Commit Graph

76 Commits

Author SHA1 Message Date
Paul Eggleton
ccc1fa775b Report charset for text & CSV views
I noticed in the branch comparison plain text view ("Plain text" button
in the Tools -> Branch Comparison page) that in current versions of
Firefox the ellipses were coming through corrupted, though they looked
fine in the HTML version, and it turns out this is because I wasn't
specifying a character set encoding. It should be UTF-8, so add a
charset to the content type stating as such for this and other similar
views.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.microsoft.com>
2021-10-20 11:07:46 +13:00
Paul Eggleton
fdc8209b29 rrs: make changes to support Django 2.2
Removed obsolete references to django.core.urlresolvers. Added the
newly required on_delete parameters to foreign key relationships in
models.py and in all migrations.

(Amber Elliot's 2d526f9b0d served as a
reference for these changes).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.microsoft.com>
2021-10-20 10:02:58 +13:00
Meh Mbeh Ida Delphine
2314fb2811 Add SPDX license headers to layerindex-web source files
Added SPDX identifiers to all .py files except those in migrations directory.

Fixes: [YOCTO #13527]

Signed-off-by: Meh Mbeh Ida Delphine <idadelm@gmail.com>
Signed-off-by: Paul Eggleton <bluelightning@bluelightning.org>
2021-04-21 08:22:51 +12:00
Paul Eggleton
254dc0c7db RRS: enable grouping recipe upgrades by license
Going back in OE-Core recipe upgrade history, we kept GPLv2 and GPLv3
versions of a number of recipes around, so this is the source of quite a
few situations where we had multiple versions of recipes with the same
recipe name around. Add means of grouping upgrades by license so that we
can keep these versions separate in the upgrade history instead of
detecting lots of apparent upgrades and downgrades if they are
intermingled.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-11-21 02:51:30 +13:00
Paul Eggleton
29741015d1 RRS: ensure upgrades recorded at exact same time are correctly ordered
In meta-oe there are two commits (d91f92cf04 and 57492d40b5) which have
the same commit date and thus don't deterministically order; the result
was that the mercurial-native recipe might or might not show up. Add id
to the order_by to make it deterministic.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-11-21 02:27:39 +13:00
Paul Eggleton
51d7f139f8 RRS: support grouping upgrades by version for multi-version recipes
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-11-21 02:27:39 +13:00
Paul Eggleton
ecb053472b RRS: handle recipe moves without overwriting data
If we ever want to analyse the upgrade chain later on then we need to
avoid overwriting the paths when we identify a moved recipe - instead,
store a "move" upgrade record (not shown in the UI) that we can later
pick up when we are going through and deleting.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-11-21 02:27:39 +13:00
Paul Eggleton
5540a84434 RRS: Add deleted recipe handling
Now that we're using RecipeSymbols we have the complete list of recipes
that ever existed in a layer. We only want to see the ones that are
valid for the selected milestone, so when a recipe gets deleted (or
renamed or moved outside of the layer subdirectory, if any) we need to
record that - do so using a RecipeUpgrade record with a new field
upgrade_type set to 'R'. Additionally we need to store the file path so
that deletion events (where we don't parse the contents of the recipe,
thus we don't have PN) are easy to match up with RecipeUpgrade records;
naturally we need to keep the paths "up-to-date" when we notice recipe
files being moved around.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-11-21 02:27:39 +13:00
Paul Eggleton
c0b8439182 RRS: collect history independent of current recipes
Recipes come and go over time, so when a recipe gets deleted the history
for it goes away, which means that if you look back in time you do not
see an accurate picture - you only see the subset of recipes that are
currently present. Introduce an indirection between recipes and history
that allows for old recipes to persist (mostly in name only).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-11-21 02:27:39 +13:00
Paul Eggleton
03b39e89c0 RRS: fix showing all recipes for multi-layer maintenance plans
If you had more than one layer on a maintenance plan, only the recipes
from the last layer were shown because we were using a recipe list
variable from the last iteration. The fix just turned out to be to
indent the code that read from that list so it runs every iteration
instead of at the end.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-21 14:15:52 +13:00
Paul Eggleton
7ab7766ec7 RRS: Fix raw SQL statements to use parameters
For security reasons it's best practice to use parameters to pass values
into SQL statements and not substitute them in as strings (with Django's
database API the distinction is subtle, but we pass in the parameters in
a second list parameter instead of using % to substitute them before
passing the query in).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-21 14:15:42 +13:00
Paul Eggleton
0929289465 Add links to other branch recipes in recipe detail
Add links to the same recipe in other branches in the recipe detail page
(and RRS recipe detail page) so that you can see which versions are
available in other branches and drill down to the detail if you want to.

Implements [YOCTO #13019].

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-12-06 09:25:06 +13:00
Paul Eggleton
631116a1c4 Replace nvd3 with Chart.js
nvd3 and its python/django wrappers appear to be no longer actively
maintained, and at least the wrappers were a bit clunky to use. Looking
around for a suitable replacement, Chart.js seems capable, has no
additional dependencies and is fairly simple to use. As a bonus we get
to drop a few Python dependencies from our list.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-06 10:33:39 +13:00
Paul Eggleton
c629787630 rrs: show warnings for missing current release/milestone
If there is no release or milestone covering the current date, show an
error message at the top of the recipes list page to alert the user that
they can't view current data (since this is a common cause of not being
able to see that which may not be immediately apparent).

Additionally, show a warning within rrs_upgrade_history when this
happens.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-09-06 11:08:43 +12:00
Paul Eggleton
e071ebab29 rrs: add flag to MaintenancePlan to specify layer-wide maintainers
Most layers do not track maintenance on a per-recipe basis, and for
those layers we will hide some of the per-recipe maintainer features
and on the recipe detail show the layer maintainer(s) as the
maintainer(s) of the recipe.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
7606664eeb rrs: link maintenance/upstream history to layerbranch
RecipeUpstreamHistory was not linked to the layer it was produced from,
which meant that it wasn't easy to query for a different maintenance
plan (i.e. a different layer) and thus the maintenance plan selection
on the recipe list didn't really work. Add a link field, populate it in
a migration and then make it required.

We had added a link earlier from RecipeMaintainerHistory to LayerBranch
but it was optional; for the same reasons we now populate it and make it
required.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
da3bfff3a1 rrs: drop a couple of unused functions from Raw class
These two functions aren't being used anywhere. In the interest of
having as little code directly reading the database using SQL as
possible, drop them.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
183ba0f7eb rrs: add charts page
Add some basic charts to show recipe upstream / patch status.

Implements [YOCTO #7909].

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
130ff8e803 rrs: add patch listing to recipe list and detail
Expose the newly added patch information in the RRS:

* Add a table to the recipe detail listing the patches for the recipe
* Add pending / total counts to the recipe list page

Implements [YOCTO #7909].

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
a6aaa5c8ef Implement layer web repo commit URL
The Recipe Reporting System needs to be able to provide links to commits
in the web interface for the repository, but we can only do this if we
have a custom template URL just like we do for file/tree links, since
it's different for different git web interfaces. Add support in all the
various places for such a URL and make use of it in the RRS.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
0d112b1d97 rrs/views: replace mimetype with content_type
Adapt to this change that happened sometime in Django 1.7.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
8a2839b870 rrs: add breadcrumb to recipe detail page
Make it possible to get back to the maintenance plan from the recipe
detail page.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
ea6270f386 rrs: handle dependency field differences
The old RRS branch had its own addition of dependency support, but in the
mean time we added that to the layer index in the master branch using a
different structure. Adapt the RRS recipe detail page to that structure.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
b2a67d718c Add link to maintenance plan in layer detail
If the RRS is enabled, then add a way to get from the layer detail page
to any maintenance plans in which the layer is included.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
1733170583 rrs: add maintenance plan selection
Add a drop-down for selecting the maintenance plan from the recipes
page.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
e77b652192 rrs/views: fix SQL parameter in get_reup_by_last_updated()
We were passing in a parameter value but not using that in the query.
Add a WHERE clause to fix that.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
8517822d43 rrs/views: fix case in table names
Table names should be lowercase. I'm unsure if the collation settings
have an effect on this, but with the mariadb database I set up here I
needed to make this change or else I got errors about missing tables.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
b8f51c7f9e rrs: handle maintenance plans in views
Insert maintenance plan into views, their corresponding URLs and
templates.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
6ca3d6649e Use this layer index's URL for layer branch
If we're integrating the layer index and the RRS, no need to jump off to
the OE index if for example this is an internal index.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:52 +12:00
Paul Eggleton
e50c2e6841 Rename RRS URLs to have rrs_ prefix
Make these distinct from the layerindex ones.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:52 +12:00
Paul Eggleton
f58754c998 rrs/models: move Raw class to views.py
This is something that really belongs in with the views, so move it
there.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:52 +12:00
Aníbal Limón
66193c153b rrs/views.py: When export to csv use release_milestone.csv
To be more descriptive about what report is.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Aníbal Limón
48375d481c rrs: Frontend update to work with Python3 and django
[YOCTO #9746]

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Aníbal Limón
86d77504f2 rrs: Fix frontpage redirect
We was experimenting fixed redirects due to urls are
cached at init of rrs.

Move the redirect Release, Milestone selection to a view
called FrontPageRedirect.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Aníbal Limón
3ccb422aeb rrs/recipes: Add support for persist search and filters.
Now when search something the URL is modified and you can
share the URL for access to the data.

[YOCTO #7809]

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Mariano Lopez
ef32d52b29 rrs/views: Fix mismatch in summary and recipe list
This fixes the issue with the mismatch in the
statistics bar and the recipe list. The mismatch
is caused by the gcc-source recipe, because every
version of the recipe is counted in Upstream History
but not in Recipe Upgrade.

Also added the TODO.rrs to track bugs and issues with
rrs.

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Mariano Lopez
6aa369757c rrs/views.py: Added percentages for navbar
This adds the percentage for all the recipes types
(up-to-date, not updated, unknown, can't be updated)
so it can be displayed in the navbar.

This also adds the number of the recipes not updated
at the begining of the period and number of recipes
updated in the period.

The changes in the frontend are still missing, this
just adds the functionality.

[YOCTO #8020]

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Mariano Lopez
b08004f778 views.py: Show last updated date for all recipes
This changes the behavior of the last update column
and show the date for all the recipes, including the
up to date recipes.

[YOCTO #8018]

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2018-05-04 23:57:52 +12:00
Aníbal Limón
b2e04c0a9e rrs: Fix last updated.
Don't conditional last updated to upstream status.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Aníbal Limón
6added950f rrs/views.py: RecipeListView always use recipes from history.
If recipe don't exist in history means that don't need to be displayed.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Mariano Lopez
45ba3a280d views.py: Changed date format to iso
Changed the day format to iso. Also, the outdated
field in the recipes views was changed from days
to the date the recipe was last updated.

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Mariano Lopez
7f24895f6e views.py: Changed the behavior of percentage done
This changes the percetage done from recipes up to date
in the period to percentage of not updated recipes that
were updated in the period.

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Mariano Lopez
58ea26418e views.py: Fix current up to date percentage
There was an issue where the current up to date was always
using the latest upstream version to get the percentage.
This uses the upstream version in the period displayed.

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2018-05-04 23:57:52 +12:00
Mariano Lopez
034bf57f71 views.py: Fix version in current milestone
Fixed the version displayed in current milestone.
Before the data was obtained from recipe upgrad table
and this allows to fetch the data from the recipe table.

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2018-05-04 23:57:52 +12:00
Mariano Lopez
faafae13fa views.py: Add how long a recipe hasn't been updated
This add a tooltip in the upstream status field that
show how long the recipe hasn't been updated.

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Mariano Lopez
a9aff3cae6 rrs/views.py: Changed ORM to raw SQL calls
Allow to use the new raw SQL call to get better performance in the
web page. Also removed SQL queries inside loops.
This is only for two views: recipes and maintainers

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Aníbal Limón
c64ec37699 rrs/views.py: RecipeList always use recipe from Recipe upgrades.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Mariano Lopez
887737027c views.py: Added validation in recipe detail
Added the validation for the maintainer in the
recipe detail view. Also added a check if the
recipe exists.

[YOCTO #7904]

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Mariano Lopez
ab7f6443ac rrs/views.py: Fix exception in Maintainer view
When try to get Maintainer view with empty database a exception
is raised due to don't have intervals.

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Aníbal Limón
0f6c74b68b rrs/views.py: Recipes remove unnedded extra_url_params
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:51 +12:00