Commit Graph

46 Commits

Author SHA1 Message Date
Paul Eggleton
32617fc366 rrs: fix unique constraint on RecipeMaintainerHistory sha1 field
Although it's unlikely to be an issue, technically we shouldn't be
insisting the sha1 field be unique globally, just within each
layerbranch, so adjust the constraints.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
e071ebab29 rrs: add flag to MaintenancePlan to specify layer-wide maintainers
Most layers do not track maintenance on a per-recipe basis, and for
those layers we will hide some of the per-recipe maintainer features
and on the recipe detail show the layer maintainer(s) as the
maintainer(s) of the recipe.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
7606664eeb rrs: link maintenance/upstream history to layerbranch
RecipeUpstreamHistory was not linked to the layer it was produced from,
which meant that it wasn't easy to query for a different maintenance
plan (i.e. a different layer) and thus the maintenance plan selection
on the recipe list didn't really work. Add a link field, populate it in
a migration and then make it required.

We had added a link earlier from RecipeMaintainerHistory to LayerBranch
but it was optional; for the same reasons we now populate it and make it
required.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
baa3f78498 rrs: improve admin for Release/Milestone objects
* Ensure the Release and Milestone names are separated by a space when
  listing Milesones
* Include the maintenance plan name in the name shown for each
  Release/Milestone
* Allow filtering Releases/Milestones by maintenance plan

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
1f037470fb rrs: releases should be unique by plan and name, not just name
I missed changing this constraint when adding the plan field. We want to
be able to have the same named release on another plan.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
a6aaa5c8ef Implement layer web repo commit URL
The Recipe Reporting System needs to be able to provide links to commits
in the web interface for the repository, but we can only do this if we
have a custom template URL just like we do for file/tree links, since
it's different for different git web interfaces. Add support in all the
various places for such a URL and make use of it in the RRS.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
6a332c5d8f rrs: handle linking maintainership
Provide a mechanism set the maintainer for things like gcc-cross-<arch>
to the same as gcc. (We do have entries in the .inc file for these,
however they aren't useful as they don't match the recipe name when we
parse it, and due to the fact that RecipeMaintainer objects link
directly to Recipe objects, we can't handle entries that don't map to a
real recipe).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
58c56883fd rrs_maintainer_history: add --fullreload option
Add an option that deletes all maintainer history records for the
current layer branch so that they can then be reloaded from scratch.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
7831eec543 rrs_upgrade_history: use date/commit of last recipe upgrade import
Instead of arbitrarily importing the last 8 days of upgrades, record the
date and commit when we do an import, and then use that information the
next time the script is run.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
ba27f9fee0 rrs_upgrade_history.py: enable to work across python2/3 versions
If you want to go back and get history for the earlier releases (krogoth
and previous) then we need to be able to support both python 2 and 3,
which practically means we need the same split for this script as we
have for the main layer index update script.

The catch here is that since we are going back and following the history
of changes forward, we basically need to use the same version of bitbake
that was current at that time. This works except for around the
transition between python 2 to 3 where the metadata lagged behind a bit,
so we need to take that into account. In order to keep things generic we
have a date field on the maintenance plan layer branch that specifies
the date in the metadata where we should switch over to python 3, and
then link to PythonEnvironment records that should be used for python 2
and 3 respectively.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
1733170583 rrs: add maintenance plan selection
Add a drop-down for selecting the maintenance plan from the recipes
page.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
24db787f68 rrs/models: handle All missing in milestones
If we call objects.get() with no matching record then the result will be
an exception, not a null return, so handle that properly.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
b8f51c7f9e rrs: handle maintenance plans in views
Insert maintenance plan into views, their corresponding URLs and
templates.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
735353ebd1 rrs/models: add fields for more flexible email handling
* Add a flag to say whether emails should be sent
* Add fields for from/to/subject (to replace what's currently in
  settings)
* Add user link for administrator to replace the hardcoded values in the
  email template

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:53 +12:00
Paul Eggleton
edb1261d2b rrs: fix model string representations for Python 3
__unicode__ doesn't work anymore, we need to use __str__ for models
instead.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:52 +12:00
Paul Eggleton
24960bb7be rrs: add Release link to MaintenancePlan
If any Releases exist then we create a default MaintenancePlan and then
link all Releases to it - this needs to be done in multiple upgrade
steps since the field needs to be non-null.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:52 +12:00
Paul Eggleton
1f49943aee rrs: add maintenance plans
The MaintenancePlan will provide some context for the RRS records so we
can effectively enable RRS functionality only on certain layers where we
want it. You can also consider the maintenance of multiple layers
together under a single plan if desired.

Here we add just the MaintenancePlan and the corresponding migration;
a non-null link from the Release requires a separate migration and thus
that will be done in a separate commit.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:52 +12:00
Paul Eggleton
f58754c998 rrs/models: move Raw class to views.py
This is something that really belongs in with the views, so move it
there.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-05-04 23:57:52 +12:00
Mariano Lopez
ef32d52b29 rrs/views: Fix mismatch in summary and recipe list
This fixes the issue with the mismatch in the
statistics bar and the recipe list. The mismatch
is caused by the gcc-source recipe, because every
version of the recipe is counted in Upstream History
but not in Recipe Upgrade.

Also added the TODO.rrs to track bugs and issues with
rrs.

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Aníbal Limón
b2e04c0a9e rrs: Fix last updated.
Don't conditional last updated to upstream status.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Mariano Lopez
ac50ffd82e models.py: Code cleanup
Cleaned up the class Raw to have better readability.
Also added docstring to methods in this class.

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Mariano Lopez
aab05507d5 models.py: Added Raw SQL for percentage updated
This adds SQL queries that will be used for the
percentage of recipes updated in a period.

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Mariano Lopez
58ea26418e views.py: Fix current up to date percentage
There was an issue where the current up to date was always
using the latest upstream version to get the percentage.
This uses the upstream version in the period displayed.

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2018-05-04 23:57:52 +12:00
Mariano Lopez
034bf57f71 views.py: Fix version in current milestone
Fixed the version displayed in current milestone.
Before the data was obtained from recipe upgrad table
and this allows to fetch the data from the recipe table.

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
2018-05-04 23:57:52 +12:00
Mariano Lopez
faafae13fa views.py: Add how long a recipe hasn't been updated
This add a tooltip in the upstream status field that
show how long the recipe hasn't been updated.

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Mariano Lopez
35b4d0fb34 rrs/models.py: Added raw SQL calls
The raw calls are going to be used instead of Django's ORM.
This improves the performance of the webpage.

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Aníbal Limón
954e81d27a rrs/models.py: RecipeUpgrade fix selection by milestone.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Aníbal Limón
7174fab269 rrs: Add missing date indexes in RecipeUpstreamHistory
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Aníbal Limón
d4c3693f49 rrs: Add database indexes in models.py
Add database indexes in models.py also create migration for upgrade
the current database in order to improve performance.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:52 +12:00
Aníbal Limón
6752c23362 rrs: RecipeDetail show milestone instead of all in upgrades table
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:51 +12:00
Aníbal Limón
665dfae577 rrs: Release/Milestone improvments
rrs/models.py: Milestone now have unique in Release and Name fields
instead of only field. When try to get current milestone give priority
to Mn instead of All.

templates/rrs/base_toplevel.html: Display only release name when
milestone is All.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:51 +12:00
Aníbal Limón
d61f95164b rrs: Maintainers statistics fixed sorting of weeks
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:51 +12:00
Aníbal Limón
f042c706da rrs/models.py: RecipeMaintainerHistory if not found return the first record
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:51 +12:00
Aníbal Limón
cec364510f rrs: Maintainers add support for display Milestone intervals
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:51 +12:00
Aníbal Limón
73a90437c3 rrs: Add support for Release
Now Recipes and Maintainers page can be access by Release and Milestone,
to support this a url namespace was add also update views/templates
handle new URL's.

rrs/models.py: Add support model for store Release also foregin key in
Milestone.
rrs/admin.py: Add admin site for Release model.
rrs/fixtures/initial_data.json: Add initial data with Release/Milestone
relation.
rrs/{views, urls}.py: Add support for handle Release/Milestone.
templates/rrs: Update to handle new URL's.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:51 +12:00
Aníbal Limón
446220e8c9 rrs: Add support to display version by Milestone based on RecipeUpgrade's
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:51 +12:00
Aníbal Limón
0013db9bc7 rrs: Add support for display Can't be updated recipe upstream status.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:51 +12:00
Aníbal Limón
f3f70210af rrs: Now display recipes when not found upstream info.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:51 +12:00
Aníbal Limón
5534515a8a rrs: Maintainers add support for display statistics of recipe upgrades by week in Milestone.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:51 +12:00
Aníbal Limón
b8e16428d8 rrs/models.py: fix RecipeMaintainerHistory get_by_end_date.
If not maintainer info is found return None instead of the first
one entry in the table, this made reporting more consistency.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:51 +12:00
Aníbal Limón
ac485c0cc4 rrs: Add support for Recipe maintainership history
Now can you get maintainers based on Milestones.

Feature changes,

	rrs/models.py: Add RecipeMaintainerHistory to store maintainer updates.
	rrs/admin.py: Add admin site for RecipeMaintainerHistory.

Adapt changes,

Now you need to specify RecipeMaintainerHistory to get RecipeMaintainer,
changes into rrs/view.py.

These set of changes are incompatible with the previous version you need
to do initial setup described in README.rrs.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:51 +12:00
Aníbal Limón
4e00337d93 rrs: Add support for Recipe detail page
Recipe detail page gives information about Recipe like summary,
section, license, file, etc. also display's upgrade history.

rrs/models.py: Milestone add get_by_date and rewrite get_current
for use get_by_date and RecipeDistro add get_distros_by_recipe.

rrs/urls.py: Add url for recipe_detail with pk.
rrs/views.py: Add RecipeUpgradeDetail view.
templates/rrs/recipedetail.html: Add recipedetail template.

templates/rrs/recipes.html: Add link to Recipe detail by row.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:51 +12:00
Aníbal Limón
a021f8d441 rrs: Recipes add url validation
Add url validation for Milestone and Upstream status and maintainer name
params if isn't valid raise HTTP 404.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:51 +12:00
Aníbal Limón
a7ede0d126 rrs: Add initial support of recipes page
templates/rrs/base_toplevel.html: Add support for display statistics by
Milestone.
templates/rrs/recipes.html: Add initial page that display Recipe
status by Milestone also details of every recipe.
rrs/views.py: Add RecipeLitView for support recipes page.
rrs/models.py: Add helper functions.
rrs/static/*: Add css and js resources.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:51 +12:00
Aníbal Limón
2a651efc9a rrs: Add support for store RecipeUpstreamHistory
To provide statistics about RecipeUpstream information based on
Milestones we need to store RecipeUpstreamHistory that contains when the
update script was executed.

rrs/admin.py: Add admin page for RecipeUpstreamHistory.
rrs/models.py: Add model for RecipeUpstreamHistory with helper functions
for get last and last by date range also add migration.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:51 +12:00
Aníbal Limón
8e2d772e00 rrs: Add application
Add models for store Milestone, Maintainer and Recipe{Maintainer,
Upgrade, Upstream, Distro}, add initial data for Milestone and
Maintainer, initial migration and admin sites.

Add rrs/context_processors.py for return the site_name and application
for use in templates.

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2018-05-04 23:57:51 +12:00