Fixtures aren't supported in current Django versions. Add some code to
the initial migration to add the data instead.
Also add releases/milestones up to and including 2.5.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We were passing in a parameter value but not using that in the query.
Add a WHERE clause to fix that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Table names should be lowercase. I'm unsure if the collation settings
have an effect on this, but with the mariadb database I set up here I
needed to make this change or else I got errors about missing tables.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If we call objects.get() with no matching record then the result will be
an exception, not a null return, so handle that properly.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
With Django 1.10+, if you use get_template() to retrieve a template,
then you can't pass a context when calling .render() on it, you need to
pass a dict instead.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Use maintenance plans to get layerbranches
* Use from/to/subject and admin contact from maintenance plan
* Use an actual template to render the email (and drop tabulate
dependency)
* Improve grammar in the email text
* Use a single line to represent the most recent commit
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Add a flag to say whether emails should be sent
* Add fields for from/to/subject (to replace what's currently in
settings)
* Add user link for administrator to replace the hardcoded values in the
email template
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This function is no longer needed - we now rely upon the layer index's
code to do this (update through the update script, and checkout through
setup_layer()).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Remove hardcoded references to the poky repository, and process
layerbranches for all enabled maintenance plans.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Without this it's not clear what's happening if debug mode is enabled
and you are waiting for the parsing step.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If the Recipe object doesn't exist here then an exception will be raised
rather than None being returned, and this will also trigger if multiple
recipes match. This may have never triggered in the past because this
would have been run right after updating all the recipes in the layer and
clearing out duplicates (which we were doing earlier), and thus what is
in the database would match the recipe files in the repository, assuming
no errors occurred during parsing). We can't remove duplicates though so
we need to switch over to using filter() and taking the first recipe.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Remove hardcoded references to the poky repository, and process
layerbranches for all enabled maintenance plans.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This is never going to work, the only way we can practically do parallel
execution is to run these things in a task which would basically amount
to distrodata's do_checkpkg; we may move to that in future but for now
just drop the threading code.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Instead of hardcoded references to the poky repository, look for any
maintainers.inc file in layers associated with the layerbranches for all
enabled maintenance plans. At present few layers have this file, but at
least it will now work generically in any layer index instance.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Instead of processing all layerbranches, only process those associated
with an enabled maintenance plan. This is one step towards being able to
use the RRS together with a more general layer index database.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We don't need to create branches here, and we don't need to actually
check anything out unless we're going to parse, so we can save a bit of
time by not doing so.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If any Releases exist then we create a default MaintenancePlan and then
link all Releases to it - this needs to be done in multiple upgrade
steps since the field needs to be non-null.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The MaintenancePlan will provide some context for the RRS records so we
can effectively enable RRS functionality only on certain layers where we
want it. You can also consider the maintenance of multiple layers
together under a single plan if desired.
Here we add just the MaintenancePlan and the corresponding migration;
a non-null link from the Release requires a separate migration and thus
that will be done in a separate commit.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This is of course a new Django migration rather than the previous South
one. This will have to be applied using --fake-initial on a database
that already has the RRS tables.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add the ability to run the scripts without writing changes back to the
database, for debugging purposes.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If we're integrating the layer index and the RRS, no need to jump off to
the OE index if for example this is an internal index.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We can't just delete arbitrary recipes that do exist in the layer if
we're in a general layer index database. We'll need to handle this in a
different way, or just live with the fact that there will be duplicate
entries.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Due to bitbake client/server changes now the meta path isn't included
to sys.path when load tinfoil.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
This are causing a failure when access the remote datastore.
File "rrs/rrs/tools/rrs_upstream_history.py", line 169, in <module>
set_regexes(recipe_data)
File "rrs/rrs/tools/rrs_upstream_history.py", line 53, in set_regexes
if any(d.getVar(var, True) for var in variables):
File "rrs/rrs/tools/rrs_upstream_history.py", line 53, in <genexpr>
if any(d.getVar(var, True) for var in variables):
File "/srv/rrs/layers/bitbake/lib/bb/data_smart.py", line 608, in
getVar
return self.getVarFlag(var, "_content", expand, noweakdefault,
parsing)
File "/srv/rrs/layers/bitbake/lib/bb/data_smart.py", line 723, in
getVarFlag
local_var, overridedata = self._findVar(var)
File "/srv/rrs/layers/bitbake/lib/bb/data_smart.py", line 480, in
_findVar
return connector.getVar(var)
File "/srv/rrs/layers/bitbake/lib/bb/tinfoil.py", line 62, in getVar
value = self.tinfoil.run_command('dataStoreConnectorFindVar',
self.dsindex, name)
File "/srv/rrs/layers/bitbake/lib/bb/tinfoil.py", line 316, in
run_command
raise TinfoilCommandFailed(result[1])
bb.tinfoil.TinfoilCommandFailed: Traceback (most recent call last):
File "/srv/rrs/layers/bitbake/lib/bb/command.py", line 81, in
runCommand
result = command_method(self, commandline)
File "/srv/rrs/layers/bitbake/lib/bb/command.py", line 465, in
dataStoreConnectorFindVar
datastore = command.remotedatastores[dsindex]
File "/srv/rrs/layers/bitbake/lib/bb/remotedata.py", line 39, in
__getitem__
return self.datastores[key]
TypeError: unhashable type: 'list'
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
The new tinfoil API acts as a client/server so isn't need to work
around creating process to avoid errors in cooker with different
data revisions.
Also add a try/except when create an upgrade display the error.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
The new client/server API of tinfoil requires explicit call of
shutdown method to send the event for finalize cooker process.
[YOCTO #10863]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Django 1.6 provides a context manager for atomic transactions so
update the way for make db transactions.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Due to multi-configuration builds in bitbake now is needed
to use a tinfoil cache instance instead of call directly in
bb.cache.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Now create_distro_packages_list requires d to be passed, this change was
done to enable proxy support into create_distro_packages_list function.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
We was experimenting fixed redirects due to urls are
cached at init of rrs.
Move the redirect Release, Milestone selection to a view
called FrontPageRedirect.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
instead of deprecated upstream tracking file.
Since upstream_tracking.inc file is deprecated because now every recipe
has it's own upstream information then update email message according it.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Now when search something the URL is modified and you can
share the URL for access to the data.
[YOCTO #7809]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Don't redirect permanently in main view because the first URL needs
to point to the current release, so use permanent = False to return
a 302..
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
The status email sent to the Yocto Project mailing
list by the Recipe Reporting Tool still shows the
old URL (http://packages.yoctoproject.org).
This patch changes it to the new URL:
http://recipes.yoctoproject.org
Signed-off-by: Belen Barros Pena <belen.barros.pena@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Make difference when handling suffixes or preffixes like 'nativesdk-'
when try to use regexes in packages that have suffixes like '-crosssdk'
it can contain the arch into it like binutils-crosssdk-x86_64.
Use split method in suffixes and replace method into preffixes, this
fixes issues with suffixes containing archs at end.
[YOCTO #8102]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
This fixes the issue with the mismatch in the
statistics bar and the recipe list. The mismatch
is caused by the gcc-source recipe, because every
version of the recipe is counted in Upstream History
but not in Recipe Upgrade.
Also added the TODO.rrs to track bugs and issues with
rrs.
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
This provides changes in the front end for the
navbar, now it shows the percentage of recipes
up-to-date, not update, unknown and can't be
updated along with the number of recipes.
This also moves the update percentage to the
end and adds clarity to what it means.
This also moves the export list button the the
top bar.
[YOCTO #8020]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
This adds the percentage for all the recipes types
(up-to-date, not updated, unknown, can't be updated)
so it can be displayed in the navbar.
This also adds the number of the recipes not updated
at the begining of the period and number of recipes
updated in the period.
The changes in the frontend are still missing, this
just adds the functionality.
[YOCTO #8020]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
When SPECIAL_PKGSUFFIX packages don't have regexes use it from package
without SPECIAL_PKGSUFFIX.
[YOCTO #8102]
For example: python-native use regex from python if don't have one.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
This changes the behavior of the last update column
and show the date for all the recipes, including the
up to date recipes.
[YOCTO #8018]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Don't load all recipes avaiable in layer because was previous
filtered to have the last version.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
without versioning.
Don't detect upgrade in git recipes that don't have versioning only
PV=git.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Cleaned up the class Raw to have better readability.
Also added docstring to methods in this class.
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Changed the day format to iso. Also, the outdated
field in the recipes views was changed from days
to the date the recipe was last updated.
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
This changes the percetage done from recipes up to date
in the period to percentage of not updated recipes that
were updated in the period.
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
This adds SQL queries that will be used for the
percentage of recipes updated in a period.
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
There was an issue where the current up to date was always
using the latest upstream version to get the percentage.
This uses the upstream version in the period displayed.
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Fixed the version displayed in current milestone.
Before the data was obtained from recipe upgrad table
and this allows to fetch the data from the recipe table.
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
This add a tooltip in the upstream status field that
show how long the recipe hasn't been updated.
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Allow to use the new raw SQL call to get better performance in the
web page. Also removed SQL queries inside loops.
This is only for two views: recipes and maintainers
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
The raw calls are going to be used instead of Django's ORM.
This improves the performance of the webpage.
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
When add new recipes sometimes don't have maintainer defined causing
errors on frontend when search for one.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
bb.cache.Cache.loadDatafull() expects an filepath in asciii, if not
is provided it fails with character mapping error.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Tinfoil shutdown was not accepted as patch into bitbake because
this is a simple solution to a big problem.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Added the validation for the maintainer in the
recipe detail view. Also added a check if the
recipe exists.
[YOCTO #7904]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Don't remove native, cross, crossinitial and sdk recipes now only
keep major version of recipe based by pn.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
When runs recipe parse with different revisions in same process it
generates errors due to cooker parser uses global data cache's.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Import of FetchError when try to expand PV somethings requires to have
SRCREV/SRCPV causing an FetcherError.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Copied get_recipe_pv_without_srcpv from oe-core/recipeutils library
to avoid import errors caused by have imported twice oe-core from poky
and from tinfoil.
rrs/tools/rrs_upgrade_history.py: Update to use get_recipe_pv_without_srcpv
from common.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
In the RRS only one recipe (PN) per layerbranch makes sense because
the RRS reports the current status of the recipe.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Added links to recipe page using the update status filter in the header.
Changed the <th> tags in the footer to <td>.
Added color to the footer format instead of bold.
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
When try to get Maintainer view with empty database a exception
is raised due to don't have intervals.
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Add database indexes in models.py also create migration for upgrade
the current database in order to improve performance.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
rrs/models.py: Milestone now have unique in Release and Name fields
instead of only field. When try to get current milestone give priority
to Mn instead of All.
templates/rrs/base_toplevel.html: Display only release name when
milestone is All.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Display no update reason when upstream status filter is Can't be updated
this helps to review the reason.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Give some space to the tabs,
Add some top margin to the nav-pills class so we
create some breathing space between the milestone
overview and the tabs.
Change 'No Maintainer' to 'No maintainer'
Just to keep the capitalisation style consistent
across the interface.
Apply the muted class to all not-sortable table headings
The class was missing from the Recipe and Maintainer
columns in the recipes table; and from the Not updated
and % done column in the maintainers table.
Remove the strong tag from the recipe status information
This is just to match the presentation of the milestone
overview information in the base_toplevel.html template.
Separate the footer from the bottom of the viewport,
It's hard to see the footer on click on its links when
they are so close to the bottom of the veiwport, so
add some margin at the bottom of the footer <div>.
Change the label of the recipes tab,
From 'Recipes status' to 'Recipes upstream status' to
match the label of the 'Upstream status' filter in the
recipes table.
Signed-off-by: Belen Barros Pena <belen.barros.pena@intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Maintainer statistics use '' instead of zeros.
RecipeDetail don't display internal Downgrade status use Unknown instead.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Now Recipes and Maintainers page can be access by Release and Milestone,
to support this a url namespace was add also update views/templates
handle new URL's.
rrs/models.py: Add support model for store Release also foregin key in
Milestone.
rrs/admin.py: Add admin site for Release model.
rrs/fixtures/initial_data.json: Add initial data with Release/Milestone
relation.
rrs/{views, urls}.py: Add support for handle Release/Milestone.
templates/rrs: Update to handle new URL's.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Show percentage sign for every item in percentage column.
Don't display zero's in week columns instead use blank '' to
avoid noisy.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Now filter recipes are done by JS this avoid to made request for every
change in the filters also add support for share filters between pages.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
templates/rrs/base_toplevel.html: Disable top functions.
templates/rrs/recipedetail.html: Fix class type for different
upstream status and better display of recipe info don't display
element if no have content.
rrs/views.py: Don't display percentages with two decimals and
separate filters for set and elements.
templates/rrs/recipes.html: Add support for display filters with
set and element separation.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
In order to provide only one status when recipe upstream backend
script fails display display Unknown as Unknown + Downgrade.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Maintainer statistics page gives information by Milestone and
Maintainers assigned recipes, status of recipes (up-to-date,
not-update, unknown) and percertange of work done.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Modularize get milestone statistics not at private function in order
to use in Manitainers view.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
rrs/views.py: Now is necessary to test if a RecipeMaintainer exists by
milestone to avoid errors for undefined variables.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
If not maintainer info is found return None instead of the first
one entry in the table, this made reporting more consistency.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Now can you get maintainers based on Milestones.
Feature changes,
rrs/models.py: Add RecipeMaintainerHistory to store maintainer updates.
rrs/admin.py: Add admin site for RecipeMaintainerHistory.
Adapt changes,
Now you need to specify RecipeMaintainerHistory to get RecipeMaintainer,
changes into rrs/view.py.
These set of changes are incompatible with the previous version you need
to do initial setup described in README.rrs.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Improve CSS in order to handle column width better also
use styles to display upstream status column.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Recipe detail page gives information about Recipe like summary,
section, license, file, etc. also display's upgrade history.
rrs/models.py: Milestone add get_by_date and rewrite get_current
for use get_by_date and RecipeDistro add get_distros_by_recipe.
rrs/urls.py: Add url for recipe_detail with pk.
rrs/views.py: Add RecipeUpgradeDetail view.
templates/rrs/recipedetail.html: Add recipedetail template.
templates/rrs/recipes.html: Add link to Recipe detail by row.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Add url validation for Milestone and Upstream status and maintainer name
params if isn't valid raise HTTP 404.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
templates/rrs/base_toplevel.html: Add support for display statistics by
Milestone.
templates/rrs/recipes.html: Add initial page that display Recipe
status by Milestone also details of every recipe.
rrs/views.py: Add RecipeLitView for support recipes page.
rrs/models.py: Add helper functions.
rrs/static/*: Add css and js resources.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
To provide statistics about RecipeUpstream information based on
Milestones we need to store RecipeUpstreamHistory that contains when the
update script was executed.
rrs/admin.py: Add admin page for RecipeUpstreamHistory.
rrs/models.py: Add model for RecipeUpstreamHistory with helper functions
for get last and last by date range also add migration.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Add models for store Milestone, Maintainer and Recipe{Maintainer,
Upgrade, Upstream, Distro}, add initial data for Milestone and
Maintainer, initial migration and admin sites.
Add rrs/context_processors.py for return the site_name and application
for use in templates.
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
