Errors deleting bitbake.sock and bitbake.lock have been observed when
shutting down tinfoil at the end of some of these scripts. Move the code
used in the main layer index update script to a function in utils.py and
use it everywhere in order to avoid the issue.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add links to the same recipe in other branches in the recipe detail page
(and RRS recipe detail page) so that you can see which versions are
available in other branches and drill down to the detail if you want to.
Implements [YOCTO #13019].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We were using uitablefilter.js to provide live filtering of table rows
based upon a search field value, but it turns out this module really
isn't necessary - we can accomplish the same thing using simple jQuery
code. While we're at it, enable the search field on the layers list page
to work in conjunction with with the drop-down layer type selection, fix
pasting into the search field and refreshing with a search specified.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Make it possible for users to delete their own account and all associated
information from the database, should they decide they no longer wish to
use it.
(I checked the implications of doing this on our model structure -
anything with a foreign key to user is safe to delete with the exception
of RRS MaintenancePlan.admin which I needed to change on_delete for so
that it doesn't get deleted with the user).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
nvd3 and its python/django wrappers appear to be no longer actively
maintained, and at least the wrappers were a bit clunky to use. Looking
around for a suitable replacement, Chart.js seems capable, has no
additional dependencies and is fairly simple to use. As a bonus we get
to drop a few Python dependencies from our list.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The code in recipeparse.setup_layer() was trying to log a warning in the
case where LAYERRECOMMENDS not being satisfied, however there is no
actual logger object in this context. Pass it in via a parameter and
update all callers to pass it.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Use a more modern version of Bootstrap and take the opportunity to
upgrade jQuery to the latest version at the same time. This provides
better browser compatibility, moves to MIT license, allows us to make
the site more responsive for different devices in future, and provides
theming capabilities for custom installs among other improvements.
(I chose to upgrade to v3 for now rather than straight to v4 as it was
easier to do this gradually.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If you're creating a new release then you will need to create milestones
within it, so add in-line forms for these. Unfortunately there's no
capability yet to automatically split up the release into n milestones
(or perhaps copy milestones from a previous release), that will have to
wait until later - for now this makes things a little easier.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
In the current RRS we should expect that more than one recipe with the
same PN can exist in a layer - in fact it is common to have this (i.e.
multiple versions of the same recipe). Use the filename to match the
recipe record instead. At the same time, fix the exception handling.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If there is no release or milestone covering the current date, show an
error message at the top of the recipes list page to alert the user that
they can't view current data (since this is a common cause of not being
able to see that which may not be immediately apparent).
Additionally, show a warning within rrs_upgrade_history when this
happens.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If a release has no milestones, we shouldn't be selecting it as the
default to be linked to in the maintenance plan drop-down, so filter
those out.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
In the case of dry-run a couple of the scripts were breaking out after
one layerbranch had been processed due to the code structure. Handle the
exception within the block for the layerbranch to avoid this.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We do not want to be prompting the user for a password during layer
updates or upstream checks, e.g. in the case where a repo requires
authentication, or on github where any fetch of a nonexistent repo
apparently triggers authentication.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
REGEX, REGEX_URI and GITTAGREGEX are replaced by UPSTREAM_CHECK_REGEX,
UPSTREAM_CHECK_URI and UPSTREAM_CHECK_GITTAGREGEX
Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add logger.setLevel to enable set log level.
Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Although it's unlikely to be an issue, technically we shouldn't be
insisting the sha1 field be unique globally, just within each
layerbranch, so adjust the constraints.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If automated emails are enabled, we need to ensure that the other email
fields are populated, so validate that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Most layers do not track maintenance on a per-recipe basis, and for
those layers we will hide some of the per-recipe maintainer features
and on the recipe detail show the layer maintainer(s) as the
maintainer(s) of the recipe.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If we're in a repository containing multiple layers, we don't care about
commits that don't affect the layer we are processing, so skip those
commits rather than passing them to upgrade_history_internal.py which
will ignore them (which is significantly slower).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The processing code can't currently handle if a layerbranch is part of
more than one plan, so disallow that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The patterns() function is deprecated in Django 1.8 and gone in 1.10, so
we should switch over to the new list format.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
RecipeUpstreamHistory was not linked to the layer it was produced from,
which meant that it wasn't easy to query for a different maintenance
plan (i.e. a different layer) and thus the maintenance plan selection
on the recipe list didn't really work. Add a link field, populate it in
a migration and then make it required.
We had added a link earlier from RecipeMaintainerHistory to LayerBranch
but it was optional; for the same reasons we now populate it and make it
required.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
These two functions aren't being used anywhere. In the interest of
having as little code directly reading the database using SQL as
possible, drop them.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add an option to specify which maintenance plan to operate on (largely
for debugging purposes).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Ensure the Release and Milestone names are separated by a space when
listing Milesones
* Include the maintenance plan name in the name shown for each
Release/Milestone
* Allow filtering Releases/Milestones by maintenance plan
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's a pain to have to add all the releases when adding a new
maintenance plan. Since these are likely to be the same (or similar) for
every plan, then duplicate them across from the first plan when you save
a new one.
Also add "default" milestones on the assumption that other layers
probably won't want to use the 4-milestone split per release, but there
do have to be some milestone records, so just create one milestone for
each release.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's a bit of a pain to have to set the two python environment fields on
every record in order to have things set correctly, and it can easily
get forgotten, so try to set them automatically by default (assuming
reasonable naming).
Note that this does introduce an annoying behaviour whereby if you click
"Add another Maintenance plan layer branch" and then decide you don't
want it, the admin form will insist you fill in the fields unless you
clear out the python2/3 environment fields. I'm not sure how to fix
that, so I'm leaving it as-is for now.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
I missed changing this constraint when adding the plan field. We want to
be able to have the same named release on another plan.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Expose the newly added patch information in the RRS:
* Add a table to the recipe detail listing the patches for the recipe
* Add pending / total counts to the recipe list page
Implements [YOCTO #7909].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Consolidate the code for checking out a repository, using the newly
added utils.checkout_repo() function
* Check out a layer's dependencies, not just the layer itself
* Only check out if the desired revision isn't already checked out
(mostly useful for bitbake which we would otherwise be checking out
much more frequently than necessary since it may not have changed
even if we've moved to a new commit in the layer).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The Recipe Reporting System needs to be able to provide links to commits
in the web interface for the repository, but we can only do this if we
have a custom template URL just like we do for file/tree links, since
it's different for different git web interfaces. Add support in all the
various places for such a URL and make use of it in the RRS.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Provide a mechanism set the maintainer for things like gcc-cross-<arch>
to the same as gcc. (We do have entries in the .inc file for these,
however they aren't useful as they don't match the recipe name when we
parse it, and due to the fact that RecipeMaintainer objects link
directly to Recipe objects, we can't handle entries that don't map to a
real recipe).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We don't actually need to check out the repository until we actually
analyse a commit, so avoid doing so. Additionally, there's not much
point in checking out master at the end, let the next script invocation
do that if needed (if it needs to, it should since otherwise there's no
guarantee what state the repository is in).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add an option that deletes all maintainer history records for the
current layer branch so that they can then be reloaded from scratch.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We don't really need to clean up the repository, we'll take care of it
on subsequent runs.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If a recipe upgrade only consists of a .inc file changing, we were not
picking it up, since we were only looking specifically for recipes
(.bb). If a .inc file changes, assume all .bb files in the same
directory (if any) should be parsed to look for an upgrade.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
load_recipes() was leaving files around in /tmp; on my Fedora system
this eventually resulted in /tmp running out of space which we do not
want.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Sometimes it's useful to be able to re-try processing a particular
commit (generally in conjunction with --dry-run), so add an option to
enable that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We were parsing recipes that were in the repository but not inside the
actual layer we're dealing with (e.g. we have meta-selftest within the
OE-Core repository, containing a number of recipes that are only
intended for testing purposes and should not be looked at by this
script).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If an exception occurs during parsing, let's actually see what kind of
exception it was in the output.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
In OE-Core revision e0531174119bff21e9014b95ed1bbd0e1c01af26 we
accidentally committed a new e2fsprogs recipe with ..bb at the end of
its name instead of .bb. This was fixed immediately afterwards, but when
the RRS hits this commit, it doesn't fail immediately, but the bogus
version "1.43." gets into the database and all subsequent commits
touching the e2fsprogs recipe cause bb.utils.vercmp_part() to blow up
because one of the version parts in the "previous" version in the database
is apparently empty. To work around this and any similar issues, just
reject any change that results in such a broken version string (on the
assumption that it'll be corrected in a subsequent commit and thus we
will get to re-parse the recipe then and therefore not miss the
upgrade.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We check out different revisions while we do this processing, and so
does the layer index update script, so we shouldn't be allowing both to
run at once or nasty stuff will happen.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Since we're now executing a separate script per commit, we should try
not to do that unless the commit actually touches recipe files in order
to avoid wasting time.
(Whilst it's possible that a change to a bbclass might alter what's in
the recipe, we can ignore that since we are only concerned with actual
upgrades which would always require some sort of change to the recipe or
an include file, so we can safely skip commits that don't do that.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Instead of arbitrarily importing the last 8 days of upgrades, record the
date and commit when we do an import, and then use that information the
next time the script is run.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
