Add an option to stop at a particular commit (so we can then repeat a
specific commit afterwards easily for debugging purposes).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Record the previous version in RecipeUpgrades, and use it to more
accurately record upgrades where there are multiple versions present at
a given time (common with e.g. kernel recipes).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Make it possible to re-collect all the history for a given path.
(Typically this would only be used for debugging, as it saves time if
you are trying to correct an issue with upgrade data collection for a
single recipe.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If we ever want to analyse the upgrade chain later on then we need to
avoid overwriting the paths when we identify a moved recipe - instead,
store a "move" upgrade record (not shown in the UI) that we can later
pick up when we are going through and deleting.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Silently catching all exceptions like this is evil. Use .first() and
check the return instead.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Version downgrades (or what appear to be downgrades) do occasionally
happen, and if they did then the RRS was previously simply ignoring
them, resulting in the latest version being reported incorrectly.
Allow downgrades to be recorded as an upgrade with a new 'Downgrade'
type option set, and display a label on such records in the UI.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
There are a range of commits in OE-Core which cause parsing problems;
map them to the one that fixes it in order to avoid the problem. (This
will only be done if we're dealing with OE-Core as a dependency, rather
than the actual layer we're parsing).
(The second set are some commits during the python 3 conversion time.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Now that we're using RecipeSymbols we have the complete list of recipes
that ever existed in a layer. We only want to see the ones that are
valid for the selected milestone, so when a recipe gets deleted (or
renamed or moved outside of the layer subdirectory, if any) we need to
record that - do so using a RecipeUpgrade record with a new field
upgrade_type set to 'R'. Additionally we need to store the file path so
that deletion events (where we don't parse the contents of the recipe,
thus we don't have PN) are easy to match up with RecipeUpgrade records;
naturally we need to keep the paths "up-to-date" when we notice recipe
files being moved around.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Recipes come and go over time, so when a recipe gets deleted the history
for it goes away, which means that if you look back in time you do not
see an accurate picture - you only see the subset of recipes that are
currently present. Introduce an indirection between recipes and history
that allows for old recipes to persist (mostly in name only).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's best practice for security reasons to use shell=False and pass
command line arguments as a list; it also avoids some pain with
escaping, so let's use it everywhere we can (in fact we're only left
with one place in layerindex/tasks.py where we now pass shell=True).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The original TableSorter is unmaintained, move to the latest version of
the active fork:
https://mottie.github.io/tablesorter/docs/
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This should have been added in 2c8f979f9c
to make it possible to delete an account that is the administrator for a
maintenance plan without removing the maintenance plan.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If you had more than one layer on a maintenance plan, only the recipes
from the last layer were shown because we were using a recipe list
variable from the last iteration. The fix just turned out to be to
indent the code that read from that list so it runs every iteration
instead of at the end.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
For security reasons it's best practice to use parameters to pass values
into SQL statements and not substitute them in as strings (with Django's
database API the distinction is subtle, but we pass in the parameters in
a second list parameter instead of using % to substitute them before
passing the query in).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The recipe upstream history can't be reconstructed, so we need to be
able to export it out of an old database an import it into the new one.
There's also a "remove-duplicates" function; whilst not strictly
necessary it did help us to remove duplicate records resulting from
running the import twice (due to a bug) and may be useful in future
under similar circumstances.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The revision that happens to be checked out at the moment could be
anything and might not actually have a maintainers.inc, so check out the
proper branch before looking for it.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Best practices state that you should use a mechanism that ensures files
get closed in case of any error, so let's do that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Errors deleting bitbake.sock and bitbake.lock have been observed when
shutting down tinfoil at the end of some of these scripts. Move the code
used in the main layer index update script to a function in utils.py and
use it everywhere in order to avoid the issue.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add links to the same recipe in other branches in the recipe detail page
(and RRS recipe detail page) so that you can see which versions are
available in other branches and drill down to the detail if you want to.
Implements [YOCTO #13019].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We were using uitablefilter.js to provide live filtering of table rows
based upon a search field value, but it turns out this module really
isn't necessary - we can accomplish the same thing using simple jQuery
code. While we're at it, enable the search field on the layers list page
to work in conjunction with with the drop-down layer type selection, fix
pasting into the search field and refreshing with a search specified.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Make it possible for users to delete their own account and all associated
information from the database, should they decide they no longer wish to
use it.
(I checked the implications of doing this on our model structure -
anything with a foreign key to user is safe to delete with the exception
of RRS MaintenancePlan.admin which I needed to change on_delete for so
that it doesn't get deleted with the user).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
nvd3 and its python/django wrappers appear to be no longer actively
maintained, and at least the wrappers were a bit clunky to use. Looking
around for a suitable replacement, Chart.js seems capable, has no
additional dependencies and is fairly simple to use. As a bonus we get
to drop a few Python dependencies from our list.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The code in recipeparse.setup_layer() was trying to log a warning in the
case where LAYERRECOMMENDS not being satisfied, however there is no
actual logger object in this context. Pass it in via a parameter and
update all callers to pass it.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Use a more modern version of Bootstrap and take the opportunity to
upgrade jQuery to the latest version at the same time. This provides
better browser compatibility, moves to MIT license, allows us to make
the site more responsive for different devices in future, and provides
theming capabilities for custom installs among other improvements.
(I chose to upgrade to v3 for now rather than straight to v4 as it was
easier to do this gradually.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If you're creating a new release then you will need to create milestones
within it, so add in-line forms for these. Unfortunately there's no
capability yet to automatically split up the release into n milestones
(or perhaps copy milestones from a previous release), that will have to
wait until later - for now this makes things a little easier.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
In the current RRS we should expect that more than one recipe with the
same PN can exist in a layer - in fact it is common to have this (i.e.
multiple versions of the same recipe). Use the filename to match the
recipe record instead. At the same time, fix the exception handling.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If there is no release or milestone covering the current date, show an
error message at the top of the recipes list page to alert the user that
they can't view current data (since this is a common cause of not being
able to see that which may not be immediately apparent).
Additionally, show a warning within rrs_upgrade_history when this
happens.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If a release has no milestones, we shouldn't be selecting it as the
default to be linked to in the maintenance plan drop-down, so filter
those out.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
In the case of dry-run a couple of the scripts were breaking out after
one layerbranch had been processed due to the code structure. Handle the
exception within the block for the layerbranch to avoid this.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We do not want to be prompting the user for a password during layer
updates or upstream checks, e.g. in the case where a repo requires
authentication, or on github where any fetch of a nonexistent repo
apparently triggers authentication.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
REGEX, REGEX_URI and GITTAGREGEX are replaced by UPSTREAM_CHECK_REGEX,
UPSTREAM_CHECK_URI and UPSTREAM_CHECK_GITTAGREGEX
Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add logger.setLevel to enable set log level.
Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Although it's unlikely to be an issue, technically we shouldn't be
insisting the sha1 field be unique globally, just within each
layerbranch, so adjust the constraints.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If automated emails are enabled, we need to ensure that the other email
fields are populated, so validate that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Most layers do not track maintenance on a per-recipe basis, and for
those layers we will hide some of the per-recipe maintainer features
and on the recipe detail show the layer maintainer(s) as the
maintainer(s) of the recipe.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If we're in a repository containing multiple layers, we don't care about
commits that don't affect the layer we are processing, so skip those
commits rather than passing them to upgrade_history_internal.py which
will ignore them (which is significantly slower).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The processing code can't currently handle if a layerbranch is part of
more than one plan, so disallow that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The patterns() function is deprecated in Django 1.8 and gone in 1.10, so
we should switch over to the new list format.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
RecipeUpstreamHistory was not linked to the layer it was produced from,
which meant that it wasn't easy to query for a different maintenance
plan (i.e. a different layer) and thus the maintenance plan selection
on the recipe list didn't really work. Add a link field, populate it in
a migration and then make it required.
We had added a link earlier from RecipeMaintainerHistory to LayerBranch
but it was optional; for the same reasons we now populate it and make it
required.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
These two functions aren't being used anywhere. In the interest of
having as little code directly reading the database using SQL as
possible, drop them.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add an option to specify which maintenance plan to operate on (largely
for debugging purposes).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
