Version downgrades (or what appear to be downgrades) do occasionally
happen, and if they did then the RRS was previously simply ignoring
them, resulting in the latest version being reported incorrectly.
Allow downgrades to be recorded as an upgrade with a new 'Downgrade'
type option set, and display a label on such records in the UI.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
There are a range of commits in OE-Core which cause parsing problems;
map them to the one that fixes it in order to avoid the problem. (This
will only be done if we're dealing with OE-Core as a dependency, rather
than the actual layer we're parsing).
(The second set are some commits during the python 3 conversion time.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Now that we're using RecipeSymbols we have the complete list of recipes
that ever existed in a layer. We only want to see the ones that are
valid for the selected milestone, so when a recipe gets deleted (or
renamed or moved outside of the layer subdirectory, if any) we need to
record that - do so using a RecipeUpgrade record with a new field
upgrade_type set to 'R'. Additionally we need to store the file path so
that deletion events (where we don't parse the contents of the recipe,
thus we don't have PN) are easy to match up with RecipeUpgrade records;
naturally we need to keep the paths "up-to-date" when we notice recipe
files being moved around.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Recipes come and go over time, so when a recipe gets deleted the history
for it goes away, which means that if you look back in time you do not
see an accurate picture - you only see the subset of recipes that are
currently present. Introduce an indirection between recipes and history
that allows for old recipes to persist (mostly in name only).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's best practice for security reasons to use shell=False and pass
command line arguments as a list; it also avoids some pain with
escaping, so let's use it everywhere we can (in fact we're only left
with one place in layerindex/tasks.py where we now pass shell=True).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Errors deleting bitbake.sock and bitbake.lock have been observed when
shutting down tinfoil at the end of some of these scripts. Move the code
used in the main layer index update script to a function in utils.py and
use it everywhere in order to avoid the issue.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Consolidate the code for checking out a repository, using the newly
added utils.checkout_repo() function
* Check out a layer's dependencies, not just the layer itself
* Only check out if the desired revision isn't already checked out
(mostly useful for bitbake which we would otherwise be checking out
much more frequently than necessary since it may not have changed
even if we've moved to a new commit in the layer).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If a recipe upgrade only consists of a .inc file changing, we were not
picking it up, since we were only looking specifically for recipes
(.bb). If a .inc file changes, assume all .bb files in the same
directory (if any) should be parsed to look for an upgrade.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
load_recipes() was leaving files around in /tmp; on my Fedora system
this eventually resulted in /tmp running out of space which we do not
want.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We were parsing recipes that were in the repository but not inside the
actual layer we're dealing with (e.g. we have meta-selftest within the
OE-Core repository, containing a number of recipes that are only
intended for testing purposes and should not be looked at by this
script).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If an exception occurs during parsing, let's actually see what kind of
exception it was in the output.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
In OE-Core revision e0531174119bff21e9014b95ed1bbd0e1c01af26 we
accidentally committed a new e2fsprogs recipe with ..bb at the end of
its name instead of .bb. This was fixed immediately afterwards, but when
the RRS hits this commit, it doesn't fail immediately, but the bogus
version "1.43." gets into the database and all subsequent commits
touching the e2fsprogs recipe cause bb.utils.vercmp_part() to blow up
because one of the version parts in the "previous" version in the database
is apparently empty. To work around this and any similar issues, just
reject any change that results in such a broken version string (on the
assumption that it'll be corrected in a subsequent commit and thus we
will get to re-parse the recipe then and therefore not miss the
upgrade.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We should expect multiple matches for layerbranch + pn, so use filter()
instead of get() and take the first id that matches.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Use try...finally to ensure we shut down tinfoil, but since we are
potentially dealing with older bitbake releases when importing older
upgrades, only call shutdown() if it's actually there (and although it's
unlikely, guard against the broken shutdown() in fido as we do in the
main layer index update script).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If you want to go back and get history for the earlier releases (krogoth
and previous) then we need to be able to support both python 2 and 3,
which practically means we need the same split for this script as we
have for the main layer index update script.
The catch here is that since we are going back and following the history
of changes forward, we basically need to use the same version of bitbake
that was current at that time. This works except for around the
transition between python 2 to 3 where the metadata lagged behind a bit,
so we need to take that into account. In order to keep things generic we
have a date field on the maintenance plan layer branch that specifies
the date in the metadata where we should switch over to python 3, and
then link to PythonEnvironment records that should be used for python 2
and 3 respectively.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
