If a LAYERRECOMMENDS relationship is not satisfied, we shouldn't be
erroring out - it's a recommendation, not a hard dependency. Just show a
warning and allow processing to continue.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If a file is modified and renamed it will show up in both
iter_change_type('M') and iter_change_type('R'), however naturally the
file that will exist will be the b path and not the a one, so we should
be looking at the b path or we will get errors.
FYI you can reproduce this with OE-Core (in a scratch database) using
the following procedure:
1) (in the OE-Core layer directory):
git checkout 59285b324f6d9ed270b0bef209ef5da22a620a83
2) update.py -l openembedded-core -b master -x --nofetch -r --fullreload
3) (in the OE-Core layer directory):
git checkout 086308aa2a5e332de6f00ed397c4a55d132f158f
4) update.py -l openembedded-core -b master -x --nofetch
Without this change you'll see the following error:
ERROR: Unable to read /opt/layerindex/layers/git___git_openembedded_org_openembedded-core/meta/recipes-devtools/python-numpy/python-numpy_1.13.1.bb: Traceback (most recent call last):
File "/opt/layerindex/layers/bitbake/lib/bb/command.py", line 84, in runCommand
result = command_method(self, commandline)
File "/opt/layerindex/layers/bitbake/lib/bb/command.py", line 568, in parseRecipeFile
envdata = bb.cache.parse_recipe(config_data, fn, appendfiles)['']
File "/opt/layerindex/layers/bitbake/lib/bb/cache.py", line 315, in parse_recipe
bb_data = bb.parse.handle(bbfile, bb_data)
File "/opt/layerindex/layers/bitbake/lib/bb/parse/__init__.py", line 117, in handle
return h['handle'](fn, data, include)
File "/opt/layerindex/layers/bitbake/lib/bb/parse/parse_py/BBHandler.py", line 132, in handle
abs_fn = resolve_file(fn, d)
File "/opt/layerindex/layers/bitbake/lib/bb/parse/__init__.py", line 141, in resolve_file
raise IOError(errno.ENOENT, "file %s not found" % fn)
FileNotFoundError: [Errno 2] file /opt/layerindex/layers/git___git_openembedded_org_openembedded-core/meta/recipes-devtools/python-numpy/python-numpy_1.13.1.bb not found
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If you're running a testing / internal instance then you really don't
want to be emailing maintainers on publish, so provide a setting you can
use to disable that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Fixed:
Assume there is no master branch in hello layer:
$ update.py -l hello -b master
INFO: Skipping update of layer hello - branch master doesn't exist
This is correct since hello layer doesn't have master branch, but when --nocheckout:
$ update.py -l hello -b master --nocheckout
[snip]
INFO: Sorting layers for branch mater:
WARNING: Cannot find required collections on branch master:
WARNING: hello: LAYERDEPENDS: <snip>
This is incorrect, this patch fixed the problem, now it skips it since the
branch doesn't exists when --nocheckout.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
When layer_a RECOMMENDS layer_b, try to add layer_b before layer_a, but if
layer_b is not found, still add layer_a.
And print summary error mesage:
$ update.py -b master
ERROR: Issues found on branch master:
openembedded-core: Added without LAYERRECOMMENDS
meta-secure-env: Failed to add since LAYERDEPENDS is not satisfied
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Fixed:
$ git clone <url>
warning: remote HEAD refers to nonexistent ref, unable to checkout.
$ git rev-parse HEAD
HEAD
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
Catch the error and avoid that.
And use "git reset --hard" to replace of "git reset --hard HEAD", HEAD is
default for git reset, so they are the same, but the later one reports error
when remote HEAD doesn't exist:
$ git reset --hard HEAD
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
[snip]
$ git reset --hard
No errors.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Fixed:
$ update.py -b <new_branch>
[snip]
NOTE: Starting bitbake server...
Traceback (most recent call last):
File "update_layer.py", line 471, in main
utils.setup_core_layer_sys_path(settings, branch.name)
File "/buildarea1/lyang1/layerindex-web/layerindex/utils.py", line 376, in setup_core_layer_sys_path
core_layerdir = os.path.join(core_repodir, core_layerbranch.vcs_subdir)
AttributeError: 'NoneType' object has no attribute 'vcs_subdir'
[snip]
This is because core_layerbranch is not in database yet for completely new
branch, so it is None and we will get the error. Avoid calling
setup_core_layer_sys_path() when "update_layer.py --initial" will fix the
problem.
And also only add core layer's sys path when it is present, since core layer
may not be added yet for completely new branch.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
REGEX, REGEX_URI and GITTAGREGEX are replaced by UPSTREAM_CHECK_REGEX,
UPSTREAM_CHECK_URI and UPSTREAM_CHECK_GITTAGREGEX
Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add logger.setLevel to enable set log level.
Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add the ability to export the recipe listing for a layer to a CSV file
for importing into external tools. At the moment we include name,
version and license, but there is a parameter that lets you specify the
fields to include in the URL if desired.
Implements [YOCTO #12722].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We're about to replace this with a proper CSV export function, so we
don't need this dead code hanging around anymore.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Although it's unlikely to be an issue, technically we shouldn't be
insisting the sha1 field be unique globally, just within each
layerbranch, so adjust the constraints.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If automated emails are enabled, we need to ensure that the other email
fields are populated, so validate that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Most layers do not track maintenance on a per-recipe basis, and for
those layers we will hide some of the per-recipe maintainer features
and on the recipe detail show the layer maintainer(s) as the
maintainer(s) of the recipe.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If we're in a repository containing multiple layers, we don't care about
commits that don't affect the layer we are processing, so skip those
commits rather than passing them to upgrade_history_internal.py which
will ignore them (which is significantly slower).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The processing code can't currently handle if a layerbranch is part of
more than one plan, so disallow that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
I can't quite tell which Django version broke this, but in any case
based on what's in pagination.html it seems we ought to have been
using == already.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The patterns() function is deprecated in Django 1.8 and gone in 1.10, so
we should switch over to the new list format.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
RecipeUpstreamHistory was not linked to the layer it was produced from,
which meant that it wasn't easy to query for a different maintenance
plan (i.e. a different layer) and thus the maintenance plan selection
on the recipe list didn't really work. Add a link field, populate it in
a migration and then make it required.
We had added a link earlier from RecipeMaintainerHistory to LayerBranch
but it was optional; for the same reasons we now populate it and make it
required.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
These two functions aren't being used anywhere. In the interest of
having as little code directly reading the database using SQL as
possible, drop them.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add an option to specify which maintenance plan to operate on (largely
for debugging purposes).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Ensure the Release and Milestone names are separated by a space when
listing Milesones
* Include the maintenance plan name in the name shown for each
Release/Milestone
* Allow filtering Releases/Milestones by maintenance plan
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's a pain to have to add all the releases when adding a new
maintenance plan. Since these are likely to be the same (or similar) for
every plan, then duplicate them across from the first plan when you save
a new one.
Also add "default" milestones on the assumption that other layers
probably won't want to use the 4-milestone split per release, but there
do have to be some milestone records, so just create one milestone for
each release.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's a bit of a pain to have to set the two python environment fields on
every record in order to have things set correctly, and it can easily
get forgotten, so try to set them automatically by default (assuming
reasonable naming).
Note that this does introduce an annoying behaviour whereby if you click
"Add another Maintenance plan layer branch" and then decide you don't
want it, the admin form will insist you fill in the fields unless you
clear out the python2/3 environment fields. I'm not sure how to fix
that, so I'm leaving it as-is for now.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
I missed changing this constraint when adding the plan field. We want to
be able to have the same named release on another plan.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Expose the newly added patch information in the RRS:
* Add a table to the recipe detail listing the patches for the recipe
* Add pending / total counts to the recipe list page
Implements [YOCTO #7909].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Collect information about patches applied by a recipe, and record each
patch along with the upstream status, presenting them in the recipe
detail.
Implements [YOCTO #7909].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Upon consideration, for the width of the information we want to present
we do actually want full-width pages for the RRS. When this was changed
earlier in the rrs branch it was changed in the base template, but we
want to keep the same style elsewhere, so put a block in that will let
use the "container-fluid" style (full width) in the RRS pages and
"container" by default everywhere else.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's often useful to add a link to another page for more information
when adding a layer note, so turn any included URLs into actual links.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Consolidate the code for checking out a repository, using the newly
added utils.checkout_repo() function
* Check out a layer's dependencies, not just the layer itself
* Only check out if the desired revision isn't already checked out
(mostly useful for bitbake which we would otherwise be checking out
much more frequently than necessary since it may not have changed
even if we've moved to a new commit in the layer).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Checking out a revision in the bitbake/layer repositories is something
we are doing in a few places, so add a checkout_repo() function that
does this, ensuring that we don't get tripped up by any junk files,
and avoids checking out if the repository is already at the desired
revision (thus avoiding the clean operation and e.g. preserving any
.pyc files that aren't stale and would speed things up a little). Note
that we do the clean before checking out in case there are untracked
files that are tracked in the commit we are checking out.
In addition to adding this function, change the existing code that we
use in the update script to check out a layer use the new function.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It would be useful in some scenarios to get the complete list of
recursive dependencies for a layer, so add a function to do that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The Recipe Reporting System needs to be able to provide links to commits
in the web interface for the repository, but we can only do this if we
have a custom template URL just like we do for file/tree links, since
it's different for different git web interfaces. Add support in all the
various places for such a URL and make use of it in the RRS.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
In the rrs branch we used to just store SRC_URI in a field, however we
now have a proper model to store sources, so use that in the RRS recipe
detail page.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Save each remote SRC_URI so we can use these for the recipe reporting
system. This replaces an earlier implementation in the rrs branch where
we simply stored SRC_URI verbatim.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Every time we add something that links to Recipe we had to add it to the
exclusions list in the readonly_fields line for RecipeAdmin (and
ClassicRecipeAdmin), which is tedious and easily forgotten. We can avoid
this by looking at each field and excluding it by its attributes rather
than having a hardcoded list of names.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If the RRS is enabled, then add a link to it in the tools menu. I don't
expect this to be used a lot, but otherwise the only way you'd get there
would be either externally or via the link from the layer detail.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add a convenience link to the layer detail to the breadcrumb (also as an
indicator, since it's possible to have more than one layer in the
maintenance plan).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Provide a mechanism set the maintainer for things like gcc-cross-<arch>
to the same as gcc. (We do have entries in the .inc file for these,
however they aren't useful as they don't match the recipe name when we
parse it, and due to the fact that RecipeMaintainer objects link
directly to Recipe objects, we can't handle entries that don't map to a
real recipe).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We don't actually need to check out the repository until we actually
analyse a commit, so avoid doing so. Additionally, there's not much
point in checking out master at the end, let the next script invocation
do that if needed (if it needs to, it should since otherwise there's no
guarantee what state the repository is in).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add an option that deletes all maintainer history records for the
current layer branch so that they can then be reloaded from scratch.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We don't really need to clean up the repository, we'll take care of it
on subsequent runs.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If a recipe upgrade only consists of a .inc file changing, we were not
picking it up, since we were only looking specifically for recipes
(.bb). If a .inc file changes, assume all .bb files in the same
directory (if any) should be parsed to look for an upgrade.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
load_recipes() was leaving files around in /tmp; on my Fedora system
this eventually resulted in /tmp running out of space which we do not
want.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Drop outdated information, fix typos/spelling/grammar, improve
formatting and add some new steps.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
