Make layerupdate collection slightly more reliable and make it easier
to see when updates have actually been captured:
* Split layerbranch into separate layer and branch fields, since there
may not be a layerbranch in existence but we might want to log an
error relating to the branch and layer.
* Show all layerupdates on the update detail page, not just those with
log messages
* Record before and after revisions and show these in the update detail
and layerupdate detail (with links)
* Record return code of update_layer process
* Highlight layer updates with a non-zero return code, errors or
warnings in the output on the update detail page
* Show duration on the layerupdate detail page
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If other distro comparison package(s) exist for a recipe being shown in
the RRS recipe detail page, link to the page for each package as well as
any extra URLs.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Add a few items
* Merge the top feature lines with the bottom ones
* Drop an item that has basically been satisfied by way of the REST API
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If a file is modified and renamed it will show up in both
iter_change_type('M') and iter_change_type('R'), however naturally the
file that will exist will be the b path and not the a one, so we should
be looking at the b path or we will get errors.
FYI you can reproduce this with OE-Core (in a scratch database) using
the following procedure:
1) (in the OE-Core layer directory):
git checkout 59285b324f6d9ed270b0bef209ef5da22a620a83
2) update.py -l openembedded-core -b master -x --nofetch -r --fullreload
3) (in the OE-Core layer directory):
git checkout 086308aa2a5e332de6f00ed397c4a55d132f158f
4) update.py -l openembedded-core -b master -x --nofetch
Without this change you'll see the following error:
ERROR: Unable to read /opt/layerindex/layers/git___git_openembedded_org_openembedded-core/meta/recipes-devtools/python-numpy/python-numpy_1.13.1.bb: Traceback (most recent call last):
File "/opt/layerindex/layers/bitbake/lib/bb/command.py", line 84, in runCommand
result = command_method(self, commandline)
File "/opt/layerindex/layers/bitbake/lib/bb/command.py", line 568, in parseRecipeFile
envdata = bb.cache.parse_recipe(config_data, fn, appendfiles)['']
File "/opt/layerindex/layers/bitbake/lib/bb/cache.py", line 315, in parse_recipe
bb_data = bb.parse.handle(bbfile, bb_data)
File "/opt/layerindex/layers/bitbake/lib/bb/parse/__init__.py", line 117, in handle
return h['handle'](fn, data, include)
File "/opt/layerindex/layers/bitbake/lib/bb/parse/parse_py/BBHandler.py", line 132, in handle
abs_fn = resolve_file(fn, d)
File "/opt/layerindex/layers/bitbake/lib/bb/parse/__init__.py", line 141, in resolve_file
raise IOError(errno.ENOENT, "file %s not found" % fn)
FileNotFoundError: [Errno 2] file /opt/layerindex/layers/git___git_openembedded_org_openembedded-core/meta/recipes-devtools/python-numpy/python-numpy_1.13.1.bb not found
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add another tab to search for classes.
Fixes [YOCTO #11207]
Signed-off by: Amanda Brindle <amanda.r.brindle@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This patch adds asynchronous task execution using a Celery backend
and RabbitMQ task queue, so that the layer submission process to
proceed even in the event that sending the notification email fails,
and establishing an asynchronous execution mechanism that we can use
in the future e.g. for triggering parse operations from the web UI.
This pertains to bug 11197:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=11197
It updates the README to reflect the installation and configuration
of a basic RabbitMQ setup, adds a 'tasks.py' file to contain task
definitions, updates the 'edit_layer_view' function to send
emails to administrators about new and updated layers asynchronously,
modifies the 'settings.py' to include a default configuration
for a RabbitMQ connection, and updates the Dockerfile to start a Celery
worker alongside the Gunicorn daemon.
Fixes [YOCTO #11197].
Signed-off-by: Diana Thayer <garbados@gmail.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
On the main layer index page we want openembedded-core to appear before
meta-oe - naturally openembedded-core is the layer everyone starts from.
We already have an "index_preference" column that we use for search
results, so just sort by that here before sorting by name.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
At the moment it's a bit difficult to get update logs out of the
environment in which the update script is being run. In order to make
the logs more accessible, create a LayerUpdate model to record the
output of update_layer.py separately for each layerbranch and tie the
created LayerUpdates together with a single Update model per session.
We provide two ways to look at this - a Tools->Updates page for
logged-in users, and there's also an "Updates" tab on each layer that is
accessible to anyone; which one is useful depends on whether you are
looking at the index as a whole or an individual layer.
Update records older than 30 days are deleted automatically by default.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Fix "data truncated" warnings/errors when loading data for OE-Core:
* PROVIDES for recipe packagegroup-base is ~1452 characters
* LICENSE for recipe linux-firmware is ~1053 characters
(These aren't shown with SQLite, only with something like MariaDB where
column sizes are enforced.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Fix a greedy regex in the recipe detail view resulting in the git recipe
listing all bbappends named *_git.bbappend as its bbappends (quite a few
in the public instance).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We took care of showing blacklist info and searching for the git recipe
recently. We've used "git fetch" rather than "git pull" for some time
now so we won't be hitting merge situations.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If you want to check what the same search returns on a different branch
it makes things easier if you don't have to re-type the search after
selecting the branch.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Drop the front page - this just gets in the way. Redirect to the
layers list instead. This has meant adding a touch more text to the
about page and adding the FAQ link to the footer.
* Use a separate navbar to hold the branch selector and the main
top-level pages (Layers, Recipes, Machines) instead of tabs
* All pages depending on a branch selection are now under
branch/<branchname>/ so we don't need to have the branch selection on
every page.
* Use breadcrumbs on recipe detail and layer detail pages instead of
tabs
* Add title to recipe detail page
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Having the page content dance about when switching between pages just
because one has a scrollbar and the other doesn't is annoying, so just
force the scrollbar to show all of the time to avoid this.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This allows users to easily tick and untick several items at once
without having to drop the filter box down multiple times. Thanks to
Belen for this snippet.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The recipe detail pages use the recipe record id, so in case users
bookmark links to these pages, try to preserve the id when a recipe
upgrade occurs (which usually shows up as a delete and an add). This
also works when doing a full update with -r so that we can capture the
newly recorded items (bbclasses, bbappends) without having all the
recipe id values change.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This provides a way to set "meta" fields (SUMMARY, DESCRIPTION,
HOMEPAGE, BUGTRACKER, SECTION, and LICENSE) for a number of recipes at
once, and then download those changes in the form of one or more patch
files which can be submitted for merging into the layer.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Collect bbappend/bbclass info during the update process and display it
on the layer detail page.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Show BUGTRACKER in the recipe detail page if set
* Track the other two variables for possible future use
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add an index_preference field to enable control over which layer's
duplicate recipes get de-emphasised in the recipe search results.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Save individual field changes into revision comment and display this
comment on the history page. Now we're ready to add a link at the
bottom of every page so the history is easily visible.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Report layer which failed to fetch in error message
* Don't retry fetching a repo if it already failed for another layer
(where more than one layer is in the same repository)
* Exit immediately if all fetches failed
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's easy to forget you need to use -l to specify the list of layers to
update and previously we just ignored unused arguments, so check for
them and error out if present.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If another recipe exists with the same name in a different layer and
that other layer is "older" (by layer ID) and is a software or base
layer then lighten the recipe entry in the search results as it may
not be the preferred version (e.g. recipes in BSP or distro layers may
have been customised specifically for the machine or distro).
This has had a performance impact on the recipes list; as a result
showing all recipes by default has been disabled. If the user really
wants to see all recipes they can just leave the search box blank and
hit the search button.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add a badge next to the Review link (when shown for users with the
publish permission) on all pages showing how many layers need to be
reviewed, if any.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This should prevent spamming even though this is less likely with this
kind of site.
The CAPTCHA does not show when editing, only submitting, and is also not
shown for authenticated users.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If we get bogus or duplicate entries we'll want the ability to delete
them easily before publishing (without needing to have access to the
admin interface), so add this ability. Being able to delete a published
layer might be a bit dangerous and is less likely to be needed so that
is disallowed for now.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This allows adding an important notice to a layer e.g. "this layer is
deprecated, please use layer xyz instead". Only one layer note can be
added through the interface although the data structures allow multiple,
so notes may be added programmatically without disturbing user-added
ones.
With this change we also add a get_absolute_url() function to the
LayerItem model and change the calls to reverse() for layers to use it.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Show "unpublished" rather than "new" in dependencies list in submit
form, and mention publishing as the operation in the "thanks for
submitting" page.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Allow users with publish permission to edit any layer, and users with
the same email address as one of the maintainers of a layer to edit that
layer.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Look for conf/machine/*.conf and add a record for each file found. These
are displayed in the layer detail page if any are present.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
These can be used to show important notes (e.g. "This layer is
deprecated, use layer xyz instead.") as eye-catching alerts at the top
of the detail page.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Changed to full screen layout to improve the table display
and get rid of the accordion for the layer information.
Signed-off-by: Belen Barros <belen.barros.pena@intel.com>
