I noticed in the branch comparison plain text view ("Plain text" button
in the Tools -> Branch Comparison page) that in current versions of
Firefox the ellipses were coming through corrupted, though they looked
fine in the HTML version, and it turns out this is because I wasn't
specifying a character set encoding. It should be UTF-8, so add a
charset to the content type stating as such for this and other similar
views.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.microsoft.com>
If a BBHandledException occurs that means some error was logged, so we
need to handle any pending events so that we can actually have the error
logged. Tinfoil should really be doing this for us but at this stage in
the release we can't really fix this there, so do it here for now.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.microsoft.com>
So with honister / current master we can no longer get away with
bypassing BBLAYERS - it now needs to point to the core layer at
minimum. This is fine, we just need to skip parsing layer.conf if we're
parsing the core layer or we get some extra warnings we don't need.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.microsoft.com>
Added SPDX identifiers to all .py files except those in migrations directory.
Fixes: [YOCTO #13527]
Signed-off-by: Meh Mbeh Ida Delphine <idadelm@gmail.com>
Signed-off-by: Paul Eggleton <bluelightning@bluelightning.org>
Removed all obsolete references to django.core.urlresolvers. Added the
newly required on_delete fields to foreign key relationships in models.py
and in all migrations.
Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
Signed-off-by: Paul Eggleton <bluelightning@bluelightning.org>
In OE-Core commit 8c9ef587fe499c612a878a1ab42092eb79b334ef an
AVAILABLE_LICENSES variable was added in license.bbclass where its value
is assigned with immediate expansion ( := ) with the result that it
looks in LICENSE_PATH for licenses; in turn LICENSE_PATH refers to
COREBASE. You might assume that COREBASE is always set, and normally it
is (since it's set OE-Core's layer.conf) - but in the layer index
context we do not parse layer.conf until a bit later, so it immediately
fails. The quick way to fix this is just to set our own (correct) value
for COREBASE and then AVAILABLE_LICENSES can be expanded successfully.
You might ask why we don't instead just set BBLAYERS such that we *do*
parse OE-Core's layer.conf - the answer is that that can have other
effects such as BBFILE_COLLECTIONS being set, and at least at the moment
the rest of the code isn't expecting that.
Fixes [YOCTO #13723].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Reloading an existing update task page was taking an extremely long time
to fetch down the task log and then pegging the client CPU such that the
browser gave a warning. Digging into it, logs from a Clear Linux update
task can be of the order of 500MB in size (with all of the line
refreshing using CRs that happens during downloads), causing (a) the
transfer to take a long time and (b) the JS code that updates the log
text box to be extremely busy. If we're loading the entire log from
scratch (as we are when we refresh the page) rather than just getting an
update since the last poll, we don't need any of those line refreshes -
so squash them out before returning the data.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If a recipe dependency (either static or dynamic) is removed from the
recipe when it is parsed, then we should ensure it gets removed from the
database as well.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Quite a few bugs fixed, as far as I know none that we observed, but good
to have. Details can be found here:
https://github.com/chartjs/Chart.js/releases
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
When using the layer: advanced query term, if you want to match on
OE-Core, its actual layer name is "openembedded-core", but people will
naturally assume that "oe-core" should work, so make it so (case
insensitive).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Allow filtering only on recipe name - i.e., searching for "git" finds
any recipe with "git" in the name or description. Now, you can search
for "pn:git" which will return only recipes with the name "git".
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Use shlex.split() to split the query string so that quotes can be used
to delimit strings containing spaces to be matched as a whole. This
worked with the previous code, but it did not support single quotes -
these caused an error in Django's filter code and thus an internal
server error (as did querying for ""). Add some additional checks for
single quotes as it is still possible to get them past shlex.split()
e.g. with something like "'hello'" (with quotes).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Fixes the bitbakepath variable not being defined with -x/--nofetch
specified.
(Regression introduced in c91372587bbddd4c595d7202e51a8740b787a06e.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This was added when we brought over a patch from the Clear Linux
Dissector, but here we're not using diff2html here so we shouldn't have
this either.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add an extra tool that lets you view all of the recipe dependencies in
a layer. There is also a mode that shows only cross-layer dependencies,
which can be useful to find dependencies on recipes in other layers
that aren't declared in the layer's dependencies (or conversely where a
layer dependency is no longer necessary).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
You'd think this is very unlikely to happen, but back in
meta-openembedded commit 415e213ad75ec9a93171c963395a1c4b92c6233b and
the commits preceding it, a recipe was added to the root of the
repository and then moved into place, and os.path.relpath() does not
like to be called with a blank path and thus raises an exception. To
avoid the exception, get the relative path to the filename and then chop
that off instead of the other way around.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add the ability to compare available recipes and their versions between
two branches for a selection of layers (default is just OE-Core). This
was mainly intended to help us with the Yocto Project release notes
preparation (hence the "Plain text" button at the bottom of the page)
but is also useful in its own right.
Note: for readability, SRCREVs are only shown when PV has not changed.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
For the purposes of the branch comparison function I'm about to add it
would be useful to track the value of SRCREV, so we can see if it has
changed even if PV hasn't.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Just because git.yoctoproject.org is in the URL, doesn't mean we can or
should force the vcs_web_url to be a specific value. If it starts with
git://git.yoctoproject.org then we can do this. git.openembedded.org
already did this.
This also changes github, gitlab and bitbucket references.
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add a new BITBAKE_PATH to the settings file to specify the path within the
BITBAKE_REPO_URL where bitbake lives. This is useful when using a combined
repository, such as poky, that contains bitbake, openembedded-core and other
layers.
This change also changes the default path, in the fetch directory, for the
bitbake checkout. It no longer uses the path 'bitbake', but instead uses the
same URL processing as the layer fetching.
There is a side effect that, when using a shared fetch, the branch of the
layer will be used instead of the specified bitbake branch. Generally this
is a reasonable compromise, since in a combined repository bitbake and
openembedded-core component should already match.
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Without this change the system will fail parsing various URL components
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This change is to record PE (epoch) and PR (release) values for recipes.
These values will be reflected in the REST API but will not be exposed
in the UI. The aim of this change is to enrich metadata availability to
consumers of the feed.
Signed-off-by: Pranay Mankad <pranaymankad@gmail.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
On some systems it seems the order of fields in a values() query aren't
guaranteed, so in order to avoid the contents dancing around too much,
explicitly write out the fields in the order we expect them to be in.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Unfortunately the "updated" date gets changed every time the repo is
fetched (i.e. vcs_last_fetch changes), but that doesn't mean the
layerbranch or anything under it has actually changed. Use vcs_last_rev
instead although unfortunately that won't catch manual edits to the
layerbranch alone.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Provide a way to export and import other distro recipe coverage in
update_classic_status.py.
(Based on the Clear Linux Dissector commit
ff38e9582093453e13b90e06af125374ca4b0a16).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
I'm about to add a few command line options to this script, so switch
over to argparse before that which is more modern and a bit more
flexible.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
I forgot the lesson of f5922091b4 which
was that djangorestframework has a very silly default of registering
the model name as the key for finding the ViewSet, with the result that
the recently created RecipesExtendedViewSet was also being used for the
recipes URL which we expect to point to RecipeViewSet; this broke the
bitbake-layers layerindex-* commands.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The new viewset has a different URL and uses pagination, so we need to
take that into account.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Adding these extra child items to the standard "recipes" viewset (which
we did recently in 684a06a383) means that
some current usage is impractical due to the size of the returned list
of items. Instead, create a recipesExtended viewset, move the new child
items to that and add pagination to avoid result size issues.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If a LayerBranch is on a branch that is in the remote layer index (and
that branch is in any branch list specified with -b/--branch) and the
layer for the LayerBranch is not found in the remote layerindex then it
should be deleted, otherwise we could end up with stale data.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add an option to import only certain layers - mostly for debugging
(can make testing a lot quicker).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We only work with branches that exist in both the local database and the
remote index, but allow the user to restrict the list further if
desired.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This is not a comprehensive test, but does provide a way to easily
compare that what was originally fetched for a recipe from the API is
the same once it's been imported into a new instance using
import_layers.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Import recipe data from the REST API, including sub-recipe objects
(sources, patches, PACKAGE_CONFIGs, dependencies etc.) now that they are
available from the API. Also, with the data being more or less complete,
it's also reasonable to also set the vcs_* fields so that we can tell
how up-to-date it is.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Don't assume we need to create these objects - check the updated field
value against what the API reports and update the existing object if the
latter is newer.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>