Typo, there has to be a ? in front of the id or otherwise we don't get
linked to the right commit. This would have affected recently added layers
with a cgit web frontend.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add the ability to compare available recipes and their versions between
two branches for a selection of layers (default is just OE-Core). This
was mainly intended to help us with the Yocto Project release notes
preparation (hence the "Plain text" button at the bottom of the page)
but is also useful in its own right.
Note: for readability, SRCREVs are only shown when PV has not changed.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
For the purposes of the branch comparison function I'm about to add it
would be useful to track the value of SRCREV, so we can see if it has
changed even if PV hasn't.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Just because git.yoctoproject.org is in the URL, doesn't mean we can or
should force the vcs_web_url to be a specific value. If it starts with
git://git.yoctoproject.org then we can do this. git.openembedded.org
already did this.
This also changes github, gitlab and bitbucket references.
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add a new BITBAKE_PATH to the settings file to specify the path within the
BITBAKE_REPO_URL where bitbake lives. This is useful when using a combined
repository, such as poky, that contains bitbake, openembedded-core and other
layers.
This change also changes the default path, in the fetch directory, for the
bitbake checkout. It no longer uses the path 'bitbake', but instead uses the
same URL processing as the layer fetching.
There is a side effect that, when using a shared fetch, the branch of the
layer will be used instead of the specified bitbake branch. Generally this
is a reasonable compromise, since in a combined repository bitbake and
openembedded-core component should already match.
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Without this change the system will fail parsing various URL components
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Where we use glyphicons to mark items in a list, ensure there is a space
between the item and the icon to make things look a bit neater.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Bump a few versions where we can
* Drop anyjson - this used to be a dependency of kombu but not anymore,
and nothing else needs it.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This change is to record PE (epoch) and PR (release) values for recipes.
These values will be reflected in the REST API but will not be exposed
in the UI. The aim of this change is to enrich metadata availability to
consumers of the feed.
Signed-off-by: Pranay Mankad <pranaymankad@gmail.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
On some systems it seems the order of fields in a values() query aren't
guaranteed, so in order to avoid the contents dancing around too much,
explicitly write out the fields in the order we expect them to be in.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Drop some explicit outdated version numbers
* You do actually have to create the database with MariaDB etc. so add
instructions as to how for convenience
* Add pointer to virtualenv in python requirements item
* Add libssl-dev package to install command
* Use python3-venv instead of virtualenv package in install command
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Unfortunately the "updated" date gets changed every time the repo is
fetched (i.e. vcs_last_fetch changes), but that doesn't mean the
layerbranch or anything under it has actually changed. Use vcs_last_rev
instead although unfortunately that won't catch manual edits to the
layerbranch alone.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If you enable the --letsencrypt option when you run dockersetup.py, the
script will modify the volume mount for the certificates to point to
/etc/letsencrypt instead of /opt/cert. If you then run dockersetup.py
again (with -r/--reinstall) without --letsencrypt, we want the path to
be set back to /opt/cert, so ensure that it does. Additionally, the code
wasn't actually setting the path for the layerscertbot service since
editing that section was done separately. (Admittedly, the letsencrypt
functionality has not been well-tested.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Provide a way to export and import other distro recipe coverage in
update_classic_status.py.
(Based on the Clear Linux Dissector commit
ff38e9582093453e13b90e06af125374ca4b0a16).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
I'm about to add a few command line options to this script, so switch
over to argparse before that which is more modern and a bit more
flexible.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Don't confuse proxy options being defaulted through from the environment
and the user explicitly specifying them. Also look at no_proxy option.
Fixes https://github.com/intel/clear-linux-dissector-web/issues/13
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add the ability to uninstall the application for the user's convenience.
(Note that this does not undo the changes to the configuration, it only
removes the Docker containers and volumes.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* We need a SOCKS proxy to support fetching from git:// or ssh:// URLs
within the container, so add an option to specify it
* It's possible for the http and https proxy settings to be the same, so
set one from the other if only one of them is set.
* If we want to be able to fetch from internal servers inside the proxy
then we also need a "no-proxy" list, so add support for that.
* It's not unlikely that machines within networks requiring use of a
proxy for external network access will have all of the proxy settings
set in the environment, so we can try to pick up the defaults from
there.
* Ensure that we can switch from proxy to no proxy (when reinstalling)
which means we always need to edit the config files and ensure the
proxy options get commented out if we don't want them set.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's easy to get the proxy settings wrong and not realise until you've
got quite a long way into the process of setting things up. Thus, add a
check where we actually try to fetch various things within the container
environment and fail reasonably early if things aren't working.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
I forgot the lesson of f5922091b4 which
was that djangorestframework has a very silly default of registering
the model name as the key for finding the ViewSet, with the result that
the recently created RecipesExtendedViewSet was also being used for the
recipes URL which we expect to point to RecipeViewSet; this broke the
bitbake-layers layerindex-* commands.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The new viewset has a different URL and uses pagination, so we need to
take that into account.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Adding these extra child items to the standard "recipes" viewset (which
we did recently in 684a06a383) means that
some current usage is impractical due to the size of the returned list
of items. Instead, create a recipesExtended viewset, move the new child
items to that and add pagination to avoid result size issues.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This is now be practical to use for updating as well as the initial
import. Additionally the user doesn't need to be using the debug
option by default so drop it from the example command.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If a LayerBranch is on a branch that is in the remote layer index (and
that branch is in any branch list specified with -b/--branch) and the
layer for the LayerBranch is not found in the remote layerindex then it
should be deleted, otherwise we could end up with stale data.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add an option to import only certain layers - mostly for debugging
(can make testing a lot quicker).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We only work with branches that exist in both the local database and the
remote index, but allow the user to restrict the list further if
desired.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This is not a comprehensive test, but does provide a way to easily
compare that what was originally fetched for a recipe from the API is
the same once it's been imported into a new instance using
import_layers.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Import recipe data from the REST API, including sub-recipe objects
(sources, patches, PACKAGE_CONFIGs, dependencies etc.) now that they are
available from the API. Also, with the data being more or less complete,
it's also reasonable to also set the vcs_* fields so that we can tell
how up-to-date it is.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Don't assume we need to create these objects - check the updated field
value against what the API reports and update the existing object if the
latter is newer.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
For LayerDependency, LayerMaintainer and LayerNote objects, properly
sync the objects in the database with the objects fetched from the API.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
These are a little less useful than the other items but if we want to be
able to create a clone of a layer index then we need them.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We're tracking these items for each recipe, so they should be
accessible via the API as well.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The try-specfile subcommand is for testing, so it's entirely possible
the spec file we are testing with will be in the current directory;
os.path.dirname() returns '' in this case, which os.path.relpath() then
chokes on. Avoid this in the easiest way possible by just prepending ./
to the path in this case.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's not common, but if someone saves a spec file with a non-UTF8
encoding then the surrogate characters were preventing the data from
being saved to the database (at least with MariaDB). Instead of assuming
UTF8, try a few common encodings on the first 100 lines.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add a SECURITY_QUESTIONS_REQUIRED setting that defaults to True, but if
set to False then a user who has not set security questions will still
be allowed to reset their password. This is convenient for the OE Layer
index because there are a number of existing accounts, none of which
will have security questions set.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
