If a LayerBranch is on a branch that is in the remote layer index (and
that branch is in any branch list specified with -b/--branch) and the
layer for the LayerBranch is not found in the remote layerindex then it
should be deleted, otherwise we could end up with stale data.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add an option to import only certain layers - mostly for debugging
(can make testing a lot quicker).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We only work with branches that exist in both the local database and the
remote index, but allow the user to restrict the list further if
desired.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This is not a comprehensive test, but does provide a way to easily
compare that what was originally fetched for a recipe from the API is
the same once it's been imported into a new instance using
import_layers.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Import recipe data from the REST API, including sub-recipe objects
(sources, patches, PACKAGE_CONFIGs, dependencies etc.) now that they are
available from the API. Also, with the data being more or less complete,
it's also reasonable to also set the vcs_* fields so that we can tell
how up-to-date it is.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Don't assume we need to create these objects - check the updated field
value against what the API reports and update the existing object if the
latter is newer.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
For LayerDependency, LayerMaintainer and LayerNote objects, properly
sync the objects in the database with the objects fetched from the API.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
These are a little less useful than the other items but if we want to be
able to create a clone of a layer index then we need them.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We're tracking these items for each recipe, so they should be
accessible via the API as well.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The try-specfile subcommand is for testing, so it's entirely possible
the spec file we are testing with will be in the current directory;
os.path.dirname() returns '' in this case, which os.path.relpath() then
chokes on. Avoid this in the easiest way possible by just prepending ./
to the path in this case.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's not common, but if someone saves a spec file with a non-UTF8
encoding then the surrogate characters were preventing the data from
being saved to the database (at least with MariaDB). Instead of assuming
UTF8, try a few common encodings on the first 100 lines.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add a SECURITY_QUESTIONS_REQUIRED setting that defaults to True, but if
set to False then a user who has not set security questions will still
be allowed to reset their password. This is convenient for the OE Layer
index because there are a number of existing accounts, none of which
will have security questions set.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If you clicked on logout in the admin site then clicked on the "Login"
button and logged in, you were redirected to the logout page because we
weren't detecting that the redirection should not be applied.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This is the only dependency we can update right now; the rest will cause
problems given that the layer index needs python 2 support still.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Tweak message when one or more (but not all) security question answers
have been changed so that it's clear that all are required
* Ensure success message gets shown upon saving
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Require all three answers to be correct (previously only two had to
be)
* Give the user three attempts instead of just two
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Use venv module instead of deprecated virtualenv command
* Upgrade pip at the start so we don't get warnings about it
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We should be being explicit about the MIME type rather than allowing the
browser to guess.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Try to cleanup the typical uncommitted files list. The added files:
* The .json files are generated when running in development mode
* requirements* are generated by check_requirements.sh
* The others relate to dockersetup.sh
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Some SMTP servers are configured to require encrypted connections and
username & password, so provide options to enable those.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Just pass back the args object instead of individual arguments to avoid
having to change two places for every trivial argument we add.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If no email host has been specified that will prevent certain things
from working, so warn the user if they haven't specified one.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Use the port number in the domain so that emails have working URLs
* Don't include the port in the URL printed at the end of installation
if it's the default
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We use django-axes to lock out IP addresses after a set number of
attempts at logging in, and separately we use django-reversion to
record change history. As part of the history tracking, the default
behaviour of django-reversion is to wrap all POST requests in
"with transaction.atomic()", with the result that if an exception is
raised any changes get rolled back; unfortunately when authentication
fails for the final time, axes updates the database and then raises
PermissionDenied - with the result that the database changes are rolled
back, and the user's IP is not locked out, in fact it can never be
locked out. To work around this, disable the atomic mode on
ReversionMiddleware using a subclass. (I don't like having to do this,
but this is the quickest solution for now.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
For running this script in a cron job we want to see just the
interesting output since that will be sent as an email, so make the
following changes:
* Silence the normal output of virtualenv and pip
* Add a -q option for the script to silence progress messages
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Update to latest versions; as per last update we skip django-reversion
and django-axes. (Now however I've realised that django-axes > 4.5.4
probably isn't found within the container because in my development
setup, pip == pip3 whereas in the container pip is the Python 2 version.
For now let's stick with something that we can still use with the layer
index that still requires Python 2).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If you supply a database file then you may already have ad admin user
defined and thus adding another one is superfluous. Add an option to
make it easy to disable that step.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If a database file is specified we probably want to stick with whatever
site name is defined in there and not overwrite it.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Simplify things a bit. We lose the digg-style pagination but the new
behaviour is good enough and improves maintainability.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Allow specifying the project name (either with a --project-name
command-line option or COMPOSE_PROJECT_NAME) and save it so that
future upgrade operations work properly by default.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We need the real project name here otherwise we are mounting something
nonexistent instead of the real volume we want to populate. This fixes
a bug where the static assets were not available (thus the web interface
looks awful) if the directory you installed the application from was not
called "layerindex-web".
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Good for consistency, but also useful once we remove the hardcoded
container names from docker-compose.yml.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
In testing, lockouts are happening far too frequently. Thus make the
following changes to Axes configuration:
* Lock out only after 4 incorrect attempts, not 3
* Set a 1-hour cooling-off period, after which the lockout will be
removed
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Most Linux distributions do not have "recipes", they have "packages" so
use the correct term (as we are in other places).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
There's no reason for this directory to be present in the source tree.
(The rest of the docker files are already excluded via .dockerignore,
but files in docker/ need to be picked up by COPY directives so we can't
do that for these as well.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
When running a task (e.g. importing other distribution data), the web
frontend polls to get task output every second. In the view handling
this request we check to see if the underlying Celery task is finished
by getting the AsyncResult and then calling result.ready().
Unfortunately that latter call seems to be failing some of the time in
the development Docker setup that I am using, throwing a
ConnectionResetError. Because the polling is regular it doesn't really
matter if this fails as there'll be another chance on the next poll, so
just allow the call to fail silently. (It pains me to put in workarounds
like this but at the moment I can't determine the real cause of the
issue, and this stops the stream of ultimately useless error report
emails that I'm getting as an admin of the development instance).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Calling communicate() blocks the process; but since we're writing the
output directly to a file and not sending any input we don't actually
need to call communicate(), just poll() (so that we can check
the returncode attribute). Subjectively this does appear to improve
performance although it has not fixed the ConnectionResetError issues.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
