* We need a SOCKS proxy to support fetching from git:// or ssh:// URLs
within the container, so add an option to specify it
* It's possible for the http and https proxy settings to be the same, so
set one from the other if only one of them is set.
* If we want to be able to fetch from internal servers inside the proxy
then we also need a "no-proxy" list, so add support for that.
* It's not unlikely that machines within networks requiring use of a
proxy for external network access will have all of the proxy settings
set in the environment, so we can try to pick up the defaults from
there.
* Ensure that we can switch from proxy to no proxy (when reinstalling)
which means we always need to edit the config files and ensure the
proxy options get commented out if we don't want them set.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's easy to get the proxy settings wrong and not realise until you've
got quite a long way into the process of setting things up. Thus, add a
check where we actually try to fetch various things within the container
environment and fail reasonably early if things aren't working.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
I forgot the lesson of f5922091b4 which
was that djangorestframework has a very silly default of registering
the model name as the key for finding the ViewSet, with the result that
the recently created RecipesExtendedViewSet was also being used for the
recipes URL which we expect to point to RecipeViewSet; this broke the
bitbake-layers layerindex-* commands.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The new viewset has a different URL and uses pagination, so we need to
take that into account.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Adding these extra child items to the standard "recipes" viewset (which
we did recently in 684a06a383) means that
some current usage is impractical due to the size of the returned list
of items. Instead, create a recipesExtended viewset, move the new child
items to that and add pagination to avoid result size issues.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This is now be practical to use for updating as well as the initial
import. Additionally the user doesn't need to be using the debug
option by default so drop it from the example command.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If a LayerBranch is on a branch that is in the remote layer index (and
that branch is in any branch list specified with -b/--branch) and the
layer for the LayerBranch is not found in the remote layerindex then it
should be deleted, otherwise we could end up with stale data.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add an option to import only certain layers - mostly for debugging
(can make testing a lot quicker).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We only work with branches that exist in both the local database and the
remote index, but allow the user to restrict the list further if
desired.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This is not a comprehensive test, but does provide a way to easily
compare that what was originally fetched for a recipe from the API is
the same once it's been imported into a new instance using
import_layers.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Import recipe data from the REST API, including sub-recipe objects
(sources, patches, PACKAGE_CONFIGs, dependencies etc.) now that they are
available from the API. Also, with the data being more or less complete,
it's also reasonable to also set the vcs_* fields so that we can tell
how up-to-date it is.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Don't assume we need to create these objects - check the updated field
value against what the API reports and update the existing object if the
latter is newer.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
For LayerDependency, LayerMaintainer and LayerNote objects, properly
sync the objects in the database with the objects fetched from the API.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
These are a little less useful than the other items but if we want to be
able to create a clone of a layer index then we need them.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We're tracking these items for each recipe, so they should be
accessible via the API as well.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The try-specfile subcommand is for testing, so it's entirely possible
the spec file we are testing with will be in the current directory;
os.path.dirname() returns '' in this case, which os.path.relpath() then
chokes on. Avoid this in the easiest way possible by just prepending ./
to the path in this case.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's not common, but if someone saves a spec file with a non-UTF8
encoding then the surrogate characters were preventing the data from
being saved to the database (at least with MariaDB). Instead of assuming
UTF8, try a few common encodings on the first 100 lines.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add a SECURITY_QUESTIONS_REQUIRED setting that defaults to True, but if
set to False then a user who has not set security questions will still
be allowed to reset their password. This is convenient for the OE Layer
index because there are a number of existing accounts, none of which
will have security questions set.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If you clicked on logout in the admin site then clicked on the "Login"
button and logged in, you were redirected to the logout page because we
weren't detecting that the redirection should not be applied.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This is the only dependency we can update right now; the rest will cause
problems given that the layer index needs python 2 support still.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Tweak message when one or more (but not all) security question answers
have been changed so that it's clear that all are required
* Ensure success message gets shown upon saving
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Require all three answers to be correct (previously only two had to
be)
* Give the user three attempts instead of just two
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Use venv module instead of deprecated virtualenv command
* Upgrade pip at the start so we don't get warnings about it
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We should be being explicit about the MIME type rather than allowing the
browser to guess.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Try to cleanup the typical uncommitted files list. The added files:
* The .json files are generated when running in development mode
* requirements* are generated by check_requirements.sh
* The others relate to dockersetup.sh
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Some SMTP servers are configured to require encrypted connections and
username & password, so provide options to enable those.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Just pass back the args object instead of individual arguments to avoid
having to change two places for every trivial argument we add.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If no email host has been specified that will prevent certain things
from working, so warn the user if they haven't specified one.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Use the port number in the domain so that emails have working URLs
* Don't include the port in the URL printed at the end of installation
if it's the default
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We use django-axes to lock out IP addresses after a set number of
attempts at logging in, and separately we use django-reversion to
record change history. As part of the history tracking, the default
behaviour of django-reversion is to wrap all POST requests in
"with transaction.atomic()", with the result that if an exception is
raised any changes get rolled back; unfortunately when authentication
fails for the final time, axes updates the database and then raises
PermissionDenied - with the result that the database changes are rolled
back, and the user's IP is not locked out, in fact it can never be
locked out. To work around this, disable the atomic mode on
ReversionMiddleware using a subclass. (I don't like having to do this,
but this is the quickest solution for now.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
For running this script in a cron job we want to see just the
interesting output since that will be sent as an email, so make the
following changes:
* Silence the normal output of virtualenv and pip
* Add a -q option for the script to silence progress messages
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Update to latest versions; as per last update we skip django-reversion
and django-axes. (Now however I've realised that django-axes > 4.5.4
probably isn't found within the container because in my development
setup, pip == pip3 whereas in the container pip is the Python 2 version.
For now let's stick with something that we can still use with the layer
index that still requires Python 2).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If you supply a database file then you may already have ad admin user
defined and thus adding another one is superfluous. Add an option to
make it easy to disable that step.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If a database file is specified we probably want to stick with whatever
site name is defined in there and not overwrite it.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
