Within the layersapp container, if you try to run layerindex/update.py:
File "/opt/layerindex/settings.py", line 280, in <module>
RABBIT_BROKER = 'amqp://' + os.getenv('RABBITMQ_DEFAULT_USER') + ':' + os.getenv('RABBITMQ_DEFAULT_PASS') + '@layersrabbit:5672/'
TypeError: can only concatenate str (not "NoneType") to str
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
In addition to ALLOWED_HOSTS, we now must have CSRF_TRUSTED_ORIGINS defined.
This variable requires the scheme (http:// or https://).
Like ALLOWED_HOSTS, CSRF_TRUSTED_ORIGINS is a list of strings, with one
entry for each host which is trusted for POST requests.
https://docs.djangoproject.com/en/4.2/ref/settings/#csrf-trusted-origins
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
New in Django 3.2
To quiet warnings, set DEFAULT_AUTO_FIELD to the
default value 'django.db.models.AutoField'
NOTE: The default value for newly created Django 3.2
projects is django.db.models.BigAutoField, but
this causes the need for a migration in 'captcha'.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Added SPDX identifiers to all .py files except those in migrations directory.
Fixes: [YOCTO #13527]
Signed-off-by: Meh Mbeh Ida Delphine <idadelm@gmail.com>
Signed-off-by: Paul Eggleton <bluelightning@bluelightning.org>
The docker/settings.py file also needs the change from
"Changing MIDDLEWARE_CLASSES to MIDDLEWARE."
Signed-off-by: Tim Orling <ticotimo@gmail.com>
Signed-off-by: Paul Eggleton <bluelightning@bluelightning.org>
Add a new BITBAKE_PATH to the settings file to specify the path within the
BITBAKE_REPO_URL where bitbake lives. This is useful when using a combined
repository, such as poky, that contains bitbake, openembedded-core and other
layers.
This change also changes the default path, in the fetch directory, for the
bitbake checkout. It no longer uses the path 'bitbake', but instead uses the
same URL processing as the layer fetching.
There is a side effect that, when using a shared fetch, the branch of the
layer will be used instead of the specified bitbake branch. Generally this
is a reasonable compromise, since in a combined repository bitbake and
openembedded-core component should already match.
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* We need a SOCKS proxy to support fetching from git:// or ssh:// URLs
within the container, so add an option to specify it
* It's possible for the http and https proxy settings to be the same, so
set one from the other if only one of them is set.
* If we want to be able to fetch from internal servers inside the proxy
then we also need a "no-proxy" list, so add support for that.
* It's not unlikely that machines within networks requiring use of a
proxy for external network access will have all of the proxy settings
set in the environment, so we can try to pick up the defaults from
there.
* Ensure that we can switch from proxy to no proxy (when reinstalling)
which means we always need to edit the config files and ensure the
proxy options get commented out if we don't want them set.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's easy to get the proxy settings wrong and not realise until you've
got quite a long way into the process of setting things up. Thus, add a
check where we actually try to fetch various things within the container
environment and fail reasonably early if things aren't working.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add a SECURITY_QUESTIONS_REQUIRED setting that defaults to True, but if
set to False then a user who has not set security questions will still
be allowed to reset their password. This is convenient for the OE Layer
index because there are a number of existing accounts, none of which
will have security questions set.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We should be being explicit about the MIME type rather than allowing the
browser to guess.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Some SMTP servers are configured to require encrypted connections and
username & password, so provide options to enable those.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We use django-axes to lock out IP addresses after a set number of
attempts at logging in, and separately we use django-reversion to
record change history. As part of the history tracking, the default
behaviour of django-reversion is to wrap all POST requests in
"with transaction.atomic()", with the result that if an exception is
raised any changes get rolled back; unfortunately when authentication
fails for the final time, axes updates the database and then raises
PermissionDenied - with the result that the database changes are rolled
back, and the user's IP is not locked out, in fact it can never be
locked out. To work around this, disable the atomic mode on
ReversionMiddleware using a subclass. (I don't like having to do this,
but this is the quickest solution for now.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Simplify things a bit. We lose the digg-style pagination but the new
behaviour is good enough and improves maintainability.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
In testing, lockouts are happening far too frequently. Thus make the
following changes to Axes configuration:
* Lock out only after 4 incorrect attempts, not 3
* Set a 1-hour cooling-off period, after which the lockout will be
removed
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Limit to TLS 1.2 and the most secure ciphers
* Set some other recommended ssl_* settings
* Disable gzip
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We need the task log dir to be shared between the app and celery
containers, so mount a volume and point both of them into it.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's likely an image manifest tarball will be more than 1mb in size;
increase it to 16mb to handle where a large number of patches are
included in an image containing a lot of packages from different
recipes.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add user security questions upon registration as extra authentication
for password reset. Three unique security questions must be chosen and
answered. Answers are then stored in the database with the same hashing
algorithm as the users's password.
On password reset, users get two chances to get two out of three
security questions answered correctly. After a second failure their
account is locked and email is sent to the admin. The same template is
shown for the axes lockout. Super user cannot reset their password until
they set security questions.
Users can update their security questions or add them if they weren't
originally set (in the case of super user) in Edit Profile.
Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
This involves changing how registration templates are referenced
and how the activation email is sent on user's email address change.
Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
Make the app should use a database account that has only the privileges
it needs rather than the root account for improved security.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add settings for user/password for the RabbitMQ server and make
dockersetup.py set it up. (The rabbitmq container intrinsically
understands RABBITMQ_DEFAULT_*, and for the sake of consistency I've
reused those variables for the other containers.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The application is most likely to be used in an internal setting,
however should someone want to make it accessible externally, add
support for Let's Encrypt certificates for serving via HTTPS.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If we want a minimum level of security we should enable HTTPS. However,
the only practical way we can do that without the user having to do further
infrastructure setup and/or pay a certification authority is to use a
self-signed certificate. Do this by default, and also provide an option
to specify a previously obtained certificate/key pair.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Adding setup script for docker containers. The script will edit all
necessary configuration files, build and launch all containers, and
do the initial database setup - including populating the database
with data supplied by the user.
Changed docker/README to reflect new setup instructions.
Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This makes deployment (and updates) much easier. You still need to
follow some setup instructions in the docker/README file, but the number
of steps is reduced and most commands simplified.
Thanks to Konrad Scherer <Konrad.Scherer@windriver.com> for suggesting
this (although the docker-compose.yml here was written from scratch).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Use Django's built-in password validators with reasonable settings, and
add a basic complexity validator since there isn't one provided.
Additionally, fix the registration form so that it shows the help text
which includes a description of what the password requirements are.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Repeated failed login attempts should trigger lockout to prevent
brute-forcing and user enumeration - django-axes does this using account
and IP-based lockout.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
To enable audio support for django-simple-captcha for accessibility
purposes, you need to provide the full paths to the flite and sox
binaries, so add these settings (commented out by default to avoid extra
dependencies for those that don't need it).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
nvd3 and its python/django wrappers appear to be no longer actively
maintained, and at least the wrappers were a bit clunky to use. Looking
around for a suitable replacement, Chart.js seems capable, has no
additional dependencies and is fairly simple to use. As a bonus we get
to drop a few Python dependencies from our list.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Use a more modern version of Bootstrap and take the opportunity to
upgrade jQuery to the latest version at the same time. This provides
better browser compatibility, moves to MIT license, allows us to make
the site more responsive for different devices in future, and provides
theming capabilities for custom installs among other improvements.
(I chose to upgrade to v3 for now rather than straight to v4 as it was
easier to do this gradually.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We weren't giving the layersapp container access to the layer
repositories, which meant that the "Bulk change" function (which lets
you generate patches on top of recipes to change certain variable
values) could not work. Enable the volume and rearrange the order so
that it does, and name the volume more appropriately.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Several distro-provided versions of Docker I have used are too old to
support the --mount option, so rather than making users find and replace
it in the instructions, just revert to the old-style -v option.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The Docker-based setup method is preferred for production, so rearrange
things a little to make it a bit easier to follow.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Put NGINX, Celery, and RabbitMQ into their own separate containers
* Use a docker network instead of the deprecated --link
* Allow for collecting the static files properly
* Create a copy of settings.py specifically for the docker setup. This
will need to be kept in sync with the main example settings.py, but
it avoids the user having to edit it too much.
* Add optional SSL configuration using letsencrypt certificate
* Create some volumes for static files / fetched repos
* Add some more helpful setup instructions
Largely based upon work by Michael Halstead <michael@yoctoproject.org>.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If we're starting a new database, or one of the other applications
(i.e. dependencies of the main layerindex application) has been
upgraded, we need to be migrate all of the applications rather than just
layerindex, so have migrate.sh do that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Python 3 is a lot more sensitive to locale, plus we will definitely be
dealing with non-ASCII names and email addresses, so we need to get this
right.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Allow updating multiple branches, and if no branches are specified,
update all branches that have a new "updates_enabled" flag field set to
True. This avoids the need to have a separate shell script which runs
update.py for each branch (and thus has hardcoded knowledge of each
active branch in the index, i.e. it needs to be kept up-to-date in
addition to the database.)
The migration will default updates_enabled to True for all branches so
if you wish to take advantage of this functionality, the flag will need
to be set to False for any branches that shouldn't be updated.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Replicate production setup in Docker containers
[YOCTO #7575]
Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>