Added SPDX identifiers to all .py files except those in migrations directory.
Fixes: [YOCTO #13527]
Signed-off-by: Meh Mbeh Ida Delphine <idadelm@gmail.com>
Signed-off-by: Paul Eggleton <bluelightning@bluelightning.org>
Add a new BITBAKE_PATH to the settings file to specify the path within the
BITBAKE_REPO_URL where bitbake lives. This is useful when using a combined
repository, such as poky, that contains bitbake, openembedded-core and other
layers.
This change also changes the default path, in the fetch directory, for the
bitbake checkout. It no longer uses the path 'bitbake', but instead uses the
same URL processing as the layer fetching.
There is a side effect that, when using a shared fetch, the branch of the
layer will be used instead of the specified bitbake branch. Generally this
is a reasonable compromise, since in a combined repository bitbake and
openembedded-core component should already match.
Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add a SECURITY_QUESTIONS_REQUIRED setting that defaults to True, but if
set to False then a user who has not set security questions will still
be allowed to reset their password. This is convenient for the OE Layer
index because there are a number of existing accounts, none of which
will have security questions set.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We use django-axes to lock out IP addresses after a set number of
attempts at logging in, and separately we use django-reversion to
record change history. As part of the history tracking, the default
behaviour of django-reversion is to wrap all POST requests in
"with transaction.atomic()", with the result that if an exception is
raised any changes get rolled back; unfortunately when authentication
fails for the final time, axes updates the database and then raises
PermissionDenied - with the result that the database changes are rolled
back, and the user's IP is not locked out, in fact it can never be
locked out. To work around this, disable the atomic mode on
ReversionMiddleware using a subclass. (I don't like having to do this,
but this is the quickest solution for now.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Simplify things a bit. We lose the digg-style pagination but the new
behaviour is good enough and improves maintainability.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
In testing, lockouts are happening far too frequently. Thus make the
following changes to Axes configuration:
* Lock out only after 4 incorrect attempts, not 3
* Set a 1-hour cooling-off period, after which the lockout will be
removed
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add user security questions upon registration as extra authentication
for password reset. Three unique security questions must be chosen and
answered. Answers are then stored in the database with the same hashing
algorithm as the users's password.
On password reset, users get two chances to get two out of three
security questions answered correctly. After a second failure their
account is locked and email is sent to the admin. The same template is
shown for the axes lockout. Super user cannot reset their password until
they set security questions.
Users can update their security questions or add them if they weren't
originally set (in the case of super user) in Edit Profile.
Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
This involves changing how registration templates are referenced
and how the activation email is sent on user's email address change.
Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
Use Django's built-in password validators with reasonable settings, and
add a basic complexity validator since there isn't one provided.
Additionally, fix the registration form so that it shows the help text
which includes a description of what the password requirements are.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Repeated failed login attempts should trigger lockout to prevent
brute-forcing and user enumeration - django-axes does this using account
and IP-based lockout.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
To enable audio support for django-simple-captcha for accessibility
purposes, you need to provide the full paths to the flite and sox
binaries, so add these settings (commented out by default to avoid extra
dependencies for those that don't need it).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
nvd3 and its python/django wrappers appear to be no longer actively
maintained, and at least the wrappers were a bit clunky to use. Looking
around for a suitable replacement, Chart.js seems capable, has no
additional dependencies and is fairly simple to use. As a bonus we get
to drop a few Python dependencies from our list.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We were refreshing the page constantly in order to show output while
a task was running, which basically worked but is horrible. Instead,
write the task output to a file and then use AJAX calls to request
whatever output has been written to the file since the last call
and call this roughly every second. Put the output in a scrollable <pre>
element instead of making it the length of the page, and auto-scroll
to the end (unless the user grabs the scrollbar and pulls it upwards -
it may not be immediately obvious that you can do this if there is a lot
of output since you have to pull it up when the scrolling animation is
not running, but it is possible).
An alternative would be to have used some kind of long-lived HTTP
session or a websocket, but those come with their own set of problems
so I elected to use this much simpler method.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Use a more modern version of Bootstrap and take the opportunity to
upgrade jQuery to the latest version at the same time. This provides
better browser compatibility, moves to MIT license, allows us to make
the site more responsive for different devices in future, and provides
theming capabilities for custom installs among other improvements.
(I chose to upgrade to v3 for now rather than straight to v4 as it was
easier to do this gradually.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If you're running a testing / internal instance then you really don't
want to be emailing maintainers on publish, so provide a setting you can
use to disable that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Use maintenance plans to get layerbranches
* Use from/to/subject and admin contact from maintenance plan
* Use an actual template to render the email (and drop tabulate
dependency)
* Improve grammar in the email text
* Use a single line to represent the most recent commit
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add RRS-specific settings and bring in RRS. Based on work by
Aníbal Limón <anibal.limon@linux.intel.com>.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Django 1.8 introduced this as a soft requirement, it is a hard
requirement for later Django releases.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This can save a lot of time, here is my testing data when PARALLEL_JOBS is 10,
this is the fetch time only, I hacked it to stop when the fetch is done to get
the data (124 layers):
$ update.py -b <branch>
Before: 2m30
Now: 16s
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This patch adds asynchronous task execution using a Celery backend
and RabbitMQ task queue, so that the layer submission process to
proceed even in the event that sending the notification email fails,
and establishing an asynchronous execution mechanism that we can use
in the future e.g. for triggering parse operations from the web UI.
This pertains to bug 11197:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=11197
It updates the README to reflect the installation and configuration
of a basic RabbitMQ setup, adds a 'tasks.py' file to contain task
definitions, updates the 'edit_layer_view' function to send
emails to administrators about new and updated layers asynchronously,
modifies the 'settings.py' to include a default configuration
for a RabbitMQ connection, and updates the Dockerfile to start a Celery
worker alongside the Gunicorn daemon.
Fixes [YOCTO #11197].
Signed-off-by: Diana Thayer <garbados@gmail.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
At the moment the URL that is presented in the review email will have
http:// or https:// prefix depending on what the user who submitted the
layer was using, but that's irrelevant - we actually want https:// if
the server is capable of it since the reviewer may be redirected to log
in (and Django's login_required decorator will always redirect to the
login page if http is being used as far as I observed, which is a bit
annoying if you are already logged in.) Add a setting which if enabled
will substitute https:// as the prefix.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Fixed:
1) set LAYERDEPENDS_openembedded-layer = "core"
2) $ "update.py -l meta-oe -b master"
Check from web, its dependency is "openembedded-core"
3) Change LAYERDEPENDS_openembedded-layer = "foo"
4) Run "update.py -l meta-oe -b master"
5) Check from web, its dependency is "openembedded-core and foo", this might
be incorrect, now if set REMOVE_LAYER_DEPENDENCIES to true, the old
dependency openembedded-core will be removed, the default is False which
prints warnings to notify users.
And also the existing checking should filter(required=required), otherwise it
can't work well when a layer is in both depends and recommends, this can't
happen in a normal case, but it would surprise the user when this happens.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
At the moment it's a bit difficult to get update logs out of the
environment in which the update script is being run. In order to make
the logs more accessible, create a LayerUpdate model to record the
output of update_layer.py separately for each layerbranch and tie the
created LayerUpdates together with a single Update model per session.
We provide two ways to look at this - a Tools->Updates page for
logged-in users, and there's also an "Updates" tab on each layer that is
accessible to anyone; which one is useful depends on whether you are
looking at the index as a whole or an individual layer.
Update records older than 30 days are deleted automatically by default.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
I'd like to be upgrading to 1.8 but that causes problems with South, and
we're not quite ready to dispense with our existing migrations yet.
Part of the implementation for [YOCTO #9620].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We add the corsheaders application in order to
enable the CORS policy for the read-only REST API.
This is needed to allow AJAX queries to the layerindex
application from a browser environment.
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
This patch enables a read-only REST API for the layerindex
application using Django REST Framework.
The objects of types Branch, LayerBranch and LayerItem are
exposed to queries so that the layerindex application can
function as a Layer Source in Toaster.
The library dependencies are documented in the requirements.txt
file.
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Add a script for doing a one-time import of OE-Classic recipe
information, so comparisons against OE-Core can be performed; this
is stored using a new ClassicRecipe model supporting additional fields
for tracking migration status. The migration status fields can be
updated as well as viewed and summarised in graph format.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Change the data structure to support multiple branches. At the top level
there is a set list of Branch objects, and then a LayerBranch object
between each layer and the maintainers, dependencies, recipes and
machines, so that the set of each can be different per branch. The
branch is a session option, and can be selected via a drop-down that is
shown for all pages.
Additionally, with this change we avoid the need to run the update
script within a build environment set up with oe-init-build-env - since
we need a specific version of BitBake per branch we now use our own copy
of BitBake which is fetched by the script itself. The update script will
need to be called multiple times however - once per branch.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This should prevent spamming even though this is less likely with this
kind of site.
The CAPTCHA does not show when editing, only submitting, and is also not
shown for authenticated users.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This should be set when the application is deployed, not set to some
default value out of the box.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Use a Django snippet to make pagination display a bit nicer (compressed
form instead of showing all page numbers). In order to interact properly
with GET requests, an additional snippet was required to add parameters
to the URL (why doesn't Django provide this out of the box?).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Collect history for changes, and allow browsing / reverting / comparison
within the admin interface.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
