Use Django's built-in password validators with reasonable settings, and
add a basic complexity validator since there isn't one provided.
Additionally, fix the registration form so that it shows the help text
which includes a description of what the password requirements are.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add a script that creates a virtualenv and does the following:
1) Tests pip install -r requirements.txt
2) Runs pip freeze and diffs the output to requirements.txt (to
check if any dependencies have been missed when it was last updated)
3) Runs pip list --outdated and writes the output to a file so you can
see which packages might need updating (of course some may not be
able to be updated easily e.g. if an API change has been made)
4) Installs safety and runs "safety check" to check if there are any
known vulnerabilities
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Upgrade GitPython and trim out old dependencies: gitdb and smmap are no
longer needed by current GitPython, gitdb2 and smmap2 have replaced
them.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Repeated failed login attempts should trigger lockout to prevent
brute-forcing and user enumeration - django-axes does this using account
and IP-based lockout.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
On top of the CAPTCHA, prevent more than 10 name change attempts for the
same user with the Edit Profile page so that username enumeration is
made even more difficult.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Users may want to change their usernames for a number of different
reasons, but at the moment we require them to contact an admin to do
that. Provided we validate the new username correctly and add a CAPTCHA
to make automated enumeration difficult, we can add username to the Edit
Profile form and then users can do that any time they wish.
While we're doing this, show a message when the profile is successfully
updated.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Make it possible for users to delete their own account and all associated
information from the database, should they decide they no longer wish to
use it.
(I checked the implications of doing this on our model structure -
anything with a foreign key to user is safe to delete with the exception
of RRS MaintenancePlan.admin which I needed to change on_delete for so
that it doesn't get deleted with the user).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Extend and override the default views so we can extend and override the
default forms to add a CAPTCHA field. This should prevent the automated
account creation requests we've been seeing on layers.openembedded.org
(luckily failing anyway due to bad domain names), but in any case this
also improves security by making it harder to do user enumeration.
For the registration page in particular, because Django's forms logic
tries to be helpful by showing all errors at once, we need to change it
so that if there's an error for the CAPTCHA then you only see that error
and no other - in particular you won't see "that username already
exists" if that is the case.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This was copy-pasted from the DistroSearchView and I clearly forgot to
replace the reference here.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
To enable audio support for django-simple-captcha for accessibility
purposes, you need to provide the full paths to the flite and sox
binaries, so add these settings (commented out by default to avoid extra
dependencies for those that don't need it).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's been reported to me that it's possible in a custom setup to have a
bitbake repo that intentionally doesn't have a master branch, so pick
up the bitbake branch from the master Branch record if it exists and the
bitbake_branch field is set.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
nvd3 and its python/django wrappers appear to be no longer actively
maintained, and at least the wrappers were a bit clunky to use. Looking
around for a suitable replacement, Chart.js seems capable, has no
additional dependencies and is fairly simple to use. As a bonus we get
to drop a few Python dependencies from our list.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's not too common but there are instances where people have copied
.inc files into their own layer and modified them, and if you are using
such a layer that could result in unexpected behaviour. In order to get
a handle on when this is being done, collect data about all .inc files
and show duplicates in the Duplicates screen.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Split out the code used in the recipe search views to its own function
and use that same function in three different places rather than having
a copy of largely the same code. Also take the opportunity to add some
comments.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
On layers.openembedded.org we're seeing requests from some search engine
crawlers requesting the CSV export URL with an invalid branch for the
layer. I couldn't see the referer anywhere in the logs but I suspect it
has to do with some recent cleanup work I did in the database where I
deleted some invalid LayerBranch records - they were probably following
links in a cached version of the webpage. In any event we want to return
404 in this situation rather than an internal server error.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
In order to show bbappends on the recipe detail page we are doing a
regex query to find any whose names match up with the recipe. In the
layer index instance at layers.openembedded.org viewing the recipe
detail page for any recipe whose name contains ++ (e.g. libsigc++-2.0 in
meta-oe) results in an invalid regex and causes a database error. Escape
any + signs in the name used within the regex in order to fix this.
(I wasn't actually able to reproduce this on my own setup despite also
using MariaDB, but I did find that the unescaped query was not correctly
matching records so it needed to be fixed anyway.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
When you make changes to the infrastructure it can be useful to test
that email sending is working, since for that to work that involves the
code, Celery, RabbitMQ and SMTP being functional. However, up until now
to run a test you needed to submit a fake layer which is a bit annoying.
Add an explicit "Test email" option to the Tools drop-down for staff
users to allow them to send an email to themselves.
Note: the page will come back when the Celery job has been created, it
does not check and report on the job status - you need to look on the
server side to see that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Sometimes we get massively long lines from the update script
(particularly if there's an error) so ensure that long lines get
wrapped.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If we want to be able to read in patch information on python2-based
branches (e.g. fido) then we need to use codecs.open() instead of open()
here since python2's open() did not support the encoding parameter.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If you update a python2-based branch then the python2-compatible version
of bitbake will be checked out, but we are calling into bitbake's
bb.utils directly here from python 3 and thus you get an error about
commands.getstatusoutput being missing (since that is not available in
python 3 and the old version of bitbake refers to it). To fix this,
check out origin/master in the bitbake repo right before we call the
code in question.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
When we print a warning about the value of a CharField being truncated,
print out the string representation of the object so we have a chance of
finding the offending object.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Enable searching on vcs_url on LayerItem and layer name and vcs_url on
LayerBranch. This makes it easier to find the layers/branches in a
particular repository (e.g. meta-openembedded).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
I've come across at least one layer that is now hosted on gitlab.com, so
add support in the layer submission/edit form and import_layer.py for
automatically determining the other fields for gitlab.com URLs.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We don't want to allow any other arguments to be injected into these
commands, so disable the shell and pass the parameters in the form of a
list to prevent that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Sometimes layers get created on master and then the master branch is
removed in favour of a release branch. In that case it can be useful to
switch the existing layerbranch record rather than having to create a
blank one and copy everything over.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The code in recipeparse.setup_layer() was trying to log a warning in the
case where LAYERRECOMMENDS not being satisfied, however there is no
actual logger object in this context. Pass it in via a parameter and
update all callers to pass it.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
In c26604146a I made a fix to change where
the bitbake code writes out bitbake.lock and other files it creates
during parsing, but didn't adequately test it and it turns out our
call to delete the temp directory races against bitbake deleting
bitbake.lock and bitbake.sock. For now the simplest way to deal with
this is to ignore the errors since we don't care about these files,
we just want the temp dir gone.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Our setup when parsing recipes is a little unusual in that we have no
bblayers.conf, thus findTopdir() which is used to find where to put
bitbake.lock (and bitbake-cookerdaemon.log as of the recent bitbake
commit 1620dbc48ffb2a882371cf9174a7b12648befc8a) defaults to the
parent's parent of where bitbake.conf can be found, which is the meta/
subdirectory of the OE-Core repo, thus that's where we now find
bitbake-cookerdaemon.log gets written out. We really don't want to be
writing anything into the metadata repositories so create a fake
conf/bblayers.conf in our temp directory to make findTipdir() pick that
instead.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
On the machines/distros/classes pages, if you type a keyword and then
click "search", and there are no matches, then you click on "browse",
there shouldn't be any search text in the box anymore because you're
viewing all items, so use a bit of javascript to ensure that.
While I'm at it, set reasonable ids for the search field on each page
(including the recipes page, although there is no browse button there so
that is just for consistency).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If there are no search results, focus the search input field and select
any text in it so that the user can just start typing a keyword
immediately.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If you clicked on "Select" then cancelled the modal that appears, then
clicked on "No match" then the title of the modal in the second instance
retains the recipe name from the first time which is wrong. Rearrange
the logic so that this cannot happen (and make it tidier at the same
time).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This isn't a visual thing, this select element must remain hidden, so it
seems a bit more appropriate to me to specify the style directly on the
element rather than using a CSS class to do it.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
For situations where the user launches a distro comparison update
process and then shortly afterwards realises it is operating with the
wrong configuration (or is otherwise broken) and is going to take a long
time to finish, add a button to the task page to stop the task. This was
tricky to get working, since the default behaviour of Celery's revoke()
would either terminate both the Celery task process along with the update
process (leaving us with no log saved to the database) or worse not even
kill the update process, depending on the signal sent. To avoid this,
send SIGUSR2, trap it in the task process and kill the child process,
returning gracefully. To make that possible I had to rewrite runcmd() to
use subprocess.Popen() instead of subprocess.check_call() as otherwise
we can't get the child's PID.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If a distro comparison update task fails (returning a non-zero value to
indicate as such) we were not able to see this easily from the frontend.
Show success/failure in the form of a label on the task page and general
update list/detail, and if the task fails while we're watching then make
the progress bar go red as well. Also make a distinction between the
process failing (retcode > 0) and being terminated (retcode < 0, e.g.
process was killed).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Provide a mechanism for distro comparison update tasks to display
progress. In practice this means the update command needs to write the
progress percentage to a file and then the log view (which is polled by
the frontend) reads this file. Originally I was going to use a FIFO for
this but that turned out to be a but unreliable; I also tried to use
Celery's state mechanism to pass it back but I simply could not get it
to work. The file-based mechanism is good enough though.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We were refreshing the page constantly in order to show output while
a task was running, which basically worked but is horrible. Instead,
write the task output to a file and then use AJAX calls to request
whatever output has been written to the file since the last call
and call this roughly every second. Put the output in a scrollable <pre>
element instead of making it the length of the page, and auto-scroll
to the end (unless the user grabs the scrollbar and pulls it upwards -
it may not be immediately obvious that you can do this if there is a lot
of output since you have to pull it up when the scrolling animation is
not running, but it is possible).
An alternative would be to have used some kind of long-lived HTTP
session or a websocket, but those come with their own set of problems
so I elected to use this much simpler method.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If the layer has an actual_branch set, then show it underneath the URL
on the layer detail page so that the reader knows which branch they need
to check out to see what the index shows.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
In a bunch of places we needed to get the branch we were supposed to
be checking out (which is actual_branch if that is set, otherwise the
normal branch name). Add a function to do that.
Additionally, instead of showing the normal branch name next to the
"last update" date, use the result of this new function.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
At the moment the only way to get to the index is to manually type in
the URL, which is a little inconvenent. Add a link to the Tools
drop-down (visible only for users with admin access) that will take you
to it for convenience.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Use a more modern version of Bootstrap and take the opportunity to
upgrade jQuery to the latest version at the same time. This provides
better browser compatibility, moves to MIT license, allows us to make
the site more responsive for different devices in future, and provides
theming capabilities for custom installs among other improvements.
(I chose to upgrade to v3 for now rather than straight to v4 as it was
easier to do this gradually.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We were missing some import statements here, so clearly I didn't test
this as I thought I had.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Now the logic is:
Use options.layer_type if specified, and guess if not. Default to 'M'.
Note choices=['A', 'B', 'S', 'D', 'M', ''], the '' is for default='', we can't
use default='M' here, otherwise we don't know whether the 'M' is specified by
user or is the default value, we don't guess if it is specified by user,
otherwise, guess.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If you're creating a new release then you will need to create milestones
within it, so add in-line forms for these. Unfortunately there's no
capability yet to automatically split up the release into n milestones
(or perhaps copy milestones from a previous release), that will have to
wait until later - for now this makes things a little easier.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
In the current RRS we should expect that more than one recipe with the
same PN can exist in a layer - in fact it is common to have this (i.e.
multiple versions of the same recipe). Use the filename to match the
recipe record instead. At the same time, fix the exception handling.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
