Commit Graph

798 Commits

Author SHA1 Message Date
Paul Eggleton
dba1fbe5d1 RRS: add missing migration
This should have been added in 2c8f979f9c
to make it possible to delete an account that is the administrator for a
maintenance plan without removing the maintenance plan.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-05-28 09:57:58 +12:00
Paul Eggleton
8f37d77598 RRS: move Export button and restore Tools drop-down menu
Move the Export Recipe List button, change it to match the Export CSV
button on the layer detail page and put the Tools menu back (useful for
accessing the Admin site if not just for consistency). The Submit Layer
button remains hidden.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-21 15:47:56 +13:00
Paul Eggleton
03b39e89c0 RRS: fix showing all recipes for multi-layer maintenance plans
If you had more than one layer on a maintenance plan, only the recipes
from the last layer were shown because we were using a recipe list
variable from the last iteration. The fix just turned out to be to
indent the code that read from that list so it runs every iteration
instead of at the end.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-21 14:15:52 +13:00
Paul Eggleton
7ab7766ec7 RRS: Fix raw SQL statements to use parameters
For security reasons it's best practice to use parameters to pass values
into SQL statements and not substitute them in as strings (with Django's
database API the distinction is subtle, but we pass in the parameters in
a second list parameter instead of using % to substitute them before
passing the query in).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-21 14:15:42 +13:00
Paul Eggleton
a17652256f Add admin handling for Patch objects
Allow searching by path and filtering on layer / branch.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:19:10 +13:00
Paul Eggleton
975d2927d1 Record patch application order and use as default sorting
Patches often need to be applied in a specific order. For OE recipes we
were always storing the Patch objects correct order as they are
refreshed every time the recipe itself is refreshed, however for other
distro comparisons, import_otherdistro.py attempts to preserve existing
records, adds new ones and then deletes whatever is left over, which may
result in the order getting messed up over time. To avoid this issue,
record the order next to the patch and set the model meta-info to use
this to sort Patch queries by default.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:17:21 +13:00
Paul Eggleton
f9c5143700 Split out patch status read to a method on the Patch model
Make it easier to call this outside of the context of the update
process.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:14:50 +13:00
Paul Eggleton
ce5141fc39 RRS: add tool to import/export upstream history data
The recipe upstream history can't be reconstructed, so we need to be
able to export it out of an old database an import it into the new one.
There's also a "remove-duplicates" function; whilst not strictly
necessary it did help us to remove duplicate records resulting from
running the import twice (due to a bug) and may be useful in future
under similar circumstances.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:05:53 +13:00
Paul Eggleton
c74e4bd4d9 rrs_maintainer_history: check out layer branch before looking for maintainers.inc
The revision that happens to be checked out at the moment could be
anything and might not actually have a maintainers.inc, so check out the
proper branch before looking for it.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:05:53 +13:00
Paul Eggleton
0d19f49dd7 RRS: fix sorting arrow positioning
I'm not sure when or how they got broken but the sorting arrows were
overlapping the text, so move them back into the proper location, 8px
from the right of the column.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:05:53 +13:00
Paul Eggleton
53e6ed36b6 docker: add wget to dependencies
The upstream version checking used by the RRS needs wget to fetch
http/https URLs, so add it into the container.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:05:53 +13:00
Paul Eggleton
f6f747fb92 Replace use of assert with exceptions
asserts don't really belong in non-test code, let's handle these
situations properly instead.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:05:53 +13:00
Paul Eggleton
9c65bf254e Use try...finally or with to ensure files get closed
Best practices state that you should use a mechanism that ensures files
get closed in case of any error, so let's do that.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:05:53 +13:00
Paul Eggleton
24739d9369 local.conf: allow for upstream_tracking.inc to be missing
In OE-Core commit 22447eba250656489a6ed636fe58cb304d74e975,
conf/distro/include/upstream_tracking.inc has been removed, so allow the
file to be missing by changing require to include.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:05:53 +13:00
Paul Eggleton
2c3c287a33 Fix errors due to races deleting bitbake temp files
Errors deleting bitbake.sock and bitbake.lock have been observed when
shutting down tinfoil at the end of some of these scripts. Move the code
used in the main layer index update script to a function in utils.py and
use it everywhere in order to avoid the issue.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:05:53 +13:00
Paul Eggleton
0929289465 Add links to other branch recipes in recipe detail
Add links to the same recipe in other branches in the recipe detail page
(and RRS recipe detail page) so that you can see which versions are
available in other branches and drill down to the detail if you want to.

Implements [YOCTO #13019].

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-12-06 09:25:06 +13:00
Paul Eggleton
eeffb2d675 Send people an email when another user adds them as a maintainer
If someone adds you as a maintainer of a layer as a matter of courtesy
it would be nice if you get an email.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-12-06 08:18:49 +13:00
Paul Eggleton
61e9b04859 Fix drop-down alignment on duplicates page
Hack the dropdown HTML to fix the dropdown menu appearing at the bottom
of the "Duplicate recipes" section (which may be below the bottom of the
browser window). This is probably not the correct fix, but it works.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-20 11:58:42 +13:00
Paul Eggleton
e10d21240c Fix Submit/Edit layer form field rendering issue
Some controls on the Submit / Edit Layer page were getting the wrong
width and any "feedback" elements were being pulled to the right because
of the size of their parent divs. Instead of pulling in the size of the
form control, act on the div to fix the problem. Additionally the
help text should now be allowed to expand to the full width of the
parent div since that is now sensible.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-20 11:58:42 +13:00
Paul Eggleton
4f25afe805 Make login button return user to previous page
Send the current URL path as a parameter to the login URL so that after
successfully logging in, you come back to where you were beforehand.
Also hide the login/user drop-down on the login page since it's
superfluous.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-20 10:57:56 +13:00
Paul Eggleton
b767d709ff Return to previous page after Edit profile
Use a parameter to pass the current page to the Edit Profile URL so that
saving or cancelling returns you to that same page.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:42 +13:00
Paul Eggleton
61dd9527ba requirements.txt: bump versions
Upgrade everything with the exception of django-registration 3.0 and
django-reversion 3.0.x which both require some work to migrate to.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:42 +13:00
Paul Eggleton
fc45dfc8dc README: add missing mention of TableSorter
The RRS code uses this but we weren't mentioning that here.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:42 +13:00
Paul Eggleton
ad2701003f Add search to layer dependencies list on edit/submit layer form
The layer index has quite a number of layers in it these days and thus
this list is quite long to scroll through. Use the same layer filtering
method as we do on the layers page for the dependencies list.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:42 +13:00
Paul Eggleton
403e94a0f1 Add search to duplicates/comparison layer filter dialog
The layer index has quite a number of layers in it these days and thus
this list is quite long to scroll through. Use the same layer filtering
method as we do on the layers page, taking care to ensure that the
"select all" and "select none" buttons only affect what you can see when
the layer list is filtered.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:42 +13:00
Paul Eggleton
aa10299356 Replace uitablefilter usage with jQuery
We were using uitablefilter.js to provide live filtering of table rows
based upon a search field value, but it turns out this module really
isn't necessary - we can accomplish the same thing using simple jQuery
code. While we're at it, enable the search field on the layers list page
to work in conjunction with with the drop-down layer type selection, fix
pasting into the search field and refreshing with a search specified.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:42 +13:00
Paul Eggleton
b940e7c431 Use prop() instead of attr() to set checked property
Using attr('checked') is not reliable, the jQuery documentation
recommends using prop() with jQuery 1.6 or later and it definitely works
more reliably in my testing.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:42 +13:00
Paul Eggleton
3c25eda61e Exclude "Distro-specific" from "unknown/not available" in other distro search
If you're searching for items that are of status "unknown/not available"
in the other distro packages, you're usually doing so to find things
that haven't been dispositioned. Originally I had included them here
because the "available" status correctly excludes items with the
"distro-specific" status and "unknown/not available" seemed like it
should be the inverse, but usage seems to require that it not be.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:42 +13:00
Paul Eggleton
eaae0d8b02 tools/github-fetch: fix marking deleted
I used the wrong path variable here resulting in failure if anything was
deleted.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:42 +13:00
Paul Eggleton
17ced0ee38 tools/github-fetch: handle non-fast-forward merges
We don't particularly care if someone force-pushed master, just fetch
the latest repo.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:42 +13:00
Paul Eggleton
f2daec4318 Tweak activation complete page
* Add a convenience link to the login page
* Make the language a bit less terse

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:42 +13:00
Paul Eggleton
e7e43ce03b Require re-registration if user changes email address
If a user goes to Edit Profile and changes their email address,
deactivate their account temporarily and make them go through the
registration process to confirm that the new email address is in fact
valid and theirs.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:39 +13:00
Paul Eggleton
54c9ac8e43 Improve password form formatting
Borrow the formatting from some of our other forms which looks much
nicer (and shows field errors properly).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-06 13:58:46 +13:00
Paul Eggleton
047fc704f7 Add some text to the password reset email
What we had before was a little bit terse, so add some reasonable text.

Also mention in the confirmation page that sending an email is
predicated on there actually being an account matching the specified
email address (and we deliberately don't specify whether there is or
not, in order to prevent user enumeration).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-06 13:58:40 +13:00
Paul Eggleton
5d308d943e Enable password strength validation by default
Use Django's built-in password validators with reasonable settings, and
add a basic complexity validator since there isn't one provided.

Additionally, fix the registration form so that it shows the help text
which includes a description of what the password requirements are.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-06 13:58:32 +13:00
Paul Eggleton
55d6840072 Add a script to help keep requirements.txt up to date
Add a script that creates a virtualenv and does the following:
1) Tests pip install -r requirements.txt
2) Runs pip freeze and diffs the output to requirements.txt (to
   check if any dependencies have been missed when it was last updated)
3) Runs pip list --outdated and writes the output to a file so you can
   see which packages might need updating (of course some may not be
   able to be updated easily e.g. if an API change has been made)
4) Installs safety and runs "safety check" to check if there are any
   known vulnerabilities

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-06 13:58:30 +13:00
Paul Eggleton
476348a598 requirements.txt: update GitPython dependencies
Upgrade GitPython and trim out old dependencies: gitdb and smmap are no
longer needed by current GitPython, gitdb2 and smmap2 have replaced
them.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-06 13:58:07 +13:00
Paul Eggleton
dff454815f Use django-axes to lockout after multiple failed logins
Repeated failed login attempts should trigger lockout to prevent
brute-forcing and user enumeration - django-axes does this using account
and IP-based lockout.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-06 13:58:03 +13:00
Paul Eggleton
f609ce3919 Restrict the number of username change attempts
On top of the CAPTCHA, prevent more than 10 name change attempts for the
same user with the Edit Profile page so that username enumeration is
made even more difficult.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-06 11:38:25 +13:00
Paul Eggleton
091f5e4ef8 Allow user to edit their username in the Edit Profile form
Users may want to change their usernames for a number of different
reasons, but at the moment we require them to contact an admin to do
that. Provided we validate the new username correctly and add a CAPTCHA
to make automated enumeration difficult, we can add username to the Edit
Profile form and then users can do that any time they wish.

While we're doing this, show a message when the profile is successfully
updated.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-06 11:38:21 +13:00
Paul Eggleton
2c8f979f9c Implement "delete account" function
Make it possible for users to delete their own account and all associated
information from the database, should they decide they no longer wish to
use it.

(I checked the implications of doing this on our model structure -
anything with a foreign key to user is safe to delete with the exception
of RRS MaintenancePlan.admin which I needed to change on_delete for so
that it doesn't get deleted with the user).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-06 11:31:57 +13:00
Paul Eggleton
23194fc5d4 Add CAPTCHA to registration/password reset forms
Extend and override the default views so we can extend and override the
default forms to add a CAPTCHA field. This should prevent the automated
account creation requests we've been seeing on layers.openembedded.org
(luckily failing anyway due to bad domain names), but in any case this
also improves security by making it harder to do user enumeration.

For the registration page in particular, because Django's forms logic
tries to be helpful by showing all errors at once, we need to change it
so that if there's an error for the CAPTCHA then you only see that error
and no other - in particular you won't see "that username already
exists" if that is the case.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-06 11:31:07 +13:00
Paul Eggleton
5d7cf9e5ae views: fix incorrect class usage in ClassSearchView
This was copy-pasted from the DistroSearchView and I clearly forgot to
replace the reference here.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-06 10:37:31 +13:00
Paul Eggleton
0bfb98971c settings: add settings for CAPTCHA accessibility
To enable audio support for django-simple-captcha for accessibility
purposes, you need to provide the full paths to the flite and sox
binaries, so add these settings (commented out by default to avoid extra
dependencies for those that don't need it).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-06 10:37:31 +13:00
Paul Eggleton
4acb0b2c91 update: handle if there is no origin/master branch in bitbake repo
It's been reported to me that it's possible in a custom setup to have a
bitbake repo that intentionally doesn't have a master branch, so pick
up the bitbake branch from the master Branch record if it exists and the
bitbake_branch field is set.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-06 10:33:39 +13:00
Paul Eggleton
631116a1c4 Replace nvd3 with Chart.js
nvd3 and its python/django wrappers appear to be no longer actively
maintained, and at least the wrappers were a bit clunky to use. Looking
around for a suitable replacement, Chart.js seems capable, has no
additional dependencies and is fairly simple to use. As a bonus we get
to drop a few Python dependencies from our list.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-06 10:33:39 +13:00
Paul Eggleton
e591d1820a Track and enable reporting on duplicate inc files
It's not too common but there are instances where people have copied
.inc files into their own layer and modified them, and if you are using
such a layer that could result in unexpected behaviour. In order to get
a handle on when this is being done, collect data about all .inc files
and show duplicates in the Duplicates screen.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-06 10:33:39 +13:00
Paul Eggleton
24b6e13442 docker: add TASK_LOG_DIR to docker settings file
This was missed when the main settings.py got updated recently.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-10-04 11:15:39 +13:00
Paul Eggleton
accbf15a8b views: split out recipe search query to its own function
Split out the code used in the recipe search views to its own function
and use that same function in three different places rather than having
a copy of largely the same code. Also take the opportunity to add some
comments.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-10-04 09:18:51 +13:00
Paul Eggleton
f76e811403 Improve handling of invalid branch in layer CSV export view
On layers.openembedded.org we're seeing requests from some search engine
crawlers requesting the CSV export URL with an invalid branch for the
layer. I couldn't see the referer anywhere in the logs but I suspect it
has to do with some recent cleanup work I did in the database where I
deleted some invalid LayerBranch records - they were probably following
links in a cached version of the webpage. In any event we want to return
404 in this situation rather than an internal server error.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-10-04 08:43:02 +13:00