Commit Graph

818 Commits

Author SHA1 Message Date
Paul Eggleton
dad461cd0f dockerignore: add docker files and tests
These files shouldn't be copied into containers, so add them to
the .dockerignore file.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
3bc7dce405 docker: set mariadb wait_timeout to upstream default
The default wait_timeout we are getting from the official MariaDB
container (as inherited from Debian) is 600s, which is too short if you
walk away while the setup script is running and it stops on the
"manage.py createsuperuser" step for longer than 10 minutes. We don't
really need this timeout, so just use the upstream default of 28800 (8h)
instead.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
95c650d7c4 nginx: set some limits for DDOS protection
Set some reasonable default limits in the nginx configuration.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
efe74a451f dockersetup: add letsencrypt support
The application is most likely to be used in an internal setting,
however should someone want to make it accessible externally, add
support for Let's Encrypt certificates for serving via HTTPS.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
c67313593f dockersetup: Show intro message
Apply some branding, improve user friendliness and give the user a
chance to break out if they actually don't want to run the installation.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
cb4955fe0b dockersetup: add HTTPS support and use by default
If we want a minimum level of security we should enable HTTPS. However,
the only practical way we can do that without the user having to do further
infrastructure setup and/or pay a certification authority is to use a
self-signed certificate. Do this by default, and also provide an option
to specify a previously obtained certificate/key pair.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
c717a827d3 docker: use quoted values
Avoids issues with certain characters in values.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
29f30b5d38 docker/nginx*.conf: fix up indentation
Stick to four spaces, no tabs.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
2afc25cb57 dockersetup: tweak portmapping option and display port
* Use default macro in command line help text to avoid duplication
* Make config editing slightly more robust
* Display a message at the end telling the user how to access the
  application based upon the port mapping

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
e1f26deacb dockersetup: add some error checking to the setup script
Ensure we stop processing if there's a failure.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
f160927765 dockersetup: formatting fixes
- Move copyright/license statement to top
- Tabs to spaces
- Drop trailing spaces

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Amber Elliot
aa07c1b451 docker: add setup script
Adding setup script for docker containers. The script will edit all
necessary configuration files, build and launch all containers, and
do the initial database setup - including populating the database
with data supplied by the user.

Changed docker/README to reflect new setup instructions.

Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
dde0a82a35 import_otherdistro: improve display of deleted items
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
c84024a51b import_otherdistro: add description option
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
cd88e0c0ff update_classic_status: categorise perl packages
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
33fba17414 update_classic_status: set category for KDE packages
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
b05a24d41f update_classic_status: recognise pythonhosted.org as python
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
08c979ac58 docker: Add docker-compose file
This makes deployment (and updates) much easier. You still need to
follow some setup instructions in the docker/README file, but the number
of steps is reduced and most commands simplified.

Thanks to Konrad Scherer <Konrad.Scherer@windriver.com> for suggesting
this (although the docker-compose.yml here was written from scratch).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
d1be5af067 Split out recipe dependency handling to its own function
Make it easier to call this outside of the context of the update
process.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
bcedcb7006 Add ability to hide branches
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
dba1fbe5d1 RRS: add missing migration
This should have been added in 2c8f979f9c
to make it possible to delete an account that is the administrator for a
maintenance plan without removing the maintenance plan.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-05-28 09:57:58 +12:00
Paul Eggleton
8f37d77598 RRS: move Export button and restore Tools drop-down menu
Move the Export Recipe List button, change it to match the Export CSV
button on the layer detail page and put the Tools menu back (useful for
accessing the Admin site if not just for consistency). The Submit Layer
button remains hidden.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-21 15:47:56 +13:00
Paul Eggleton
03b39e89c0 RRS: fix showing all recipes for multi-layer maintenance plans
If you had more than one layer on a maintenance plan, only the recipes
from the last layer were shown because we were using a recipe list
variable from the last iteration. The fix just turned out to be to
indent the code that read from that list so it runs every iteration
instead of at the end.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-21 14:15:52 +13:00
Paul Eggleton
7ab7766ec7 RRS: Fix raw SQL statements to use parameters
For security reasons it's best practice to use parameters to pass values
into SQL statements and not substitute them in as strings (with Django's
database API the distinction is subtle, but we pass in the parameters in
a second list parameter instead of using % to substitute them before
passing the query in).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-21 14:15:42 +13:00
Paul Eggleton
a17652256f Add admin handling for Patch objects
Allow searching by path and filtering on layer / branch.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:19:10 +13:00
Paul Eggleton
975d2927d1 Record patch application order and use as default sorting
Patches often need to be applied in a specific order. For OE recipes we
were always storing the Patch objects correct order as they are
refreshed every time the recipe itself is refreshed, however for other
distro comparisons, import_otherdistro.py attempts to preserve existing
records, adds new ones and then deletes whatever is left over, which may
result in the order getting messed up over time. To avoid this issue,
record the order next to the patch and set the model meta-info to use
this to sort Patch queries by default.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:17:21 +13:00
Paul Eggleton
f9c5143700 Split out patch status read to a method on the Patch model
Make it easier to call this outside of the context of the update
process.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:14:50 +13:00
Paul Eggleton
ce5141fc39 RRS: add tool to import/export upstream history data
The recipe upstream history can't be reconstructed, so we need to be
able to export it out of an old database an import it into the new one.
There's also a "remove-duplicates" function; whilst not strictly
necessary it did help us to remove duplicate records resulting from
running the import twice (due to a bug) and may be useful in future
under similar circumstances.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:05:53 +13:00
Paul Eggleton
c74e4bd4d9 rrs_maintainer_history: check out layer branch before looking for maintainers.inc
The revision that happens to be checked out at the moment could be
anything and might not actually have a maintainers.inc, so check out the
proper branch before looking for it.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:05:53 +13:00
Paul Eggleton
0d19f49dd7 RRS: fix sorting arrow positioning
I'm not sure when or how they got broken but the sorting arrows were
overlapping the text, so move them back into the proper location, 8px
from the right of the column.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:05:53 +13:00
Paul Eggleton
53e6ed36b6 docker: add wget to dependencies
The upstream version checking used by the RRS needs wget to fetch
http/https URLs, so add it into the container.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:05:53 +13:00
Paul Eggleton
f6f747fb92 Replace use of assert with exceptions
asserts don't really belong in non-test code, let's handle these
situations properly instead.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:05:53 +13:00
Paul Eggleton
9c65bf254e Use try...finally or with to ensure files get closed
Best practices state that you should use a mechanism that ensures files
get closed in case of any error, so let's do that.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:05:53 +13:00
Paul Eggleton
24739d9369 local.conf: allow for upstream_tracking.inc to be missing
In OE-Core commit 22447eba250656489a6ed636fe58cb304d74e975,
conf/distro/include/upstream_tracking.inc has been removed, so allow the
file to be missing by changing require to include.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:05:53 +13:00
Paul Eggleton
2c3c287a33 Fix errors due to races deleting bitbake temp files
Errors deleting bitbake.sock and bitbake.lock have been observed when
shutting down tinfoil at the end of some of these scripts. Move the code
used in the main layer index update script to a function in utils.py and
use it everywhere in order to avoid the issue.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:05:53 +13:00
Paul Eggleton
0929289465 Add links to other branch recipes in recipe detail
Add links to the same recipe in other branches in the recipe detail page
(and RRS recipe detail page) so that you can see which versions are
available in other branches and drill down to the detail if you want to.

Implements [YOCTO #13019].

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-12-06 09:25:06 +13:00
Paul Eggleton
eeffb2d675 Send people an email when another user adds them as a maintainer
If someone adds you as a maintainer of a layer as a matter of courtesy
it would be nice if you get an email.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-12-06 08:18:49 +13:00
Paul Eggleton
61e9b04859 Fix drop-down alignment on duplicates page
Hack the dropdown HTML to fix the dropdown menu appearing at the bottom
of the "Duplicate recipes" section (which may be below the bottom of the
browser window). This is probably not the correct fix, but it works.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-20 11:58:42 +13:00
Paul Eggleton
e10d21240c Fix Submit/Edit layer form field rendering issue
Some controls on the Submit / Edit Layer page were getting the wrong
width and any "feedback" elements were being pulled to the right because
of the size of their parent divs. Instead of pulling in the size of the
form control, act on the div to fix the problem. Additionally the
help text should now be allowed to expand to the full width of the
parent div since that is now sensible.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-20 11:58:42 +13:00
Paul Eggleton
4f25afe805 Make login button return user to previous page
Send the current URL path as a parameter to the login URL so that after
successfully logging in, you come back to where you were beforehand.
Also hide the login/user drop-down on the login page since it's
superfluous.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-20 10:57:56 +13:00
Paul Eggleton
b767d709ff Return to previous page after Edit profile
Use a parameter to pass the current page to the Edit Profile URL so that
saving or cancelling returns you to that same page.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:42 +13:00
Paul Eggleton
61dd9527ba requirements.txt: bump versions
Upgrade everything with the exception of django-registration 3.0 and
django-reversion 3.0.x which both require some work to migrate to.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:42 +13:00
Paul Eggleton
fc45dfc8dc README: add missing mention of TableSorter
The RRS code uses this but we weren't mentioning that here.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:42 +13:00
Paul Eggleton
ad2701003f Add search to layer dependencies list on edit/submit layer form
The layer index has quite a number of layers in it these days and thus
this list is quite long to scroll through. Use the same layer filtering
method as we do on the layers page for the dependencies list.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:42 +13:00
Paul Eggleton
403e94a0f1 Add search to duplicates/comparison layer filter dialog
The layer index has quite a number of layers in it these days and thus
this list is quite long to scroll through. Use the same layer filtering
method as we do on the layers page, taking care to ensure that the
"select all" and "select none" buttons only affect what you can see when
the layer list is filtered.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:42 +13:00
Paul Eggleton
aa10299356 Replace uitablefilter usage with jQuery
We were using uitablefilter.js to provide live filtering of table rows
based upon a search field value, but it turns out this module really
isn't necessary - we can accomplish the same thing using simple jQuery
code. While we're at it, enable the search field on the layers list page
to work in conjunction with with the drop-down layer type selection, fix
pasting into the search field and refreshing with a search specified.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:42 +13:00
Paul Eggleton
b940e7c431 Use prop() instead of attr() to set checked property
Using attr('checked') is not reliable, the jQuery documentation
recommends using prop() with jQuery 1.6 or later and it definitely works
more reliably in my testing.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:42 +13:00
Paul Eggleton
3c25eda61e Exclude "Distro-specific" from "unknown/not available" in other distro search
If you're searching for items that are of status "unknown/not available"
in the other distro packages, you're usually doing so to find things
that haven't been dispositioned. Originally I had included them here
because the "available" status correctly excludes items with the
"distro-specific" status and "unknown/not available" seemed like it
should be the inverse, but usage seems to require that it not be.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:42 +13:00
Paul Eggleton
eaae0d8b02 tools/github-fetch: fix marking deleted
I used the wrong path variable here resulting in failure if anything was
deleted.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:42 +13:00
Paul Eggleton
17ced0ee38 tools/github-fetch: handle non-fast-forward merges
We don't particularly care if someone force-pushed master, just fetch
the latest repo.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2018-11-19 16:44:42 +13:00