Commit Graph

840 Commits

Author SHA1 Message Date
Paul Eggleton
604330f41a dockersetup: move superuser account creation to the end
If we move the superuser account creation to the end, it's practical to
Ctrl+C it in case you already have a superuser in the data that got
imported.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:01 +12:00
Paul Eggleton
b99143d2e3 dockersetup: import database dump before running migrations
If the database dump is older than the application, there may be
migrations to run, so we need to run them after importing and not
beforehand.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:01 +12:00
Paul Eggleton
303d7ca235 Use shell=False where possible with utils.runcmd()
It's best practice for security reasons to use shell=False and pass
command line arguments as a list; it also avoids some pain with
escaping, so let's use it everywhere we can (in fact we're only left
with one place in layerindex/tasks.py where we now pass shell=True).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:01 +12:00
Amber Elliot
ecd584f609 requirements.txt: Require secure version of Django.
Change minimum required version of Django to 1.11.11.

Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:01 +12:00
Paul Eggleton
c38d1c37a1 docker: enable SECURE_BROWSER_XSS_FILTER
This is a rudimentary extra protection against XSS, we might as well
enable it.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:01 +12:00
Paul Eggleton
df189e3455 Disable caching on auth views
These pages might contain confidential information, so they should not
be cached.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:01 +12:00
Paul Eggleton
3cc532f435 Update TableSorter to latest version of active fork
The original TableSorter is unmaintained, move to the latest version of
the active fork:

https://mottie.github.io/tablesorter/docs/

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:01 +12:00
Paul Eggleton
a052d8d7f7 docker: restart services automatically unless stopped
Restarting these services automatically is what you'd want in
production.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
6051c734e1 docker: pass through options for migrate script
Allow specifying arguments to the migrate command (e.g. --fake) via this
shortcut script.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Amber Elliot
5eecb5bf5c layerindex/urls.py: improve formatting
Indentation, spacing, and wrapping of a long import line.

Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Amber Elliot
9a9bbeb8b6 Add user security questions
Add user security questions upon registration as extra authentication
for password reset. Three unique security questions must be chosen and
answered. Answers are then stored in the database with the same hashing
algorithm as the users's password.

On password reset, users get two chances to get two out of three
security questions answered correctly. After a second failure their
account is locked and email is sent to the admin. The same template is
shown for the axes lockout. Super user cannot reset their password until
they set security questions.

Users can update their security questions or add them if they weren't
originally set (in the case of super user) in Edit Profile.

Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
2019-07-17 11:30:56 +12:00
Amber Elliot
0f3b3e42a6 Upgrade django-registration to version 3.0.
This involves changing how registration templates are referenced
and how the activation email is sent on user's email address change.

Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
2019-07-17 11:30:56 +12:00
Amber Elliot
d333657e9d admin.py: Add custom SiteAdmin model.
This allows users to see the site ID in the admin page.

Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
24dcd4618a dockersetup: support importing gzip compressed database dumps
Database dumps are simply SQL (i.e. plain text) so they tend to be
both large and easy to compress, so having them gzipped is something
worth supporting.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
841124a662 .gitignore: add some more files/dirs
Ignore some files and directories I typically have around in my
development tree.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
e9915dd359 dockersetup: use with open in readfile/writefile
Best practice to ensure files always get closed.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
7a248df9f5 dockersetup: use separate db account with lower privileges
Make the app should use a database account that has only the privileges
it needs rather than the root account for improved security.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
3775efc848 dockersetup: warn if http proxy specified without https
You really need to specify these both together since during updates we
will very likely fetch over both HTTP and HTTPS.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
f50f48cf01 dockersetup: fix auto-generated passwords
* We can't use # because it can result in part of the password being
  trimmed off when it gets put in a configuration file.
* Similarly, our parsing splits on = so it is unwise to allow as a valid
  character.
* Add uppercase characters for a little more randomisation.
2019-07-17 11:30:56 +12:00
Paul Eggleton
99a86d7e0a dockersetup: support update/reinstall mode
Make it easier to do updates and also reinstall from scratch; if the
application is already installed, we force the user be explicit about
which one they want to do so that there is less chance of data loss.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
4370564b91 dockersetup: move HTTPS code to its own function
Refactor out HTTPS configuration code to a new function to give the code
a bit more structure.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
e78c4dae9c docker: enable user/password for RabbitMQ server
Add settings for user/password for the RabbitMQ server and make
dockersetup.py set it up. (The rabbitmq container intrinsically
understands RABBITMQ_DEFAULT_*, and for the sake of consistency I've
reused those variables for the other containers.)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
dad461cd0f dockerignore: add docker files and tests
These files shouldn't be copied into containers, so add them to
the .dockerignore file.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
3bc7dce405 docker: set mariadb wait_timeout to upstream default
The default wait_timeout we are getting from the official MariaDB
container (as inherited from Debian) is 600s, which is too short if you
walk away while the setup script is running and it stops on the
"manage.py createsuperuser" step for longer than 10 minutes. We don't
really need this timeout, so just use the upstream default of 28800 (8h)
instead.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
95c650d7c4 nginx: set some limits for DDOS protection
Set some reasonable default limits in the nginx configuration.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
efe74a451f dockersetup: add letsencrypt support
The application is most likely to be used in an internal setting,
however should someone want to make it accessible externally, add
support for Let's Encrypt certificates for serving via HTTPS.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
c67313593f dockersetup: Show intro message
Apply some branding, improve user friendliness and give the user a
chance to break out if they actually don't want to run the installation.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
cb4955fe0b dockersetup: add HTTPS support and use by default
If we want a minimum level of security we should enable HTTPS. However,
the only practical way we can do that without the user having to do further
infrastructure setup and/or pay a certification authority is to use a
self-signed certificate. Do this by default, and also provide an option
to specify a previously obtained certificate/key pair.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
c717a827d3 docker: use quoted values
Avoids issues with certain characters in values.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
29f30b5d38 docker/nginx*.conf: fix up indentation
Stick to four spaces, no tabs.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
2afc25cb57 dockersetup: tweak portmapping option and display port
* Use default macro in command line help text to avoid duplication
* Make config editing slightly more robust
* Display a message at the end telling the user how to access the
  application based upon the port mapping

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
e1f26deacb dockersetup: add some error checking to the setup script
Ensure we stop processing if there's a failure.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
f160927765 dockersetup: formatting fixes
- Move copyright/license statement to top
- Tabs to spaces
- Drop trailing spaces

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Amber Elliot
aa07c1b451 docker: add setup script
Adding setup script for docker containers. The script will edit all
necessary configuration files, build and launch all containers, and
do the initial database setup - including populating the database
with data supplied by the user.

Changed docker/README to reflect new setup instructions.

Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
dde0a82a35 import_otherdistro: improve display of deleted items
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
c84024a51b import_otherdistro: add description option
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
cd88e0c0ff update_classic_status: categorise perl packages
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
33fba17414 update_classic_status: set category for KDE packages
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
b05a24d41f update_classic_status: recognise pythonhosted.org as python
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
08c979ac58 docker: Add docker-compose file
This makes deployment (and updates) much easier. You still need to
follow some setup instructions in the docker/README file, but the number
of steps is reduced and most commands simplified.

Thanks to Konrad Scherer <Konrad.Scherer@windriver.com> for suggesting
this (although the docker-compose.yml here was written from scratch).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
d1be5af067 Split out recipe dependency handling to its own function
Make it easier to call this outside of the context of the update
process.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
bcedcb7006 Add ability to hide branches
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
dba1fbe5d1 RRS: add missing migration
This should have been added in 2c8f979f9c
to make it possible to delete an account that is the administrator for a
maintenance plan without removing the maintenance plan.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-05-28 09:57:58 +12:00
Paul Eggleton
8f37d77598 RRS: move Export button and restore Tools drop-down menu
Move the Export Recipe List button, change it to match the Export CSV
button on the layer detail page and put the Tools menu back (useful for
accessing the Admin site if not just for consistency). The Submit Layer
button remains hidden.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-21 15:47:56 +13:00
Paul Eggleton
03b39e89c0 RRS: fix showing all recipes for multi-layer maintenance plans
If you had more than one layer on a maintenance plan, only the recipes
from the last layer were shown because we were using a recipe list
variable from the last iteration. The fix just turned out to be to
indent the code that read from that list so it runs every iteration
instead of at the end.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-21 14:15:52 +13:00
Paul Eggleton
7ab7766ec7 RRS: Fix raw SQL statements to use parameters
For security reasons it's best practice to use parameters to pass values
into SQL statements and not substitute them in as strings (with Django's
database API the distinction is subtle, but we pass in the parameters in
a second list parameter instead of using % to substitute them before
passing the query in).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-21 14:15:42 +13:00
Paul Eggleton
a17652256f Add admin handling for Patch objects
Allow searching by path and filtering on layer / branch.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:19:10 +13:00
Paul Eggleton
975d2927d1 Record patch application order and use as default sorting
Patches often need to be applied in a specific order. For OE recipes we
were always storing the Patch objects correct order as they are
refreshed every time the recipe itself is refreshed, however for other
distro comparisons, import_otherdistro.py attempts to preserve existing
records, adds new ones and then deletes whatever is left over, which may
result in the order getting messed up over time. To avoid this issue,
record the order next to the patch and set the model meta-info to use
this to sort Patch queries by default.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:17:21 +13:00
Paul Eggleton
f9c5143700 Split out patch status read to a method on the Patch model
Make it easier to call this outside of the context of the update
process.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:14:50 +13:00
Paul Eggleton
ce5141fc39 RRS: add tool to import/export upstream history data
The recipe upstream history can't be reconstructed, so we need to be
able to export it out of an old database an import it into the new one.
There's also a "remove-duplicates" function; whilst not strictly
necessary it did help us to remove duplicate records resulting from
running the import twice (due to a bug) and may be useful in future
under similar circumstances.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-02-14 10:05:53 +13:00