Commit Graph

1098 Commits

Author SHA1 Message Date
Paul Eggleton
7e99440afd docker: Increase nginx max upload size
It's likely an image manifest tarball will be more than 1mb in size;
increase it to 16mb to handle where a large number of patches are
included in an image containing a lot of packages from different
recipes.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
348bd2661c Disable autocomplete on sensitive fields
Disable autocomplete for registration/login/password change/password
reset forms, and security question / captcha fields on Edit Profile
form.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
0196eda3f1 dockersetup: enable error report emails
Error reports get emailed to the addresses specified in the ADMINS
setting, but up until now dockersetup wasn't setting this and so the
details of any internal error were not visible anywhere. If we're not
updating, prompt the user for an email address early in the process,
and use this for the ADMINS setting, the superuser account, and Let's
Encrypt (if selected).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
8d52470654 dockersetup: add option to skip database migrations
If we've messed around with the migrations during development then they
may need to be run carefully (with some fake steps), so provide an
option to update the container and then do the migrations manually
afterwards.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
0d19d76d8a Add a script to create initial db dump
For the purposes of creating a dump suitable for importing into a fresh
database during initial setup, add a script that creates a dump of an
existing database with the complete structure but without any data that
involves users (or anything else we don't want).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
e033154f4f import_layers: fix URL construction
If you specified only the layer index URL and not the full API URL i.e.
https://layers.openembedded.org then we put an extra slash in the URL
which resulted in the request being redirected to the main page which
breaks the request.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
b5892b0fdb Rework README documents
Change the focus to really be on the docker-based setup:

* Integrate docker/README into the main README
* Move all documentation of the standalone setup to README.devel

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
ee7e5a0564 dockersetup: set site info and email host
In order to have registration/password reset emails working properly we
need to have both the site domain and email host/port set correctly.
Email host/port is easy to set (environment variable read into the
configuration). However, to set the site info, create a simple script
that we can call to do so.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
604330f41a dockersetup: move superuser account creation to the end
If we move the superuser account creation to the end, it's practical to
Ctrl+C it in case you already have a superuser in the data that got
imported.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:01 +12:00
Paul Eggleton
b99143d2e3 dockersetup: import database dump before running migrations
If the database dump is older than the application, there may be
migrations to run, so we need to run them after importing and not
beforehand.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:01 +12:00
Paul Eggleton
303d7ca235 Use shell=False where possible with utils.runcmd()
It's best practice for security reasons to use shell=False and pass
command line arguments as a list; it also avoids some pain with
escaping, so let's use it everywhere we can (in fact we're only left
with one place in layerindex/tasks.py where we now pass shell=True).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:01 +12:00
Amber Elliot
ecd584f609 requirements.txt: Require secure version of Django.
Change minimum required version of Django to 1.11.11.

Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:01 +12:00
Paul Eggleton
c38d1c37a1 docker: enable SECURE_BROWSER_XSS_FILTER
This is a rudimentary extra protection against XSS, we might as well
enable it.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:01 +12:00
Paul Eggleton
df189e3455 Disable caching on auth views
These pages might contain confidential information, so they should not
be cached.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:01 +12:00
Paul Eggleton
3cc532f435 Update TableSorter to latest version of active fork
The original TableSorter is unmaintained, move to the latest version of
the active fork:

https://mottie.github.io/tablesorter/docs/

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:01 +12:00
Paul Eggleton
a052d8d7f7 docker: restart services automatically unless stopped
Restarting these services automatically is what you'd want in
production.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
6051c734e1 docker: pass through options for migrate script
Allow specifying arguments to the migrate command (e.g. --fake) via this
shortcut script.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Amber Elliot
5eecb5bf5c layerindex/urls.py: improve formatting
Indentation, spacing, and wrapping of a long import line.

Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Amber Elliot
9a9bbeb8b6 Add user security questions
Add user security questions upon registration as extra authentication
for password reset. Three unique security questions must be chosen and
answered. Answers are then stored in the database with the same hashing
algorithm as the users's password.

On password reset, users get two chances to get two out of three
security questions answered correctly. After a second failure their
account is locked and email is sent to the admin. The same template is
shown for the axes lockout. Super user cannot reset their password until
they set security questions.

Users can update their security questions or add them if they weren't
originally set (in the case of super user) in Edit Profile.

Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
2019-07-17 11:30:56 +12:00
Amber Elliot
0f3b3e42a6 Upgrade django-registration to version 3.0.
This involves changing how registration templates are referenced
and how the activation email is sent on user's email address change.

Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
2019-07-17 11:30:56 +12:00
Amber Elliot
d333657e9d admin.py: Add custom SiteAdmin model.
This allows users to see the site ID in the admin page.

Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
24dcd4618a dockersetup: support importing gzip compressed database dumps
Database dumps are simply SQL (i.e. plain text) so they tend to be
both large and easy to compress, so having them gzipped is something
worth supporting.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
841124a662 .gitignore: add some more files/dirs
Ignore some files and directories I typically have around in my
development tree.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
e9915dd359 dockersetup: use with open in readfile/writefile
Best practice to ensure files always get closed.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
7a248df9f5 dockersetup: use separate db account with lower privileges
Make the app should use a database account that has only the privileges
it needs rather than the root account for improved security.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
3775efc848 dockersetup: warn if http proxy specified without https
You really need to specify these both together since during updates we
will very likely fetch over both HTTP and HTTPS.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
f50f48cf01 dockersetup: fix auto-generated passwords
* We can't use # because it can result in part of the password being
  trimmed off when it gets put in a configuration file.
* Similarly, our parsing splits on = so it is unwise to allow as a valid
  character.
* Add uppercase characters for a little more randomisation.
2019-07-17 11:30:56 +12:00
Paul Eggleton
99a86d7e0a dockersetup: support update/reinstall mode
Make it easier to do updates and also reinstall from scratch; if the
application is already installed, we force the user be explicit about
which one they want to do so that there is less chance of data loss.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
4370564b91 dockersetup: move HTTPS code to its own function
Refactor out HTTPS configuration code to a new function to give the code
a bit more structure.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
e78c4dae9c docker: enable user/password for RabbitMQ server
Add settings for user/password for the RabbitMQ server and make
dockersetup.py set it up. (The rabbitmq container intrinsically
understands RABBITMQ_DEFAULT_*, and for the sake of consistency I've
reused those variables for the other containers.)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
dad461cd0f dockerignore: add docker files and tests
These files shouldn't be copied into containers, so add them to
the .dockerignore file.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
3bc7dce405 docker: set mariadb wait_timeout to upstream default
The default wait_timeout we are getting from the official MariaDB
container (as inherited from Debian) is 600s, which is too short if you
walk away while the setup script is running and it stops on the
"manage.py createsuperuser" step for longer than 10 minutes. We don't
really need this timeout, so just use the upstream default of 28800 (8h)
instead.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
95c650d7c4 nginx: set some limits for DDOS protection
Set some reasonable default limits in the nginx configuration.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
efe74a451f dockersetup: add letsencrypt support
The application is most likely to be used in an internal setting,
however should someone want to make it accessible externally, add
support for Let's Encrypt certificates for serving via HTTPS.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
c67313593f dockersetup: Show intro message
Apply some branding, improve user friendliness and give the user a
chance to break out if they actually don't want to run the installation.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
cb4955fe0b dockersetup: add HTTPS support and use by default
If we want a minimum level of security we should enable HTTPS. However,
the only practical way we can do that without the user having to do further
infrastructure setup and/or pay a certification authority is to use a
self-signed certificate. Do this by default, and also provide an option
to specify a previously obtained certificate/key pair.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
c717a827d3 docker: use quoted values
Avoids issues with certain characters in values.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
29f30b5d38 docker/nginx*.conf: fix up indentation
Stick to four spaces, no tabs.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
2afc25cb57 dockersetup: tweak portmapping option and display port
* Use default macro in command line help text to avoid duplication
* Make config editing slightly more robust
* Display a message at the end telling the user how to access the
  application based upon the port mapping

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
e1f26deacb dockersetup: add some error checking to the setup script
Ensure we stop processing if there's a failure.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
f160927765 dockersetup: formatting fixes
- Move copyright/license statement to top
- Tabs to spaces
- Drop trailing spaces

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Amber Elliot
aa07c1b451 docker: add setup script
Adding setup script for docker containers. The script will edit all
necessary configuration files, build and launch all containers, and
do the initial database setup - including populating the database
with data supplied by the user.

Changed docker/README to reflect new setup instructions.

Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
dde0a82a35 import_otherdistro: improve display of deleted items
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
c84024a51b import_otherdistro: add description option
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
cd88e0c0ff update_classic_status: categorise perl packages
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
33fba17414 update_classic_status: set category for KDE packages
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
b05a24d41f update_classic_status: recognise pythonhosted.org as python
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
08c979ac58 docker: Add docker-compose file
This makes deployment (and updates) much easier. You still need to
follow some setup instructions in the docker/README file, but the number
of steps is reduced and most commands simplified.

Thanks to Konrad Scherer <Konrad.Scherer@windriver.com> for suggesting
this (although the docker-compose.yml here was written from scratch).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
d1be5af067 Split out recipe dependency handling to its own function
Make it easier to call this outside of the context of the update
process.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
bcedcb7006 Add ability to hide branches
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00