Commit Graph

46 Commits

Author SHA1 Message Date
Paul Eggleton
19e40eff5a dockersetup: minor command line comment fix
We do allow compressed SQL files.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02 14:13:47 +13:00
Paul Eggleton
16dc1eb933 dockersetup: allow updating if proxy config set in environment
Don't confuse proxy options being defaulted through from the environment
and the user explicitly specifying them. Also look at no_proxy option.

Fixes https://github.com/intel/clear-linux-dissector-web/issues/13

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02 14:13:47 +13:00
Paul Eggleton
050fbb0878 dockersetup: handle ArgumentTypeError properly
Print ArgumentTypeError (which we are raising) instead of showing a
traceback.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02 14:13:47 +13:00
Paul Eggleton
5e78505dbd dockersetup: add uninstall option
Add the ability to uninstall the application for the user's convenience.
(Note that this does not undo the changes to the configuration, it only
removes the Docker containers and volumes.)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02 14:13:47 +13:00
Paul Eggleton
2c709617bc dockersetup: proxy support fixes
* We need a SOCKS proxy to support fetching from git:// or ssh:// URLs
  within the container, so add an option to specify it
* It's possible for the http and https proxy settings to be the same, so
  set one from the other if only one of them is set.
* If we want to be able to fetch from internal servers inside the proxy
  then we also need a "no-proxy" list, so add support for that.
* It's not unlikely that machines within networks requiring use of a
  proxy for external network access will have all of the proxy settings
  set in the environment, so we can try to pick up the defaults from
  there.
* Ensure that we can switch from proxy to no proxy (when reinstalling)
  which means we always need to edit the config files and ensure the
  proxy options get commented out if we don't want them set.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02 14:13:47 +13:00
Paul Eggleton
b5ba406a07 dockersetup.py: add connectivity check
It's easy to get the proxy settings wrong and not realise until you've
got quite a long way into the process of setting things up. Thus, add a
check where we actually try to fetch various things within the container
environment and fail reasonably early if things aren't working.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-10-02 14:13:47 +13:00
Paul Eggleton
9ce9f7d2d1 dockersetup: fix regression with https enabled
Fix a regression introduced in bfecbb091b.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-31 09:03:46 +12:00
Paul Eggleton
bc6984b582 dockersetup: fix error when printing URL with https enabled
Copy-paste error.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-25 16:25:48 +12:00
Paul Eggleton
31a2e3f043 dockersetup: support encryption/authentication for SMTP
Some SMTP servers are configured to require encrypted connections and
username & password, so provide options to enable those.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
bfecbb091b dockersetup: refactor arguments handling slightly
Just pass back the args object instead of individual arguments to avoid
having to change two places for every trivial argument we add.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
f16ca9da35 dockersetup: show a warning if no email host specified
If no email host has been specified that will prevent certain things
from working, so warn the user if they haven't specified one.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
a8bb9e64bb dockersetup: handle http/https port properly
* Use the port number in the domain so that emails have working URLs
* Don't include the port in the URL printed at the end of installation
  if it's the default

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
0c617f1064 dockersetup: add option to disable adding admin user
If you supply a database file then you may already have ad admin user
defined and thus adding another one is superfluous. Add an option to
make it easy to disable that step.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
d39929b405 dockersetup: do not set site name if a database file is specified
If a database file is specified we probably want to stick with whatever
site name is defined in there and not overwrite it.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
513be9d8ef dockersetup: add option to specify docker-compose project name
Allow specifying the project name (either with a --project-name
command-line option or COMPOSE_PROJECT_NAME) and save it so that
future upgrade operations work properly by default.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
6b2d929261 dockersetup: obtain and use the real project name instead of hardcoding
We need the real project name here otherwise we are mounting something
nonexistent instead of the real volume we want to populate. This fixes
a bug where the static assets were not available (thus the web interface
looks awful) if the directory you installed the application from was not
called "layerindex-web".

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
04979eb03f dockersetup: Use -T option with docker-compose exec
We need to disable the terminal with -T, otherwise it doesn't work
properly.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
a9729491a8 dockersetup: use docker-compose exec instead of docker exec
Good for consistency, but also useful once we remove the hardcoded
container names from docker-compose.yml.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
3cd5976a7a Set permissions for all volumes
We weren't setting ownership for the recently introduced logvolume and
srcvolume volumes (though keep srcvolume optional as I often modify
docker-compose.yml to mount it from a local directory). At the same time
convert the call to shell=False to keep bandit happy.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
c0b85ba29c dockersetup: update nginx-ssl.conf in update mode
If the base SSL configuration has been updated, and we then run
dockersetup.py -u then we want the configuration changes to be reflected
in the web server configuration, however that was not happening because
unlike how the other configuration files are handled, nginx-ssl.conf
gets copied and then we modify the copy due to the nature of the edits
made. To fix it, when in update mode, read in the old values from the
modified configuration file and then copy and modify the base
configuration using those values.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
6647131689 dockersetup: remove redundant code to update non-HTTPS lines in nginx-ssl.conf
The non-HTTPS configuration was removed some time ago from
nginx-ssl.conf, and the code here that would have updated it didn't have
access to http_port so it couldn't have worked anyway, so just remove
it.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
3c844beab2 dockersetup: require python 3.4.3 or later
Prior to this version, HTTPSConnection was not properly secure (as
reported by Bandit).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Terri Oda
7bd189b8e2 dockersetup: quote user input sent to subprocess
Strengthen things a little where shell=True is still being used.

Signed-off-by: Terri Oda <terri.oda@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
9f46418eb3 dockersetup: 'easy' subprocess conversions to not use shell=True
Avoid using shell=True where we can easily do so so that bandit doesn't
complain. Patch courtesy of Terri Oda <terri.oda@intel.com>.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
a823789ab2 dockersetup: require email address
Do not allow proceeding until an email address has been entered, because
we need it later on for creating the superuser account.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
0196eda3f1 dockersetup: enable error report emails
Error reports get emailed to the addresses specified in the ADMINS
setting, but up until now dockersetup wasn't setting this and so the
details of any internal error were not visible anywhere. If we're not
updating, prompt the user for an email address early in the process,
and use this for the ADMINS setting, the superuser account, and Let's
Encrypt (if selected).

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
8d52470654 dockersetup: add option to skip database migrations
If we've messed around with the migrations during development then they
may need to be run carefully (with some fake steps), so provide an
option to update the container and then do the migrations manually
afterwards.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
ee7e5a0564 dockersetup: set site info and email host
In order to have registration/password reset emails working properly we
need to have both the site domain and email host/port set correctly.
Email host/port is easy to set (environment variable read into the
configuration). However, to set the site info, create a simple script
that we can call to do so.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:04 +12:00
Paul Eggleton
604330f41a dockersetup: move superuser account creation to the end
If we move the superuser account creation to the end, it's practical to
Ctrl+C it in case you already have a superuser in the data that got
imported.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:01 +12:00
Paul Eggleton
b99143d2e3 dockersetup: import database dump before running migrations
If the database dump is older than the application, there may be
migrations to run, so we need to run them after importing and not
beforehand.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:31:01 +12:00
Paul Eggleton
24dcd4618a dockersetup: support importing gzip compressed database dumps
Database dumps are simply SQL (i.e. plain text) so they tend to be
both large and easy to compress, so having them gzipped is something
worth supporting.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
e9915dd359 dockersetup: use with open in readfile/writefile
Best practice to ensure files always get closed.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
7a248df9f5 dockersetup: use separate db account with lower privileges
Make the app should use a database account that has only the privileges
it needs rather than the root account for improved security.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
3775efc848 dockersetup: warn if http proxy specified without https
You really need to specify these both together since during updates we
will very likely fetch over both HTTP and HTTPS.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
f50f48cf01 dockersetup: fix auto-generated passwords
* We can't use # because it can result in part of the password being
  trimmed off when it gets put in a configuration file.
* Similarly, our parsing splits on = so it is unwise to allow as a valid
  character.
* Add uppercase characters for a little more randomisation.
2019-07-17 11:30:56 +12:00
Paul Eggleton
99a86d7e0a dockersetup: support update/reinstall mode
Make it easier to do updates and also reinstall from scratch; if the
application is already installed, we force the user be explicit about
which one they want to do so that there is less chance of data loss.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
4370564b91 dockersetup: move HTTPS code to its own function
Refactor out HTTPS configuration code to a new function to give the code
a bit more structure.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
e78c4dae9c docker: enable user/password for RabbitMQ server
Add settings for user/password for the RabbitMQ server and make
dockersetup.py set it up. (The rabbitmq container intrinsically
understands RABBITMQ_DEFAULT_*, and for the sake of consistency I've
reused those variables for the other containers.)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
efe74a451f dockersetup: add letsencrypt support
The application is most likely to be used in an internal setting,
however should someone want to make it accessible externally, add
support for Let's Encrypt certificates for serving via HTTPS.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
c67313593f dockersetup: Show intro message
Apply some branding, improve user friendliness and give the user a
chance to break out if they actually don't want to run the installation.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
cb4955fe0b dockersetup: add HTTPS support and use by default
If we want a minimum level of security we should enable HTTPS. However,
the only practical way we can do that without the user having to do further
infrastructure setup and/or pay a certification authority is to use a
self-signed certificate. Do this by default, and also provide an option
to specify a previously obtained certificate/key pair.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
c717a827d3 docker: use quoted values
Avoids issues with certain characters in values.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
2afc25cb57 dockersetup: tweak portmapping option and display port
* Use default macro in command line help text to avoid duplication
* Make config editing slightly more robust
* Display a message at the end telling the user how to access the
  application based upon the port mapping

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
e1f26deacb dockersetup: add some error checking to the setup script
Ensure we stop processing if there's a failure.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Paul Eggleton
f160927765 dockersetup: formatting fixes
- Move copyright/license statement to top
- Tabs to spaces
- Drop trailing spaces

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00
Amber Elliot
aa07c1b451 docker: add setup script
Adding setup script for docker containers. The script will edit all
necessary configuration files, build and launch all containers, and
do the initial database setup - including populating the database
with data supplied by the user.

Changed docker/README to reflect new setup instructions.

Signed-off-by: Amber Elliot <amber.n.elliot@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2019-07-17 11:30:56 +12:00