272f0eded2
* Put NGINX, Celery, and RabbitMQ into their own separate containers * Use a docker network instead of the deprecated --link * Allow for collecting the static files properly * Create a copy of settings.py specifically for the docker setup. This will need to be kept in sync with the main example settings.py, but it avoids the user having to edit it too much. * Add optional SSL configuration using letsencrypt certificate * Create some volumes for static files / fetched repos * Add some more helpful setup instructions Largely based upon work by Michael Halstead <michael@yoctoproject.org>. Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
5.5 KiB
Layerindex example docker instructions
This is set up to make a cluster of 5 containers:
- layersapp: the application
- layersdb: the database
- layersweb: NGINX web server (as a proxy and for serving static content)
- layerscelery: Celery (for running background jobs)
- layersrabbit: RabbitMQ (required by Celery)
First, find and replace layers.openembedded.org below with your hostname
You'll probably also want to replace the database password "testingpw".
If you are using a version of Docker older than 17.06 then you'll need to replace --mount src=,dst= with -v :
If you want to change any of the application configuration, edit docker/settings.py as desired.
Some settings have been set so that values can be passed in via environment variables.
You will definitely need to set SECRET_KEY.
If you are on a network that requires a proxy to get out to the internet, then you'll need to:
- Uncomment several lines in Dockerfile (search for "proxy")
- Edit docker/.gitconfig and docker/git-proxy
Build the main container from the root of the repo.
docker build -t halstead/layerindex-app .
Build the static web server container
(for SSL, first move docker/nginx-ssl.conf to docker/nginx.conf and edit as needed.)
docker build -t halstead/layerindex-web -f Dockerfile.web .
Add a network for our containers
docker network create layerindex
Start a database server - here we use MariaDB, though you can obviously use something else and change docker/settings.py as appropriate
run one of the following.
To use an existing dump run the following and wait for startup and import
docker run --detach --name layersdb --network layerindex --mount src=layers-database-dump.sql,dst=/docker-entrypoint-initdb.d/layerdb.sql -e MYSQL_ROOT_PASSWORD=testingpw mariadb:10.2 --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
Or to start fresh
docker run --detach --name layersdb --network layerindex -e MYSQL_DATABASE=layersdb -e MYSQL_ROOT_PASSWORD=testingpw mariadb:10.2 --character-set-server=utf8mb4 --collation-server=utf8mb4_unicode_ci
Start the RabbitMQ container
docker run --detach --network layerindex --name layersrabbit rabbitmq:alpine
Start the Celery container
docker run --detach --network layerindex --name layerscelery -e DATABASE_PASSWORD=testingpw -e DATABASE_HOST=layersdb halstead/layerindex-app /usr/local/bin/celery -A layerindex.tasks worker --loglevel=info --workdir=/opt/layerindex
Apply any pending layerindex migrations / initialize the database
docker run --rm --network layerindex --env DATABASE_HOST=layersdb --env DATABASE_PASSWORD=testingpw halstead/layerindex-app /opt/migrate.sh
For a fresh database, create an admin account
docker run --rm -it --network layerindex --env DATABASE_HOST=layersdb --env DATABASE_PASSWORD=testingpw halstead/layerindex-app /opt/layerindex/manage.py createsuperuser
Start the layerindex application
docker run --detach --network layerindex --name layersapp --hostname layers.openembedded.org -e DATABASE_PASSWORD=testingpw -e DATABASE_HOST=layersdb halstead/layerindex-app
Create a volume for static assets
docker volume create layersstatic
Set the volume permissions using debian:stretch since we recently fetched it
docker run --mount src=layersstatic,dst=/usr/share/nginx/html debian:stretch chown 500 /usr/share/nginx/html
Generate static assets. Run this command again to regenerate at any time (when static assets in the code are updated)
docker run --env STATIC_ROOT=/usr/share/nginx/html -ti --rm --network layerindex --hostname layers.openembedded.org --name generatestatic --mount src=layersstatic,dst=/usr/share/nginx/html --env DATABASE_HOST=layersdb --env DATABASE_PASSWORD=testingpw halstead/layerindex-app python3 /opt/layerindex/manage.py collectstatic
Start the reverse proxy
run one of the following:
A) for local/test use forward port 8080:
docker run --detach --network layerindex -p 8080:80 --mount src=layersstatic,dst=/usr/share/nginx/html --name layersweb --hostname layers.openembedded.org halstead/layerindex-web
B) with SSL for production:
Make sure your DNS is setup and then run the following to get the certs
docker run -it --rm -p 80:80 -p 443:443 --name certbot --mount src=layerscerts,dst=/etc/letsencrypt --mount src=certbotvar,dst=/var/lib/letsencrypt certbot/certbot certonly #renew
then start the proxy with ssl
docker run --detach --network layerindex -p 80:80 -p 443:443 --mount src=layersstatic,dst=/usr/share/nginx/html --mount src=layerscerts,dst=/etc/letsencrypt --name layersweb --hostname layers.openembedded.org halstead/layerindex-web
Create a workdir to prevent downloading repos fresh each time
docker volume create update-workdir
Set the volume permissions using debian:stretch since we recently fetched it
docker run --mount src=update-workdir,dst=/opt/workdir debian:stretch chown 500 /opt/workdir
Run the layer updates
docker run --rm --network layerindex --hostname updatelayers.openembedded.org --name updatelayers-throwaway --mount src=update-workdir,dst=/opt/workdir --env DATABASE_HOST=layersdb --env DATABASE_PASSWORD=testingpw halstead/layerindex-app python3 /opt/layerindex/layerindex/update.py
Or do a full refresh
docker run --rm --network layerindex --hostname updatelayers.openembedded.org --name updatelayers-throwaway --mount src=update-workdir,dst=/opt/workdir --env DATABASE_HOST=layersdb --env DATABASE_PASSWORD=testingpw halstead/layerindex-app python3 /opt/layerindex/layerindex/update.py -r