To build mysqlclient==2.2.1 we need to add default-libmysqlclient-dev
and pkg-config
Drop conflicting libmariadb-dev-compat
Drop python3-mysqldb to allow requirements.txt to override version
Upgrade pip and setuptools before installing from requirements.txt
as newer versions are required to properly build mysqlclient wheel.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
ubuntu:jammy provides python 3.10.6 which supports current bitbake.
Dropped unneeded python2 packages.
Moved locale generation high to reduce error messages during image building.
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
The current version of django-simple-captcha requires libz and freetype
development libraries.
Refactor to apply. Fix typo. Fix previous mixed use of space vs tab.
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
We have some layers and especially older releases that still support
python2. Add python2 dependencies back to the container so that the
older releases still function.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
* Define LANGUAGE to quiet perl warnings
* Set DEBIAN_FRONTEND=noninteractive before apt call to quiet TERM/Dialog
warnings
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Debian 10 "buster" actually has Django 2.2 available
python2 is now EOL, so drop all usage
Signed-off-by: Tim Orling <ticotimo@gmail.com>
Signed-off-by: Paul Eggleton <bluelightning@bluelightning.org>
* We need a SOCKS proxy to support fetching from git:// or ssh:// URLs
within the container, so add an option to specify it
* It's possible for the http and https proxy settings to be the same, so
set one from the other if only one of them is set.
* If we want to be able to fetch from internal servers inside the proxy
then we also need a "no-proxy" list, so add support for that.
* It's not unlikely that machines within networks requiring use of a
proxy for external network access will have all of the proxy settings
set in the environment, so we can try to pick up the defaults from
there.
* Ensure that we can switch from proxy to no proxy (when reinstalling)
which means we always need to edit the config files and ensure the
proxy options get commented out if we don't want them set.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's easy to get the proxy settings wrong and not realise until you've
got quite a long way into the process of setting things up. Thus, add a
check where we actually try to fetch various things within the container
environment and fail reasonably early if things aren't working.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
There's no reason for this directory to be present in the source tree.
(The rest of the docker files are already excluded via .dockerignore,
but files in docker/ need to be picked up by COPY directives so we can't
do that for these as well.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We really don't want the layers user to be able to write to the
application code / settings in /opt/workdir or /opt/bin within the
layersapp container - only selected directories under /opt. It appears
that we only need to set the ownership on /opt/workdir for now.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We use rpm/rpm2cpio/cpio to deal with the derivative packages, so we
need them in the application container.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Currently, the processing of larger images takes some time, for example
a manifest generated from core-image-sato-sdk seems to take about 45s to
load into the database which is longer than gunicorn's default worker
timeout of 30s. Increase the timeout to 60s as a workaround until we
optimise the import.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The upstream version checking used by the RRS needs wget to fetch
http/https URLs, so add it into the container.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We can always deploy these files since the default versions have all the
settings commented out - save proxy users from needing to uncomment
these (it's annoying if you miss doing so).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The home directory path in the commented line here had not been updated
when we switched to running the app as a normal user ("layers"), so do
that now. Additionally we need to copy the file *after* we create the
user, otherwise the home directory will be created by the COPY operation
first and will thus end up being owned by root which we don't want.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Docker commits changes after each RUN stage and commands that delete
files must part of the sane RUN command to actually reduce image size.
Debian stretch contains pip 9.0.3 and the upgrade to 10 caused pip
install to fail.
Signed-off-by: Konrad Scherer <Konrad.Scherer@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Put NGINX, Celery, and RabbitMQ into their own separate containers
* Use a docker network instead of the deprecated --link
* Allow for collecting the static files properly
* Create a copy of settings.py specifically for the docker setup. This
will need to be kept in sync with the main example settings.py, but
it avoids the user having to edit it too much.
* Add optional SSL configuration using letsencrypt certificate
* Create some volumes for static files / fetched repos
* Add some more helpful setup instructions
Largely based upon work by Michael Halstead <michael@yoctoproject.org>.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's best practice to run as a normal account, so create one and use it.
Thanks to Michael Halstead <michael@yoctoproject.org> for this.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Use plain debian instead of buildpack-deps as a base (and install just
the additional packages we need on top)
* Purge the development packages after we're done with them (they aren't
used at runtime)
* Run apt-get clean to drop package cache files
Thanks to Konrad Scherer <konrad.scherer@windriver.com> for this idea.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If we split dealing with requirements.txt out before we copy the entire
source tree then docker can keep that part cached more often when we're
making changes to the code and thus rebuilds are faster. Thanks to
Konrad Scherer <konrad.scherer@windriver.com> for this idea.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Python 3 is a lot more sensitive to locale, plus we will definitely be
dealing with non-ASCII names and email addresses, so we need to get this
right.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Python 2 is only supported for indexing layers in older branches - the
web interface should be running under Python 3.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Use LABEL instead of the deprecated MAINTAINER
* Use COPY instead of ADD. There's no difference in operation here, it's
just that we don't need the extra magic that ADD provides and best
practice is to use COPY in that case.
* Fix the mkdir /opt/workdir line - it wasn't being run since it was a
CMD not RUN, and thus was overridden by a later CMD directive.
* Drop the CMD line to run celery - having more than one CMD directive
does not work. We'll launch a separate container to run it instead.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
This patch adds asynchronous task execution using a Celery backend
and RabbitMQ task queue, so that the layer submission process to
proceed even in the event that sending the notification email fails,
and establishing an asynchronous execution mechanism that we can use
in the future e.g. for triggering parse operations from the web UI.
This pertains to bug 11197:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=11197
It updates the README to reflect the installation and configuration
of a basic RabbitMQ setup, adds a 'tasks.py' file to contain task
definitions, updates the 'edit_layer_view' function to send
emails to administrators about new and updated layers asynchronously,
modifies the 'settings.py' to include a default configuration
for a RabbitMQ connection, and updates the Dockerfile to start a Celery
worker alongside the Gunicorn daemon.
Fixes [YOCTO #11197].
Signed-off-by: Diana Thayer <garbados@gmail.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Replicate production setup in Docker containers
[YOCTO #7575]
Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
