Good for consistency, but also useful once we remove the hardcoded
container names from docker-compose.yml.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
In testing, lockouts are happening far too frequently. Thus make the
following changes to Axes configuration:
* Lock out only after 4 incorrect attempts, not 3
* Set a 1-hour cooling-off period, after which the lockout will be
removed
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Most Linux distributions do not have "recipes", they have "packages" so
use the correct term (as we are in other places).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
There's no reason for this directory to be present in the source tree.
(The rest of the docker files are already excluded via .dockerignore,
but files in docker/ need to be picked up by COPY directives so we can't
do that for these as well.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
When running a task (e.g. importing other distribution data), the web
frontend polls to get task output every second. In the view handling
this request we check to see if the underlying Celery task is finished
by getting the AsyncResult and then calling result.ready().
Unfortunately that latter call seems to be failing some of the time in
the development Docker setup that I am using, throwing a
ConnectionResetError. Because the polling is regular it doesn't really
matter if this fails as there'll be another chance on the next poll, so
just allow the call to fail silently. (It pains me to put in workarounds
like this but at the moment I can't determine the real cause of the
issue, and this stops the stream of ultimately useless error report
emails that I'm getting as an admin of the development instance).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Calling communicate() blocks the process; but since we're writing the
output directly to a file and not sending any input we don't actually
need to call communicate(), just poll() (so that we can check
the returncode attribute). Subjectively this does appear to improve
performance although it has not fixed the ConnectionResetError issues.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If an update task's output contains a single quote for example (') then
this will be escaped to an HTML entity before it gets sent as a
response. However, that means that the length of the data in the response
will be greater than the length of the original data, resulting in
characters getting missed out when we read the next chunk - so we
can't use the escaped length to set the next position to read from. The
easiest thing to do is have the Django view send us the actual position
we're at and then we don't have to try to calculate it on the JS side.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
There's no particular reason these fields should be read-only (unlike
the fields picked up from Recipe, they aren't derived from the recipe
itself.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If you had a cover comment set on a comparison recipe record then you
received an internal server error when that recipe got rendered in the
search results. Use the current correct URL name to fix it.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We really don't want the layers user to be able to write to the
application code / settings in /opt/workdir or /opt/bin within the
layersapp container - only selected directories under /opt. It appears
that we only need to set the ownership on /opt/workdir for now.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We weren't setting ownership for the recently introduced logvolume and
srcvolume volumes (though keep srcvolume optional as I often modify
docker-compose.yml to mount it from a local directory). At the same time
convert the call to shell=False to keep bandit happy.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Update to the latest versions, with the exception of django-reversion
which I suspect will trigger some regressions; plus django-axes which
for some reason pip did not want to fetch 5.x versions of within the
container.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If the base SSL configuration has been updated, and we then run
dockersetup.py -u then we want the configuration changes to be reflected
in the web server configuration, however that was not happening because
unlike how the other configuration files are handled, nginx-ssl.conf
gets copied and then we modify the copy due to the nature of the edits
made. To fix it, when in update mode, read in the old values from the
modified configuration file and then copy and modify the base
configuration using those values.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
The non-HTTPS configuration was removed some time ago from
nginx-ssl.conf, and the code here that would have updated it didn't have
access to http_port so it couldn't have worked anyway, so just remove
it.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Strengthen things a little where shell=True is still being used.
(For the most part input that passes through here would already be
sanitised, but let's be careful anyway).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Strengthen things a little where shell=True is still being used.
Signed-off-by: Terri Oda <terri.oda@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Avoid using shell=True where we can easily do so so that bandit doesn't
complain. Patch courtesy of Terri Oda <terri.oda@intel.com>.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
I have been seeing repeated emailed errors from Django reporting
"ConnectionResetError: [Errno 104] Connection reset by peer" in the call
to get task status i.e:
File "/opt/layerindex/layerindex/views.py" in task_log_view
1572. if result.ready():
Digging around this seems to be some sort of known bug:
https://github.com/celery/celery/issues/4817https://github.com/celery/celery/issues/4980
The workaround suggested is to disable the broker heartbeat, so try
that in order to avoid the errors.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
I've seen "MySQL server has gone away" errors in the logs, and this time
it isn't the timeout; one of the suggested fixes is to increase the max
packet size (and I have also seen errors in the mariadb logs indicating
that the max has been exceeded).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Limit to TLS 1.2 and the most secure ciphers
* Set some other recommended ssl_* settings
* Disable gzip
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Once we get to the package definitions we don't want to be gathering
field values (or we'll just get the last package's summary value for the
overall summary for example).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Record the configure script options when importing recipe / package
information so we can display them.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
When we're running commands as part of a task that use carriage returns
(\r) to update the currently displayed line, we want to see the same
output in the web representation, so if we encounter a \r in the output
we need to look back to the last newline, truncate to that and then
start appending.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We need the task log dir to be shared between the app and celery
containers, so mount a volume and point both of them into it.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Allow passing a list to run_update_command() instead of a string which
will run the command directly instead of within the shell.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We use rpm/rpm2cpio/cpio to deal with the derivative packages, so we
need them in the application container.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Without this, celery tasks were attempting to connect with the default
user (root) and failing because the passwords are different.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
There's a little more to patch handling in spec files than "patchX:"
fields - the patch actually has to be applied in the %prep section as
well, though this can take one of several forms (%autosetup / %autopatch
which apply all patches, individual %patchX directives, or even direct
application (rare). There's also the matter of the striplevel (-p option
to the patch command). Add fields to record whether or not a patch is
applied and the striplevel.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
I came across several spec files in Clear Linux that has a PNG file in
the description value. No idea how that got there, but avoid choking and
ignore the garbage description if this happens.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
We don't actually need this for anything at the moment, but it would be
useful if we get to the point where we need to access imported files
within the application after the import process (e.g. to compare
patches).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Do not allow proceeding until an email address has been entered, because
we need it later on for creating the superuser account.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If a source points to a local file, get the sha256sum of it and save it
into the field we just added.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Allow us to record a checksum for local sources (and potentially remote
ones if it is recorded in the recipe). This just adds the field, but
doesn't populate it.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Make it easier to import different versions of the other distro by not
requiring you to manually create the layer and branch.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
If for some reason we do not find any spec files in the specified
directory, error out instead of marking all packages as deleted. (This
can happen if the wrong directory is specified.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
* Make patch / user fields non-editable for existing records (patch
mostly because the list is huge for a drop-down)
* User must be a superuser or the dispositioner to edit
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Add the ability to mark each patch with a disposition indicating whether
the patch is interesting or not.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Editing on this page is done by clicking through to another page, so
it should be a DetailView rather than an UpdateView and shouldn't have a
form class set.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Currently, the processing of larger images takes some time, for example
a manifest generated from core-image-sato-sdk seems to take about 45s to
load into the database which is longer than gunicorn's default worker
timeout of 30s. Increase the timeout to 60s as a workaround until we
optimise the import.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
It's likely an image manifest tarball will be more than 1mb in size;
increase it to 16mb to handle where a large number of patches are
included in an image containing a lot of packages from different
recipes.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
