To build mysqlclient==2.2.1 we need to add default-libmysqlclient-dev
and pkg-config
Drop conflicting libmariadb-dev-compat
Drop python3-mysqldb to allow requirements.txt to override version
Upgrade pip and setuptools before installing from requirements.txt
as newer versions are required to properly build mysqlclient wheel.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Some layers now have one branch with many supported
LAYERSERIES_COMPAT. If this branch name does not match
one of the stable releases, LayerBranches might not
have been created. When actual_branch is set, it is
only set in a LayerBranch object. We previously
could not update (create) a stable branch with actual_branch
except manually in the admin interface.
Add --force-create option to be used in conjunction with
--actual-branch (which already requires --branch) in the
update.py script. This tells the script to ignore the
fact that no layerbranch exists already.
Add --actual-branch to update_layer.py so that we can create
(and more importantly checkout) an actual_branch for the
given stable --branch.
Update utils.py to allow checking out of actual_branch when
a LayerBranch does not yet exist.
While we are at it, ensure that any Branch that is marked
as no update will be skipped even with --force-create. The
main reason that a Branch has updates disabled is because the
bitbake or python syntax has changed enough to cause exceptions.
This script can now be run with:
./layerindex/update.py \
--layer meta-weird-one \
--branch kirkstone \
--actual-branch=nonstandard \
--force-create
Which will attempt to create a meta-weird-one:kirkstone layerbranch
checked out at the 'nonstandard' branch from that layer's git repo.
This allows layerindex admins to at least populate the database
without tedious creation of layerbranches in the admin interface.
Helps make the "branch mapping" actually work and be useful:
[YOCTO #8008]
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
* Add an "Update Layer" button to the layer detail view.
- This allows a user that is a member of is_staff to trigger
an update of the current layer (for the current branch)
* Add an "Update Layer" button to the reviewdetail view
- This allows a user that is a member of is_staff and has
publish_layer permissions to trigger an update attempt
of the layer under review (even in the un-published state)
* The update is run as a task with Celery
NOTE:
You must have the RABBITMQ_ and DATABASE_ credentials set
correctly in the docker/settings.py file or set via
environment variables or you will get authentication errors
talking to layersdb or layersrabbit containers.
[YOCTO #12484]
layerindex/views.py: add update_layer_view
layerindex/urls.py: add update_layer_view
layerindex/urls_branch.py: add update_layer_view
templates/layerindex/reviewdetail.html: add Update Layer button
templates/layerindex/detail.html: add Update Layer button
templates/layerindex: add updatelayer.html
TODO:
While the update is happening, the AJAX rendering of the
update.log is showing the b'' characters and not adding
any new lines. If you go back to the same task view
afterwards, the log is rendered as expected.
TODO:
After the update is completed, it would be nice to have a
button to return you to the page from where you called the
"Update Layer".
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
For layers which do not follow standard branch names (including
the inclusive naming move away from "master" to "main") we have
the actual_branch field set in a LayerBranch object. Previously
this was only exposed via the admin interface.
Allow layer maintainers (including upon submitting a new layer)
to set the 'Actual branch' in the web UI.
Add a check to make sure the actual_branch is a valid branch
name using 'git check-ref-format --branch <actual_branch>'
since we are not using full refs.
[YOCTO #8008]
NOTE:
Only existing LayerBranches will be editable. A new layer
can be submitted with a different branch for "master", but
only the "master" LayerBranch will be created.
Further changes to the update.py script will be needed to
make creation of new stable branches with an actual_branch
possible.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
The '--staging' argument to certbot has now been changed
to '--test-cert'. We previously only allowed using the
dockersetup.py tool to create Staging environment certs,
which are still marked as invalid by browsers. Add a
'--letsencrypt-production' knob to allow for valid, trusted
certs to be created. If they already exist in the workspace
and have not expired, re-use them (to avoid hitting rate
limits). Continue to '--force-renewal' for staging certs.
NOTE:
If you have previously created staging certs in your
workspace, you will want to clean docker/certs before
creating production certs for the same domain. Certbot
will not overwrite those staging certs and the newly
created ones will not be in the path passed in by
dockersetup.py.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Within the layersapp container, if you try to run layerindex/update.py:
File "/opt/layerindex/settings.py", line 280, in <module>
RABBIT_BROKER = 'amqp://' + os.getenv('RABBITMQ_DEFAULT_USER') + ':' + os.getenv('RABBITMQ_DEFAULT_PASS') + '@layersrabbit:5672/'
TypeError: can only concatenate str (not "NoneType") to str
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
The Branch Comparison view would throw an error because of a parser_view instead of parse_view typo.
[YOCTO #15332]
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
File "/opt/layerindex/layerindex/update.py", line 525
failed_layers[branch].append('%s: Failed to add since LAYERDEPENDS [%s ...] is not
SyntaxError: '(' was never closed
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Make errors like this more useful:
ERROR: Issues found on branch nanbield:
meta-luneos: Failed to add since LAYERDEPENDS is not satisfied
meta-luneui: Failed to add since LAYERDEPENDS is not satisfied
In this case, meta-luneos depends on meta-luneui, but we cannot create a
'nanbield' layer branch, because meta-luneui LAYERDEPENDS on qt6-layer
collection, which currently has no 'nanbield' branch.
Use next(iter(value['deps'])) to get the first element in the OrderedDict
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Currently, the error message is something like:
ERROR: conf/layer.conf not found for layer meta-doom - is subdirectory set correctly?
This is because in this case meta-doom has a 'langdale' branch without
a conf/layer.conf
Another example is:
ERROR: conf/layer.conf not found for layer meta-st-stm32mp - is subdirectory set correctly?
This is because meta-st-stm32mp has a master branch, but it only contains
a README.md telling you to use the stable branches (e.g. nanbield).
Make it more obvious what the source of the error is by also displaying
the branch that was being attempted.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
The regex for Upstream-Status had (\w+), but this
does not include hyphen/dash. Explicitly add it to
the pattern.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
For both meta-poky/meta-yocto and meta-xilinx/meta-xilinx-core
we have a situation where the "collections" stayed the same
("yocto" and "xilinx" respectively) but the layer/layerbranch
changed. Without the "branch" argument to get_dependency_layers,
we were always defaulting to the older layer which first
defined the "collection".
Instead, add an option to use "branch" to filter on the expected
LayerBranch object. Keep the old behavior just in case someone
depends upon it.
[YOCTO #15221]
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
We have never been checking out dependent layers at the same
release/branch. With the introduction of 'addpylib', this
became obvious due to parsing errors.
Ensure that known LayerDependency objects are checked out at
the expected branch/release. Since openembedded-core has already
been handled elsewhere, we skip it.
[YOCTO #15236]
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
When a new release is run for the first time, no LayerBranch exists
yet and the update_enabled cannot be queried. The unintended result
is that all layers are skipped. Move the update_enabled check into
the code path where the layerbranch already exists.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Add a script which can either mark one --layer --branch or
use data in a --from-file to mark multiple layer:branch objects
as Yocto Project Compatible.
The --from-file is a json file is compatible with or which can be
generated by:
yocto-autobuilder-helper/scripts/list-yp-compatible-layers.py
[YOCTO #15093]
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Especially since LTS branches have become more popular as the
only active branch, we need to be able to turn off updates on
a LayerBranch basis rather than the current Layer or Branch
heavier hammers.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
To make it easier to quickly see what version we are using, add
the version to the filenames.
Also, use minified flavor in templates/base.html
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
In addition to ALLOWED_HOSTS, we now must have CSRF_TRUSTED_ORIGINS defined.
This variable requires the scheme (http:// or https://).
Like ALLOWED_HOSTS, CSRF_TRUSTED_ORIGINS is a list of strings, with one
entry for each host which is trusted for POST requests.
https://docs.djangoproject.com/en/4.2/ref/settings/#csrf-trusted-origins
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Wrap the is_pagination usage of bootstrap_pagination with comment/endcomment
to avoid incompatible module.
We will need to re-write the pagination code.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
https://raw.githubusercontent.com/mbi/django-simple-captcha/master/CHANGES
Version 0.5.20
--------------
* Still support Django 3.2 (#222, thanks @petrklus)
Version 0.5.19
--------------
* SECURITY ISSUE: reset the random seed after an image was generated (#221, thanks @ibuler)
Version 0.5.18
--------------
* Fix some typos in documentation (#210, thanks @stweil)
* Test against Django 4.2
* Stopped testing Django < 3.2
* BaseCaptchaTextInput should set autocomplete=off on the hashkey HiddenInput (#201, thanks @eerotal)
* Test against Django 4.2a
* Fix some deprecation warnings in Pillow 9.2+
* Removed old unused conditional imports
* Format code with pre-commit and black
Version 0.5.17
--------------
* English translation created from Spanish ones (#209, thanks @tpazderka)
Version 0.5.16
--------------
* Adds a migration missing from 0.5.15 (#208, thanks @atodorov)
Version 0.5.15
--------------
* Updated test matrix, drop tests against Python3.6, test against Python3.9
* Remove dependency on six
* Test against Django 4.0a1
* Test with Python 3.10 (Django 3.2 and Django 4.0)
* Remove warning for django 3.2 (#206, thanks @MiStErLu)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
zstd compression and decompression are significantly faster than gzip.
zstd is also "splittable" and is streaming/real-time friendly.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
The url returned by the reverse lookup ends in /submit/thanks/ but
the regex was improperly searching for a string without the terminating "/".
Thank you to Pawel Zalewski for reporting the 404 error.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
bump gitpython and pillow to version with fixes for these CVEs
pillow: CVE-2022-22817 CVE-2022-24303 CVE-2022-45198
gitpython: CVE-2022-24439
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Bitbake's api has been changed via:
afb8478d3 parse: Add support for addpylib conf file directive and BB_GLOBAL_PYMODULES
The conf file won't be parsed without baseconfig=True:
bb.parse.ParseError: ParseError at /path/to/oe-core/meta/conf/layer.conf:132: unparsed line: 'addpylib ${LAYERDIR}/lib oe'
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
ubuntu:jammy provides python 3.10.6 which supports current bitbake.
Dropped unneeded python2 packages.
Moved locale generation high to reduce error messages during image building.
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
The runcmd() would print an "ERROR" on failure which causes confusion since
the failure is expected on old branches, so subprocess.getstatusoutput to fix
the problem.
Minor rewording.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
The following commit has added on_delete=models.CASCADE to the ForeignKey:
===
commit 2d526f9b0d
Author: Amber Elliot <amber.n.elliot@intel.com>
Date: Thu Jun 25 15:57:34 2020 -0700
Updating models and imports for Django 2.2 upgrade.
===
The on_delete=models.CASCADE will remove the objects which reference the
removed objects, so there will be errors when removing a LayerBranch
with admin permissions:
===
Cannot delete layer branch
Deleting the selected layer branch would result in deleting related objects, but your account doesn't have permission to delete the following types of objects:
Append
distro
===
This is because has_delete_permission() always return False, remove it to use
Django's implementation to fix the problem.
Minor rewording.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
The commit 38e6288c7d has added oe-core to
BBLAYERS by default, and then there will be multiple BBFILE_COLLECTIONS which
causes update.py failure to update layers (most of the dependencies or
recommends are missing), use get_layer_var() to get BBFILE_COLLECTIONS to fix
the problem since it can ignore the extra oe-core from BBFILE_COLLECTIONS.
Fix minor typos. Minor rewording.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
