Commit Graph

1098 Commits

Author SHA1 Message Date
Tim Orling
9961b58146 update.py: display first unsatisfied LAYERDEPENDS
Make errors like this more useful:

ERROR: Issues found on branch nanbield:
    meta-luneos: Failed to add since LAYERDEPENDS is not satisfied
    meta-luneui: Failed to add since LAYERDEPENDS is not satisfied

In this case, meta-luneos depends on meta-luneui, but we cannot create a
'nanbield' layer branch, because meta-luneui LAYERDEPENDS on qt6-layer
collection, which currently has no 'nanbield' branch.

Use next(iter(value['deps'])) to get the first element in the OrderedDict

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-11-17 12:15:49 -08:00
Tim Orling
cbd9cf632e update.py: show branch if missing conf/layer.conf
Currently, the error message is something like:
ERROR: conf/layer.conf not found for layer meta-doom - is subdirectory set correctly?

This is because in this case meta-doom has a 'langdale' branch without
a conf/layer.conf

Another example is:
ERROR: conf/layer.conf not found for layer meta-st-stm32mp - is subdirectory set correctly?

This is because meta-st-stm32mp has a master branch, but it only contains
a README.md telling you to use the stable branches (e.g. nanbield).

Make it more obvious what the source of the error is by also displaying
the branch that was being attempted.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-11-17 12:15:49 -08:00
Tim Orling
0bcaf71deb layerindex/models.py: fix Inactive-Upstream check
The regex for Upstream-Status had (\w+), but this
does not include hyphen/dash. Explicitly add it to
the pattern.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-11-17 12:15:49 -08:00
Tim Orling
dd38760c18 utils.py: get dependent layers at same branch
For both meta-poky/meta-yocto and meta-xilinx/meta-xilinx-core
we have a situation where the "collections" stayed the same
("yocto" and "xilinx" respectively) but the layer/layerbranch
changed. Without the "branch" argument to get_dependency_layers,
we were always defaulting to the older layer which first
defined the "collection".

Instead, add an option to use "branch" to filter on the expected
LayerBranch object. Keep the old behavior just in case someone
depends upon it.

[YOCTO #15221]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-11-16 07:09:09 -08:00
Tim Orling
94d8f3f20e update_layer.py: checkout dep layers as well
We have never been checking out dependent layers at the same
release/branch. With the introduction of 'addpylib', this
became obvious due to parsing errors.

Ensure that known LayerDependency objects are checked out at
the expected branch/release. Since openembedded-core has already
been handled elsewhere, we skip it.

[YOCTO #15236]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-11-16 07:09:09 -08:00
Tim Orling
b3bfbd1479 update.py: fix updates_enabled for newbranch
When a new release is run for the first time, no LayerBranch exists
yet and the update_enabled cannot be queried. The unintended result
is that all layers are skipped. Move the update_enabled check into
the code path where the layerbranch already exists.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-11-16 07:09:09 -08:00
Tim Orling
3dc125d9c1 layerindex/tools: add mark_yp_compatible_layers.py
Add a script which can either mark one --layer --branch or
use data in a --from-file to mark multiple layer:branch objects
as Yocto Project Compatible.

The --from-file is a json file is compatible with or which can be
generated by:

yocto-autobuilder-helper/scripts/list-yp-compatible-layers.py

[YOCTO #15093]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-11-16 07:09:09 -08:00
Tim Orling
9fdcfad873 Add SECURITY.md
Add the generic Yocto Project tools SECURITY.md as a first step.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-11-16 07:09:00 -08:00
Tim Orling
22ee6617b4 bootstrap: fix typo in minified file name
Fixes dropdown menus so they are functional again.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-19 14:01:47 -07:00
Tim Orling
00a430c666 update.py: skip unless layerbranch.updates_enabled
If layerbranch.updates_enabled is false, skip the update.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-19 10:32:58 -07:00
Tim Orling
3e121980bd models.py: add updates_enabled to LayerBranch
Especially since LTS branches have become more popular as the
only active branch, we need to be able to turn off updates on
a LayerBranch basis rather than the current Layer or Branch
heavier hammers.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-19 10:32:58 -07:00
Tim Orling
99c97660fb Chart.js: upgrade from 2.9.3 to 2.9.4
https://github.com/chartjs/Chart.js/releases/tag/v2.9.4

CVE: CVE-2020-7746

https://nvd.nist.gov/vuln/detail/CVE-2020-7746

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-16 17:44:40 -07:00
Tim Orling
28f06c5ac9 Chart.js: add version to filenames
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-16 17:44:40 -07:00
Tim Orling
2566208185 bootstrap: add version to filenames
To make it easier to quickly see what version we are using, add
the version to the filenames.

Also, use minified flavor in templates/base.html

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-16 17:40:01 -07:00
Tim Orling
79172de651 jquery: upgrade 3.4.1 -> 3.7.1
Use minified version in templates/base.html

Fixes some vulernabilities:
https://nvd.nist.gov/vuln/search/results?adv_search=true&isCpeNameSearch=true&query=cpe%3A2.3%3Aa%3Ajquery%3Ajquery%3A3.4.1%3A*%3A*%3A*%3A*%3Anode.js%3A*%3A*

CVE: CVE-2020-23064
CVE: CVE-2020-11022
CVE: CVE-2020-11023

For full changelog see:
https://github.com/jquery/jquery/compare/3.4.1...3.7.1

License-Update: Copyright OpenJS Foundation (from JS Foundation)

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-16 17:37:37 -07:00
Tim Orling
378ae0d018 docker/settings.py: add CSRF_TRUSTED_ORIGINS
In addition to ALLOWED_HOSTS, we now must have CSRF_TRUSTED_ORIGINS defined.
This variable requires the scheme (http:// or https://).

Like ALLOWED_HOSTS, CSRF_TRUSTED_ORIGINS is a list of strings, with one
entry for each host which is trusted for POST requests.

https://docs.djangoproject.com/en/4.2/ref/settings/#csrf-trusted-origins

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-06 08:43:18 -07:00
Tim Orling
7dcdd0e2a1 password_validation.py: fix deprecated ugettext
Use django.utils.translation.gettext instead

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-05 21:04:41 -07:00
Tim Orling
7989e61fae requirements.txt: update dependencies
Use results of:
pipenv install
pipenv update
pip freeze > requirements-freeze.txt

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-05 20:51:39 -07:00
Tim Orling
eceb742205 requirements.txt: upgrade all to latest
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-05 20:49:35 -07:00
Tim Orling
fcce3cdcde Dockerfile: drop python3-pil
This prevents upgrading Pillow in requirements.txt

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-05 20:12:27 -07:00
Tim Orling
abef2b6a19 Refactor usage of django.conf.urls
django.conf.urls.url() was removed in Django 4.0:
https://docs.djangoproject.com/en/4.2/releases/4.0/#features-removed-in-4-0

Replace all usage with django.urls.re_path()
Replace all django.conf.urls imports with equivalent django.urls modules

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-05 20:10:09 -07:00
Tim Orling
bd58fbe7df bootstrap_pagination: disable via comment
Wrap the is_pagination usage of bootstrap_pagination with comment/endcomment
to avoid incompatible module.

We will need to re-write the pagination code.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-05 20:09:12 -07:00
Tim Orling
e86a2c2610 Drop django-bootstrap-pagination: incompatible
Upstream django-bootstrap-pagination is no longer maintained and is
incompatible with Django 4.x.

https://github.com/staticdev/django-pagination-bootstrap was archived on June 26, 2023.

ugettext was removed in Django 4.0:
https://docs.djangoproject.com/en/4.2/releases/4.0/#features-removed-in-4-0

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-05 20:06:21 -07:00
Tim Orling
bb79b3885e docker-compose.yml: bump mariadb from 10.3 to 10.4
Django 4.2 requires a minimum of 10.4

https://docs.djangoproject.com/en/4.2/releases/4.2/#dropped-support-for-mariadb-10-3

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-05 20:04:57 -07:00
Tim Orling
082f5aab79 requirements.txt: upgrade Django 3.2 -> 4.2
Update to 4.2 LTS release

https://docs.djangoproject.com/en/4.2/releases/4.2/
https://docs.djangoproject.com/en/4.1/releases/4.1/
https://docs.djangoproject.com/en/4.0/releases/4.0/

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-05 20:04:38 -07:00
Tim Orling
b4d9c92d5d requirements: bump django-simple-captcha to 0.5.20
https://raw.githubusercontent.com/mbi/django-simple-captcha/master/CHANGES

Version 0.5.20
--------------
* Still support Django 3.2 (#222, thanks @petrklus)

Version 0.5.19
--------------
* SECURITY ISSUE: reset the random seed after an image was generated (#221, thanks @ibuler)

Version 0.5.18
--------------
* Fix some typos in documentation (#210, thanks @stweil)
* Test against Django 4.2
* Stopped testing Django < 3.2
* BaseCaptchaTextInput should set autocomplete=off on the hashkey HiddenInput (#201, thanks @eerotal)
* Test against Django 4.2a
* Fix some deprecation warnings in Pillow 9.2+
* Removed old unused conditional imports
* Format code with pre-commit and black

Version 0.5.17
--------------
* English translation created from Spanish ones (#209, thanks @tpazderka)

Version 0.5.16
--------------
* Adds a migration missing from 0.5.15 (#208, thanks @atodorov)

Version 0.5.15
--------------
* Updated test matrix, drop tests against Python3.6, test against Python3.9
* Remove dependency on six
* Test against Django 4.0a1
* Test with Python 3.10 (Django 3.2 and Django 4.0)
* Remove warning for django 3.2 (#206, thanks @MiStErLu)

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-05 19:28:35 -07:00
Tim Orling
d020a5c04e dockersetup.py: databasefile zstd compression
zstd compression and decompression are significantly faster than gzip.
zstd is also "splittable" and is streaming/real-time friendly.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-05 18:33:49 -07:00
dependabot[bot]
d447eb7e45 build(deps): bump pillow from 9.3.0 to 10.0.1
Bumps [pillow](https://github.com/python-pillow/Pillow) from 9.3.0 to 10.0.1.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](https://github.com/python-pillow/Pillow/compare/9.3.0...10.0.1)

---
updated-dependencies:
- dependency-name: pillow
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-10-05 17:57:38 -07:00
Tim Orling
a2e23f1e20 build(deps): bump gitpython from 3.1.32 to 3.1.37
This release contains another security fix that further improves validation
of symbolic references and thus properly fixes this CVE:
https://github.com/advisories/GHSA-cwvm-v4w8-q58c (CVE-2023-41040).

https://github.com/gitpython-developers/GitPython/blob/main/doc/source/changes.rst

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-09-29 08:06:53 -07:00
Tim Orling
7b316cddfb layerindex/urls.py: fix submit/thanks/ url
The url returned by the reverse lookup ends in /submit/thanks/ but
the regex was improperly searching for a string without the terminating "/".

Thank you to Pawel Zalewski for reporting the 404 error.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-09-29 07:57:59 -07:00
dependabot[bot]
a3a5a209a4 build(deps): bump gitpython from 3.1.30 to 3.1.32
Bumps [gitpython](https://github.com/gitpython-developers/GitPython) from 3.1.30 to 3.1.32.
- [Release notes](https://github.com/gitpython-developers/GitPython/releases)
- [Changelog](https://github.com/gitpython-developers/GitPython/blob/main/CHANGES)
- [Commits](https://github.com/gitpython-developers/GitPython/compare/3.1.30...3.1.32)

---
updated-dependencies:
- dependency-name: gitpython
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-08-25 13:08:00 -07:00
dependabot[bot]
6b56e5c067 build(deps): bump pillow from 9.2.0 to 9.3.0
Bumps [pillow](https://github.com/python-pillow/Pillow) from 9.2.0 to 9.3.0.
- [Release notes](https://github.com/python-pillow/Pillow/releases)
- [Changelog](https://github.com/python-pillow/Pillow/blob/main/CHANGES.rst)
- [Commits](https://github.com/python-pillow/Pillow/compare/9.2.0...9.3.0)

---
updated-dependencies:
- dependency-name: pillow
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2023-08-25 13:08:00 -07:00
Chee Yang Lee
7e6854335a requirements.txt: bump gitpython and pillow
bump gitpython and pillow to version with fixes for these CVEs

pillow: CVE-2022-22817 CVE-2022-24303 CVE-2022-45198

gitpython: CVE-2022-24439

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
2023-08-25 13:08:00 -07:00
Robert Yang
5976f9fc68 layerindex/utils.py: Add to baseconfig=True for bb.parse.handle()
Bitbake's api has been changed via:
afb8478d3 parse: Add support for addpylib conf file directive and BB_GLOBAL_PYMODULES

The conf file won't be parsed without baseconfig=True:
bb.parse.ParseError: ParseError at /path/to/oe-core/meta/conf/layer.conf:132: unparsed line: 'addpylib ${LAYERDIR}/lib oe'

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-12-20 08:38:45 -08:00
Tim Orling
0cc6d8eb1f global: replace deprecated distutils
Replace distutils.version.LooseVersion with pkg_resource.parse_version

https://majornetwork.net/2021/05/comparing-version-numbers-in-python/

[YOCTO #14990]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-12-16 08:59:37 -08:00
Michael Halstead
13c94088b4 Dockerfile: switch to ubuntu:jammy for >python3.8
ubuntu:jammy provides python 3.10.6 which supports current bitbake.
Dropped unneeded python2 packages.
Moved locale generation high to reduce error messages during image building.

Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-12-16 08:59:37 -08:00
Robert Yang
61c857a7df utils.py: Fix for is_commit_ancestor()
The runcmd() would print an "ERROR" on failure which causes confusion since
the failure is expected on old branches, so subprocess.getstatusoutput to fix
the problem.

Minor rewording.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-12-16 08:59:37 -08:00
Robert Yang
2d7906ff3b admin.py: Remove has_delete_permission()
The following commit has added on_delete=models.CASCADE to the ForeignKey:

===
commit 2d526f9b0d
Author: Amber Elliot <amber.n.elliot@intel.com>
Date:   Thu Jun 25 15:57:34 2020 -0700

    Updating models and imports for Django 2.2 upgrade.
===

The on_delete=models.CASCADE will remove the objects which reference the
removed objects, so there will be errors when removing a LayerBranch
with admin permissions:

===
Cannot delete layer branch
Deleting the selected layer branch would result in deleting related objects, but your account doesn't have permission to delete the following types of objects:

Append
distro
===

This is because has_delete_permission() always return False, remove it to use
Django's implementation to fix the problem.

Minor rewording.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-12-16 08:59:37 -08:00
Robert Yang
edb574ae7e utils.py: Use get_layer_var() to get BBFILE_COLLECTIONS
The commit 38e6288c7d has added oe-core to
BBLAYERS by default, and then there will be multiple BBFILE_COLLECTIONS which
causes update.py failure to update layers (most of the dependencies or
recommends are missing), use get_layer_var() to get BBFILE_COLLECTIONS to fix
the problem since it can ignore the extra oe-core from BBFILE_COLLECTIONS.

Fix minor typos. Minor rewording.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-12-16 08:59:37 -08:00
Michael Halstead
c7b02096e0 Dockerfile: add captcha dependencies
The current version of django-simple-captcha requires libz and freetype
development libraries.

Refactor to apply. Fix typo. Fix previous mixed use of space vs tab.

Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-12-16 08:59:37 -08:00
Tim Orling
d8ab1beebe layerindex/utils.py: ignore 'core' in BBFILES_COLLECTIONS
Many layers append BBFILE_COLLECTIONS and therefore have 'core <layer>'

During update.py, this means we are likely not handling the collection we
expect:

WARNING: /opt/workdir/git___git_openembedded_org_meta-openembedded/meta-oe: multiple collections found, handling first one (core) only
BBFILE_COLLECTIONS = "core"

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-04-28 08:15:22 -07:00
Tim Orling
242db27bf9 recipe{desc,parse}.py: BB_ENV_PASSTHROUGH_ADDITIONS
ERROR: Variable BB_ENV_EXTRAWHITE has been renamed to BB_ENV_PASSTHROUGH_ADDITIONS
ERROR: Variable BB_ENV_EXTRAWHITE from the shell environment has been renamed to BB_ENV_PASSTHROUGH_ADDITIONS
ERROR: Exiting to allow enviroment variables to be corrected

Replace BB_ENV_EXTRAWHITE with new variable BB_ENV_PASSTHROUGH_ADDITIONS

In order to be backward compatible with older branches, we must first check
for the presence of the bitbake commit which implemented the variable name
change, using layerindex.utils.is_commit_ancestor().

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-04-28 08:15:22 -07:00
Tim Orling
ee30a2e037 layerindex/utils.py: add is_commit_ancestor check
Add a helper function to check if a given SHA1 hash is an ancestor in
the currently checked out branch, using:

git merge-branch --is-ancestor <commit> HEAD

NOTE: This will not match commits which have been cherry-picked.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-04-28 08:15:22 -07:00
Tim Orling
4dd7f0ee8f layerindex/models.py: add Inactive-Upstream
Add the newish Inactive-Upstream upstream status.

Add 0046_alter_patch_status.py migration.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>

layerindex/migrations: update patch status

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-04-18 12:56:51 -07:00
Tim Orling
7ef8152266 layerindex/urls.py: fix about url pattern
The url pattern was not including the trailing /

[YOCTO #14445]

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-04-18 12:56:51 -07:00
Tim Orling
796d2455bb templates/*: staticfiles -> static
In Django 3.0 the staticfiles tag was changed to static

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 23:36:22 -08:00
Tim Orling
f074aece70 requirements.txt: bump all to latest
Run pip-upgrade and update all to latest

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 21:16:46 -08:00
Tim Orling
88b13592c3 Dockerfile: add back python2 dependencies
We have some layers and especially older releases that still support
python2. Add python2 dependencies back to the container so that the
older releases still function.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 21:00:38 -08:00
Tim Orling
722f1f3a37 rrs: add 0030_alter_recipeupgrade_maintainer.py miagration
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 21:00:36 -08:00
Tim Orling
c99a1ed4e7 Dockerfile: fix warnings
* Define LANGUAGE to quiet perl warnings
* Set DEBIAN_FRONTEND=noninteractive before apt call to quiet TERM/Dialog
  warnings

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
2022-01-13 21:00:33 -08:00