Fixed a broken link to the OpenEmbedded FAQ.
Fixes [YOCTO #15645]
Reported-by: <kweihmann@outlook.com>
Co-developed-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Chuck Wolber <chuckwolber@gmail.com>
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
In production, we use settings.DEFAULT_FROM_EMAIL as our primary source
of from_email, rather than settings.SUBMIT_EMAIL_FROM
Default to DEFAULT_FROM_EMAIL if it is not Null, fall back to SUBMIT_EMAIL_FROM
The real consumer (in production) is a Celery task in layerindex/tasks.py
@tasks.task
def send_email(subject, text_content, from_email=settings.DEFAULT_FROM_EMAIL, to_emails=[]):
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Change the canonical, definitive place to send patches to
the new yocto-patches@lists.yoctoproject.org list.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
There is nothing preventing a period in a layer name from the
BitBake perspective.
For example, https://github.com/webosose/meta-webosose has:
meta-webos-backports/meta-webos-backports-4.1
[YOCTO #15373]
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
https://github.com/webosose/meta-webosose has layers like:
meta-webos-backports/meta-webos-backports-4.1
which is 45 characters.
Allow for up to 60 characters in the vcs_subdir field in the LayerBranch class.
[YOCTO #15374]
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Exposing the actual_branch field of LayerBranch in the UI
was intending to _enable_ setting a non-default value, but
the check was accidentally left in a state that _requires_
a value.
[YOCTO #15376]
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Now that we have different BBClass subtypes, it is handy to have
a UI indication of the different types. Add "badges" to display
"Global" (for 'classes-global' paths) or "Recipe" (for
'classes-recipe' paths).
[YOCTO #15238]
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Since commits:
bitbake f33ce7e7
'BBHandler/cooker: Implement recipe and global classes'
oe-core f5c12800
'classes: Update classes to match new bitbake class scope functionality'
we now have a lot of the oe-core recipe classes in meta/classes-recipe.
We are also missing any bbclasses in meta/classes-global
[YOCTO #15238]
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Add support for classes-global and classes-recipe, but use
"proxy=True" to not create new tables for the new classes.
Rather, we use the bbclass_type field.
[YOCTO #15238]
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Since we are only using parse_version for comparison (typically checking
that we are greater than some minimum version for tool or package), one
would think we can use packaging.version.parse as if it was parse_version
Unfortunately, this requires conforming to PEP-440 version definitions,
which does not work for e.g. autotools (2.72d) nor older openssl (1.1.1p).
We rely in these (and to be sure other) cases on the LegacyVersion behavior.
https://packaging.python.org/en/latest/specifications/version-specifiers/#summary-of-differences-from-pkg-resources-parse-version
"This specification purposely restricts the syntax which constitutes a
valid version while pkg_resources.parse_version attempts to provide some
meaning from any arbitrary string."
In order to have the least impact to the overall code, we instead add
packaging_legacy to requirements.txt and use packaging_legacy.version.parse
as if it was parse_version.
https://pypi.org/project/packaging-legacy/https://github.com/pypa/packaging/pull/407
Since pypi.org itself is depending on packaging_legacy (in fact, a pypi dev
developed the package), we can expect it to be supported for quite some time.
https://github.com/pypi/warehouse/pull/13500
[YOCTO #15348]
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
'result' is a list of RecipeUpstream objects... drop legacy 'recipe, ru'
syntax. In the current code base, result.append((recipe, ru)) was
appending a tuple of the last value of 'recipe' (in 'for recipe_qry:' under
'for maintplan in maintplans:') and the RecipeUpstream object.
Fixes:
AttributeError: 'tuple' object has no attribute 'history'
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
The get_recipe_pv_without_srcpv function was renamed to
get_recipe_pv_with_pfx_sfx in:
84794b59 lib/oe/recipeutils.py: accommodate SRCPV being optional and deprecated in version check regex
Try to import/call the old method and fail over to the new method.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Fixes:
Traceback (most recent call last):
File "/opt/layerindex/./rrs/tools/rrs_upstream_history.py", line 210, in <module>
set_regexes(recipe_data)
File "/opt/layerindex/./rrs/tools/rrs_upstream_history.py", line 73, in set_regexes
bb.data.update_data(localdata)
AttributeError: module 'bb.data' has no attribute 'update_data'
bb.data.update_data was dropped in:
584989ed 'data/data_smart/build: Clean up datastore finalize/update_data references'
While we are here, drop some trailing whitespace.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
When performing rrs_upgrade_history for commit date 8/24/2023,
the 'fetch2: Add new srcrev fetcher API' bitbake commit is not
automatically added.
Fix this by remapping 'cooker: Fix error message' (8/22/2023) to
the tag on 8/24/2023 'fetch2: git: Check if clone directory is
a git repo'.
This allows upgrade history for oe-core to continue across the
8/24/2023 date.
Fixes:
2024-01-08 23:59:07,114: DEBUG: run cmd '['git', 'show', '-s', '--format=%ci']' in /opt/workdir/git___git_openembedded_org_openembedded-core
2024-01-08 23:59:07,215: DEBUG: output: 2023-08-24 13:34:35 +0100
2024-01-08 23:59:07,227: DEBUG: run cmd '['git', 'rev-parse', 'HEAD']' in /opt/workdir/bitbake
2024-01-08 23:59:07,329: DEBUG: output: a06619951a43acb80b80d92e0caac560657ca249
2024-01-08 23:59:07,448: DEBUG: Running "GIT_DIR=/opt/workdir/bitbake/.git git merge-base --is-ancestor 87104b6a167188921da157c7dba45938849fb22a HEAD"
NOTE: Starting bitbake server...
ERROR: Failure expanding variable fetcher_hashes_dummyfunc[vardepvalue], expression was ${@bb.fetch.get_hashvalue(d)} which triggered exception AttributeError: module 'bb.fetch2' has no attribute 'get_hashvalue'
The variable dependency chain for the failure is: fetcher_hashes_dummyfunc[vardepvalue]
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
* Allow the regex for RECIPE_MAINTAINER to use either _ or : override
syntax to allow processing maintainer changes across the change
boundary.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Bump all the dependencies to latest, using
pur -r requirements.txt
We are now able to build the mysqlclient==2.2.1 package,
so the only constraint is Django>=4.2,<4.3 to keep us
pinned at the desired LTS version.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
To build mysqlclient==2.2.1 we need to add default-libmysqlclient-dev
and pkg-config
Drop conflicting libmariadb-dev-compat
Drop python3-mysqldb to allow requirements.txt to override version
Upgrade pip and setuptools before installing from requirements.txt
as newer versions are required to properly build mysqlclient wheel.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Some layers now have one branch with many supported
LAYERSERIES_COMPAT. If this branch name does not match
one of the stable releases, LayerBranches might not
have been created. When actual_branch is set, it is
only set in a LayerBranch object. We previously
could not update (create) a stable branch with actual_branch
except manually in the admin interface.
Add --force-create option to be used in conjunction with
--actual-branch (which already requires --branch) in the
update.py script. This tells the script to ignore the
fact that no layerbranch exists already.
Add --actual-branch to update_layer.py so that we can create
(and more importantly checkout) an actual_branch for the
given stable --branch.
Update utils.py to allow checking out of actual_branch when
a LayerBranch does not yet exist.
While we are at it, ensure that any Branch that is marked
as no update will be skipped even with --force-create. The
main reason that a Branch has updates disabled is because the
bitbake or python syntax has changed enough to cause exceptions.
This script can now be run with:
./layerindex/update.py \
--layer meta-weird-one \
--branch kirkstone \
--actual-branch=nonstandard \
--force-create
Which will attempt to create a meta-weird-one:kirkstone layerbranch
checked out at the 'nonstandard' branch from that layer's git repo.
This allows layerindex admins to at least populate the database
without tedious creation of layerbranches in the admin interface.
Helps make the "branch mapping" actually work and be useful:
[YOCTO #8008]
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
* Add an "Update Layer" button to the layer detail view.
- This allows a user that is a member of is_staff to trigger
an update of the current layer (for the current branch)
* Add an "Update Layer" button to the reviewdetail view
- This allows a user that is a member of is_staff and has
publish_layer permissions to trigger an update attempt
of the layer under review (even in the un-published state)
* The update is run as a task with Celery
NOTE:
You must have the RABBITMQ_ and DATABASE_ credentials set
correctly in the docker/settings.py file or set via
environment variables or you will get authentication errors
talking to layersdb or layersrabbit containers.
[YOCTO #12484]
layerindex/views.py: add update_layer_view
layerindex/urls.py: add update_layer_view
layerindex/urls_branch.py: add update_layer_view
templates/layerindex/reviewdetail.html: add Update Layer button
templates/layerindex/detail.html: add Update Layer button
templates/layerindex: add updatelayer.html
TODO:
While the update is happening, the AJAX rendering of the
update.log is showing the b'' characters and not adding
any new lines. If you go back to the same task view
afterwards, the log is rendered as expected.
TODO:
After the update is completed, it would be nice to have a
button to return you to the page from where you called the
"Update Layer".
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
For layers which do not follow standard branch names (including
the inclusive naming move away from "master" to "main") we have
the actual_branch field set in a LayerBranch object. Previously
this was only exposed via the admin interface.
Allow layer maintainers (including upon submitting a new layer)
to set the 'Actual branch' in the web UI.
Add a check to make sure the actual_branch is a valid branch
name using 'git check-ref-format --branch <actual_branch>'
since we are not using full refs.
[YOCTO #8008]
NOTE:
Only existing LayerBranches will be editable. A new layer
can be submitted with a different branch for "master", but
only the "master" LayerBranch will be created.
Further changes to the update.py script will be needed to
make creation of new stable branches with an actual_branch
possible.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
The '--staging' argument to certbot has now been changed
to '--test-cert'. We previously only allowed using the
dockersetup.py tool to create Staging environment certs,
which are still marked as invalid by browsers. Add a
'--letsencrypt-production' knob to allow for valid, trusted
certs to be created. If they already exist in the workspace
and have not expired, re-use them (to avoid hitting rate
limits). Continue to '--force-renewal' for staging certs.
NOTE:
If you have previously created staging certs in your
workspace, you will want to clean docker/certs before
creating production certs for the same domain. Certbot
will not overwrite those staging certs and the newly
created ones will not be in the path passed in by
dockersetup.py.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Within the layersapp container, if you try to run layerindex/update.py:
File "/opt/layerindex/settings.py", line 280, in <module>
RABBIT_BROKER = 'amqp://' + os.getenv('RABBITMQ_DEFAULT_USER') + ':' + os.getenv('RABBITMQ_DEFAULT_PASS') + '@layersrabbit:5672/'
TypeError: can only concatenate str (not "NoneType") to str
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
The Branch Comparison view would throw an error because of a parser_view instead of parse_view typo.
[YOCTO #15332]
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
File "/opt/layerindex/layerindex/update.py", line 525
failed_layers[branch].append('%s: Failed to add since LAYERDEPENDS [%s ...] is not
SyntaxError: '(' was never closed
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Make errors like this more useful:
ERROR: Issues found on branch nanbield:
meta-luneos: Failed to add since LAYERDEPENDS is not satisfied
meta-luneui: Failed to add since LAYERDEPENDS is not satisfied
In this case, meta-luneos depends on meta-luneui, but we cannot create a
'nanbield' layer branch, because meta-luneui LAYERDEPENDS on qt6-layer
collection, which currently has no 'nanbield' branch.
Use next(iter(value['deps'])) to get the first element in the OrderedDict
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Currently, the error message is something like:
ERROR: conf/layer.conf not found for layer meta-doom - is subdirectory set correctly?
This is because in this case meta-doom has a 'langdale' branch without
a conf/layer.conf
Another example is:
ERROR: conf/layer.conf not found for layer meta-st-stm32mp - is subdirectory set correctly?
This is because meta-st-stm32mp has a master branch, but it only contains
a README.md telling you to use the stable branches (e.g. nanbield).
Make it more obvious what the source of the error is by also displaying
the branch that was being attempted.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
The regex for Upstream-Status had (\w+), but this
does not include hyphen/dash. Explicitly add it to
the pattern.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
For both meta-poky/meta-yocto and meta-xilinx/meta-xilinx-core
we have a situation where the "collections" stayed the same
("yocto" and "xilinx" respectively) but the layer/layerbranch
changed. Without the "branch" argument to get_dependency_layers,
we were always defaulting to the older layer which first
defined the "collection".
Instead, add an option to use "branch" to filter on the expected
LayerBranch object. Keep the old behavior just in case someone
depends upon it.
[YOCTO #15221]
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
We have never been checking out dependent layers at the same
release/branch. With the introduction of 'addpylib', this
became obvious due to parsing errors.
Ensure that known LayerDependency objects are checked out at
the expected branch/release. Since openembedded-core has already
been handled elsewhere, we skip it.
[YOCTO #15236]
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
When a new release is run for the first time, no LayerBranch exists
yet and the update_enabled cannot be queried. The unintended result
is that all layers are skipped. Move the update_enabled check into
the code path where the layerbranch already exists.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Add a script which can either mark one --layer --branch or
use data in a --from-file to mark multiple layer:branch objects
as Yocto Project Compatible.
The --from-file is a json file is compatible with or which can be
generated by:
yocto-autobuilder-helper/scripts/list-yp-compatible-layers.py
[YOCTO #15093]
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Especially since LTS branches have become more popular as the
only active branch, we need to be able to turn off updates on
a LayerBranch basis rather than the current Layer or Branch
heavier hammers.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
