The new devtool ide plugin provides the eSDK and configures an IDE to
work with the eSDK. In doing so, bitbake should be used to generate the
IDE configuration and update the SDK, but it should no longer play a
role when working on the source code. The work on the source code should
take place exclusively with the IDE, which, for example, calls cmake
directly to compile the code and execute the unit tests from the IDE.
The plugin works for recipes inheriting the cmake or the meson bbclass.
Support for more programming languages and build tools may be added in
the future.
There are various IDEs that can be used for the development of embedded
Linux applications. Therefore, devtool ide-sdk, like devtool itself,
supports plugins to support IDEs.
VSCode is the default IDE for this first implementation. Additionally,
some generic helper scripts can be generated with --ide none instead of
a specific IDE configuration. This can be used for any IDE that
supports calling some scripts.
There are two different modes supported:
- devtool modify mode (default):
devtool ide-sdk configures the IDE to manage the build-tool used by the
recipe (e.g. cmake or meson). The workflow looks like:
$ devtool modify a-recipe
$ devtool ide-sdk a-recipe a-image
$ code "$BUILDDIR/workspace/sources/a-recipe"
Work in VSCode, after installing the proposed plugins
Deploying the artifacts to the target device and running a remote
debugging session is supported as well.
This first implementation still calls bitbake and devtool to copy the
binary artifacts to the target device. In contrast to compiling,
installation and copying must be performed with the file rights of the
target device. The pseudo tool must be used for this. Therefore
bitbake -c install a-recipe && devtool deploy-target a-recipe
are called by the IDE for the deployment. This might be improved later
on.
Executing the unit tests out of the IDE is supported via Qemu user if
the build tool supports that. CMake (if cmake-qemu.bbclass is
inherited) and Meson support Qemu usermode.
- Shared sysroots mode: bootstraps the eSDK with shared sysroots for
all the recipes passed to devtool ide-sdk. This is basically a wrapper
for bitbake meta-ide-support && bitbake build-sysroots. The workflow
looks like:
$ devtool ide-sdk --share-sysroots a-recipe another-recipe
vscode where/the/sources/are
If the IDE and the build tool support it, the IDE gets configured to
offer the cross tool-chain provided by the eSDK. In case of VSCode and
cmake a cmake-kit is generated. This offers to use the cross
tool-chain from the UI of the IDE.
Many thanks to Enguerrand de Ribaucourt for testing and bug fixing.
(From OE-Core rev: 3f8af7a36589cd05fd07d16cbdd03d6b3dff1f82)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A missing space when using :append would lead to run-on URIs if there
was no whitespace at the end of the original SRC_URI.
(From OE-Core rev: 4de0c679e4b1a3cb394f348d625b97ad73f1efe7)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
All the regexes throw a warning like this:
WARNING: scripts/lib/recipetool/create_buildsys.py:140:
SyntaxWarning: invalid escape sequence '\s'
proj_re = re.compile('project\s*\(([^)]*)\)', re.IGNORECASE)
Python 3 interprets string literals as Unicode strings, and therefore
\s is treated as an escaped Unicode character which is not correct.
Declaring the RegEx pattern as a raw string instead of unicode is
required for Python 3.
(From OE-Core rev: 24b0ba00d4f0b4d9834f7693ecb6032dfc534a80)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we can't find tomllib or tomli then we can just tell the user politely
that we can't parse the pyproject.toml file, there's no need to dump
exception stack traces.
Move the parser exception handler to catch the actual parse, as otherwise
it will never be used.
Whilst here, also add some debug statements to make it clear what of the
handlers is being called.
(From OE-Core rev: 1a3ba4c312844d80ae382912b319e60ad8b30737)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- os is missing
- glob is not used
(From OE-Core rev: f560bc8972e97d2e497d7b976586e4a79ffc74ec)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
GPT based disks have a disk guid apart from the 32-bit disk identifier.
This commit implements reproducible disk guid by using SOURCE_DATE_EPOCH (if available) value as a random seed
(From OE-Core rev: 150e079589e207fe174d2dceb40cd8f3d3972c5a)
Signed-off-by: Adithya Balakumar <Adithya.Balakumar@toshiba-tsip.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make the deploy function independent from d. This allows to call the
function also from Python code not running in bitbake.
This is needed to for the devtool ide plugin which will call the
do_install task and the code from devtool deploy-target independently
from a bitbake server. This allows a much quicker workflow.
(From OE-Core rev: c8697d1132cbd4b2a2502b4c48e7d91fc18de786)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a step towards a deploy function which can be called without
passing the d variable.
(From OE-Core rev: 3e8c41ec373a1766f68932d3b979d00e4e3bb3a9)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Provide a function exec_fakeroot_no_d which does the same like
exec_fakeroot does, but is usable independenlty from bitbake. This
allows to use the fanction from scripts where the d variable is not
available.
(From OE-Core rev: 6f5980d96844559639fc3660bb975b0bdd1aa523)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[YOCTO #15162]
when doing devtool modify, sources are extracted into a devtool
temporary workdir. The main source is moved inside
build/workspace/sources/${BPN}/ and local files are moved inside
build/workspace/sources/${BPN}/oe-local-files. Secondary sources are
currently not handled and are lost.
Here is the output of devtool modify/build on bzip2 recipe:
NOTE: bzip2: compiling from external source tree <...>/build/workspace/sources/bzip2
ERROR: bzip2-1.0.8-r0 do_install_ptest_base: ExecutionError('<...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368', 1, None, None)
ERROR: Logfile of failure stored in: <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/log.do_install_ptest_base.3368
Log data follows:
| DEBUG: Executing shell function do_install_ptest_base
| NOTE: make -j 16 DESTDIR=<...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest install-ptest
| sed -n '/^runtest:/,/^install-ptest:/{/^install-ptest:/!p}' \
| ../../../../../../workspace/sources/bzip2/Makefile.am > <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/Makefile
| cp ../../../../../../workspace/sources/bzip2/sample1.ref <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
| cp ../../../../../../workspace/sources/bzip2/sample2.ref <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
| cp ../../../../../../workspace/sources/bzip2/sample3.ref <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
| cp ../../../../../../workspace/sources/bzip2/sample1.bz2 <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
| cp ../../../../../../workspace/sources/bzip2/sample2.bz2 <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
| cp ../../../../../../workspace/sources/bzip2/sample3.bz2 <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
| ln -s /usr/bin/bzip2 <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/bzip2
| cp: cannot stat '<...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/git/commons-compress': No such file or directory
| WARNING: <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368:189 exit 1 from 'cp -r <...>/build/tmp/work/core2-64-poky-linux/bzip2/
1.0.8/git/commons-compress <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/bzip2-tests/commons-compress'
| WARNING: Backtrace (BB generated script):
| #1: do_install_ptest, <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368, line 189
| #2: do_install_ptest_base, <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368, line 158
| #3: main, <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368, line 226
ERROR: Task (<...>/poky/meta/recipes-extended/bzip2/bzip2_1.0.8.bb:do_install_ptest_base) failed with exit code '1'
NOTE: Tasks Summary: Attempted 776 tasks of which 765 didn't need to be rerun and 1 failed.
Summary: 1 task failed:
<...>/poky/meta/recipes-extended/bzip2/bzip2_1.0.8.bb:do_install_ptest_base
externalsrc class modify SRC_URI to keep only:
* 'file', 'npmsw' and 'crate' sources
* url with type parameter matching 'kmeta' or 'git-dependency'
So by forcing to add type='git-dependency' on secondary sources, we
ensure that when building the recipe, the secondary sources can be
unpacked into WORKDIR.
This allows recipes containing several sources to be built under a
devtool context, but it has some limitations:
* user would not be able to generate patches for the secondary sources
* type="git-dependency" is added for secondary sources even on non git
sources, so we may want to rename this parameter
(From OE-Core rev: cfd5ee890163a3d975093359016dda104e7b71df)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Major module version is a part of name, but not necessary part of the
actual URL (See https://go.dev/ref/mod#module-path).
Nevertheless, name detection function can't handle that suffix, so get
rid of it to determine component name.
For replaced modules that name might be different that the actual module
name defined in go.mod file.
(From OE-Core rev: 0cccfa1041d48f0ae3a2dc89a129cf7884fc08f0)
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whenever the recipe uses a CLOSED license, the list is going to be
empty. It's a discouraged practice not to have a license, but proceed
anyway to finish recipe generation.
(From OE-Core rev: 5ca920284d0946346f5b06f5e443c80d9d8b85ce)
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Local modules are usually referenced with a 'replace' directive in
go.mod file. If that's the case, remove them from populating SRC_URI.
(From OE-Core rev: 9f220f61e3e44a650a46ee997b47f1d87b7c4ef0)
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Root Cause:
initial_revs is an empty dictionary and do not have "." key.
Traceback (most recent call last):
File "scripts/devtool", line 349, in <module>
ret = main()
File "scripts/devtool", line 336, in main
ret = args.func(args, config, basepath, workspace)
File "scripts/lib/devtool/standard.py", line 922, in modify
if not initial_revs["."]:
KeyError: '.'
Solution:
check key exists, then get its value.
(From OE-Core rev: fb0db5c48abb4d56233a175fdd349d18b972e452)
Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'devtool check-upgrade-status' is for reporting upgradable
status for recipes. The output should always be printed out. So
we should just use 'print' instead of 'logger.info' as the latter
will be suppressed if '-q' parameter is supplied to devtool.
(From OE-Core rev: 2c7bf9c8a833bec13a1ebabdce30933cbe691108)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
E2FSPROGS_FAKE_TIME: sets fixed times for the inodes in the file system.
hash_seed: creates reproducible directory indexes in the file system.
Reference commit in e2fsprogs: e1f7100643a46456be107b33098f6034b0835e6d
(From OE-Core rev: bb822ab75de0020572058090439b93cc56bbf7e0)
Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the new python_maturin PEP-517 backend
Add selftest for 'pydantic-core' pypi package.
(From OE-Core rev: 69b679380616a94a631681caa05d9bf7610f9372)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoids the following error when running `devtool modify` on a
recipe that has a menuconfig task, but does not have
KCONFIG_CONFIG_ENABLE_MENUCONFIG set.
.../temp/run.do_configure.4163366: line 152:
${@ oe.types.boolean('${KCONFIG_CONFIG_ENABLE_MENUCONFIG}') }: bad substitution
WARNING: .../temp/run.do_configure.4163366:152 exit 1 from
'[ ${@ oe.types.boolean('${KCONFIG_CONFIG_ENABLE_MENUCONFIG}') } = True ]'
(From OE-Core rev: e87e6fa84a0c4b5ac8e736dc62f6e08390ba2436)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The pypi change:
"85a2a6f68af recipetool: create_buildsys_python: add pypi support"
deleted all the SRC_URI variables, including the SRC_URI checksums.
These are not generated by the pypi.bbclass (how could they be trusted?)
Without the checksum(s), we are vulnerable to a man-in-the-middle attack
and zero checks on the validity of the downloaded tarball from pypi.org.
Fix by only setting S and SRC_URI to None.
(From OE-Core rev: 560181a52111569f7bc57b09139b42510e0d0325)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoids the following error when using --no-extract, introduced in
commit 900129cbdf (devtool: add support for git submodules):
Traceback (most recent call last):
File ".../scripts/devtool", line 349, in <module>
ret = main()
File ".../scripts/devtool", line 336, in main
ret = args.func(args, config, basepath, workspace)
File ".../scripts/lib/devtool/standard.py", line 995, in modify
for commit in commits[name]:
KeyError: '.'
(From OE-Core rev: 6a44fb10ef4b90ee24f470362007d56183abc593)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In addition to updating the sha256sum and removing the md5sum, update
all other existing checksums. If the only existing checksum is md5sum,
then replace it with the default expected checksums (currently only
sha256sum).
(From OE-Core rev: 8ea8827ee49b7f0443b1c4bd47d1344a689d73a3)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than including all SRC_URI checksums, include the ones that are
expected. These are the same as are output if no checksums are included
when building the recipe.
(From OE-Core rev: c2af83eb5e8573480179b6c0bcce50606b547099)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than having a static list of crunched MD5 checksums for some of
the most common licenses, calculate it for all common licenses. This
should improve the identification of license text variantions.
(From OE-Core rev: 377f9513dc56e9b8e5f5813c1535be0206756949)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a new option -u/--update-recipe to update the recipe itself instead
of creating/updating a bbappend file
(From OE-Core rev: c9a80f55ed0bf9ce249205d6a9ba670fbb7f59ba)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
appendsrc function relies on oe.recipeutils.bbappend_recipe to
copy files and add the corresponding entries in SRC_URI.
Currently, appendsrc function build itself the new SRC_URI entry to add the
correct subdir param, and gives it using the extralines parameter.
This has 2 drawbacks:
- oe.recipeutils.bbappend_recipe can already do this if we specify the
correct params, so we have duplicate code
- the duplicated code is not fully functional: for example, it doesn't
take into account the -m/--machine parameter
So fix this by not using extralines but give correctly formatted params.
Also remove the check for already existing entries as
oe.recipeutils.bbappend_recipe already implement it
The new bbappend file now have the SRC_URI entry after the
FILESEXTRAPATHS so fix the selftest.
Update test_recipetool_appendsrcfile_existing_in_src_uri_diff_params
test because recipetool appendsrcfiles used to not add new src_uri entry
if the entry already exist even with different parameters while
oe.recipeutils.bbappend_recipe adds it if parameters are different (and
remove the old entry)
(From OE-Core rev: cd5de8d53849a6f3bb6f82e45fb301e39892c789)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add dry-run mode for recipetool appendsrcfile and appendsrcfiles, with
if necessary, a diff of changes
(From OE-Core rev: 456c726e6b52f4dc57ff605e1cf1687097537002)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds features to explicitly write zeros to the start of the
partition. This is useful to overwrite old content like
filesystem signatures which may be re-recognized otherwise.
The new features can be enabled with
'--soucreparams="[fill|size=<N>[S|s|K|k|M|G]][,][bs=<N>[S|s|K|k|M|G]]"'
Conflicting or missing options throw errors.
The features are:
- fill
Fill the entire partition with zeros. Requires '--fixed-size' option
to be set.
- size=<N>[S|s|K|k|M|G]
Set the first N bytes of the partition to zero. Default unit is 'K'.
- bs=<N>[S|s|K|k|M|G]
Write at most N bytes at a time during source file creation.
Defaults to '1M'. Default unit is 'K'.
(From OE-Core rev: d19d4529e7a2056caeb526fed980cc1df19a5f6e)
Signed-off-by: Malte Schmidt <malte.schmidt@weidmueller.com>
Signed-off-by: Lukas Funke <lukas.funke@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Today, we can use devtool/recipetool to create recipes for python projects
using the github url or the direct release tarball of the project, but the
create_buildsys_python plugin doesn't support the pypi class, since we cannot
know from the extracted source if the package is available on pypi or not.
By implementing the new optional process_url callback, we can detect
that the url is a pypi one (i.e 'https://pypi.org/project/<package>')
and retrieve the release tarball location.
Also detect if the url points to a release tarball hosted on
"files.pythonhosted.iorg" (i.e https://files.pythonhosted.org/packages/...)
In both cases, adds the pypi class, remove 'S' and 'SRC_URIxxx'
variables from the created recipe as they will be handled by the pypi class
and add the PYPI_PACKAGE variable
This helps to produce cleaner recipes when package is hosted on pypi.
If the url points to a github url or a release tarball not coming from
"files.pythonhosted.org", the created recipe is the same as before.
One can also use the newly added "--no-pypi" switch to NOT inherit
from pypi class on matching url, to keep legacy behaviour.
To create a recipe for a pypi package, one can now use one of the
new following syntax (using recipetool create / devtool add):
* recipetool create https://pypi.org/project/<package>
* recipetool create https://pypi.org/project/<package>/<version>
* recipetool create https://pypi.org/project/<package> --version <version>
or the old syntax:
* recipetool create https://files.pythonhosted.org/packages/<...>
(From OE-Core rev: 097a43846cd99a7d74d004efc57f583ce78970a4)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a new process_url callback that plugins can optionally implement if
they which to handle url.
Plugins can implement this callback for example, to:
* transform the url
* add special variables using extravalues
* add extra classes
* ...
If a plugin handles the url, it must append 'url' to the handled
list and must return the fetchuri
No functional changes expected for plugins non implementing this
optional callback
(From OE-Core rev: 3e5ce351e7dbe283562bf1db1f2ce3b121c49b53)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
create_recipe() function relies on oe.recipeutils.patch_recipe_lines()
which relies on bb.utils.edit_metadata(). edit_metada expect lines to
have trailing newlines, so add it to each lines before calling
patch_recipe_lines, otherwise edit_metadata will not be able to squash
blank line if there are two consecutive blanks after a removal
(From OE-Core rev: 31b27cc76a1b669e2b126c332608fd772c124d69)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the case pyproject.toml doesn't contains metadatas, the metadata
variable is not initialized and the plugin throws an error and falls back
to another plugin, which is not the desired behaviour. So just ignore
metadata if we don't have them
(From OE-Core rev: 88d15877ba18309c521740d7a9649e14d77189bc)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The issue with installing the kernel image to both rootfs
and boot partition is that some systems rely on the kernel image in
rootfs and not in the boot partition.
This leads to duplication of the kernel image, which can cause
unnecessary storage usage.
This patch provides a solution to the problem by adding a new
parameter "install-kernel-into-boot-dir" to the wic kickstart file.
If this parameter is set to 'true', the plugin will install the
kernel image to the boot partition. If the parameter is set to
'false', the plugin will skip installing the kernel image, avoiding
duplication.
(From OE-Core rev: d3599afe5f604ea5afd9411e114934dcb52b2d48)
Signed-off-by: Kareem Zarka <kareem.zarka@huawei.com>
Signed-off-by: Christian Taedcke <christian.taedcke@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding the support of submodules required a lot of changes on the
internal data structures:
* initial_rev/startcommit used as a starting point for looking at new
/ updated commits was replaced by a dictionary where the keys are the
submodule name ("." for main repo) and the values are the
initial_rev/startcommit
* the extractPatches function now extracts patch for the main repo and
for all submodules and stores them in a hierarchical way describing the
submodule path
* store initial_rev/commit also for all submodules inside the recipe
bbappend file
* _export_patches now returns dictionaries that contains the 'patchdir'
parameter (if any). This parameter is used to add the correct
'patchdir=' parameter on the recipe
Also, recipe can extract a secondary git tree inside the workdir.
By default, at the end of the do_patch function, there is a hook in
devtool that commits everything that was modified to have a clean
repository. It uses the command: "git add .; git commit ..."
The issue here is that, it adds the secondary git tree as a submodule
but in a wrong way. Doing "git add <git dir>" declares a submodule but do
not adds a url associated to it, and all following "git submodule foreach"
commands will fail.
So detect that a git tree was extracted inside S and correctly add it
using "git submodule add <url> <path>", so that it will be considered as a
regular git submodule
(From OE-Core rev: 900129cbdf25297a42ab5dbd02d1adbea405c935)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the case of a repository with submodules, we need to add the
"devtool-base" and "devtool-patched" tag on all submodules in order to
properly detect the added/removed/modified patches
(From OE-Core rev: 241da68805d177d4ec4b302c8a997645cc645286)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When specifying --mode / -m srcrev with devtool finish/update-recipe on
recipes that are not fetched from a SCM repository we get the following
error:
Traceback (most recent call last):
[..]
File "<...>/poky/meta/lib/oe/patch.py", line 49, in runcmd
raise CmdError(cmd, exitstatus >> 8, "stdout: %s\nstderr: %s" % (stdout, stderr))
oe.patch.CmdError: Command Error: 'sh -c 'git format-patch --no-signature --no-numbered INVALID -o /tmp/oepatchbj7pfmzj -- .'' exited with 0 Output:
stdout:
stderr: fatal: bad revision 'INVALID'
Fix this by adding a check and abort with a proper error message.
(From OE-Core rev: 5b77f60e85d07921ae5e808daa1fd7d8c3dc40ea)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running devtool update-recipe with --mode=srcrev AND --append switch
in dry-run, we get the following error:
Traceback (most recent call last):
[...]
Exception: destpath should be set here
Fix this by removing a misplaced else statement in _update_recipe_srcrev
(From OE-Core rev: 85ba125703d5b442133fd7c470b915460ee68ac9)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit c304fcbe05 introduced a grouping when
listing regressions. This grouping has been added only for ptests. It has
been observed that any other kind of tests could benefit from it. For
example, current regression reports can show the following:
1 regression(s) for oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash
oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash: PASSED -> SKIPPED
1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_help
oescripts.OEPybootchartguyTests.test_pybootchartguy_help: PASSED -> SKIPPED
1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_pdf_output
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_pdf_output: PASSED -> SKIPPED
1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_png_output
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_png_output: PASSED -> SKIPPED
1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_svg_output
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_svg_output: PASSED -> SKIPPED
[...]
This output is not so useful in its current state and should be grouped per
test type too.
Enable grouping for all kind of tests, to make it llok like the following
in reports:
5 regression(s) for oescripts
oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash: PASSED -> SKIPPED
oescripts.OEPybootchartguyTests.test_pybootchartguy_help: PASSED -> SKIPPED
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_pdf_output: PASSED -> SKIPPED
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_png_output: PASSED -> SKIPPED
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_svg_output: PASSED -> SKIPPED
(From OE-Core rev: 982798ef96e3a32bf15341bdd3bb7c4356709412)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
One of the newer PEP-517 backends to be added was python_hatchling.bbclass
but it was not included in the recent improvements.
Add selftest for 'jsonschema' pypi package.
(From OE-Core rev: d99b4883b4fee82bc588fd235ba90fedf1550cb8)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was writing out locked-sigs.inc into cwd with every
'bitbake -S' invocation. When the intent is only to to get task
stamps (-S none), or print the difference between them (-S printdiff),
the file is unnecessary clutter.
A couple of selftests/scripts were however relying on this, so they're
adjusted to explicitly request the file.
eSDK code calls dump_lockedsigs() separately via
oe.copy_buildsystem.generate_locked_sigs() and so isn't affected.
(From OE-Core rev: ad57c3cac2a8d3e60222e3cca0685f582dcea135)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add basic support for PEP517 [1] for the 3 following backends that are
supported by bitbake:
- setuptools.build_meta
- poetry.core.masonry.api
- flit_core.buildapi
If a pyproject.toml file is found, use it to create the recipe,
otherwise fallback to the old setup.py method.
Some projects can declare a minimal pyproject.toml file, and put all
the metadata in setup.py/setup.cfg/requirements.txt .. theses cases
are not handled. If a pyproject.toml file is found, assumes it has all
necessary metadata.
As for the old setup.py method, version numbers for dependencies are not
handled.
Some features may be missing, such as the extra dependencies.
[YOCTO #14737]
[1]: https://peps.python.org/pep-0517/
(From OE-Core rev: c7d8d15b2d0a9ecd210bd247fa0df31d9f458873)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to prepare the support for pyproject.toml (PEP517 [1]) enabled
projects, refactor the code and move setup.py specific code into a
specific class in order to allow sharing the PythonRecipeHandler class
No functionnal changes expected
[1]: https://peps.python.org/pep-0517/#source-tree
(From OE-Core rev: 2281e93347da4129062cfb40710df03c87c63168)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By convention, all python recipes start with "python3-" so update
create_buildsys_python to do this
This rule doesn't apply for packages already starting with "python"
Update recipetool's selftest accordingly
(From OE-Core rev: b0d87440e610b80f763d09784d4a90a148bb3e7b)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
License field of setup is not always standardized, so we usually use the
classifier to determine the correct license format to use in the recipe.
A warning note is added above the LICENSE field of the create recipe
in case a license is provided in setup. But when the plugin is called,
"LICENSE =" is not yet present so we can never display this note.
Replace the "LICENSE =" condition with "##LICENSE_PLACEHOLDER##"
to actually be able to display the note message
(From OE-Core rev: b7c26ca2028aa60f740464de85a11a01a531f32e)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since "matched" and "improved" tests are not as important as regressions,
reduce the place they take in the regression report:
- merge "matched" and "improved" tests, while removing the label
- add a single line of additional info per pair
Those changes make the "Matches and improvements" look like the following
sample:
oeselftest_almalinux-9.2_qemux86-64_20230910083156
oeselftest_almalinux-8.8_qemux86-64_20231018010951
-> +7 test(s) present
oeselftest_almalinux-9.2_qemux86-64_20230911010538
oeselftest_debian-11_qemux86-64_20231017150459
oeselftest_debian-11_qemux86-64_20230910012927
oeselftest_debian-11_qemux86-64_20231017151319
-> +7 test(s) present
[...]
(From OE-Core rev: 6de4426d9a7da67deed7d3a3918892fb56238ff3)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Regressions reports currently reports matching pairs and improved pairs
first, then regressions.
Change order to print regressions first, which is the most valuable info in
the report, and then print improvements and matches at the bottom.
(From OE-Core rev: 599267467430e70fa4dc8ba6b2a8b126bf6da359)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Most of the changes list generated in regression reports fall in one
of the two following categories:
- there is only a few (<10) changes listed and the info is
valuable/relevant
- the list is huge (> 100 ? 1000 ?) and basically tells us that the whole
tests category suffers the same status (test missing, test failing, test
skipped, etc)
Prevent those huge, worthless lists by limiting the output for each test
group:
- current default limit is arbitrarily set to 50
- limit can still be overriden with a new "-l"/"--limit" flag, either with
custom value, or with 0 to print the whole lists of changes
- limit is applied per test family: currently it distinguishes only types
of ptests, but it can be adapted to other kind of tests
(From OE-Core rev: cec118406f3ad81cb4709f6e6ae1cef65799658e)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #15222]
In some situations its more practical to keep git configuration
at `/etc/gitconfig` instead of `$HOME/.gitconfig` (e.g., when mounting
git configuration into a docker container).
This change makes `devtool upgrade` consider any available
git configuration instead of only checking `--global`.
(From OE-Core rev: 30a9f7de45050c8bac49d4b37419cc2e067a75fa)
Signed-off-by: Marcus Flyckt <marcus.flyckt@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'imp' was deprecated in Python 3.4 and removed in 3.12. The
piece of importlib we use has been around since 3.3.
(From OE-Core rev: 457f0dad87b4e45a53865b5ad2c150215bd74019)
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently our wic test images boot up without kernel output on the consoles
which means we have no way to debug if anything goes wrong. Add the console
parameters runqemu would have added if the kernel wasn't built into an image
to improve our chances of debugging.
(From OE-Core rev: 63ea1efdaaf1173ef4f2f69b5a3403afef4b556a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Debug message about using custom configuration file includes file name
with incorrect extension. Correct file name to "extlinux.conf".
(From OE-Core rev: dd63e1520454b2d53a48b72eaae126059af9809b)
Signed-off-by: Daniel Semkowicz <dse@thaumatec.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake has changed to require notification when metadata changes in the middle of tinfoil
sessions. Add the required function calls at the places metadata is changed.
(From OE-Core rev: e5574163ab49a8f51b2b34fd37acfd1cad9b7595)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop some unused imports.
(From OE-Core rev: 432446ef402ff42fe0c90172b77376fa5981524a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support for formatting a disk with a hybrid MBR & GPT partition
scheme. In this scheme, the primary partitioning method is GPT, but a
valid MBR header is also written than can point to a subset of the GPT
partitions on the disk (any partitions marked with the `--mbr` flag will
be included in this MBR). The primary purpose of this method is to allow
for SoCs that can only find a bootloader in an MBR partition to use GPT
once the bootloader is running. As an example, older versions of the
Raspberry Pi firmware can only parse MBR partitions to find a kernel (or
other bootloader like u-boot), but once those have booted GPT partitions
can be used.
In addition to the partitions annotated with the `--mbr`, a "protective"
GPT partition of type 0xEE is added, as the existence of such a
partition is the indication to tooling that this a hybrid MBR and that
the GPT partition table should be parsed instead.
(From OE-Core rev: e50e4c2a5ada6947b3503ca4d8e9c30d359e8a5d)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that SRCPV isn't needed we can simplify things in a few places...
(From OE-Core rev: 843f82a246a535c353e08072f252d1dc78217872)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The kickstart parser defaults fstype to "vfat". This leads to an attempt
to create an empty file system even for regions configured with "--no-table"
if used without fstype when no --sourceparams given.
The fix tests for fstype "none" or no_table in Partition prepare method.
This will omit the file system creation an the potential error for small
region with --no-table option.
(From OE-Core rev: db771a4cd36bf291a8b68edfd905e03243f2c8b3)
Signed-off-by: Markus Niebel <Markus.Niebel@ew.tq-group.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Avoid a divide by zero traceback if unfortunate test counts are encountered.
(From OE-Core rev: c5aeea53dfacb53dedb8445cb3523dc3a8cb6dca)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Section VMA's are currently hardcoded. This doesn't work anymore starting
with systemd-boot v254.
Follow the actually solution to this which is documented here:
https://wiki.archlinux.org/title/Unified_kernel_image#Manually
This is also used by dracut. Later on, we may want to switch to ukify
instead but this is not ready yet.
(From OE-Core rev: 3abf99a6c6bde2fb8770f54dba609b35f6c6ee5a)
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whenever a test result is present in base test result but absent from
target test results, we have an entry in regression report looking like the
following one:
ptestresult.apr.testfile: PASSED -> None
The "None" status may be a bit confusing, so replace it with a custom
string which really states what "None" means in this context:
ptestresult.apr.testfile: PASSED -> No matching test result
(From OE-Core rev: b5f6da6e13f3484f51dfa07f6b3500aea7a21825)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a STATUS_STRINGS dictionnary matching raw statuses to custom strings.
Whenever a regression must be reported, raw status is searched in the
custom statuses dict (key search is case insensitive). If no custom string
is found, raw status is kept and used in regression report
(From OE-Core rev: 9d22bfc9d0c4092dba1af0ee11a4c51b7b270786)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With newly added recipe_qa task, the function needs to filter out its output as well.
The issue wouldn't appear in selftest because it's triggered by buildhistory and
selftest disables that.
(From OE-Core rev: da1284c990957827ee89685921c53b64490a32b2)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This can happen if running unpack task produces unrelated files and directories
(e.g. if recipe_qa or other tasks run); in this case it's better to stop, rather
than allow devtool to continue and error out much later with a message that has
nothing to do with where the problem originated.
The idea here was to handle tarballs that don't contain a top level directory and thus
the source tree is one level up; this basically never happens, and if it does we
should find a less brittle way to handle such tarballs.
(From OE-Core rev: b25e922b271794906b22450c7e6cc18fcab51ff8)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The immediate reason is that the task is sstate enabled, and due
to the way devtool sets up source extraction this causes bogus
values to appear in tmp/sstate-control/index-* files.
Also, recipe_qa may contain hard failures in the future, and it
would be counterproductive for the failures to prevent devtool
workflows (which may perhaps be about fixing the failure).
An alternative would have been to not insert recipe_qa task before
fetch task, but that would greatly delay the checks in larger
bitbake builds, and it's better to run them as early as possible.
(From OE-Core rev: 6f9db3f8de3dca9b0c87382561c70857d5b39e0f)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some regression reports show a lot of "PASSED->None" transitions. When such
big lot of identical transitions are observed, it could be that tests are
now failing, but it could also be that some tests has been renamed.
To detect such case, add a log in regression report to report the number of
new tests (i.e: tests that are present in target results but not in base
result). This new log also allows to know about newly added tests bases
(From OE-Core rev: 01b5cefd07e01c7407bc663842b8a8d502358a6d)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
non-release indexes will continue to generate when test output is
corrupted.
(From OE-Core rev: 1a9157684a6bff8406c9bb470cb2e16ee006bbe9)
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
NPM changed its manner to handle peer dependencies over its versions.
Before NPM 3: NPM installs automatically peer dependencies
between NPM 3 and 7: NPM shows a warning about peer dependencies
After NPM 3: NPM reworked its manner how to handle peer dependencies
The shrinkwrap doesn't have the parameters of the peer dependencies, so we cannot
fetch them. in the same time peer dependencies are not direct dependencies, they should
be installed as run time dependencies.
(From OE-Core rev: f36021a749974ef3d4a6abe4d5429544a815071a)
Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The shrinkwrap file changed its format, but npm does not version this file. So we can use it properly.
The actual changes make the script check if the npm package has dependencies in the actual shrinkwrap format.
(From OE-Core rev: 488d17c2af0c927ec66f0eee124bf6fc5b7f7c95)
Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Npm packages do not have yocto friendly names. fore instance we can have names like
"@example/npmPackage"
npm fetcher has a function that convert these names to yocto friendly names.
But in recipe tool we have an other function (duplicate).
(From OE-Core rev: 18e5438de5389b58c8b6a548d4474128d510a28d)
Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
native and nativesdk classes are special and must be inherited last :
put them at the end of the gathered classes to inherit.
(From OE-Core rev: a6614fd800cbe791264aeb102d379ba79bd145c2)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
devtool crashes when running "update-recipe" and append changes on the recipe.
"$ devtool update-recipe -a <layer> <recipe>"
Traceback (most recent call last):
...
File "/ovss/ovss_quanta/poky/scripts/lib/devtool/standard.py", line 1636, in srcuri_entry
return 'file://%s%s' % (basepath, paramstr)
^^^^^^^^
NameError: cannot access free variable 'basepath' where it is not associated with a value in enclosing scope
The input variable 'fname' should have the same meaning as the variable 'basepath'.
Modify the 'fname' to 'basepath' and solve the issue.
(From OE-Core rev: c3231756bbc2cb5641204414ad3670d7f8607ed3)
Signed-off-by: Charlie Wu <chiachiwu@google.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, uboot do_menuconfig task is breaking when UBOOT_CONFIG is
chosen rather than UBOOT_MACHINE, it simply fails with the following
errors:
| make: *** No rule to make target 'menuconfig'. Stio.
| Command failed.
| Press any key to continue...
this is due to the work directory of do_menuconfig is set to ${B} but
not ${B}/$config.
We should distinguish two situations:
1) When there is only one config item in UBOOT_CONFIG, do_menuconfig
should work just like how it works for UBOOT_MACHINE.
2) When there are multiple config items in UBOOT_CONFIG, do_menuconfig
should print out some information saying it's not supported other
than just failing.
This patch mainly aims to fix that by introducing a extra variable
KCONFIG_CONFIG_ENABLE_MENUCONFIG, it would be set to 'false' for
situation 2), and when it's set to 'true', then set
KCONFIG_CONFIG_ROOTDIR correctly in uboot-config.bbclass to let
do_menuconfig task work.
DEVTOOL_DISABLE_MENUCONFIG could be replaced by this new variable
KCONFIG_CONFIG_ENABLE_MENUCONFIG.
(From OE-Core rev: f9e834e317880cf47dbb4f8285bc36d743beae5e)
Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I've encountered issues reproducing initramfs and UKI image builds,
which will be fixed with this patch.
1. initramfs
There's a symbolic link to /sbin/init, which is appended to the cpio archive after creation.
The links timestamp needs to be static and the cpio append command needs the '--reproducible' flag to produce deterministic outcomes.
2. Unified Kernel Image
'--preserve-dates' is required for a static 'Time/Date' entry.
I've added '--enable-deterministic-archives' although in my case this
didn't change anything.
(From OE-Core rev: fd027729bafb4e085ba0949e38e724f3a8cad102)
Signed-off-by: Frieder Paape <frieder@konvera.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add argument --hidden to avoid MS Windows prompting to format
partition after flashing to a USB stick, SD card on another media.
Set Bit 0 (RequiredPartition) to mark that the partition is
required for the platform to function on GUID Partition Table
(GPT).
The new argument simplifies setting RequiredPartition on GPT
through a WKS file and the default imager plugin. Otherwise,
without this feature, to achieve the same result a new imager
plugin has to be implemented and set in WIC_CREATE_EXTRA_ARGS.
(From OE-Core rev: 7a111ff58d7390b79e2e63c8059f6c25f40f8977)
Signed-off-by: Leon Anavi <leon.anavi@konsulko.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use appropriate kernel image name instead of hard-coded vmlinuz for bootimg-pcbios plugin.
(From OE-Core rev: 8d9f00ba456fe76e0f4ef0e68ec64fc538c90d89)
Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a bit of a compatibility issue more than anything. Some devices
get upset if the FAT file system contains less blocks than the
partition.
The fixed-size argument is currently respected by the partition creation
step but not by the file system creation step. Let's make it so the file
system respects this value as well.
(From OE-Core rev: 38e1a235f5eceade7c871f96dc97f6c384384c7b)
Signed-off-by: Randolph Sapp <rs@ti.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
So far part.label has been used to define GPT partition label even if
part.part_name was defined.
Fix the code to use part.part_name whenever available, as it makes sense
to have a GPT partition label which is different from the contained
filesystem label.
(From OE-Core rev: 7704d5fc36eb065224792bf4d5543814eaa5fed3)
Signed-off-by: Dit Kozmaj <dit.kozmaj@kynetics.com>
Signed-off-by: Diego Rondini <diego.rondini@kynetics.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The logic in this script validates that the length of data sections are
evenly divisible by the block size. On most systems the block size is
4KB and all is good. Some systems though, such as ppc64le, have a block
size larger then 4KB. For example on a POWER9 based ppc64le system, the
block size is 64KB.
This results in this script failing with errors like this when building
wic images:
|440, in _do_get_mapped_ranges
| assert extent_len % self.block_size == 0
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| AssertionError
In this case the data section size was 268KB and the block size was
64KB, resulting in the above assert failure.
Resolves https://bugzilla.yoctoproject.org/show_bug.cgi?id=15075
(From OE-Core rev: 1e23b803af6991fc20e4a4e88a0ef0541399e722)
Signed-off-by: Andrew Geissler <geissonator@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we try to parse a buildstats directory which was either aborted or
is still being built then the top-level build_stats file doesn't
contain an elapsed value which causes an exception:
UnboundLocalError: local variable 'elapsed' referenced before assignment
Default both start and elapsed to 0 so that the parse succeeds.
(From OE-Core rev: 701d985aa8f2e9c2b9c0736fa25b424f3701889e)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the move to a ptest per image, the multiple ptest log scenario
is much more likely. Tweak the handling to only warn if there are
overlapping files.
(From OE-Core rev: 8391f1668fcbe932ba846299b0ec22de09e06dd4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a convenience argument to the log subcommand to list all of the
ptest logs in a testresults file.
(From OE-Core rev: cd2d7adf02005d46f19c93a40db1e10ce01ac261)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
ptestresult_get_log() looked for a key called 'ptestresuls.sections',
which should be 'ptestresult.sections'
(From OE-Core rev: 7c8c9f7283e54bf8b1521fbaad7dceb66a8fcdbb)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If it exists, there is no need to delete it, and if it does not,
devtool prints an ugly traceback.
(From OE-Core rev: af82e59e8f08369aabd5fa6eb43022982d4e59a7)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
resulttool regression module simply compare a base test status to a target test
result status. This approach raises many false positives since all XXX -> PASS
transitions (XXX being any status different from PASS) are flagged as
regression.
- Do not list XXX -> PASS transitions in regression report, instead count them
and print a summary of "newly passing tests"
- If an inspected pair has only "newly passing tests", do not print detailed
list and print it as "Improvement" instead of "Regression"
Updated output example looks like the following:
[...]
Improvement: oeselftest_fedora-37_qemux86-64_20230127010225
oeselftest_ubuntu-22.04_qemux86-64_20230226120516
(+1 test(s) passing)
[...]
Match: oeselftest_almalinux-8.7_qemuarm64_20230127015830
oeselftest_almalinux-8.7_qemuarm64_20230227015258
[...]
Regression: oeselftest_almalinux-9.1_qemumips_20230127000217
oeselftest_opensuseleap-15.4_qemumips_20230226130046
ptestresult.glibc-user.debug/tst-read-chk-cancel: PASS -> None
ptestresult.glibc-user.nptl/tst-mutexpi4: PASS -> FAIL
ptestresult.glibc-user.nptl/tst-mutexpi5a: PASS -> FAIL
Additionally, 44 previously failing test(s) is/are now passing
(From OE-Core rev: c335f96f687c73fde443ac330ca3e17113794d9e)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some ptests results may be wrongly sampled, resulting in some part of the error
being propagated in test name. For example:
"ptestresult.binutils-ld.in testcase /home/pokybuild/yocto-worker/qemumips/build/build-st-1666126/tmp/work/mips32r2-poky-linux/binutils-cross-testsuite/2.40-r0/git/ld/testsuite/ld-ctf/ctf.exp"
"ptestresult.gcc.Couldn't create remote directory /tmp/runtest.455781 on ssh"
"ptestresult.gcc-libstdc++-v3.Couldn't create remote directory /tmp/runtest.3814266 on target"
While the root errors must be fixed in corresponding ptests packages for those
tests, the test results history must still be usable even with this issue
Add new filters to detect if temporary test directories (build-st-*,
/tmp/runtime.*) are present in name. If so, truncate test name. As a side
effect, it will aggregate multiple failing errors into one, but the regression
will still be raised
(From OE-Core rev: 601eecfddd26bfe2954835a73ed1116bb520235f)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
ptests names not only need to be fixed for regression based on git testresults
but also for testsresults provided "manually"
Move ptests naming fixup in regression_common to share the fixup between both
regression use cases
(From OE-Core rev: f772ccd108dc3d618db9d479d672c0f3edd203ca)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some test case naming is poor and contains random strings, particularly
lttng/babeltrace but also curl. Truncating the test names works since they
contain file and line number identifiers which allows us to match them
without the random components, or in the case or curl, test IDs.
Going forward we may be able to improve the test names but this
tweak allows historical test results to work in reports.
(From OE-Core rev: 541a2e2683531355e678fd93524a0c4a8c43a8ff)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a test result contains LTP test results, it should only be compared with
other runs containing LTP test results.
(From OE-Core rev: 4dbbf2f4a85620a08dc2fa65095dc17fe6c530f8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When generating regression reports, many false positive can be observed since
some tests results are compared while the corresponding tests sets are not the
same, as it can be seen for example for oeselftest tests (oeselftest is run
multiple time but with different parameters, resulting in different tests sets)
Add a filtering mechanism in resulttool regression module to enable a better
matching between tests. The METADATA_MATCH_TABLE defines that when the TEST_TYPE
is "oeselftest", then resulttool should filter pairs based on
OESELFTEST_METADATA appended to test configuration. If metadata is absent from
test results, in order to keep compatibility with older results, add a
"guessing" mechanism to generate the missing OESELFTEST_METADATA. The guessed
data is tightly coupled to the autobuilder configuration, where all oe-selftest
executions are described
(From OE-Core rev: 94ab7c2b892bf292dd86619ca9c63ddd7bf53f3c)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wic plugin crashed if IMAGE_EFI_BOOT_FILES is not empty and uefi-kernel
loader specified because of preliminary return from the function.
[Yocto #15033]
Fixes:
| File "/mnt/builds/yocto/sources/scripts/lib/wic/plugins/source/bootimg-efi.py", line 371, in do_prepare_partition
| for src_path, dst_path in cls.install_task:
| ^^^^^^^^^^^^^^^^
| AttributeError: type object 'BootimgEFIPlugin' has no attribute 'install_task'. Did you mean: 'do_install_disk'?
(From OE-Core rev: a6372f70f31b39ce9867b705d02205621d7a8685)
Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
yocto-check-layer to check all .patch file in layer for
Upstream-status and list down all .patch file without Upstream-Status.
set this test as expected failure for now, so this wont fail the check
layer while still able to capture any patches with malformed or missing
Upstream-Status in report.
[YOCTO #14642]
(From OE-Core rev: 237c1b66e5014123c1e5c3e78f9ab0357bcd62dc)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
So that patch fuzz issues can actually be fixed,
as extracting source with 'devtool modify' is the first step
for that.
(From OE-Core rev: 7067abd31d9dd4b98ec70c1c7effbe2904797cd1)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a task is aborted the buildstats file isn't complete, so calculate
when the build finished and use that as a end time.
(From OE-Core rev: 23ebaec476dc46aebe5997f025661137f3e341bd)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows to specify partition with fstype=none in the wks file
to have partition created but without following mkfs. The none fstype
is in the list already but the usage is not documented.
Example;
part /data --ondisk mmcblk0 --fstype=none --align 4096 --fixed-size 512
will create a partition, filesystem may be created manualy on the host
or target and data will be preserved if the device is reflashed using
same wks. Works with bmaptool and probably does not work with dd.
Use case is persistent filesystem/data between reflashing of the image.
(From OE-Core rev: 351cb64da37aa43113e5192605d04436652aa3b8)
Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* True is default since 2016 and most layers were already updated
not to pass this parameter where not necessary, e.g. oe-core was
updated couple times, first in:
https://git.openembedded.org/openembedded-core/commit/?id=7c552996597faaee2fbee185b250c0ee30ea3b5f
Updated with the same regexp as later oe-core update:
https://git.openembedded.org/openembedded-core/commit/?id=9f551d588693328e4d99d33be94f26684eafcaba
with small modification to replace not only d.getVar, but also data.getVar as in e.g.:
e.data.getVar('ERR_REPORT_USERNAME', True)
and for getVarFlag:
sed -e 's|\(d\.getVarFlag \?\)( \?\([^,()]*, \?[^,()]*\), \?True)|\1(\2)|g' \
-i $(git grep -E 'getVarFlag ?\( ?([^,()]*), ?([^,()]*), ?True\)' \
| cut -d':' -f1 \
| sort -u)
(From OE-Core rev: 26c74fd10614582e177437608908eb43688ab510)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
oe-selftest efibootpartition.GenericEFITest.test_boot_efi was failing for 6.1 kernels with:
| ERROR: _exec_cmd: export PATH=[...] mcopy -i [...]/rootfs_boot.1.vfat -s [...]/rootfs1/* ::/ returned '1' instead of 0
| output: Disk full
I believe we hit a file boundary size and having "0" overhead in the image meant
the files couldn't be installed. Allow a small amount of overhead to avoid
the error.
(From OE-Core rev: 16e0b8a8fc36f5525b1801888851958f0dbe84c2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
devtool modify/upgrade are not currently equipped to handle conditional local files
in SRC_URI, and provide only the main no-override set in a workspace under
source/component/oe-local-files/ (this is done via meta/classes/devtool-source.bbclass).
On the other hand, updating the changes from workspace into a recipe
is run iteratively against all overrides; this works for patches (as they
all are directed into their own override branches in the workspace
git source tree), but breaks down when trying to match local files
in a workspace against local files in overridden SRC_URI lists, resulting in
bad recipe breakage.
(there's an additional twist here: existing code has a guard against this
but the guard relies on metadata in workspace .bbappend that is only there
in modify operations, but not upgrades. This commit replaces the guard
with a general check that will work everywhere).
Implementing multiple sets of local files is significant work; let's for now
simply not touch local files in recipes except when on the no-override variant.
Also, adjust the selftest cases to include conditional local files in sample
recipes, so the situation is covered by the tests.
(From OE-Core rev: 3a8654b860fa98f94e80c3c3fff359ffed14bbe7)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'devtool modify' writes additional settings to workspace .bbappend so that this
can be handled correctly, but 'devtool upgrade' does not. This adds the missing
settings.
In particular, local files should not anymore mysteriously disappear from
SRC_URIs on upgrades.
(From OE-Core rev: 0817aa5537a8d7cc9591c53dfaa1d225f4c327f7)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For unknown reasons we've never seemingly run the check layer script
against OE-Core itself. This isn't entirely straightforward as the core
layer is a bit of a special case, we can't for example compare signatures
against ourselve and we can't remove core from bblayers.conf.
Core does have distro, machine and software components too, in the case
of distro, our fallback default settings. Whilst the qemu machines could
be split into a seperate layer directory, core wouldn't then parse at all
standalone due to the lack of any machine so it seems a bit pointless to
do that.
These changes tweak the script to handle core's special cases, specifically
to allow distro and machine directories and to account for the README placed
a directory level higher than other layers.
(From OE-Core rev: ba312ed228507d05f280aeb96819d671b01400b8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake additions for the addpylib API mean we need to update the parsing
function call to be clear we're parsing in configuration context.
(From OE-Core rev: ef7677dc90fac089f8b9f6da301cca022ed7284c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The /proc/pressure support in buildstats is creating directories in the
buildstats tree called reduced_proc_pressure, which confuses the parsing
logic as that cannot be parsed as a name-epoc-version-revision tuple.
Explicitly skip this directory to solve the problem.
(From OE-Core rev: 24f0331f0b7e51161b1fa43d4592b491d2037fe9)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ext2/3/4 FS contains not only mtime, but also ctime, atime and crtime.
Currently, all the files are being added into the rootfs image using
mkfs -d functionality which affects all the timestamps excluding mtime.
This patch ensures these timestamps inside the FS image equal to
the SOURCE_DATE_EPOCH if it is set.
(From OE-Core rev: 75d2dd0ea7790db2e8ee921784ca373abff2df65)
Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In case user requested to build a binary repeatable package,
it's required to honor the SOURCE_DATE_EPOCH environment
variable. So forcefully set mtime inside all the routines
which modify fstab in case it is updated.
(From OE-Core rev: 99719a3712a88dce8450994d995803e126e49115)
Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When SOURCE_DATE_EPOCH variable is set, binary repeatable build
is expected. This commit implements reproducable disk identifiers
in such a case using its value as a Random seed.
(From OE-Core rev: 2c0c54e12169e76f16fb3398904bc897a9190397)
Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
--include-path can be used for placing files on ESP,
like dtbs.
(From OE-Core rev: 46ca73f504b0792766dc4d4bbad38beb04ef4ed0)
Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit 20d43a2 allows changes in fstab for the / mount. However, if the
fstab contains / already then this results in two entries for / in the
fstab, which is confusing and results in systemd producing errors on
boot:
systemd-fstab-generator[11101]: Failed to create unit file
/run/systemd/generator/-.mount, as it already exists. Duplicate entry
in /etc/fstab?
Wic should be extended to support merging mount points in fstab, but as
we're about to release revert the patch that introduced this change as
it hasn't been in a release yet, and the issue which prompted the patch
in the first place can be revisited post-release.
[ YOCTO #14865 ]
(From OE-Core rev: 4a7b3ef0a69b7e08030519b712a6fe4dd199c571)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These are typically auto-extracted with modify/upgrade from recipes
and can be easily recreated. On the rare occasions where they need
to be reused, they are still available under workspace/attic (which
is already used for old recipes and appends), so nothing gets lost.
This avoids the annoyance of devtool refusing to proceed because
there is a previous source tree in workspace/sources.
For independent source trees behave as before: do nothing.
Adjust the test that previously deleted those trees by hand.
(From OE-Core rev: 9bfb95d070d68d5ab5adfe0ea096f5fbf9cad8b0)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's not possible to set the label (of gpt entry) normally
when I want to use non-listed fstype as a rawcopy.
Example)
part ? --source rawcopy --ondisk mmcblk0 --label mypart --sourceparams file=mypart.raw
To resolve this problem, this patch addes a 'none' fstype
and ignore do_image_label on rawcopy (that actually set the partition label.)
(From OE-Core rev: 949cf797eb54f3d22a093910087e937633ce9b23)
Signed-off-by: JeongBong Seo <jb.seo@lge.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Where there isn't a copyright statement, add one to make it explicit.
Also drop editor config lines where they were present and add license
identifiers as MIT if there isn't one.
(From OE-Core rev: deb3ccec53e0bd63bc4235cf2b0d3fc781687361)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a coding error in the second invocation of _upgrade_error:
rf is passed into it before it is initialized in the try: block. And so
bogus recipes are left behind in the workspace, causing breakage.
Instead, rewrite the functions to take the recipe directory name in the
workspace layer, which can be calculated in advance.
(From OE-Core rev: e653996369c1d2b5ac8367ad85f4816d679b6c98)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For device tree using systems, add support to set a custom devices tree
during UEFI boot. This requires to copy the DTB file to the boot
partition and to add the respective loader entries to the configuration
files.
Both grub and systemd-boot support only loading a specific device tree.
Therefore refuse to work if the 'dtb' parameter contains more than one
entry.
Out of scope for now are overlays (only supported by systemd-boot).
(From OE-Core rev: c5a14f39a6717a99b510cb97aa2fb403d4b98d99)
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
[Luca: rebased on commit 98e9d435b278 ("wic/bootimg-efi: use cross objcopy when building unified kernel image")]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The paths for configuring grub and systemd-boot have some common bits
around copying the initrd files. This will even grow when adding dtb
support. Factor this out into a class function.
Along this, avoid evaluating 'create-unified-kernel-image' multiple
times in do_configure_systemdboot and suppress a bogus warning about
"Ignoring missing initrd" when it is turned on.
(From OE-Core rev: c700cfd88473b9ed4e12a6620fb089f41bd95a9e)
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We can't rely on the host objcopy knowing how to process target binaries,
so use the cross objcopy in the sysroot instead.
Also construct the command argument-by-argument as the format expression
was getting unwieldy.
(From OE-Core rev: 0264aeedbf21e9e7a104243c11b3b57f00e38bda)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We might want to run a cross tool, such as objcopy, in wic. These are
in a TARGET_SYS/ subdirectory under /usr/bin, so add that directory to
the search path too.
(From OE-Core rev: c523549141e5c31edc75281f581d97867b7d251d)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix "NameError: name 'orig_path' is not defined".
It's a typo from when this error was handled outside this function.
(From OE-Core rev: 2124ec0d9f9de2da476f0024a0ccf70da987420f)
Signed-off-by: Mihai Lindner <mihai.lindner@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If devtool finish needs to create a patch and have it applied to the
sources for a recipe where S points to a subdirectory of the sources,
then the patch needs to be applied at the root of the repo i.e. we need
to add a patchdir= parameter to the SRC_URI entry.
(From OE-Core rev: ad3736d9ca14cac14a7da22c1cfdeda219665e6f)
Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If (perhaps foolishly) at your configuration level you have e.g.
SRC_URI_append_pn-recipename = " file://patchname.patch"
and then run devtool modify on a different recipe, an error occurs:
INFO: SRC_URI contains some conditional appends/prepends - will create branches to represent these
...
ERROR: [Errno 2] No such file or directory: '/path/to/downloads/patchname.patch'
pn- overrides would not constitute an alternative configuration that we
should handle in this context, so just ignore them to avoid the issue.
(From OE-Core rev: 3f2a812ade42ece0bb59b2d303125a91b29936dd)
Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Substitute expressions or whitespace from python egg requires.txt when
generating PACKAGECONFIG
Pysetuptools sees the uvicorn.egg-info/requires.txt as extra requirements.
Recipetool parses this information to generate the PACKAGECONFIG.
These extra requirements contain expressions and whitespace, which are not allowed in PACKGAGECONFIG.
This patch substitute them by hyphens to make PACKAGECONFIG parsable and readable.
Also adding an oe-selftest for this.
[YOCTO #14446]
(From OE-Core rev: a854d95a79e64f3f82abfa4cc1daec750abf4249)
Signed-off-by: Thomas Roos <throos@amazon.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow wic to also manipulate the rootfs entry in fstab, which it
currently refuses to write. Reasons one might want to do that include
using systemd-growfs via --fsoptions on /
With this change / is now handled exactly the same as other
mountpoints, the former exception seemingly was not even documented.
(From OE-Core rev: 20d43a2599d7622b96e2fb0da87a886da1a3794a)
Signed-off-by: Tobias Schmidl <tobiasschmidl@siemens.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The `fspassno` parameter allows to overwrite the value of the last
column (`fs_passno`) in the /etc/fstab of the target root file system.
This allows to have periodic file system checks.
(From OE-Core rev: b9b9f71e6f37bfbf954ade518391b242669481e3)
Signed-off-by: Claudius Heine <ch@denx.de>
Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patches makes locating the file database containing the file and
folder usernames and permissions more reliable.
In addition to locating it relative to the partition directory, we also
try to locate it relative to the IMAGE_ROOTFS.
Prior to this patch, the database was not found when using
--rootfs-dir=${IMAGE_ROOTFS}/<x> in the WIC script, leading to erronous file
permissions and ownership.
(From OE-Core rev: 09e18ee246da8b56f446c4db548fb9c7e895142b)
Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to build erofs filesystems, wic must have the erofs-utils
package installed into its sysroot.
Fixes: 30375ce97 ("Add support for erofs filesystems")
(From OE-Core rev: 68e364340c439a1341d37c3f7a2b0e6aad8e1e56)
Signed-off-by: Sean Anderson <sean.anderson@seco.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using "msdos" partition tables and "--label" but not "--use-uuid"
one can generate images which will not find their root, because
PARTLABEL does not work for "msdos".
Fix that by simply not going the PARTLABEL path in case of "msdos".
Fixes: 2fb247c5ecf0 ("wic: support rootdev identified by partition label")
(From OE-Core rev: 9ea1a838b946020e026edc032039552b723fcaa4)
Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove mutable default arguments in Python because they can lead to all
sorts of nasty and horrible bugs.
https://florimond.dev/en/posts/2018/08/python-mutable-defaults-are-the-source-of-all-evil/
Revert `recipetool: Change default paramter fallback_licenses of
function split_pkg_licenses from None to []` and instead check
fallback_licenses before use.
(From OE-Core rev: 99dee60b8db557f54783bf0f61098587badc683c)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some of these are hard to know what to do with since the original source
files for the checksum aren't present. The safe option is to use "-only"
as often the main license is ambiguous and the source files themselves
determine the "or-later" possibility. The "-only" option therefore is
realistically what we need to use in this code.
(From OE-Core rev: 2b0cbafc7854de0308a624b17b8aaba704b031d5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are still a couple of cases where the license may be set as, e.g.,
"GPL" or "GPL-2.0" since there is not enough information to decide the
actual SPDX license. It is then assumed that the developer will have to
correct the information.
(From OE-Core rev: e7df51f8d2361e9fe2d67669d2e17f0a5d01004e)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add all OSI approved licenses from https://pypi.org/classifiers/. Also
add support for Other/Proprietary (Proprietary) and Public Domain (PD).
(From OE-Core rev: 99ef134d1019e5b98b845cf71f3eb39871218f9d)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After the change to bitbake, update the references in OE-Core to match the updates.
(From OE-Core rev: 193affb9f28b0116c3fd619834f145326fee08c5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is useful when needing to test layer's recipes, where this special
bbclass can define a global python function that gets called on each
recipe parsing during "bitbake -S none world" signature dump and be able
to fail layer's check accordingly.
First test being added is to detect recipes skipping "installed-vs-shipped"
QA check. As "installed-vs-shipped" is a packaging QA check, it happens very
late in the build process and failing it could mean some potential issues
with packaging, especially when recipe uses BBCLASSEXTEND="nativesdk" and
resulting package is used in an SDK.
In OE-Core failing this QA check leads to an error, but other layers can
suppress it or change it to a warning. Detecting weird packaging problems
with SDKs is quite difficult and time consuming. Also, waiting for the
actual "installed-vs-shipped" packaging QA check to fail means that all
recipes in the layer under test have to run through all standard tasks in
the build chain, equivalent to a multi-hour world-build.
Hence yocto-check-layer takes a shortcut and only detects a mere attempt
at skipping "installed-vs-shipped" QA check in the INSANE_SKIP list during
initial parsing when dumping the signature information for the layer.
(From OE-Core rev: e8baa75535fc888f1d768b23a0140475e832c910)
Signed-off-by: Denys Dmytriyenko <denis@denix.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In particular this resolves devtool's inability to pick a branch
when the same tagged commit is avaiable in main and in a release maintenance
branch.
Thanks to Peter Kjellerstedt for the suggestion.
(From OE-Core rev: bcb21ee2760a2c76039412a56c6cda43fbf96fd0)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support for packed images to wic rawcopy handler do minimize disk
usage in deploy directory and reuse of packed images between wic and
swupdate. Add `unpack` to sourceparams to unpack an bz2, gz and xz
archives.
Example:
part / --source rawcopy --sourceparams="file=core-image-minimal-qemu.ext4.gz,unpack"
(From OE-Core rev: 4c97d25791389ece041565981ba3207ce9949a1a)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Accept valueless keys in sourceparams without equals sign (=) to match
the comment and support Boolean entries.
(From OE-Core rev: 052a9b0f7d2ee5c284cf1b4f1f45caed0489221a)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake is now requiring there be a call to get_srcrev() when urls contain
floating revisions. This breaks some recipetool workflows since it is running
these queries to determine those revisions. Add a PV entry which triggers
that call to avoid the errors.
(From OE-Core rev: bdaf0c798da21f9c15d9cb30ab4258a3c75c8bbf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
deploy-target may fail the second time with "pseudo abort" because
devtool-deploy-target-stripped is deleted outside of pseudo's fakeroot
context.
(From OE-Core rev: 2338a33b690b0bbe279cde3f73764911b239cb50)
Signed-off-by: Florian Amstutz <florian.amstutz@scs.ch>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
detect_layers() is very greedy and if it recurses into poky or bitbake
it will find the test suite layers, such as
bitbake/lib/layerindexlib/tests/testdata/layer4. This is a dummy layer
which claims to be openembedded-layer, so if the real openembedded-layer
is a dependency then layer4 may be used instead, which will cause
errors: initially because it's only compatible with Sumo, but later
because it doesn't contain any recipes.
Add a check that the set of layers we've found doesn't contain any
duplicate collection names with different patterns, and abort if that is
the case as the test will be non-deterministic.
(From OE-Core rev: 0df4bae4ec67d38442620fa08c839528b425e2a8)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When creating a new recipe.bb file for a binary, don't use BP which
includes the version information, instead use BPN which is just the
name base Package Name.
Since PB is not specified, it takes the default:
PV = "1.0+git${SRCPV}"
But SRCPV is defined in terms of the SRC_URI, which leads to infinite
recursion (traceback below). Here are the pertinent variables which
cause the recursion:
SRC_URI = "git://github.com/lvc/abi-dumper;protocol=https;subdir=${BP}"
BP = "${BPN}-${PV}"
PV = "1.0+git${SRCPV}"
SRCPV = "${@bb.fetch2.get_srcrev(d)}"
def get_srcrev(d, method_name='sortable_revision'):
# ... trimmed
scms = []
fetcher = Fetch(d.getVar('SRC_URI').split(), d)
# ... trimmed
[YOCTO #14040]
(From OE-Core rev: 3b8d43fc53ee13d39abc3b2a1f706a97fcf752aa)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
