Make the deploy function independent from d. This allows to call the
function also from Python code not running in bitbake.
This is needed to for the devtool ide plugin which will call the
do_install task and the code from devtool deploy-target independently
from a bitbake server. This allows a much quicker workflow.
(From OE-Core rev: c8697d1132cbd4b2a2502b4c48e7d91fc18de786)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a step towards a deploy function which can be called without
passing the d variable.
(From OE-Core rev: 3e8c41ec373a1766f68932d3b979d00e4e3bb3a9)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Provide a function exec_fakeroot_no_d which does the same like
exec_fakeroot does, but is usable independenlty from bitbake. This
allows to use the fanction from scripts where the d variable is not
available.
(From OE-Core rev: 6f5980d96844559639fc3660bb975b0bdd1aa523)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Pass the plugins search path to plugins via context. This allows plugins
to search for their own plugins at the same paths.
(From OE-Core rev: e9c3c996f2e9a52462c4426f394798a211e53ae1)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[YOCTO #15162]
when doing devtool modify, sources are extracted into a devtool
temporary workdir. The main source is moved inside
build/workspace/sources/${BPN}/ and local files are moved inside
build/workspace/sources/${BPN}/oe-local-files. Secondary sources are
currently not handled and are lost.
Here is the output of devtool modify/build on bzip2 recipe:
NOTE: bzip2: compiling from external source tree <...>/build/workspace/sources/bzip2
ERROR: bzip2-1.0.8-r0 do_install_ptest_base: ExecutionError('<...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368', 1, None, None)
ERROR: Logfile of failure stored in: <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/log.do_install_ptest_base.3368
Log data follows:
| DEBUG: Executing shell function do_install_ptest_base
| NOTE: make -j 16 DESTDIR=<...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest install-ptest
| sed -n '/^runtest:/,/^install-ptest:/{/^install-ptest:/!p}' \
| ../../../../../../workspace/sources/bzip2/Makefile.am > <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/Makefile
| cp ../../../../../../workspace/sources/bzip2/sample1.ref <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
| cp ../../../../../../workspace/sources/bzip2/sample2.ref <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
| cp ../../../../../../workspace/sources/bzip2/sample3.ref <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
| cp ../../../../../../workspace/sources/bzip2/sample1.bz2 <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
| cp ../../../../../../workspace/sources/bzip2/sample2.bz2 <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
| cp ../../../../../../workspace/sources/bzip2/sample3.bz2 <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/
| ln -s /usr/bin/bzip2 <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/bzip2
| cp: cannot stat '<...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/git/commons-compress': No such file or directory
| WARNING: <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368:189 exit 1 from 'cp -r <...>/build/tmp/work/core2-64-poky-linux/bzip2/
1.0.8/git/commons-compress <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/image/usr/lib/bzip2/ptest/bzip2-tests/commons-compress'
| WARNING: Backtrace (BB generated script):
| #1: do_install_ptest, <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368, line 189
| #2: do_install_ptest_base, <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368, line 158
| #3: main, <...>/build/tmp/work/core2-64-poky-linux/bzip2/1.0.8/temp/run.do_install_ptest_base.3368, line 226
ERROR: Task (<...>/poky/meta/recipes-extended/bzip2/bzip2_1.0.8.bb:do_install_ptest_base) failed with exit code '1'
NOTE: Tasks Summary: Attempted 776 tasks of which 765 didn't need to be rerun and 1 failed.
Summary: 1 task failed:
<...>/poky/meta/recipes-extended/bzip2/bzip2_1.0.8.bb:do_install_ptest_base
externalsrc class modify SRC_URI to keep only:
* 'file', 'npmsw' and 'crate' sources
* url with type parameter matching 'kmeta' or 'git-dependency'
So by forcing to add type='git-dependency' on secondary sources, we
ensure that when building the recipe, the secondary sources can be
unpacked into WORKDIR.
This allows recipes containing several sources to be built under a
devtool context, but it has some limitations:
* user would not be able to generate patches for the secondary sources
* type="git-dependency" is added for secondary sources even on non git
sources, so we may want to rename this parameter
(From OE-Core rev: cfd5ee890163a3d975093359016dda104e7b71df)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Major module version is a part of name, but not necessary part of the
actual URL (See https://go.dev/ref/mod#module-path).
Nevertheless, name detection function can't handle that suffix, so get
rid of it to determine component name.
For replaced modules that name might be different that the actual module
name defined in go.mod file.
(From OE-Core rev: 0cccfa1041d48f0ae3a2dc89a129cf7884fc08f0)
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whenever the recipe uses a CLOSED license, the list is going to be
empty. It's a discouraged practice not to have a license, but proceed
anyway to finish recipe generation.
(From OE-Core rev: 5ca920284d0946346f5b06f5e443c80d9d8b85ce)
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Local modules are usually referenced with a 'replace' directive in
go.mod file. If that's the case, remove them from populating SRC_URI.
(From OE-Core rev: 9f220f61e3e44a650a46ee997b47f1d87b7c4ef0)
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Root Cause:
initial_revs is an empty dictionary and do not have "." key.
Traceback (most recent call last):
File "scripts/devtool", line 349, in <module>
ret = main()
File "scripts/devtool", line 336, in main
ret = args.func(args, config, basepath, workspace)
File "scripts/lib/devtool/standard.py", line 922, in modify
if not initial_revs["."]:
KeyError: '.'
Solution:
check key exists, then get its value.
(From OE-Core rev: fb0db5c48abb4d56233a175fdd349d18b972e452)
Signed-off-by: Jamin Lin <jamin_lin@aspeedtech.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Also change path.exists() and !path.isdir() to a single
path.isfile() which should be equal.
Enables running tests against image recipes which are not called
"bla-image" but plain "bla". Currently they fail with do_testimage/runqemu
error:
runqemu - ERROR - Unknown path arg /home/builder/src/base/build/tmp_qemuarm64/deploy/images/qemuarm64/img-qemuarm64.rootfs.wic
Suggested-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(From OE-Core rev: 107d0db3dc3678e6f0ae4035f4c0b86c6b421168)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When invoking runqemu with Python 3.12, the following warning is
encountered:
|SyntaxWarning: invalid escape sequence '\.'
This is because the interpreter scans the string before it is processed
by the regex module, and it interprets the backslash as part of an
escape sequence, but not a standard one. This will be registered as an
error rather than a warning in future Python versions. To avoid the it,
simply add an extra backslash so that Python doesn't misinterpret the
string, while the regex parser still sees an escaped '.' character.
(From OE-Core rev: 0e8a4142bb90a92d175df6b2537d24a372356f98)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'devtool check-upgrade-status' is for reporting upgradable
status for recipes. The output should always be printed out. So
we should just use 'print' instead of 'logger.info' as the latter
will be suppressed if '-q' parameter is supplied to devtool.
(From OE-Core rev: 2c7bf9c8a833bec13a1ebabdce30933cbe691108)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since there is a faster python version, drop the slower shell one.
(From OE-Core rev: 0551fa2ae3dacf51825ff5c1d081e713a150ddd2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This (should be) a drop in replacement for sstate-cache-management.sh.
(From OE-Core rev: 2fa1b25d7485bfbb92bcc33067beb6751218b36a)
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
E2FSPROGS_FAKE_TIME: sets fixed times for the inodes in the file system.
hash_seed: creates reproducible directory indexes in the file system.
Reference commit in e2fsprogs: e1f7100643a46456be107b33098f6034b0835e6d
(From OE-Core rev: bb822ab75de0020572058090439b93cc56bbf7e0)
Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the new python_maturin PEP-517 backend
Add selftest for 'pydantic-core' pypi package.
(From OE-Core rev: 69b679380616a94a631681caa05d9bf7610f9372)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoids the following error when running `devtool modify` on a
recipe that has a menuconfig task, but does not have
KCONFIG_CONFIG_ENABLE_MENUCONFIG set.
.../temp/run.do_configure.4163366: line 152:
${@ oe.types.boolean('${KCONFIG_CONFIG_ENABLE_MENUCONFIG}') }: bad substitution
WARNING: .../temp/run.do_configure.4163366:152 exit 1 from
'[ ${@ oe.types.boolean('${KCONFIG_CONFIG_ENABLE_MENUCONFIG}') } = True ]'
(From OE-Core rev: e87e6fa84a0c4b5ac8e736dc62f6e08390ba2436)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support for qmp sockets and defaults to unix:qmp.sock if unspecified
(From OE-Core rev: 380631797f0d63124a8c21efa93ab672dbd79283)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The pypi change:
"85a2a6f68af recipetool: create_buildsys_python: add pypi support"
deleted all the SRC_URI variables, including the SRC_URI checksums.
These are not generated by the pypi.bbclass (how could they be trusted?)
Without the checksum(s), we are vulnerable to a man-in-the-middle attack
and zero checks on the validity of the downloaded tarball from pypi.org.
Fix by only setting S and SRC_URI to None.
(From OE-Core rev: 560181a52111569f7bc57b09139b42510e0d0325)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoids the following error when using --no-extract, introduced in
commit 900129cbdf (devtool: add support for git submodules):
Traceback (most recent call last):
File ".../scripts/devtool", line 349, in <module>
ret = main()
File ".../scripts/devtool", line 336, in main
ret = args.func(args, config, basepath, workspace)
File ".../scripts/lib/devtool/standard.py", line 995, in modify
for commit in commits[name]:
KeyError: '.'
(From OE-Core rev: 6a44fb10ef4b90ee24f470362007d56183abc593)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In addition to updating the sha256sum and removing the md5sum, update
all other existing checksums. If the only existing checksum is md5sum,
then replace it with the default expected checksums (currently only
sha256sum).
(From OE-Core rev: 8ea8827ee49b7f0443b1c4bd47d1344a689d73a3)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than including all SRC_URI checksums, include the ones that are
expected. These are the same as are output if no checksums are included
when building the recipe.
(From OE-Core rev: c2af83eb5e8573480179b6c0bcce50606b547099)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than having a static list of crunched MD5 checksums for some of
the most common licenses, calculate it for all common licenses. This
should improve the identification of license text variantions.
(From OE-Core rev: 377f9513dc56e9b8e5f5813c1535be0206756949)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Try to execute both gtk-update-icon-cache and
gtk4-update-icon-cache after checking whether the
commands are available.
This attempts to match what gtk-icon-cache.bbclass is doing.
This fixes running update_gtk_icon_cache during do_rootfs
for an image that contains only GTK4 related packages.
(From OE-Core rev: 1e01ef59a5864f1261e9c0ca76fcccee2eb7e7f3)
Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a new option -u/--update-recipe to update the recipe itself instead
of creating/updating a bbappend file
(From OE-Core rev: c9a80f55ed0bf9ce249205d6a9ba670fbb7f59ba)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
appendsrc function relies on oe.recipeutils.bbappend_recipe to
copy files and add the corresponding entries in SRC_URI.
Currently, appendsrc function build itself the new SRC_URI entry to add the
correct subdir param, and gives it using the extralines parameter.
This has 2 drawbacks:
- oe.recipeutils.bbappend_recipe can already do this if we specify the
correct params, so we have duplicate code
- the duplicated code is not fully functional: for example, it doesn't
take into account the -m/--machine parameter
So fix this by not using extralines but give correctly formatted params.
Also remove the check for already existing entries as
oe.recipeutils.bbappend_recipe already implement it
The new bbappend file now have the SRC_URI entry after the
FILESEXTRAPATHS so fix the selftest.
Update test_recipetool_appendsrcfile_existing_in_src_uri_diff_params
test because recipetool appendsrcfiles used to not add new src_uri entry
if the entry already exist even with different parameters while
oe.recipeutils.bbappend_recipe adds it if parameters are different (and
remove the old entry)
(From OE-Core rev: cd5de8d53849a6f3bb6f82e45fb301e39892c789)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add dry-run mode for recipetool appendsrcfile and appendsrcfiles, with
if necessary, a diff of changes
(From OE-Core rev: 456c726e6b52f4dc57ff605e1cf1687097537002)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds features to explicitly write zeros to the start of the
partition. This is useful to overwrite old content like
filesystem signatures which may be re-recognized otherwise.
The new features can be enabled with
'--soucreparams="[fill|size=<N>[S|s|K|k|M|G]][,][bs=<N>[S|s|K|k|M|G]]"'
Conflicting or missing options throw errors.
The features are:
- fill
Fill the entire partition with zeros. Requires '--fixed-size' option
to be set.
- size=<N>[S|s|K|k|M|G]
Set the first N bytes of the partition to zero. Default unit is 'K'.
- bs=<N>[S|s|K|k|M|G]
Write at most N bytes at a time during source file creation.
Defaults to '1M'. Default unit is 'K'.
(From OE-Core rev: d19d4529e7a2056caeb526fed980cc1df19a5f6e)
Signed-off-by: Malte Schmidt <malte.schmidt@weidmueller.com>
Signed-off-by: Lukas Funke <lukas.funke@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Today, we can use devtool/recipetool to create recipes for python projects
using the github url or the direct release tarball of the project, but the
create_buildsys_python plugin doesn't support the pypi class, since we cannot
know from the extracted source if the package is available on pypi or not.
By implementing the new optional process_url callback, we can detect
that the url is a pypi one (i.e 'https://pypi.org/project/<package>')
and retrieve the release tarball location.
Also detect if the url points to a release tarball hosted on
"files.pythonhosted.iorg" (i.e https://files.pythonhosted.org/packages/...)
In both cases, adds the pypi class, remove 'S' and 'SRC_URIxxx'
variables from the created recipe as they will be handled by the pypi class
and add the PYPI_PACKAGE variable
This helps to produce cleaner recipes when package is hosted on pypi.
If the url points to a github url or a release tarball not coming from
"files.pythonhosted.org", the created recipe is the same as before.
One can also use the newly added "--no-pypi" switch to NOT inherit
from pypi class on matching url, to keep legacy behaviour.
To create a recipe for a pypi package, one can now use one of the
new following syntax (using recipetool create / devtool add):
* recipetool create https://pypi.org/project/<package>
* recipetool create https://pypi.org/project/<package>/<version>
* recipetool create https://pypi.org/project/<package> --version <version>
or the old syntax:
* recipetool create https://files.pythonhosted.org/packages/<...>
(From OE-Core rev: 097a43846cd99a7d74d004efc57f583ce78970a4)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a new process_url callback that plugins can optionally implement if
they which to handle url.
Plugins can implement this callback for example, to:
* transform the url
* add special variables using extravalues
* add extra classes
* ...
If a plugin handles the url, it must append 'url' to the handled
list and must return the fetchuri
No functional changes expected for plugins non implementing this
optional callback
(From OE-Core rev: 3e5ce351e7dbe283562bf1db1f2ce3b121c49b53)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
create_recipe() function relies on oe.recipeutils.patch_recipe_lines()
which relies on bb.utils.edit_metadata(). edit_metada expect lines to
have trailing newlines, so add it to each lines before calling
patch_recipe_lines, otherwise edit_metadata will not be able to squash
blank line if there are two consecutive blanks after a removal
(From OE-Core rev: 31b27cc76a1b669e2b126c332608fd772c124d69)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the case pyproject.toml doesn't contains metadatas, the metadata
variable is not initialized and the plugin throws an error and falls back
to another plugin, which is not the desired behaviour. So just ignore
metadata if we don't have them
(From OE-Core rev: 88d15877ba18309c521740d7a9649e14d77189bc)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The issue with installing the kernel image to both rootfs
and boot partition is that some systems rely on the kernel image in
rootfs and not in the boot partition.
This leads to duplication of the kernel image, which can cause
unnecessary storage usage.
This patch provides a solution to the problem by adding a new
parameter "install-kernel-into-boot-dir" to the wic kickstart file.
If this parameter is set to 'true', the plugin will install the
kernel image to the boot partition. If the parameter is set to
'false', the plugin will skip installing the kernel image, avoiding
duplication.
(From OE-Core rev: d3599afe5f604ea5afd9411e114934dcb52b2d48)
Signed-off-by: Kareem Zarka <kareem.zarka@huawei.com>
Signed-off-by: Christian Taedcke <christian.taedcke@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding the support of submodules required a lot of changes on the
internal data structures:
* initial_rev/startcommit used as a starting point for looking at new
/ updated commits was replaced by a dictionary where the keys are the
submodule name ("." for main repo) and the values are the
initial_rev/startcommit
* the extractPatches function now extracts patch for the main repo and
for all submodules and stores them in a hierarchical way describing the
submodule path
* store initial_rev/commit also for all submodules inside the recipe
bbappend file
* _export_patches now returns dictionaries that contains the 'patchdir'
parameter (if any). This parameter is used to add the correct
'patchdir=' parameter on the recipe
Also, recipe can extract a secondary git tree inside the workdir.
By default, at the end of the do_patch function, there is a hook in
devtool that commits everything that was modified to have a clean
repository. It uses the command: "git add .; git commit ..."
The issue here is that, it adds the secondary git tree as a submodule
but in a wrong way. Doing "git add <git dir>" declares a submodule but do
not adds a url associated to it, and all following "git submodule foreach"
commands will fail.
So detect that a git tree was extracted inside S and correctly add it
using "git submodule add <url> <path>", so that it will be considered as a
regular git submodule
(From OE-Core rev: 900129cbdf25297a42ab5dbd02d1adbea405c935)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the case of a repository with submodules, we need to add the
"devtool-base" and "devtool-patched" tag on all submodules in order to
properly detect the added/removed/modified patches
(From OE-Core rev: 241da68805d177d4ec4b302c8a997645cc645286)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When specifying --mode / -m srcrev with devtool finish/update-recipe on
recipes that are not fetched from a SCM repository we get the following
error:
Traceback (most recent call last):
[..]
File "<...>/poky/meta/lib/oe/patch.py", line 49, in runcmd
raise CmdError(cmd, exitstatus >> 8, "stdout: %s\nstderr: %s" % (stdout, stderr))
oe.patch.CmdError: Command Error: 'sh -c 'git format-patch --no-signature --no-numbered INVALID -o /tmp/oepatchbj7pfmzj -- .'' exited with 0 Output:
stdout:
stderr: fatal: bad revision 'INVALID'
Fix this by adding a check and abort with a proper error message.
(From OE-Core rev: 5b77f60e85d07921ae5e808daa1fd7d8c3dc40ea)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running devtool update-recipe with --mode=srcrev AND --append switch
in dry-run, we get the following error:
Traceback (most recent call last):
[...]
Exception: destpath should be set here
Fix this by removing a misplaced else statement in _update_recipe_srcrev
(From OE-Core rev: 85ba125703d5b442133fd7c470b915460ee68ac9)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit c304fcbe05 introduced a grouping when
listing regressions. This grouping has been added only for ptests. It has
been observed that any other kind of tests could benefit from it. For
example, current regression reports can show the following:
1 regression(s) for oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash
oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash: PASSED -> SKIPPED
1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_help
oescripts.OEPybootchartguyTests.test_pybootchartguy_help: PASSED -> SKIPPED
1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_pdf_output
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_pdf_output: PASSED -> SKIPPED
1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_png_output
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_png_output: PASSED -> SKIPPED
1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_svg_output
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_svg_output: PASSED -> SKIPPED
[...]
This output is not so useful in its current state and should be grouped per
test type too.
Enable grouping for all kind of tests, to make it llok like the following
in reports:
5 regression(s) for oescripts
oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash: PASSED -> SKIPPED
oescripts.OEPybootchartguyTests.test_pybootchartguy_help: PASSED -> SKIPPED
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_pdf_output: PASSED -> SKIPPED
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_png_output: PASSED -> SKIPPED
oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_svg_output: PASSED -> SKIPPED
(From OE-Core rev: 982798ef96e3a32bf15341bdd3bb7c4356709412)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows easier replication of esdk environment (which provides
a curated, limited set of tools that for example does not include bitbake)
in a standard yocto build. Switchover between various sets can be achieved
via PATH manipulation.
(From OE-Core rev: 20c548f2edca3888152adb63de7b23d84e3848e7)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Not all mbox 'from' fields will contain angle brackets, so the
re.findall invocation used for getting a reply_address may fail. Use a
simpler reference to the field to get the sender's email address.
(From OE-Core rev: 86e9afe09a346586114133f5a7470304d2ed733f)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rework the script for sending results to use send_raw_email and specify
the 'In-Reply-To' field so that patchtest replies to the emails, rather
than sending them standalone to the submitter and mailing list.
(From OE-Core rev: 0c45c92e7f26aea4edf2cfa577b7ba51384e59d3)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Modify patchtest-send-results so that it extracts the submitter's email
address and responds to them with the patch testresults. Also make a
minor adjustment to the suggestions provided with each email and include
a link to the Patchtest wiki page for additional clarification on
specific failures.
(From OE-Core rev: 64ed88e32cf9e04772319ff6e66c602d1cff4fd7)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
git show-ref looks at the _remote_ ref called HEAD, which is fine when it
matches the local HEAD but problematic when you're iterating a series of
commits.
Use rev-parse to resolve the local name to a proper hash.
(From OE-Core rev: 3c04747b681cf6090ba9c77752f6c2f304dbbe17)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
One of the newer PEP-517 backends to be added was python_hatchling.bbclass
but it was not included in the recent improvements.
Add selftest for 'jsonschema' pypi package.
(From OE-Core rev: d99b4883b4fee82bc588fd235ba90fedf1550cb8)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move most imports to the top of the file.
(From OE-Core rev: d2c287db0739b249604cd1beaa03ec38512ba718)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The autobuilder scripts post-process the generated JSON to inject recipe
and commit counts into the data. We can do this easily in patchreview
instead.
(From OE-Core rev: 77c96e43090cbf485aec612cc2315b85e5635dda)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A previous patch[1] added the ability to allow the search pattern for
patches to be changed, so that patchreview can be used across the entire
meta-oe repository by changing the patterns.
However, this means the caller needs to write long patterns when calling
patchreview.
Instead, we can see if the specified directory contains a layer by
checking if conf/layer.conf exists. If it does, then search for patches
inside this directory. If it doesn't, assume that the specified
directory is a repository that contains sublayers (such as
meta-openembedded) and look through each of the directories that match
the pattern meta-*.
This means patchreview can both scan either a single layer (eg
.../poky/meta) or a repository of sublayers (eg .../meta-openembedded).
[1] oe-core 599046ea9302af0cf856d3fcd827f6a2be75b7e1
(From OE-Core rev: a3a868519beab1b9cac94fefd7dbeffb09d047e9)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was writing out locked-sigs.inc into cwd with every
'bitbake -S' invocation. When the intent is only to to get task
stamps (-S none), or print the difference between them (-S printdiff),
the file is unnecessary clutter.
A couple of selftests/scripts were however relying on this, so they're
adjusted to explicitly request the file.
eSDK code calls dump_lockedsigs() separately via
oe.copy_buildsystem.generate_locked_sigs() and so isn't affected.
(From OE-Core rev: ad57c3cac2a8d3e60222e3cca0685f582dcea135)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add basic support for PEP517 [1] for the 3 following backends that are
supported by bitbake:
- setuptools.build_meta
- poetry.core.masonry.api
- flit_core.buildapi
If a pyproject.toml file is found, use it to create the recipe,
otherwise fallback to the old setup.py method.
Some projects can declare a minimal pyproject.toml file, and put all
the metadata in setup.py/setup.cfg/requirements.txt .. theses cases
are not handled. If a pyproject.toml file is found, assumes it has all
necessary metadata.
As for the old setup.py method, version numbers for dependencies are not
handled.
Some features may be missing, such as the extra dependencies.
[YOCTO #14737]
[1]: https://peps.python.org/pep-0517/
(From OE-Core rev: c7d8d15b2d0a9ecd210bd247fa0df31d9f458873)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to prepare the support for pyproject.toml (PEP517 [1]) enabled
projects, refactor the code and move setup.py specific code into a
specific class in order to allow sharing the PythonRecipeHandler class
No functionnal changes expected
[1]: https://peps.python.org/pep-0517/#source-tree
(From OE-Core rev: 2281e93347da4129062cfb40710df03c87c63168)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By convention, all python recipes start with "python3-" so update
create_buildsys_python to do this
This rule doesn't apply for packages already starting with "python"
Update recipetool's selftest accordingly
(From OE-Core rev: b0d87440e610b80f763d09784d4a90a148bb3e7b)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
License field of setup is not always standardized, so we usually use the
classifier to determine the correct license format to use in the recipe.
A warning note is added above the LICENSE field of the create recipe
in case a license is provided in setup. But when the plugin is called,
"LICENSE =" is not yet present so we can never display this note.
Replace the "LICENSE =" condition with "##LICENSE_PLACEHOLDER##"
to actually be able to display the note message
(From OE-Core rev: b7c26ca2028aa60f740464de85a11a01a531f32e)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Check that the maximum line length of the testresult file is less than
220 characters, to help guard against malicious changes being sent in
email responses. If any line exceeds this length, replace the normal
testresults used in the response with a line stating that tests failed,
but the results could not be processed. Also clean up the respone
substrings slightly to go along with the change.
(From OE-Core rev: b0d53cf587dc9afb97f00c1089e45b758e96dd7c)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Pull the actual email's subject line from the .mbox file and use that in
patchtest's test results response, so that it's clearer which patch it
is replying to.
(From OE-Core rev: 98ca0b151517b3544454fd5c1656a2de631c4897)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a "-l"/"--limit" option to allow changing the display limit in
resulttool.
- If no value is passed, resulttool uses its default value.
- If 0 is passed, the display limit is removed and every regression will be
displayed
- If a custom value is passed, this value overrides the vlaue configured in
resulttool
(From OE-Core rev: d3f536b3fc3f7027f6f5cf8bdaf5d7c050c7974b)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since "matched" and "improved" tests are not as important as regressions,
reduce the place they take in the regression report:
- merge "matched" and "improved" tests, while removing the label
- add a single line of additional info per pair
Those changes make the "Matches and improvements" look like the following
sample:
oeselftest_almalinux-9.2_qemux86-64_20230910083156
oeselftest_almalinux-8.8_qemux86-64_20231018010951
-> +7 test(s) present
oeselftest_almalinux-9.2_qemux86-64_20230911010538
oeselftest_debian-11_qemux86-64_20231017150459
oeselftest_debian-11_qemux86-64_20230910012927
oeselftest_debian-11_qemux86-64_20231017151319
-> +7 test(s) present
[...]
(From OE-Core rev: 6de4426d9a7da67deed7d3a3918892fb56238ff3)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Regressions reports currently reports matching pairs and improved pairs
first, then regressions.
Change order to print regressions first, which is the most valuable info in
the report, and then print improvements and matches at the bottom.
(From OE-Core rev: 599267467430e70fa4dc8ba6b2a8b126bf6da359)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Most of the changes list generated in regression reports fall in one
of the two following categories:
- there is only a few (<10) changes listed and the info is
valuable/relevant
- the list is huge (> 100 ? 1000 ?) and basically tells us that the whole
tests category suffers the same status (test missing, test failing, test
skipped, etc)
Prevent those huge, worthless lists by limiting the output for each test
group:
- current default limit is arbitrarily set to 50
- limit can still be overriden with a new "-l"/"--limit" flag, either with
custom value, or with 0 to print the whole lists of changes
- limit is applied per test family: currently it distinguishes only types
of ptests, but it can be adapted to other kind of tests
(From OE-Core rev: cec118406f3ad81cb4709f6e6ae1cef65799658e)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using a squashfs filesystem type, runqemu requires specifying the
full path to the image because it doesn't list squashfs types in its
fstypes variable. Add them to provide the same support as other
filesystem types.
(From OE-Core rev: c9c9a077e85b56f495f09187483548149f142a8d)
Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Underscores previously caused the next character in the label to be
printed using subscript due to the enhanced string support in gnuplot.
(From OE-Core rev: 282b48f90f77e0766993018d22fe03dd303febdc)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When reading patches from a directory it's important to sort the output
of os.listdir(), as that returns the files in an effectively random
order. We can't test the patches apply if they're applied in the wrong
order, and typically patch filenames are prefixed with a counter to
ensure the order is correct.
(From OE-Core rev: 4d6b586d37ab4528ed6dae6779cd730af9ef09c2)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[YOCTO #15243]
Avoid overwriting local changes when running patchtest by checking for
anything unstaged or uncommitted in the target repo, and logging an
error if something is found. This will provide the user helpful feedback
if (for example) they forgot to commit a change for their patch under
test, and will leave the target repository in a reasonable state (rather
than a temporary branch created by patchtest).
(From OE-Core rev: 2cc2dda6c3d5a97173b3fc434cc16ff2a930f43f)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is not documented or tested, and indeed hasn't been producing useful reports
for some time.
The script works by redirecting STAMPS_DIR into a separate location,
then running bitbake -S none, then comparing the two sets of stamp
filenames with regexes:
Match the stamp's filename
group(1): PE_PV (may no PE)
group(2): PR
group(3): TASK
group(4): HASH
stamp_re = re.compile("(?P<pv>.*)-(?P<pr>r\d+)\.(?P<task>do_\w+)\.(?P<hash>[^\.]*)")
Then there's some code that finds out what changed in the above between the two sets.
Messing about with STAMPS_DIR like that isn't supported, and will either do nothing,
or remove the original stamps. Also stamp filenames aren't really a 'public API'.
For finding out the changes between two builds, 'bitbake -s printdiff' is a supported
and tested option. It may be a bit too verbose, but that can be more easily fixed than
rewriting bitbake-whatchanged into a working state.
(From OE-Core rev: f8193978eb0944e693e6a5cfbf9035e104e489f0)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replace full license headers with SPDX identifiers and adjust all
patchtest-related code to use GPL-2.0-only.
(From OE-Core rev: 9bea6b39074296bb8d8719a3300636e316f19d1b)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since patchtest is in oe-core, the Python os module's methods can be
used to retrieve the repository path and tests directory by default.
This reduces the number of mandatory arguments for invocation of
patchtest unless the user wants to use a custom test suite or test
patches against a different repo. The REPO and TESTDIR arguments are
likewise adjusted so that they are optional. Also, make it more obvious
what the --startdir flag is meant for on the command line by renaming it
to --testdir, and update the scripts/patchtest.README file to be
consistent with the new usage.
(From OE-Core rev: bae7421ece4806f5148f164293810b9fe75e0756)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the following from the patchtest repo:
- patchtest: core patch testing tool
- patchtest-get-branch: determine the target branch of a patch
- patchtest-get-series: pull patch series from Patchwork
- patchtest-send-results: send test results to selected mailing list
- patchtest-setup-sharedir: create sharedir for use with patchtest guest
mode
- patchtest.README: instructions for using patchtest based on the README
in the original repository
Note that the patchtest script was modified slightly from the repo
version to retain compatibility with the oe-core changes.
patchtest-send-results and patchtest-setup-sharedir are also primarily
intended for automated testing in guest mode, but are added for
consistency.
(From OE-Core rev: cf318c3c05fc050b8c838c04f28797325c569c5c)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This introduces support for specifying a search pattern with the -p/--pattern
option in the patchreview.py script. This is designed to accommodate
the directory structure of meta-openembedded.
(From OE-Core rev: 599046ea9302af0cf856d3fcd827f6a2be75b7e1)
Signed-off-by: Mickael RAMILISON <mickael.ramilison@smile.fr>
Reviewed-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This disables Nagle's algorithm for our tcp serial connections which may
be causing data transfer issues.
(From OE-Core rev: f8eff4c427881a98333fdf7c42f66ed6603e4f03)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bblock script allows to lock/unlock recipes to latest task signatures.
The idea is to prevent some recipes to be rebuilt during development.
For example when working on rust recipe, one may not want rust-native to be
rebuilt.
This tool can be used, with proper environment set up, using the following
command:
bblock <recipe_name>
See help for more details
if a <recipe_name>'s task signature change, this task will not be built again and
sstate cache will be used.
[YOCTO #13425]
(From OE-Core rev: 2d9ab0cfd7f3cacc347954676f1323342a6b286f)
Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fixes [YOCTO #15222]
In some situations its more practical to keep git configuration
at `/etc/gitconfig` instead of `$HOME/.gitconfig` (e.g., when mounting
git configuration into a docker container).
This change makes `devtool upgrade` consider any available
git configuration instead of only checking `--global`.
(From OE-Core rev: 30a9f7de45050c8bac49d4b37419cc2e067a75fa)
Signed-off-by: Marcus Flyckt <marcus.flyckt@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'imp' was deprecated in Python 3.4 and removed in 3.12. The
piece of importlib we use has been around since 3.3.
(From OE-Core rev: 457f0dad87b4e45a53865b5ad2c150215bd74019)
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously _is_repo_git_repo() would return a result containing b'true\n' or
b'false\n' if 'git rev-parse' ran successfully, instead of True of False.
While this can be solved using e.g. result.strip().decode("utf-8") == "true",
there are some other cases to consider.
First, .git can be a file and not a directory when using a worktree.
Second, an emtpy .git directory in 'repodir' for which some parent of
'repodir' is an actual git repo will still return True in this case.
To account for these cases as well, use 'git rev-parse --show-toplevel'
and compare the result against 'repodir' instead of
using 'git rev-parse --is-inside-git-dir'.
(From OE-Core rev: 0830f53900dd7fd669a7d6492325559ad3225302)
Signed-off-by: Jermain Horsman <jermain.horsman@nedap.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently our wic test images boot up without kernel output on the consoles
which means we have no way to debug if anything goes wrong. Add the console
parameters runqemu would have added if the kernel wasn't built into an image
to improve our chances of debugging.
(From OE-Core rev: 63ea1efdaaf1173ef4f2f69b5a3403afef4b556a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I have a theory that some of the console boot issues we're seeing are due to
starting images with three serial ports yet only starting gettys on two of them.
This means that occasionally, depending on the port numbering we may not get
a login prompt on the console we expect it on.
To fix this, change the runqemu code so that if serial ports are passed in on
the commandline (as is the case in automated testing), we don't add any other
GUI serial consoles.
We do need to make sure we do have at least two serial ports since we don't want
getty timeout warnings.
(From OE-Core rev: 1b0348535dce3b776efbcf26406b94730a51eb85)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Debug message about using custom configuration file includes file name
with incorrect extension. Correct file name to "extlinux.conf".
(From OE-Core rev: dd63e1520454b2d53a48b72eaae126059af9809b)
Signed-off-by: Daniel Semkowicz <dse@thaumatec.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake has changed to require notification when metadata changes in the middle of tinfoil
sessions. Add the required function calls at the places metadata is changed.
(From OE-Core rev: e5574163ab49a8f51b2b34fd37acfd1cad9b7595)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop some unused imports.
(From OE-Core rev: 432446ef402ff42fe0c90172b77376fa5981524a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This only occurs in debian package builds when populating the sdk
and is a work around that seems to work. Eventually we should look
at why this is failing (I have ideas, it's somewhere in
lib/oe/package_management/deb/sdk.py), but for now, do this so we can
fix the core issue with nativesdk-intercepts.
(From OE-Core rev: a411123a95114233c5efd762dbcc8eb513030aab)
Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will match other deviation subtask of the same main task,
a couple of them can be found on oe-core layer:
do_compile_kernelmodules
do_compile_ptest
cmake_do_configure
setuptools3_do_configure
cargo_common_do_configure
python_pyo3_do_configure
python_setuptools3_rust_do_configure
This task will be also painted with the same color of the main task
but using alpha blending.
(From OE-Core rev: f10582b1c9a5639b48a4663453d201652facb179)
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
qemu itself is not helpful when render nodes exist, but can't be opened:
qemu-system-x86_64: egl: render node init failed
To fix this, users likely need to
* modprobe vgem (presence when physical graphic card is absent or has a driver without
support for render nodes, such as many older cards found in server machines)
* add their user to "render" group to write to /dev/dri/renderD* (permissions)
With this change runqemu should print hints for the above as appropriate from probing the nodes.
(From OE-Core rev: acd85925cb197b7a31a25b60e8de762e2c3697ef)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Running either of these ends up corrupting the os.execv args.
If we run:
./scripts/nativesdk-intercept/chown -R foo:foo bar
The loop here ends up missing the conversion of foo:foo to root:root because
it sees sys.argv[0] and assumes that it's the user:group argument and that we
should convert that. We end up a os.execv(path, args) that have the following
args:
['root:root', '-R', 'foo:foo', 'bar']
As os.execv ignores args[0], we can just populate it with sys.argv[0] and then
loop through sys.argv[1:]. As both chgrp and chown would have either flags and
USER[:GROUP] next, this fixes the issue.
(From OE-Core rev: 2a75f647ec7696d353f4b09099d777ba53f34d36)
Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of bitbake and grep, just use bitbake-getvar.
(From OE-Core rev: b5011a2fc248d88b5491cf6af1fc15e5974f6e45)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The '-w' option is not giving very helpful information. For example,
if we add 'spice' to IMAGE_INSTALL, bitbake -g core-image-minimal,
and then run `oe-depends-dot -k nspr -w task-depends.dot', the result is:
$ oe-depends-dot -k nspr -w task-depends.dot
Because: core-image-minimal nss
core-image-minimal -> nss -> nspr
The result is not showing the full dependency chain which brings in nspr.
With this patch, the result is:
$ oe-depends-dot -k nspr -w task-depends.dot
Because: core-image-minimal nss libcacard spice
core-image-minimal -> spice -> libcacard -> nss -> nspr
This patch also fixes a typo in help message: recipe-depends.dot -> task-depends.dot.
(From OE-Core rev: 222302810c472c8eb2efceaa757a253dcac5618f)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support for formatting a disk with a hybrid MBR & GPT partition
scheme. In this scheme, the primary partitioning method is GPT, but a
valid MBR header is also written than can point to a subset of the GPT
partitions on the disk (any partitions marked with the `--mbr` flag will
be included in this MBR). The primary purpose of this method is to allow
for SoCs that can only find a bootloader in an MBR partition to use GPT
once the bootloader is running. As an example, older versions of the
Raspberry Pi firmware can only parse MBR partitions to find a kernel (or
other bootloader like u-boot), but once those have booted GPT partitions
can be used.
In addition to the partitions annotated with the `--mbr`, a "protective"
GPT partition of type 0xEE is added, as the existence of such a
partition is the indication to tooling that this a hybrid MBR and that
the GPT partition table should be parsed instead.
(From OE-Core rev: e50e4c2a5ada6947b3503ca4d8e9c30d359e8a5d)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that SRCPV isn't needed we can simplify things in a few places...
(From OE-Core rev: 843f82a246a535c353e08072f252d1dc78217872)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The system requirements in the current version of the reference manual
state that Python 3.8.0 is the minimum version, but oe-buildenv-internal
still only checks for 3.5.0 or newer. Update the script to match.
(From OE-Core rev: 3e655a07f2de64dbf39189c3ece56803f6bc334e)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The kickstart parser defaults fstype to "vfat". This leads to an attempt
to create an empty file system even for regions configured with "--no-table"
if used without fstype when no --sourceparams given.
The fix tests for fstype "none" or no_table in Partition prepare method.
This will omit the file system creation an the potential error for small
region with --no-table option.
(From OE-Core rev: db771a4cd36bf291a8b68edfd905e03243f2c8b3)
Signed-off-by: Markus Niebel <Markus.Niebel@ew.tq-group.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Avoid a divide by zero traceback if unfortunate test counts are encountered.
(From OE-Core rev: c5aeea53dfacb53dedb8445cb3523dc3a8cb6dca)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Also remove the git.pokylinux.org URL, no longer used.
(From OE-Core rev: 47b88d143c2fc61ce0e03b7eb3a9dbcffadbf5b1)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Section VMA's are currently hardcoded. This doesn't work anymore starting
with systemd-boot v254.
Follow the actually solution to this which is documented here:
https://wiki.archlinux.org/title/Unified_kernel_image#Manually
This is also used by dracut. Later on, we may want to switch to ukify
instead but this is not ready yet.
(From OE-Core rev: 3abf99a6c6bde2fb8770f54dba609b35f6c6ee5a)
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With kernel 6.2 and later network devices are renamed by systemd. This does not
match with the current network device naming assumed in our configuration.
We may or may not change that naming but for now, pass the right kernel commandline
so things work as expected with newer kernels and removing a blocker on upgrading
to the 6.4 kernel by default.
(From OE-Core rev: 9e9c33d51e401fe2b4a632db74ccb3449e4b23ee)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whenever a test result is present in base test result but absent from
target test results, we have an entry in regression report looking like the
following one:
ptestresult.apr.testfile: PASSED -> None
The "None" status may be a bit confusing, so replace it with a custom
string which really states what "None" means in this context:
ptestresult.apr.testfile: PASSED -> No matching test result
(From OE-Core rev: b5f6da6e13f3484f51dfa07f6b3500aea7a21825)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a STATUS_STRINGS dictionnary matching raw statuses to custom strings.
Whenever a regression must be reported, raw status is searched in the
custom statuses dict (key search is case insensitive). If no custom string
is found, raw status is kept and used in regression report
(From OE-Core rev: 9d22bfc9d0c4092dba1af0ee11a4c51b7b270786)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With newly added recipe_qa task, the function needs to filter out its output as well.
The issue wouldn't appear in selftest because it's triggered by buildhistory and
selftest disables that.
(From OE-Core rev: da1284c990957827ee89685921c53b64490a32b2)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This can happen if running unpack task produces unrelated files and directories
(e.g. if recipe_qa or other tasks run); in this case it's better to stop, rather
than allow devtool to continue and error out much later with a message that has
nothing to do with where the problem originated.
The idea here was to handle tarballs that don't contain a top level directory and thus
the source tree is one level up; this basically never happens, and if it does we
should find a less brittle way to handle such tarballs.
(From OE-Core rev: b25e922b271794906b22450c7e6cc18fcab51ff8)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The immediate reason is that the task is sstate enabled, and due
to the way devtool sets up source extraction this causes bogus
values to appear in tmp/sstate-control/index-* files.
Also, recipe_qa may contain hard failures in the future, and it
would be counterproductive for the failures to prevent devtool
workflows (which may perhaps be about fixing the failure).
An alternative would have been to not insert recipe_qa task before
fetch task, but that would greatly delay the checks in larger
bitbake builds, and it's better to run them as early as possible.
(From OE-Core rev: 6f9db3f8de3dca9b0c87382561c70857d5b39e0f)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some regression reports show a lot of "PASSED->None" transitions. When such
big lot of identical transitions are observed, it could be that tests are
now failing, but it could also be that some tests has been renamed.
To detect such case, add a log in regression report to report the number of
new tests (i.e: tests that are present in target results but not in base
result). This new log also allows to know about newly added tests bases
(From OE-Core rev: 01b5cefd07e01c7407bc663842b8a8d502358a6d)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
non-release indexes will continue to generate when test output is
corrupted.
(From OE-Core rev: 1a9157684a6bff8406c9bb470cb2e16ee006bbe9)
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The function draw_label_in_box chooses the correct side based on
the maximum width length argument 'maxx'.
Currently the labels are always drawn on the left side and this
is because we are inadvertently passing the 'maxx' argument wrong.
(From OE-Core rev: b1cea47c52899a85428e0bddf82ba6a07689673b)
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is another variable named 'w' on the function which is redefined.
The full width is needed in the function so rename it as this must be unique.
(From OE-Core rev: 81389ccdf22d3ea845ae8d894a0e4e0cdf589f2a)
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The function draw_label_in_box have logic to chose the rigth side
to call the draw_text, so use it instaed of forcing the cordinates
of the right side.
(From OE-Core rev: b369e9bb0136fecc65f49e0965e5506aa50f489e)
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instaed of calculate the elapsed time over the loop when needed
do it on the loop startup.
(From OE-Core rev: 63a19626f10873cd7a4aabee7c9777fdede83bcb)
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since commit 569d4cd325, if one is using a custom template directory
containing a conf-notes.txt, this file is only displayed when creating
a new environment. When entering an already existing environment, only
the default poky conf-notes.txt is displayed.
This patch copies the conf-notes.txt to display to the build directory,
so that the good file is shown, even when templateconf.cfg is not used.
(From OE-Core rev: 845e8292f218d740ee653fa68bc3110aec1af3c5)
Signed-off-by: Stéphane Veyret <sveyret@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
NPM changed its manner to handle peer dependencies over its versions.
Before NPM 3: NPM installs automatically peer dependencies
between NPM 3 and 7: NPM shows a warning about peer dependencies
After NPM 3: NPM reworked its manner how to handle peer dependencies
The shrinkwrap doesn't have the parameters of the peer dependencies, so we cannot
fetch them. in the same time peer dependencies are not direct dependencies, they should
be installed as run time dependencies.
(From OE-Core rev: f36021a749974ef3d4a6abe4d5429544a815071a)
Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The shrinkwrap file changed its format, but npm does not version this file. So we can use it properly.
The actual changes make the script check if the npm package has dependencies in the actual shrinkwrap format.
(From OE-Core rev: 488d17c2af0c927ec66f0eee124bf6fc5b7f7c95)
Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Npm packages do not have yocto friendly names. fore instance we can have names like
"@example/npmPackage"
npm fetcher has a function that convert these names to yocto friendly names.
But in recipe tool we have an other function (duplicate).
(From OE-Core rev: 18e5438de5389b58c8b6a548d4474128d510a28d)
Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
native and nativesdk classes are special and must be inherited last :
put them at the end of the gathered classes to inherit.
(From OE-Core rev: a6614fd800cbe791264aeb102d379ba79bd145c2)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- Drop the native-sysroot-basedir parameter
still allow it to keep backward compatibility
write a warning to stderr
- Add a space after ! in the if as suggested by shellcheck
- Support the new OE_TAP_NAME variable as well
(From OE-Core rev: be72e5e32da5a251db14b42d3e9c0951178e216d)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When runqemu tries to call runqemu-ifup to create tap devices, it checks the
output of runqemu-ifup to get the newly created tap device.
The behavior of runqemu-ifup was recently modified along with its output, it
no longer expects the uid parameter to be passed and it prints out a warning
if it was, since this warning was now part of the output runqemu tries to parse
it and convert it to an int() which proved impossible.
Pass the correct arguments to the runqemu-ifup call and echo the warning
to stderr instead to make sure its not being parsed and used by runqemu in any
case.
(From OE-Core rev: ce3a2f4cec28290c8e530989f17243f1ada4e3bd)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandro@enedino.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
logger.warn() has been deprecated, logger.warning() should be used instead.
(From OE-Core rev: 9263497880b3154d65ed713498749f906534a055)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandro@enedino.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Support the new environment variable OE_TAP_NAME.
(From OE-Core rev: 6795dddb4074274279b7ff7b9639d15786f06a40)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ignore itnerfaces with other names than what the runqemu scripts created.
(From OE-Core rev: e99ec79a423d09b9308e3d42f0832ee583b1456d)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Feature: Hard-coding the interface names to tap* is not always a good
idea.
(From OE-Core rev: 739f20481d14a66b23d6195c3837f83d7370434a)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The uid parameter is no longer needed since ip tuntap is used
internally. Remove it. Backward compatibility to 3 or 4 parameters is
still supported.
(From OE-Core rev: a11c87b0a13f81beea8a9f33ef8379c9d2593e59)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The parameter staging_bindir_native is no longer used. Remove it.
For now the script is backward compatible. With 4 parameters it logs an
error message but still works.
(From OE-Core rev: c47322297fcd0408c5e290e74eee4344754e03e4)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If there are other tap interfaces than the interfaces created by the
runqemu-* scripts, these interfaces are not ignored. This is now fixed
by filtering the interfaces for a specific prefix in the interface name.
(From OE-Core rev: f65fb5fce45d13c2881acd8295d417635449c52d)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Recent patches changed the index of the tap interfaces. They start now
with tap1 instead of tap0. Also the IP address starts with 192.168.7.3
instead of 192.168.7.1. This gets reverted to the previous behavior.
(From OE-Core rev: eaf9d2dfd8f0270f543b612fbd2df775adc7a394)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Feature: Hard-coding the interface names to tap* is not always a good
idea. Introduce an environment variable which allows to change this:
OE_TAP_NAME.
(From OE-Core rev: c9ea17c13935e805a5b7497b14e1f049b0548779)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
ip tuntap does not need the uid, it was an unused variable/parameter.
Backward compatibility should be fine.
(From OE-Core rev: 9ccbabc06d02addd429a21dbe15a1a42738c58d6)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
openSUSE RPMs are compressing the RPM payload using zstd, that
correspond to the magic ID 0x28, 0xb5, 0x2f.
This patch update the script to the last version from the rpm project,
and add support to this compression format, and extract the cpio payload
using the "unzstd" binary.
(From OE-Core rev: 3aba44a75dd565b192f7328f2a0150a313de3cc1)
Signed-off-by: Alberto Planas <aplanas@suse.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since https://git.yoctoproject.org/poky/commit/?id=51063c1e6ac
we need to pass exactly 2 arguments to runqemu-ifup, otherwise
the script will return an error since bindir is no longer being
used.
However the call to runqemu-ifup from runqemu is still passing
bindir as an argument resulting in the error mentioned above,
remove the bindir argument to fix this issue.
[YOCTO #15150]
(From OE-Core rev: b9ef82727e719389b6d8ca2e9f642bfb328219b7)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandro@enedino.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The use of sort means that tap10 and tap1 don't sort correctly. Tweak
the shell expression to correctly handle numbers of tap devices of 10
or more.
(From OE-Core rev: 4c6d0f3ddbb76bde7f35edeb280fe56a9a00f91b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The third parameter was dropped in a previous commit but changes were not made
to the argument checks. Fix this.
(From OE-Core rev: 5d4818e836747a83f55bad8a00a9d3cd6e80c5dc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit d43c41fcaf refactors the script but
there's a missing '"' in one line, leading to the following message:
poky/scripts/runqemu-gen-tapdevs: command substitution: line 62: unexpected EOF while looking for matching `"'
(From OE-Core rev: 94b9b1333a5f779316b9698bb3e4fa7246bae78e)
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The IFCONFIG variable is really the path to ip and hasn't been
ifconfig for a long time. Rename the variable to something less
confusing.
(From OE-Core rev: eb1c947a056f3e2c80e46e5e606423e85da46caa)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
ip tuntap provides the functionality that we obtained from tunctl. We only
needed tunctl when ifconfig was more available than ip. That isn't the case
now so we can drop tunctl and all the hoops we need to jump through to build
and provide it.
(From OE-Core rev: 2abfbca690ff00cb58ce08a65cde006578ee3de9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
devtool crashes when running "update-recipe" and append changes on the recipe.
"$ devtool update-recipe -a <layer> <recipe>"
Traceback (most recent call last):
...
File "/ovss/ovss_quanta/poky/scripts/lib/devtool/standard.py", line 1636, in srcuri_entry
return 'file://%s%s' % (basepath, paramstr)
^^^^^^^^
NameError: cannot access free variable 'basepath' where it is not associated with a value in enclosing scope
The input variable 'fname' should have the same meaning as the variable 'basepath'.
Modify the 'fname' to 'basepath' and solve the issue.
(From OE-Core rev: c3231756bbc2cb5641204414ad3670d7f8607ed3)
Signed-off-by: Charlie Wu <chiachiwu@google.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is already a neat check_free_port() function for finding an available port
atomically, so use that and make two additional tweaks:
- no need to allocate two separate ports; per unfsd documentation they can be the same
- move lockfile release until after unfsd has been shut down and the port(s) used has been freed
[YOCTO #15077]
(From OE-Core rev: dee96e82fb04ea99ecd6c25513c7bd368df3bd37)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The *ip* command supports the creation and destruction of TAP devices since
2009 and might be more likely installed on systems then *tunctl*. Therefore
it should be tried to setup or teardown the TAP interface with *ip* before
falling back to *tunctl*.
https://git.kernel.org/pub/scm/network/iproute2/iproute2.git/commit/?id=580fbd88f75cc9eea0d28a48c025b090eb9419a7
(From OE-Core rev: 424ede206baae1c228583aab1df6c18513ac104f)
Signed-off-by: Jörg Sommer <joerg.sommer@navimatix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The changes are mostly about early exit which causes indentation changes;
check with `git diff -w`. Another change is the check for ip by simply
calling it and deciding upon the exit code, if it's fine or not.
(From OE-Core rev: 351577761d0712a005eda9dde9215558ca9a1fe9)
Signed-off-by: Jörg Sommer <joerg.sommer@navimatix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, uboot do_menuconfig task is breaking when UBOOT_CONFIG is
chosen rather than UBOOT_MACHINE, it simply fails with the following
errors:
| make: *** No rule to make target 'menuconfig'. Stio.
| Command failed.
| Press any key to continue...
this is due to the work directory of do_menuconfig is set to ${B} but
not ${B}/$config.
We should distinguish two situations:
1) When there is only one config item in UBOOT_CONFIG, do_menuconfig
should work just like how it works for UBOOT_MACHINE.
2) When there are multiple config items in UBOOT_CONFIG, do_menuconfig
should print out some information saying it's not supported other
than just failing.
This patch mainly aims to fix that by introducing a extra variable
KCONFIG_CONFIG_ENABLE_MENUCONFIG, it would be set to 'false' for
situation 2), and when it's set to 'true', then set
KCONFIG_CONFIG_ROOTDIR correctly in uboot-config.bbclass to let
do_menuconfig task work.
DEVTOOL_DISABLE_MENUCONFIG could be replaced by this new variable
KCONFIG_CONFIG_ENABLE_MENUCONFIG.
(From OE-Core rev: f9e834e317880cf47dbb4f8285bc36d743beae5e)
Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I've encountered issues reproducing initramfs and UKI image builds,
which will be fixed with this patch.
1. initramfs
There's a symbolic link to /sbin/init, which is appended to the cpio archive after creation.
The links timestamp needs to be static and the cpio append command needs the '--reproducible' flag to produce deterministic outcomes.
2. Unified Kernel Image
'--preserve-dates' is required for a static 'Time/Date' entry.
I've added '--enable-deterministic-archives' although in my case this
didn't change anything.
(From OE-Core rev: fd027729bafb4e085ba0949e38e724f3a8cad102)
Signed-off-by: Frieder Paape <frieder@konvera.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, finding the elapsed time of each task in buildtimes.svg
is a manual effort of checking the top axis and finding and subtracting
the end and start time of the task.
This change adds the elapsed time for each task, so that
manual effort of comparing start/end time is avoided.
(From OE-Core rev: 3efebd3404de548f0757863da237f2d18ce60013)
Signed-off-by: Mauro Queiros <Mauro.Queiros@criticaltechworks.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add argument --hidden to avoid MS Windows prompting to format
partition after flashing to a USB stick, SD card on another media.
Set Bit 0 (RequiredPartition) to mark that the partition is
required for the platform to function on GUID Partition Table
(GPT).
The new argument simplifies setting RequiredPartition on GPT
through a WKS file and the default imager plugin. Otherwise,
without this feature, to achieve the same result a new imager
plugin has to be implemented and set in WIC_CREATE_EXTRA_ARGS.
(From OE-Core rev: 7a111ff58d7390b79e2e63c8059f6c25f40f8977)
Signed-off-by: Leon Anavi <leon.anavi@konsulko.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Util now, only paths added for current project are removed
to avoid PATH growing unnecessarily. This is to handle the case
of sourcing the init script into different build directories.
However, if we source the init script from different projects into
different build directories, the paths added by previous projects
are not cleaned up.
To avoid this, we record the paths added into OE_ADDED_PATHS, and
remove it in the next sourcing.
The OE_ADDED_PATHS variable is exported mainly for the two reasons:
1. users can clearly see it in the `env' command's output.
2. if the prefixed PATH is carried into the subprocess (e.g., a subshell),
so should this OE_ADDED_PATHS variable that prefixes it.
Note that the paths, "$OEROOT/scripts:$BITBAKEDIR/bin:", are added
as a whole. A previous commit, "oe-buildenv-internal: Add paths to $PATH individually",
made the change to treat these two paths separately, the reason was
not "assuming the path to the scripts directory always is in $PATH
directly before the bitbake directory". But this is exactly the effect
of the codes. I see no reason why we should complicate things.
(From OE-Core rev: 3405a3221b8f6641a8e42b04bc7acd6e032aeab8)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Somehow these two got left behind and hence on older hosts that
are using buildtools for a newer python - they will still fail.
(From OE-Core rev: f1b11b83a6dd991f393940741c930691463e6d53)
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use appropriate kernel image name instead of hard-coded vmlinuz for bootimg-pcbios plugin.
(From OE-Core rev: 8d9f00ba456fe76e0f4ef0e68ec64fc538c90d89)
Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a bit of a compatibility issue more than anything. Some devices
get upset if the FAT file system contains less blocks than the
partition.
The fixed-size argument is currently respected by the partition creation
step but not by the file system creation step. Let's make it so the file
system respects this value as well.
(From OE-Core rev: 38e1a235f5eceade7c871f96dc97f6c384384c7b)
Signed-off-by: Randolph Sapp <rs@ti.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Default network configuration requires tun/tap module and while being
usable it conflicts with tap devices created by VPN clients sometimes
and requires root permissions to use . While it's possible to work
this around it's not always feasible if network is not required
Add nonetwork option which can be specified if the network connectivity is
not needed and SDL/serial is enough to communicate with the image.
(From OE-Core rev: d4073dedbb234ff3c6bbebafc836fedf90d96569)
Signed-off-by: Pavel Zhukov <pazhukov@suse.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bzip2 is in HOSTTOOLS already and used in few other places already.
This fixes bin_package class for RPM packages without adding bunzip2 to
HOSTTOOLS.
(From OE-Core rev: eb3ec7469fff857c819332371ad1d586f43c79c3)
Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
So far part.label has been used to define GPT partition label even if
part.part_name was defined.
Fix the code to use part.part_name whenever available, as it makes sense
to have a GPT partition label which is different from the contained
filesystem label.
(From OE-Core rev: 7704d5fc36eb065224792bf4d5543814eaa5fed3)
Signed-off-by: Dit Kozmaj <dit.kozmaj@kynetics.com>
Signed-off-by: Diego Rondini <diego.rondini@kynetics.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* when searching for qemuboot.conf
* don't assume that IMAGE_LINK_NAME is always
<rootfs>-<machine> (with <rootfs>-<machine>.qemuboot.conf)
* runqemu: use IMAGE_LINK_NAME set by testimage.bbclass or query with bitbake -e
* testimage.bbclass was setting DEPLOY_DIR which I don't see used
anywhere else, so I assume it was supposed to be DEPLOY_DIR_IMAGE as mentioned
in corresponding runqemu code, do the same with IMAGE_LINK_NAME variable
* add virtual/kernel as bitbake -e target in run_bitbake_env to make
sure IMAGE_LINK_NAME is defined (kernel-artifact-names.bbclass inherits
image-artifact-names.bbclass as well)
* improve .qemuboot.conf search
1st search for file matching the rootfs and only when not found
try again with .rootfs suffix removed
[YOCTO #12937]
(From OE-Core rev: 716eb55bb963db7b02d985849cb025898aabc855)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit 82e67b82ea8e12aa0b7b9db1d84fec0436dec71b.
It was commited as part of https://bugzilla.yoctoproject.org/show_bug.cgi?id=12301
for kernels < 4.15, as of now oldest builder kernel we have is 4.15 on
ubuntu 18.04 so we should not require this workaround. Moreover, this
fixes an smp problem with qemux86 where no matter what -smp <x> option
is used, qemu always starts with single core.
(From OE-Core rev: fa8a7c0608fc800c48d0ff1cd832ad63c51eeab1)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The logic in this script validates that the length of data sections are
evenly divisible by the block size. On most systems the block size is
4KB and all is good. Some systems though, such as ppc64le, have a block
size larger then 4KB. For example on a POWER9 based ppc64le system, the
block size is 64KB.
This results in this script failing with errors like this when building
wic images:
|440, in _do_get_mapped_ranges
| assert extent_len % self.block_size == 0
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| AssertionError
In this case the data section size was 268KB and the block size was
64KB, resulting in the above assert failure.
Resolves https://bugzilla.yoctoproject.org/show_bug.cgi?id=15075
(From OE-Core rev: 1e23b803af6991fc20e4a4e88a0ef0541399e722)
Signed-off-by: Andrew Geissler <geissonator@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Regression reports are not generated on some integration branches because
yocto_testresults_query.py truncates branches names with slashes when it passes
it to resulttool. For example, "abelloni/master-next" is truncated to "abelloni"
Fix this unwanted branch truncation by fix tag parsing in yocto-testresults
(From OE-Core rev: b2fb7d7b0c05bb198a2271bbf1742645ba220ea3)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The commands passed to subprocess are tuples which when passed to a %
format are treated as multiple format arguments instead of a single
argument to be coerced by "%s". This results in a TypeError being
raised with python claiming that not all arguments were consumed.
Fix this by wrapping the command tuple in a str() call, as is done for
the logging strings
[YOCTO #15078]
(From OE-Core rev: 3e5d04d9ebbee4e11fb39bf353b6d4c3133e166a)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we try to parse a buildstats directory which was either aborted or
is still being built then the top-level build_stats file doesn't
contain an elapsed value which causes an exception:
UnboundLocalError: local variable 'elapsed' referenced before assignment
Default both start and elapsed to 0 so that the parse succeeds.
(From OE-Core rev: 701d985aa8f2e9c2b9c0736fa25b424f3701889e)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This script will write a summary of the buildstats to the terminal,
sorted by start time or duration, optionally hiding short tasks, and
highlighting long running tasks.
(From OE-Core rev: 253d2c0eb048ea38822844ebb69ad76d55b5c3ef)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With mesa 23.0, it is not longer possible to use the host drivers, as
mesa upstream has added strict checks for matching builds between
drivers and libraries that load them.
Add a check and a hint to runqemu so that there is a helpful error when
there is no native/nativesdk opengl/virgl support.
(From OE-Core rev: f0946844df7270fe368858d8929e6b380675b78b)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* to better indicate how it's used in get_first_file
* cmd* is used in other places for actual shell commands
to execute
* RunQemuError('KERNEL not found: %s, %s or %s' % cmds)
also looked weird to me, but that works (to my python-noob surprise)
[YOCTO #12937]
(From OE-Core rev: 7c26e9dcc999a7d6a365831c39d25d98890be6d0)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the move to a ptest per image, the multiple ptest log scenario
is much more likely. Tweak the handling to only warn if there are
overlapping files.
(From OE-Core rev: 8391f1668fcbe932ba846299b0ec22de09e06dd4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous addition of pressure values to the chart didn't fix the extents
function which meant the bottom of the chart was cut off. Fix that.
(From OE-Core rev: cc8cef69e717e08f80d10f775f0fffc644267b59)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix:
scripts/pybootchartgui/pybootchartgui/parsing.py:134: SyntaxWarning: "is" with a literal. Did you mean "=="?
if pid is 0:
(From OE-Core rev: c9a6511ae618035b8efad01646e37ba28ce1e3f8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Address:
DeprecationWarning: 'pipes' is deprecated and slated for removal in Python 3.13
pipes.quote is an alias for shlex.quote so switch to that.
(From OE-Core rev: 7b1c1dd9985a6f1645271a928dda7f1897a7ba8a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a convenience argument to the log subcommand to list all of the
ptest logs in a testresults file.
(From OE-Core rev: cd2d7adf02005d46f19c93a40db1e10ce01ac261)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
ptestresult_get_log() looked for a key called 'ptestresuls.sections',
which should be 'ptestresult.sections'
(From OE-Core rev: 7c8c9f7283e54bf8b1521fbaad7dceb66a8fcdbb)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support to the runqemu script for a new option, 'guestagent', that
enables the virtio serial port for host-to-guest communication.
(From OE-Core rev: 21a1e52079089c5bbeee8ffc9c504471f4a8732a)
Signed-off-by: Brenda Streiff <brenda.streiff@ni.com>
Signed-off-by: Clément Péron <peron.clem@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If it exists, there is no need to delete it, and if it does not,
devtool prints an ugly traceback.
(From OE-Core rev: af82e59e8f08369aabd5fa6eb43022982d4e59a7)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The script currently only works if base and target can be found on default
branches. It breaks if we try to generate a regression report between revisions
that live on different branches (as needed on integration and testing branches).
For example, the following command:
./scripts/yocto_testresults_query.py regression-report yocto-4.0.6 yocto-4.0.7
ends with the follwing error:
[...]
ERROR: Only 1 tester revisions found, unable to generate report
[...]
Read branches from tags names in test results repository, and pass those
branches to resulttool when generating the report
(From OE-Core rev: 6c472b326bcc718459483cc29b310b884742df86)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
resulttool regression module simply compare a base test status to a target test
result status. This approach raises many false positives since all XXX -> PASS
transitions (XXX being any status different from PASS) are flagged as
regression.
- Do not list XXX -> PASS transitions in regression report, instead count them
and print a summary of "newly passing tests"
- If an inspected pair has only "newly passing tests", do not print detailed
list and print it as "Improvement" instead of "Regression"
Updated output example looks like the following:
[...]
Improvement: oeselftest_fedora-37_qemux86-64_20230127010225
oeselftest_ubuntu-22.04_qemux86-64_20230226120516
(+1 test(s) passing)
[...]
Match: oeselftest_almalinux-8.7_qemuarm64_20230127015830
oeselftest_almalinux-8.7_qemuarm64_20230227015258
[...]
Regression: oeselftest_almalinux-9.1_qemumips_20230127000217
oeselftest_opensuseleap-15.4_qemumips_20230226130046
ptestresult.glibc-user.debug/tst-read-chk-cancel: PASS -> None
ptestresult.glibc-user.nptl/tst-mutexpi4: PASS -> FAIL
ptestresult.glibc-user.nptl/tst-mutexpi5a: PASS -> FAIL
Additionally, 44 previously failing test(s) is/are now passing
(From OE-Core rev: c335f96f687c73fde443ac330ca3e17113794d9e)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some ptests results may be wrongly sampled, resulting in some part of the error
being propagated in test name. For example:
"ptestresult.binutils-ld.in testcase /home/pokybuild/yocto-worker/qemumips/build/build-st-1666126/tmp/work/mips32r2-poky-linux/binutils-cross-testsuite/2.40-r0/git/ld/testsuite/ld-ctf/ctf.exp"
"ptestresult.gcc.Couldn't create remote directory /tmp/runtest.455781 on ssh"
"ptestresult.gcc-libstdc++-v3.Couldn't create remote directory /tmp/runtest.3814266 on target"
While the root errors must be fixed in corresponding ptests packages for those
tests, the test results history must still be usable even with this issue
Add new filters to detect if temporary test directories (build-st-*,
/tmp/runtime.*) are present in name. If so, truncate test name. As a side
effect, it will aggregate multiple failing errors into one, but the regression
will still be raised
(From OE-Core rev: 601eecfddd26bfe2954835a73ed1116bb520235f)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
ptests names not only need to be fixed for regression based on git testresults
but also for testsresults provided "manually"
Move ptests naming fixup in regression_common to share the fixup between both
regression use cases
(From OE-Core rev: f772ccd108dc3d618db9d479d672c0f3edd203ca)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you build an image with lots of manpages in, then each package will
run mandb inside qemu-user at rootfs time. This is a slow operation
and should be done once when all of the packages have installed using an
intercept instead.
The call to mandb has been changed too. mandb doesn't actually allow
the configuration file to be read from stdin so that was being ignored,
instead write the file to a temporary file and use that.
This means we then don't need to tell it where to search explicitly, and
it writes the indexes to the correct paths so we don't need to move
files afterwards either.
Sadly we do still need to run mandb inside qemu-user, as the underlying
database is a gdbm file and they are byte-order dependent.
For my test case of core-image-base with api-documentation
DISTRO_FEATURES and doc-pkgs IMAGE_FEATURES enabled, the performance
gain is significant:
core-image-base do_rootfs -1303.1s -73.6% 1771.6s -> 468.5s
(From OE-Core rev: fbd8a57aa307bfda70a08cb78af3c97f05c39a3a)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If yocto_testresults_query.py is run from oe-core instead of poky, the
script will very likely fail since poky tags do no exist in oe-core. If
one or both revisions are not found, log the error and a suggestion
about the reason (the script being run in oe-core instead of poky)
(From OE-Core rev: 758ac050ffd91524d400c196865b1ed27ece8776)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will produce a more useful hint for those setting
up or testing virgl headless:
runqemu - ERROR - No render nodes found in /dev/dri: ['by-path', 'card0']. If /dev/dri/renderD* is absent due to lack of suitable GPU, 'modprobe vgem' will create one suitable for mesa llvmpipe software renderer.
as qemu itself isn't helpful:
alex@Zen2:/srv/storage/alex/yocto/build-64-alt$ qemu-system-x86_64 -display egl-headless
qemu-system-x86_64: egl: no drm render node available
qemu-system-x86_64: egl: render node init failed
(From OE-Core rev: cbbada6a6c9b0a2e97f7395117dad986555f2db9)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some test case naming is poor and contains random strings, particularly
lttng/babeltrace but also curl. Truncating the test names works since they
contain file and line number identifiers which allows us to match them
without the random components, or in the case or curl, test IDs.
Going forward we may be able to improve the test names but this
tweak allows historical test results to work in reports.
(From OE-Core rev: 541a2e2683531355e678fd93524a0c4a8c43a8ff)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a test result contains LTP test results, it should only be compared with
other runs containing LTP test results.
(From OE-Core rev: 4dbbf2f4a85620a08dc2fa65095dc17fe6c530f8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add yocto-testresults-query script. This is a thin wrapper over resulttool which
is able to translate tags or branch name to specific revisions, and then to work
with those "guessed" revisions with resulttool
(From OE-Core rev: b1460201b0f3d8fd7f977ac82f218bf9010d5573)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When generating regression reports, many false positive can be observed since
some tests results are compared while the corresponding tests sets are not the
same, as it can be seen for example for oeselftest tests (oeselftest is run
multiple time but with different parameters, resulting in different tests sets)
Add a filtering mechanism in resulttool regression module to enable a better
matching between tests. The METADATA_MATCH_TABLE defines that when the TEST_TYPE
is "oeselftest", then resulttool should filter pairs based on
OESELFTEST_METADATA appended to test configuration. If metadata is absent from
test results, in order to keep compatibility with older results, add a
"guessing" mechanism to generate the missing OESELFTEST_METADATA. The guessed
data is tightly coupled to the autobuilder configuration, where all oe-selftest
executions are described
(From OE-Core rev: 94ab7c2b892bf292dd86619ca9c63ddd7bf53f3c)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wic plugin crashed if IMAGE_EFI_BOOT_FILES is not empty and uefi-kernel
loader specified because of preliminary return from the function.
[Yocto #15033]
Fixes:
| File "/mnt/builds/yocto/sources/scripts/lib/wic/plugins/source/bootimg-efi.py", line 371, in do_prepare_partition
| for src_path, dst_path in cls.install_task:
| ^^^^^^^^^^^^^^^^
| AttributeError: type object 'BootimgEFIPlugin' has no attribute 'install_task'. Did you mean: 'do_install_disk'?
(From OE-Core rev: a6372f70f31b39ce9867b705d02205621d7a8685)
Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
yocto-check-layer to check all .patch file in layer for
Upstream-status and list down all .patch file without Upstream-Status.
set this test as expected failure for now, so this wont fail the check
layer while still able to capture any patches with malformed or missing
Upstream-Status in report.
[YOCTO #14642]
(From OE-Core rev: 237c1b66e5014123c1e5c3e78f9ab0357bcd62dc)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
So that patch fuzz issues can actually be fixed,
as extracting source with 'devtool modify' is the first step
for that.
(From OE-Core rev: 7067abd31d9dd4b98ec70c1c7effbe2904797cd1)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
qemu doesn't always behave well and can hang too.
kill it with force if it was still alive. Move clean up
commands into cleanup() function.
(From OE-Core rev: 079c2935d2f585ce49e1c7daab2155fcf0094c48)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The script is operating on layer repositories, which can and do sometimes contain
several layers. This distinction is important as the script will be tweaked
to write a record of actual layer locations.
(From OE-Core rev: 833965e6001db98039c0aa816ae661232213bcea)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a task is aborted the buildstats file isn't complete, so calculate
when the build finished and use that as a end time.
(From OE-Core rev: 23ebaec476dc46aebe5997f025661137f3e341bd)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This sets the ground for standardizing (and enforcing) the location of
configuration templates: let's start with the default one.
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows to specify partition with fstype=none in the wks file
to have partition created but without following mkfs. The none fstype
is in the list already but the usage is not documented.
Example;
part /data --ondisk mmcblk0 --fstype=none --align 4096 --fixed-size 512
will create a partition, filesystem may be created manualy on the host
or target and data will be preserved if the device is reflashed using
same wks. Works with bmaptool and probably does not work with dd.
Use case is persistent filesystem/data between reflashing of the image.
(From OE-Core rev: 351cb64da37aa43113e5192605d04436652aa3b8)
Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* True is default since 2016 and most layers were already updated
not to pass this parameter where not necessary, e.g. oe-core was
updated couple times, first in:
https://git.openembedded.org/openembedded-core/commit/?id=7c552996597faaee2fbee185b250c0ee30ea3b5f
Updated with the same regexp as later oe-core update:
https://git.openembedded.org/openembedded-core/commit/?id=9f551d588693328e4d99d33be94f26684eafcaba
with small modification to replace not only d.getVar, but also data.getVar as in e.g.:
e.data.getVar('ERR_REPORT_USERNAME', True)
and for getVarFlag:
sed -e 's|\(d\.getVarFlag \?\)( \?\([^,()]*, \?[^,()]*\), \?True)|\1(\2)|g' \
-i $(git grep -E 'getVarFlag ?\( ?([^,()]*), ?([^,()]*), ?True\)' \
| cut -d':' -f1 \
| sort -u)
(From OE-Core rev: 26c74fd10614582e177437608908eb43688ab510)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Deleting data files whilst bitbake is running isn't a great idea.
Make sure memory resident bitbake is stopped if present and make
the file deletions a little more specific and safer too while we're
here.
Note: The cache directory layout is different on older releases (some
are split level) so this won't be a direct backport to some.
(From OE-Core rev: fa3f1f67abae1d90bede2e98bf7293603fd1307c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This extra options to be appended to the nfs rootfs options in kernel
boot arg.
Example config with qemuppc64 in machine config:
add r/w size in the nfs rootfs extra option:
QB_NFSROOTFS_EXTRA_OPT = "wsize=524288,rsize=524288"
re-build and runqemu with nfs again, we can observe the kernel command
line added our defined value in QB_NFSROOTFS_EXTRA_OPT:
Kernel command line: root=/dev/nfs nfsroot=10.0.2.2:/home/xchen5/testing/build/tmp-glibc/deploy/images/qemuppc64/testnfs,nfsvers=3,port=3049,tcp,mountport=3048,wsize=524288,rsize=524288 rw mem=256M ip=dhcp console=hvc0 console=hvc0 nohugevmalloc
(From OE-Core rev: 43a97f5bf3f90c5c1fd603f7dca2b3db2c0e3040)
Signed-off-by: Xiangyu Chen <xiangyu.chen@windriver.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
oe-selftest efibootpartition.GenericEFITest.test_boot_efi was failing for 6.1 kernels with:
| ERROR: _exec_cmd: export PATH=[...] mcopy -i [...]/rootfs_boot.1.vfat -s [...]/rootfs1/* ::/ returned '1' instead of 0
| output: Disk full
I believe we hit a file boundary size and having "0" overhead in the image meant
the files couldn't be installed. Allow a small amount of overhead to avoid
the error.
(From OE-Core rev: 16e0b8a8fc36f5525b1801888851958f0dbe84c2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The effect of subsequent setup-layers executions is now either a NOOP
or the minimal set of changes required to ensure layers precisely match
the JSON configuration.
This change allows setup-layers to be incorporated into a team's
configuration management strategy. In particular, the configuration
JSON manages a "pinning policy" that documents the oversight of sources
of change (a requirement for embedded development in highly regulated
industries).
One model for this strategy would work as follows. Team level policy is
developed to regularly review upstream commits that occur between the
current upstream HEAD and the previously pinned revision. The JSON
configuration is periodically updated after a review, test, and approval
process. In the rare instance that an upstream change is considered
problematic, the bbappend mechanism can be used to make relevant
changes in the team's project repository. This approach also requires
that team developers regularly run the project repository copy of
setup-layers. This is most easily accomplished by including setup-layers
in a wrapper script that all team developers use to interact with the
bitbake tool suite (e.g. "bb bitbake foo-image"). Project level policy
and oversight is effectively "contained" within this wrapper script,
thereby reducing a significant source of human error.
Left unstated, but acknowledged here, are a number of nuances required
to successfully implement the above strategy. The details are out of
scope for this explanation. What should be clear is that a larger
configuration management strategy can now benefit from the utility
provided by setup-layers.
Note: Neither the above configuration management strategy example nor
the change itself is intended to alter the original intent to use
"bitbake-layers create-layers-setup destdir" to keep pace with upstream
activity for those who wish to use it that way.
(From OE-Core rev: da2e01cacd98715318a5307fe0618dbca0cf1fe7)
Signed-off-by: Chuck Wolber <chuck.wolber@boeing.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
devtool modify/upgrade are not currently equipped to handle conditional local files
in SRC_URI, and provide only the main no-override set in a workspace under
source/component/oe-local-files/ (this is done via meta/classes/devtool-source.bbclass).
On the other hand, updating the changes from workspace into a recipe
is run iteratively against all overrides; this works for patches (as they
all are directed into their own override branches in the workspace
git source tree), but breaks down when trying to match local files
in a workspace against local files in overridden SRC_URI lists, resulting in
bad recipe breakage.
(there's an additional twist here: existing code has a guard against this
but the guard relies on metadata in workspace .bbappend that is only there
in modify operations, but not upgrades. This commit replaces the guard
with a general check that will work everywhere).
Implementing multiple sets of local files is significant work; let's for now
simply not touch local files in recipes except when on the no-override variant.
Also, adjust the selftest cases to include conditional local files in sample
recipes, so the situation is covered by the tests.
(From OE-Core rev: 3a8654b860fa98f94e80c3c3fff359ffed14bbe7)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These enable running custom shell setup and cleanup commands
before and after qemu. Enables machine configurations to for
example run qemu with swtpm to emulate TPM devices.
Example config with meta-tpm2 based swtpm in machine config:
* image recipe depens on swtpm-native to get the native tools to PATH,
same handling as qemu itself
do_testimage[depends] += "swtpm-native:do_populate_sysroot"
* startup commands for swtpm daemon, note that swtpm
socket file has 107 character limit and absolute paths to build environment
will not work
QB_SETUP_CMD = " \
test -d '${IMAGE_BASENAME}_swtpm' || ( mkdir -p '${IMAGE_BASENAME}_swtpm' && \
swtpm_setup --tpmstate '${IMAGE_BASENAME}_swtpm' --tpm2 --pcr-banks sha256 ); \
swtpm socket --tpmstate dir='${IMAGE_BASENAME}_swtpm' \
--ctrl type=unixio,path='${IMAGE_BASENAME}_swtpm/swtpm-sock' \
--log level=40 --tpm2 -t -d \
"
* qemu startup command in machine config is configured enable swtpm device
QB_OPT_APPEND += "-chardev socket,id=chrtpm,path='${IMAGE_BASENAME}_swtpm/swtpm-sock' -tpmdev emulator,id=tpm0,chardev=chrtpm -device tpm-tis-device,tpmdev=tpm0"
* in this case, swtpm daemon stops automatically with qemu machine, but
QB_CLEANUP_CMD could be used to kill a specific process and wipe
temporary files
Now runqemu and testimage.bbclass can be used with swtpm.
(From OE-Core rev: d5c38964a4458aa31ec37810773ecc4f5d410dbe)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'devtool modify' writes additional settings to workspace .bbappend so that this
can be handled correctly, but 'devtool upgrade' does not. This adds the missing
settings.
In particular, local files should not anymore mysteriously disappear from
SRC_URIs on upgrades.
(From OE-Core rev: 0817aa5537a8d7cc9591c53dfaa1d225f4c327f7)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For unknown reasons we've never seemingly run the check layer script
against OE-Core itself. This isn't entirely straightforward as the core
layer is a bit of a special case, we can't for example compare signatures
against ourselve and we can't remove core from bblayers.conf.
Core does have distro, machine and software components too, in the case
of distro, our fallback default settings. Whilst the qemu machines could
be split into a seperate layer directory, core wouldn't then parse at all
standalone due to the lack of any machine so it seems a bit pointless to
do that.
These changes tweak the script to handle core's special cases, specifically
to allow distro and machine directories and to account for the README placed
a directory level higher than other layers.
(From OE-Core rev: ba312ed228507d05f280aeb96819d671b01400b8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake additions for the addpylib API mean we need to update the parsing
function call to be clear we're parsing in configuration context.
(From OE-Core rev: ef7677dc90fac089f8b9f6da301cca022ed7284c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It seems some layers want to subvert the intent of LAYERSERIES_COMPAT
so bitbake is going to have to become stricter about the values there.
To work with this, use LAYERSERIES_CORENAMES to generate the entries in
LAYERSERIES_COMPAT instead of the current magic LAYERSERIES_COMPAT_core
value which may not continue to work.
The downside to this is when migating between releases, people would
need to update devtool workspace layer.conf files. I guess you could
argue this is a feature!
(From OE-Core rev: 96ff9baa8ead57504f40f362ed3a4aaa776d1b58)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When starting to use combo-layer, or if someone else is using it too,
the local last_revision may be incorrect.
This command will forcibly update the last_revision config values to the
latest SHA on the remote branch that is tracked.
(From OE-Core rev: 2bb5d12ecd1b0273983f7c05699f34dd64b11c25)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake may not be configured, and bb isn't imported anyway.
Instead just use os.rename(), and take the filename from the file object
instead of duplicating logic.
(From OE-Core rev: 528f4fb3683d048537604e4562ea758968060d62)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The /proc/pressure support in buildstats is creating directories in the
buildstats tree called reduced_proc_pressure, which confuses the parsing
logic as that cannot be parsed as a name-epoc-version-revision tuple.
Explicitly skip this directory to solve the problem.
(From OE-Core rev: 24f0331f0b7e51161b1fa43d4592b491d2037fe9)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather, set it similarly to the overall network config.
(From OE-Core rev: d5e2ef31b151825613b62e58034e81f72526a944)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The eliminates having to do extra steps (e.g. meta-ide-support)
when booting an image with a nfs mount as rootfs - startng runqemu
with a nfs mount starts to 'just work' after building an image.
(From OE-Core rev: f73e370bec16d206592a7ca01b4a86b1d1316ada)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is the first release in 13 years;
I have reviewed the accumulated patches, and dropped some of them
where purpose or issue being fixed is not clear. Specifically:
0001-Add-listen-action-for-a-tcp-socket.patch
0001-daemon.c-Libtirpc-porting-fixes.patch
fixed upstream in
84ab475f93
0001-attr-fix-utime-for-symlink.patch
addresses an open issue in
https://github.com/unfs3/unfs3/issues/4
please rebase and re-submit as a PR if the problem is still present.
alternate_rpc_ports.patch
unnecessary as of
https://git.yoctoproject.org/poky/commit/?id=6bb9860ef7ba9c84fe9bd3a81aa6555f67ebd38e
Command line options introduced by the patch no longer used anywhere.
fix_compile_warning.patch
merged upstream.
fix_pid_race_parent_writes_child_pid.patch
rebased and re-submitted upstream.
no-yywrap.patch
dropped as backport.
relative_max_socket_path_len.patch
needs to be re-submitted by the original author, purpose and reproducer scenario unclear.
rename_fh_cache.patch
merged upstream.
tcp_no_delay.patch
purpose and use case for oe unclear.
unfs3_parallel_build.patch
fixed upstream in
987d32ca12a39a78995c
Drop -N option from oeqa nfs helper and runqemu helper;
the option was provided by tcp_no_delay.patch
and is not needed for the tests or qemu.
Drop ad hoc libtirpc support; upstream supports it directly now.
Drop the check for portmap/rpcbind, it is unnecessary as of
https://git.yoctoproject.org/poky/commit/?id=6bb9860ef7ba9c84fe9bd3a81aa6555f67ebd38e
(From OE-Core rev: fa2f7cf545137b071db97015bca5b70d77566cd8)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With default slirp port forwarding config qemu listens on TCP ports
2222 and 2323 on all IP addresses available on the build host. Most
use cases with runqemu only need it for localhost and it is not
safe to run qemu images with root login without password enabled
and listening on all available, possibly Internet reachable network
interfaces. Limit qemu port forwarding to localhost 127.0.0.1 IP
address. Now qemu machine SSH and telnet ports are only
reachable from the build host machine, not full Internet.
If qemu machine needs to be reachable from network, then it can
be enabled via local.conf or machine config variable QB_SLIRP_OPT:
QB_SLIRP_OPT = "-netdev user,id=net0,hostfwd=tcp::2222-:22"
(From OE-Core rev: c6b1e3d50bf2feea80b70a42c6fad868fa9e6042)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds argument parsing to the conversion script so that the fields that
the script uses to do conversions can be customized on the command line.
The intention is to allows easier customization without having to fork
the script, and allow automated checking on 3rd party layers via CI
without false positives
(From OE-Core rev: b9551f9180bf9f13fb1c480b5b7892fdc831ffcd)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ext2/3/4 FS contains not only mtime, but also ctime, atime and crtime.
Currently, all the files are being added into the rootfs image using
mkfs -d functionality which affects all the timestamps excluding mtime.
This patch ensures these timestamps inside the FS image equal to
the SOURCE_DATE_EPOCH if it is set.
(From OE-Core rev: 75d2dd0ea7790db2e8ee921784ca373abff2df65)
Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of changing the script environment to affect the child
processes, make a copy of the environment with modifications and pass
that to subprocess.
Specifically, when dri rendering is enabled, LD_PRELOAD was being passed
to all processes created by the script which resulted in other commands
(e.g. stty) exiting with a failure like:
/bin/sh: symbol lookup error: sysroots-uninative/x86_64-linux/lib/librt.so.1: undefined symbol: __libc_unwind_link_get, version GLIBC_PRIVATE
Making a copy of the environment fixes this because the LD_PRELOAD is
now only passed to qemu itself.
(From OE-Core rev: 2232599d330bd5f2a9e206b490196569ad855de8)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In case user requested to build a binary repeatable package,
it's required to honor the SOURCE_DATE_EPOCH environment
variable. So forcefully set mtime inside all the routines
which modify fstab in case it is updated.
(From OE-Core rev: 99719a3712a88dce8450994d995803e126e49115)
Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code to parse arguments was inadvertently skipping all arguments in
the elif block after gl-es if it was specified on the command line.
(From OE-Core rev: 718bb8d56f6a24c86e67830a7d13af54df2ebb4e)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When SOURCE_DATE_EPOCH variable is set, binary repeatable build
is expected. This commit implements reproducable disk identifiers
in such a case using its value as a Random seed.
(From OE-Core rev: 2c0c54e12169e76f16fb3398904bc897a9190397)
Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
--include-path can be used for placing files on ESP,
like dtbs.
(From OE-Core rev: 46ca73f504b0792766dc4d4bbad38beb04ef4ed0)
Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since the commit "populate_sdk_base/images: Drop use of 'meta' class and
hence do_build dependencies"[1], builds of images or SDKs don't
recursively depend on the top-level do_build target. This is typically
a good thing: images just depend on the packages themselves and those
dependencies already exist, but they don't need each recipes sysroot to
be populated.
However, eSDK generation is partly done via the script oe-check-sstate,
which does a 'dry-run' build of the target and collates all of the
sstate that is used. With this commit the sstate that is used is a
fraction of what would be needed in the SDK, specifically there are no
sysroots populated during the build, so there are no sysroots in the
SDK.
This is obviously a problem, as the entire point of an eSDK is to
contain a sysroot. Resolve this problem by forcing bitbake to run the
build task for all targets, so that all potentially needed sstate is
collated.
[YOCTO #14626]
[1] 41d7f1aa2c
Tested-by: Andrej Valek <andrej.valek@siemens.com>
(From OE-Core rev: 1b62344f919b5122f048b6409d09386d7d6dd3cd)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The scriptutils import isn't used, there's no need to run bitbake
in a shell environment, and invoke bitbake as a list instead of a
string.
(From OE-Core rev: 663aa284adf312eb5c8a471e5dbff2634e87897d)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Support installing buildtools-make-tarball that is built in version 4.1
and later for build hosts with a broken make version. Also update the
default version values to 4.1.
(From OE-Core rev: 5d539268d0c7b8fad1ba9352c7f2d4b81e78b75c)
Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Also, write to it only if it does not already exist.
As discussed here[1], reading from templateconf.cfg serves no purpose:
it exists only if the build directory has already been initialized, and so
the scripts will not change anything in the build directory anyway.
My adjustment is to keep the file however; it is useful as a pointer
to the original template, which can be utilized by config management
tools in the future.
[1] https://lists.openembedded.org/g/openembedded-architecture/topic/template_handling_in_oe_core/93968540
(From OE-Core rev: bcfaeb8ed503ce1a5add3ec882f0e0c3bccfd3d2)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit 20d43a2 allows changes in fstab for the / mount. However, if the
fstab contains / already then this results in two entries for / in the
fstab, which is confusing and results in systemd producing errors on
boot:
systemd-fstab-generator[11101]: Failed to create unit file
/run/systemd/generator/-.mount, as it already exists. Duplicate entry
in /etc/fstab?
Wic should be extended to support merging mount points in fstab, but as
we're about to release revert the patch that introduced this change as
it hasn't been in a release yet, and the issue which prompted the patch
in the first place can be revisited post-release.
[ YOCTO #14865 ]
(From OE-Core rev: 4a7b3ef0a69b7e08030519b712a6fe4dd199c571)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Many git repos prefer https:// nowadays and many removed support
for git://.
This breaks the script when using github.com even when selected remote
is ssh (git@github.com:openembedded/...), it will re-write it to git://
before calling git pull-request causing:
openembedded-core $ scripts/create-pull-request -u github -b jansa/artifacts -o pull-kernel
NOTE: Assuming local branch HEAD, use -l to override.
fatal: unable to connect to github.com:
github.com[0: 140.82.121.3]: errno=Connection timed out
warn: No match for commit ea003bd026aa24bb4c8b7562f44ed6512e921259 found at git://github.com/shr-distribution/oe-core
warn: Are you sure you pushed 'jansa/artifacts' there?
ERROR: git request-pull reported an error
(From OE-Core rev: 64c466920b808c35d1ac87b47cf438bc79becea7)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to be able to debug issues on the host that have an impact on the
guest, it is useful to get the uptime of the host while starting so we can
match with the events in dmesg. Also include the uptime when cleaning up.
(From OE-Core rev: 2d96499823f7de6e16a461426491e015ba63c1ec)
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This holds true for the default core/poky template, but not
anything else. Eventually there will be fragments and associated
tooling.
(From OE-Core rev: 3e4015d54e91ce4d312964b036784576a4b22566)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The intent of the templateconf.cfg file is to cache the value of
$TEMPLATECONF. To do this, it needs to be updated in case TEMPLATECONF
is manually specified.
Before, the following commands:
TEMPLATECONF=foo . oe-init-build-env
TEMPLATECONF=bar . oe-init-build-env
. oe-init-build-env
would cause the templates in foo to be used for the last source of
oe-init-build-env, while afterwards, bar is used as expected.
(From OE-Core rev: 37c122dfdac03a98bf2bf8c8f7676ec25136d402)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before commit 7b96dc80 (scripts/oe-setup-builddir: write to
conf/templateconf.cfg after the build is set up), the path written to
templateconf.cfg was relative if $TEMPLATECONF was relative, but
afterwards it became absolute.
Restore the original behavior of saving the relative path to
templateconf.cfg.
(From OE-Core rev: 49e75009780a09873dc3987ae1c40f2ac95bc667)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoid the following warnings:
* SC2086: Double quote to prevent globbing and word splitting.
* SC2164: Use 'cd ... || exit' or 'cd ... || return' in case cd fails.
* SC2166: Prefer [ p ] || [ q ] as [ p -o q ] is not well defined.
* SC2236: Use -n instead of ! -z.
(From OE-Core rev: a5aa5065d5ebe9f320cb1415c6ff4d5d5772f630)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The validation of the templates directory is supposed to be run as long
as $TEMPLATECONF is defined, but it was only done if the directory did
not exist.
(From OE-Core rev: dcca9ee6f06e9eacd6507f57bd0a5012ea343aa9)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This sets the following order:
1. TEMPLATECONF set in the unix environment (for example as a
prefix to '. oe-init-build-env').
2. If empty then $BUILDDIR/conf/templateconf.cfg
3. If absent then $OEROOT/.templateconf (oe-init-build-env sets OEROOT
to where it is, which is poky or oe-core).
I believe this is more logical and consistent with common practice;
it also avoids situations where the user is stuck with cryptic
errors because templateconf.cfg contains something invalid and
that something cannot be overridden from command line.
(From OE-Core rev: 7296c42b8c58e6e08f4210eb7b51d852b1ee950e)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was done before any of the sanity checks or actual setup, which
resulted in templateconf.cfg always getting written out, whether it
would be valid or not, and the user becoming stuck with it.
(From OE-Core rev: 7b96dc80298af6f669c885056a80a48bdd50e547)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a leftover from one of the RFC iterations, where
the property contained available machines, distros and templates.
As all of those were dropped from the final version, there is no
reason to list the layers either anymore.
Normally this would be a backwards incompatible change, but as
the layer setup itself was just merged, I think we can do a quick
fixup :-)
(From OE-Core rev: 64a774de0e154ef81f20853fec903b17d9985a72)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is done only for default oe-core/poky templates; for anything
else the locations themselves need to be migrated first, and there
is no way to tell where they would be.
(From OE-Core rev: 9da0dd350a451676e3d4a1f09f90ec0234047ab7)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This script can be used directly from poky or oe-core, or can be copied directly
into alayer or any other repository - it is self-suffucient and requires only python3
and git on the host where it will run. It is also copied by the bitbake-layers
layers-setup plugin together with the json, unless requested otherwise.
1. How to restore the layers from the saved configuration:
a) Clone the bootstrap layer or some other repository to obtain the json config and the setup script that can use it.
(use 'bitbake-layers create-layer-setup' from the previous commit to create them)
b) Running with default options:
(note: this will work to update an existing checkout as well)
alex@Zen2:/srv/work/alex/my-build$ meta-alex/setup-layers
Note: not checking out source meta-alex, use --force-bootstraplayer-checkout to override.
Setting up source meta-intel, revision 15.0-hardknott-3.3-310-g0a96edae, branch master
Running 'git init -q /srv/work/alex/my-build/meta-intel'
Running 'git remote remove origin > /dev/null 2>&1; git remote add origin git://git.yoctoproject.org/meta-intel' in /srv/work/alex/my-build/meta-intel
Running 'git fetch -q origin || true' in /srv/work/alex/my-build/meta-intel
Running 'git checkout -q 0a96edae609a3f48befac36af82cf1eed6786b4a' in /srv/work/alex/my-build/meta-intel
Setting up source poky, revision 4.1_M1-372-g55483d28f2, branch akanavin/setup-layers
Running 'git init -q /srv/work/alex/my-build/poky'
Running 'git remote remove origin > /dev/null 2>&1; git remote add origin git://git.yoctoproject.org/poky' in /srv/work/alex/my-build/poky
Running 'git fetch -q origin || true' in /srv/work/alex/my-build/poky
Running 'git remote remove poky-contrib > /dev/null 2>&1; git remote add poky-contrib ssh://git@push.yoctoproject.org/poky-contrib' in /srv/work/alex/my-build/poky
Running 'git fetch -q poky-contrib || true' in /srv/work/alex/my-build/poky
Running 'git checkout -q 11db0390b02acac1324e0f827beb0e2e3d0d1d63' in /srv/work/alex/my-build/poky
2. Command line options:
alex@Zen2:/srv/work/alex/my-build$ meta-alex/setup-layers -h
usage: setup-layers [-h] [--force-bootstraplayer-checkout] [--destdir DESTDIR] [--jsondata JSONDATA]
A self contained python script that fetches all the needed layers and sets them to correct revisions
optional arguments:
-h, --help show this help message and exit
--force-bootstraplayer-checkout
Force the checkout of the layer containing this file (by default it is presumed that as this script is in it, the layer is already in place).
--destdir DESTDIR Where to check out the layers (default is /srv/work/alex/my-build).
--jsondata JSONDATA File containing the layer data in json format (default is /srv/work/alex/my-build/meta-alex/setup-layers.json).
(From OE-Core rev: 58f94471675aef9ac6d15637ac5d8e69cc304c7a)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
specifically that ../../layer.conf exists, and that second-from-last
component in the path is 'templates'.
This requires tweaking template.conf creation in eSDK bbclass, as
we need to ensure that the path in it is valid, and exists
(which may not be the case if the SDK is poky-based).
(From OE-Core rev: c6f2b57be8893ee58f20cc29d8ec3a5a6edf7c07)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These are typically auto-extracted with modify/upgrade from recipes
and can be easily recreated. On the rare occasions where they need
to be reused, they are still available under workspace/attic (which
is already used for old recipes and appends), so nothing gets lost.
This avoids the annoyance of devtool refusing to proceed because
there is a previous source tree in workspace/sources.
For independent source trees behave as before: do nothing.
Adjust the test that previously deleted those trees by hand.
(From OE-Core rev: 9bfb95d070d68d5ab5adfe0ea096f5fbf9cad8b0)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's not possible to set the label (of gpt entry) normally
when I want to use non-listed fstype as a rawcopy.
Example)
part ? --source rawcopy --ondisk mmcblk0 --label mypart --sourceparams file=mypart.raw
To resolve this problem, this patch addes a 'none' fstype
and ignore do_image_label on rawcopy (that actually set the partition label.)
(From OE-Core rev: 949cf797eb54f3d22a093910087e937633ce9b23)
Signed-off-by: JeongBong Seo <jb.seo@lge.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Where there isn't a copyright statement, add one to make it explicit.
Also drop editor config lines where they were present and add license
identifiers as MIT if there isn't one.
(From OE-Core rev: deb3ccec53e0bd63bc4235cf2b0d3fc781687361)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If there is insufficent space to change the interpreter, we were
printing an error here but the overall script did not return an error
code, and thus the SDK installation appeared to succeed - but some of
the binaries will not be in a working state. Allow the relocation to
proceed (so we still get a full list of the failures) but error out at
the end so that the installation is halted.
(From OE-Core rev: c5a9a448e462d3e5457e8403c5a1a54148ecd224)
Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* easier to find than searching for the highest value in the graph, when the graph is really wide
(From OE-Core rev: 615e04554aa8442d89ea186b30f23adeb1e64762)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
scripts/pybootchartgui/pybootchartgui/draw.py:820: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if (OPTIONS.show_pid or OPTIONS.show_all) and ipid is not 0:
scripts/pybootchartgui/pybootchartgui/draw.py:918: SyntaxWarning: "is not" with a literal. Did you mean "!="?
if i is not 0:
(From OE-Core rev: 8d996616f0ca57220d939a41ca9ba6d696ea2a4f)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* memory pressure is already collected in buildstats, render it as well
when available
(From OE-Core rev: 42010d0812246a418f30b4f1d9fbd3f374a3bbe9)
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
