Print error messages to stderr instead of stdout.
The commands run as part of send-auto-cc-cmd and send-auto-to-cmd b4
commands will make b4 raise an Exception if they return a non-zero
return code and it will only print the content of stderr before that.
Because print defaults to stdout, b4 would raise the exception and not
tell the user why. This commit should now provide a hint to the user
about what went wrong.
Reported-by: Khem Raj <raj.khem@gmail.com>
(From OE-Core rev: 225990c55a5cbf8498e2aa775e6406c27a4c0ea7)
Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Running, e.g., `devtool reset sdbus-c++` would result in the following
error:
re.error: multiple repeat at position 35
This was due to the ++ in the recipe name, which would be treated as an
incorrect regular expression in _reset().
Use re.escape() to make sure all characters in the recipe name are
treated literally.
(From OE-Core rev: 6e73bd9b3e6d529752db93879f2c0ed53873dd1a)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If tag.gpgSign is configured in Git's configuration, then creating the
devtool-base tag will fail (if Git's core.editor is not configured) or
it will hang (when trying to open the editor). This is beacause
tag.gpgSign causes git tag to create annotated tags. To avoid this,
specify --no-sign as argument to git tag.
(From OE-Core rev: f96e955b29aad6a9e0b20f8e9a4987ace3808c91)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Auxiliary script to clean the hashserver database based on the files
available in the sstate directory.
It makes used of the new "hashclient gc-mark-stream" command to mark all sstate
relevant hashes as "alive" and removes everything else from the
database.
Usage example:
```
./scripts/clean-hashserver-database \
--sstate-dir ~/build/sstate-cache \
--hashclient ./bitbake/bin/bitabke-hashclient \
--hashserver-address "ws://localhost:8688/ws" \
--mark "alive" \
--threshold-age 60 \
--clean-db
```
(From OE-Core rev: f6737c762ac11f7653a64fac58428428c4222d0f)
Signed-off-by: Alexander Marques <c137.marques@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This extends the previous commit, so that download URLs are preserved
in addition to actual artefacts. By default it's all written to a temporary
directory and erased together with artefacts, but users can choose
to preserve both:
$ install-buildtools -D --downloads-directory=.
...
$ ls -1
buildtools_url
checksum_url
x86_64-buildtools-extended-nativesdk-standalone-5.1.2.sh
x86_64-buildtools-extended-nativesdk-standalone-5.1.2.sh.sha256sum
Additionally, rename check_url variable into checksum_url as it's
a better name.
(From OE-Core rev: 76748d27fd5088971259d08fdeae5c86a054452b)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default the script puts everything it downloads into a temporary
directory and erases it after unpacking and installing the buildtools.
This isn't great for traceability and reproducibility of builds
(being able to see what was downloaded exactly, and being able
to reproduce setting up a build, especially if the buildtools
download location isn't available for whatever reason).
This commit adds an option to download items into a specified directory
and keep them there. I would particularly like to use it with
bitbake-setup, where an optional feature to install the buildtools
(exact implementation details tbd) would ensure the tarball remains
available on local disk.
(From OE-Core rev: fc8cedd899f7e5d06215a71808dd0827ccdcf849)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update to the 5.1.2 release of the 5.1 series for buildtools.
(From OE-Core rev: 6766bce806a7e4fd62f6666db731eefc1b98c5ff)
Signed-off-by: Aleksandar Nikolic <aleksandar.nikolic@zeiss.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds a template for a python project that processes the SPDX 3.0.1
output from a build and lists all the files on the root file system with
their checksums
This is intended to be an example to show how to deal with the SPDX data
to do common tasks.
(From OE-Core rev: 3d9c5588ce6181b519810e3378b55826ffcaee49)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
While multiple dynamic loader existed, in order to make executable file is
interpreted by the expected dynamic loader, relocating interpreter only if
the new dynamic loader and executable file have the same arch
[ YOCTO #15722 ]
(From OE-Core rev: 0c1f2bf3d8e31e66d28aaa81a7c1658f160bb79e)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, the perf report charts use start_time which is when the build
is run. If running builds for commits in the past, this is inaccurate so
switch to using the recently added commit_time if the field is present.
Over time, the reports will switch to using the more accurate values.
(From OE-Core rev: 8c22efd444b0b1486c43f01b41cba891a47c7520)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Both OE-Core and poky git repos have a .b4-config, both at the same
location and both different.
Therefore, changes made to .b4-config for OE-Core need to be sent to the
OE-Core mailing list and changes made to .b4-config for poky sent to the
poky mailing list.
Changes made to OE-Core's .b4-config can only be made from a vanilla
OE-Core git repo, not from poky git repo. With the current content of
OE-Core's .b4-config, b4-wrapper-poky.py script is NOT called to figure
out where to send patches, b4 gets it from the hardcoded value in
send-series-to.
Thus, we can say that if b4-wrapper-poky.py detects changes for
.b4-config, those changes are for poky since this script wouldn't be run
by b4 outside of poky git repo.
(From OE-Core rev: 1006712027812326fdeade2ab539b42bdf5b10e6)
Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow the '+' symbol as a valid character in filenames listed in the
IMAGE_EFI_BOOT_FILES variable.
The '+' symbol might be used to support boot counting for boot entries,
as described in the UAPI Boot Loader Specification [1]:
The boot counting data is stored in the name of the boot loader entry.
A boot loader entry file name may contain a plus (+) followed by a
number. This may optionally be followed by a minus (-)
followed by a second number. The dot (.) and file name suffix
(conf or efi) must immediately follow.
Boot counting is enabled for entries which match this pattern.
Example:
IMAGE_EFI_BOOT_FILES:append = " entry.conf;loader/entries/entry+3.conf"
[1] https://uapi-group.org/specifications/specs/boot_loader_specification/#boot-counting
(From OE-Core rev: 3f25822281eb9423ff86105eaebb0bed48663648)
Signed-off-by: Igor Opaniuk <igor.opaniuk@foundries.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We triggered a test of an older revision to narrow down when performance
changed. The issue is that git's timestamps are granular to 1s. We'll
usually merge a set of commits at the same time so they will all have
the same timestamp for a block of them. This means that even if we use
the commit date, all the points can't be distinguished on the graph.
The author date doesn't work either as the commits are not merged in
author date order.
To solve this this patch adds the commit_count chart as a separate tab
next to the start_time chart
(From OE-Core rev: b263edd33f6c895238d81ef148c0445fcd0aa268)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
poky is a combo-layer containing BitBake, OpenEmbedded-Core and Yocto
Documentation source code into one big repo. It is not uncommon to have
people develop patches for either of those projects from a poky git
repo.
However, it is unlikely those patches are to be sent to the poky mailing
list as very few files contained in the poky git repo actually are
poky-specific. So we need a way to identify to which mailing list a
patch is destined to be sent.
Additionally, because the source code in openembedded-core is
merged/imported at the root of the git repo of poky, its .b4-config
introduced in the previous commit will be used if not overridden (which
will be done in a separate commit specific to the poky git repo). We
need to provide a different .b4-config in poky.
Therefore, this wrapper is used to identify automatically which mailing
list a patch series needs to be sent to (via b4 prep --auto-to-cc) and
does some additional checks (via b4 prep --check) such as making sure a
patch doesn't modify two different projects at the same time or that
multiple projects are modified by different patches in the same patch
series.
This wrapper script is meant to be used by poky's .b4-config. Ideally
the b4 prep --check part could be offloaded to `patchtest` once it
supports running on source different from OE-Core.
Note that b4 makes sure that an address doesn't appear twice in the
recipient list. There's also no priority in the type of recipient list,
so if the address appears first in Cc and then in To, only the Cc will
be added. The opposite is true as well.
(From OE-Core rev: a1d629a0823cf724bda607ef8d88ee4f722c3a08)
Signed-off-by: Quentin Schulz <quentin.schulz@cherry.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
SGI-1 is not a SPDX license identifier. However, the SGI-1 license has
the same license text as SGI-OpenGL. Map the old SGIv1 name to SGI-OpenGL.
(From OE-Core rev: e97a9c3c86a8fe27a26ad69174ba50e5228846e5)
Signed-off-by: Bastian Germann <bage@debian.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The goal is to make this more Pythonic, as it was a little cryptic the
first time I looked at it.
(From OE-Core rev: 80285b0f9b716ebad87a2feb08f12f87d70c27e3)
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The ide-sdk command bootstraps the SDK from the bitbake environment
before the IDE configuration is generated. In the case of the eSDK
installer, the bootstrapping is performed during the installation of
the eSDK installer. Running the ide-sdk plugin from an eSDK installer
based setup would require skipping the bootstrapping and probably taking
some other differences into account when generating the IDE
configurations.
This would be possible. But it will probably never be implemented, as
running devtool ide-sdk directly from the bitbake environment is much
more flexible.
Also, some of the recent improvements that have made it into the core
have the potential to make the eSDK installer obsolete at some point in
the future:
- bitbake-layers create-layers-setup replicates the layers
- bitbake-config-build replicates the build configuration
- The new sstate mirror features replicate the sstate
- bblock locks the sstate more flexible than the eSDK installer
- devtool ide-sdk bootstraps the SDK directly from the bitbake
environment. The same environment-setup... file is provided with
--mode=shared.
The devtool modify based workflow is supported since always by devtool
and also the default --mode of devtool ide-sdk.
These functions essentially cover what the eSDK installer does without
a need for the current implementation of the eSDK installer and the
populate_sdk_ext, which is hard to maintain and takes a lot of time to
build.
This means that instead of making the ide-sdk plugin compatible with the
eSDK installer, we should rather replace the current implementation of
the eSDK installer and populate_sdk_ext with an implementation that can
replicate a normal bitbake environment in a convenient way where the
ide-sdk plugin also just works without additional complexity.
(From OE-Core rev: 177aa72b37f2061ff3311ec5dbb33aa56a5ba006)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These imports are not needed.
(From OE-Core rev: c0e9e35843004aaac5bdcc12fa1f6bf8d2da0abb)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It never had to be a global anyway
(From OE-Core rev: 2d5557f4f5d228e94bc7a55ae5fd72b70b5c93e4)
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'read_workspace' can now access it via the 'context' that's passed in
(From OE-Core rev: bfc525f6fdc8990b312123ac22d93118322b4e34)
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Please excuse the usage of 'typing' slipping in here - it's just how
dataclasses work :/.
(From OE-Core rev: 207cdead039383780bd39adbaf2a17b679889c63)
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add wic kickstart help for undocumented options for part/partition and
bootloader command.
(From OE-Core rev: 59786183c41b93b49fe97926dd77904c8d66b356)
Signed-off-by: Stefan Gloor <code@stefan-gloor.ch>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Also check that 'tools-debug' is not set in
IMAGE_FEATURES to determine if gdbserver is missing.
(From OE-Core rev: 28a8b35826302a40e7bb49f4bd3213fe7026f480)
Signed-off-by: Georgi, Tom <tom.georgi@karlstorz.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Otherwise the tool simply prints what went wrong and exits without error,
which makes it impossible for tools like bitbake-setup to determine that
the requested operation did not succeed.
(From OE-Core rev: d44ab1c3abb25ef08307558430d76a47cde07cc1)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit fe167e082cbde1c6d186ecdda531abef610ac2ac switched to requiring
lz4 instead of lz4c which allows us to support distros dropping lz4c.
However, it wasn't only OE that was still using the legacy lz4c, there's
a number of upstreams as well. For instance, it's only in the 6.13
kernel that CONFIG_KERNEL_LZ4 makes the switch from lz4c to lz4.
So, while this all gets ironed out, simply intercept calls to lz4c and
convert them to use lz4. This was picked instead of adding lz4c to
HOSTTOOLS_NONFATAL due to concerns about builds becoming
non-deterministic and failing late:
https://lore.kernel.org/openembedded-core/9c3143ebb7f9e17cfbd318ef0e17994aae7264be.camel@linuxfoundation.org/
(From OE-Core rev: c10b94d82d10058a9e26f7d6919a0d6d721a7c75)
Signed-off-by: Justin Bronder <jsbronder@cold-front.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The debug_build_config function was never called. Compiling with debug
optimized compiler flags was not working. Even with the
--debug-build-config flag set, the build configuration from the recipe
was used.
The devtool ide-sdk --debug-build-config approach didn't work very well
anyway. The problem is that changing the bbappend file doesn't work
while bitbake uses the bbappend file. As a workaround, it would be
possible to parse the recipe, get DEBUG_BUILD and the path to the append
file, exit tinfoil, change the bbappend file, reopen tinfoil and do what
ide-sdk is supposed to do. Such an implementation would be complicated
and slow.
Therefore, the code that was originally supposed to implement this is
removed from ide-sdk and the new --debug-build function of devtool
modify is used instead. Additionally, a hint should be given on how to
manually add DEBUG_BUILD = '1' to bbappend.
This is compatible with the VSCode Bitbake plug-in, which does not
support this parameter anyway.
(From OE-Core rev: 65950eb601c6c8aac0e4bc8683e544305346229d)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sort the keys of the generated CMakeUserPreset.json file to make it
easier to search and compare.
(From OE-Core rev: b886c26bf893878ba8eb6bee80dd0507e5cb0d2d)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a new option --debug-builds to automatically add DEBUG_BUILD = “1”
to the bbappend file of this recipe. This is especially useful when
invoking devtool modify before invoking devtool ide-sdk to perform a
remote debugging session.
(From OE-Core rev: fc17808799d2b667afbe4ea9837b66af70d47007)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update to the 5.1.1 release of the 5.1 series for buildtools.
(From OE-Core rev: 0b746655fcc133db563d791673484c4ca82cda8a)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Don't access private variables, instead use the new bb.build.listtasks()
function (from bitbake 185c4b)
(From OE-Core rev: d31a7718c16bd26efd6e174cb5e97fb088aad4bd)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the RawTextHelpFormatter formatter to preserve formatting in the
argpars help message.
(From OE-Core rev: ee94d7f99ff3c7e922fdb502fabbda3fd0cd8700)
Signed-off-by: Aleksandar Nikolic <aleksandar.nikolic@zeiss.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We want the results directory to be visable to other users, tweak the
permissions of the created directory to ensure this is the case.
(From OE-Core rev: ed9d887e8d71a800db19826264de552f7736dc6a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update to the 5.1 release of the 5.1 series for buildtools.
(From OE-Core rev: f6c1e5541ee8d215a0327263c9dd3f4cc128cd56)
Signed-off-by: Aleksandar Nikolic <aleksandar.nikolic@zeiss.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having all oe-selftest results on top of each other results in a large 640MB
json file which is hard to use. Split the results out per machine and test type.
This also stops the toolchain raw logs from overwriting each other meaning more
than one MACHINE is preserved.
(From OE-Core rev: 4b890f04bc7d147b4a11b824a84f3d2abd75ac54)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The duration values have pointless amounts of precision. Removing some of the
least significant digits reduces result size and makes the results easier to read.
(From OE-Core rev: a789a2e6d97bb8efd663226a17db8d1ca6c1e40f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We've improved the data stored for reproduicible builds. Teach resulttool how
to apply those cleanups when reprocessing data so we can reduce results file
sizes and make the data easier to process.
(From OE-Core rev: b799c57ae6d61c1b1c7035c8a2c4ba6ee08d1a81)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Improve the rawlogs handling to include ltp logs as well as the ptest ones to
reduce the size of the results git repos.
(From OE-Core rev: a0a1954d559609c2c1ca16936d0d68eb3c4c6b45)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Storing the log files inside the testresults git repo isn't scaling and isn't
really appropriate use of a git repository. Allow these to be optionally stored
in a separate filesystem location so the git repo can remain managable.
(From OE-Core rev: 1afc0f3d7e93fa8496be241e9622d3b9a6904bd5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using store mode, --all was broken as not all files were being preserved.
Fix this by limiting the scope of the git rm command.
(From OE-Core rev: 9604561d2022b6c76b1cb4186d40800d1affdd2b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using 4 space indentation in resulted in hundreds of megabytes of extra file size
in general use. Reduce this to make filesizes more managable and reduce the processing
cost. Some level of indentation and spacing does make the files more readable and allows
use of git diff so we need to retain some of it.
(From OE-Core rev: a274cdcaf852cca9497f0358f44dda99c06aacbe)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
