Some of the host distributions build the drivers in a way (RPATH/RUNPATH)
that tricks uninative loader into loading pieces of the host libc, if
the same pieces haven't been previously loaded by native binaries. Mixing
the two libc versions leads to failures.
This change ensures that the correct (uninative) versions are always in use.
(From OE-Core rev: 39c10816d5ec9d9c7952d786d7a3f942d25d0c27)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The wrapper is executed by host bash, and host bash refuses to run
when pseudo libc is preloaded via LD_PRELOAD (which is the case
when gl is enabled).
Only the fontconfig setting is carried over as local testing showed that
only that is necessary for the gui to look ok nowadays; adjust further
if necessary.
(From OE-Core rev: 34f152de14f803fcfe5c92c515bf585838bba10a)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
deploy-target may fail the second time with "pseudo abort" because
devtool-deploy-target-stripped is deleted outside of pseudo's fakeroot
context.
(From OE-Core rev: 2338a33b690b0bbe279cde3f73764911b239cb50)
Signed-off-by: Florian Amstutz <florian.amstutz@scs.ch>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
detect_layers() is very greedy and if it recurses into poky or bitbake
it will find the test suite layers, such as
bitbake/lib/layerindexlib/tests/testdata/layer4. This is a dummy layer
which claims to be openembedded-layer, so if the real openembedded-layer
is a dependency then layer4 may be used instead, which will cause
errors: initially because it's only compatible with Sumo, but later
because it doesn't contain any recipes.
Add a check that the set of layers we've found doesn't contain any
duplicate collection names with different patterns, and abort if that is
the case as the test will be non-deterministic.
(From OE-Core rev: 0df4bae4ec67d38442620fa08c839528b425e2a8)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The runqemu-ifup script performs a bunch of setup steps that
runqemu-ifdown attempts to undo later on. While a bunch of said setup
operations are considered fatal should they fail, the iptables based NAT
setup notably is not.
The tear down procedure in runqemu-ifdown, however, has the iptables
based tear down as the last operation, with the status of it determining
the overall status of the script. Hence, if this step fails, the script
is considered a failure overall. That is arguably inconsistent: If the
NAT setup did not succeed, the tear down cannot succeed either.
To ensure similarity of the two paths, let's not treat the last iptables
tear down operation any special and allow it to fail the runqemu-ifdown
script, but just ignore failures.
Background: we have seen a NAT related setup problem on the ifup path
(which didn't cause script failure), but then saw an issue bubbled up
when this operation was meant to be undone on the ifdown path.
(From OE-Core rev: 0ebcfb034bcad81efef5f746f0aa0b69772901a0)
Signed-off-by: Daniel Müller <muellerd@fb.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When creating a new recipe.bb file for a binary, don't use BP which
includes the version information, instead use BPN which is just the
name base Package Name.
Since PB is not specified, it takes the default:
PV = "1.0+git${SRCPV}"
But SRCPV is defined in terms of the SRC_URI, which leads to infinite
recursion (traceback below). Here are the pertinent variables which
cause the recursion:
SRC_URI = "git://github.com/lvc/abi-dumper;protocol=https;subdir=${BP}"
BP = "${BPN}-${PV}"
PV = "1.0+git${SRCPV}"
SRCPV = "${@bb.fetch2.get_srcrev(d)}"
def get_srcrev(d, method_name='sortable_revision'):
# ... trimmed
scms = []
fetcher = Fetch(d.getVar('SRC_URI').split(), d)
# ... trimmed
[YOCTO #14040]
(From OE-Core rev: 3b8d43fc53ee13d39abc3b2a1f706a97fcf752aa)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Prelink is being dropped by glibc in 2.36. It already causes issues with
binary corruption, has a number of open bugs and is of questionable benefit
without disabling load address randomization and PIE executables.
We disabled it by default a while back but left people able to use it.
We would be unable to maintain it alone without glibc support so remove
the remaining pieces.
(From OE-Core rev: 23c0be78106f1d1e2bb9c724174a1bb8c56c2469)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since commit 044fb04d in bitbake (fetch2: Allow whitespace only mirror
entries) there is no need to separate the entries in SSTATE_MIRRORS
with "\n".
(From OE-Core rev: 6e66ec3cd201d32d3d97df2222f41bb8c475fb69)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When debugging weird yocto-check-layer output it is useful to know
what the tool found when looking for layers.
(From OE-Core rev: 711e2d4d7baf36f8497741c14268d7f72d0db016)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'distutils' is deprecated in Python 3.10 with removal in Python 3.12
(~October 2023). Replace 'distutils.command.build_py' with
'setuptools.command.build_py'.
To avoid an AttributeError, we call super().__init__() which provides
the missing 'distribution' attribute. However, for some reason, __init__()
in 'setuptools.command.build_py.build_py' class requires a 'dist' positional
argument which must be a 'Distribution' instance. It is not clear why
'distutils.command.build_py.build_py' class does not require this.
There is still a check which decides to inherit setuptools3 vs distutils3
that will need to be refactored when we add pyproject.toml and setup.cfg
support for more modern PEP 517 packaging.
Once distutils3.bbclass is dropped, any recipe inheriting distutils3
will throw a parsing error. The plan is to move distutils*.bbclasses to
meta-python. However if meta-python is not in bblayers, the parsing
error would still occur.
[YOCTO #14610]
(From OE-Core rev: 619a3eb1266459daf16e10386113e9201fbf9cf5)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This updates buildtools to use the tarball from the 3.4 release which
contains some bug fixes and is what the autobuilder currently uses for
testing on older distros.
(From OE-Core rev: 7479861c60a1c205b9502c1a811ac3a9dc51cd07)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
New systemd is actually parsing this in systemd-network-generator
and fails if it is not fully formed. 'off' means 'static ip, do nothing':
https://www.kernel.org/doc/Documentation/filesystems/nfs/nfsroot.txt
(From OE-Core rev: 2cf12c8dde0f05917797f8b4a80883dc0647b95d)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is deprecated in python 3.12 and Fedora 35 is throwing warnings so
move to the new functions.
(From OE-Core rev: 655cd3f614d736416eab0d708b7c49674bf5c977)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The use of distutils.version.LooseVersion to check for GitPython > 0.3.1
is not really needed anymore since any supported distribution has at least
1.0.0 (centos-7 via epel7, debian-9, ubuntu-16.04)
If we want to reinstate this check, alternatives would be to require
python3-packaging on all hosts and use packaging.version.Version or
use an imported LooseVersion in bb.version.
[YOCTO #14610]
(From OE-Core rev: bc90dcae9f53ddc246942f4d9b8ae8943e3b9754)
Signed-off-by: Tim Orling <timothy.t.orling@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Separate licenses with & operator since it should be satisfied most use
cases and it is a reasonable assumption that all the licenses apply.
Furthermore flat, split and sort the licenses to minimize license string
changes.
Separate package licenses with & operator:
-LICENSE:${PN} = "MIT ISC"
+LICENSE:${PN} = "ISC & MIT"
Respect | and brackets in LICENSE:
-LICENSE = "BSD-3-Clause & (ISC & | & MIT)"
+LICENSE = "BSD-3-Clause & (ISC | MIT)"
Sort licenses:
-LICENSE = "MIT & BSD-3-Clause & ISC"
+LICENSE = "BSD-3-Clause & ISC & MIT"
Remove duplicates:
-LICENSE = "MIT & ISC & MIT"
+LICENSE = "ISC & MIT"
(From OE-Core rev: 60a84ecc53d20118c5e7f86dd3e3cafbfed1cf0a)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sort the output of guess_license function by license file to be
deterministic and to support self-testing.
(From OE-Core rev: ff2a7520813b3f775f96879d1905222e08b8c83f)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The commit 'meta/scripts: Manual git url branch additions (dc53fe75cc)'
sets the branch= parameter too early to master and thereby breaks the
-B/--srcbranch option.
ERROR: branch= parameter and -B/--srcbranch option cannot both be specified - use one or the other
(From OE-Core rev: 34ece8030e7a6a100b5e3e7b94e6c786c0e199a6)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If runqemu is killed, check that we have a valid PID for the qemu before
sending a kill() to it.
[ YOCTO #14651 ]
(From OE-Core rev: 0f3afbd3a6a6bef668612f818517df7543c0a683)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Relaxes the case requirements for checks in the README file so that word
like "Maintainer" and "Patch" are allowed
(From OE-Core rev: fdedf94d406ad3da85cb45d43ef87d3fdc8c14d4)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is permissible to have a layer's README reference another README. If
this is the case, avoid other checks. Do this by checking for the word
README (case insensitive) in the README file. This might be too
permissive, but follows the pattern of looking for exact words (like
"patch" or "maintainer") in READMEs.
(From OE-Core rev: 443c73d638519d6a7ea44d1c0e80d76306687ddc)
Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds:
- Test case to review if the README file specifies the maintainer.
- Test case to review if the README file specifies the word patch.
- Test case to review if the README file contains an email address.
[YOCTO #11131]
(From OE-Core rev: 5d4937bce191b7e22b807c0595d4845c88ecc560)
Signed-off-by: Dhruva Gole <goledhruva@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a configure.ac uses LIBCURL_CHECK_CONFIG it wants curl.
(From OE-Core rev: 16e830ca1352cee61e4ae4b98b1203f849bf71f3)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
GitHub URLs are automatically transformed to git: fetches, so handle
GitLab URLs too.
(From OE-Core rev: 651fb951819840fe122458ddbd852ee6c7ec0455)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a wrapper around ar calls for native/cross recipes. This wrapper adds
the -D option so that deterministic archives are built for native/cross
output. This improves the changes of hash equivalence matches and hence
build artefact reuse.
We don't need this in the target case since we compile binutils-cross
with an option making this the default. We need a wrapper since we need
to remove the "u" option and replace it with "D" but also allow things like
"--version" to continue to work too.
(From OE-Core rev: 59922c95fcb20c66634c5677012d490be2246b0b)
Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We disable the useradd code for nativesdk targets since we don't support
postinstalls or multiple users in those cases. This means any usage
of chown/chgrp inside do_install tasks won't work and would have to be
conditional. Rather than require all recipes to do that, add intercepts
of the calls and map those to root/root user/groups. We can't just ignore
them as some calls are used to remove host contamination from the host
user ID so they need to be made, just as root.
(From OE-Core rev: e1f40670c438e33cae87678425de72ca03566888)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We already support specifying the rootfs by PARTUUID. This adds general
support for letting the kernel find the rootfs by PARTLABEL.
(From OE-Core rev: 2fb247c5ecf057bb96649a3c0234794b4991c050)
Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use shutil.which to find the executable instead to silence the below warning:
$ cat tmp/work/intel_x86_64-poky-linux/core-image-base/1.0-r5/temp/log.do_image_wic
[snip]
DEBUG: Executing shell function do_image_wic
/path/layers/oe-core/scripts/wic:27: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
from distutils import spawn
INFO: Creating image(s)...
[snip]
[RP: Added conversion for missed function reference]
(From OE-Core rev: 3966cbf5c8a2dbc3a4f0f3eefdbeeb83f522bf87)
Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
lnr is a script that implements the same behaviour as 'ln --relative
--symlink', as at the time of creation --relative was only available in
coreutils 8.16 onwards which was too new for the older supported distros.
Now, however, everyone has a new enough coreutils, so we can remove this
script.
All users of lnr should be replaced with ln --relative --symbolic.
(From OE-Core rev: 723b6e40f5943426364bffce7c58ade65c4abbba)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Optionally allow to set QB_KERNEL_ROOT to e.g. "/dev/vda ro" to mount the
rootfs reay-only in Qemu.
(From OE-Core rev: 448eb1fd21287ba16b17e9402ce040b86ae3638c)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The URLs describing Kickstart are no longer valid and do not redirect to
the correct location. Update them with the correct location.
(From OE-Core rev: e5ac75f93c8128b0761af5fee99e8603ddd1657d)
Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Give the user a proper error message if there aren't packages built,
rather than a less friendly traceback.
[YOCTO #14619]
(From OE-Core rev: b14c176b7dd74b7d63ca0f72e6e00fbf209f5a0b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After the overrides change, the format of pkgdata changed and this
usage of configparser no longer works. This change is a bandaid to make
things work but the pkgdata format isn't very similar to ini files
so this may need to be reimplmented in a better way in the long run.
[YOCTO #14619]
(From OE-Core rev: 25a8ec6e2891b71bc280aacaf5f62ecc4b0bd1d1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There may be none, one or more spaces including tab before backslash in
SRC_URI. Use regex to check and update. It helps to avoid malformed uri
such as recipe open-iscsi-user in meta-openstack:
SRC_URI = "git://github.com/open-iscsi/open-iscsi.git;protocol=https ;branch=master \
And help to check more recipes such as concurrent-ruby in the same
layer:
SRC_URI = "git://github.com/ruby-concurrency/concurrent-ruby.git;protocol=https;tag=v1.1.6\
(From OE-Core rev: a69a53573b1987ee5834a6fc27763f9bbf5fe5a4)
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is not invoked with bitbake context as a result bb.utils is not
visible when this function is called during image creation and builds
fail e.g.
NameError: name 'bb' is not defined
(From OE-Core rev: df9dca9fe4dd1abfe5f3986389a8e8ff524da5d7)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Cc: Joshua Watt <JPEWhacker@gmail.com>
Cc: Devendra Tewari <devendra.tewari@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Not all machine definitions need to have SERIAL_CONSOLES defined, but
runqemu currently will fail with the following script error if not
present:
Traceback (most recent call last):
File "/builder/meta-arm/poky/scripts/runqemu", line 1604, in main
config.setup_final()
File "/builder/meta-arm/poky/scripts/runqemu", line 1446, in setup_final
self.setup_serial()
File "/builder/meta-arm/poky/scripts/runqemu", line 1381, in setup_serial
self.kernel_cmdline_script += ' console=%s' %entry.split(';')[1]
IndexError: list index out of range
To get around this issue, add a sanity check to runqemu to avoid the
parsing of SERIAL_CONSOLES if empty.
(From OE-Core rev: e8359000e765f876c041b6ccabe63f069f83efec)
Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Github are dropping support for git:// protocol fetching. Update the script
to learn about corner cases found in the previous conversion and
support remapping the github urls as needed too.
(From OE-Core rev: e59fe8279b209f67ff79b9d6dbb69389a64db236)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Following the scripted conversion adding branches to git://
SRC_URI entries, add the remaining references, mainly in the selftests
and recipetool.
(From OE-Core rev: 5340c0d688036c1be6c938f05d8a8c1e3b49ec38)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is uncertainty about the default branch name in git going forward.
To try and cover the different possible outcomes, add branch names to all
git:// and gitsm:// SRC_URI entries.
This update was made with the script added to contrib in this patch which
aims to help others convert other layers.
(From OE-Core rev: b51c405faf6f8c0365f7533bfaf470d79152a463)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Correct two issues:
1. Looking for dri.pc is only needed when gl is enabled.
2. virtio-vga-gl works only when gl is enabled via -display,
otherwise virtio-vga needs to be selected.
(From OE-Core rev: d6d5fe44510d2087b735758ac85e804533a1778c)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure the jinja2 module is available during bringup.
(From OE-Core rev: c46a6ec91bd40a458cb0ef5ec84bc0cc274d9cef)
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch switches the compressor from Gzip to ZStandard for ssate cache
files.
Zstandard compression provides a significant improvement in
decompression speed as well as improvement in compression speed and disk
usage over the 'tgz' format in use. Furthermore, its configurable
compression level offers a trade-off between time spent compressing
sstate cache files and disk space used by those files. The reduced disk
usage also contributes to saving network traffic for those sharing their
sstate cache with others.
Zstandard should therefore be a good choice when:
* disk space is at a premium
* network speed / resources are limited
* the CI server can sstate packages can be created at high compression
* less CPU on the build server should be used for sstate decompression
(From OE-Core rev: 0710e98f40ec7394567ede1f88f7c7bae306b925)
Signed-off-by: Henry Kleynhans <hkleynhans@fb.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the ability to replace DEPLOY_DIR_IMAGE with that path in
QB_OPT_APPEND. This allows for anything present in DEPLOY_DIR_IMAGE to
be passed into the qemu parameters. This is especially useful if you
want to run multiple flash images (as -bios only allows for one).
(From OE-Core rev: 1f0bec0421617e8aa9645c385195a755f0d44e75)
Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the README as license fallback if a license file is missing. Use the
linenumbers parameter of get_license_md5sums function to determine the
license text inside the README.
(From OE-Core rev: eff85c86f36673a1cb5a5dc8c66598e0dc457374)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The package.json files doesn't contain any licenses. The name of the
license doesn't comply the license requirements of most liceneses.
(From OE-Core rev: 194df9c240378b6befeb10183889093ec7bb4d5f)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support for linenumbers (begin and end lines) to licenses.csv. Add
an optional linenumbers parameter to get_license_md5sums to support
different use cases.
(From OE-Core rev: d5c4979669f125e73c24dcc73fa3c4f3787bbb62)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For consistency with "pydevshell" which is also used.
This addresses [YOCTO #14531]
(From OE-Core rev: 958133f4e97275d1f2c74a7e715c4506cca3582f)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"A unified kernel image is a single EFI PE executable combining an EFI
stub loader, a kernel image, an initramfs image, and the kernel command
line.
[...]
Images of this type have the advantage that all metadata and payload
that makes up the boot entry is monopolized in a single PE file that can
be signed cryptographically as one for the purpose of EFI
SecureBoot."[1]
This commit adds a create-unified-kernel-image=true option to the
bootimg-efi plugin for creating a Unified Kernel Image[1] and installing
it into $BOOT/EFI/Linux/ with a .efi extension per the the Boot Loader
Specification[1][2]. This is useful for implementing Secure Boot.
systemd-boot is the only mainstream bootloader implementing the
specification, but GRUB should be able to boot the EFI binary, this
commit however doesn't implement the necessary changes to the GRUB
config generation logic to boot the Unified Kernel Image.
[1] https://systemd.io/BOOT_LOADER_SPECIFICATION/#type-2-efi-unified-kernel-images
[2] https://systemd.io/BOOT_LOADER_SPECIFICATION/
(From OE-Core rev: b0573f240525df561ddef6e47cb285b217d38487)
Signed-off-by: Kristian Klausen <kristian@klausen.dk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a check before deleting path when using recipetool commands to avoid the following type of errors:
Traceback (most recent call last):
File "<workdir>/sources/core/scripts/lib/scriptutils.py", line 218, in fetch_url
shutil.rmtree(path)
File "/usr/local/lib/python3.7/shutil.py", line 476, in rmtree
onerror(os.lstat, path, sys.exc_info())
File "/usr/local/lib/python3.7/shutil.py", line 474, in rmtree
orig_st = os.lstat(path)
FileNotFoundError: [Errno 2] No such file or directory: '<workdir>/build/tmp/work/recipetool-usg7o81n/work/recipe-sysroot'
ERROR: Command 'script -e -q -c "recipetool --color=always create --devtool -o /tmp/devtool5sq_op37 'file:///<SRCTREE>' -x <workdir>/build/workspace/sources/devtoolsrcxc1b9zjq -N test" /dev/null' failed
(From OE-Core rev: b6aa8b47e023004ffd6958d1cec18c2d9c95d77b)
Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wic fstab-update creates invalid entries for partitons that are not supposed to
be mounted from userspace eg u-boot partitions.
The following lines were added to fstab on a rock-pi-4:
/dev/mmcblk1p1 loader1 vfat defaults 0 0
/dev/mmcblk1p2 reserved1 vfat defaults 0 0
/dev/mmcblk1p3 reserved2 vfat defaults 0 0
/dev/mmcblk1p4 loader2 vfat defaults 0 0
/dev/mmcblk1p5 atf vfat defaults 0 0
/dev/mmcblk1p6 /boot vfat defaults 0 0
With this patch only valid entries should be added
/dev/mmcblk1p6 /boot vfat defaults 0 0
(From OE-Core rev: 7aa678ce804c21dc1dc51b9be442671bc33c4041)
Signed-off-by: MarkusVolk <f_l_k@t-online.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This class was added by me back in 2007 and has changed one for a whitespace issue
since. It only has two lines and neither are particularly useful, one was replaced
by the nopackages class, the other adding recursive dependencies also is now
mainly problematic adding tons of unneeded dependencies. The name is hard to
understand and the class doesn't have a clear purpose. Drop it.
Remove the references in devtool (which may be the one reason to keep it around
but in my view still not worth it).
(From OE-Core rev: 6e2cbfc561dac89bf9183d24d90e52f7d9117826)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I've puzzled over why I see local eSDK test failures. It is due to having a
messy git tree locally, git then runs git gc in the background which races
against the deletion code for the publish test. Disable git gc to avoid this.
(From OE-Core rev: d938a74097202396beda2e3bb4f582342d6dfa1b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Avoid a rare divide by zero error if there isn't data point spread.
[YOCTO #14547]
(From OE-Core rev: d7e36d01e87ddf89f76f164a0b7d98f597a53fa5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The corrected line accidentally converted it to float,
which causes problems later on with python 3.10:
| File "/home/alex/development/poky/scripts/lib/wic/partition.py", line 278, in prepare_rootfs_ext
| os.ftruncate(sparse.fileno(), rootfs_size * 1024)
| TypeError: 'float' object cannot be interpreted as an integer
(From OE-Core rev: d1d260dd2d196d10379ed9e238bcb34f39f3a3b7)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
collections.abc is available since 3.3 and doesn't need special handling.
(From OE-Core rev: 01152c9410ba00274c8415a5d914dc33dfd0cf46)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows extra space to be added after the last partition and is
especially useful when free space is needed for ex: adding partitions on
first boot with ex: systemd-repart[1] and the image is tested in QEMU.
[1] https://www.freedesktop.org/software/systemd/man/systemd-repart.html
(From OE-Core rev: f81b188bcf5aa18746fd622eb7b5c0dcb0b5c93d)
Signed-off-by: Kristian Klausen <kristian@klausen.dk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The PyPI license classifiers include "OSI Approved: BSD", which does not
specify which of the many variations of BSD license it actually means.
The generic "BSD" license in the oe-core set is actually BSD-3-Clause.
>From a random sample of ten PyPI modules that use "BSD", they are all
BSD-3-Clause. As we expect the recipe maintainer to verify the license
anyway, and this matches the previous license text, explicitly set the
license to BSD-3-Clause.
(From OE-Core rev: a879fff9af31e45b1acc3f19a3c2a7eaf6319ad4)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, buildhistory does not produce a single file combining relevant
information of installed packages. Produce an output file
"installed-package-info.txt" listing a package's runtime name, buildtime name,
its recipe, version, and size to avoid having to look up each package externally.
Leave the existing package list files as-is for backwards compatibility.
In order to support this efficiently, extend oe-pkgdata-util to accept multiple keys
for its read-value argument.
(From OE-Core rev: 1e18b514bf1f960d324a21db608c8e8e5af007ef)
Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is particularly useful for llvm-accelerated GL rendering from qemu guest to
an offscreen buffer (accessible over vnc or spice) using llvmpipe on the
host, rather than using unaccelerated swrast renderer in the guest.
This is the best that can be done in the absence of a host GPU with render node
support (such as old Matrox cards common in servers, or fully virtualized cloud
environments with no GPU at all).
Note: even though NVidia blob drivers do support render nodes, they do not
support gbm (yet?), and so rendering will fall back to llvmpipe as well even when
the system has a 3000 euro NVidia GPU. Cue Linus picture.
(From OE-Core rev: 9b1daa173481f7f560e00e0dc22b4010ff1dc0ec)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When embedding a rootfs image (e.g. 'rootfs-dir') as a partition we
might want to keep the stock fstab for that image. In such a case, use
this option to not update the fstab and use the stock one instead.
This option allows you to specify which partitions get the fstab
updated and which get the stock fstab.
The option matches the argument you can pass to wic itself where the
same action is performed but for all the partitions.
Example:
part /export --source rootfs --rootfs-dir=hockeycam-image
--fstype=ext4 --label export --align 1024 --no-fstab-update
part / --source rootfs --fstype=ext4 --label rootfs --align 1024
(From OE-Core rev: ab4c95af8ecd15dc136194ab761afae756db5803)
Signed-off-by: Daniel Gomez <daniel@qtec.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a long standing bug where FILES_INFO isn't written into pkgdata
with a package suffix. This means if the files are read into the datastore
as intended, the last one "wins".
Fix this to work as intended. Most of the call sites using the data need
to be updated to handle this and the overrides change correctly.
Also fix some other problematic references noticed along the way.
(From OE-Core rev: a1190903e0a61a12c9854c96af918ae8d12c6327)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Company name is properly spelled as "Red Hat".
(From OE-Core rev: cd60f20fe40bba5b80ab604ee34fbf4ec2b0b4a2)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts part of oe-core eecbe62555, which was a previous attempt
to solve the Y2038 problem. This is now solved centrally in e2fsprogs,
so doesn't need to be dealt with in wic.
We don't revert the commit entirely, to retain the warning if a
filesystem has small inodes.
(From OE-Core rev: 7e8017208bed98b6c90735cb641fc9d7aedf9140)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently lines like below are converted.
e.g.
# IMAGE_INSTALL_append = " A"
But lines without a leading whitespace are not converted.
e.g.
#IMAGE_INSTALL_append = " A"
We should be converting both.
(From OE-Core rev: 1994e3844c1aa6b595c0c18040e4f8240fa04438)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
e.g. VIRTUAL-RUNTIME_com.webos.service.flowmanager_armv4 weren't replaced
with VIRTUAL-RUNTIME_com.webos.service.flowmanager:armv4 or when package
name contains a dot like in:
RDEPENDS:gstreamer1.0-meta-base:remove
(From OE-Core rev: 41bff44bd26c09573eb3139bb6735e5ecfda18b7)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For consistency, use override syntax for these variables as well since
it is more consistent with the rest of the image code. We may be able to use
these as proper overrides in due course.
(From OE-Core rev: 52674c4b1fdf79829095031b2e342d44fb0dc181)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The IMAGE_TYPEDEP variable would make more sense to match the form of the
other image override variables, convert it to use the overrides format.
(From OE-Core rev: 8573f6b2a7af9867da0b21936ffd2cd2a417de1d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add task-configure and few more supported values of TARGET_OS override.
(From OE-Core rev: 1172f9593902f28ddd8da47de6bd51cda9a0f86a)
Signed-off-by: Denys Dmytriyenko <denis@denix.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will allow the use of a bios (e.g. U-Boot) in combination with
a WIC file (from which U-Boot loads the kernel).
(From OE-Core rev: 5fc65c9d18679b62b13fead7b91126c665525884)
Signed-off-by: Matthias Klein <matthias@extraklein.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is handy to be able to have the script convert a single patch file
and it turns out to be straightforward to make it do this.
(From OE-Core rev: 21df7acc969f47d615d1701ee71f19571de94949)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The automated conversion of OE-Core to use the new override sytax isn't
perfect. This patches some mis-converted lines and some lines which were missed
by the automation.
(From OE-Core rev: 4e9a06b64b43131b731fb59a0305f78a98e27fbd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is the result of automated script conversion:
scripts/contrib/convert-overrides.py <oe-core directory>
converting the metadata to use ":" as the override character instead of "_".
(From OE-Core rev: 42344347be29f0997cc2f7636d9603b1fe1875ae)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* on bigger layer this script takes couple minutes without showing any progress
* add a version to reference it in the conversion commit (to easily figure out
if it's worth re-converting the layer).
(From OE-Core rev: 39edf868deafe79ffd2ea57620af70840ec4c208)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds a script I've developed to migrate metadata to use the new override
syntax. It is a bit rough but since its for a single use with validation, it
doesn't need to be perfect. It is run simply as:
scripts/contrib/convert-overrides.py <directory>
It is setup and has been tested to work with OE-Core, Bitbake, yocto-docs,
meta-yocto, meta-gplv2 and meta-mingw. For OE-Core, it converts around 10,100
lines with about 34 manual fixes needed.
For other layers it would need updating for override names and exclusions for
functions/variable names with "append", "prepend" or "remove" in them.
(From OE-Core rev: e8fc67f6f6baf54ccdf548b947c35dea926842c3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to be compliant with the YP compatible status, a layer also
needs to ensure that all its dependencies are compatible
too. Currently yocto-check-layer only checks the requested layer,
without testing any dependencies.
With this change, all dependencies are also checked by default, so the
summary printed at the end will give a clear picture whether all
dependencies pass the script or not.
Using --no-auto-dependency can be used to skip that.
(From OE-Core rev: 45d59b774b95c91193a8376b83c05291d555e5c8)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
What this function does is really to find a layer, not a 'depends'. We
are using this function to find a dependent layer, but the name is
confusing.
(From OE-Core rev: e9b7690ab30d0e7c07471034f6cb89ccc3168a11)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Split add_layer_dependencies() into 2 parts. First search for layer
dependencies, and then add them to the config. That allows us to
call get_layer_dependencies() independently.
(From OE-Core rev: 08edf928aac3f2daaa0c256d4c21e56e2db72bff)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The first 2 calls to add_layer_dependencies() are here to add all
dependencies for the 'layer under test' and the additional layers
provided by the users.
In both cases, we use misssing_dependencies boolean to indicate if any
dependency is missing. But we then never really use
missing_dependencies. Instead the script is calling
add_layer_dependencies() again (for both the layer under test, and the
additional layers) to detect if there are any missing dependency. As a
result, we are trying to add again all dependencies, and we can see
that from the traces:
INFO: Detected layers:
INFO: meta-aws: LayerType.SOFTWARE, /work/oe/sources/meta-aws
INFO: checklayer: Doesn't have conf/layer.conf file, so ignoring
INFO:
INFO: Setting up for meta-aws(LayerType.SOFTWARE), /work/oe/sources/meta-aws
INFO: Adding layer meta-python
INFO: Adding layer meta-oe
INFO: Adding layer meta-networking
-->
INFO: Adding layer meta-python
INFO: meta-python is already in /work/oe/poky/master/build-checklayer/conf/bblayers.conf
INFO: Adding layer meta-oe
INFO: meta-oe is already in /work/oe/poky/master/build-checklayer/conf/bblayers.conf
INFO: Adding layer meta-networking
INFO: meta-networking is already in /work/oe/poky/master/build-checklayer/conf/bblayers.conf
<--
INFO: Getting initial bitbake variables ...
The code appears more complex than it should, and we can simply
replace the complex if statement by using missing_dependencies, and
avoid duplicating the call to add_layer_dependencies().
(From OE-Core rev: fceb84f7bc472731b8f96ee1ebf0f4485943226c)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This updates the link to the YP docs
and proposes to access the OE website through https
(From OE-Core rev: 87686233aeffc639c3f412fd5c4898b32b15013b)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Nicolas Dechesne spotted there was duplicate code I had introduced with a
previous fix. Remove the second statement since the earlier one is correct
all that is needed.
(From OE-Core rev: f3e8d9f0e53e73de5498fccce81d049a88f6473b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When S is set to a sub-directory of upstream source, the license
checks and the bbappend writing (specifically, setting EXTERNALSRC)
need to operate on that sub-directory.
'devtool modify' already has similar logic, and it was copied from there
and adjusted.
(From OE-Core rev: fd77e356d4507405fde352c8bba7d3842518bbdd)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Options:
-c | --count <amount> dd (transfer) <amount> KiB of data within specified timeout to detect latency.
Must enable -t option.
-t | --timeout <time> timeout in seconds for the <count> amount of data to be transferred.
-l | --log-only run the commands without performing the data transfer.
-h | --help show help
(From OE-Core rev: 302bc6c99226a4d050e4e454afc461a25e127632)
Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The qemu tap device handling is potentially race ridden. We pass the
fd to the main qemu subprocess which is good as it means the lock is held
as long as the qemu process exists. This means we shouldn't unlock it
ourselves though, only close the file. We also can't delete the file
as we have no idea if qemu is still using it. We could try and obtain
an exclusive new lock, then the file would be safe to unlink but it
doesn't seem worth it.
Also fix the same issue in the port lock code.
(From OE-Core rev: 2a87bddabf816d09ec801e33972879e6983627eb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit a2db4fa127 added ${WORKDIR}/deploy-* to
PSEUDO_IGNORE_PATHS. This breaks the --strip mode since ${D} is copied to
deploy-target-stripped. Use the directory devtool-deploy-target-stripped
instead.
[YOCTO #14451]
(From OE-Core rev: 02661f20faf11d0fa2f1874bd423f5d9fa7a31c9)
Signed-off-by: Florian Amstutz <florian.amstutz@scs.ch>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously the bootimg-pcbios wic plugin was not respecting
the --label option provided from the wks file. The plugin
was setting the label to 'boot'. With this fix, the --label
option is use. If no option are specified, then the default
is 'boot'.
(From OE-Core rev: 0fd7a73c1bd2486b7a022f0f69bbcb2e0d9cb141)
Signed-off-by: jbouchard <jeanbouch418@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's a number of recipes (such as nettle or glib) which were upgraded
incorrectly due to rebasing only the default set of patches in main SRC_URI.
Native-only or musl-only patches were not handled, causing do_patch fails
and overall misery.
Unfortunately, this also necessitates aborting any incomplete rebases as
devtool needs to be able to return to the original branch to complete
the upgrade command.
Also, do not add devtool-patched-%version tags, as they are not used anywhere,
and it's unclear how to extend them to cover multiple override branches.
(From OE-Core rev: 00131cbc969f6a73caffe068fcb983287cbe577f)
(From OE-Core rev: 588d04748dd89b5371ca9144666e97ec82dd6204)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Measure and display the time it takes to copy the rootfs to its new
location (usually in tmpfs) to try to understand whether it sometimes takes
a lot of time on the autobuilders.
(From OE-Core rev: be213114c1349e02ca48fd617a6c3badae8aa101)
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since Linux 5.4 a new read-only filesystem is available, erofs.
Compared to squashfs it offers much better read performance with and
without compression enabled.
It suppports two optional compressors, lz4 and lz4hc.
>From the mkfs.erofs man page:
EROFS is a new enhanced lightweight linux read-only filesystem with
modern designs (eg. no buffer head, reduced metadata, inline
xattrs/data, etc.) for scenarios which need high-performance read-only
requirements, e.g. Android OS for smartphones and LIVECDs.
It also provides fixed-sized output compression support, which improves
storage density, keeps relatively higher compression ratios, which is
more useful to achieve high performance for embedded devices with
limited memory since it has unnoticable memory overhead and page cache
thrashing.
This commit adds support for three new filesystem targets:
erofs: erofs without compression
erofs-lz4: erofs with lz4 compresssion enabled
erofs-lz4hc: erofs with lz4hc compression enabled
(From OE-Core rev: 41dead1ff8ccc49e6cd6e6f5d41a59d164693e0d)
Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Each of the cores is mapped to a thread on the host, this
should speed up things inside qemu which can take advantage of that.
(From OE-Core rev: 56f98d12a79585a4d4c459160f39403e05b620a5)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Dynamic layers may have a conf directory, but don't need (or
want) a conf/layer.conf
This isn't an error, so we can just log it and indicate the
layer is being skipped.
A full layer without a conf file isn't all that useable, so we
aren't letting anything subtle slip through by just logging
it.
(From OE-Core rev: 9841a7c70bc21c5cd3bbea98d5f8654434577f61)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Incremental build in Docker fails with:
OSError: [Errno 18] Invalid cross-device link
when source and destination are on different overlay filesystems.
Rather than adding fallback code to every call site, use a new wrapper
in bitbake which detects this case and falls back to shutil.move
which is slower but will handtle the overlay docker filesystems correctly.
[YOCTO #14301]
(From OE-Core rev: 656a65b2b84e7d529b89cf5de7eb838f902d84a2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are many Python scripts in oe-core that want to use Tinfoil, and
right now they have to know where they are to work out where BitBake is
likely to be.
This is suboptimal as BitBake could be somewhere else, so this
approach doesn't scale to other layers at all.
Solve this by adding BITBAKEDIR/lib to PYTHONPATH in oe-buildenv-internal,
so that Python has BitBake on its search path once the build system is
configured.
(From OE-Core rev: a48178f6d00e7f97a09f42d5a164204e9dcffa9f)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When disk stats don't run frequenctly enough, we see divide by zero
errors. The code already has a fallback path so ensure we use it
for this case too.
[YOCTO #14360]
(From OE-Core rev: b71d30aef5dc2c360432c0dd4147859dd303ea48)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Collect the last 30 lines from the cooker.log
whenever the timeout is exceeded.
(From OE-Core rev: 58f7cd4d6186525f08f3027975530d647cbfa26b)
Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If iterating a layer with multiple components and auto-adding dependencies
the tests can break since layers are never removed and order isn't guaranteed
to account for that.
Fix this by resetting the layer list back to the original list each time
before auto-adding the dependencies in each case.
This fixes scanning of meta-openembedded in particular where the sublayers
may not be added in order of minimal dependency.
(From OE-Core rev: bf1b467dacf345379cd5d84a1c9b3b0d844d5c91)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to cleanup snapshot files if we make a copy of them to ensure
the tmpfs doesn't run out of space. There is already NFS code needing
this so make it a generic code path.
(From OE-Core rev: a3e0eec5a4785a0c4859455eb10b43aa832e606d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Particularly:
- nesting subprocess.run() inside subprocess.check_output() does not work at all.
How was this tested?
- -R and -U options can be combined; no need to separate the invocations based
on packaging format
- both exception handlers are unnecessary; we can simply print the hint if
invocation did not succeed
- to run debuginfod from its own sysroot, '-c addto_recipe_sysroot' for elfutils-native
must be executed
(From OE-Core rev: 9e57bf636ec63e74d56f1ac48b5a27c5b80f1877)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the previous timeout of 5 seconds, there would be
builds such as:
https://autobuilder.yocto.io/pub/non-release/20210417-13/
which produced 17 files with top output with top running 454 times
and that's a bit too much data to analyze for each run. By
increasing the timeout, we'll find the worse problems
first, fix them and then we can decrease the timeout if needed.
(From OE-Core rev: 92b29a09b4c442597d212337b785afb76129ac7c)
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
image-manifest: script to generate product/image specific BOM
The image-manifest script generates image specific reports based on
an image manifest file. Currently there is data generated by buildhistory,
pkgdata, and license manifest but this data is poorly formated and spread
across multiple text files. This script can generate a single JSON output
file that is machine readable by other tools.
The manifest-info collects package information and stores the information
in a tarball. manifest-info can be configured using a json configuration
file. The default configuration including all possible options can be
dumped using the dump-config subcommand.
image-manifest takes an image manifest file as input to get the runtime
dependencies. As an option image-manifest can also use the build dependency
file, pn-buildlist, to get the build dependencies excluding native
packages.
This script extends the oe-image-manifest script [0] done by Paul Eggleton
[0]
https://github.com/intel/clear-linux-dissector-web/blob/master/layerindex/static/files/oe-image-manifest
------------------------------------------------------
usage: image-manifest [-h] [-d] [-q] <subcommand> ...
Image manifest utility
options:
-h, --help show this help message and exit
-d, --debug Enable debug output
-q, --quiet Print only errors
subcommands:
recipe-info Get recipe info
list-depends List dependencies
list-recipes List recipes producing packages within an image
list-packages List packages within an image
list-layers List included layers
dump-config Dump default config
manifest-info Export recipe info for a manifest
Use image-manifest <subcommand> --help to get help on a specific command
Co-developed-by: Paul Eggleton <bluelightning@bluelightning.org>
(From OE-Core rev: ad8fec9ce1704866df925bda18a240d6889b1ed5)
Signed-off-by: Anders Wallin <anders.wallin@windriver.com>
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Improvements:
- increase width to 512
- pass -c option to show full command-line
(From OE-Core rev: aeae9467af5609c3c7bf8d0379d5546d9797ead5)
Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com>
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have a working theory that IO queues on the autobuilder are impacting
runtime testing under qemu, particularly async writes which inice does not
influence. We already pass the snapshot option to qemu which copies the
image and runs out of the copy. Add in the ability to copy the image to
a specificed location which can be a tmpfs. This means that writes to the
image would no longer be blocked by other writes to disk in the system.
Preliminary tests show that this does improve the qemu errors at the expense
of sometimes showing qemu startup timeouts as on a loaded system with a large
test image, it can take longer than 120s to copy the image to tmpfs. Having
a most consistent failure mode for loaded tests is probably desireable though.
(From OE-Core rev: fd1c26ab426c3699ffd8082b83d65a84c8eb8bff)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some options such as egl-headless are fully compatible with it, so
there is no need to quit.
(From OE-Core rev: 66d11106f9e76d19e397ba3d14c3a22726033567)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
oe-time-dd-test records how much time it takes to
write <count> number of kilobytes to the filesystem.
It also records the number of processes that are in
running (R), uninterruptible sleep (D) and interruptible
sleep (S) state from the output of "top" command.
The purporse of this script is to find which part of
the build system puts stress on the filesystem io and
log all the processes.
(From OE-Core rev: 26d82f92e5ed2ebdde158d66b86eaf248f56892a)
Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
anonscm.debian.org is not used any more, update the URL.
(From OE-Core rev: 3c89071a2c19a522c1df5919f500b88339ec6294)
Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update after tinfoil API changes in bitake for REQUIRED_VERSION.
(From OE-Core rev: 1645c9b093bebf7ced67cbee0009d81d1a215966)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Pretty sure that trailing quote should be a period; it
appears to work properly.
(From OE-Core rev: 579f9ada19bd174bff0002cd6a731d12a1868252)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The do_kernel_configme task is no longer part of SRCTREECOVEREDTASKS,
its been removed from the kernel-yocto.bbclass since b72dbb2e4, but
there wasnt a matching patch for devtool for those changes.
This patch enables us to invoke the do_kernel_configme task when
using a devtool workspace, it also prepends a check for an existing
.config file in the source directory and moves it if thats the case,
since when using devtool modify a .config is created and
do_kernel_configme complains about it, this is not the case when
using bitbake since the .config file would be on B instead.
Alowing do_kernel_configme to run also fixes the flow where testing a
new config fragment from devtool workspace isnt added properly
(config queue shows it as ///frg.cfg) and as a side effect it never
gets merged into the final config.
(From OE-Core rev: 08dcc0e68095dcf2a159546a48b29d40c9aabc0b)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alhe@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Passing 'novga' only adds '-vga none' to the qemu commandline and does
not prevent other vga devices from being added, contrary to the
documentation/help. Clean-up the vga logic and add the ability to
prevent other vga devices from being added.
(From OE-Core rev: 9f8d049dbbe0b1760979d9f3b745124abfc54c90)
Signed-off-by: Jon Mason <jon.mason@arm.com>
Change-Id: I7ff8f9f3e419ce8bae3f3847d75f9275ca30406a
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some platforms do not use ttyS* for their serial consoles (e.g., qemuarm
and qemuarm64). The hardcoding of this can cause issues. Modify
runqemu to use the serial consoles defined in SERIAL_CONSOLES instead of
hardcoding.
(From OE-Core rev: 9dea4cd2f9f46ab3a75562639a22d8f56b4d26af)
Signed-off-by: Jon Mason <jon.mason@arm.com>
Change-Id: I746d56de5669c955c5e29d3ded70c0a4d3171f17
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a new runqemu field for VGA devices. Currently, these are being set
in QB_OPT_APPEND, which can make them difficult to override if importing
the config file into another one.
(From OE-Core rev: 695c98b6522be4373806c154a2999eaeef205556)
Signed-off-by: Jon Mason <jon.mason@arm.com>
Change-Id: I8cb9527954c5b06c083c42fe2466cb3338584b7d
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The sstate-diff-machines.sh script's --analyze option was only supporting
ipk packages, this adds rpm, deb and tar packages, checking all the used
package types simultaneously.
Without this changing sstate between machines would not report an error
if the difference occured in packages and ipk packaging was not enabled.
(From OE-Core rev: b27bff26a9640dfdfba4fc711a9b28acb4501d2b)
Signed-off-by: Diego Santa Cruz <Diego.SantaCruz@spinetix.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bin is not an acceptable format for qemu and will throw an error is set.
The correct format in this case is raw.
(From OE-Core rev: cd279aeb986b4676ea8ecb1e7b12c7e29e83460a)
Signed-off-by: Jon Mason <jon.mason@arm.com>
Change-Id: I9d78e356d39ecb15bef886c19870b029d9badd65
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added a script that fetches the package manager directory and runs the elfutils-native debuginfod on DEPLOY_DIR
Added a check to ensure that PACKAGECONFIG options is set in local.conf
(From OE-Core rev: 410083f93e2c986247cf3a2ff5050847e10cf359)
Signed-off-by: Dorinda Bassey <dorindabassey@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Modern dosfstools automatically determines the appropriate size
and will error out if something that doesn't make sense is
supplied on the command line.
(From OE-Core rev: b85a09ea450a5e8f49418f4a930805fbb88dc83b)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We are getting closer and closer to the year 2038 where the 32 bit
time_t overflow will happen. While products (= embedded systems) with an
expected life time of 15 years are still save the situation may change
if your system has to survive the next 20 years.
ext2 and ext3 filesystems are always affected by the time overflow, so
let's warn the user if these filesystems are still being used.
If ext4 is affected depends on the inode size chosen during filesystem
creation. At least 256 bytes are necessary to be safe. As ext4 is
used very often (and partitions may be created small first and extended
later) this might be an issue for many users.
Some filesystems created during CI runs were already affected by the Y2038
problem. By using `--mkfs-extraopts "-T default"` we tell mke2fs not to
auto-detect the usage type based on the filesystem size. mke2fs will use
the default values for tuning parameters instead. The inode size is one
of these parameters.
(From OE-Core rev: eecbe625558406680121d2a7e84917fea45ea9dc)
Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using nographic, explicitly disabling vga is needed some cases
since some qemu firmware (OpenFirmware ) defaults to std vga, and
when vga is enabled then it disables output to serial and redirects
that to vga which is by design, hwoever we expect the console output
to go to serial when using nographic, therefore its important to
disable vga with nographic especially on qemu based ppc platforms
(From OE-Core rev: eabbdb3c4f78ab23348a6b988f59d9d98b5e2c7c)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Cc: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tinfoil doesn't behave well if environment is not initialized, this check ensures a proper error log if environment is not initialized.
[YOCTO #12096]
(From OE-Core rev: e88073e16f1b4cfd0f97c81a988640a84adad674)
Signed-off-by: Dorinda Bassey <dorindabassey@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
files in wic tmp directory can be usefull for debugging, so do not remove
tmp directory when wic create run with debugging mode (-D or --debug).
also update wic.Wic.test_debug_short and wic.Wic.test_debug_long to
check for tmp directory.
[YOCTO#14216]
(From OE-Core rev: a122e2418b67d38f691edcf8dd846c167d6b4fa9)
Signed-off-by: Lee Chee Yang <Chee.Yang.Lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add relative path if file is under directory.
[YOCTO #13738]
(From OE-Core rev: 19ddacc1b38f9ebb86a9359963ccc3c707f7125e)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If do_patch task is disabled then prepare do_configure dependencies to
fetch external sources and create symlink to ${S} in devtool workspace.
[YOCTO #13036]
(From OE-Core rev: 9e47319bfe62d289b90f7545a64dbdc1cbde7f1d)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The fix for [Yocto #13994] required the rootfs directory to be copied
(using hardlinks if possible) when modifying the fstab file under wic.
We can optimise this copy away for filesystems where we have the tools
to modify the contents of the partition image after it is created. For
ext2/3/4 filesystems we have the debugfs tool and for msdos/vfat
filesystems we have the mcopy tool. So for any of these filesystems we
skip the modification of the fstab file in the rootfs directory (and
skip the associated copy unless it is otherwise necessary) and update
the contents of fstab directly in the partition image.
(From OE-Core rev: 5fb8ae0e9159597d7eaa9307a3a8543800bf9405)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default, wic updates the /etc/fstab in the rootfs to include details
of additional partitions described in the selected wks file. If this
modification is performed in place, other tasks which create an image
file from the rootfs directory (e.g. do_image_tar and do_image_ext4)
will pick up the modified fstab file which would not be appropriate for
those images as they do not include the additional partitions described
in the wks file. wic does undo modifications to the fstab file once it
has finished creating the filesystem image, however this leaves open a
race condition if one of the other tasks reads the contents of the fstab
file from the rootfs directory between the point where wic modifies the
fstab file and the point where wic restores the files original content.
This could be solved by adding a lockfile for tasks which use the rootfs
directory to ensure that no other such task is reading the rootfs
directory while do_image_wic is running. This would serialize several
do_image_* tasks and result in slower builds, especially for large
images. Another drawback of this solution is that it is hard to
selectively optimise - adding lockfiles to do_image_* tasks would result
in these tasks always being serialized even if no fstab modification
will take place.
An alternative solution is to copy the rootfs directory when fstab needs
to be modified. The code to do this in wic already exists as it is
needed when including or excluding content in the rootfs. This still
results in an impact on build times but the copy uses hardlinks if
possible (so little data is actually copied) and we can make selective
optimisations to improve things. The rootfs copy will only take place if
fstab modification is required (or if it was already needed to include
or exclude rootfs content). We can also follow up with further
optimisations after this commit. So this second solution is chosen.
Fixes [Yocto #13994]
(From OE-Core rev: ce682a73b7447652f898ce1d1d0416a456df5416)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To exclude content from the rootfs, wic makes a copy (using hardlinks if
possible) of the rootfs directory and associated pseudo db, then removes
files & directories as needed. However if these files and directories
are removed using the python functions os.remove and shutil.rmtree, the
copied pseudo db will not be updated correctly. For files copied from
the original rootfs, if hardlinks were used successfully when copying
the rootfs this should mean that the relevant inodes can't be reused and
so the risk of pseudo aborts should be avoided. However, this logic
doesn't apply for directories (as they can't be hardlinked) or for files
added via the '--include-path' argument (as they weren't present in the
original rootfs) and so there remains some risk of inodes being reused
and the pseudo db becoming corrupted.
To fix this, use the 'rm' command under pseudo when removing files &
directories from the copied rootfs to ensure that the copied pseudo db
is updated.
(From OE-Core rev: d5db7e268947f0392c2126137571a44acd29ccd6)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a path is specified for the internal wic working directory using
the -w/--workdir argument then it must not already exist. Re-using a
previous workdir could easily result in rootfs and intermediate files
from a previous build being added to the current image.
(From OE-Core rev: 2e40c8d4109024ff704c5ce40d98050ca7f34dd5)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before adding a npmsw fetcher to a recipe we
should first check if the generated shrinkwrap file
contains dependencies.
(From OE-Core rev: ef153ad36d0299e83a03af8f207686d0d8a238b3)
Signed-off-by: Kamel Bouhara <kamel.bouhara@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes a crash with "find" when running a native tool and *-native
directories do not exist under the binary directory in the sysroot.
This happened because the directory wildcard was passed as part of the
root directory.
The directory wildcard is now passed by "-name", which returns an empty
result if no matching directory.
(From OE-Core rev: f6c90ed0ad24b7d4f892e22e088b1578824eb1d3)
Signed-off-by: Adrian Herrera <adrian.herrera@arm.com>
Change-Id: Iba7acd8bbd7e0beb4d25c984f6af7a4fd21486e6
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows programs from HOSTTOOLS (e.g. 'install', 'rm', 'mv', etc) to
be more easily executed by wic. Without this change only programs from
an actual *-native recipe built by bitbake can be executed by wic.
(From OE-Core rev: 8eb186acdecfbb3151c9a0ab148358e3fe5cce39)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows the path for the temporary workdir used by wic to be set
when running wic from bitbake or directly from the command line.
(From OE-Core rev: 2e7314ac3a3cab89de93c932e8efc75de0feb0a7)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
