I've puzzled over why I see local eSDK test failures. It is due to having a
messy git tree locally, git then runs git gc in the background which races
against the deletion code for the publish test. Disable git gc to avoid this.
(From OE-Core rev: d938a74097202396beda2e3bb4f582342d6dfa1b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Avoid a rare divide by zero error if there isn't data point spread.
[YOCTO #14547]
(From OE-Core rev: d7e36d01e87ddf89f76f164a0b7d98f597a53fa5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The corrected line accidentally converted it to float,
which causes problems later on with python 3.10:
| File "/home/alex/development/poky/scripts/lib/wic/partition.py", line 278, in prepare_rootfs_ext
| os.ftruncate(sparse.fileno(), rootfs_size * 1024)
| TypeError: 'float' object cannot be interpreted as an integer
(From OE-Core rev: d1d260dd2d196d10379ed9e238bcb34f39f3a3b7)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
collections.abc is available since 3.3 and doesn't need special handling.
(From OE-Core rev: 01152c9410ba00274c8415a5d914dc33dfd0cf46)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows extra space to be added after the last partition and is
especially useful when free space is needed for ex: adding partitions on
first boot with ex: systemd-repart[1] and the image is tested in QEMU.
[1] https://www.freedesktop.org/software/systemd/man/systemd-repart.html
(From OE-Core rev: f81b188bcf5aa18746fd622eb7b5c0dcb0b5c93d)
Signed-off-by: Kristian Klausen <kristian@klausen.dk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The PyPI license classifiers include "OSI Approved: BSD", which does not
specify which of the many variations of BSD license it actually means.
The generic "BSD" license in the oe-core set is actually BSD-3-Clause.
>From a random sample of ten PyPI modules that use "BSD", they are all
BSD-3-Clause. As we expect the recipe maintainer to verify the license
anyway, and this matches the previous license text, explicitly set the
license to BSD-3-Clause.
(From OE-Core rev: a879fff9af31e45b1acc3f19a3c2a7eaf6319ad4)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, buildhistory does not produce a single file combining relevant
information of installed packages. Produce an output file
"installed-package-info.txt" listing a package's runtime name, buildtime name,
its recipe, version, and size to avoid having to look up each package externally.
Leave the existing package list files as-is for backwards compatibility.
In order to support this efficiently, extend oe-pkgdata-util to accept multiple keys
for its read-value argument.
(From OE-Core rev: 1e18b514bf1f960d324a21db608c8e8e5af007ef)
Signed-off-by: Andres Beltran <abeltran@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is particularly useful for llvm-accelerated GL rendering from qemu guest to
an offscreen buffer (accessible over vnc or spice) using llvmpipe on the
host, rather than using unaccelerated swrast renderer in the guest.
This is the best that can be done in the absence of a host GPU with render node
support (such as old Matrox cards common in servers, or fully virtualized cloud
environments with no GPU at all).
Note: even though NVidia blob drivers do support render nodes, they do not
support gbm (yet?), and so rendering will fall back to llvmpipe as well even when
the system has a 3000 euro NVidia GPU. Cue Linus picture.
(From OE-Core rev: 9b1daa173481f7f560e00e0dc22b4010ff1dc0ec)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When embedding a rootfs image (e.g. 'rootfs-dir') as a partition we
might want to keep the stock fstab for that image. In such a case, use
this option to not update the fstab and use the stock one instead.
This option allows you to specify which partitions get the fstab
updated and which get the stock fstab.
The option matches the argument you can pass to wic itself where the
same action is performed but for all the partitions.
Example:
part /export --source rootfs --rootfs-dir=hockeycam-image
--fstype=ext4 --label export --align 1024 --no-fstab-update
part / --source rootfs --fstype=ext4 --label rootfs --align 1024
(From OE-Core rev: ab4c95af8ecd15dc136194ab761afae756db5803)
Signed-off-by: Daniel Gomez <daniel@qtec.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a long standing bug where FILES_INFO isn't written into pkgdata
with a package suffix. This means if the files are read into the datastore
as intended, the last one "wins".
Fix this to work as intended. Most of the call sites using the data need
to be updated to handle this and the overrides change correctly.
Also fix some other problematic references noticed along the way.
(From OE-Core rev: a1190903e0a61a12c9854c96af918ae8d12c6327)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Company name is properly spelled as "Red Hat".
(From OE-Core rev: cd60f20fe40bba5b80ab604ee34fbf4ec2b0b4a2)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts part of oe-core eecbe62555, which was a previous attempt
to solve the Y2038 problem. This is now solved centrally in e2fsprogs,
so doesn't need to be dealt with in wic.
We don't revert the commit entirely, to retain the warning if a
filesystem has small inodes.
(From OE-Core rev: 7e8017208bed98b6c90735cb641fc9d7aedf9140)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently lines like below are converted.
e.g.
# IMAGE_INSTALL_append = " A"
But lines without a leading whitespace are not converted.
e.g.
#IMAGE_INSTALL_append = " A"
We should be converting both.
(From OE-Core rev: 1994e3844c1aa6b595c0c18040e4f8240fa04438)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
e.g. VIRTUAL-RUNTIME_com.webos.service.flowmanager_armv4 weren't replaced
with VIRTUAL-RUNTIME_com.webos.service.flowmanager:armv4 or when package
name contains a dot like in:
RDEPENDS:gstreamer1.0-meta-base:remove
(From OE-Core rev: 41bff44bd26c09573eb3139bb6735e5ecfda18b7)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For consistency, use override syntax for these variables as well since
it is more consistent with the rest of the image code. We may be able to use
these as proper overrides in due course.
(From OE-Core rev: 52674c4b1fdf79829095031b2e342d44fb0dc181)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The IMAGE_TYPEDEP variable would make more sense to match the form of the
other image override variables, convert it to use the overrides format.
(From OE-Core rev: 8573f6b2a7af9867da0b21936ffd2cd2a417de1d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add task-configure and few more supported values of TARGET_OS override.
(From OE-Core rev: 1172f9593902f28ddd8da47de6bd51cda9a0f86a)
Signed-off-by: Denys Dmytriyenko <denis@denix.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will allow the use of a bios (e.g. U-Boot) in combination with
a WIC file (from which U-Boot loads the kernel).
(From OE-Core rev: 5fc65c9d18679b62b13fead7b91126c665525884)
Signed-off-by: Matthias Klein <matthias@extraklein.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is handy to be able to have the script convert a single patch file
and it turns out to be straightforward to make it do this.
(From OE-Core rev: 21df7acc969f47d615d1701ee71f19571de94949)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The automated conversion of OE-Core to use the new override sytax isn't
perfect. This patches some mis-converted lines and some lines which were missed
by the automation.
(From OE-Core rev: 4e9a06b64b43131b731fb59a0305f78a98e27fbd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is the result of automated script conversion:
scripts/contrib/convert-overrides.py <oe-core directory>
converting the metadata to use ":" as the override character instead of "_".
(From OE-Core rev: 42344347be29f0997cc2f7636d9603b1fe1875ae)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* on bigger layer this script takes couple minutes without showing any progress
* add a version to reference it in the conversion commit (to easily figure out
if it's worth re-converting the layer).
(From OE-Core rev: 39edf868deafe79ffd2ea57620af70840ec4c208)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds a script I've developed to migrate metadata to use the new override
syntax. It is a bit rough but since its for a single use with validation, it
doesn't need to be perfect. It is run simply as:
scripts/contrib/convert-overrides.py <directory>
It is setup and has been tested to work with OE-Core, Bitbake, yocto-docs,
meta-yocto, meta-gplv2 and meta-mingw. For OE-Core, it converts around 10,100
lines with about 34 manual fixes needed.
For other layers it would need updating for override names and exclusions for
functions/variable names with "append", "prepend" or "remove" in them.
(From OE-Core rev: e8fc67f6f6baf54ccdf548b947c35dea926842c3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to be compliant with the YP compatible status, a layer also
needs to ensure that all its dependencies are compatible
too. Currently yocto-check-layer only checks the requested layer,
without testing any dependencies.
With this change, all dependencies are also checked by default, so the
summary printed at the end will give a clear picture whether all
dependencies pass the script or not.
Using --no-auto-dependency can be used to skip that.
(From OE-Core rev: 45d59b774b95c91193a8376b83c05291d555e5c8)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
What this function does is really to find a layer, not a 'depends'. We
are using this function to find a dependent layer, but the name is
confusing.
(From OE-Core rev: e9b7690ab30d0e7c07471034f6cb89ccc3168a11)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Split add_layer_dependencies() into 2 parts. First search for layer
dependencies, and then add them to the config. That allows us to
call get_layer_dependencies() independently.
(From OE-Core rev: 08edf928aac3f2daaa0c256d4c21e56e2db72bff)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The first 2 calls to add_layer_dependencies() are here to add all
dependencies for the 'layer under test' and the additional layers
provided by the users.
In both cases, we use misssing_dependencies boolean to indicate if any
dependency is missing. But we then never really use
missing_dependencies. Instead the script is calling
add_layer_dependencies() again (for both the layer under test, and the
additional layers) to detect if there are any missing dependency. As a
result, we are trying to add again all dependencies, and we can see
that from the traces:
INFO: Detected layers:
INFO: meta-aws: LayerType.SOFTWARE, /work/oe/sources/meta-aws
INFO: checklayer: Doesn't have conf/layer.conf file, so ignoring
INFO:
INFO: Setting up for meta-aws(LayerType.SOFTWARE), /work/oe/sources/meta-aws
INFO: Adding layer meta-python
INFO: Adding layer meta-oe
INFO: Adding layer meta-networking
-->
INFO: Adding layer meta-python
INFO: meta-python is already in /work/oe/poky/master/build-checklayer/conf/bblayers.conf
INFO: Adding layer meta-oe
INFO: meta-oe is already in /work/oe/poky/master/build-checklayer/conf/bblayers.conf
INFO: Adding layer meta-networking
INFO: meta-networking is already in /work/oe/poky/master/build-checklayer/conf/bblayers.conf
<--
INFO: Getting initial bitbake variables ...
The code appears more complex than it should, and we can simply
replace the complex if statement by using missing_dependencies, and
avoid duplicating the call to add_layer_dependencies().
(From OE-Core rev: fceb84f7bc472731b8f96ee1ebf0f4485943226c)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This updates the link to the YP docs
and proposes to access the OE website through https
(From OE-Core rev: 87686233aeffc639c3f412fd5c4898b32b15013b)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Nicolas Dechesne spotted there was duplicate code I had introduced with a
previous fix. Remove the second statement since the earlier one is correct
all that is needed.
(From OE-Core rev: f3e8d9f0e53e73de5498fccce81d049a88f6473b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When S is set to a sub-directory of upstream source, the license
checks and the bbappend writing (specifically, setting EXTERNALSRC)
need to operate on that sub-directory.
'devtool modify' already has similar logic, and it was copied from there
and adjusted.
(From OE-Core rev: fd77e356d4507405fde352c8bba7d3842518bbdd)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Options:
-c | --count <amount> dd (transfer) <amount> KiB of data within specified timeout to detect latency.
Must enable -t option.
-t | --timeout <time> timeout in seconds for the <count> amount of data to be transferred.
-l | --log-only run the commands without performing the data transfer.
-h | --help show help
(From OE-Core rev: 302bc6c99226a4d050e4e454afc461a25e127632)
Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The qemu tap device handling is potentially race ridden. We pass the
fd to the main qemu subprocess which is good as it means the lock is held
as long as the qemu process exists. This means we shouldn't unlock it
ourselves though, only close the file. We also can't delete the file
as we have no idea if qemu is still using it. We could try and obtain
an exclusive new lock, then the file would be safe to unlink but it
doesn't seem worth it.
Also fix the same issue in the port lock code.
(From OE-Core rev: 2a87bddabf816d09ec801e33972879e6983627eb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit a2db4fa127 added ${WORKDIR}/deploy-* to
PSEUDO_IGNORE_PATHS. This breaks the --strip mode since ${D} is copied to
deploy-target-stripped. Use the directory devtool-deploy-target-stripped
instead.
[YOCTO #14451]
(From OE-Core rev: 02661f20faf11d0fa2f1874bd423f5d9fa7a31c9)
Signed-off-by: Florian Amstutz <florian.amstutz@scs.ch>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously the bootimg-pcbios wic plugin was not respecting
the --label option provided from the wks file. The plugin
was setting the label to 'boot'. With this fix, the --label
option is use. If no option are specified, then the default
is 'boot'.
(From OE-Core rev: 0fd7a73c1bd2486b7a022f0f69bbcb2e0d9cb141)
Signed-off-by: jbouchard <jeanbouch418@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's a number of recipes (such as nettle or glib) which were upgraded
incorrectly due to rebasing only the default set of patches in main SRC_URI.
Native-only or musl-only patches were not handled, causing do_patch fails
and overall misery.
Unfortunately, this also necessitates aborting any incomplete rebases as
devtool needs to be able to return to the original branch to complete
the upgrade command.
Also, do not add devtool-patched-%version tags, as they are not used anywhere,
and it's unclear how to extend them to cover multiple override branches.
(From OE-Core rev: 00131cbc969f6a73caffe068fcb983287cbe577f)
(From OE-Core rev: 588d04748dd89b5371ca9144666e97ec82dd6204)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Measure and display the time it takes to copy the rootfs to its new
location (usually in tmpfs) to try to understand whether it sometimes takes
a lot of time on the autobuilders.
(From OE-Core rev: be213114c1349e02ca48fd617a6c3badae8aa101)
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since Linux 5.4 a new read-only filesystem is available, erofs.
Compared to squashfs it offers much better read performance with and
without compression enabled.
It suppports two optional compressors, lz4 and lz4hc.
>From the mkfs.erofs man page:
EROFS is a new enhanced lightweight linux read-only filesystem with
modern designs (eg. no buffer head, reduced metadata, inline
xattrs/data, etc.) for scenarios which need high-performance read-only
requirements, e.g. Android OS for smartphones and LIVECDs.
It also provides fixed-sized output compression support, which improves
storage density, keeps relatively higher compression ratios, which is
more useful to achieve high performance for embedded devices with
limited memory since it has unnoticable memory overhead and page cache
thrashing.
This commit adds support for three new filesystem targets:
erofs: erofs without compression
erofs-lz4: erofs with lz4 compresssion enabled
erofs-lz4hc: erofs with lz4hc compression enabled
(From OE-Core rev: 41dead1ff8ccc49e6cd6e6f5d41a59d164693e0d)
Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Each of the cores is mapped to a thread on the host, this
should speed up things inside qemu which can take advantage of that.
(From OE-Core rev: 56f98d12a79585a4d4c459160f39403e05b620a5)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Dynamic layers may have a conf directory, but don't need (or
want) a conf/layer.conf
This isn't an error, so we can just log it and indicate the
layer is being skipped.
A full layer without a conf file isn't all that useable, so we
aren't letting anything subtle slip through by just logging
it.
(From OE-Core rev: 9841a7c70bc21c5cd3bbea98d5f8654434577f61)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Incremental build in Docker fails with:
OSError: [Errno 18] Invalid cross-device link
when source and destination are on different overlay filesystems.
Rather than adding fallback code to every call site, use a new wrapper
in bitbake which detects this case and falls back to shutil.move
which is slower but will handtle the overlay docker filesystems correctly.
[YOCTO #14301]
(From OE-Core rev: 656a65b2b84e7d529b89cf5de7eb838f902d84a2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are many Python scripts in oe-core that want to use Tinfoil, and
right now they have to know where they are to work out where BitBake is
likely to be.
This is suboptimal as BitBake could be somewhere else, so this
approach doesn't scale to other layers at all.
Solve this by adding BITBAKEDIR/lib to PYTHONPATH in oe-buildenv-internal,
so that Python has BitBake on its search path once the build system is
configured.
(From OE-Core rev: a48178f6d00e7f97a09f42d5a164204e9dcffa9f)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
