There are still a couple of cases where the license may be set as, e.g.,
"GPL" or "GPL-2.0" since there is not enough information to decide the
actual SPDX license. It is then assumed that the developer will have to
correct the information.
(From OE-Core rev: e7df51f8d2361e9fe2d67669d2e17f0a5d01004e)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add all OSI approved licenses from https://pypi.org/classifiers/. Also
add support for Other/Proprietary (Proprietary) and Public Domain (PD).
(From OE-Core rev: 99ef134d1019e5b98b845cf71f3eb39871218f9d)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After the change to bitbake, update the references in OE-Core to match the updates.
(From OE-Core rev: 193affb9f28b0116c3fd619834f145326fee08c5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is useful when needing to test layer's recipes, where this special
bbclass can define a global python function that gets called on each
recipe parsing during "bitbake -S none world" signature dump and be able
to fail layer's check accordingly.
First test being added is to detect recipes skipping "installed-vs-shipped"
QA check. As "installed-vs-shipped" is a packaging QA check, it happens very
late in the build process and failing it could mean some potential issues
with packaging, especially when recipe uses BBCLASSEXTEND="nativesdk" and
resulting package is used in an SDK.
In OE-Core failing this QA check leads to an error, but other layers can
suppress it or change it to a warning. Detecting weird packaging problems
with SDKs is quite difficult and time consuming. Also, waiting for the
actual "installed-vs-shipped" packaging QA check to fail means that all
recipes in the layer under test have to run through all standard tasks in
the build chain, equivalent to a multi-hour world-build.
Hence yocto-check-layer takes a shortcut and only detects a mere attempt
at skipping "installed-vs-shipped" QA check in the INSANE_SKIP list during
initial parsing when dumping the signature information for the layer.
(From OE-Core rev: e8baa75535fc888f1d768b23a0140475e832c910)
Signed-off-by: Denys Dmytriyenko <denis@denix.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In particular this resolves devtool's inability to pick a branch
when the same tagged commit is avaiable in main and in a release maintenance
branch.
Thanks to Peter Kjellerstedt for the suggestion.
(From OE-Core rev: bcb21ee2760a2c76039412a56c6cda43fbf96fd0)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support for packed images to wic rawcopy handler do minimize disk
usage in deploy directory and reuse of packed images between wic and
swupdate. Add `unpack` to sourceparams to unpack an bz2, gz and xz
archives.
Example:
part / --source rawcopy --sourceparams="file=core-image-minimal-qemu.ext4.gz,unpack"
(From OE-Core rev: 4c97d25791389ece041565981ba3207ce9949a1a)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Accept valueless keys in sourceparams without equals sign (=) to match
the comment and support Boolean entries.
(From OE-Core rev: 052a9b0f7d2ee5c284cf1b4f1f45caed0489221a)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake is now requiring there be a call to get_srcrev() when urls contain
floating revisions. This breaks some recipetool workflows since it is running
these queries to determine those revisions. Add a PV entry which triggers
that call to avoid the errors.
(From OE-Core rev: bdaf0c798da21f9c15d9cb30ab4258a3c75c8bbf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
deploy-target may fail the second time with "pseudo abort" because
devtool-deploy-target-stripped is deleted outside of pseudo's fakeroot
context.
(From OE-Core rev: 2338a33b690b0bbe279cde3f73764911b239cb50)
Signed-off-by: Florian Amstutz <florian.amstutz@scs.ch>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
detect_layers() is very greedy and if it recurses into poky or bitbake
it will find the test suite layers, such as
bitbake/lib/layerindexlib/tests/testdata/layer4. This is a dummy layer
which claims to be openembedded-layer, so if the real openembedded-layer
is a dependency then layer4 may be used instead, which will cause
errors: initially because it's only compatible with Sumo, but later
because it doesn't contain any recipes.
Add a check that the set of layers we've found doesn't contain any
duplicate collection names with different patterns, and abort if that is
the case as the test will be non-deterministic.
(From OE-Core rev: 0df4bae4ec67d38442620fa08c839528b425e2a8)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When creating a new recipe.bb file for a binary, don't use BP which
includes the version information, instead use BPN which is just the
name base Package Name.
Since PB is not specified, it takes the default:
PV = "1.0+git${SRCPV}"
But SRCPV is defined in terms of the SRC_URI, which leads to infinite
recursion (traceback below). Here are the pertinent variables which
cause the recursion:
SRC_URI = "git://github.com/lvc/abi-dumper;protocol=https;subdir=${BP}"
BP = "${BPN}-${PV}"
PV = "1.0+git${SRCPV}"
SRCPV = "${@bb.fetch2.get_srcrev(d)}"
def get_srcrev(d, method_name='sortable_revision'):
# ... trimmed
scms = []
fetcher = Fetch(d.getVar('SRC_URI').split(), d)
# ... trimmed
[YOCTO #14040]
(From OE-Core rev: 3b8d43fc53ee13d39abc3b2a1f706a97fcf752aa)
Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since commit 044fb04d in bitbake (fetch2: Allow whitespace only mirror
entries) there is no need to separate the entries in SSTATE_MIRRORS
with "\n".
(From OE-Core rev: 6e66ec3cd201d32d3d97df2222f41bb8c475fb69)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'distutils' is deprecated in Python 3.10 with removal in Python 3.12
(~October 2023). Replace 'distutils.command.build_py' with
'setuptools.command.build_py'.
To avoid an AttributeError, we call super().__init__() which provides
the missing 'distribution' attribute. However, for some reason, __init__()
in 'setuptools.command.build_py.build_py' class requires a 'dist' positional
argument which must be a 'Distribution' instance. It is not clear why
'distutils.command.build_py.build_py' class does not require this.
There is still a check which decides to inherit setuptools3 vs distutils3
that will need to be refactored when we add pyproject.toml and setup.cfg
support for more modern PEP 517 packaging.
Once distutils3.bbclass is dropped, any recipe inheriting distutils3
will throw a parsing error. The plan is to move distutils*.bbclasses to
meta-python. However if meta-python is not in bblayers, the parsing
error would still occur.
[YOCTO #14610]
(From OE-Core rev: 619a3eb1266459daf16e10386113e9201fbf9cf5)
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is deprecated in python 3.12 and Fedora 35 is throwing warnings so
move to the new functions.
(From OE-Core rev: 655cd3f614d736416eab0d708b7c49674bf5c977)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Separate licenses with & operator since it should be satisfied most use
cases and it is a reasonable assumption that all the licenses apply.
Furthermore flat, split and sort the licenses to minimize license string
changes.
Separate package licenses with & operator:
-LICENSE:${PN} = "MIT ISC"
+LICENSE:${PN} = "ISC & MIT"
Respect | and brackets in LICENSE:
-LICENSE = "BSD-3-Clause & (ISC & | & MIT)"
+LICENSE = "BSD-3-Clause & (ISC | MIT)"
Sort licenses:
-LICENSE = "MIT & BSD-3-Clause & ISC"
+LICENSE = "BSD-3-Clause & ISC & MIT"
Remove duplicates:
-LICENSE = "MIT & ISC & MIT"
+LICENSE = "ISC & MIT"
(From OE-Core rev: 60a84ecc53d20118c5e7f86dd3e3cafbfed1cf0a)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sort the output of guess_license function by license file to be
deterministic and to support self-testing.
(From OE-Core rev: ff2a7520813b3f775f96879d1905222e08b8c83f)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The commit 'meta/scripts: Manual git url branch additions (dc53fe75cc)'
sets the branch= parameter too early to master and thereby breaks the
-B/--srcbranch option.
ERROR: branch= parameter and -B/--srcbranch option cannot both be specified - use one or the other
(From OE-Core rev: 34ece8030e7a6a100b5e3e7b94e6c786c0e199a6)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Relaxes the case requirements for checks in the README file so that word
like "Maintainer" and "Patch" are allowed
(From OE-Core rev: fdedf94d406ad3da85cb45d43ef87d3fdc8c14d4)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is permissible to have a layer's README reference another README. If
this is the case, avoid other checks. Do this by checking for the word
README (case insensitive) in the README file. This might be too
permissive, but follows the pattern of looking for exact words (like
"patch" or "maintainer") in READMEs.
(From OE-Core rev: 443c73d638519d6a7ea44d1c0e80d76306687ddc)
Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds:
- Test case to review if the README file specifies the maintainer.
- Test case to review if the README file specifies the word patch.
- Test case to review if the README file contains an email address.
[YOCTO #11131]
(From OE-Core rev: 5d4937bce191b7e22b807c0595d4845c88ecc560)
Signed-off-by: Dhruva Gole <goledhruva@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a configure.ac uses LIBCURL_CHECK_CONFIG it wants curl.
(From OE-Core rev: 16e830ca1352cee61e4ae4b98b1203f849bf71f3)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
GitHub URLs are automatically transformed to git: fetches, so handle
GitLab URLs too.
(From OE-Core rev: 651fb951819840fe122458ddbd852ee6c7ec0455)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We already support specifying the rootfs by PARTUUID. This adds general
support for letting the kernel find the rootfs by PARTLABEL.
(From OE-Core rev: 2fb247c5ecf057bb96649a3c0234794b4991c050)
Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use shutil.which to find the executable instead to silence the below warning:
$ cat tmp/work/intel_x86_64-poky-linux/core-image-base/1.0-r5/temp/log.do_image_wic
[snip]
DEBUG: Executing shell function do_image_wic
/path/layers/oe-core/scripts/wic:27: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
from distutils import spawn
INFO: Creating image(s)...
[snip]
[RP: Added conversion for missed function reference]
(From OE-Core rev: 3966cbf5c8a2dbc3a4f0f3eefdbeeb83f522bf87)
Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The URLs describing Kickstart are no longer valid and do not redirect to
the correct location. Update them with the correct location.
(From OE-Core rev: e5ac75f93c8128b0761af5fee99e8603ddd1657d)
Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is not invoked with bitbake context as a result bb.utils is not
visible when this function is called during image creation and builds
fail e.g.
NameError: name 'bb' is not defined
(From OE-Core rev: df9dca9fe4dd1abfe5f3986389a8e8ff524da5d7)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Cc: Joshua Watt <JPEWhacker@gmail.com>
Cc: Devendra Tewari <devendra.tewari@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Following the scripted conversion adding branches to git://
SRC_URI entries, add the remaining references, mainly in the selftests
and recipetool.
(From OE-Core rev: 5340c0d688036c1be6c938f05d8a8c1e3b49ec38)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the README as license fallback if a license file is missing. Use the
linenumbers parameter of get_license_md5sums function to determine the
license text inside the README.
(From OE-Core rev: eff85c86f36673a1cb5a5dc8c66598e0dc457374)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The package.json files doesn't contain any licenses. The name of the
license doesn't comply the license requirements of most liceneses.
(From OE-Core rev: 194df9c240378b6befeb10183889093ec7bb4d5f)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support for linenumbers (begin and end lines) to licenses.csv. Add
an optional linenumbers parameter to get_license_md5sums to support
different use cases.
(From OE-Core rev: d5c4979669f125e73c24dcc73fa3c4f3787bbb62)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"A unified kernel image is a single EFI PE executable combining an EFI
stub loader, a kernel image, an initramfs image, and the kernel command
line.
[...]
Images of this type have the advantage that all metadata and payload
that makes up the boot entry is monopolized in a single PE file that can
be signed cryptographically as one for the purpose of EFI
SecureBoot."[1]
This commit adds a create-unified-kernel-image=true option to the
bootimg-efi plugin for creating a Unified Kernel Image[1] and installing
it into $BOOT/EFI/Linux/ with a .efi extension per the the Boot Loader
Specification[1][2]. This is useful for implementing Secure Boot.
systemd-boot is the only mainstream bootloader implementing the
specification, but GRUB should be able to boot the EFI binary, this
commit however doesn't implement the necessary changes to the GRUB
config generation logic to boot the Unified Kernel Image.
[1] https://systemd.io/BOOT_LOADER_SPECIFICATION/#type-2-efi-unified-kernel-images
[2] https://systemd.io/BOOT_LOADER_SPECIFICATION/
(From OE-Core rev: b0573f240525df561ddef6e47cb285b217d38487)
Signed-off-by: Kristian Klausen <kristian@klausen.dk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a check before deleting path when using recipetool commands to avoid the following type of errors:
Traceback (most recent call last):
File "<workdir>/sources/core/scripts/lib/scriptutils.py", line 218, in fetch_url
shutil.rmtree(path)
File "/usr/local/lib/python3.7/shutil.py", line 476, in rmtree
onerror(os.lstat, path, sys.exc_info())
File "/usr/local/lib/python3.7/shutil.py", line 474, in rmtree
orig_st = os.lstat(path)
FileNotFoundError: [Errno 2] No such file or directory: '<workdir>/build/tmp/work/recipetool-usg7o81n/work/recipe-sysroot'
ERROR: Command 'script -e -q -c "recipetool --color=always create --devtool -o /tmp/devtool5sq_op37 'file:///<SRCTREE>' -x <workdir>/build/workspace/sources/devtoolsrcxc1b9zjq -N test" /dev/null' failed
(From OE-Core rev: b6aa8b47e023004ffd6958d1cec18c2d9c95d77b)
Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wic fstab-update creates invalid entries for partitons that are not supposed to
be mounted from userspace eg u-boot partitions.
The following lines were added to fstab on a rock-pi-4:
/dev/mmcblk1p1 loader1 vfat defaults 0 0
/dev/mmcblk1p2 reserved1 vfat defaults 0 0
/dev/mmcblk1p3 reserved2 vfat defaults 0 0
/dev/mmcblk1p4 loader2 vfat defaults 0 0
/dev/mmcblk1p5 atf vfat defaults 0 0
/dev/mmcblk1p6 /boot vfat defaults 0 0
With this patch only valid entries should be added
/dev/mmcblk1p6 /boot vfat defaults 0 0
(From OE-Core rev: 7aa678ce804c21dc1dc51b9be442671bc33c4041)
Signed-off-by: MarkusVolk <f_l_k@t-online.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This class was added by me back in 2007 and has changed one for a whitespace issue
since. It only has two lines and neither are particularly useful, one was replaced
by the nopackages class, the other adding recursive dependencies also is now
mainly problematic adding tons of unneeded dependencies. The name is hard to
understand and the class doesn't have a clear purpose. Drop it.
Remove the references in devtool (which may be the one reason to keep it around
but in my view still not worth it).
(From OE-Core rev: 6e2cbfc561dac89bf9183d24d90e52f7d9117826)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The corrected line accidentally converted it to float,
which causes problems later on with python 3.10:
| File "/home/alex/development/poky/scripts/lib/wic/partition.py", line 278, in prepare_rootfs_ext
| os.ftruncate(sparse.fileno(), rootfs_size * 1024)
| TypeError: 'float' object cannot be interpreted as an integer
(From OE-Core rev: d1d260dd2d196d10379ed9e238bcb34f39f3a3b7)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
collections.abc is available since 3.3 and doesn't need special handling.
(From OE-Core rev: 01152c9410ba00274c8415a5d914dc33dfd0cf46)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows extra space to be added after the last partition and is
especially useful when free space is needed for ex: adding partitions on
first boot with ex: systemd-repart[1] and the image is tested in QEMU.
[1] https://www.freedesktop.org/software/systemd/man/systemd-repart.html
(From OE-Core rev: f81b188bcf5aa18746fd622eb7b5c0dcb0b5c93d)
Signed-off-by: Kristian Klausen <kristian@klausen.dk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The PyPI license classifiers include "OSI Approved: BSD", which does not
specify which of the many variations of BSD license it actually means.
The generic "BSD" license in the oe-core set is actually BSD-3-Clause.
>From a random sample of ten PyPI modules that use "BSD", they are all
BSD-3-Clause. As we expect the recipe maintainer to verify the license
anyway, and this matches the previous license text, explicitly set the
license to BSD-3-Clause.
(From OE-Core rev: a879fff9af31e45b1acc3f19a3c2a7eaf6319ad4)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When embedding a rootfs image (e.g. 'rootfs-dir') as a partition we
might want to keep the stock fstab for that image. In such a case, use
this option to not update the fstab and use the stock one instead.
This option allows you to specify which partitions get the fstab
updated and which get the stock fstab.
The option matches the argument you can pass to wic itself where the
same action is performed but for all the partitions.
Example:
part /export --source rootfs --rootfs-dir=hockeycam-image
--fstype=ext4 --label export --align 1024 --no-fstab-update
part / --source rootfs --fstype=ext4 --label rootfs --align 1024
(From OE-Core rev: ab4c95af8ecd15dc136194ab761afae756db5803)
Signed-off-by: Daniel Gomez <daniel@qtec.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a long standing bug where FILES_INFO isn't written into pkgdata
with a package suffix. This means if the files are read into the datastore
as intended, the last one "wins".
Fix this to work as intended. Most of the call sites using the data need
to be updated to handle this and the overrides change correctly.
Also fix some other problematic references noticed along the way.
(From OE-Core rev: a1190903e0a61a12c9854c96af918ae8d12c6327)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Company name is properly spelled as "Red Hat".
(From OE-Core rev: cd60f20fe40bba5b80ab604ee34fbf4ec2b0b4a2)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts part of oe-core eecbe62555, which was a previous attempt
to solve the Y2038 problem. This is now solved centrally in e2fsprogs,
so doesn't need to be dealt with in wic.
We don't revert the commit entirely, to retain the warning if a
filesystem has small inodes.
(From OE-Core rev: 7e8017208bed98b6c90735cb641fc9d7aedf9140)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The automated conversion of OE-Core to use the new override sytax isn't
perfect. This patches some mis-converted lines and some lines which were missed
by the automation.
(From OE-Core rev: 4e9a06b64b43131b731fb59a0305f78a98e27fbd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is the result of automated script conversion:
scripts/contrib/convert-overrides.py <oe-core directory>
converting the metadata to use ":" as the override character instead of "_".
(From OE-Core rev: 42344347be29f0997cc2f7636d9603b1fe1875ae)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
What this function does is really to find a layer, not a 'depends'. We
are using this function to find a dependent layer, but the name is
confusing.
(From OE-Core rev: e9b7690ab30d0e7c07471034f6cb89ccc3168a11)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Split add_layer_dependencies() into 2 parts. First search for layer
dependencies, and then add them to the config. That allows us to
call get_layer_dependencies() independently.
(From OE-Core rev: 08edf928aac3f2daaa0c256d4c21e56e2db72bff)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When S is set to a sub-directory of upstream source, the license
checks and the bbappend writing (specifically, setting EXTERNALSRC)
need to operate on that sub-directory.
'devtool modify' already has similar logic, and it was copied from there
and adjusted.
(From OE-Core rev: fd77e356d4507405fde352c8bba7d3842518bbdd)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit a2db4fa127 added ${WORKDIR}/deploy-* to
PSEUDO_IGNORE_PATHS. This breaks the --strip mode since ${D} is copied to
deploy-target-stripped. Use the directory devtool-deploy-target-stripped
instead.
[YOCTO #14451]
(From OE-Core rev: 02661f20faf11d0fa2f1874bd423f5d9fa7a31c9)
Signed-off-by: Florian Amstutz <florian.amstutz@scs.ch>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously the bootimg-pcbios wic plugin was not respecting
the --label option provided from the wks file. The plugin
was setting the label to 'boot'. With this fix, the --label
option is use. If no option are specified, then the default
is 'boot'.
(From OE-Core rev: 0fd7a73c1bd2486b7a022f0f69bbcb2e0d9cb141)
Signed-off-by: jbouchard <jeanbouch418@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's a number of recipes (such as nettle or glib) which were upgraded
incorrectly due to rebasing only the default set of patches in main SRC_URI.
Native-only or musl-only patches were not handled, causing do_patch fails
and overall misery.
Unfortunately, this also necessitates aborting any incomplete rebases as
devtool needs to be able to return to the original branch to complete
the upgrade command.
Also, do not add devtool-patched-%version tags, as they are not used anywhere,
and it's unclear how to extend them to cover multiple override branches.
(From OE-Core rev: 00131cbc969f6a73caffe068fcb983287cbe577f)
(From OE-Core rev: 588d04748dd89b5371ca9144666e97ec82dd6204)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since Linux 5.4 a new read-only filesystem is available, erofs.
Compared to squashfs it offers much better read performance with and
without compression enabled.
It suppports two optional compressors, lz4 and lz4hc.
>From the mkfs.erofs man page:
EROFS is a new enhanced lightweight linux read-only filesystem with
modern designs (eg. no buffer head, reduced metadata, inline
xattrs/data, etc.) for scenarios which need high-performance read-only
requirements, e.g. Android OS for smartphones and LIVECDs.
It also provides fixed-sized output compression support, which improves
storage density, keeps relatively higher compression ratios, which is
more useful to achieve high performance for embedded devices with
limited memory since it has unnoticable memory overhead and page cache
thrashing.
This commit adds support for three new filesystem targets:
erofs: erofs without compression
erofs-lz4: erofs with lz4 compresssion enabled
erofs-lz4hc: erofs with lz4hc compression enabled
(From OE-Core rev: 41dead1ff8ccc49e6cd6e6f5d41a59d164693e0d)
Signed-off-by: Richard Weinberger <richard@nod.at>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Incremental build in Docker fails with:
OSError: [Errno 18] Invalid cross-device link
when source and destination are on different overlay filesystems.
Rather than adding fallback code to every call site, use a new wrapper
in bitbake which detects this case and falls back to shutil.move
which is slower but will handtle the overlay docker filesystems correctly.
[YOCTO #14301]
(From OE-Core rev: 656a65b2b84e7d529b89cf5de7eb838f902d84a2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The do_kernel_configme task is no longer part of SRCTREECOVEREDTASKS,
its been removed from the kernel-yocto.bbclass since b72dbb2e4, but
there wasnt a matching patch for devtool for those changes.
This patch enables us to invoke the do_kernel_configme task when
using a devtool workspace, it also prepends a check for an existing
.config file in the source directory and moves it if thats the case,
since when using devtool modify a .config is created and
do_kernel_configme complains about it, this is not the case when
using bitbake since the .config file would be on B instead.
Alowing do_kernel_configme to run also fixes the flow where testing a
new config fragment from devtool workspace isnt added properly
(config queue shows it as ///frg.cfg) and as a side effect it never
gets merged into the final config.
(From OE-Core rev: 08dcc0e68095dcf2a159546a48b29d40c9aabc0b)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alhe@linux.microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Modern dosfstools automatically determines the appropriate size
and will error out if something that doesn't make sense is
supplied on the command line.
(From OE-Core rev: b85a09ea450a5e8f49418f4a930805fbb88dc83b)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We are getting closer and closer to the year 2038 where the 32 bit
time_t overflow will happen. While products (= embedded systems) with an
expected life time of 15 years are still save the situation may change
if your system has to survive the next 20 years.
ext2 and ext3 filesystems are always affected by the time overflow, so
let's warn the user if these filesystems are still being used.
If ext4 is affected depends on the inode size chosen during filesystem
creation. At least 256 bytes are necessary to be safe. As ext4 is
used very often (and partitions may be created small first and extended
later) this might be an issue for many users.
Some filesystems created during CI runs were already affected by the Y2038
problem. By using `--mkfs-extraopts "-T default"` we tell mke2fs not to
auto-detect the usage type based on the filesystem size. mke2fs will use
the default values for tuning parameters instead. The inode size is one
of these parameters.
(From OE-Core rev: eecbe625558406680121d2a7e84917fea45ea9dc)
Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
files in wic tmp directory can be usefull for debugging, so do not remove
tmp directory when wic create run with debugging mode (-D or --debug).
also update wic.Wic.test_debug_short and wic.Wic.test_debug_long to
check for tmp directory.
[YOCTO#14216]
(From OE-Core rev: a122e2418b67d38f691edcf8dd846c167d6b4fa9)
Signed-off-by: Lee Chee Yang <Chee.Yang.Lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add relative path if file is under directory.
[YOCTO #13738]
(From OE-Core rev: 19ddacc1b38f9ebb86a9359963ccc3c707f7125e)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If do_patch task is disabled then prepare do_configure dependencies to
fetch external sources and create symlink to ${S} in devtool workspace.
[YOCTO #13036]
(From OE-Core rev: 9e47319bfe62d289b90f7545a64dbdc1cbde7f1d)
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The fix for [Yocto #13994] required the rootfs directory to be copied
(using hardlinks if possible) when modifying the fstab file under wic.
We can optimise this copy away for filesystems where we have the tools
to modify the contents of the partition image after it is created. For
ext2/3/4 filesystems we have the debugfs tool and for msdos/vfat
filesystems we have the mcopy tool. So for any of these filesystems we
skip the modification of the fstab file in the rootfs directory (and
skip the associated copy unless it is otherwise necessary) and update
the contents of fstab directly in the partition image.
(From OE-Core rev: 5fb8ae0e9159597d7eaa9307a3a8543800bf9405)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default, wic updates the /etc/fstab in the rootfs to include details
of additional partitions described in the selected wks file. If this
modification is performed in place, other tasks which create an image
file from the rootfs directory (e.g. do_image_tar and do_image_ext4)
will pick up the modified fstab file which would not be appropriate for
those images as they do not include the additional partitions described
in the wks file. wic does undo modifications to the fstab file once it
has finished creating the filesystem image, however this leaves open a
race condition if one of the other tasks reads the contents of the fstab
file from the rootfs directory between the point where wic modifies the
fstab file and the point where wic restores the files original content.
This could be solved by adding a lockfile for tasks which use the rootfs
directory to ensure that no other such task is reading the rootfs
directory while do_image_wic is running. This would serialize several
do_image_* tasks and result in slower builds, especially for large
images. Another drawback of this solution is that it is hard to
selectively optimise - adding lockfiles to do_image_* tasks would result
in these tasks always being serialized even if no fstab modification
will take place.
An alternative solution is to copy the rootfs directory when fstab needs
to be modified. The code to do this in wic already exists as it is
needed when including or excluding content in the rootfs. This still
results in an impact on build times but the copy uses hardlinks if
possible (so little data is actually copied) and we can make selective
optimisations to improve things. The rootfs copy will only take place if
fstab modification is required (or if it was already needed to include
or exclude rootfs content). We can also follow up with further
optimisations after this commit. So this second solution is chosen.
Fixes [Yocto #13994]
(From OE-Core rev: ce682a73b7447652f898ce1d1d0416a456df5416)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To exclude content from the rootfs, wic makes a copy (using hardlinks if
possible) of the rootfs directory and associated pseudo db, then removes
files & directories as needed. However if these files and directories
are removed using the python functions os.remove and shutil.rmtree, the
copied pseudo db will not be updated correctly. For files copied from
the original rootfs, if hardlinks were used successfully when copying
the rootfs this should mean that the relevant inodes can't be reused and
so the risk of pseudo aborts should be avoided. However, this logic
doesn't apply for directories (as they can't be hardlinked) or for files
added via the '--include-path' argument (as they weren't present in the
original rootfs) and so there remains some risk of inodes being reused
and the pseudo db becoming corrupted.
To fix this, use the 'rm' command under pseudo when removing files &
directories from the copied rootfs to ensure that the copied pseudo db
is updated.
(From OE-Core rev: d5db7e268947f0392c2126137571a44acd29ccd6)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a path is specified for the internal wic working directory using
the -w/--workdir argument then it must not already exist. Re-using a
previous workdir could easily result in rootfs and intermediate files
from a previous build being added to the current image.
(From OE-Core rev: 2e40c8d4109024ff704c5ce40d98050ca7f34dd5)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before adding a npmsw fetcher to a recipe we
should first check if the generated shrinkwrap file
contains dependencies.
(From OE-Core rev: ef153ad36d0299e83a03af8f207686d0d8a238b3)
Signed-off-by: Kamel Bouhara <kamel.bouhara@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows programs from HOSTTOOLS (e.g. 'install', 'rm', 'mv', etc) to
be more easily executed by wic. Without this change only programs from
an actual *-native recipe built by bitbake can be executed by wic.
(From OE-Core rev: 8eb186acdecfbb3151c9a0ab148358e3fe5cce39)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows the path for the temporary workdir used by wic to be set
when running wic from bitbake or directly from the command line.
(From OE-Core rev: 2e7314ac3a3cab89de93c932e8efc75de0feb0a7)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
vfat/msdos filesystem should have fsuuid in format 0xYYYYYYYY where "0x"
in front follow with 8 hexadecimal number in uppercase. In wic, when using
custom fsuuid for vfat/msdos partition in wks, it is able to set the value
in any length, with or without leading "0x". This can cause fsuuid
missaligned when fstab updates, fstab expect exactly 10 character
fsuuid for vfat/msdos partition and all in uppercase.
if custom fsuuid for vfat/msdos is set, check the length and format,
error if it exceed the format size. Amend it so it is align with format
0xYYYYYYYY. This is done before image create and fstab update to ensure the
fsuuid are same in all followup process. if custom fsuuid length less than
expected, fill in "0".
[YOCTO #14161]
(From OE-Core rev: d9686ae511ef10a504becfd81bfe296b788e1456)
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The empty wic plugin is used to create unformatted empty partitions for wic
images.
To use it you must pass "empty" as argument for the "--source" parameter in
the wks file. For example:
part foo --source empty --ondisk sda --size="1024" --align 1024
Also adds a selftest for this plugin where the 'Fstype' column from 'wic
ls' should be empty for the second partition as listed in
test_empty_plugin.wks.
(From OE-Core rev: 77d174fc80663403ef76c5b808aafc1117d3545c)
Signed-off-by: Diego Sueiro <diego.sueiro@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add "payload_dir" parameter so user can add other files/dirs
to the ISO.
(From OE-Core rev: bbe84a33c81066fee96dd28a4009d8404b51e642)
Signed-off-by: Chaitanya Vadrevu <chaitanya.vadrevu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If nothing else is specified, 'git init' uses its default repository
template from the install location (e.g. /usr/share/git-core/templates),
which already includes an info/ subdirectory. However, when setting
init.templateDir to a different template path in ~/.gitconfig, this
isn't necessarily the case, and it can lead to setup_git_repo() failing
with stack traces like:
File: '.../scripts/lib/devtool/__init__.py', lineno: 234, function: setup_git_repo
0230: pass
0231: if 'singletask.lock\n' not in excludes:
0232: excludes.append('singletask.lock\n')
0233: bb.warn("try writing excludefile")
*** 0234: with open(excludefile, 'w') as f:
0235: for line in excludes:
0236: f.write(line)
0237:
0238: bb.process.run('git checkout -b %s' % devbranch, cwd=repodir)
Exception: FileNotFoundError: [Errno 2] No such file or directory: '.../devtooltmp-6m36b181/workdir/foobar-1.0.1/.git/info/exclude'
Fix this edge case by creating the .git/info/ directory first.
Fixes: 334ba846c795fc0d8c73 (2018-02-01, "devtool: set up git repos so that singletask.lock is ignored")
(From OE-Core rev: 148a23e4d5ceaf655ccacb52deca4ba501f12975)
Signed-off-by: Roland Hieber <rhi@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This variable isn't used at all.
(From OE-Core rev: 9676ee24d190b144cb50be6ebde3990644a6df2a)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adjust wic to correctly handle the new PSEUDO_IGNORE_PATH variable and avoid
inode corruption issues.
(From OE-Core rev: 13500f5234361385c365c7c35e83f99435500481)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Due to recent changes in bootimg-efi to include IMAGE_BOOT_FILES,
when both bootimg-partition and bootimg-efi occur in a single .wks
and IMAGE_BOOT_FILES are defined, files listed in IMAGE_BOOT_FILES
will be duplicated in both partition.
Since IMAGE_BOOT_FILES are crucial for bootimg-partition, but
optional for bootimg-efi, hence allowing bootimg-efi to have the option
to ignore it.
The new variable, IMAGE_EFI_BOOT_FILES, was added to help handle this
issue. Its basic usage is the same as IMAGE_BOOT_FILES.
Usage example:
${IMGDEPLOYDIR}/${IMAGE_BASENAME}-${MACHINE}.ext4;rootfs.img \
This commit is also squashed with the updated testcase to cover for
this change.
[YOCTO #14011]
(From OE-Core rev: 945339e06b273df1935cfd784f548ef57e0b7f4c)
Signed-off-by: Khairul Rohaizzat Jamaluddin <khairul.rohaizzat.jamaluddin@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a package contains hard links to a file, the file size is added for
each hard link instead of once for the file. Therefore, the calculated
size may be much larger than the actual package size.
For example, the mesa-megadriver package contains several hard links to
the same library.
Keep track of the inode numbers when listing the files that are
installed and use the actual size only for the first occurrence of an
inode. All further hard links to the same inode are added to the file
list, but accounted with size 0.
All file names need to be added to the file list, because the list is
used for preserving the files/hard links on the target.
(From OE-Core rev: 17e92572278980d1a7f06de9d72c68baf57698f1)
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
/bin is also a valid path where one can find executables. Add
that to the search path.
(From OE-Core rev: ca0a6025351cb2135e87cecf828633cf12aa34c6)
Signed-off-by: Vijai Kumar K <vijaikumar.kanagarajan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* I have recipes which use variable inside SRC_URI, e.g.:
ROS_BRANCH ?= "branch=release/melodic/swri_nodelet"
SRC_URI = "git://github.com/swri-robotics-gbp/marti_common-release;${ROS_BRANCH};protocol=https"
and devtool modify works fine, but devtool finish fails with:
$ devtool finish --force-patch-refresh swri-nodelet meta-ros/meta-ros1-melodic/
...
Traceback (most recent call last):
File "/OE/openembedded-core/scripts/devtool", line 334, in <module>
ret = main()
File "/OE/openembedded-core/scripts/devtool", line 321, in main
ret = args.func(args, config, basepath, workspace)
File "/OE/openembedded-core/scripts/lib/devtool/standard.py", line 2082, in finish
updated, appendfile, removed = _update_recipe(args.recipename, workspace, rd, args.mode, appendlayerdir, wildcard_version=True, no_remove=False, no_report_remove=removing_original, initial_rev=args.initial_rev, dry_run_outdir=dry_run_outdir, no_overrides=args.no_overrides, force_patch_refresh=args.force_patch_refresh)
File "/OE/openembedded-core/scripts/lib/devtool/standard.py", line 1737, in _update_recipe
mode = _guess_recipe_update_mode(srctree, rd)
File "/OE/openembedded-core/scripts/lib/devtool/standard.py", line 1721, in _guess_recipe_update_mode
params = bb.fetch.decodeurl(uri)[5]
File "/OE/bitbake/lib/bb/fetch2/__init__.py", line 390, in decodeurl
raise MalformedUrl(url, "The URL: '%s' is invalid: parameter %s does not specify a value (missing '=')" % (url, s))
bb.fetch2.MalformedUrl: The URL: 'git://github.com/swri-robotics-gbp/marti_common-release;${ROS_BRANCH};protocol=https' is invalid: parameter ${ROS_BRANCH} does not specify a value (missing '=')
let it expand the SRC_URI before trying to decode it.
(From OE-Core rev: 3535cfdbf3d77f550b804276f957acf859da484f)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allows the --offset argument to use the "s" or "S" suffix to specify
that it is reporting the number of 512 byte sectors.
This is required for some SoCs where the mask ROM looks for an item at a
sector that isn't aligned to a 1KB boundary.
(From OE-Core rev: 938595d1dc4abaf5f7f3a7900add3f0492b805d0)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
layer under test should absent from BBLAYERS when running
yocto-check-layer. This allow to get signatures before layer
under test. There are existing steps to add the layer under
test to BBLAYERS after getting initial signatures.
add steps to check for layer under test in BBLAYERS before
running any test, skip test for the layer if the layer under
test exist in BBLAYERS.
[YOCTO #13176]
(From OE-Core rev: be02e8dbfb0d1decce125322f9f1e11a649756c0)
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
List of files defined using IMAGE_BOOT_FILES are installed into
the boot partition when preparing an image using the wic tool with
the bootimg-efi source plugin.
The code snippet introduced is taken as is from bootimg-partition.py
Change-Id: I8dbb6b4e7c24870f587a6f31e6ae4a87d7033782
Issue-Id: PLATFORMS-3134
(From OE-Core rev: a44ab3a4ee5b3c57812909c6194456f299d6ba7f)
Signed-off-by: Khasim Syed Mohammed <khasim.mohammed@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For some filesystems, such as aufs which may be used by docker container,
don't support either the SEEK_DATA/HOLE or FIEMAP to get the block
map. So add a FileNobmap class to fall back to standard copy when there
is no way to get the block map.
[Yocto #12988]
(From OE-Core rev: 7934ed49179242f15b413c0275040a3bb6b68876)
Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This method is not used by any code, so drop it.
(From OE-Core rev: 6513fd9302b9989f97fc9d95e76e06ad5d266774)
Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This method is not used by any code, so drop it.
(From OE-Core rev: fb62a15349597ee026c67a0bb0a6ca2cc9bfe420)
Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python now expects regex strings to be prepended with r.
Silence pylint/autopep8 and similar warnings by identifying
these regex patterns as... regex patterns.
(From OE-Core rev: 0eae98a369d80340e48dc690d09a1364cde97973)
Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[Bug 13971] https://bugzilla.yoctoproject.org/show_bug.cgi?id=13971
A recipe using LICENSE=CLOSED would normally not have LIC_FILES_CHKSUM
set.
If LIC_FILES_CHKSUM was not set, it would cause `devtool upgrade` to fail.
This fixes the issue by defaulting to an empty string if
LIC_FILES_CHKSUM is not found.
[YOCTO #13971]
(From OE-Core rev: 95e112f0f465868e98285509ea0d1a624a439ac2)
Signed-off-by: Bjarne Michelsen <bjarne_michelsen@hotmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
LAYERDEPENDS is a string of this format:
"DEPEND1 (optional version) DEPEND2 (optional version) ..."
However when we parse LAYERDEPENDS in _get_layer_collections() we
parse it as a simple string, and if any optional versions are there the
'depends' field is wrong. For example, running yocto-check-layer
might result in such errors:
ERROR: Layer meta-python depends on (>= and isn't found.
ERROR: Layer meta-python depends on 12) and isn't found.
Let's use bb.utils.explode_dep_versions2() to parse LAYERDEPENDS, and
create a string that contains all dependencies, effectively
skipping/ignoring any optional versions.
[YOCTO #13957]
(From OE-Core rev: f81f07afc200fe06c5c06ea81a4f8a3a43436faf)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When explicitly passing a branch using --srcbranch in 'devtool add' or
'recipetool create', the branch name is not included in the params of
bb.fetch2.encodeurl and default 'master' branch is used instead.
(From OE-Core rev: 0424df825f1e509faf6cd44403c0736bb91b57c3)
Signed-off-by: Tuomas Salokanto <tuomas.salokanto@gmail.com>
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The error message was reporting the calculated offset instead of the
current offset, which made it confusing.
(From OE-Core rev: 2be775cfe1b49ce3889b5dc326e2b67a9667f18a)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If there are multiple appends, ensure we concatenate compatible things:
Traceback (most recent call last):
File "/home/akiernan/poky/scripts/recipetool", line 111, in <module>
ret = main()
File "/home/akiernan/poky/scripts/recipetool", line 100, in main
ret = args.func(args)
File "/home/akiernan/poky/scripts/lib/recipetool/edit.py", line 38, in edit
return scriptutils.run_editor([recipe_path] + appends, logger)
TypeError: can only concatenate list (not "tuple") to list
(From OE-Core rev: 4c1e74bdf4922519d168434afd69c9bebcb9bd82)
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
467f84e12b ("wic: Add --offset argument for partitions") broke the
--extra-space argument handling in wic. Fix the option and add a unit
test for the argument.
(From OE-Core rev: 87722a92c18f94917c8f70afc8cd0763462a5c25)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support for an --offset argument when defining a partition. Many
SoCs require that boot partitions be located at specific offsets. Prior
to this argument, most WKS files were using the --align attribute to
specify the location of these fixed partitions but this is not ideal
because in the event that the partition couldn't be placed in the
specified location, wic would move it to the next sector with that
alignment, often preventing the device from booting. Unlike the --align
argument, wic will fail if a partition cannot be placed at the exact
offset specified with --offset.
Changes in V2:
* Fixed a small typo that prevented test_fixed_size_error from passing
(From OE-Core rev: 467f84e12b96bc977d57575023517dd6f8ef7f29)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* I see a case where a tarball contains .gitignore and bunch of files
which are normally ignored in git, but still included in the tarball
(e.g. configure script next to configure.ac)
* when devtool is creating a git repo in workspace it won't include these
files from tarball in the initial devtool-base commit, because
git ls-files won't list them
* but then the first .patch file (without git headers) when applied with
GitApplyTree._applypatch() will add all these still ignored files to a
commit which used to only modify some files, because it's using -f:
# Add all files
shellcmd = ["git", "add", "-f", "-A", "."]
output += runcmd(["sh", "-c", " ".join(shellcmd)], self.dir)
at least in this case it would be better to add all ignored files in
the initial devtool-base commit and then --force-patch-refresh will just
include the small modification as before instead of adding unrelated
files, just because they were initially ignored - this behavior will
also match with the do_patch task in the actual build where the
.gitignore is ignored when unpacking some tarball
* my use-case is fixed in setup_git_repo, but similar function is in
devtool upgrade, I've changed it there as well
(From OE-Core rev: 06a24a615549af3550302a56ea08147000a608f3)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently only ptest logs are accessible with the log command, this
adds support so the ltp logs can be extracted too.
(From OE-Core rev: 64a2121a875ce128959ee0a62e310d5f91f87b0d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I've long since wondered why there was some odd output in result reports,
remove the leftover debug which was causing it.
(From OE-Core rev: 66e96bf70753933714ff8edcc13a1f35a052656f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As in other places in the file, skip layers that don't define a
collection when searching for a layer to resolve a dependency. Fixes
KeyError exceptions when attempting to access the layer collections
later
(From OE-Core rev: 26090a2861ebe21224aaf89d7be0c0a89ca58e48)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will drop them md5sums from recipes that still have them,
and will not re-add them for recipes where they're already
removed.
[RP: Added fix for the test recipe]
(From OE-Core rev: 1609e77a4db04a4c01cbbff17261cd57e8f816ee)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Combine all the common path validation in a function to avoid code
duplication.
Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: ebd12b10d17db0b4176b0188407d7e9b8420eab1)
Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow specifying an optional destination to include-path and make the
option aware of permissions and owners.
It is very useful for making a partition that contains the rootfs for a
host and a target Eg:
/ -> Roofs for the host
/export/ -> Rootfs for the target (which will netboot)
Although today we support making a partition for "/export" this might
not be compatible with some upgrade systems, or we might be limited by
the number of partitions.
With this patch we can use something like:
part / --source rootfs --fstype=ext4 --include-path core-image-minimal-mtdutils export/ --include-path hello
on the .wks file.
Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: e8c21c6ebaebde88151697381bdb2452f1171090)
Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Executables like tar won't be available on the native sysroot, as they
are part of the ASSUME_PROVIDED variable.
Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: 2f574d535f8665b26dab65c14668cf8fc7b751c0)
Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the source of the rootfs is not a bitbake cooked image, or it is not
pointing to the root of one, we call pseudo again, which will produce
a new pseudo folder at rootfs/../pseudo
Eg:
part /etc --source rootfs --rootfs-dir=${IMAGE_ROOTFS}/home --fstype=ext4
Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: 799a24ae78655f7a3eda7456b1a0ffaf3e43ec16)
Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If an excuded path does not exist, continue without an error.
This allows to seamleasly reuse .wks among different projects.
Eg:
part / --source rootfs --fstype=ext4 --exclude-path=opt/private_keys
Where /opt/private_keys in only populated by some of the image.bb files.
Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: 2dbd692b8e563cf991fb4ae1ef6129fda0d7e3c4)
Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This option allows to specify which part of a rootfs is going to be
included, the same way the -C argument on tar.
Thanks to this option we can make sure the permissions and usernames
on the target partition are respected, and also simplify the creation of
splitted partitons, not neeting to invoke external vars or using .wks.in
files. Eg:
part / --source rootfs --ondisk sda --fstype=ext4 --exclude-path=etc/
part /etc --source rootfs --fstype=ext4 --change-directory=etc
Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: 2265d089a58e1f78f26d623ee667c420cb1c3bd4)
Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to support .wks files with multiple images inside bitbake we
need to explicitly set the pseudo database in use.
Eg: If we try this .mks:
part / --source rootfs --ondisk sda --fstype=ext4
part /export --source rootfs --rootfs=core-image-minimal-mtdutils --fstype=ext4
The username for all the files under /export will be set to the runner
of bitbake (usually UID 1000).
Before we run wic, we need to make sure that the pseudo database will be
flushed, and contains all the data needed.
Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: dde90a5dd2b22a539095d1bac82acc15c6380ac8)
Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When parameters include_path or exclude_path are passed to the rootfs
plugin, it will copy the partition content into a folder and make all
the modifications there.
This is done using copyhardlinktree(), which does not take into
consideration the content of the pseudo folder, which contains the
information about the right permissions and ownership of the folders.
This results in a rootfs owned by the user that is running the wic
command (usually UID 1000), which makes some rootfs unbootable.
This bug can be easily triggerd with the following .wks
part / --source rootfs --fstype=ext4 --exclude-path=home
And this sequence:
$ wic create test-permissions -e core-image-minimal -o test/
$ sudo mount test/test-permissions-202004080823-sda.direct.p1 /mnt
$ ls -la /mnt/etc/shadow
To fix this we copy the content of the pseudo folders to the new folder
and modify the pseudo database using the "pseudo -B" command.
If the rootfs is not a rootfs generated by bitbake a warning is shown
making the user aware that the permissions on the target might not match
what he expects.
WARNING: /tmp/test/../pseudo folder does not exist. Usernames and permissions will be invalid
Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: fd739c15cdba221d9d497d3402b7d929c0be2ca4)
Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
part.get("uuid") return the Partition UUID from the partition
table instead of Filesystem UUID. This lead to swap partition
UUID not match/change when wic write expand swap partition.
change it to read the filesystem UUID using blkid. The output
from blkid should looks like this:
wic-partvzhiwq3s: LABEL="swap1" UUID="04e55c19-3f3f-4491-9e32-44eea8daa827" VERSION="1" TYPE="swap" USAGE="other"
[YOCTO #13313]
(From OE-Core rev: 683297eefebe83f848daad9927871242ab28ef91)
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit 36993eea89d1c011397b7692b9b8d61b499d0171.
After discussion on the mailing list it was felt these changes
were not ready yet.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit efdcf94801f6abe8e4099e324d9a3deccd8d4384.
After discussion on the mailing list it was felt these
changes were not ready yet.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This option adds the content of a rootfs on a specific location on the
rootfs.
It is very useful for making a partition that contains the rootfs for a
host and a target Eg:
/ -> Roofs for the host
/export/ -> Rootfs for the target (which will netboot)
Although today we support making a partition for "/export" this might
not be compatible with some upgrade systems, or we might be limited by
the number of partitions.
With this patch we can use something like:
part / --source rootfs --embed-rootfs target-image /export --embed-rootfs target-image2 /export2
on the .wks file.
(From OE-Core rev: efdcf94801f6abe8e4099e324d9a3deccd8d4384)
Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When parameters include_path or exclude_path are passed to the rootfs
plugin, it will copy the partition content into a folder and make all
the modifications there.
This is done using copyhardlinktree(), which does not take into
consideration the content of the pseudo folder, which contains the
information about the right permissions and ownership of the folders.
This results in a rootfs owned by the user that is running the wic
command (usually UID 1000), which makes some rootfs unbootable.
To fix this we copy the content of the pseudo folders to the new folder
and modify the pseudo database using the "pseudo -B" command.
(From OE-Core rev: 36993eea89d1c011397b7692b9b8d61b499d0171)
Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wic look for external layer source plugin under scripts/lib/
while other tools look for lib/.
allow wic to check for source plugin at both scripts/lib/ and
lib/ to align with other tools while avoid breaking any existing
source plugin in external layer.
[YOCTO #13056]
(From OE-Core rev: 986baff26bd96a6265f5fe2d631818fff9f66374)
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
extended partition should reserve at least 2 unallocated sectors
between the start of the extended partition and the start of a
logical partition.
[YOCTO #13658]
(From OE-Core rev: 884c8a0e50a49bdca5e048197a9dc0ff37cc8d1a)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the requested partition does not exist in this request "wic ls {path}:pnum"
display a nice message not a trackback
Also fix displaying the pnum and not "%s"
(From OE-Core rev: 15d1722950a22649905cf8a5789d3cfe48a2a892)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
since commit 03154d2cf25c1d5ce908da9e72d324004b8d5722 ("sfdisk: add -J between mutually exclusive options")
in util-linux [1], the '-J' can't be used with the '-d' option anymore.
So, since we're using json format output, drop the '-d' option.
Fixes:
ERROR: _exec_cmd: <snip>/usr/sbin/sfdisk -dJ <snip>/core-image-minimal-qemux86-64.wic returned '1' instead of 0
output: sfdisk: mutually exclusive arguments: --list-free --json --dump
[1] - 03154d2cf2 (diff-31d5cc325380dbdafe64d0e019846f12)
(From OE-Core rev: 49e7f3e8e3a775510d8b0e2826a06d000934706b)
Signed-off-by: Pierre-Jean Texier <pjtexier@koncepto.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was added ages ago to enable GL passthrough with
vmware driver, and is no longer relevant, as std or virgl is used
instead nowadays.
Original commit:
commit 072545b111
Author: Richard Purdie <rpurdie@linux.intel.com>
Date: Wed Jan 21 17:40:51 2009 +0000
scripts/poky-qemu-internal: Add support for GL passthrough in qemux86 images
(From OE-Core rev: 857078ba8eda153f4a097683db551a7d310ecc01)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
APPEND is used to add additional parameters to kernel by features, such
as read-only-rootfs. So we should honor this variable when we compose
the kernel parameter in wic. I know we also can resolve this kind of
issue by using the .wks.in template introduced by commit 42e870c5ed
("image_types.bbclass: support template .wks.in files for wic"), but the
APPEND is needed by all the wks, it would seems pretty ridiculous to me
to change all the .wks to .wks.in and then foist the APPEND into them.
So the APPEND is definitely deserved to export to the wic directly.
[Yocto #12809]
(From OE-Core rev: 18981b8a457104391dfd94938c247eac04e4ed50)
Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This error handling didn't work as expected since upon failure it would
inject bytestreams back into the code leading to tracebacks.
Instead, ignore the decode errors. Fixes:
Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/a-full/build/scripts/resulttool", line 78, in <module>
sys.exit(main())
File "/home/pokybuild/yocto-worker/a-full/build/scripts/resulttool", line 72, in main
ret = args.func(args, logger)
File "/home/pokybuild/yocto-worker/a-full/build/scripts/lib/resulttool/store.py", line 70, in store
resultutils.save_resultsdata(results, tempdir, ptestlogs=True)
File "/home/pokybuild/yocto-worker/a-full/build/scripts/lib/resulttool/resultutils.py", line 178, in save_resultsdata
f.write(sectionlog)
TypeError: write() argument must be str, not bytes
(From OE-Core rev: 3aa0d0fd0bbf8c9c13719251a2fd1e85a22c7336)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enable reporting selected test case result given the user provided
the selected test case id. If both test result id and test case id
were provided, report the selected test case result from the
selected test result id.
(From OE-Core rev: 7161310ce32d6e0c397d0132808d556bdc80d183)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
u-boot.inc supports u-boot recipes with or without menuconfig [1].
However, running devtool on a u-boot recipe that does not support menuconfig
results in an error:
cp: cannot stat '/home/r60874/upstream/fsl-xwayland/tmp/work/imx8mmevk-fsl-linux/u-boot-imx/2018.03-r0/u-boot-imx-2018.03//.config': No such file or directory
The problem is the devtool logic assumes that any recipe with a do_menuconfig task
will generate a .config in do_configure().
Fix the problem by removing the assumption with a flag that the recipe can control,
like this:
do_configure() {
if [ menuconfig-supported ]; then
...
else
DEVTOOL_DISABLE_MENUCONFIG=true
fi
}
[1] 11278e3b2c
(From OE-Core rev: 6cc17e658c3dbadc772353f0f200561d2d988658)
Signed-off-by: Tom Hochstein <tom.hochstein@nxp.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit removes the 'noverify' parameter which was added to the url
to fix warnings with the shrinkwrap / lockdown file generation. This is
not needed anymore with the new npm fetcher.
(From OE-Core rev: 401a9818c633373854d3c7dec032b9f455f2b2b4)
Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When creating a recipe using devtool, a workspace is created to store
the new recipe, the recipe source and some append files. These append
files are used by devtool to build the recipe using externalsrc (to use
the source which are in the workspace). They can also have some
additional actions according to the class of the recipe.
This commit updates the append file for the npm recipes. The
devtool / externalsrc files are removed in the npm package directory
instead of the install directory.
(From OE-Core rev: a213a76e56e9bc215a61399ae35a3602863f5850)
Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As usual the 'LICENSE' and the 'LIC_FILES_CHKSUM' values reflects all
the license files discovered in the source tree (including the
dependencies).
For npm recipes the 'LIC_FILES_CHKSUM' value contains also the status of
the 'package.json' file of every packages as it contains license
informations.
Finally each package has a separate 'LICENSE_${PN}-package-name' value
which describes its license.
(From OE-Core rev: 9a70d4996c84b277f423eda5aac4acbe344599f4)
Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit refactors the npm recipe creation handler to use the new npm
behavior. The process is kept as simple as possible and only generates
the shrinkwrap file.
To avoid naming issues the recipe name is now extracted from the npm
package name and not directly mapped.
(From OE-Core rev: 1deccb0f0c204cd02fb8606f180d8a13df9f31db)
Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit renames the '--fetch-dev' option into '--npm-dev' as it is a
npm only option.
(From OE-Core rev: 2b75cc848ceebee4067788a621299bfd5fb62231)
Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
OE wic plugins create temporary file with the index of the line
tmp file name. This causes race in case several builds run in time.
If source_params['file'] is an absolute path, the cr_workdir prefix
is not applied by os.path.join(). So instead it writes to a ".1"
file next to the original image - this is outside the WORKDIR
and at risk of collision.
(From OE-Core rev: c68d8a37ba4348fe1c0e75c63b5668187d326ec2)
Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
Suggested-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Suggested-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This option adds the contents of the given path to a partition built
with the rootfs source plugin. The path is relative to the directory in
which wic is running not the rootfs itself so use of an absolute path
is recommended. This option is most useful when multiple copies of the
rootfs are added to an image and it is required to add extra content to
only one of these copies. This option only has an effect with the
rootfs source plugin.
(From OE-Core rev: d4cd27a9837426e809190548a83c6c7c76505114)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When debugging issues or trying to add new features in wic it's useful
to examine the intermediate files created by the tool such as images of
the individual partitions. Previously these files were created in a
subdirectory of IMGDEPLOYDIR and, deleted by wic itself and then the
working subdirectory completely removed at the end of IMAGE_CMD_wic.
Instead we should work in a staging directory named build-wic and leave
behind intermediate files for analysis if necessary.
This has the added benefit of allowing individual partition images to
be preserved by adding commands to IMAGE_CMD_wic_append() in an image
recipe if needed. This is useful where an update system requires exact
sized and pre-formatted partition images for deployment.
(From OE-Core rev: 07b6b284204e912b5d08bdd0ca51b766ab7dd9c1)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some file systems don't support fetching the block size (notably the
file system Docker uses for containers), so if iotctl() fail, try to use
failback via os.stat() to get block size.
(From OE-Core rev: e219f5175177a640dd62833082ea19adc1c13d42)
Signed-off-by: Kalle lampila <kalle.lampila@lempea.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
imp was replaced by importlib as it's deprecated, but the import
line was left, so remove this useless line.
(From OE-Core rev: 95574a7d18a8acfbed1b956677e72044bdfa7ebc)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
currently 'wic cp' only works for copy file from local storage to
wic image.
enhance 'wic cp' to copy file/directory from wic image to local storage.
include selftest and 'wic help' updates.
[YOCTO#12169]
(From OE-Core rev: bd669c1809a378f93580eb9e0679a26ec6746cb8)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the reproducible_build class is inherited then there may be a
"source-date-epoch" subdirectory in a fetched source tree; devtool
upgrade was not expecting that in the upgraded source. Take a small
snippet of code from recipetool create which already handles this,
and make it a shared function that can be used in both places.
Additionally, fix an assumption that the source is always in a
subdirectory in the cleanup code that blocked debugging this.
[YOCTO #13635]
(From OE-Core rev: 0d642861cd9cf034b8d4951433980addc215d4fd)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit 41d225f4a3.
Unfortunately this change broke 'devtool upgrade' functionality,
causing 'devtool finish' to write out an upgraded recipe that no
longer includes the original upstream source in SRC_URI.
(From OE-Core rev: 2d6e55192dba0bf7f6e23e5ab5b3dbc68835bb28)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The extra diretory created by reproducibile_builds was confusing recipetool
and hence devtool. Ignore this directory to restore the correct behaviour and
allow it to work with the class.
(From OE-Core rev: 188ba30ef290efab6d5a9b95fba8e991b779ab33)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wic currently unable to remove non-empty directory in ext* partition.
enable wic rm to remove non-empty directory and all the sub-content
with -r flag.
update help documents for 'wic rm'.
[YOCTO #12404]
(From OE-Core rev: 5cb7a329d0aaac8fe5328eb2001692c540aa5ade)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enable the option to add extra test environment data to the
configuration of each test result (as optional).
Example of optional test environment data include:
- custom packages included for runtime test
- detail machine specification used as target
- detail host environment used for bitbake
(From OE-Core rev: 3cc887a72c5726024c3699e6a3deb72fad4992a9)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add total passed, failed, and skipped statistic to test result.
(From OE-Core rev: 7ad74f99b6105f23bcf930712627aac279c6af10)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In case of debugging, report user need to acccess the raw
test result. Instead of going back to source file/directory/URL
to manually pull out the raw result, provide alternative
way to let report showing raw test results by providing
the result id (optional).
(From OE-Core rev: 8b2485bc0d0fa2f72b37e8bafbfa663d456ea98c)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default, report will use the store_map to generate the key
to reference each result set. In some situation when using store_map
with multiple set of tests sharing similar test configurations,
the report will only showing partial result set for results
that having identical result_id (use of multiconfig to run tests
where it generate identical result_id).
Enable report to have the option to use the regression_map (optional)
instead of the default store_map, where it will take larger
set of configurations to generate the key to reference each
result set, this will prevent the report from only showing
partial result set.
(From OE-Core rev: 8577abce6feb1f529537da554d39e86a5b05ca2d)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The Wic help returned to the user is unreadable.
Use a custom ArgumentParser to override argparse help message.
change help message as suggest in
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12205
[YOCTO #12205]
changes applies to 'wic help', 'wic -h', 'wic --h' and 'wic --help'
(From OE-Core rev: d29d553cc096f4e1208c44dc595e1cf365c3dff3)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add "esp_extra_blocks" plugin parameter so that caller may change
ESP's free space from the default 100 blocks.
(From OE-Core rev: db4be094b0d6741ca4f9e2cbb499fc9c224fa690)
Signed-off-by: Haris Okanovic <haris.okanovic@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[YOCTO #13312]
see https://bugzilla.yoctoproject.org/show_bug.cgi?id=13312
wic/engine.Disk._get_part_image was looking at variable fstypes for
supported fstype which is 'swap' but image build with 'linux-swap'.
supported fstype should be 'linux-swap'.
(From OE-Core rev: 7e6da22fe4faf841bcec02e55f376b4dae04d6a8)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enable --remove-work option for devtool reset command that allows user
to clean up source directory within workspace.
Currently devtool reset command only removes recipes and user is forced
to manually remove the sources directory within the workspace before
running devtool modify again.
Using devtool reset -r or devtool reset --remove-work option, user can
cleanup the sources directory along with the recipe instead of manually
cleaning it.
syntax: devtool reset -r <recipename>
Ex: devtool reset -r zip
devtool finish -r <recipename> <layer-name>
Ex: devtool finish -r zip meta-yocto-bsp
(From OE-Core rev: 0d6dd01b8633ad63d77c24568a4b20852aaa366f)
Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the source file is located in a subdirectory of DEPLOY_DIR
rawcopy will currently fail in sparse_copy function on
open(dst_fname, 'wb'), as the parent directory for destination
file does not exist.
This patch helps to avoid that by recursively creating
parent directories.
(From OE-Core rev: 073c435644091c2801e45c6d02afa917de575082)
Signed-off-by: Eugene Smirnov <evgenii.smirnov@here.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When wks describes extra partitions that aren't in the partition
table (e.g. boot loader) and exactly four primary MBR partitions, the
last partition gets added to fstab as partition #5 instead of #4.
[YOCTO #13560]
(From OE-Core rev: 7537580b3dd21bd512fb26e56e92b6553c549fa8)
Signed-off-by: Michael Cooper <michaelcooper81@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
All devtool commands right now are filtering out the devtool workspace
bbappends in build/workspace/appends when calling parse_recipe. While
this may make sense for devtool add and modify, we need devtool finish
to include those appends.
A specific breakage that is caused because devtool finish filters devtool
appends is the cmake/cml1 flow where a file is created in the WORKDIR
that finish needs access to, to commit those files. Particularly for
git packages with SRCPV in PV, SRCPV is only changed to 999 when using
external source, hence when creating the cfg or cmake config files using
for instance bitbake -c diffconfig, these files are created in the
git999 workdir correctly (as in the devtool bbapends, we are inheriting
externalsrc class). But when devtool finish is run, the devtool appends
are not parsed, hence SRCPV is not changed to 999 and devtool is looking
for the fragment files in the wrong WORKDIR.
Changing the parse_recipe call just in devtool finish to not filter out
the devtool workspace.
Fixes [YOCTO #13533]
(From OE-Core rev: aa94f00eec64ef936acc80dfa826fd309daa294f)
Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Do not decode the log content into a string only to re-encode it as
binary data again. Some logs might un-intentionally contain bytes that
do not decode as utf-8, as such preserve the log file content as it was
on disk.
Handle the decoding on the resulttool side, but also handle the failure
to decode the data.
(From OE-Core rev: 20531dc0b8f76a6e37cc856f36cd94077b6aba50)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This works just like the already existing --no-clean option to the
`devtool reset` command.
(From OE-Core rev: 4433d3133ce53cdf7db17f5e7914fe8c4d78dba3)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The _get_patchset_revs() function returns the patches in an
OrderedDict to keep them ordered. However, this information was lost
when the patches were added to the bbappend file.
(From OE-Core rev: 69b7a2ba3af1280cc220ab236032f8466246ee93)
Signed-off-by: Niclas Svensson <niclas.svensson@axis.com>
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The commit 8e48b4d6c4 makes wic ignores IMAGE_ROOTFS_SIZE for rootfs
size and makes it uses the computed one only. Re-add support for
IMAGE_ROOTFS_SIZE variable and compute roots size only if the former
is not defined. Moreover the size of a provided directory with
--rootfs-dir="" in the .wks file should always be computed on the fly,
else every partition will be constrained to be the same size as what
ever value was in ROOTFS_SIZE.
(From OE-Core rev: 0217a0392b8cc534fcc0d760b7663a1845b306f5)
Signed-off-by: Alessio Igor Bogani <alessio.bogani@elettra.eu>
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the OEPTestResultTestCase class as a mix-in class to provide helper
functions for interacting with ptestresults within the extraresults
object generated by the test case.
This class also provides default compression of log text and log files.
Also add support to resulttool for decoding/decompressing log files
embedded in the test results.
(From OE-Core rev: 06cba9883a5964320969301fd05eeb6bec3e786d)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Handle multiple results series having ptestresults content. The contents
are merged on a per-result basis where duplicates are ignored (with a
warning message printed). The 'ptestresults.sections' collection is also
merged on a per-suite basis.
(From OE-Core rev: 47edd51970ed0c33edbe04fd72abd1cfc6ecd3d1)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds an argument to the log subcommand to extract the raw logs from the
reproducible selftest.
To prevent ambiguity, the "--raw" argument has been renamed
"--raw-ptest", although the old "--raw" argument is kept around for
compatibility.
[YOCTO #13324]
(From OE-Core rev: 7a4ebb361ff1efc22e7dafadfa60c98bc8a79ed4)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Similar to bitbake, implement a 'keepalive' output to the console to
ensure CI systems do not kill the process. The default timeout for
bitbake is 5000s.
(From OE-Core rev: 77939cca96fa5467c88eafa3ac0db2db4aef09d6)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows us to generate a rootFS with a large filesystem for use with
QEMU.
(From OE-Core rev: e06439200e44999c1e2f88d7d6c651da13698ca7)
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When extracting the sources for a recipe that has S == WORKDIR and no
local files in the SRC_URI (which, e.g., can happen for a recipe with
a URI that has the unpack=false attribute), the extraction fails with
the following backtrace:
Traceback (most recent call last):
File ".../scripts/devtool", line 344, in <module>
ret = main()
File ".../scripts/devtool", line 331, in main
ret = args.func(args, config, basepath, workspace)
File ".../poky/scripts/lib/devtool/standard.py", line 762, in
modify
initial_rev, _ = _extract_source(srctree, args.keep_temp,
args.branch, False, config, basepath, workspace,
args.fixed_setup, rd, tinfoil, no_overrides=args.no_overrides)
File ".../poky/scripts/lib/devtool/standard.py", line 647, in
_extract_source
bb.process.run('git %s commit -a -m "Committing local file
symlinks\n\n%s"' % (' '.join(useroptions),
oe.patch.GitApplyTree.ignore_commit_prefix), cwd=srctree)
File ".../poky/bitbake/lib/bb/process.py", line 178, in run
raise ExecutionError(cmd, pipe.returncode, stdout, stderr)
bb.process.ExecutionError: Execution of 'git commit -a -m
"Committing local file symlinks
%% ignore"' failed with exit code 1:
On branch devtool
nothing to commit, working tree clean
This is because no files were found in the oe-local-files directory
and consequently no symbolic links were added using `git add`, but the
`git commit` command was still executed.
(From OE-Core rev: 3fdf304e72a1fb5de8bf9bc21e5b598fefb08648)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's become more commone for files to be named "readme" or "Readme" on github servers
in recent time. So adjust the scanning to allow any mix of case.
(From OE-Core rev: afe46eca15b6ddfa15c75cb7b707d6dd9aae3eae)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add additional result types into the dictionary to handle dejagnu style
test results. These include PASS, FAIL, XPASS, XFAIL, UNSUPPORTED,
UNTESTED, UNRESOLVED and ERROR.
(From OE-Core rev: 684fc36402a23760b203f4761f284043031c799c)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Right now `devtool build` runs populate_sysroot and packagedata tasks.
Adding deploy to this list, if the recipe has the deploy task, so that
the newly built artifacts are available in the deploy directory.
Applicable only for packages with deploy task, such as kernel.
[YOCTO#13382]
(From OE-Core rev: b38a1328f0c7bc4b4102a05daee4058fd3214489)
Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com>
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, if a test occurs multiple times over different series, the
code will sum these. This can lead to confusion over the desired
results. Change the code to report the redundant tests and skip adding
an additional values.
(From OE-Core rev: caeaa0648dff64c56a33f52e45e82bfab6719e3e)
Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The layer checks are designed to work with OEBasicHash so ensure that handler
is in use rather than the new hash equivalency one as an example.
(From OE-Core rev: a10bf92516a4771e2dc49ba9f74323d7a87a1619)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using different root directories with a wks file wic is using the
value from the original ROOTFS_SIZE which is not correct. Example:
Number Start End Size File system Name Flags
1 20.5kB 318MB 318MB fat16 otaefi legacy_boot, msftdata
2 318MB 636MB 318MB ext4 otaboot
3 636MB 1709MB 1074MB ext4 otaroot
4 1709MB 2027MB 318MB ext4 otaboot_b
5 2027MB 3101MB 1074MB ext4 otaroot_b
6 3101MB 5249MB 2147MB ext4 fluxdata
The partitions 1, 2, and 3 incorrectly inherit the size, instead of using
a computed size. With the patch applied it is working properly:
Number Start End Size File system Name Flags
1 20.5kB 14.5MB 14.5MB fat16 otaefi legacy_boot, msftdata
2 14.5MB 65.3MB 50.8MB ext4 otaboot
3 65.3MB 1139MB 1074MB ext4 otaroot
4 1139MB 1190MB 50.8MB ext4 otaboot_b
5 1190MB 2264MB 1074MB ext4 otaroot_b
6 2264MB 4411MB 2147MB ext4 fluxdata
As for the removal of the bb.warn, if the size is not specified, it is
not something to warn the end user about. Some of my default images
make use of the head room + a computed directory size or while generating
images.
(From OE-Core rev: 8e48b4d6c4d0ed213089a7449fea63aa0656e786)
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to create a msdos partition table disk image that can auto
expand after the image is copied to an SD card, wic needs the ability
to have a primary partition as the last entry. The desired use case
is to be able to create an A/B update partition image scheme with a
/var volume that can be auto expanded to the remainder of the SD card
at run time.
The typical .wks file will look similar to the following:
bootloader --ptable msdos
part / --source rawcopy --sourceparams="file=u-boot.imx" \
--ondisk mmcblk --no-table --align 1 --size 1
part /boot --source bootimg-partition \
--ondisk mmcblk --fstype=vfat --label boot --active --align 4 --size 16
part / --source rawcopy --sourceparams="file=imx6_boot.otaimg" \
--ondisk mmcblk --fstype=ext4 --label otaboot --align 4 --type logical
part / --source rawcopy --sourceparams="file=imx6.otaimg" \
--ondisk mmcblk --fstype=ext4 --label otaroot --align 4 --type logical
part / --source rawcopy --sourceparams="file=imx6_boot.otaimg" \
--ondisk mmcblk --fstype=ext4 --label otaboot_b --align 4 --type logical
part / --source rawcopy --sourceparams="file=imx6.otaimg" \
--ondisk mmcblk --fstype=ext4 --label otaroot_b --align 4 --type logical
part /var --source rawcopy --sourceparams="file=imx6_var.otaimg" \
--ondisk mmcblk --fstype=ext4 --label fluxdata --align 4
Without the patch applied, wic will generate an SD card image that looks like:
Disk boot.img: 890940s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 2056s 48001s 45946s primary fat16 lba
2 48008s 132467s 84460s primary ext4
3 132472s 454467s 321996s primary ext4
4 454471s 890939s 436469s extended lba
5 454472s 538931s 84460s logical ext4
6 538936s 860931s 321996s logical ext4
7 860936s 890939s 30004s logical ext4 boot
With the patch applied a primary partition can be created at the end
of the image which can be expanded to fill the free space on the media
where the image has been copied, which looks like:
Disk boot.img: 890940s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 2056s 48001s 45946s primary fat16 lba
2 48007s 860931s 812925s extended lba
5 48008s 132467s 84460s logical ext4
6 132472s 454467s 321996s logical ext4
7 454472s 538931s 84460s logical ext4
8 538936s 860931s 321996s logical ext4
3 860936s 890939s 30004s primary ext4 boot
(From OE-Core rev: 56add7cc547e0113cdf980579d1421b14cc233e5)
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a source plugin that support both EFI and legacy PC-Bios.
While using this plugin, both bootloaders configurations reside
in the same /boot partitions.
This plugin has very little code : to avoid code duplication,
we simply re-import bootimg-pcbios and bootmg-efi source and
call both their SourcePlugin methods.
(From OE-Core rev: c8545d54139c6c48bffd1dd1d39d79891626c6f7)
Signed-off-by: William Bourque <wbourque@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For now, the temp dir is left in system, although the temporary
source directory has been cleaned up. So we clean it up too.
(From OE-Core rev: 8a0602327d5afcf4f36850d3f05c9721305852af)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit 450335ba5e73a375eb9932b4c4cf37979640dbfc copies the pseudo
database to the working directory in order to have ownership information
when the filesystem is generated.
Unfortunately this does not work anymore. The filenames on the database
are absolute and there is no information about the new directory.
Instead of fixing the database, we could redo a bit the way we patch the
fstab file. Now I am saving the old contents of fstab, modifying the
file and then reverting the changes on exit.
This is faster than the previous approach, although it can cause
indeterminism if the application is killed before finishing.
(From OE-Core rev: dcbf7b864dd1713b54a172d8714ce1508482f086)
Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
when INITRAMFS_IMAGE_BUNDLE and INITRAMFS_IMAGE are set, wic should
look for kernel with initramfs image bundled.
Include required variable MACHINE, INITRAMFS_IMAGE_BUNDLE,
INITRAMFS_IMAGE, INITRAMFS_LINK_NAME and KERNEL_IMAGETYPE in WICVARS.
No longer require default value for variable kernel as KERNEL_IMAGETYPE
is not optional variable and included in WICVARS.
image_types_wic to inherit kernel-artifact-names to obtain default
INITRAMFS_LINK_NAME when INITRAMFS_IMAGE_BUNDLE are set.
update wic.Wic2.test_image_env test case to filter optional
variable INITRAMFS_LINK_NAME, INITRAMFS_IMAGE and INITRAMFS_IMAGE_BUNDLE.
(From OE-Core rev: bac984fbb2d5ad5d13ba3275c8a3e878d8753c58)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds a command line option to filter out the buildstats-diff report by
one more more tasks. e.g.:
buildstats-diff --only-task do_compile A B
will only show the differences for do_compile tasks. The --only-task
option can be specified multiple times to filter out multiple tasks at
once.
(From OE-Core rev: a8c7960d24c48107fd3703e49c38f890e84e2226)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
All packages that support the menuconfig task will be able to run
devtool menuconfig command. This would allow the user to modify the
current configure options and create a config fragment which can be
added to a recipe using devtool finish.
1. The patch checks if devtool menuconfig command is called for a valid
package.
2. It checks for oe-local-files dir within source and creates one if
needed, this directory is needed to store the final generated config
fragment so that devtool finish can update the recipe.
3. Menuconfig command is called for users to make necessary changes.
After saving the changes, diffconfig command is run to generate the
fragment.
Syntax:
devtool menuconfig <package name>
Ex: devtool menuconfig linux-yocto
The config fragment is saved as devtool-fragment.cfg within
oe-local-files dir.
Ex:
<workspace_path>/sources/linux-yocto/oe-local-files/devtool-fragment.cfg
Run devtool finish to update the recipe by appending the config fragment
to SRC_URI and place a copy of the fragment within the layer where the
recipe resides.
Ex: devtool finish linux-yocto meta
[YOCTO #10416]
(From OE-Core rev: 417feb559a74b367315e8658d6ba868a4f8d1340)
Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com>
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If kernel source is not already downloaded i.e staging kernel dir is
empty, place a copy of the source when the user runs devtool modify
linux-yocto. This way the kernel source is available for other packages
that use it.
[YOCTO #10416]
(From OE-Core rev: bb42ab90835e8ec2f1dfbb35056c353784693266)
Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com>
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the regular devtool modify flow, the kernel source is fetched by
running do_fetch task. This is an overhead in time and space.
This patch updates modify command to check if the kernel source is
already downloaded. If so, then instead of calling do_fetch, copy the
source from work-shared to devtool workspace by creating hard links
else run the usual devtool modify flow and call do_fetch task.
[YOCTO #10416]
(From OE-Core rev: 3c3a9bae296f849dbfe03942282f44036e6fa1fb)
Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com>
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As part of upgrading a recipe we create the upgraded recipe file in the
workspace and then try to parse it so we can then make further
modifications. If for some reason that parsing fails then the failure
was not being handled very well - the broken recipe was being left in
place, breaking parsing until it was removed by hand. Fix that by adding
a call to the cleanup function, and fix the following issues:
* Fix the cleanup function which doesn't look like it has ever worked
due to a typo in the function call
* Fix double-printing the error message
* Remove usage of DevtoolError in this case (DevtoolError is for simple
usage errors, not this kind of issue which may be the result of a
bug).
We're still printing a traceback in this scenario but at least it
doesn't break the build system requiring manual cleanup. I also
introduced a command-line option to preserve the broken upgraded recipe
file(s) for debugging purposes.
(The reproducer for this is "devtool upgrade libnewt-python", however
you need to check out revision b82ea144e1
or earlier since that recipe has now been absorbed into the libnewt
recipe. The libnewt-python recipe was causing an issue with the upgrade
because it actually included the libnewt recipe using ${PV} in the
include statement, and of course PV was changing in the upgrade.)
Fixes [YOCTO #13404].
(From OE-Core rev: c519ac360796675d7fc09a5250d21f0f5b6236fc)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a setup.py file exists it ought to have something in it before we
consider the source tree to be a Python module and treating it as such.
(A counter-example is https://www.bro.org/downloads/binpac-0.50.tar.gz -
it's not clear why this has a zero-length setup.py in it but we should
pay no attention to it.)
Fixes [YOCTO #12923].
(From OE-Core rev: 548a5c8f42c6ac1b0f7962926d05276e71505678)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This switches the code to build pkgdata specific to the current recipe
which means that its filtered to the recipes dependencies and can perform
better as we can drop the lockfile.
It uses a similar method to the staging code to do this, using BB_TASKDEPDATA
to construct a list of packagedata task output which this recipe should "see".
The original pkgdata store is left unaltered so existing code works.
The lock file was there to prevent files disappearing as they were read or as
directories were listed. Since we have a copy of the data and only access output
from completed tasks (as per their manifests), we can remove the lock.
The lock was causing starvation issues on systems with parallelism.
There was also a potential determinism problem as the current code could "see"
data from recipes which it doesn't depend upon.
[YOCTO #13412]
(From OE-Core rev: 1951132576bfb95675b4879287f8b3b7c47524fa)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(From OE-Core rev: 9303f92599343adf645fee5d2434fadd97e7febb)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently `devtool finish RECIPE meta' will silently succeed even
if there are multiple layers having the same base name of 'meta'.
e.g. meta layer from oe-core and meta layer from meta-secure-core.
We should at least give user a warning in such case. With the patch,
we will get warning like below.
WARNING: Multiple layers have the same base name 'meta', use the first one '<PROJ_DIR>/oe-core/meta'.
WARNING: Consider using path instead of base name to specify layer:
<PROJ_DIR>/oe-core/meta
<PROJ_DIR>/meta-secure-core/meta
(From OE-Core rev: 2c8740f543c38dbaef3345e40827ef48b3f75405)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow plugin bootimg-efi to configure with multiple initrd
through source parameter.
Uses ; to separate each initrd.
e.g:
--sourceparams="loader=${EFI_PROVIDER},initrd=initrd1;initrd2"
(From OE-Core rev: c7b0823f9ab6a9959aaa809b8c3f70d199feb64d)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current QA team need to merge test result files from multiple sources.
Adding TESTSERIES configuration too early will have negative
implication to report and regression. Enable control to add TESTSERIES
when needed. Also enable adding EXECUTED_BY configuration when
needed.
(From OE-Core rev: 651d8d371e78e77599d56681228d5782664f7743)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current results stored does not have information needed to trace who
executed the tests. Enable store to add EXECUTED_BY configuration
to results file in order to track who executed the tests.
(From OE-Core rev: dca2a57d54163a2e63b06e2f140fea3bd49cef0d)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current resultutils library always add "TESTSERIES" configuration
to results. Enhance this to allow control of adding "TESTSERIES"
configuration as well as allow adding extra configurations
when needed.
(From OE-Core rev: 443c0acc14ef2451b10878fc83dd11b46805daf0)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
replaced hardcoded kernel image with KERNEL_IMAGETYPE.
set kernel image to "bzImage" incase KERNEL_IMAGETYPE not set.
(From OE-Core rev: 88a9fef761c5e67b2964fedc85a7e8ad37067564)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support for generating python3 recipes using the recipetool / devtool.
Drop python2 support at the same time.
Tested with:
oe-selftest -r recipetool.RecipetoolTest
[YOCTO #13264]
(From OE-Core rev: d8b2f58974482b3b1ccc65c5f93104d0d7ba87bc)
Signed-off-by: Maciej Pijanowski <maciej.pijanowski@3mdeb.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some file systems don't support fetching the block size (notably the file system
Docker uses for containers), so handle the iotctl() failing and raise the
expected error.
(From OE-Core rev: 3757073726a00c5250556aae3d0daac76b88085e)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
allow wic to list and search for kickstart file in .wks.in extension.
basename show by wic list images to fully exclude extension.
(From OE-Core rev: 2c0a292a790ad069648e37b1b29fcea656fcf3e4)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Printing the lack of a test is not necessary (per feedback). Remove
this from the template to quieten it.
(From OE-Core rev: b1fe6ae66360e160eeaeafe456536f335a0eab60)
Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow the creation of test case configuration file based on user inputs.
Where this testcase configuration file will be used by the the manual
execution to run selected test cases for a module rather than compulsory
run all test cases in manual json file.
(From OE-Core rev: 73d2a747c17779da0ca972da776b3cf02c2e1cbc)
Signed-off-by: sangeeta jain <sangeeta.jain@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, the code will sum all of the different machine results into a
single report of the tests results. This can lead to confusion as to
which machine may be experiencing issues. Modify the code to store the
results in a per machine basis and report them accordingly.
(From OE-Core rev: 16d4031ea5df8a4ddfdb937d35464c09e1abd10e)
Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add new source parameter label to allow custom boot.conf/grub.cfg label,
so far it's hardcoded to "Boot".
Default label to "Boot" for systemd-boot and blank for grub-efi when source
parameter label are not set.
(From OE-Core rev: 7a0aab1aa31e66e6bc94c04c2f6c1043b64a8967)
Signed-off-by: Chee Yang Lee <chee.yang.lee at intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
volume name should refer to --label in .wks.
Replace the hardcoded volume name with label.
set "ESP" as default name when no lable specified.
(From OE-Core rev: 5621aceaf39ef0dc097b16c83e73b9882c987a7c)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds SPDX license headers in place of the wide assortment of things
currently in our script headers. We default to GPL-2.0-only except for the
oeqa code where it was clearly submitted and marked as MIT on the most part
or some scripts which had the "or later" GPL versioning.
The patch also drops other obsolete bits of file headers where they were
encoountered such as editor modelines, obsolete maintainer information or
the phrase "All rights reserved" which is now obsolete and not required in
copyright headers (in this case its actually confusing for licensing as all
rights were not reserved).
More work is needed for OE-Core but this takes care of the bulk of the scripts
and meta/lib directories.
The top level LICENSE files are tweaked to match the new structure and the
SPDX naming.
(From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds an option to dump all the ptest logs to individual files in a
specified directory. If multiple test runs are present, the
'--prepend-run' argument will create separate directories for each test
run under the target directory and put the logs there to prevent each
test run from clobbering the others.
[YOCTO #13331]
(From OE-Core rev: f262b9e7482e38f04dfa162f0aaf65b17e86f8b2)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If attempting to find a fixed SRCREV fails because the directory doesn't exit,
avoid failing with:
Traceback (most recent call last):
File "/home/vagrant/poky/scripts/recipetool", line 121, in <module>
ret = main()
File "/home/vagrant/poky/scripts/recipetool", line 110, in main
ret = args.func(args)
File "/home/vagrant/poky/scripts/lib/recipetool/create.py", line 707, in create_recipe
srcrev = stdout.rstrip()
UnboundLocalError: local variable 'stdout' referenced before assignment
Fixes: 000480c42797 ("recipetool / devtool: set a fixed SRCREV by default when fetching from git")
(From OE-Core rev: 8e4b301573071ea04911b0df6533c25bc4596b93)
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current manualexecution required user to exceute all test cases defined inside a "modulename.json" file in oeqa/manual
There are cases when all test cases all not required to run for a module on specific DUT.
Enable manualexecution to have the optional feature where it will use pre-defined json format test case configuration file
where user will be able to select test cases from the "modulename.json" instead of running all of them. This will help
in reducing testing time and reporting unneccesary skip or failures.
Example pre-defined json format test case configuration file (for build-applince):
{
"testcases" : [
"build-appliance.build-appliance.Create_core-image-sato-sdk_using_build_appliance",
"build-appliance.build-appliance.Build_a_image_without_error_(added_recipe)"
]
}
(From OE-Core rev: 3950c28a34f94b5907d37b579bdaee5a59794652)
Signed-off-by: sangeeta jain <sangeeta.jain@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit f7dfb4d43247d3c13a4e0a3853007d63b9512b83.
FAT16 volumes do not have a minimum size of 8250 blocks
(== slightly over 4MB). Exhibit A: floppy disks.
The original commit message suggests this was a workaround for a bug in
parted - in which case we should fix it there, or use the wic
`--fixed-size` option to pass down an enforced minimum from a more
context-aware point in the callstack.
(From OE-Core rev: e0a1ae3863bc229512d43e4a5248551bf25950e5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix AttributeError: module 'urllib' has no attribute 'request' when
using remote http urls.
(From OE-Core rev: 9acc47236e2a9ac7f97f80040f5e508f23f01611)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds a subcommand for dumping various logs from test results
(From OE-Core rev: 454b8d2cdc6f79a51e610dae92c22352850c3f7c)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds support for resulttool to load JSON files directly from a http://
or https:// URL
(From OE-Core rev: 235bcf0c504e7ea253ccfb28d600898117c64c1f)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove duplicate codes. Replace unnecessary class variables with
local variables. Rename variables and arguments with simple and
standard name.
(From OE-Core rev: 6bef61d36f3328fad003c0dc9c010d3f76ba96d8)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When I use recipetool to add a new recipe, it says:
[snip]
sys.exit(14)
NameError: name 'sys' is not defined
[snip]
(From OE-Core rev: 051e1c27bbf6a5898f2cf13b78dd3d8d1ec31586)
Signed-off-by: Lei Yang <Lei.Yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow the creation of configuration option file based on user inputs.
Where this configuration option file will be used by the the manual
execution to display options for configuration rather than user
need to inputs configuration manually.
(From OE-Core rev: 23c7a037a85a825aa088bfac3a144c14087a9f49)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current manualexecution required user to input configuration manually
where there were inconsistent inputs and human typo issues.
Enable manualexecution to have the optional feature where it
will use pre-compiled configuration options file where user will
be able to select configuration from the pre-compiled list instead
of manual key-in the configuration. This will eliminate human error.
Expect the pre-compiled configuration options file in json format below
{
"bsps-hw": {
"IMAGE_BASENAME": {
"1": "core-image-sato-sdk"
},
"MACHINE": {
"1": "beaglebone-yocto",
"2": "edgerouter",
"3": "mpc8315e-rdb",
"4": "genericx86",
"5": "genericx86-64"
}
},
"bsps-qemu": {
"IMAGE_BASENAME": {
"1": "core-image-sato-sdk"
},
"MACHINE": {
"1": "qemuarm",
"2": "qemuarm64",
"3": "qemumips",
"4": "qemumips64",
"5": "qemuppc",
"6": "qemux86",
"7": "qemux86-64"
}
}
}
(From OE-Core rev: 1af7e04223b415a64e100a9f91d60f5e9b0f789a)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The cml1 and ccmake bbclasses generate configuration fragment source
files that must be exported from the WORKDIR as a source file to be
preserved across builds. This change adds detection of the current
recipes inherited classes and for cml1 and ccmake classes checks for the
specific generated configuration fragment files. These files are then
exported by devtool and included as SRC_URI files from within the target
layer.
(From OE-Core rev: 7a1fabe4a24552ce8a50c8d6009969ca52a5fd27)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the manual execution display step by sorting
the step as string, where steps were not being sorted
correctly when there are more than 9 steps.
Fixed the step sorting by sorting step as integer.
(From OE-Core rev: 192c255d9f597b3526a5b94fcf3eedc4527189ed)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current manualexecution required pressing enter button to show each step
information, where this was wasting execution time. Enable display
full steps without needing to any press enter button.
(From OE-Core rev: e44c9a018c13208fef0dcef4479ce71977628cd4)
Signed-off-by: Mazliana <mazliana.mohamad@intel.com>
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current input checking does not match the standard input practiced
by QA team. Change the input checking to match the standard
input practiced by the QA team.
(From OE-Core rev: 630a93c740359a65249a198f314f15040042f8d3)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than simply discarding the ptest data, change the code to discard
it when writing out the new testresult files, or optionally either preserve
it, or write it as seperate discrete logs.
This means the autobuilder should start writing out individual ptest log
files as well as allowing ueers to extract these manually.
(From OE-Core rev: a1e0944bf260ef50dd7dfcb10db248fdd7f45bc9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Store operations using a single file as a source weren't working as the os.walk
command didn't like being given a single file. Fix the store operation to
work for single files.
(From OE-Core rev: eb5bbe613d8c7cbcd8b74d8ac0073c2217970410)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
On some hosts the wic.Wic2.test_wic_cp_ext selftest was failing as files weren't
being copied into the rootfs with "wic cp". This was due to a bug added by:
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=89d4a8df074598cfb3a76e41db7c45d845afd961
where there should be a second newline added at the end of the expression due
to the difference in the way echo -e and printf behave.
[YOCTO #13237]
(From OE-Core rev: 2f3adb21cc5067458a12964d0bde235966a20a60)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Devtool creates a git repository for extracted sources in a temporary
directory and then moves it to a final destination after patching is
done. Unfortunately devtool is not aware that some of its git operations
may have caused git garbage collector to start in background. If timing
is just right a repository move fails because GC is removing git objects
while they are being moved.
The issue was hit on Krogoth release, but the code that moves created
git repository is still the same.
Disable gc.autodetach to make GC run synchronously and block git until
it is done.
Traceback (most recent call last):
File "<snip>/openembedded-core/scripts/devtool", line 342, in <module>
ret = main()
File "<snip>/openembedded-core/scripts/devtool", line 329, in main
ret = args.func(args, config, basepath, workspace)
File "<snip>/openembedded-core/scripts/lib/devtool/standard.py", line 352, in extract
initial_rev = _extract_source(srctree, args.keep_temp, args.branch, False, rd)
File "<snip>/openembedded-core/scripts/lib/devtool/standard.py", line 644, in _extract_source
shutil.move(srcsubdir, srctree)
File "/usr/lib/python2.7/shutil.py", line 299, in move
copytree(src, real_dst, symlinks=True)
File "/usr/lib/python2.7/shutil.py", line 208, in copytree
raise Error, errors
Error: [('/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/5e', '<snip>/build/grpc/grpc_src/.git/objects/5e', "[Errno 2] No such file or directory: '/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/5e'"), ('/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/57', '<snip>/build/grpc/grpc_src/.git/objects/57', "[Errno 2] No such file or directory: '/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/57'"), many git objects ... ]
(From OE-Core rev: 3da578a79195a63fb6d4bbd68d8f1548defd205f)
Signed-off-by: Taras Kondratiuk <takondra@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We found that manualexecution does not capture test suite values
correctly if there are more than one test suite in test cases.
After verification has made we found out we should retrieved
full test cases value <test_module.test_suite.test_case> from
oeqa/manual/ json file rather than split it them into new
variables test_suite and test_cases.
(From OE-Core rev: db08ec5f48badd5be2fa24cc0d89a4386ab44a59)
Signed-off-by: Mazliana <mazliana.mohamad@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enable roll-up all test results belong to a commit
and to provide a roll-up report.
(From OE-Core rev: a98a75a3db1fe9d327fd21919fb8ea09e76fb1e3)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
do_patch rule of SDK's workspace/appends/linux-*.bbhappend may fail if script are not written in Python
that was the case with Phytec's BSP, the fix was to replace the do_patch rule with :
do_patch[noexec]="1" when the file was generated in scripts/lib/devtool/standard.py
(From OE-Core rev: e51cb385e5573c7069de6e8308b9b68eb4a3a162)
Signed-off-by: Yann CARDAILLAC <yann.cardaillac@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To enable store for testresults.json file from manualexecution,
add layers metadata to configuration and add "manual" map to
resultutils.store_map.
To enable regression for manual, add "manual" map to
resultutils.regression_map. Also added compulsory configurations
('MACHINE', 'IMAGE_BASENAME') to manualexecution.
(From OE-Core rev: 7521834f8557565d3898605257c7fd9505ea22bb)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than recreating handlers and forcing them, iterate over the handlers
and enable color on ones we can handle. This makes it easier to handle color
properly when we introduce the bb.msg default log filters.
(From OE-Core rev: 713f44e0e0cde9b818c214002fd8b730d422fafa)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sorted regression results to provide friendly viewing of report.
(From OE-Core rev: 59d827889dd8b790a7c9323c69565d6bba3041bd)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
ptest suites with no results don't show up on the reports even though we have
a duration for them. Fix this so the fact they report no tests is visible.
(From OE-Core rev: 58268151704246a81ec8dae46c26346023057554)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently some older results files cause the code to give tracebacks.
Handle these missing sections more cleanly.
(From OE-Core rev: f778c191dbd5740173b3be07f4c1655d85a07bb2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently we cant store results if the results files span multiple
different build revisons. Remove this limitation by iterating.
(From OE-Core rev: 4da12c00963b02508056b87ce9b972528ce3a1be)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow the use of an alternate SSH executable for deploy-target and
undeploy-target. The command can be specified with -e / --ssh-exec.
(From OE-Core rev: c2684740d7fd3aeb60b3ec7888000097effe82f4)
Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a combined patch of the various tweaks and improvements I
made to resulttool:
* Avoid subprocess.run() as its a python 3.6 feature and we
have autobuilder workers with 3.5.
* Avoid python keywords as variable names
* Simplify dict accesses using .get()
* Rename resultsutils -> resultutils to match the resultstool ->
resulttool rename
* Formalised the handling of "file_name" to "TESTSERIES" which the code
will now add into the json configuration data if its not present, based
on the directory name.
* When we don't have failed test cases, print something saying so
instead of an empty table
* Tweak the table headers in the report to be more readable (reference
"Test Series" instead if file_id and ID instead of results_id)
* Improve/simplify the max string length handling
* Merge the counts and percentage data into one table in the report
since printing two reports of the same data confuses the user
* Removed the confusing header in the regression report
* Show matches, then regressions, then unmatched runs in the regression
report, also remove chatting unneeded output
* Try harder to "pair" up matching configurations to reduce noise in
the regressions report
* Abstracted the "mapping" table concept used to pairing in the
regression code to general code in resultutils
* Created multiple mappings for results analysis, results storage and
'flattening' results data in a merge
* Simplify the merge command to take a source and a destination,
letting the destination be a directory or a file, removing the need for
an output directory parameter
* Add the 'IMAGE_PKGTYPE' and 'DISTRO' config options to the regression
mappings
* Have the store command place the testresults files in a layout from
the mapping, making commits into the git repo for results storage more
useful for simple comparison purposes
* Set the oe-git-archive tag format appropriately for oeqa results
storage (and simplify the commit messages closer to their defaults)
* Fix oe-git-archive to use the commit/branch data from the results file
* Cleaned up the command option help to match other changes
* Follow the model of git branch/tag processing used by oe-build-perf-report
and use that to read the data using git show to avoid branch change
* Add ptest summary to the report command
* Update the tests to match the above changes
(From OE-Core rev: ff2c029b568f70aa9960dde04ddd207829812ea0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Integrated “manualexecution” operation to resulttool scripts.
Manual execution script is a helper script to execute all manual
test cases in baseline command, which consists of user guideline
steps and the expected results. The last step will ask user to
provide their input to execute result. The input options are
passed/failed/blocked/skipped status. The result given will be
written in testresults.json including log error from the user
input and configuration if there is any.The output test result
for json file is created by using OEQA library.
The configuration part is manually key-in by the user. The system
allow user to specify how many configuration they want to add and
they need to define the required configuration name and value pair.
In QA perspective, "configuration" means the test environments and
parameters used during QA setup before testing can be carry out.
Example of configurations: image used for boot up, host machine
distro used, poky configurations, etc.
The purpose of adding the configuration is to standardize the
output test result format between automation and manual execution.
To use these scripts, first source oe environment, then run the
entry point script to look for help.
$ resulttool
To execute manual test cases, execute the below
$ resulttool manualexecution <manualjsonfile>
By default testresults.json store in <build_dir>/tmp/log/manual/
[YOCTO #12651]
(From OE-Core rev: f24dc9e87085a8fe5410feee10c7a3591fe9d816)
Signed-off-by: Mazliana <mazliana.mohamad@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
OEQA outputs test results into json files and these files were
archived by Autobuilder during QA releases. Example: each oe-selftest
run by Autobuilder for different host distro generate a
testresults.json file.
These scripts were developed as a test result tools to manage
these testresults.json file.
Using the "store" operation, user can store multiple testresults.json
files as well as the pre-configured directories used to hold those files.
Using the "merge" operation, user can merge multiple testresults.json
files to a target file.
Using the "report" operation, user can view the test result summary
for all available testresults.json files inside a ordinary directory
or a git repository.
Using the "regression-file" operation, user can perform regression
analysis on testresults.json files specified. Using the "regression-dir"
and "regression-git" operations, user can perform regression analysis
on directory and git accordingly.
These resulttool operations expect the testresults.json file to use
the json format below.
{
"<testresult_1>": {
"configuration": {
"<config_name_1>": "<config_value_1>",
"<config_name_2>": "<config_value_2>",
...
"<config_name_n>": "<config_value_n>",
},
"result": {
"<testcase_namespace_1>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
"<testcase_namespace_2>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
...
"<testcase_namespace_n>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
}
},
...
"<testresult_n>": {
"configuration": {
"<config_name_1>": "<config_value_1>",
"<config_name_2>": "<config_value_2>",
...
"<config_name_n>": "<config_value_n>",
},
"result": {
"<testcase_namespace_1>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
"<testcase_namespace_2>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
...
"<testcase_namespace_n>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
}
},
}
To use these scripts, first source oe environment, then run the
entry point script to look for help.
$ resulttool
To store test result from oeqa automated tests, execute the below
$ resulttool store <source_dir> <git_branch>
To merge multiple testresults.json files, execute the below
$ resulttool merge <base_result_file> <target_result_file>
To report test report, execute the below
$ resulttool report <source_dir>
To perform regression file analysis, execute the below
$ resulttool regression-file <base_result_file> <target_result_file>
To perform regression dir analysis, execute the below
$ resulttool regression-dir <base_result_dir> <target_result_dir>
To perform regression git analysis, execute the below
$ resulttool regression-git <source_dir> <base_branch> <target_branch>
[YOCTO# 13012]
[YOCTO# 12654]
(From OE-Core rev: 78a322d7be402a5b9b5abf26ad35670a8535408a)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Just stating the recipe counts are different isn't helpful, showing
the differences makes it much easier to understand what changed.
(From OE-Core rev: 1d84b782e3af6f0e6922d72895c905877cc33739)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
OE-Core rev: a88bcbae850a2e6d182291d3f8e167aabdbe4842 broke the ability
to find parted as it may be in sbin which is not in PATH for some users
on some distros.
Iterate on the original patch to fix this and also fix the original problem.
(From OE-Core rev: af3803e5189d7814f9dbd238fb6dab200f351e1a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Load self.paths from environment variable and
if it fails, fall back to hardcoded list.
This is required for users that would need to load
different e2fsprogs binaries if their system's ones
are not recent enought
(From OE-Core rev: a88bcbae850a2e6d182291d3f8e167aabdbe4842)
Signed-off-by: William Bourque <wbourque@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python subprocess' shell=True defaults to /bin/sh[1][2], which
often refers to a POSIX-compliant shell. As the -e flag is
not defined in the POSIX standard[3], some shells may
interpret "-e" as the first argument to the echo command,
causing the rest of the command line to fail.
In this particular case, "echo -e 'cd {}'" is interpreted
as "-e cd {}", which causes the first line of the command
to fail, and causing cp to always place the source file
in the filesystem's root.
Replacing "echo -e" for a printf command makes this
more portable.
This issue only affects "wic cp" for ext* filesystems.
[1] https://docs.python.org/2/library/subprocess.html
[2] https://docs.python.org/3/library/subprocess.html
[3] http://pubs.opengroup.org/onlinepubs/9699919799
(From OE-Core rev: 9a5b703d4e60c3ccdf80b5289b8e6fc91133fcde)
Signed-off-by: Leonardo Augusto <leobsd@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Rename add_layer() to add_layers() so that add_layer_dependencies() can
re-use it.
* Avoid adding the layer if it is already present
[YOCTO #13148]
(From OE-Core rev: b9cc18d83f55ff48c3d6e60c56359f6736d5a06a)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The default behaviour is to show just recipes needing upgrades, but for
automated reports showing all recipes can be useful.
(From OE-Core rev: 29e064ac6bd8d1ef74c280bf85bf8428c37ba014)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The check_commits logic assumes that both devtool-base and args.branch
exist in the git repo that it is operating on. In order to prevent
errors at that point it's best to first ensure that both of these refs
actually exist. If they don't both exist then the check_commits logic
should just be skipped, as it would be if the repo wasn't originally
checked out by devtool.
Previously if a user removed the args.branch branch from their devtool
cloned repo this code would crash on adding the repo with -n. The crash
would look like this:
Traceback (most recent call last):
File "/home/ddedrick/src/poky/scripts/devtool", line 344, in <module>
ret = main()
File "/home/ddedrick/src/poky/scripts/devtool", line 331, in main
ret = args.func(args, config, basepath, workspace)
File "/home/ddedrick/src/poky/scripts/lib/devtool/standard.py", line 812, in modify
(stdout, _) = bb.process.run('git log devtool-base..%s' % branch, cwd=srctree)
File "/home/ddedrick/src/poky/bitbake/lib/bb/process.py", line 178, in run
raise ExecutionError(cmd, pipe.returncode, stdout, stderr)
bb.process.ExecutionError: Execution of 'git log devtool-base..devtool' failed with exit code 128:
fatal: ambiguous argument 'devtool-base..devtool': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
(From OE-Core rev: f13a3490fdb404bbd4c77e45b83540d6deec1358)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
DEVTOOL_EXTRA_OVERRIDES only needs one entry for each instance of
overrides. Previous to these changes it would find every override to
SRC_URI and add it to the list. This would duplicate instances where
SRC_URI is modified multiple times with the same override like:
SRC_URI_append_foo += "file://0001-foo.patch"
SRC_URI_append_foo += "file://0002-bar.patch"
A bbappend might also overwrite a SRC_URI override, which would also
cause multiple instances to occur.
When there are multiple instances of the same override in
DEVTOOL_EXTRA_OVERRIDES it causes devtool modify to fail when creating
override branches. The failure occurs when attempting to create the same
override branch a second time and looks like this:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:devtool_post_patch(d)
0003:
File: '/build/poky/meta/classes/devtool-source.bbclass', lineno: 202, function: devtool_post_patch
0198:
0199: for override in extra_override_list:
0200: localdata = bb.data.createCopy(d)
0201: if override in default_overrides:
*** 0202: bb.process.run('git branch devtool-override-%s %s' % (override, devbranch), cwd=srcsubdir)
0203: else:
0204: # Reset back to the initial commit on a new branch
0205: bb.process.run('git checkout %s -b devtool-override-%s' % (initial_rev, override), cwd=srcsubdir)
0206: # Run do_patch function with the override applied
File: '/build/poky/bitbake/lib/bb/process.py', lineno: 178, function: run
0174: if not stderr is None:
0175: stderr = stderr.decode("utf-8")
0176:
0177: if pipe.returncode != 0:
*** 0178: raise ExecutionError(cmd, pipe.returncode, stdout, stderr)
0179: return stdout, stderr
Exception: bb.process.ExecutionError: Execution of 'git branch devtool-override-foo devtool' failed with exit code 128:
fatal: A branch named 'devtool-override-foo' already exists.
(From OE-Core rev: 90f667db2219f04e6d61588cd61056d3d8da6d7d)
Signed-off-by: Dan Dedrick <ddedrick@lexmark.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Apparently either nobody ever used sdimage-bootpart.wks
or fstab was kept untouched due to "--no-fstab-update" usage as
some boards like IMX may have, see [1].
Otherwise addition of the following line to the target's fstab:
---------------------->8-----------------
/dev/mmcblkp1 /boot vfat defaults 0 0
---------------------->8-----------------
gets us unusable target:
---------------------->8-----------------
[* ] A start job is running for dev-mmcblkp1.device (23s / 1min 30s)
[ TIME ] Timed out waiting for device dev-mmcblkp1.device.
[DEPEND] Dependency failed for /boot.
[DEPEND] Dependency failed for Local File Systems.
You are in emergency mode. After logging in, type "journalctl -xb" to view
system logs, "systemctl reboot" to reboot, "systemctl default" or "exit"
to boot into default mode.
Press Enter for maintenance
---------------------->8-----------------
Fix it with use of "mmcblk0" device name.
[1] https://lists.yoctoproject.org/pipermail/meta-freescale/2018-February/022156.html
(From OE-Core rev: 57a925dbd949e488980b02896106efd82bf47ed2)
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Cc: Maciej Borzecki <maciej.borzecki@open-rnd.pl>
Cc: Maciek Borzecki <maciek.borzecki@gmail.com>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
image_types_wic.bbclass has a mechanism for doing variable substitution
on .wks files by simply letting the input file be called
.wks.in. However, that doesn't allow using variables in files included
via the include directive. This is unfortunate, because lacking either
the ability to include other files or variable substitution leads to
fragile and error-prone duplication between kickstarter files and
recipes/configuration files used for various boards.
This adds (somewhat naive) support for variable substitution in all
files parsed by wic. The user should add all required variables to
WICVARS to get them exported appropriately.
(From OE-Core rev: 8a75d614a8a1ff72c4af448ac3e1292d0e1d1a79)
Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a fixup for: e1ba46109e
devtool deploy-target --strip foo root@192.168.7.2
ended up with:
Traceback (most recent call last):
File "/home/user/bar_sdk/sysroots/x86_64-pokysdk-linux/usr/bin/
devtool", line 344, in <module>
ret = main()
File "/home/user/bar_sdk/sysroots/x86_64-pokysdk-linux/usr/bin/
devtool", line 331, in main
ret = args.func(args, config, basepath, workspace)
File "/home/user/bar_sdk/layers/poky/scripts/lib/devtool/
deploy.py", line 187, in deploy
rd.getVar('base_libdir'))
TypeError: strip_exbar() missing 1 required positional argument: 'd'
(From OE-Core rev: bc349b2d99d74f6d7fb590fd97750e5ff8d6e6c4)
Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The imp module is deprecated, port the code over to use importlib
as recently done for bb.utils as well.
(From OE-Core rev: 24809582d4850190d87cd8eb8180d0dce215dbdf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes the users might want to change the title showing on UEFI
booting screen, so far it's hard-coded to 'boot'.
There is not a easy way to customize it in current design, I tried
firstly with '--configfile', but that does not work with --use-uuid,
since the later option will generate a UUID and write it to boot
config, only when the former option is not enabled.
So a new source parameter 'titile' is added in this patch, it defaults
to 'boot' to be consistent with the original title.
(From OE-Core rev: 37e16188ef3b1b328eb18b3e459c051c9c9f0332)
Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
yocto-check-layer will find locked-sigs.inc under builddir,
but locked-sigs.inc is generated under current bitbake working
dir. if run yocto-check-layer outside builddir, we will met error
like "No such file or directory: *locked-sigs.inc". change to
run bitbake -S under builddir to fix this problem.
[YOCTO #12973]
(From OE-Core rev: 2ec0bc0b038bc6413978c8f34ef5c0d22b4bc3e7)
Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
os.errno doesn't work in Python 3.7 and shouldn't have ever worked, so use
import errno explicitly.
(From OE-Core rev: 82f9157e84dcaf0ad4292053b09be68c2290d197)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It fails to build multilib image such as lib32-core-image-minimal with
wic by set 'IMAGE_FSTYPES = "wic"':
| ERROR: Couldn't find correct bootimg_dir, exiting
When multilib is enabled, STAGING_DATADIR is expanded with MLPREFIX. But
dependencies of images such as syslinux is still populated to nonarch
STAGING_DATADIR.
Search nonarch STAGING_DATADIR to fix the error.
(From OE-Core rev: dbae9a6f9a1c6cc7d4dd680d7bbda3dfa40f3491)
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Running wic commands on Debian 10 systems fail in
scripts/lib/wic/engine.py:get_partitions() due to new stderr output captured
when trying to parse the output from /sbin/parted as a non-root user.
The parted command calls the dmidecode utility, which produces this error
as a non-root user:
/sys/firmware/dmi/tables/smbios_entry_point: Permission denied
/dev/mem: Permission denied
scripts/lib/wic/engine.py:get_partitions() calls misc.py:exec_cmd(),
a subprocess wrapper which returns a combined stderr and sdtdout.
These messages to stderr confuse the partition table parser in
get_partitions().
This patch has the partition table parser ignore lines before the expected
"BYT;" header string.
Running wic in Debian 9 does not have this issue.
(From OE-Core rev: d6936301d7598b7a783beaae95109555faf6cc17)
Signed-off-by: Geoff Parker <geoffhp@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(From OE-Core rev: f24c3538a27388d282a94e2d8a70cd9108d0919c)
Signed-off-by: Michael Davis <michael.davis@essvote.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove usage of a specific amount of memory and let it be controlled by
users. This was the default behaviour before it was changed by commit
3b79d9a78 that switched the wks file to be used for qemux86.
Also fixes the bitbake parsing issues seen because of memory starvation
using build appliance images.
Fixes [YOCTO #12894]
(From OE-Core rev: 18d6b668c52dc881cff7b107420e0de527eecce4)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Because the find | cpio processes execute in parallel connected via
the pipe, and the cpio outputs in the same dir find searches for
source files, the cpio will be included in itself partially, depending
on how fast the build machine creates the cpio file before cpio
gobbles it up.
This bloats the ISO image, though compression reduces the .iso file size,
once the kernel decompresses the cpio image and boots it live, it uses
up to double the RAM memory.
Fix this by creating the initrd.cpio file directly inside cr_workdir.
(From OE-Core rev: 7a7b29f5f1392f4f87af4f20cda99c66b4789fde)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We can use this parameter to make the wic use the label to name a
partition in /etc/fstab.
(From OE-Core rev: 51638edaa00befaed58e2def255d46ae44d9234f)
Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wic allows the user to create disk image and reference each created
partitions via its partition UUID.
squashfs does not support filesystem UUID but it does offer support for
partition UUID.
$ lsblk -o NAME,FSTYPE,UUID,PARTUUID
NAME FSTYPE UUID PARTUUID
sda
|-sda1 vfat 6078-ACD0 95aa4ffb-741d-4d99-adf6-0358bc890046
|-sda2 squashfs 7d46da2d-29cc-4d5a-9e39-d0381c4abb0c
|-sda3 squashfs d4a3e24e-410a-4f2c-8ad5-8f8321de177b
|-sda4 ext4 d72ec529-78e4-4cd3-b39b-781b6b1cb47c f3edc990-f4f6-4536-9587-44dae4ceb99f
`-sda5 ext4 a0b265aa-e935-40b8-9c6b-e9e58ad3890f 6e72ebaa-29ed-41cd-844c-3f76976cf41d
Modifying this constrain allows a user to create squashfs-based rootfs and boot
from them with such kernel argument 'root=PARTUUID=7d46da2d-29cc-4d5a-9e39-d0381c4abb0c'
Fixes: 2fbdcf4e59c8 ("wic: kparser.py: Check for SquashFS and use-uuid")
(From OE-Core rev: a8787bbfdf088b0ef9feb8e6b4230b743bb4086f)
Signed-off-by: Emmanuel Roullit <emmanuel.roullit@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add -c (--clean) optiont to 'devtool build' command so that users
could easily clean things up when using devtool.
I encountered a problem about do_prepare_recipe_sysroot failure
when using `devtool build' command and I found myself in a situation
where I either have to use `bitbake' command to clean things up or
use `rm' to remove the directories under ${WORKDIR}.
So add a clean option as it would be helpful when users want to clean
things up to prepare an environment for a clean build.
(From OE-Core rev: 29d790cdeff19e520a35ec5902d6deaae8665492)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
initrd.cpio is created in initrd_dir, not in cr_workdir. Gzip will
let initrd.cpio uncompressed if the path is not found.
Also, grub_image variable doesn't exist, grub_src_image should
be used instead.
(From OE-Core rev: c88875b568a66297b1d3df380c590f20796c57f6)
Signed-off-by: Alexandru Vasiu <alexandru.vasiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Only bsdcpio works with numbers for option -R to specify user:group,
while GNU cpio doesn't. Debian use GNU cpio so without this change,
you cannot create ISO images without installing bsdcpio.
(From OE-Core rev: 7c6ba368006ca94ecc0daac7b1c5e0fbe4236daf)
Signed-off-by: Alexandru Vasiu <alexandru.vasiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Automatically select one kernel type image based on a predefined
precedence list if there are multi kernel images installed.
(From OE-Core rev: d1d80566681d4cdc00aa3d4b5e4bcf5edb7132b7)
Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support to specify a custom extlinux.conf via something like:
bootloader --configfile="extlinux.conf"
(From OE-Core rev: 33f85c8bf80d70f00eeccd9ab3dfa417c0fc7df1)
Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By leveraging the distro boot command feature in the u-boot, we can
compose the corresponding extlinux.conf when creating the wic image,
and let u-boot boot the kernel automatically. For more detail about
the u-boot distro boot command feature, please see doc/README.distro
in u-boot source files.
(From OE-Core rev: bdf8ae540af12ecc9ad60efd3651b0f71d12d3bd)
Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We want to add some u-boot specific config file. Before doing this,
we need know what files will be installed into this partition. So
move the codes about parsing the IMAGE_BOOT_FILES into
do_configure_partition(). No function change.
(From OE-Core rev: 3203037471c761f635d1f1c512cb623ff6977a41)
Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We choose to pass the source parameters to the source plugins' hooks
via a local variable srcparams_dict. So the Partition.sourceparams_dict
is not used by anyone and seems pretty confused. So drop it.
(From OE-Core rev: d7130a7d0e61021f71fec1d7ae7d37ebb27f0e03)
Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Images that rely on dhcp being used won't have network setup properly
otherwise.
Fixes [YOCTO #12804]
(From OE-Core rev: fb4dd04f65ea78890c1102424765056d93e6ca65)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When trying to delete something from an ext partition using debugfs, we
don't show any error to the user when that operation fails.
Change this behavior to show the error generated by debugfs. Also,
fallback to use rmdir in case we are trying to delete a directory. However,
unlike mdeltree that is used for a FAT partition, there's no easy way to
delete a non empty directory. Show an error instead when that happens so
user can take appropriate action.
(From OE-Core rev: a405383e63c35d7b56108f192ca74755b122a639)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The warn method is deprecated. We should use the documented warning instead.
Quoting from the python's official doc:
"""
Note: There is an obsolete method warn which is functionally identical to warning.
As warn is deprecated, please do not use it - use warning instead.
"""
(From OE-Core rev: f467fd277eb77336097cfc0f5f329bdc8d0f70cb)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently we just divide up the free space by the number of partitions
that need to be re-sized. This leads to problems when a user has
explicitly specified a subset of partitions (but not all) that need
to re-sized along with the sizes. As an example, for an image with 3
partitions, if we use:
wic write image.wic /dev/sdb --expand 1:10G
This would lead to paritions 2 and 3 each being re-sized to one thirds
of the free space instead of half.
Change the behavior to use up all the free space.
(From OE-Core rev: a88f1b5d88dbc5fb28be24b9787d73b9e0cdf183)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The UEFI spec implies that GPT partitions should be assumed to be on a 2048
sector boundary (for a 512 byte sector) and the current logic just
divides the free sectors available by the number of partitions that need
re-sizing, which may or may not align and the final result might
overshoot the limits imposed after alignment.
Since we are expanding already aligned partitions, just divide up the
free space in multiples of 2048. Also use the exec_cmd wrapper instead
of the subprocess call directly.
Fixes [YOCTO #12840]
(From OE-Core rev: 5eef63f5afdfbab8e30748cb1bf42bf2e6524759)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This edits the recipe and any bbappends for the specified target.
(From OE-Core rev: 7aece42c6b4744c54a8eb05ff90bd3bf4fbb14a3)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's no reason to have that rootfs.img filesystem in the image:
it's not used for anything because both the EFI and legacy boot paths
use the /initrd which contains the same contents as the rootfs.img,
only compressed. It was probably forgotten in there :)
My iso went down from 224 to 94 mb.
Tested using UEFI/legacy boots on CD-roms, usb dongle and qemu VM's.
(From OE-Core rev: 4c784379524cb12807ef87a02ef1514ed45c1cc3)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's really good that OE supports multiple EFI_PROVIDERs and that
commit 9a1709278de87 ("wic: isoimage-isohybrid: use grub-efi from
deploy dir") makes re-use of the grub-efi built image, but we should
still respect the standard otherwise the ISO will not boot, so install
grub images as boot[x64|ia32].efi not ${PN}-boot[x64|ia32].efi.
(From OE-Core rev: 1608129692d92c239b5fb9244b649a32b9009254)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than trying to construct a string by quoting the files in an
error-prone way, parse $EDITOR to pass a list to subprocess rather than
a string.
(From OE-Core rev: c9fdf3d046606a0becb2e6b566a481c483b9021a)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make sure that we're able to use the initrd value specified in the wks
file when using grub-efi bootloader with bootimg-efi wic plugin.
Fixes [YOCTO #12689]
(From OE-Core rev: f1d5bb1f21e11714357aff4ae7e52421e3756c5a)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of showing that command 'None' was found, show the name of
actual command that wasn't found on the host machine or at the native
sysroot path provided by user.
(From OE-Core rev: 3157ebb131e01e74689b1f87a35278350f315e7d)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Let this be dictated by the module parameter value being set by qemu
machine configuration.
(From OE-Core rev: c40b241e8d40c8bc1c9e6065b12e260662f5bba4)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's possible that downloading file from updateserver fails. In
this case, we should error out instead of continue.
We have users reporting unexpected behavior of 'devtool sdk-update'.
When an invalid url is supplied, e.g., `devtool sdk-update http://invalid',
the program reports 'Note: Already up-to-date'.
This is obviously not expected. We should error out in such case.
(From OE-Core rev: 449564783dfb162536a2f772b3a8704973221e0f)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the relative difference is greater than 2%, make the text bold to highlight
it.
(From OE-Core rev: 500e28311248713d4772480b81b10777390da909)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When packaging a node application, a `NameError` can be thrown in create_npm.py if an optional npm dependency does not
support Linux.
(From OE-Core rev: 8293201d98d368d6322eaa960fb3e7cee2ba9368)
Signed-off-by: Sarah Marsh <sarah.marsh@arm.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This changes the output of "wic create mkefidisk -e core-image-sato" from:
The following build artifacts were used to create the image(s):
ROOTFS_DIR: /media/build1/poky/build/tmp.wic.ybraavmb/rootfs_copy
to:
The following build artifacts were used to create the image(s):
ROOTFS_DIR: /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-sato/1.0-r0/rootfs
which s much less confusing for the user.
[YOCTO #12564]
(From OE-Core rev: a4941af2d3624aecc5dcd7ff54b7ea8c9e9dee8b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A flaw was found when I run:
$ recipetool create "ssh://git@xxx.xxx:7999/xxx.git"
the url turned out to be: "git://git@xxx.xxx/7999/xxx.git;protocol=ssh"
after parsing, the port number was parsed as part of the path, this is
definitely wrong and lead to fetching failures.
This issue could be fixed in reformat_git_uri, by filtering out port
numbers when formatting ":".
(From OE-Core rev: 4290e04b69360b5e1da9f37166015e30f66cb335)
Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
LayerError doesn't exist and will lead to an error when this failure
code path is hit.
(From OE-Core rev: 7780482772d005c77825dc3e99e63f00911156bf)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is very useful for updating patch context so that any fuzz is eliminated.
Simply issue:
devtool modify <recipe>
devtool finish --force-patch-refresh <recipe> <layer_path>
Without this flag, devtool will not deem the commits in the workspace
different to patches in the layer, even if the commits have different,
up-to-date context line in them.
(From OE-Core rev: 7e1d1887be8faaaab9996fca9a3fd750aeb7b62f)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using --exclude-path and a wks.in file we can create an image that takes
the /boot/ directory for the boot partition, has an empty /boot/
directory in the rootfs partition. The boot partition gets mounted to
/boot/ after startup.
(From OE-Core rev: db904053e8ee80fb6930c5e7e22287927e0f25e2)
Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Set a miniumum FAT16 volume size because images are
not valid to parted if this minimum value is not respected.
The value set is determined experimentally forr a logical
sector size of 512. This fixed my local problem but, there
may be better solutions.
(From OE-Core rev: f7dfb4d43247d3c13a4e0a3853007d63b9512b83)
Signed-off-by: Paulo Neves <ptsneves@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
