Currently, if a test occurs multiple times over different series, the
code will sum these. This can lead to confusion over the desired
results. Change the code to report the redundant tests and skip adding
an additional values.
(From OE-Core rev: caeaa0648dff64c56a33f52e45e82bfab6719e3e)
Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The layer checks are designed to work with OEBasicHash so ensure that handler
is in use rather than the new hash equivalency one as an example.
(From OE-Core rev: a10bf92516a4771e2dc49ba9f74323d7a87a1619)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using different root directories with a wks file wic is using the
value from the original ROOTFS_SIZE which is not correct. Example:
Number Start End Size File system Name Flags
1 20.5kB 318MB 318MB fat16 otaefi legacy_boot, msftdata
2 318MB 636MB 318MB ext4 otaboot
3 636MB 1709MB 1074MB ext4 otaroot
4 1709MB 2027MB 318MB ext4 otaboot_b
5 2027MB 3101MB 1074MB ext4 otaroot_b
6 3101MB 5249MB 2147MB ext4 fluxdata
The partitions 1, 2, and 3 incorrectly inherit the size, instead of using
a computed size. With the patch applied it is working properly:
Number Start End Size File system Name Flags
1 20.5kB 14.5MB 14.5MB fat16 otaefi legacy_boot, msftdata
2 14.5MB 65.3MB 50.8MB ext4 otaboot
3 65.3MB 1139MB 1074MB ext4 otaroot
4 1139MB 1190MB 50.8MB ext4 otaboot_b
5 1190MB 2264MB 1074MB ext4 otaroot_b
6 2264MB 4411MB 2147MB ext4 fluxdata
As for the removal of the bb.warn, if the size is not specified, it is
not something to warn the end user about. Some of my default images
make use of the head room + a computed directory size or while generating
images.
(From OE-Core rev: 8e48b4d6c4d0ed213089a7449fea63aa0656e786)
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to create a msdos partition table disk image that can auto
expand after the image is copied to an SD card, wic needs the ability
to have a primary partition as the last entry. The desired use case
is to be able to create an A/B update partition image scheme with a
/var volume that can be auto expanded to the remainder of the SD card
at run time.
The typical .wks file will look similar to the following:
bootloader --ptable msdos
part / --source rawcopy --sourceparams="file=u-boot.imx" \
--ondisk mmcblk --no-table --align 1 --size 1
part /boot --source bootimg-partition \
--ondisk mmcblk --fstype=vfat --label boot --active --align 4 --size 16
part / --source rawcopy --sourceparams="file=imx6_boot.otaimg" \
--ondisk mmcblk --fstype=ext4 --label otaboot --align 4 --type logical
part / --source rawcopy --sourceparams="file=imx6.otaimg" \
--ondisk mmcblk --fstype=ext4 --label otaroot --align 4 --type logical
part / --source rawcopy --sourceparams="file=imx6_boot.otaimg" \
--ondisk mmcblk --fstype=ext4 --label otaboot_b --align 4 --type logical
part / --source rawcopy --sourceparams="file=imx6.otaimg" \
--ondisk mmcblk --fstype=ext4 --label otaroot_b --align 4 --type logical
part /var --source rawcopy --sourceparams="file=imx6_var.otaimg" \
--ondisk mmcblk --fstype=ext4 --label fluxdata --align 4
Without the patch applied, wic will generate an SD card image that looks like:
Disk boot.img: 890940s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 2056s 48001s 45946s primary fat16 lba
2 48008s 132467s 84460s primary ext4
3 132472s 454467s 321996s primary ext4
4 454471s 890939s 436469s extended lba
5 454472s 538931s 84460s logical ext4
6 538936s 860931s 321996s logical ext4
7 860936s 890939s 30004s logical ext4 boot
With the patch applied a primary partition can be created at the end
of the image which can be expanded to fill the free space on the media
where the image has been copied, which looks like:
Disk boot.img: 890940s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:
Number Start End Size Type File system Flags
1 2056s 48001s 45946s primary fat16 lba
2 48007s 860931s 812925s extended lba
5 48008s 132467s 84460s logical ext4
6 132472s 454467s 321996s logical ext4
7 454472s 538931s 84460s logical ext4
8 538936s 860931s 321996s logical ext4
3 860936s 890939s 30004s primary ext4 boot
(From OE-Core rev: 56add7cc547e0113cdf980579d1421b14cc233e5)
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a source plugin that support both EFI and legacy PC-Bios.
While using this plugin, both bootloaders configurations reside
in the same /boot partitions.
This plugin has very little code : to avoid code duplication,
we simply re-import bootimg-pcbios and bootmg-efi source and
call both their SourcePlugin methods.
(From OE-Core rev: c8545d54139c6c48bffd1dd1d39d79891626c6f7)
Signed-off-by: William Bourque <wbourque@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For now, the temp dir is left in system, although the temporary
source directory has been cleaned up. So we clean it up too.
(From OE-Core rev: 8a0602327d5afcf4f36850d3f05c9721305852af)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit 450335ba5e73a375eb9932b4c4cf37979640dbfc copies the pseudo
database to the working directory in order to have ownership information
when the filesystem is generated.
Unfortunately this does not work anymore. The filenames on the database
are absolute and there is no information about the new directory.
Instead of fixing the database, we could redo a bit the way we patch the
fstab file. Now I am saving the old contents of fstab, modifying the
file and then reverting the changes on exit.
This is faster than the previous approach, although it can cause
indeterminism if the application is killed before finishing.
(From OE-Core rev: dcbf7b864dd1713b54a172d8714ce1508482f086)
Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
when INITRAMFS_IMAGE_BUNDLE and INITRAMFS_IMAGE are set, wic should
look for kernel with initramfs image bundled.
Include required variable MACHINE, INITRAMFS_IMAGE_BUNDLE,
INITRAMFS_IMAGE, INITRAMFS_LINK_NAME and KERNEL_IMAGETYPE in WICVARS.
No longer require default value for variable kernel as KERNEL_IMAGETYPE
is not optional variable and included in WICVARS.
image_types_wic to inherit kernel-artifact-names to obtain default
INITRAMFS_LINK_NAME when INITRAMFS_IMAGE_BUNDLE are set.
update wic.Wic2.test_image_env test case to filter optional
variable INITRAMFS_LINK_NAME, INITRAMFS_IMAGE and INITRAMFS_IMAGE_BUNDLE.
(From OE-Core rev: bac984fbb2d5ad5d13ba3275c8a3e878d8753c58)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds a command line option to filter out the buildstats-diff report by
one more more tasks. e.g.:
buildstats-diff --only-task do_compile A B
will only show the differences for do_compile tasks. The --only-task
option can be specified multiple times to filter out multiple tasks at
once.
(From OE-Core rev: a8c7960d24c48107fd3703e49c38f890e84e2226)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
All packages that support the menuconfig task will be able to run
devtool menuconfig command. This would allow the user to modify the
current configure options and create a config fragment which can be
added to a recipe using devtool finish.
1. The patch checks if devtool menuconfig command is called for a valid
package.
2. It checks for oe-local-files dir within source and creates one if
needed, this directory is needed to store the final generated config
fragment so that devtool finish can update the recipe.
3. Menuconfig command is called for users to make necessary changes.
After saving the changes, diffconfig command is run to generate the
fragment.
Syntax:
devtool menuconfig <package name>
Ex: devtool menuconfig linux-yocto
The config fragment is saved as devtool-fragment.cfg within
oe-local-files dir.
Ex:
<workspace_path>/sources/linux-yocto/oe-local-files/devtool-fragment.cfg
Run devtool finish to update the recipe by appending the config fragment
to SRC_URI and place a copy of the fragment within the layer where the
recipe resides.
Ex: devtool finish linux-yocto meta
[YOCTO #10416]
(From OE-Core rev: 417feb559a74b367315e8658d6ba868a4f8d1340)
Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com>
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If kernel source is not already downloaded i.e staging kernel dir is
empty, place a copy of the source when the user runs devtool modify
linux-yocto. This way the kernel source is available for other packages
that use it.
[YOCTO #10416]
(From OE-Core rev: bb42ab90835e8ec2f1dfbb35056c353784693266)
Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com>
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the regular devtool modify flow, the kernel source is fetched by
running do_fetch task. This is an overhead in time and space.
This patch updates modify command to check if the kernel source is
already downloaded. If so, then instead of calling do_fetch, copy the
source from work-shared to devtool workspace by creating hard links
else run the usual devtool modify flow and call do_fetch task.
[YOCTO #10416]
(From OE-Core rev: 3c3a9bae296f849dbfe03942282f44036e6fa1fb)
Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com>
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As part of upgrading a recipe we create the upgraded recipe file in the
workspace and then try to parse it so we can then make further
modifications. If for some reason that parsing fails then the failure
was not being handled very well - the broken recipe was being left in
place, breaking parsing until it was removed by hand. Fix that by adding
a call to the cleanup function, and fix the following issues:
* Fix the cleanup function which doesn't look like it has ever worked
due to a typo in the function call
* Fix double-printing the error message
* Remove usage of DevtoolError in this case (DevtoolError is for simple
usage errors, not this kind of issue which may be the result of a
bug).
We're still printing a traceback in this scenario but at least it
doesn't break the build system requiring manual cleanup. I also
introduced a command-line option to preserve the broken upgraded recipe
file(s) for debugging purposes.
(The reproducer for this is "devtool upgrade libnewt-python", however
you need to check out revision b82ea144e1
or earlier since that recipe has now been absorbed into the libnewt
recipe. The libnewt-python recipe was causing an issue with the upgrade
because it actually included the libnewt recipe using ${PV} in the
include statement, and of course PV was changing in the upgrade.)
Fixes [YOCTO #13404].
(From OE-Core rev: c519ac360796675d7fc09a5250d21f0f5b6236fc)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a setup.py file exists it ought to have something in it before we
consider the source tree to be a Python module and treating it as such.
(A counter-example is https://www.bro.org/downloads/binpac-0.50.tar.gz -
it's not clear why this has a zero-length setup.py in it but we should
pay no attention to it.)
Fixes [YOCTO #12923].
(From OE-Core rev: 548a5c8f42c6ac1b0f7962926d05276e71505678)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This switches the code to build pkgdata specific to the current recipe
which means that its filtered to the recipes dependencies and can perform
better as we can drop the lockfile.
It uses a similar method to the staging code to do this, using BB_TASKDEPDATA
to construct a list of packagedata task output which this recipe should "see".
The original pkgdata store is left unaltered so existing code works.
The lock file was there to prevent files disappearing as they were read or as
directories were listed. Since we have a copy of the data and only access output
from completed tasks (as per their manifests), we can remove the lock.
The lock was causing starvation issues on systems with parallelism.
There was also a potential determinism problem as the current code could "see"
data from recipes which it doesn't depend upon.
[YOCTO #13412]
(From OE-Core rev: 1951132576bfb95675b4879287f8b3b7c47524fa)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(From OE-Core rev: 9303f92599343adf645fee5d2434fadd97e7febb)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently `devtool finish RECIPE meta' will silently succeed even
if there are multiple layers having the same base name of 'meta'.
e.g. meta layer from oe-core and meta layer from meta-secure-core.
We should at least give user a warning in such case. With the patch,
we will get warning like below.
WARNING: Multiple layers have the same base name 'meta', use the first one '<PROJ_DIR>/oe-core/meta'.
WARNING: Consider using path instead of base name to specify layer:
<PROJ_DIR>/oe-core/meta
<PROJ_DIR>/meta-secure-core/meta
(From OE-Core rev: 2c8740f543c38dbaef3345e40827ef48b3f75405)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow plugin bootimg-efi to configure with multiple initrd
through source parameter.
Uses ; to separate each initrd.
e.g:
--sourceparams="loader=${EFI_PROVIDER},initrd=initrd1;initrd2"
(From OE-Core rev: c7b0823f9ab6a9959aaa809b8c3f70d199feb64d)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current QA team need to merge test result files from multiple sources.
Adding TESTSERIES configuration too early will have negative
implication to report and regression. Enable control to add TESTSERIES
when needed. Also enable adding EXECUTED_BY configuration when
needed.
(From OE-Core rev: 651d8d371e78e77599d56681228d5782664f7743)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current results stored does not have information needed to trace who
executed the tests. Enable store to add EXECUTED_BY configuration
to results file in order to track who executed the tests.
(From OE-Core rev: dca2a57d54163a2e63b06e2f140fea3bd49cef0d)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current resultutils library always add "TESTSERIES" configuration
to results. Enhance this to allow control of adding "TESTSERIES"
configuration as well as allow adding extra configurations
when needed.
(From OE-Core rev: 443c0acc14ef2451b10878fc83dd11b46805daf0)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
replaced hardcoded kernel image with KERNEL_IMAGETYPE.
set kernel image to "bzImage" incase KERNEL_IMAGETYPE not set.
(From OE-Core rev: 88a9fef761c5e67b2964fedc85a7e8ad37067564)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support for generating python3 recipes using the recipetool / devtool.
Drop python2 support at the same time.
Tested with:
oe-selftest -r recipetool.RecipetoolTest
[YOCTO #13264]
(From OE-Core rev: d8b2f58974482b3b1ccc65c5f93104d0d7ba87bc)
Signed-off-by: Maciej Pijanowski <maciej.pijanowski@3mdeb.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some file systems don't support fetching the block size (notably the file system
Docker uses for containers), so handle the iotctl() failing and raise the
expected error.
(From OE-Core rev: 3757073726a00c5250556aae3d0daac76b88085e)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
allow wic to list and search for kickstart file in .wks.in extension.
basename show by wic list images to fully exclude extension.
(From OE-Core rev: 2c0a292a790ad069648e37b1b29fcea656fcf3e4)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Printing the lack of a test is not necessary (per feedback). Remove
this from the template to quieten it.
(From OE-Core rev: b1fe6ae66360e160eeaeafe456536f335a0eab60)
Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow the creation of test case configuration file based on user inputs.
Where this testcase configuration file will be used by the the manual
execution to run selected test cases for a module rather than compulsory
run all test cases in manual json file.
(From OE-Core rev: 73d2a747c17779da0ca972da776b3cf02c2e1cbc)
Signed-off-by: sangeeta jain <sangeeta.jain@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, the code will sum all of the different machine results into a
single report of the tests results. This can lead to confusion as to
which machine may be experiencing issues. Modify the code to store the
results in a per machine basis and report them accordingly.
(From OE-Core rev: 16d4031ea5df8a4ddfdb937d35464c09e1abd10e)
Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add new source parameter label to allow custom boot.conf/grub.cfg label,
so far it's hardcoded to "Boot".
Default label to "Boot" for systemd-boot and blank for grub-efi when source
parameter label are not set.
(From OE-Core rev: 7a0aab1aa31e66e6bc94c04c2f6c1043b64a8967)
Signed-off-by: Chee Yang Lee <chee.yang.lee at intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
volume name should refer to --label in .wks.
Replace the hardcoded volume name with label.
set "ESP" as default name when no lable specified.
(From OE-Core rev: 5621aceaf39ef0dc097b16c83e73b9882c987a7c)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds SPDX license headers in place of the wide assortment of things
currently in our script headers. We default to GPL-2.0-only except for the
oeqa code where it was clearly submitted and marked as MIT on the most part
or some scripts which had the "or later" GPL versioning.
The patch also drops other obsolete bits of file headers where they were
encoountered such as editor modelines, obsolete maintainer information or
the phrase "All rights reserved" which is now obsolete and not required in
copyright headers (in this case its actually confusing for licensing as all
rights were not reserved).
More work is needed for OE-Core but this takes care of the bulk of the scripts
and meta/lib directories.
The top level LICENSE files are tweaked to match the new structure and the
SPDX naming.
(From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds an option to dump all the ptest logs to individual files in a
specified directory. If multiple test runs are present, the
'--prepend-run' argument will create separate directories for each test
run under the target directory and put the logs there to prevent each
test run from clobbering the others.
[YOCTO #13331]
(From OE-Core rev: f262b9e7482e38f04dfa162f0aaf65b17e86f8b2)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If attempting to find a fixed SRCREV fails because the directory doesn't exit,
avoid failing with:
Traceback (most recent call last):
File "/home/vagrant/poky/scripts/recipetool", line 121, in <module>
ret = main()
File "/home/vagrant/poky/scripts/recipetool", line 110, in main
ret = args.func(args)
File "/home/vagrant/poky/scripts/lib/recipetool/create.py", line 707, in create_recipe
srcrev = stdout.rstrip()
UnboundLocalError: local variable 'stdout' referenced before assignment
Fixes: 000480c42797 ("recipetool / devtool: set a fixed SRCREV by default when fetching from git")
(From OE-Core rev: 8e4b301573071ea04911b0df6533c25bc4596b93)
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current manualexecution required user to exceute all test cases defined inside a "modulename.json" file in oeqa/manual
There are cases when all test cases all not required to run for a module on specific DUT.
Enable manualexecution to have the optional feature where it will use pre-defined json format test case configuration file
where user will be able to select test cases from the "modulename.json" instead of running all of them. This will help
in reducing testing time and reporting unneccesary skip or failures.
Example pre-defined json format test case configuration file (for build-applince):
{
"testcases" : [
"build-appliance.build-appliance.Create_core-image-sato-sdk_using_build_appliance",
"build-appliance.build-appliance.Build_a_image_without_error_(added_recipe)"
]
}
(From OE-Core rev: 3950c28a34f94b5907d37b579bdaee5a59794652)
Signed-off-by: sangeeta jain <sangeeta.jain@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit f7dfb4d43247d3c13a4e0a3853007d63b9512b83.
FAT16 volumes do not have a minimum size of 8250 blocks
(== slightly over 4MB). Exhibit A: floppy disks.
The original commit message suggests this was a workaround for a bug in
parted - in which case we should fix it there, or use the wic
`--fixed-size` option to pass down an enforced minimum from a more
context-aware point in the callstack.
(From OE-Core rev: e0a1ae3863bc229512d43e4a5248551bf25950e5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix AttributeError: module 'urllib' has no attribute 'request' when
using remote http urls.
(From OE-Core rev: 9acc47236e2a9ac7f97f80040f5e508f23f01611)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds a subcommand for dumping various logs from test results
(From OE-Core rev: 454b8d2cdc6f79a51e610dae92c22352850c3f7c)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds support for resulttool to load JSON files directly from a http://
or https:// URL
(From OE-Core rev: 235bcf0c504e7ea253ccfb28d600898117c64c1f)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove duplicate codes. Replace unnecessary class variables with
local variables. Rename variables and arguments with simple and
standard name.
(From OE-Core rev: 6bef61d36f3328fad003c0dc9c010d3f76ba96d8)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When I use recipetool to add a new recipe, it says:
[snip]
sys.exit(14)
NameError: name 'sys' is not defined
[snip]
(From OE-Core rev: 051e1c27bbf6a5898f2cf13b78dd3d8d1ec31586)
Signed-off-by: Lei Yang <Lei.Yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow the creation of configuration option file based on user inputs.
Where this configuration option file will be used by the the manual
execution to display options for configuration rather than user
need to inputs configuration manually.
(From OE-Core rev: 23c7a037a85a825aa088bfac3a144c14087a9f49)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current manualexecution required user to input configuration manually
where there were inconsistent inputs and human typo issues.
Enable manualexecution to have the optional feature where it
will use pre-compiled configuration options file where user will
be able to select configuration from the pre-compiled list instead
of manual key-in the configuration. This will eliminate human error.
Expect the pre-compiled configuration options file in json format below
{
"bsps-hw": {
"IMAGE_BASENAME": {
"1": "core-image-sato-sdk"
},
"MACHINE": {
"1": "beaglebone-yocto",
"2": "edgerouter",
"3": "mpc8315e-rdb",
"4": "genericx86",
"5": "genericx86-64"
}
},
"bsps-qemu": {
"IMAGE_BASENAME": {
"1": "core-image-sato-sdk"
},
"MACHINE": {
"1": "qemuarm",
"2": "qemuarm64",
"3": "qemumips",
"4": "qemumips64",
"5": "qemuppc",
"6": "qemux86",
"7": "qemux86-64"
}
}
}
(From OE-Core rev: 1af7e04223b415a64e100a9f91d60f5e9b0f789a)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The cml1 and ccmake bbclasses generate configuration fragment source
files that must be exported from the WORKDIR as a source file to be
preserved across builds. This change adds detection of the current
recipes inherited classes and for cml1 and ccmake classes checks for the
specific generated configuration fragment files. These files are then
exported by devtool and included as SRC_URI files from within the target
layer.
(From OE-Core rev: 7a1fabe4a24552ce8a50c8d6009969ca52a5fd27)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the manual execution display step by sorting
the step as string, where steps were not being sorted
correctly when there are more than 9 steps.
Fixed the step sorting by sorting step as integer.
(From OE-Core rev: 192c255d9f597b3526a5b94fcf3eedc4527189ed)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current manualexecution required pressing enter button to show each step
information, where this was wasting execution time. Enable display
full steps without needing to any press enter button.
(From OE-Core rev: e44c9a018c13208fef0dcef4479ce71977628cd4)
Signed-off-by: Mazliana <mazliana.mohamad@intel.com>
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current input checking does not match the standard input practiced
by QA team. Change the input checking to match the standard
input practiced by the QA team.
(From OE-Core rev: 630a93c740359a65249a198f314f15040042f8d3)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than simply discarding the ptest data, change the code to discard
it when writing out the new testresult files, or optionally either preserve
it, or write it as seperate discrete logs.
This means the autobuilder should start writing out individual ptest log
files as well as allowing ueers to extract these manually.
(From OE-Core rev: a1e0944bf260ef50dd7dfcb10db248fdd7f45bc9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Store operations using a single file as a source weren't working as the os.walk
command didn't like being given a single file. Fix the store operation to
work for single files.
(From OE-Core rev: eb5bbe613d8c7cbcd8b74d8ac0073c2217970410)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
On some hosts the wic.Wic2.test_wic_cp_ext selftest was failing as files weren't
being copied into the rootfs with "wic cp". This was due to a bug added by:
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=89d4a8df074598cfb3a76e41db7c45d845afd961
where there should be a second newline added at the end of the expression due
to the difference in the way echo -e and printf behave.
[YOCTO #13237]
(From OE-Core rev: 2f3adb21cc5067458a12964d0bde235966a20a60)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Devtool creates a git repository for extracted sources in a temporary
directory and then moves it to a final destination after patching is
done. Unfortunately devtool is not aware that some of its git operations
may have caused git garbage collector to start in background. If timing
is just right a repository move fails because GC is removing git objects
while they are being moved.
The issue was hit on Krogoth release, but the code that moves created
git repository is still the same.
Disable gc.autodetach to make GC run synchronously and block git until
it is done.
Traceback (most recent call last):
File "<snip>/openembedded-core/scripts/devtool", line 342, in <module>
ret = main()
File "<snip>/openembedded-core/scripts/devtool", line 329, in main
ret = args.func(args, config, basepath, workspace)
File "<snip>/openembedded-core/scripts/lib/devtool/standard.py", line 352, in extract
initial_rev = _extract_source(srctree, args.keep_temp, args.branch, False, rd)
File "<snip>/openembedded-core/scripts/lib/devtool/standard.py", line 644, in _extract_source
shutil.move(srcsubdir, srctree)
File "/usr/lib/python2.7/shutil.py", line 299, in move
copytree(src, real_dst, symlinks=True)
File "/usr/lib/python2.7/shutil.py", line 208, in copytree
raise Error, errors
Error: [('/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/5e', '<snip>/build/grpc/grpc_src/.git/objects/5e', "[Errno 2] No such file or directory: '/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/5e'"), ('/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/57', '<snip>/build/grpc/grpc_src/.git/objects/57', "[Errno 2] No such file or directory: '/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/57'"), many git objects ... ]
(From OE-Core rev: 3da578a79195a63fb6d4bbd68d8f1548defd205f)
Signed-off-by: Taras Kondratiuk <takondra@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We found that manualexecution does not capture test suite values
correctly if there are more than one test suite in test cases.
After verification has made we found out we should retrieved
full test cases value <test_module.test_suite.test_case> from
oeqa/manual/ json file rather than split it them into new
variables test_suite and test_cases.
(From OE-Core rev: db08ec5f48badd5be2fa24cc0d89a4386ab44a59)
Signed-off-by: Mazliana <mazliana.mohamad@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enable roll-up all test results belong to a commit
and to provide a roll-up report.
(From OE-Core rev: a98a75a3db1fe9d327fd21919fb8ea09e76fb1e3)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
do_patch rule of SDK's workspace/appends/linux-*.bbhappend may fail if script are not written in Python
that was the case with Phytec's BSP, the fix was to replace the do_patch rule with :
do_patch[noexec]="1" when the file was generated in scripts/lib/devtool/standard.py
(From OE-Core rev: e51cb385e5573c7069de6e8308b9b68eb4a3a162)
Signed-off-by: Yann CARDAILLAC <yann.cardaillac@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To enable store for testresults.json file from manualexecution,
add layers metadata to configuration and add "manual" map to
resultutils.store_map.
To enable regression for manual, add "manual" map to
resultutils.regression_map. Also added compulsory configurations
('MACHINE', 'IMAGE_BASENAME') to manualexecution.
(From OE-Core rev: 7521834f8557565d3898605257c7fd9505ea22bb)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than recreating handlers and forcing them, iterate over the handlers
and enable color on ones we can handle. This makes it easier to handle color
properly when we introduce the bb.msg default log filters.
(From OE-Core rev: 713f44e0e0cde9b818c214002fd8b730d422fafa)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sorted regression results to provide friendly viewing of report.
(From OE-Core rev: 59d827889dd8b790a7c9323c69565d6bba3041bd)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
ptest suites with no results don't show up on the reports even though we have
a duration for them. Fix this so the fact they report no tests is visible.
(From OE-Core rev: 58268151704246a81ec8dae46c26346023057554)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently some older results files cause the code to give tracebacks.
Handle these missing sections more cleanly.
(From OE-Core rev: f778c191dbd5740173b3be07f4c1655d85a07bb2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently we cant store results if the results files span multiple
different build revisons. Remove this limitation by iterating.
(From OE-Core rev: 4da12c00963b02508056b87ce9b972528ce3a1be)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow the use of an alternate SSH executable for deploy-target and
undeploy-target. The command can be specified with -e / --ssh-exec.
(From OE-Core rev: c2684740d7fd3aeb60b3ec7888000097effe82f4)
Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a combined patch of the various tweaks and improvements I
made to resulttool:
* Avoid subprocess.run() as its a python 3.6 feature and we
have autobuilder workers with 3.5.
* Avoid python keywords as variable names
* Simplify dict accesses using .get()
* Rename resultsutils -> resultutils to match the resultstool ->
resulttool rename
* Formalised the handling of "file_name" to "TESTSERIES" which the code
will now add into the json configuration data if its not present, based
on the directory name.
* When we don't have failed test cases, print something saying so
instead of an empty table
* Tweak the table headers in the report to be more readable (reference
"Test Series" instead if file_id and ID instead of results_id)
* Improve/simplify the max string length handling
* Merge the counts and percentage data into one table in the report
since printing two reports of the same data confuses the user
* Removed the confusing header in the regression report
* Show matches, then regressions, then unmatched runs in the regression
report, also remove chatting unneeded output
* Try harder to "pair" up matching configurations to reduce noise in
the regressions report
* Abstracted the "mapping" table concept used to pairing in the
regression code to general code in resultutils
* Created multiple mappings for results analysis, results storage and
'flattening' results data in a merge
* Simplify the merge command to take a source and a destination,
letting the destination be a directory or a file, removing the need for
an output directory parameter
* Add the 'IMAGE_PKGTYPE' and 'DISTRO' config options to the regression
mappings
* Have the store command place the testresults files in a layout from
the mapping, making commits into the git repo for results storage more
useful for simple comparison purposes
* Set the oe-git-archive tag format appropriately for oeqa results
storage (and simplify the commit messages closer to their defaults)
* Fix oe-git-archive to use the commit/branch data from the results file
* Cleaned up the command option help to match other changes
* Follow the model of git branch/tag processing used by oe-build-perf-report
and use that to read the data using git show to avoid branch change
* Add ptest summary to the report command
* Update the tests to match the above changes
(From OE-Core rev: ff2c029b568f70aa9960dde04ddd207829812ea0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Integrated “manualexecution” operation to resulttool scripts.
Manual execution script is a helper script to execute all manual
test cases in baseline command, which consists of user guideline
steps and the expected results. The last step will ask user to
provide their input to execute result. The input options are
passed/failed/blocked/skipped status. The result given will be
written in testresults.json including log error from the user
input and configuration if there is any.The output test result
for json file is created by using OEQA library.
The configuration part is manually key-in by the user. The system
allow user to specify how many configuration they want to add and
they need to define the required configuration name and value pair.
In QA perspective, "configuration" means the test environments and
parameters used during QA setup before testing can be carry out.
Example of configurations: image used for boot up, host machine
distro used, poky configurations, etc.
The purpose of adding the configuration is to standardize the
output test result format between automation and manual execution.
To use these scripts, first source oe environment, then run the
entry point script to look for help.
$ resulttool
To execute manual test cases, execute the below
$ resulttool manualexecution <manualjsonfile>
By default testresults.json store in <build_dir>/tmp/log/manual/
[YOCTO #12651]
(From OE-Core rev: f24dc9e87085a8fe5410feee10c7a3591fe9d816)
Signed-off-by: Mazliana <mazliana.mohamad@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
OEQA outputs test results into json files and these files were
archived by Autobuilder during QA releases. Example: each oe-selftest
run by Autobuilder for different host distro generate a
testresults.json file.
These scripts were developed as a test result tools to manage
these testresults.json file.
Using the "store" operation, user can store multiple testresults.json
files as well as the pre-configured directories used to hold those files.
Using the "merge" operation, user can merge multiple testresults.json
files to a target file.
Using the "report" operation, user can view the test result summary
for all available testresults.json files inside a ordinary directory
or a git repository.
Using the "regression-file" operation, user can perform regression
analysis on testresults.json files specified. Using the "regression-dir"
and "regression-git" operations, user can perform regression analysis
on directory and git accordingly.
These resulttool operations expect the testresults.json file to use
the json format below.
{
"<testresult_1>": {
"configuration": {
"<config_name_1>": "<config_value_1>",
"<config_name_2>": "<config_value_2>",
...
"<config_name_n>": "<config_value_n>",
},
"result": {
"<testcase_namespace_1>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
"<testcase_namespace_2>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
...
"<testcase_namespace_n>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
}
},
...
"<testresult_n>": {
"configuration": {
"<config_name_1>": "<config_value_1>",
"<config_name_2>": "<config_value_2>",
...
"<config_name_n>": "<config_value_n>",
},
"result": {
"<testcase_namespace_1>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
"<testcase_namespace_2>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
...
"<testcase_namespace_n>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
}
},
}
To use these scripts, first source oe environment, then run the
entry point script to look for help.
$ resulttool
To store test result from oeqa automated tests, execute the below
$ resulttool store <source_dir> <git_branch>
To merge multiple testresults.json files, execute the below
$ resulttool merge <base_result_file> <target_result_file>
To report test report, execute the below
$ resulttool report <source_dir>
To perform regression file analysis, execute the below
$ resulttool regression-file <base_result_file> <target_result_file>
To perform regression dir analysis, execute the below
$ resulttool regression-dir <base_result_dir> <target_result_dir>
To perform regression git analysis, execute the below
$ resulttool regression-git <source_dir> <base_branch> <target_branch>
[YOCTO# 13012]
[YOCTO# 12654]
(From OE-Core rev: 78a322d7be402a5b9b5abf26ad35670a8535408a)
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Just stating the recipe counts are different isn't helpful, showing
the differences makes it much easier to understand what changed.
(From OE-Core rev: 1d84b782e3af6f0e6922d72895c905877cc33739)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
OE-Core rev: a88bcbae850a2e6d182291d3f8e167aabdbe4842 broke the ability
to find parted as it may be in sbin which is not in PATH for some users
on some distros.
Iterate on the original patch to fix this and also fix the original problem.
(From OE-Core rev: af3803e5189d7814f9dbd238fb6dab200f351e1a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Load self.paths from environment variable and
if it fails, fall back to hardcoded list.
This is required for users that would need to load
different e2fsprogs binaries if their system's ones
are not recent enought
(From OE-Core rev: a88bcbae850a2e6d182291d3f8e167aabdbe4842)
Signed-off-by: William Bourque <wbourque@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python subprocess' shell=True defaults to /bin/sh[1][2], which
often refers to a POSIX-compliant shell. As the -e flag is
not defined in the POSIX standard[3], some shells may
interpret "-e" as the first argument to the echo command,
causing the rest of the command line to fail.
In this particular case, "echo -e 'cd {}'" is interpreted
as "-e cd {}", which causes the first line of the command
to fail, and causing cp to always place the source file
in the filesystem's root.
Replacing "echo -e" for a printf command makes this
more portable.
This issue only affects "wic cp" for ext* filesystems.
[1] https://docs.python.org/2/library/subprocess.html
[2] https://docs.python.org/3/library/subprocess.html
[3] http://pubs.opengroup.org/onlinepubs/9699919799
(From OE-Core rev: 9a5b703d4e60c3ccdf80b5289b8e6fc91133fcde)
Signed-off-by: Leonardo Augusto <leobsd@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Rename add_layer() to add_layers() so that add_layer_dependencies() can
re-use it.
* Avoid adding the layer if it is already present
[YOCTO #13148]
(From OE-Core rev: b9cc18d83f55ff48c3d6e60c56359f6736d5a06a)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The default behaviour is to show just recipes needing upgrades, but for
automated reports showing all recipes can be useful.
(From OE-Core rev: 29e064ac6bd8d1ef74c280bf85bf8428c37ba014)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The check_commits logic assumes that both devtool-base and args.branch
exist in the git repo that it is operating on. In order to prevent
errors at that point it's best to first ensure that both of these refs
actually exist. If they don't both exist then the check_commits logic
should just be skipped, as it would be if the repo wasn't originally
checked out by devtool.
Previously if a user removed the args.branch branch from their devtool
cloned repo this code would crash on adding the repo with -n. The crash
would look like this:
Traceback (most recent call last):
File "/home/ddedrick/src/poky/scripts/devtool", line 344, in <module>
ret = main()
File "/home/ddedrick/src/poky/scripts/devtool", line 331, in main
ret = args.func(args, config, basepath, workspace)
File "/home/ddedrick/src/poky/scripts/lib/devtool/standard.py", line 812, in modify
(stdout, _) = bb.process.run('git log devtool-base..%s' % branch, cwd=srctree)
File "/home/ddedrick/src/poky/bitbake/lib/bb/process.py", line 178, in run
raise ExecutionError(cmd, pipe.returncode, stdout, stderr)
bb.process.ExecutionError: Execution of 'git log devtool-base..devtool' failed with exit code 128:
fatal: ambiguous argument 'devtool-base..devtool': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
(From OE-Core rev: f13a3490fdb404bbd4c77e45b83540d6deec1358)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
DEVTOOL_EXTRA_OVERRIDES only needs one entry for each instance of
overrides. Previous to these changes it would find every override to
SRC_URI and add it to the list. This would duplicate instances where
SRC_URI is modified multiple times with the same override like:
SRC_URI_append_foo += "file://0001-foo.patch"
SRC_URI_append_foo += "file://0002-bar.patch"
A bbappend might also overwrite a SRC_URI override, which would also
cause multiple instances to occur.
When there are multiple instances of the same override in
DEVTOOL_EXTRA_OVERRIDES it causes devtool modify to fail when creating
override branches. The failure occurs when attempting to create the same
override branch a second time and looks like this:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:devtool_post_patch(d)
0003:
File: '/build/poky/meta/classes/devtool-source.bbclass', lineno: 202, function: devtool_post_patch
0198:
0199: for override in extra_override_list:
0200: localdata = bb.data.createCopy(d)
0201: if override in default_overrides:
*** 0202: bb.process.run('git branch devtool-override-%s %s' % (override, devbranch), cwd=srcsubdir)
0203: else:
0204: # Reset back to the initial commit on a new branch
0205: bb.process.run('git checkout %s -b devtool-override-%s' % (initial_rev, override), cwd=srcsubdir)
0206: # Run do_patch function with the override applied
File: '/build/poky/bitbake/lib/bb/process.py', lineno: 178, function: run
0174: if not stderr is None:
0175: stderr = stderr.decode("utf-8")
0176:
0177: if pipe.returncode != 0:
*** 0178: raise ExecutionError(cmd, pipe.returncode, stdout, stderr)
0179: return stdout, stderr
Exception: bb.process.ExecutionError: Execution of 'git branch devtool-override-foo devtool' failed with exit code 128:
fatal: A branch named 'devtool-override-foo' already exists.
(From OE-Core rev: 90f667db2219f04e6d61588cd61056d3d8da6d7d)
Signed-off-by: Dan Dedrick <ddedrick@lexmark.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Apparently either nobody ever used sdimage-bootpart.wks
or fstab was kept untouched due to "--no-fstab-update" usage as
some boards like IMX may have, see [1].
Otherwise addition of the following line to the target's fstab:
---------------------->8-----------------
/dev/mmcblkp1 /boot vfat defaults 0 0
---------------------->8-----------------
gets us unusable target:
---------------------->8-----------------
[* ] A start job is running for dev-mmcblkp1.device (23s / 1min 30s)
[ TIME ] Timed out waiting for device dev-mmcblkp1.device.
[DEPEND] Dependency failed for /boot.
[DEPEND] Dependency failed for Local File Systems.
You are in emergency mode. After logging in, type "journalctl -xb" to view
system logs, "systemctl reboot" to reboot, "systemctl default" or "exit"
to boot into default mode.
Press Enter for maintenance
---------------------->8-----------------
Fix it with use of "mmcblk0" device name.
[1] https://lists.yoctoproject.org/pipermail/meta-freescale/2018-February/022156.html
(From OE-Core rev: 57a925dbd949e488980b02896106efd82bf47ed2)
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Cc: Maciej Borzecki <maciej.borzecki@open-rnd.pl>
Cc: Maciek Borzecki <maciek.borzecki@gmail.com>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
image_types_wic.bbclass has a mechanism for doing variable substitution
on .wks files by simply letting the input file be called
.wks.in. However, that doesn't allow using variables in files included
via the include directive. This is unfortunate, because lacking either
the ability to include other files or variable substitution leads to
fragile and error-prone duplication between kickstarter files and
recipes/configuration files used for various boards.
This adds (somewhat naive) support for variable substitution in all
files parsed by wic. The user should add all required variables to
WICVARS to get them exported appropriately.
(From OE-Core rev: 8a75d614a8a1ff72c4af448ac3e1292d0e1d1a79)
Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a fixup for: e1ba46109e
devtool deploy-target --strip foo root@192.168.7.2
ended up with:
Traceback (most recent call last):
File "/home/user/bar_sdk/sysroots/x86_64-pokysdk-linux/usr/bin/
devtool", line 344, in <module>
ret = main()
File "/home/user/bar_sdk/sysroots/x86_64-pokysdk-linux/usr/bin/
devtool", line 331, in main
ret = args.func(args, config, basepath, workspace)
File "/home/user/bar_sdk/layers/poky/scripts/lib/devtool/
deploy.py", line 187, in deploy
rd.getVar('base_libdir'))
TypeError: strip_exbar() missing 1 required positional argument: 'd'
(From OE-Core rev: bc349b2d99d74f6d7fb590fd97750e5ff8d6e6c4)
Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The imp module is deprecated, port the code over to use importlib
as recently done for bb.utils as well.
(From OE-Core rev: 24809582d4850190d87cd8eb8180d0dce215dbdf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes the users might want to change the title showing on UEFI
booting screen, so far it's hard-coded to 'boot'.
There is not a easy way to customize it in current design, I tried
firstly with '--configfile', but that does not work with --use-uuid,
since the later option will generate a UUID and write it to boot
config, only when the former option is not enabled.
So a new source parameter 'titile' is added in this patch, it defaults
to 'boot' to be consistent with the original title.
(From OE-Core rev: 37e16188ef3b1b328eb18b3e459c051c9c9f0332)
Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
yocto-check-layer will find locked-sigs.inc under builddir,
but locked-sigs.inc is generated under current bitbake working
dir. if run yocto-check-layer outside builddir, we will met error
like "No such file or directory: *locked-sigs.inc". change to
run bitbake -S under builddir to fix this problem.
[YOCTO #12973]
(From OE-Core rev: 2ec0bc0b038bc6413978c8f34ef5c0d22b4bc3e7)
Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
os.errno doesn't work in Python 3.7 and shouldn't have ever worked, so use
import errno explicitly.
(From OE-Core rev: 82f9157e84dcaf0ad4292053b09be68c2290d197)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It fails to build multilib image such as lib32-core-image-minimal with
wic by set 'IMAGE_FSTYPES = "wic"':
| ERROR: Couldn't find correct bootimg_dir, exiting
When multilib is enabled, STAGING_DATADIR is expanded with MLPREFIX. But
dependencies of images such as syslinux is still populated to nonarch
STAGING_DATADIR.
Search nonarch STAGING_DATADIR to fix the error.
(From OE-Core rev: dbae9a6f9a1c6cc7d4dd680d7bbda3dfa40f3491)
Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Running wic commands on Debian 10 systems fail in
scripts/lib/wic/engine.py:get_partitions() due to new stderr output captured
when trying to parse the output from /sbin/parted as a non-root user.
The parted command calls the dmidecode utility, which produces this error
as a non-root user:
/sys/firmware/dmi/tables/smbios_entry_point: Permission denied
/dev/mem: Permission denied
scripts/lib/wic/engine.py:get_partitions() calls misc.py:exec_cmd(),
a subprocess wrapper which returns a combined stderr and sdtdout.
These messages to stderr confuse the partition table parser in
get_partitions().
This patch has the partition table parser ignore lines before the expected
"BYT;" header string.
Running wic in Debian 9 does not have this issue.
(From OE-Core rev: d6936301d7598b7a783beaae95109555faf6cc17)
Signed-off-by: Geoff Parker <geoffhp@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(From OE-Core rev: f24c3538a27388d282a94e2d8a70cd9108d0919c)
Signed-off-by: Michael Davis <michael.davis@essvote.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove usage of a specific amount of memory and let it be controlled by
users. This was the default behaviour before it was changed by commit
3b79d9a78 that switched the wks file to be used for qemux86.
Also fixes the bitbake parsing issues seen because of memory starvation
using build appliance images.
Fixes [YOCTO #12894]
(From OE-Core rev: 18d6b668c52dc881cff7b107420e0de527eecce4)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Because the find | cpio processes execute in parallel connected via
the pipe, and the cpio outputs in the same dir find searches for
source files, the cpio will be included in itself partially, depending
on how fast the build machine creates the cpio file before cpio
gobbles it up.
This bloats the ISO image, though compression reduces the .iso file size,
once the kernel decompresses the cpio image and boots it live, it uses
up to double the RAM memory.
Fix this by creating the initrd.cpio file directly inside cr_workdir.
(From OE-Core rev: 7a7b29f5f1392f4f87af4f20cda99c66b4789fde)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We can use this parameter to make the wic use the label to name a
partition in /etc/fstab.
(From OE-Core rev: 51638edaa00befaed58e2def255d46ae44d9234f)
Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wic allows the user to create disk image and reference each created
partitions via its partition UUID.
squashfs does not support filesystem UUID but it does offer support for
partition UUID.
$ lsblk -o NAME,FSTYPE,UUID,PARTUUID
NAME FSTYPE UUID PARTUUID
sda
|-sda1 vfat 6078-ACD0 95aa4ffb-741d-4d99-adf6-0358bc890046
|-sda2 squashfs 7d46da2d-29cc-4d5a-9e39-d0381c4abb0c
|-sda3 squashfs d4a3e24e-410a-4f2c-8ad5-8f8321de177b
|-sda4 ext4 d72ec529-78e4-4cd3-b39b-781b6b1cb47c f3edc990-f4f6-4536-9587-44dae4ceb99f
`-sda5 ext4 a0b265aa-e935-40b8-9c6b-e9e58ad3890f 6e72ebaa-29ed-41cd-844c-3f76976cf41d
Modifying this constrain allows a user to create squashfs-based rootfs and boot
from them with such kernel argument 'root=PARTUUID=7d46da2d-29cc-4d5a-9e39-d0381c4abb0c'
Fixes: 2fbdcf4e59c8 ("wic: kparser.py: Check for SquashFS and use-uuid")
(From OE-Core rev: a8787bbfdf088b0ef9feb8e6b4230b743bb4086f)
Signed-off-by: Emmanuel Roullit <emmanuel.roullit@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add -c (--clean) optiont to 'devtool build' command so that users
could easily clean things up when using devtool.
I encountered a problem about do_prepare_recipe_sysroot failure
when using `devtool build' command and I found myself in a situation
where I either have to use `bitbake' command to clean things up or
use `rm' to remove the directories under ${WORKDIR}.
So add a clean option as it would be helpful when users want to clean
things up to prepare an environment for a clean build.
(From OE-Core rev: 29d790cdeff19e520a35ec5902d6deaae8665492)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
initrd.cpio is created in initrd_dir, not in cr_workdir. Gzip will
let initrd.cpio uncompressed if the path is not found.
Also, grub_image variable doesn't exist, grub_src_image should
be used instead.
(From OE-Core rev: c88875b568a66297b1d3df380c590f20796c57f6)
Signed-off-by: Alexandru Vasiu <alexandru.vasiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Only bsdcpio works with numbers for option -R to specify user:group,
while GNU cpio doesn't. Debian use GNU cpio so without this change,
you cannot create ISO images without installing bsdcpio.
(From OE-Core rev: 7c6ba368006ca94ecc0daac7b1c5e0fbe4236daf)
Signed-off-by: Alexandru Vasiu <alexandru.vasiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Automatically select one kernel type image based on a predefined
precedence list if there are multi kernel images installed.
(From OE-Core rev: d1d80566681d4cdc00aa3d4b5e4bcf5edb7132b7)
Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support to specify a custom extlinux.conf via something like:
bootloader --configfile="extlinux.conf"
(From OE-Core rev: 33f85c8bf80d70f00eeccd9ab3dfa417c0fc7df1)
Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
