Commit Graph

3531 Commits

Author SHA1 Message Date
Alexander Kanavin
5a5ab7cdec scripts/runqemu: move render nodes check to runqemu from selftest
This will produce a more useful hint for those setting
up or testing virgl headless:

runqemu - ERROR - No render nodes found in /dev/dri: ['by-path', 'card0']. If /dev/dri/renderD* is absent due to lack of suitable GPU, 'modprobe vgem' will create one suitable for mesa llvmpipe software renderer.

as qemu itself isn't helpful:

alex@Zen2:/srv/storage/alex/yocto/build-64-alt$ qemu-system-x86_64 -display egl-headless
qemu-system-x86_64: egl: no drm render node available
qemu-system-x86_64: egl: render node init failed

(From OE-Core rev: cbbada6a6c9b0a2e97f7395117dad986555f2db9)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-27 13:27:37 +00:00
Richard Purdie
c45d58f003 resulttool/regression: Improve matching of poor ptest test names
Some test case naming is poor and contains random strings, particularly
lttng/babeltrace but also curl. Truncating the test names works since they
contain file and line number identifiers which allows us to match them
without the random components, or in the case or curl, test IDs.

Going forward we may be able to improve the test names but this
tweak allows historical test results to work in reports.

(From OE-Core rev: 541a2e2683531355e678fd93524a0c4a8c43a8ff)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-26 11:59:52 +00:00
Richard Purdie
401d022a26 resulttool/regression: Ensure LTP results are only compared against other LTP runs
If a test result contains LTP test results, it should only be compared with
other runs containing LTP test results.

(From OE-Core rev: 4dbbf2f4a85620a08dc2fa65095dc17fe6c530f8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-26 11:59:52 +00:00
Alexis Lothoré
7a70dcfea7 scripts: add new helper for regression report generation
Add yocto-testresults-query script. This is a thin wrapper over resulttool which
is able to translate tags or branch name to specific revisions, and then to work
with those "guessed" revisions with resulttool

(From OE-Core rev: b1460201b0f3d8fd7f977ac82f218bf9010d5573)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-26 11:59:52 +00:00
Alexis Lothoré
383cd86595 scripts/resulttool/regression: add metadata filtering for oeselftest
When generating regression reports, many false positive can be observed since
some tests results are compared while the corresponding tests sets are not the
same, as it can be seen for example for oeselftest tests (oeselftest is run
multiple time but with different parameters, resulting in different tests sets)

Add a filtering mechanism in resulttool regression module to enable a better
matching between tests. The METADATA_MATCH_TABLE defines that when the TEST_TYPE
is "oeselftest", then resulttool should filter pairs based on
OESELFTEST_METADATA appended to test configuration. If metadata is absent from
test results, in order to keep compatibility with older results, add a
"guessing" mechanism to generate the missing OESELFTEST_METADATA. The guessed
data is tightly coupled to the autobuilder configuration, where all oe-selftest
executions are described

(From OE-Core rev: 94ab7c2b892bf292dd86619ca9c63ddd7bf53f3c)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-26 11:59:52 +00:00
Alexis Lothoré
88345ecf47 scripts/resulttool/regression: remove unused import
(From OE-Core rev: 4d6a74fcee1479562bf66360f20d528107c70887)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-26 11:59:52 +00:00
Pavel Zhukov
4add8776ab wic: Fix populating of IMAGE_EFI_BOOT_FILES with uefi-kernel
wic plugin crashed if IMAGE_EFI_BOOT_FILES is not empty and uefi-kernel
loader specified because of preliminary return from the function.
[Yocto #15033]
Fixes:
|   File "/mnt/builds/yocto/sources/scripts/lib/wic/plugins/source/bootimg-efi.py", line 371, in do_prepare_partition
|     for src_path, dst_path in cls.install_task:
|                               ^^^^^^^^^^^^^^^^
| AttributeError: type object 'BootimgEFIPlugin' has no attribute 'install_task'. Did you mean: 'do_install_disk'?

(From OE-Core rev: a6372f70f31b39ce9867b705d02205621d7a8685)

Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-26 11:49:00 +00:00
Chee Yang Lee
80a2028813 checklayer: check for patch file upstream status
yocto-check-layer to check all .patch file in layer for
Upstream-status and list down all .patch file without Upstream-Status.

set this test as expected failure for now, so this wont fail the check
layer while still able to capture any patches with malformed or missing
Upstream-Status in report.

[YOCTO #14642]

(From OE-Core rev: 237c1b66e5014123c1e5c3e78f9ab0357bcd62dc)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-24 17:05:24 +00:00
Alexander Kanavin
ca7306f78a devtool: ignore patch-fuzz errors when extracting source
So that patch fuzz issues can actually be fixed,
as extracting source with 'devtool modify' is the first step
for that.

(From OE-Core rev: 7067abd31d9dd4b98ec70c1c7effbe2904797cd1)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-24 13:31:45 +00:00
Mikko Rapeli
bfb4d48d7c runqemu: kill qemu if it hangs
qemu doesn't always behave well and can hang too.
kill it with force if it was still alive. Move clean up
commands into cleanup() function.

(From OE-Core rev: 079c2935d2f585ce49e1c7daab2155fcf0094c48)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-23 12:11:29 +00:00
Alexander Kanavin
46c6a8d63c scripts/oe-setup-layers: do not clobber json module with a variable named 'json'
(From OE-Core rev: 91e5aef9fd710b08ead9491f1900fbab5386b157)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-20 15:18:30 +00:00
Alexander Kanavin
e8b88168f5 scripts/oe-setup-layers: correct variable names - layers should be called repos
The script is operating on layer repositories, which can and do sometimes contain
several layers. This distinction is important as the script will be tweaked
to write a record of actual layer locations.

(From OE-Core rev: 833965e6001db98039c0aa816ae661232213bcea)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-20 15:18:30 +00:00
Ross Burton
163aa6f149 lib/buildstats: handle tasks that never finished
If a task is aborted the buildstats file isn't complete, so calculate
when the build finished and use that as a end time.

(From OE-Core rev: 23ebaec476dc46aebe5997f025661137f3e341bd)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-17 18:01:01 +00:00
Alexander Kanavin
d369a45ae7 meta/conf: move default configuration templates into meta/conf/templates/default
This sets the ground for standardizing (and enforcing) the location of
configuration templates: let's start with the default one.

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-16 17:11:11 +00:00
Alexander Kanavin
4322056708 scripts/oe-setup-layers: print a note about submodules if present
(From OE-Core rev: 7f16c1c0229110c242f2e064d612d0d35823bf8d)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-15 10:21:34 +00:00
Pavel Zhukov
7f6a17a19c wic: Fix usage of fstype=none in wic
This allows to specify partition with fstype=none in the wks file
to have partition created but without following mkfs. The none fstype
is in the list already but the usage is not documented.

Example;
part /data --ondisk mmcblk0 --fstype=none  --align 4096 --fixed-size 512

will create a partition, filesystem may be created manualy on the host
or target and data will be preserved if the device is reflashed using
same wks. Works with bmaptool and probably does not work with dd.
Use case is persistent filesystem/data between reflashing of the image.

(From OE-Core rev: 351cb64da37aa43113e5192605d04436652aa3b8)

Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-15 10:21:34 +00:00
Xiaotian Wu
324b03e15f base: add support for loongarch64
(From OE-Core rev: 759baaceb4dd623d5da12ba0d01540fa080154ba)

Signed-off-by: Xiaotian Wu <wuxiaotian@loongson.cn>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-04 17:02:10 +00:00
Martin Jansa
94149c4f37 meta: remove True option to getVar and getVarFlag calls (again)
* True is default since 2016 and most layers were already updated
  not to pass this parameter where not necessary, e.g. oe-core was
  updated couple times, first in:
  https://git.openembedded.org/openembedded-core/commit/?id=7c552996597faaee2fbee185b250c0ee30ea3b5f

  Updated with the same regexp as later oe-core update:
  https://git.openembedded.org/openembedded-core/commit/?id=9f551d588693328e4d99d33be94f26684eafcaba

  with small modification to replace not only d.getVar, but also data.getVar as in e.g.:
  e.data.getVar('ERR_REPORT_USERNAME', True)

  and for getVarFlag:
  sed -e 's|\(d\.getVarFlag \?\)( \?\([^,()]*, \?[^,()]*\), \?True)|\1(\2)|g' \
      -i $(git grep -E 'getVarFlag ?\( ?([^,()]*), ?([^,()]*), ?True\)' \
          | cut -d':' -f1 \
          | sort -u)

(From OE-Core rev: 26c74fd10614582e177437608908eb43688ab510)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-02 09:50:02 +00:00
Richard Purdie
29afbb5e14 scripts/bitbake-prserv-tool: Fix to work with memres bitbake
Deleting data files whilst bitbake is running isn't a great idea.

Make sure memory resident bitbake is stopped if present and make
the file deletions a little more specific and safer too while we're
here.

Note: The cache directory layout is different on older releases (some
are split level) so this won't be a direct backport to some.

(From OE-Core rev: fa3f1f67abae1d90bede2e98bf7293603fd1307c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-01-29 19:24:22 +00:00
Xiangyu Chen
6bc30dd4a7 runqemu: add process of option QB_NFSROOTFS_EXTRA_OPT
This extra options to be appended to the nfs rootfs options in kernel
boot arg.

Example config with qemuppc64 in machine config:

add r/w size in the nfs rootfs extra option:
QB_NFSROOTFS_EXTRA_OPT = "wsize=524288,rsize=524288"

re-build and runqemu with nfs again, we can observe the kernel command
line added our defined value in QB_NFSROOTFS_EXTRA_OPT:

Kernel command line: root=/dev/nfs nfsroot=10.0.2.2:/home/xchen5/testing/build/tmp-glibc/deploy/images/qemuppc64/testnfs,nfsvers=3,port=3049,tcp,mountport=3048,wsize=524288,rsize=524288 rw  mem=256M ip=dhcp console=hvc0 console=hvc0 nohugevmalloc

(From OE-Core rev: 43a97f5bf3f90c5c1fd603f7dca2b3db2c0e3040)

Signed-off-by: Xiangyu Chen <xiangyu.chen@windriver.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-01-26 22:22:53 +00:00
Richard Purdie
c0981c9622 wic/efi-bootdisk.wks: Fix for 6.1 kernel versions
oe-selftest efibootpartition.GenericEFITest.test_boot_efi was failing for 6.1 kernels with:

| ERROR: _exec_cmd: export PATH=[...] mcopy -i [...]/rootfs_boot.1.vfat -s [...]/rootfs1/* ::/ returned '1' instead of 0
| output: Disk full

I believe we hit a file boundary size and having "0" overhead in the image meant
the files couldn't be installed. Allow a small amount of overhead to avoid
the error.

(From OE-Core rev: 16e0b8a8fc36f5525b1801888851958f0dbe84c2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-01-26 21:57:48 +00:00
Chuck Wolber
b57d4de337 scripts/oe-setup-layers: Make efficiently idempotent
The effect of subsequent setup-layers executions is now either a NOOP
or the minimal set of changes required to ensure layers precisely match
the JSON configuration.

This change allows setup-layers to be incorporated into a team's
configuration management strategy. In particular, the configuration
JSON manages a "pinning policy" that documents the oversight of sources
of change (a requirement for embedded development in highly regulated
industries).

One model for this strategy would work as follows. Team level policy is
developed to regularly review upstream commits that occur between the
current upstream HEAD and the previously pinned revision. The JSON
configuration is periodically updated after a review, test, and approval
process. In the rare instance that an upstream change is considered
problematic, the bbappend mechanism can be used to make relevant
changes in the team's project repository. This approach also requires
that team developers regularly run the project repository copy of
setup-layers. This is most easily accomplished by including setup-layers
in a wrapper script that all team developers use to interact with the
bitbake tool suite (e.g. "bb bitbake foo-image"). Project level policy
and oversight is effectively "contained" within this wrapper script,
thereby reducing a significant source of human error.

Left unstated, but acknowledged here, are a number of nuances required
to successfully implement the above strategy.  The details are out of
scope for this explanation. What should be clear is that a larger
configuration management strategy can now benefit from the utility
provided by setup-layers.

Note: Neither the above configuration management strategy example nor
the change itself is intended to alter the original intent to use
"bitbake-layers create-layers-setup destdir" to keep pace with upstream
activity for those who wish to use it that way.

(From OE-Core rev: da2e01cacd98715318a5307fe0618dbca0cf1fe7)

Signed-off-by: Chuck Wolber <chuck.wolber@boeing.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-01-18 16:42:28 +00:00
Alexander Kanavin
c9c2b6b613 devtool: process local files only for the main branch
devtool modify/upgrade are not currently equipped to handle conditional local files
in SRC_URI, and provide only the main no-override set in a workspace under
source/component/oe-local-files/ (this is done via meta/classes/devtool-source.bbclass).

On the other hand, updating the changes from workspace into a recipe
is run iteratively against all overrides; this works for patches (as they
all are directed into their own override branches in the workspace
git source tree), but breaks down when trying to match local files
in a workspace against local files in overridden SRC_URI lists, resulting in
bad recipe breakage.

(there's an additional twist here: existing code has a guard against this
but the guard relies on metadata in workspace .bbappend that is only there
in modify operations, but not upgrades. This commit replaces the guard
with a general check that will work everywhere).

Implementing multiple sets of local files is significant work; let's for now
simply not touch local files in recipes except when on the no-override variant.

Also, adjust the selftest cases to include conditional local files in sample
recipes, so the situation is covered by the tests.

(From OE-Core rev: 3a8654b860fa98f94e80c3c3fff359ffed14bbe7)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-31 17:10:21 +00:00
Mikko Rapeli
0d1023673b runqemu: add QB_SETUP_CMD and QB_CLEANUP_CMD
These enable running custom shell setup and cleanup commands
before and after qemu. Enables machine configurations to for
example run qemu with swtpm to emulate TPM devices.

Example config with meta-tpm2 based swtpm in machine config:

 * image recipe depens on swtpm-native to get the native tools to PATH,
   same handling as qemu itself

do_testimage[depends] += "swtpm-native:do_populate_sysroot"

 * startup commands for swtpm daemon, note that swtpm
   socket file has 107 character limit and absolute paths to build environment
   will not work

QB_SETUP_CMD = " \
   test -d '${IMAGE_BASENAME}_swtpm' || ( mkdir -p '${IMAGE_BASENAME}_swtpm' && \
       swtpm_setup --tpmstate '${IMAGE_BASENAME}_swtpm' --tpm2 --pcr-banks sha256 ); \
   swtpm socket --tpmstate dir='${IMAGE_BASENAME}_swtpm' \
         --ctrl type=unixio,path='${IMAGE_BASENAME}_swtpm/swtpm-sock' \
         --log level=40 --tpm2 -t -d \
"

 * qemu startup command in machine config is configured enable swtpm device

QB_OPT_APPEND += "-chardev socket,id=chrtpm,path='${IMAGE_BASENAME}_swtpm/swtpm-sock' -tpmdev emulator,id=tpm0,chardev=chrtpm -device tpm-tis-device,tpmdev=tpm0"

 * in this case, swtpm daemon stops automatically with qemu machine, but
   QB_CLEANUP_CMD could be used to kill a specific process and wipe
   temporary files

Now runqemu and testimage.bbclass can be used with swtpm.

(From OE-Core rev: d5c38964a4458aa31ec37810773ecc4f5d410dbe)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-21 10:16:31 +00:00
Alexander Kanavin
04d2a394ea devtool/upgrade: correctly handle recipes where S is a subdir of upstream tree
'devtool modify' writes additional settings to workspace .bbappend so that this
can be handled correctly, but 'devtool upgrade' does not. This adds the missing
settings.

In particular, local files should not anymore mysteriously disappear from
SRC_URIs on upgrades.

(From OE-Core rev: 0817aa5537a8d7cc9591c53dfaa1d225f4c327f7)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-18 19:48:00 +00:00
Richard Purdie
7458ad340f yocto-check-layer: Allow OE-Core to be tested
For unknown reasons we've never seemingly run the check layer script
against OE-Core itself. This isn't entirely straightforward as the core
layer is a bit of a special case, we can't for example compare signatures
against ourselve and we can't remove core from bblayers.conf.

Core does have distro, machine and software components too, in the case
of distro, our fallback default settings. Whilst the qemu machines could
be split into a seperate layer directory, core wouldn't then parse at all
standalone due to the lack of any machine so it seems a bit pointless to
do that.

These changes tweak the script to handle core's special cases, specifically
to allow distro and machine directories and to account for the README placed
a directory level higher than other layers.

(From OE-Core rev: ba312ed228507d05f280aeb96819d671b01400b8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-09 13:18:41 +00:00
Richard Purdie
9773050910 scripts/checklayer: Update to match bitbake changes
Bitbake additions for the addpylib API mean we need to update the parsing
function call to be clear we're parsing in configuration context.

(From OE-Core rev: ef7677dc90fac089f8b9f6da301cca022ed7284c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-08 11:54:39 +00:00
Richard Purdie
4d19594b8b devtool/friends: Use LAYERSERIES_CORENAMES when generating LAYERSERIES_COMPAT entries
It seems some layers want to subvert the intent of LAYERSERIES_COMPAT
so bitbake is going to have to become stricter about the values there.
To work with this, use LAYERSERIES_CORENAMES to generate the entries in
LAYERSERIES_COMPAT instead of the current magic LAYERSERIES_COMPAT_core
value which may not continue to work.

The downside to this is when migating between releases, people would
need to update devtool workspace layer.conf files. I guess you could
argue this is a feature!

(From OE-Core rev: 96ff9baa8ead57504f40f362ed3a4aaa776d1b58)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-08 11:15:47 +00:00
Ross Burton
85ec78cc24 combo-layer: add sync-revs command
When starting to use combo-layer, or if someone else is using it too,
the local last_revision may be incorrect.

This command will forcibly update the last_revision config values to the
latest SHA on the remote branch that is tracked.

(From OE-Core rev: 2bb5d12ecd1b0273983f7c05699f34dd64b11c25)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-08 10:50:30 +00:00
Ross Burton
cee485d167 combo-layer: dont use bb.utils.rename
Bitbake may not be configured, and bb isn't imported anyway.

Instead just use os.rename(), and take the filename from the file object
instead of duplicating logic.

(From OE-Core rev: 528f4fb3683d048537604e4562ea758968060d62)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-08 10:50:30 +00:00
Ross Burton
713d0e531b combo-layer: remove unused import
(From OE-Core rev: ebfab6c3034d41252d19c6e1a0ba79072aa51146)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-08 10:50:30 +00:00
Ross Burton
d9b5b8cb21 lib/buildstats: fix parsing of trees with reduced_proc_pressure directories
The /proc/pressure support in buildstats is creating directories in the
buildstats tree called reduced_proc_pressure, which confuses the parsing
logic as that cannot be parsed as a name-epoc-version-revision tuple.

Explicitly skip this directory to solve the problem.

(From OE-Core rev: 24f0331f0b7e51161b1fa43d4592b491d2037fe9)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-08 10:48:36 +00:00
Alexander Kanavin
18fc5fb41a runqemu: do not hardcode the ip address of the nfs server when using tap
Rather, set it similarly to the overall network config.

(From OE-Core rev: d5e2ef31b151825613b62e58034e81f72526a944)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-29 10:29:58 +00:00
Alexander Kanavin
dea0b2d2a9 qemu-helper: depend on unfs3 and pseudo directly
The eliminates having to do extra steps (e.g. meta-ide-support)
when booting an image with a nfs mount as rootfs - startng runqemu
with a nfs mount starts to 'just work' after building an image.

(From OE-Core rev: f73e370bec16d206592a7ca01b4a86b1d1316ada)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-29 10:29:58 +00:00
Alexander Kanavin
afa94425a0 unfs: update 0.9.22 -> 0.10.0
This is the first release in 13 years;
I have reviewed the accumulated patches, and dropped some of them
where purpose or issue being fixed is not clear. Specifically:

0001-Add-listen-action-for-a-tcp-socket.patch
0001-daemon.c-Libtirpc-porting-fixes.patch
fixed upstream in
84ab475f93

0001-attr-fix-utime-for-symlink.patch
addresses an open issue in
https://github.com/unfs3/unfs3/issues/4
please rebase and re-submit as a PR if the problem is still present.

alternate_rpc_ports.patch
unnecessary as of
https://git.yoctoproject.org/poky/commit/?id=6bb9860ef7ba9c84fe9bd3a81aa6555f67ebd38e
Command line options introduced by the patch no longer used anywhere.

fix_compile_warning.patch
merged upstream.

fix_pid_race_parent_writes_child_pid.patch
rebased and re-submitted upstream.

no-yywrap.patch
dropped as backport.

relative_max_socket_path_len.patch
needs to be re-submitted by the original author, purpose and reproducer scenario unclear.

rename_fh_cache.patch
merged upstream.

tcp_no_delay.patch
purpose and use case for oe unclear.

unfs3_parallel_build.patch
fixed upstream in
987d32ca12
a39a78995c

Drop -N option from oeqa nfs helper and runqemu helper;
the option was provided by tcp_no_delay.patch
and is not needed for the tests or qemu.

Drop ad hoc libtirpc support; upstream supports it directly now.

Drop the check for portmap/rpcbind, it is unnecessary as of
https://git.yoctoproject.org/poky/commit/?id=6bb9860ef7ba9c84fe9bd3a81aa6555f67ebd38e

(From OE-Core rev: fa2f7cf545137b071db97015bca5b70d77566cd8)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-29 10:29:58 +00:00
Mikko Rapeli
aa3cb188b8 runqemu: limit slirp host port forwarding to localhost 127.0.0.1
With default slirp port forwarding config qemu listens on TCP ports
2222 and 2323 on all IP addresses available on the build host. Most
use cases with runqemu only need it for localhost and it is not
safe to run qemu images with root login without password enabled
and listening on all available, possibly Internet reachable network
interfaces. Limit qemu port forwarding to localhost 127.0.0.1 IP
address. Now qemu machine SSH and telnet ports are only
reachable from the build host machine, not full Internet.

If qemu machine needs to be reachable from network, then it can
be enabled via local.conf or machine config variable QB_SLIRP_OPT:

QB_SLIRP_OPT = "-netdev user,id=net0,hostfwd=tcp::2222-:22"

(From OE-Core rev: c6b1e3d50bf2feea80b70a42c6fad868fa9e6042)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-20 08:29:07 +00:00
Joshua Watt
3e5ce0275a scripts: convert-overrides: Allow command-line customizations
Adds argument parsing to the conversion script so that the fields that
the script uses to do conversions can be customized on the command line.
The intention is to allows easier customization without having to fork
the script, and allow automated checking on 3rd party layers via CI
without false positives

(From OE-Core rev: b9551f9180bf9f13fb1c480b5b7892fdc831ffcd)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-14 16:19:42 +00:00
Sergei Zhmylev
be91408e10 wic: make ext2/3/4 images reproducible
Ext2/3/4 FS contains not only mtime, but also ctime, atime and crtime.
Currently, all the files are being added into the rootfs image using
mkfs -d functionality which affects all the timestamps excluding mtime.
This patch ensures these timestamps inside the FS image equal to
the SOURCE_DATE_EPOCH if it is set.

(From OE-Core rev: 75d2dd0ea7790db2e8ee921784ca373abff2df65)

Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-07 21:57:21 +00:00
ciarancourtney
d2a483338f wic: swap partitions are not added to fstab
- Regression in 7aa678ce804c21dc1dc51b9be442671bc33c4041

(From OE-Core rev: f1243572ad6b6303fe562e4eb7a9826fd51ea3c3)

Signed-off-by: Ciaran Courtney <ciaran.courtney@activeenergy.ie>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-29 16:28:36 +01:00
Joshua Watt
8b2348a75e runqemu: Do not perturb script environment
Instead of changing the script environment to affect the child
processes, make a copy of the environment with modifications and pass
that to subprocess.

Specifically, when dri rendering is enabled, LD_PRELOAD was being passed
to all processes created by the script which resulted in other commands
(e.g. stty) exiting with a failure like:

 /bin/sh: symbol lookup error: sysroots-uninative/x86_64-linux/lib/librt.so.1: undefined symbol: __libc_unwind_link_get, version GLIBC_PRIVATE

Making a copy of the environment fixes this because the LD_PRELOAD is
now only passed to qemu itself.

(From OE-Core rev: 2232599d330bd5f2a9e206b490196569ad855de8)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-26 12:28:39 +01:00
Sergei Zhmylev
0ee936da2e wic: honor the SOURCE_DATE_EPOCH in case of updated fstab
In case user requested to build a binary repeatable package,
it's required to honor the SOURCE_DATE_EPOCH environment
variable. So forcefully set mtime inside all the routines
which modify fstab in case it is updated.

(From OE-Core rev: 99719a3712a88dce8450994d995803e126e49115)

Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-26 12:28:39 +01:00
Joshua Watt
0cc7ac200d runqemu: Fix gl-es argument from causing other arguments to be ignored
The code to parse arguments was inadvertently skipping all arguments in
the elif block after gl-es if it was specified on the command line.

(From OE-Core rev: 718bb8d56f6a24c86e67830a7d13af54df2ebb4e)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-26 12:28:39 +01:00
Sergei Zhmylev
36b9f4f3d0 wic: implement binary repeatable disk identifiers
When SOURCE_DATE_EPOCH variable is set, binary repeatable build
is expected. This commit implements reproducable disk identifiers
in such a case using its value as a Random seed.

(From OE-Core rev: 2c0c54e12169e76f16fb3398904bc897a9190397)

Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-26 12:28:39 +01:00
Maxim Uvarov
2cd51cfe99 wic: bootimg-efi: implement --include-path
--include-path can be used for placing files on ESP,
like dtbs.

(From OE-Core rev: 46ca73f504b0792766dc4d4bbad38beb04ef4ed0)

Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-26 12:28:37 +01:00
Maxim Uvarov
5b0de8447e wic: add UEFI kernel as UEFI stub
Linux kernel can be compiled as UEFI stub and loaded directly
with UEFI firmware without grub or other UEFI shell.

Tested with wic file:
bootloader  --ptable gpt --timeout=0  --append="rootwait"
part /boot --source bootimg-efi --sourceparams="loader=uefi-kernel" \
 --ondisk sda  --fstype=vfat --label bootfs \
  --active --align 1024 --use-uuid
part / --source rootfs --fstype=ext4 --label rootfs \
  --align 1024 --exclude-path boot/ --use-label

(From OE-Core rev: b18025fc8a2dad963b6f4b697e24455f2511b279)

Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-26 12:28:37 +01:00
Ross Burton
7e4b96e911 scripts/oe-check-sstate: force build to run for all targets, specifically populate_sysroot
Since the commit "populate_sdk_base/images: Drop use of 'meta' class and
hence do_build dependencies"[1], builds of images or SDKs don't
recursively depend on the top-level do_build target.  This is typically
a good thing: images just depend on the packages themselves and those
dependencies already exist, but they don't need each recipes sysroot to
be populated.

However, eSDK generation is partly done via the script oe-check-sstate,
which does a 'dry-run' build of the target and collates all of the
sstate that is used.  With this commit the sstate that is used is a
fraction of what would be needed in the SDK, specifically there are no
sysroots populated during the build, so there are no sysroots in the
SDK.

This is obviously a problem, as the entire point of an eSDK is to
contain a sysroot.  Resolve this problem by forcing bitbake to run the
build task for all targets, so that all potentially needed sstate is
collated.

[YOCTO #14626]

[1] 41d7f1aa2c

Tested-by: Andrej Valek <andrej.valek@siemens.com>
(From OE-Core rev: 1b62344f919b5122f048b6409d09386d7d6dd3cd)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-25 13:42:02 +01:00
Ross Burton
4fd15f4e3a scripts/oe-check-sstate: cleanup
The scriptutils import isn't used, there's no need to run bitbake
in a shell environment, and invoke bitbake as a list instead of a
string.

(From OE-Core rev: 663aa284adf312eb5c8a471e5dbff2634e87897d)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-25 13:42:02 +01:00
Paul Eggleton
dce3019212 install-buildtools: support buildtools-make-tarball and update to 4.1
Support installing buildtools-make-tarball that is built in version 4.1
and later for build hosts with a broken make version. Also update the
default version values to 4.1.

(From OE-Core rev: 5d539268d0c7b8fad1ba9352c7f2d4b81e78b75c)

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-25 13:42:02 +01:00
Alexander Kanavin
569d4cd325 scripts/oe-setup-builddir: do not read TEMPLATECONF from templateconf.cfg
Also, write to it only if it does not already exist.

As discussed here[1], reading from templateconf.cfg serves no purpose:
it exists only if the build directory has already been initialized, and so
the scripts will not change anything in the build directory anyway.

My adjustment is to keep the file however; it is useful as a pointer
to the original template, which can be utilized by config management
tools in the future.

[1] https://lists.openembedded.org/g/openembedded-architecture/topic/template_handling_in_oe_core/93968540

(From OE-Core rev: bcfaeb8ed503ce1a5add3ec882f0e0c3bccfd3d2)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-29 21:15:50 +01:00
Ross Burton
e7619f7650 Revert "wic/plugins/images/direct: Allow changes in fstab on rootfs"
Commit 20d43a2 allows changes in fstab for the / mount. However, if the
fstab contains / already then this results in two entries for / in the
fstab, which is confusing and results in systemd producing errors on
boot:

  systemd-fstab-generator[11101]: Failed to create unit file
  /run/systemd/generator/-.mount, as it already exists. Duplicate entry
  in /etc/fstab?

Wic should be extended to support merging mount points in fstab, but as
we're about to release revert the patch that introduced this change as
it hasn't been in a release yet, and the issue which prompted the patch
in the first place can be revisited post-release.

[ YOCTO #14865 ]

(From OE-Core rev: 4a7b3ef0a69b7e08030519b712a6fe4dd199c571)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-28 08:01:11 +01:00