Commit Graph

3562 Commits

Author SHA1 Message Date
Paul Gortmaker
e9980b8b38 scripts: fix buildstats diff/summary hard bound to host python3
Somehow these two got left behind and hence on older hosts that
are using buildtools for a newer python - they will still fail.

(From OE-Core rev: f1b11b83a6dd991f393940741c930691463e6d53)

Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-05-05 11:07:26 +01:00
Sergei Zhmylev
6722f40600 wic: add support for proper kernel name to bootimg-pcbios
Use appropriate kernel image name instead of hard-coded vmlinuz for bootimg-pcbios plugin.

(From OE-Core rev: 8d9f00ba456fe76e0f4ef0e68ec64fc538c90d89)

Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-05-04 12:37:00 +01:00
Randolph Sapp
ef577b03d7 wic/bootimg-efi: if fixed-size is set then use that for mkdosfs
This is a bit of a compatibility issue more than anything. Some devices
get upset if the FAT file system contains less blocks than the
partition.

The fixed-size argument is currently respected by the partition creation
step but not by the file system creation step. Let's make it so the file
system respects this value as well.

(From OE-Core rev: 38e1a235f5eceade7c871f96dc97f6c384384c7b)

Signed-off-by: Randolph Sapp <rs@ti.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-27 14:41:31 +01:00
Pavel Zhukov
c1e24cee0e scripts/runqemu: Add possibility to disable network
Default network configuration requires tun/tap module and while being
usable it conflicts with tap devices created by VPN clients sometimes
and requires root permissions to use . While it's possible to work
this around it's not always feasible if network is not required
Add nonetwork option which can be specified if the network connectivity is
not needed and SDL/serial is enough to communicate with the image.

(From OE-Core rev: d4073dedbb234ff3c6bbebafc836fedf90d96569)

Signed-off-by: Pavel Zhukov <pazhukov@suse.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-20 11:56:06 +01:00
Pavel Zhukov
d84eb2a5e4 scripts/rpm2cpio.sh: Use bzip2 instead of bunzip2
bzip2 is in HOSTTOOLS already and used in few other places already.
This fixes bin_package class for RPM packages without adding bunzip2 to
HOSTTOOLS.

(From OE-Core rev: eb3ec7469fff857c819332371ad1d586f43c79c3)

Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-20 11:56:06 +01:00
Dit Kozmaj
18b04f9c3f wic: use part_name when defined
So far part.label has been used to define GPT partition label even if
part.part_name was defined.
Fix the code to use part.part_name whenever available, as it makes sense
to have a GPT partition label which is different from the contained
filesystem label.

(From OE-Core rev: 7704d5fc36eb065224792bf4d5543814eaa5fed3)

Signed-off-by: Dit Kozmaj <dit.kozmaj@kynetics.com>
Signed-off-by: Diego Rondini <diego.rondini@kynetics.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-13 11:56:07 +01:00
Martin Jansa
3f0acc2a61 runqemu: respect IMAGE_LINK_NAME
* when searching for qemuboot.conf
* don't assume that IMAGE_LINK_NAME is always
  <rootfs>-<machine> (with <rootfs>-<machine>.qemuboot.conf)

* runqemu: use IMAGE_LINK_NAME set by testimage.bbclass or query with bitbake -e

* testimage.bbclass was setting DEPLOY_DIR which I don't see used
  anywhere else, so I assume it was supposed to be DEPLOY_DIR_IMAGE as mentioned
  in corresponding runqemu code, do the same with IMAGE_LINK_NAME variable

* add virtual/kernel as bitbake -e target in run_bitbake_env to make
  sure IMAGE_LINK_NAME is defined (kernel-artifact-names.bbclass inherits
  image-artifact-names.bbclass as well)

* improve .qemuboot.conf search
  1st search for file matching the rootfs and only when not found
  try again with .rootfs suffix removed

[YOCTO #12937]

(From OE-Core rev: 716eb55bb963db7b02d985849cb025898aabc855)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-04 13:11:49 +01:00
Khem Raj
17c6c0d650 Revert "runqemu: Add workaround for APIC hang on pre 4.15 kernels on qemux86"
This reverts commit 82e67b82ea8e12aa0b7b9db1d84fec0436dec71b.

It was commited as part of https://bugzilla.yoctoproject.org/show_bug.cgi?id=12301
for kernels < 4.15, as of now oldest builder kernel we have is 4.15 on
ubuntu 18.04 so we should not require this workaround. Moreover, this
fixes an smp problem with qemux86 where no matter what -smp <x> option
is used, qemu always starts with single core.

(From OE-Core rev: fa8a7c0608fc800c48d0ff1cd832ad63c51eeab1)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-30 23:26:04 +01:00
Jose Quaresma
13d4b2a455 buildstats-summary: add an option to disable bold
(From OE-Core rev: b9a0ceebe9aa1e79d97508e7ab2fc39ca7c6637f)

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-28 22:28:45 +01:00
Andrew Geissler
e9e5a7910e filemap.py: enforce maximum of 4kb block size
The logic in this script validates that the length of data sections are
evenly divisible by the block size. On most systems the block size is
4KB and all is good. Some systems though, such as ppc64le, have a block
size larger then 4KB. For example on a POWER9 based ppc64le system, the
block size is 64KB.

This results in this script failing with errors like this when building
wic images:
|440, in _do_get_mapped_ranges
|     assert extent_len % self.block_size == 0
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| AssertionError

In this case the data section size was 268KB and the block size was
64KB, resulting in the above assert failure.

Resolves https://bugzilla.yoctoproject.org/show_bug.cgi?id=15075

(From OE-Core rev: 1e23b803af6991fc20e4a4e88a0ef0541399e722)

Signed-off-by: Andrew Geissler <geissonator@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-27 15:44:02 +01:00
Alexis Lothoré
aee5758336 scripts/yocto_testresults_query.py: fix regression reports for branches with slashes
Regression reports are not generated on some integration branches because
yocto_testresults_query.py truncates branches names with slashes when it passes
it to resulttool. For example, "abelloni/master-next" is truncated to "abelloni"

Fix this unwanted branch truncation by fix tag parsing in yocto-testresults

(From OE-Core rev: b2fb7d7b0c05bb198a2271bbf1742645ba220ea3)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-25 09:39:28 +00:00
Joshua Watt
7def843a55 runqemu: Fix TypeError when command fails
The commands passed to subprocess are tuples which when passed to a %
format are treated as multiple format arguments instead of a single
argument to be coerced by "%s". This results in a TypeError being
raised with python claiming that not all arguments were consumed.

Fix this by wrapping the command tuple in a str() call, as is done for
the logging strings

[YOCTO #15078]

(From OE-Core rev: 3e5d04d9ebbee4e11fb39bf353b6d4c3133e166a)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-25 09:39:28 +00:00
Ross Burton
41de1ecadd scripts/lib/buildstats: handle top-level build_stats not being complete
If we try to parse a buildstats directory which was either aborted or
is still being built then the top-level build_stats file doesn't
contain an elapsed value which causes an exception:

UnboundLocalError: local variable 'elapsed' referenced before assignment

Default both start and elapsed to 0 so that the parse succeeds.

(From OE-Core rev: 701d985aa8f2e9c2b9c0736fa25b424f3701889e)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-25 09:39:28 +00:00
Ross Burton
1908193b81 scripts: add buildstats-summary
This script will write a summary of the buildstats to the terminal,
sorted by start time or duration, optionally hiding short tasks, and
highlighting long running tasks.

(From OE-Core rev: 253d2c0eb048ea38822844ebb69ad76d55b5c3ef)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-23 22:36:46 +00:00
Jialing Zhang
bcd0a40030 Do not remove the -m option for loongarch64
(From OE-Core rev: 6f3583675d31b74a3ae1c7fae450ea1624acc2e7)

Signed-off-by: Jialing Zhang <zhangjialing@loongson.cn>
Signed-off-by: Qizheng Zhu <zhuqizheng@loongson.cn>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-23 22:36:46 +00:00
Alexander Kanavin
44375eccf1 runqemu: direct mesa to use its own drivers, rather than ones provided by host distro
With mesa 23.0, it is not longer possible to use the host drivers, as
mesa upstream has added strict checks for matching builds between
drivers and libraries that load them.

Add a check and a hint to runqemu so that there is a helpful error when
there is no native/nativesdk opengl/virgl support.

(From OE-Core rev: f0946844df7270fe368858d8929e6b380675b78b)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 14:47:29 +00:00
Martin Jansa
92c9dfe1d3 runqemu: get_first_file() rename cmd* to glob*
* to better indicate how it's used in get_first_file

* cmd* is used in other places for actual shell commands
  to execute

* RunQemuError('KERNEL not found: %s, %s or %s' % cmds)
  also looked weird to me, but that works (to my python-noob surprise)

[YOCTO #12937]

(From OE-Core rev: 7c26e9dcc999a7d6a365831c39d25d98890be6d0)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:53:29 +00:00
Richard Purdie
bedf823032 resulttool: Improve overlapping ptest result reporting
With the move to a ptest per image, the multiple ptest log scenario
is much more likely. Tweak the handling to only warn if there are
overlapping files.

(From OE-Core rev: 8391f1668fcbe932ba846299b0ec22de09e06dd4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-16 22:52:15 +00:00
Richard Purdie
f26158603b pybootchart: Fix extents handling to account for cpu/io/mem pressure changes
The previous addition of pressure values to the chart didn't fix the extents
function which meant the bottom of the chart was cut off. Fix that.

(From OE-Core rev: cc8cef69e717e08f80d10f775f0fffc644267b59)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-16 08:13:12 +00:00
Richard Purdie
539ec94f17 pybootchartui: Fix python syntax issue
Fix:

scripts/pybootchartgui/pybootchartgui/parsing.py:134: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if pid is 0:

(From OE-Core rev: c9a6511ae618035b8efad01646e37ba28ce1e3f8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-16 08:13:12 +00:00
Richard Purdie
37b5d544d3 scripts/combo-layer: Fix python deprecation warning
Address:
DeprecationWarning: 'pipes' is deprecated and slated for removal in Python 3.13

pipes.quote is an alias for shlex.quote so switch to that.

(From OE-Core rev: 7b1c1dd9985a6f1645271a928dda7f1897a7ba8a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-14 17:13:11 +00:00
Ross Burton
9db9f8c575 resulttool: add log --list-ptest
Add a convenience argument to the log subcommand to list all of the
ptest logs in a testresults file.

(From OE-Core rev: cd2d7adf02005d46f19c93a40db1e10ce01ac261)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-14 17:13:11 +00:00
Ross Burton
37ab2ecbb7 lib/resulttool: fix typo breaking resulttool log --ptest
ptestresult_get_log() looked for a key called 'ptestresuls.sections',
which should be 'ptestresult.sections'

(From OE-Core rev: 7c8c9f7283e54bf8b1521fbaad7dceb66a8fcdbb)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-14 17:13:11 +00:00
Clément Péron
1f6b7ec6a4 runqemu: add an option to enable guest-agent virtio device
Add support to the runqemu script for a new option, 'guestagent', that
enables the virtio serial port for host-to-guest communication.

(From OE-Core rev: 21a1e52079089c5bbeee8ffc9c504471f4a8732a)

Signed-off-by: Brenda Streiff <brenda.streiff@ni.com>
Signed-off-by: Clément Péron <peron.clem@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-14 17:10:00 +00:00
Alexander Kanavin
31bdac666f devtool/upgrade: do not delete the workspace/recipes directory
If it exists, there is no need to delete it, and if it does not,
devtool prints an ugly traceback.

(From OE-Core rev: af82e59e8f08369aabd5fa6eb43022982d4e59a7)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-14 17:10:00 +00:00
Alexis Lothoré
4617ef6abf scripts/yocto_testresults_query.py: set proper branches when using resulttool
The script currently only works if base and target can be found on default
branches. It breaks if we try to generate a regression report between revisions
that live on different branches (as needed on integration and testing branches).
For example, the following command:

./scripts/yocto_testresults_query.py regression-report yocto-4.0.6 yocto-4.0.7

ends with the follwing error:

[...]
ERROR: Only 1 tester revisions found, unable to generate report
[...]

Read branches from tags names in test results repository, and pass those
branches to resulttool when generating the report

(From OE-Core rev: 6c472b326bcc718459483cc29b310b884742df86)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-12 23:39:13 +00:00
Alexis Lothoré
1eae53e277 scripts/resulttool: do not count newly passing tests as regressions
resulttool regression module simply compare a base test status to a target test
result status. This approach raises many false positives since all XXX -> PASS
transitions (XXX being any status different from PASS) are flagged as
regression.

- Do not list XXX -> PASS transitions in regression  report, instead count them
  and print a summary of "newly passing tests"
- If an inspected pair has only "newly passing tests", do not print detailed
  list and print it as "Improvement" instead of "Regression"

Updated output example looks like the following:
[...]
Improvement: oeselftest_fedora-37_qemux86-64_20230127010225
             oeselftest_ubuntu-22.04_qemux86-64_20230226120516
             (+1 test(s) passing)
[...]
Match:       oeselftest_almalinux-8.7_qemuarm64_20230127015830
             oeselftest_almalinux-8.7_qemuarm64_20230227015258
[...]
Regression:  oeselftest_almalinux-9.1_qemumips_20230127000217
             oeselftest_opensuseleap-15.4_qemumips_20230226130046
    ptestresult.glibc-user.debug/tst-read-chk-cancel: PASS -> None
    ptestresult.glibc-user.nptl/tst-mutexpi4: PASS -> FAIL
    ptestresult.glibc-user.nptl/tst-mutexpi5a: PASS -> FAIL
    Additionally, 44 previously failing test(s) is/are now passing

(From OE-Core rev: c335f96f687c73fde443ac330ca3e17113794d9e)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-12 23:39:13 +00:00
Alexis Lothoré
1feffaf41a scripts/resulttool: fix ptests results containing a non reproducible path
Some ptests results may be wrongly sampled, resulting in some part of the error
being propagated in test name. For example:

"ptestresult.binutils-ld.in testcase /home/pokybuild/yocto-worker/qemumips/build/build-st-1666126/tmp/work/mips32r2-poky-linux/binutils-cross-testsuite/2.40-r0/git/ld/testsuite/ld-ctf/ctf.exp"
"ptestresult.gcc.Couldn't create remote directory /tmp/runtest.455781 on ssh"
"ptestresult.gcc-libstdc++-v3.Couldn't create remote directory /tmp/runtest.3814266 on target"

While the root errors must be fixed in corresponding ptests packages for those
tests, the test results history must still be usable even with this issue

Add new filters to detect if temporary test directories (build-st-*,
/tmp/runtime.*) are present in name. If so, truncate test name. As a side
effect, it will aggregate multiple failing errors into one, but the regression
will still be raised

(From OE-Core rev: 601eecfddd26bfe2954835a73ed1116bb520235f)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-12 23:39:13 +00:00
Alexis Lothoré
e8048a5b66 scripts/resulttool: call fixup_ptest_names in regression_common
ptests names not only need to be fixed for regression based on git testresults
but also for testsresults provided "manually"

Move ptests naming fixup in regression_common to share the fixup between both
regression use cases

(From OE-Core rev: f772ccd108dc3d618db9d479d672c0f3edd203ca)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-12 23:39:12 +00:00
Ross Burton
d42904ba0c manpages: use an intercept to run mandb
If you build an image with lots of manpages in, then each package will
run mandb inside qemu-user at rootfs time.  This is a slow operation
and should be done once when all of the packages have installed using an
intercept instead.

The call to mandb has been changed too. mandb doesn't actually allow
the configuration file to be read from stdin so that was being ignored,
instead write the file to a temporary file and use that.

This means we then don't need to tell it where to search explicitly, and
it writes the indexes to the correct paths so we don't need to move
files afterwards either.

Sadly we do still need to run mandb inside qemu-user, as the underlying
database is a gdbm file and they are byte-order dependent.

For my test case of core-image-base with api-documentation
DISTRO_FEATURES and doc-pkgs IMAGE_FEATURES enabled, the performance
gain is significant:

  core-image-base do_rootfs  -1303.1s   -73.6%   1771.6s -> 468.5s

(From OE-Core rev: fbd8a57aa307bfda70a08cb78af3c97f05c39a3a)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-08 10:21:05 +00:00
Alexis Lothoré
d0f32772bb scripts/yoct_testresults_query: manage base/target revision not found
If yocto_testresults_query.py is run from oe-core instead of poky, the
script will very likely fail since poky tags do no exist in oe-core. If
one or both revisions are not found, log the error and a suggestion
about the reason (the script being run in oe-core instead of poky)

(From OE-Core rev: 758ac050ffd91524d400c196865b1ed27ece8776)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-28 07:53:54 +00:00
Alexander Kanavin
5a5ab7cdec scripts/runqemu: move render nodes check to runqemu from selftest
This will produce a more useful hint for those setting
up or testing virgl headless:

runqemu - ERROR - No render nodes found in /dev/dri: ['by-path', 'card0']. If /dev/dri/renderD* is absent due to lack of suitable GPU, 'modprobe vgem' will create one suitable for mesa llvmpipe software renderer.

as qemu itself isn't helpful:

alex@Zen2:/srv/storage/alex/yocto/build-64-alt$ qemu-system-x86_64 -display egl-headless
qemu-system-x86_64: egl: no drm render node available
qemu-system-x86_64: egl: render node init failed

(From OE-Core rev: cbbada6a6c9b0a2e97f7395117dad986555f2db9)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-27 13:27:37 +00:00
Richard Purdie
c45d58f003 resulttool/regression: Improve matching of poor ptest test names
Some test case naming is poor and contains random strings, particularly
lttng/babeltrace but also curl. Truncating the test names works since they
contain file and line number identifiers which allows us to match them
without the random components, or in the case or curl, test IDs.

Going forward we may be able to improve the test names but this
tweak allows historical test results to work in reports.

(From OE-Core rev: 541a2e2683531355e678fd93524a0c4a8c43a8ff)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-26 11:59:52 +00:00
Richard Purdie
401d022a26 resulttool/regression: Ensure LTP results are only compared against other LTP runs
If a test result contains LTP test results, it should only be compared with
other runs containing LTP test results.

(From OE-Core rev: 4dbbf2f4a85620a08dc2fa65095dc17fe6c530f8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-26 11:59:52 +00:00
Alexis Lothoré
7a70dcfea7 scripts: add new helper for regression report generation
Add yocto-testresults-query script. This is a thin wrapper over resulttool which
is able to translate tags or branch name to specific revisions, and then to work
with those "guessed" revisions with resulttool

(From OE-Core rev: b1460201b0f3d8fd7f977ac82f218bf9010d5573)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-26 11:59:52 +00:00
Alexis Lothoré
383cd86595 scripts/resulttool/regression: add metadata filtering for oeselftest
When generating regression reports, many false positive can be observed since
some tests results are compared while the corresponding tests sets are not the
same, as it can be seen for example for oeselftest tests (oeselftest is run
multiple time but with different parameters, resulting in different tests sets)

Add a filtering mechanism in resulttool regression module to enable a better
matching between tests. The METADATA_MATCH_TABLE defines that when the TEST_TYPE
is "oeselftest", then resulttool should filter pairs based on
OESELFTEST_METADATA appended to test configuration. If metadata is absent from
test results, in order to keep compatibility with older results, add a
"guessing" mechanism to generate the missing OESELFTEST_METADATA. The guessed
data is tightly coupled to the autobuilder configuration, where all oe-selftest
executions are described

(From OE-Core rev: 94ab7c2b892bf292dd86619ca9c63ddd7bf53f3c)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-26 11:59:52 +00:00
Alexis Lothoré
88345ecf47 scripts/resulttool/regression: remove unused import
(From OE-Core rev: 4d6a74fcee1479562bf66360f20d528107c70887)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-26 11:59:52 +00:00
Pavel Zhukov
4add8776ab wic: Fix populating of IMAGE_EFI_BOOT_FILES with uefi-kernel
wic plugin crashed if IMAGE_EFI_BOOT_FILES is not empty and uefi-kernel
loader specified because of preliminary return from the function.
[Yocto #15033]
Fixes:
|   File "/mnt/builds/yocto/sources/scripts/lib/wic/plugins/source/bootimg-efi.py", line 371, in do_prepare_partition
|     for src_path, dst_path in cls.install_task:
|                               ^^^^^^^^^^^^^^^^
| AttributeError: type object 'BootimgEFIPlugin' has no attribute 'install_task'. Did you mean: 'do_install_disk'?

(From OE-Core rev: a6372f70f31b39ce9867b705d02205621d7a8685)

Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-26 11:49:00 +00:00
Chee Yang Lee
80a2028813 checklayer: check for patch file upstream status
yocto-check-layer to check all .patch file in layer for
Upstream-status and list down all .patch file without Upstream-Status.

set this test as expected failure for now, so this wont fail the check
layer while still able to capture any patches with malformed or missing
Upstream-Status in report.

[YOCTO #14642]

(From OE-Core rev: 237c1b66e5014123c1e5c3e78f9ab0357bcd62dc)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-24 17:05:24 +00:00
Alexander Kanavin
ca7306f78a devtool: ignore patch-fuzz errors when extracting source
So that patch fuzz issues can actually be fixed,
as extracting source with 'devtool modify' is the first step
for that.

(From OE-Core rev: 7067abd31d9dd4b98ec70c1c7effbe2904797cd1)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-24 13:31:45 +00:00
Mikko Rapeli
bfb4d48d7c runqemu: kill qemu if it hangs
qemu doesn't always behave well and can hang too.
kill it with force if it was still alive. Move clean up
commands into cleanup() function.

(From OE-Core rev: 079c2935d2f585ce49e1c7daab2155fcf0094c48)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-23 12:11:29 +00:00
Alexander Kanavin
46c6a8d63c scripts/oe-setup-layers: do not clobber json module with a variable named 'json'
(From OE-Core rev: 91e5aef9fd710b08ead9491f1900fbab5386b157)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-20 15:18:30 +00:00
Alexander Kanavin
e8b88168f5 scripts/oe-setup-layers: correct variable names - layers should be called repos
The script is operating on layer repositories, which can and do sometimes contain
several layers. This distinction is important as the script will be tweaked
to write a record of actual layer locations.

(From OE-Core rev: 833965e6001db98039c0aa816ae661232213bcea)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-20 15:18:30 +00:00
Ross Burton
163aa6f149 lib/buildstats: handle tasks that never finished
If a task is aborted the buildstats file isn't complete, so calculate
when the build finished and use that as a end time.

(From OE-Core rev: 23ebaec476dc46aebe5997f025661137f3e341bd)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-17 18:01:01 +00:00
Alexander Kanavin
d369a45ae7 meta/conf: move default configuration templates into meta/conf/templates/default
This sets the ground for standardizing (and enforcing) the location of
configuration templates: let's start with the default one.

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-16 17:11:11 +00:00
Alexander Kanavin
4322056708 scripts/oe-setup-layers: print a note about submodules if present
(From OE-Core rev: 7f16c1c0229110c242f2e064d612d0d35823bf8d)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-15 10:21:34 +00:00
Pavel Zhukov
7f6a17a19c wic: Fix usage of fstype=none in wic
This allows to specify partition with fstype=none in the wks file
to have partition created but without following mkfs. The none fstype
is in the list already but the usage is not documented.

Example;
part /data --ondisk mmcblk0 --fstype=none  --align 4096 --fixed-size 512

will create a partition, filesystem may be created manualy on the host
or target and data will be preserved if the device is reflashed using
same wks. Works with bmaptool and probably does not work with dd.
Use case is persistent filesystem/data between reflashing of the image.

(From OE-Core rev: 351cb64da37aa43113e5192605d04436652aa3b8)

Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-15 10:21:34 +00:00
Xiaotian Wu
324b03e15f base: add support for loongarch64
(From OE-Core rev: 759baaceb4dd623d5da12ba0d01540fa080154ba)

Signed-off-by: Xiaotian Wu <wuxiaotian@loongson.cn>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-04 17:02:10 +00:00
Martin Jansa
94149c4f37 meta: remove True option to getVar and getVarFlag calls (again)
* True is default since 2016 and most layers were already updated
  not to pass this parameter where not necessary, e.g. oe-core was
  updated couple times, first in:
  https://git.openembedded.org/openembedded-core/commit/?id=7c552996597faaee2fbee185b250c0ee30ea3b5f

  Updated with the same regexp as later oe-core update:
  https://git.openembedded.org/openembedded-core/commit/?id=9f551d588693328e4d99d33be94f26684eafcaba

  with small modification to replace not only d.getVar, but also data.getVar as in e.g.:
  e.data.getVar('ERR_REPORT_USERNAME', True)

  and for getVarFlag:
  sed -e 's|\(d\.getVarFlag \?\)( \?\([^,()]*, \?[^,()]*\), \?True)|\1(\2)|g' \
      -i $(git grep -E 'getVarFlag ?\( ?([^,()]*), ?([^,()]*), ?True\)' \
          | cut -d':' -f1 \
          | sort -u)

(From OE-Core rev: 26c74fd10614582e177437608908eb43688ab510)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-02 09:50:02 +00:00
Richard Purdie
29afbb5e14 scripts/bitbake-prserv-tool: Fix to work with memres bitbake
Deleting data files whilst bitbake is running isn't a great idea.

Make sure memory resident bitbake is stopped if present and make
the file deletions a little more specific and safer too while we're
here.

Note: The cache directory layout is different on older releases (some
are split level) so this won't be a direct backport to some.

(From OE-Core rev: fa3f1f67abae1d90bede2e98bf7293603fd1307c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-01-29 19:24:22 +00:00