Commit Graph

3475 Commits

Author SHA1 Message Date
Michael Opdenacker
79cd9301bd lib/oe/qa: remove obsolete "Accepted" string for Upstream-Status
Replaced by "Backport" since release 3.2
https://docs.yoctoproject.org/migration-guides/migration-3.2.html#miscellaneous-changes

(From OE-Core rev: 4053fa9184fd016cfa59a67ff7857e17d88bf25e)

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-22 07:45:17 +01:00
Trevor Gamblin
882efa1e45 patchtest/selftest: only split resultlines once
selftest assumes that the test result output lines will consist of two
tokens separated by whitespace, which are the actual result and the name
of the test run. As a result, the script fails if any verbosity is added
to the output of patchtest itself (e.g. by including a failure reason).
Make the call to split() only do so once in order to ensure that two
tokens are obtained.

(From OE-Core rev: b7ff1e7363261780c015a3ee3b6d4bc69567f67f)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-22 07:45:17 +01:00
Ross Burton
e5829e1e8b oeqa/runtime/parselogs: inline single-caller functions
There's no need to have one-liner functions to get the MACHINE or
WORKDIR when they're only called once.

(From OE-Core rev: 9478a665641f55cdc14f12a4409121ef95883d74)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-22 07:45:17 +01:00
Ross Burton
1ef74b08ca oeqa/runtime/parselogs: remove obsolete LSB testing support
The LSB compliance tests were removed in 2019[1], so this is obsolete.

[1] oe-core fb064356

(From OE-Core rev: a465b61bc3a16152f90032d5d5a041f87bf30108)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-22 07:45:17 +01:00
Ross Burton
b6d19e347d oeqa/runtime/parselogs: don't bother to show target hardware information
This information is pretty useless as we know what the target is from
the build configuration (be it a qemu machine or real hardware).

(From OE-Core rev: 7821c42172191e3c16ab8662a9e4f7e921830db0)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-22 07:45:17 +01:00
Ross Burton
c105791979 oeqa/runtime/parselogs: remove unused imports
(From OE-Core rev: 5c2b6ce30b3cbec6236b99ce4019dbd443ccccae)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-22 07:45:17 +01:00
Richard Purdie
b0b0b89ad7 create-spdx/sbom: Ensure files don't overlap between machines
Currently the by-id and by-namespace SPDX files are created without reference
to PACKAGE_ARCH. This means that for two machines using a common package architecture
(e.g. genericx86-64 and qqemux86-64), there would be overlapping files. This means
that the build of one can remove files from the other leading to build failures. An
example would be:

MACHINE=qemux86-64 bitbake core-image-minimal
MACHINE=genericx86-64 bitbake core-image-minimal
MACHINE=qemux86-64 bitbake linux-yocto -c clean
MACHINE=genericx86-64 bitbake core-image-minimal -C rootfs

To fix this, add PACKAGE_ARCH to the path used for the files and use a search
path based upon PACKAGE_ARCHS to access them.

(From OE-Core rev: b2db10e966438071d00d2057b84d5f347613d841)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-22 07:45:17 +01:00
Richard Purdie
a83a16b793 oeqa/selftest/bbtests: Improve and update test_non_gplv3
This test no longer worked with fixes to avoid overlapping license files
in deploy. Fix the test but also improve the logging messages so we don't
get "False is not True" messages upon failure.

(From OE-Core rev: 2fc7714dab5cce585bce4d6c4e8ffbefce2a709b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-20 23:51:11 +01:00
Richard Purdie
2564f3a432 oeqa/selftest/wic: Improve assertTrue calls
assertTrue is a problematic call use in test cases since when it fails,
you just get an unhelpful "False is not True" message.

Replace some uses with assertIn/assertNotIn which will give more helpful results
and for the rest, add msg entries which given more helpful debugging.

For example, this patch would help debugging of #15176.

(From OE-Core rev: 35d4c39e0df1a304f557471151a03d1e4b0f30c7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-20 08:57:26 +01:00
Alexander Kanavin
16bf55b8fb go-helloworld: update to latest revision
Fix up test case to match what binary prints.

(From OE-Core rev: 4683804b9669c71d31ea6a8a300e6e87e817ee12)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-18 08:53:10 +01:00
Alexander Kanavin
4d1461595a ptest: report tests that were killed on timeout
I'm not sure if this was reported correctly before, but it
currently is not. Test that is stuck is an error in itself.

(From OE-Core rev: 002e27c9932a83e46be0b03a5232594cfba7212c)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-18 08:53:10 +01:00
Eilís 'pidge' Ní Fhlannagáin
9ccde59c53 lib/oe/package_managegment: Add nativesdk-intercept PATH
[YOCTO #15023]

This patch adds (and removes after function execution) the
nativesdk-intercept/chown|chgrp PATH before target_pm.run_intercepts
calls during populate_sdk builds.

This has been tested with cleanall builds and testsdk and fails on deb
due to an issue where $D${localstatedir}/cache/man/ does not exist for
some reason. I've a work around for that in the next patch in this
series.

(From OE-Core rev: e7afdfe9da150209ab2676d09eae040de2155c6d)

Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-17 19:37:19 +01:00
Mikko Rapeli
de863bc1fa oeqa/selftest/context.py: check git command return values
Don't ignore return values from the git command lines. If something goes
wrong, fail the test right away.

(From OE-Core rev: dfc178a70d6fa60e89d4716f05d68e2c72c6ecd3)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-14 21:57:16 +01:00
Trevor Gamblin
0f5691f23c patchtest: Add README.md for selftests
Add a short README describing how to setup patchtest's selftests for
oe-core.

(From OE-Core rev: afd4f3d9fa22dd0fbb0c30bbfc4a3de37d695c76)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-14 15:20:08 +01:00
Trevor Gamblin
9788cccca7 patchtest: add requirements.txt
Add a requirements.txt file with the Python modules needed for Patchtest
to run.

(From OE-Core rev: 2fd953ac5dd86d3045a6af240d61c6e36c62d133)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-14 15:20:08 +01:00
Trevor Gamblin
0cf17394f8 patchtest/selftest: remove configurable target
The ability to pass the target (i.e. oe-core) as an argument was a
testing mechanism and isn't needed when the tests are part of the repo,
so remove it and use os.path.dirname to get it instead.

(From OE-Core rev: 87c54eae350e358f32b12ae807719fa845fd54af)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-14 15:20:08 +01:00
Trevor Gamblin
4a6f38c532 patchtest: Add tests from patchtest oe repo
Copy the core components of the patchtest-oe repo into
meta/lib/patchtest in oe-core.

(From OE-Core rev: 257f64f4e4414b78981104aec132b067beb5a92a)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-14 15:20:08 +01:00
Richard Purdie
0669b9f6c7 oeqa/selftest/runtime_test: No need to use append with TEST_RUNQEMUPARAMS
A simple += operation is fine for working with TEST_RUNQEMUPARAMS so remove
the heavier append override usage.

(From OE-Core rev: e1a12c1212013823b44aff88317e5b30df03a35c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-13 09:32:45 +01:00
Adrian Freihofer
843c6c2e05 oeqa/selftest/devtool: Refactor runqemu pre-requisites
Split runqemu pre-requisites into a function which can be re-used by
other tests as well.

(From OE-Core rev: 020a51769439f173980315f15ad64bdace8c22b2)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-12 12:44:47 +01:00
Adrian Freihofer
b31f99a6cb lib/oe/package: Refactor to make strip_execs callable without d
This allows to call strip_execs function from devtool without going
via tinfoil and a bitbake server process.

(From OE-Core rev: 3bde26d64a0c8c3ef8ffbcb398f2a268759321af)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-12 12:44:47 +01:00
Adrian Freihofer
40fdebda20 lib/oe/utils: Refactor to make multiprocess_launch callable without d
This is a preparation for making the strip_execs function callable from
devtool without going via tinfoil and a bitbake server process.

(From OE-Core rev: af8ee73cdef90b83556a7ac5e139a08108706486)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-12 12:44:47 +01:00
Mikko Rapeli
48c80dcb15 selftest runtime_test.py: add testimage.bbclass slirp test
Builds core-image-minimal with testimage.bbclass and an ssh server
and then runs tests with slirp networking. Default test suite must
pass.

With full caches on x86_64 with x86_64 target the test passes in 416 seconds:

2023-08-21 09:01:12,295 - oe-selftest - INFO - Ran 1 test in 464.955s
2023-08-21 09:01:12,295 - oe-selftest - INFO - OK
2023-08-21 09:01:19,648 - oe-selftest - INFO - RESULTS:
2023-08-21 09:01:19,648 - oe-selftest - INFO - RESULTS - runtime_test.TestImage.test_testimage_slirp: PASSED (416.62s)
2023-08-21 09:01:19,711 - oe-selftest - INFO - SUMMARY:
2023-08-21 09:01:19,711 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 464.956s
2023-08-21 09:01:19,711 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0)

(From OE-Core rev: a93147278ee92b6aba281ab2b4b7b9d79dc1c89f)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-12 12:37:56 +01:00
Mikko Rapeli
f446f46415 oeqa selftest runtime_test.py: append to TEST_RUNQEMUPARAMS
Don't overwrite TEST_RUNQEMUPARAMS since user may have added "slirp"
or similar things to it in site.conf, auto.conf or local.conf. Use
:append to add to it. Fixes failing tests when only "slirp" networking
works with qemu and tun based one is broken on the host machine.

(From OE-Core rev: 5c1d94aa207e5c9a884bf3104eb9e8d1f8b6e432)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-12 12:37:56 +01:00
Richard Purdie
6fd8af0d30 classes: Drop ';' delimiter from ROOTFS/IMAGE*COMMAND variables
Originally these were shell functions but they have long since been processed by
bb.build.exec_func(). Since we no longer need shell syntax, we can drop the ';'
delimiters and just use a space separated string.

This cleans up the variable and quietly removes any stray ';' that do happen to
still make it in.

(From OE-Core rev: c3365dfd9ddd7fbe70b62e0f11166e57a8ca6f73)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-09 22:14:41 +01:00
Alexander Kanavin
63e53fb8b6 build-sysroots: target or native sysroot population need to be selected explicitly
Running them in parallel is prone to races as postinsts from target sysroots
rely on executables from native sysroots which may or may not be fully prepared
yet. This was observed for example here:

https://autobuilder.yoctoproject.org/typhoon/#/builders/146/builds/468/steps/12/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/147/builds/467/steps/12/logs/stdio

(From OE-Core rev: 38d7a2e45b883cf999a86af05bcc0eaa875bb47c)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-09 12:04:55 +01:00
Richard Purdie
a8f3ebc4e0 Revert "oeqa selftest context.py: fix git commands and set branch name"
We have autobuilder workers which don't have the -b option unfortunately.

This reverts commit 5ec557467dda29309e25102b507bb919275bedbb.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-07 23:26:14 +01:00
Mikko Rapeli
d95c3d3b18 oeqa selftest context.py: fix git commands and set branch name
Check return values of each git command and set initial branch name
to avoid this warning on every test run:

NOTE: Starting bitbake server...
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint:   git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint:   git branch -m <name>

(From OE-Core rev: 5ec557467dda29309e25102b507bb919275bedbb)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-07 14:36:30 +01:00
Mikko Rapeli
4bbcafa914 oeqa selftest context.py: remove warning from missing meta-selftest
It's not a warning but a handled case and layer gets added
automatically. Very few build configs have this layer enabled
by default.

(From OE-Core rev: 9a2493ea83f0b30578a819de05108502aaadc7f5)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-07 14:36:30 +01:00
Mikko Rapeli
72d34fa0ab oeqa selftest context.py: whitespace fix
(From OE-Core rev: 8aed3ca405cb3cc5d346eb8da8b3307d93f75d5b)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-07 14:36:30 +01:00
Alexander Kanavin
d3e4f25993 lib/oe/recipeutils.py: accommodate SRCPV being optional and deprecated in version check regex
(From OE-Core rev: 84794b59940cb08ec44e5ee7718a464d24136812)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-30 09:56:12 +01:00
Joshua Watt
cfa0ad6822 classes/image_types: Add vfat image type
Adds support for creating FAT formatted file system images (useful for
boot partitions on some SoCs).

Note that FAT partitions are limited in what they can represent (no
symlinks or device files), so they can't really be used for general
purpose root file systems. As such, they are skipped when testing for
that purpose.

(From OE-Core rev: 440fa508d362b8a449beb1b82dd999e980b753b7)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-30 08:43:03 +01:00
Julien Stephan
060f285e58 patch.py: use --absolute-git-dir instead of --show-toplevel to retrieve gitdir
[YOCTO #14141]

Currently the gitdir is manually constructed using `git
rev-parse --show-toplevel` and appending `.git`. This is most of the time
correct but not always: `.git` can be a file with the following content:

gitdir: <some_folder>

This is the case for submodules, so when using devtool modify on a recipe
using submodules *and* patching files inside one of the submodules, do_patch
fails with the following error:

ERROR: Error executing a python function in exec_func_python() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:patch_do_patch(d)
     0003:
File: '<..>/poky/meta/classes-global/patch.bbclass', lineno: 157, function: patch_do_patch
     0153:        except Exception as exc:
     0154:            bb.utils.remove(process_tmpdir, True)
     0155:            bb.fatal("Importing patch '%s' with striplevel '%s'\n%s" % (parm['patchname'], parm['striplevel'], repr(exc).replace("\\n", "\n")))
     0156:        try:
 *** 0157:            resolver.Resolve()
     0158:        except bb.BBHandledException as e:
     0159:            bb.utils.remove(process_tmpdir, True)
     0160:            bb.fatal("Applying patch '%s' on target directory '%s'\n%s" % (parm['patchname'], patchdir, repr(e).replace("\\n", "\n")))
     0161:
File: '<..>/poky/meta/lib/oe/patch.py', lineno: 769, function: Resolve
     0765:    def Resolve(self):
     0766:        olddir = os.path.abspath(os.curdir)
     0767:        os.chdir(self.patchset.dir)
     0768:        try:
 *** 0769:            self.patchset.Push()
     0770:        except Exception:
     0771:            import sys
     0772:            os.chdir(olddir)
     0773:            raise
File: '<..>/poky/meta/lib/oe/patch.py', lineno: 274, function: Push
     0270:            else:
     0271:                next = 0
     0272:
     0273:            bb.note("applying patch %s" % self.patches[next])
 *** 0274:            ret = self._applypatch(self.patches[next], force)
     0275:
     0276:            self._current = next
     0277:            return ret
     0278:
File: '<..>/poky/meta/lib/oe/patch.py', lineno: 556, function: _applypatch
     0552:        if os.path.lexists(hooks_dir_backup):
     0553:            raise Exception("Git hooks backup directory already exists: %s" % hooks_dir_backup)
     0554:        if os.path.lexists(hooks_dir):
     0555:            shutil.move(hooks_dir, hooks_dir_backup)
 *** 0556:        os.mkdir(hooks_dir)
     0557:        commithook = os.path.join(hooks_dir, 'commit-msg')
     0558:        applyhook = os.path.join(hooks_dir, 'applypatch-msg')
     0559:        with open(commithook, 'w') as f:
     0560:            # NOTE: the formatting here is significant; if you change it you'll also need to
Exception: NotADirectoryError: [Errno 20] Not a directory: '<..>/build/tmp/work/core2-64-poky-linux/vulkan-samples/git/devtooltmp-n87_zx1i/workdir/git/third_party/spdlog/.git/hooks'

Using `git rev-parse --absolute-git-dir` instead of `git rev-parse
--show-toplevel` ensure we get the correct gitdir

(From OE-Core rev: f74879dd95b19504ce8a8554636d2310d0336806)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-29 09:41:07 +01:00
Markus Niebel
4a56d0cc93 oeqa: wic: Add test for --no-table option
Add basic test for the --no-table wic part option.

(From OE-Core rev: ad89d7e6039da2d79c1d55fe7fdc8bb9c235dacf)

Signed-off-by: Markus Niebel <Markus.Niebel@ew.tq-group.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-29 09:41:07 +01:00
Richard Purdie
65318019cd recipes/classes/scripts: Drop SRCPV usage in OE-Core
Now that SRCPV isn't needed we can simplify things in a few places...

(From OE-Core rev: 843f82a246a535c353e08072f252d1dc78217872)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-24 16:50:24 +01:00
Richard Purdie
62afa02d01 base/package: Move source revision information from PV to PKGV
Source control information being present in PV used to be a hard requirement
for bitbake to operate correctly. Now that hashes are a required part of task
stamps, this requirement no longer exists.

This means we can defer the hash pieces to PKGV and simplify PV.

Use new bitbake fetcher API to inject the source revisions directly into the hash
allowing removal of some horrible code from base.bbclass and avoiding any hardcoding
about how SRCREV may or may not be used.

Use that API to object the string to append to PKGV and append that directly.

The user visible effect of this change is that PV will no longer have revision
information in it and this will now be appended to PV through PKGV when the
packages are written. Since PV is used in STAMP and WORKDIR, users will see
small directory naming and stamp naming changes.

This will mean that sstate reuse through hash equivalence where the source
revision changes but the output does not will become possible as the sstate
naming will become less specific and no longer contain the revision.

The SRCPV variable will no longer be needed in PV and is effectively now just
a null operation. Usage can be removed over time.

(From OE-Core rev: a8e7b0f932b9ea69b3a218fca18041676c65aba0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-24 16:50:24 +01:00
Alexis Lothoré
359a105de6 oeqa/utils/gitarchive: replace warning with info when reading local tags
Whenever a script needs to list tags, if it falls back to the third method
(reading local tags only), it emits a warning. While this warning is useful
for future diagnostic if some tagging issues re-appear, it makes buildperf
autobuilds status as "Passed with warnings", which is not desirable

(From OE-Core rev: b2c1d8a83a3b2134a8f9a445cbf00103e63ed0b3)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-24 13:34:34 +01:00
Alexis Lothoré
db6ace80a0 oeqa/utils/gitarchive: fall back to local tags when listing existing tags
e9cff55e73 has switched tag listing from bare "git tag" to "git
ls-remote" to make sure not to miss remote tags which are not fetched
locally. This mechanism first checks for configured remote repository, next
for possibly passed url, and then fails if none worked.

However there are still cases where no remote repository is
configured and no url is provided (for instance: buildperf tests use an
empty git directory to store tests). Fix those cases by putting back the
old behavior (local tags check) as last resort, with at least a warning for
future diagnostics if we still encounter tagging issues

Fixes: e9cff55e73 ("oeqa/utils/gitarchive: fix tag computation when creating archive")
(From OE-Core rev: 34e1f845687d2f7169f5d6c1bb54e1a7ab5412c4)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-23 22:33:39 +01:00
Alexis Lothoré
3c1ee6d8b0 oeqa/utils/gitarchive: allow to pass a logger to get_tags
Propagate a "log" parameter to get_tags in order to know what method is
used to retrieve existing tags

(From OE-Core rev: f8212eda45444dfb330e4d930eeceefde936adeb)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-23 22:33:39 +01:00
Jasper Orschulko
92983dba65 cve_check: Fix cpe_id generation
Use "*" (wildcard) instead of "a" (application)in cpe_id generation,
as the product is not necessarily of type application, e.g.
linux_kernel, which is of type "o" (operating system).

(From OE-Core rev: cae9528b002c06143bf048b991b9d7e93968cb6b)

(From OE-Core rev: e7c1def3c3c3a72249802ef6fb64292277a7a53e)

Signed-off-by: Jasper Orschulko <jasper@fancydomain.eu>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-22 15:13:54 +01:00
Luca Ceresoli
d40fed311e Revert "oeqa/runtime/parselogs: Exclude preempt-rt error for now"
There's now a real fix in commits "linux-yocto/6.*: fix IRQ-80 warnings".

This reverts commit 1451df346a0e2433714774421ce8f339a37a844a.

(From OE-Core rev: f5c1b0f2b17814315bc2a09acc3aaa20f2522cff)

Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-22 15:13:54 +01:00
Frederic Martinsons
37ebe9907a oeqa/runtime/rust: correct rust test
Since setUp of RustCompileTest use cls.tc.copyTo, those tests needs the scp command:

NOTE: ======================================================================
NOTE: FAIL: test_cargo_compile (rust.RustCompileTest)
NOTE: ----------------------------------------------------------------------
NOTE: Traceback (most recent call last):
  File "/home/jenkins/yocto-poky-master/poky/meta/lib/oeqa/core/case.py", line 53, in _oeSetUp
    self.setUpMethod()
  File "/home/jenkins/yocto-poky-master/poky/meta/lib/oeqa/runtime/cases/rust.py", line 17, in setUp
    cls.tc.target.copyTo(src, dst)
  File "/home/jenkins/yocto-poky-master/poky/meta/lib/oeqa/core/target/ssh.py", line 132, in copyTo
    return self._run(scpCmd, ignore_status=False)
  File "/home/jenkins/yocto-poky-master/poky/meta/lib/oeqa/core/target/ssh.py", line 81, in _run
    raise AssertionError("Command '%s' returned non-zero exit "
AssertionError: Command '['scp', '-o', 'ServerAliveCountMax=2', '-o', 'ServerAliveInterval=30', '-o', 'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no', '-o', 'LogLevel=ERROR', '-r', '-P', '2222', '/home/jenkins/yocto-poky-master/poky/meta/lib/oeqa/files/test.rs', 'root@127.0.0.1:/tmp/']' returned non-zero exit status 1:
sh: scp: not found
lost connection

(From OE-Core rev: 1bf24df9be44c73e5d8e90feb446ecfcd542228c)

Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-21 16:15:35 +01:00
Frederic Martinsons
5116f2b2cf rust: provide examples for C library generation in rust
rust-c-lib-example is a little rust code which provide
a single function to print a formatted date (via the chrono crate)
from an input timestamp in millisecond. It has the necessary FFI
annotation and inherit cargo_c class for the C ABI compatible
library generation.

rust-c-lib-example is meson project for the C code which
will call the print_date function from rust-c-lib-example
if no argument is provided, if any argument is provided
it will print "Hello world in rust from C!"

add a runtime test case to check if all went well.

(From OE-Core rev: bb177c7764b1bc47157d57d7a34930e59a7acef3)

Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-21 16:15:35 +01:00
Alexis Lothoré
9519610870 oeqa/selftest/gitarchive: add tests about tags lisiting when no remote is configured
Add specific tests on gitarchive for when tag listing is required but no
remote is configured in target directory: it should either succeed if valid
url is provided, or fail is url is not provided or wrong

(From OE-Core rev: 5e9d84d82fde81d66550d8c694ea70f0911fa4f7)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-21 11:34:12 +01:00
Alexis Lothoré
e9cff55e73 oeqa/utils/gitarchive: fix tag computation when creating archive
Sporadic errors have been observed in autobuilder when trying to store new
tests results:

error: failed to push some refs to 'push.yoctoproject.org:yocto-testresults'
hint: Updates were rejected because the tag already exists in the remote.

The new tag name is generated by gitarchive based on known tags from the
repository (learnt with git tag). In autobuilder case, this repository is a
shallow clone, so git tag only returns most recent tags, which mean we
could miss some older tags which exist in remote but not locally. In this
case, gitarchive will likely create a tag which already exists in remote,
and so will fail to push

Fix this tag duplication by using git ls-remote to learn about existing
tags instead of git tag. To do so, create a helper ("get_tags") which
manages both nominal case (target directory is a git repository with a
proper remote) and fallback case (target directory is not from a clone, no
remote has been configured)

Fixes [YOCTO #15140]

(From OE-Core rev: 9cbbe9689866158825a7ae774b7965b41ff5c461)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-21 11:34:12 +01:00
Alexis Lothoré
f30867795c oeqa/selftest: introduce gitarchive tests
Add a test suite for gitarchive.py. For now, only introduce tests on
methods which needs to read existing tags

The tests rely on tmpdirs to create local, "fake" results repository in
order to allow basic git commands

(From OE-Core rev: bc9b8ef4bc3e51ba7fca372ab7e532a80975d819)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-21 11:34:12 +01:00
Richard Purdie
f9dff104bd Revert "oeqa/utils/gitarchive: fix tag computation when creating archive"
This reverts commit 5a0a7da85a3acfd4a20a07478eabefdab60f313a.

This caused failres on the build performance tests on the autobuilder.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-16 07:56:15 +01:00
Richard Purdie
d724ec6f04 lib/package_manager: Improve repo artefact filtering
If you run an arm build followed by an x86 one and then ask for a
full repo to be created, it will include all of the arm and x86 packages.
testexport will then find the arm socat package rather than the x86 one
and try and run arm binaries within an x86 qemu image with no success.

The reproducer for this was:

oe-selftest -r fitimage.FitImageTests.test_initramfs_bundle runtime_test.TestImage.test_testimage_install

This patch only symlinks in the compatible package archictures rather
than all of them which fixes the failure and the resulting autobuilder
intermittent failure too.

[YOCTO #15190]

(From OE-Core rev: 30b45bcf49bf8207fd96bb45a55d7708661f3359)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-16 07:54:38 +01:00
Richard Purdie
d57e6de496 oeqa/runtime/parselogs: Exclude preempt-rt error for now
With the new 6.4 kernel we see this preempt-rt error. It is blocking changing
to the new kernel and has sat on mailing lists unresolved for a long time. Ignore
it in testing for now and allow upgrading until we can better understand the
issues.

(From OE-Core rev: 1451df346a0e2433714774421ce8f339a37a844a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-15 08:15:25 +01:00
Alexis Lothoré
788f6a0e16 oeqa/utils/gitarchive: fix tag computation when creating archive
Sporadic errors have been observed in autobuilder when trying to store new
tests results:

error: failed to push some refs to 'push.yoctoproject.org:yocto-testresults'
hint: Updates were rejected because the tag already exists in the remote.

The new tag name is generated by gitarchive based on known tags from the
repository (learnt with git tag). In autobuilder case, this repository is a
shallow clone, so git tag only returns most recent tags, which mean we
could miss some older tags which exist in remote but not locally. In this
case, gitarchive will likely create a tag which already exists in remote,
and so will fail to push

Fix this tag duplication by using git ls-remote to learn about existing
tags instead of git tag. Two places which wrongly read only local tags has
been identified in gitarchive:  expand_tag_strings and get_test_runs

Fixes [YOCTO #15140]

(From OE-Core rev: 5a0a7da85a3acfd4a20a07478eabefdab60f313a)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-15 08:15:25 +01:00
Richard Purdie
67a8dcabda selftest: Ensure usrmerge is enabled with systemd
systemd now requires usrmerge, ensure this is always the case in
our test cases.

(From OE-Core rev: 465bf84c523403ccc7dc6ed8c2a9c32d85929e8c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-11 13:22:01 +01:00