Commit Graph

462 Commits

Author SHA1 Message Date
Khem Raj
e2c0b0ceeb oeqa: Update lzip version in buildzip testcase
(From OE-Core rev: 5200dcb39834790542c429d70cf54db07ff7a378)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-04-08 15:49:17 +01:00
Khem Raj
b76c04bdcb oeqa: Build cpio with C17 std
cpio is not yet buildable with C23 standard which is default with
GCC 15, therefore ensure to apply needed bandage to keep it compiling
in C17 mode even with GCC 15

(From OE-Core rev: 0c637099887f1be421c8e1203f99631a1e040150)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-03-24 17:29:16 +00:00
Christos Gavros
dd8195473f skeletoninit: remove broken link
The link in the comment is broken.
It's removed together with some text.

CC: Yoann Congal <yoann.congal@smile.fr>
CC: Randy MacLeod <randy.macleod@windriver.com>
(From OE-Core rev: 4d5ce199d44bbeee7a7cff1dc7deac3d45a15d35)

Signed-off-by: Christos Gavros <gavrosc@yahoo.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-02-25 12:17:22 +00:00
Richard Purdie
1460b73e01 oeqa/runtime/connman: Drop unused test
This test checks for an IP address and then tests if interface aliases work. We
don't run it on any of our automated testing as it only applies for non-qemu.

The connectivity test is unrealted to connman and pretty pointless as it depends
on ssh being working, so networking is probably ok.

The alias interface test is unrelated to commman and a general networking test
but seems out of place.

The code uses obsolete ifconfig calls and overall, the value of the test we're
never using seems low. Delete it.

(From OE-Core rev: 5e40277d1ee9df8dbc612a39b575f9a50806cd62)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-02-18 22:53:45 +00:00
Richard Purdie
261a1409b1 oeqa/runtime: Add debugging if networking fails
If networking fails, we can get useful informaiton over the serial connection. Add
this fallback code so that any issues can be more easily debugged by showing the
host and target networking states.

(From OE-Core rev: 3291f9d07ecfe7d3301dc914f5e6a80577cf1d5d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-02-11 11:44:19 +00:00
Ross Burton
3975cc6f4f oeqa/runtime/ping: don't bother trying to ping localhost
If SLIRP is being used instead of TAP for networking to the guest then
the target IP will be localhost.  There's no point in pinging localhost
to see if the target is up but whilst you'd think it is harmless, in
some containers ping doesn't actually have enough rights to work:

  ping: socktype: SOCK_RAW
  ping: socket: Operation not permitted
  ping: => missing cap_net_raw+p capability or setuid?

Look at the target address and if it's localhost or 127.0.0.* return
immediately.

(From OE-Core rev: a06ef43d2a50e16c32bd6edbdc7b32c3528687d5)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-11-18 22:09:03 +00:00
Mikko Rapeli
30cc8b17d0 oeqa runtime uki.py: add tests
If Unified Kernel Image was enabled via IMAGE_CLASSES, then
target should also boot the same uki at runtime.

(From OE-Core rev: 2ae651c869a19fd24b5a915451c1f99e1d7b32f0)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-11-12 11:31:48 +00:00
Richard Purdie
317bc90025 oeqa/runtime/parselogs: Add different PCI BAR errors with qemu 9.1.0
With qemu 9.1.0 we see parselogs failures with mips and mips64. We've long
since ignored these errors so update to continue to do so with the new form.

(From OE-Core rev: 4f3c299e7a282e953d9adb7549c30015df317171)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-22 11:39:58 +01:00
Richard Purdie
d2b2675cc4 oeqa/runtime/ssh: Fix incorrect timeout fix
I'd meant to change the command timeout in the previous change, fix
the correct one.

(From OE-Core rev: bb991988cb23be2c8947171726ada321f27e6eed)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-14 21:42:17 +01:00
Richard Purdie
2f9ba70637 oeqa/runtime/ssh: Rework ssh timeout
After the changes to improve this test, we keep seeing image testing ssh
failures, particularly on mips.

It looks like part of the problem is that on a loaded system, 5s is too short
for mips to reliably establish an ssh connection. I've seen logs where it keeps
timing out and fails to work, then the debug code successfully uses ssh later after
everything else fails.

Change the timings/retries to give slow platforms enough time to respond.

(From OE-Core rev: ba64ccf3ad6e40461219b72d60eb0fe5cb38fddd)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-11 12:17:03 +01:00
Andrew Oppelt
413f327baa testexport: support for executing tests over serial
Uses TEST_SERIALCONTROL_CMD to open a serial connection to the target
and execute commands. This is a drop in replacement for the ssh target,
fully supporting the same API. Supported with testexport.

To use, set the following in local.conf:
- TEST_TARGET to "serial"
- TEST_SERIALCONTROL_CMD to a shell command or script which connects to
  the serial console of the target and forwards that connection to
  standard input/output.
- TEST_SERIALCONTROL_EXTRA_ARGS (optional) any parameters that must be
  passed to the serial control command.
- TEST_SERIALCONTROL_PS1 (optional) A regex string representing an empty
  prompt on the target terminal. Example: "root@target:.*# ". This is
  used to find an empty shell after each command is run. This field is
  optional and will default to "root@{MACHINE}:.*# " if no other value is
  given.
- TEST_SERIALCONTROL_CONNECT_TIMEOUT (optional) Specifies the timeout in
  seconds for the initial connection to the target. Defaults to 10 if no
  other value is given.

The serial target does have some additional limitations over the ssh
target.
1. Only supports one "run" command at a time. If two threads attempt to
   call "run", one will block until it finishes. This is a limitation of
   the serial link, since two connections cannot be opened at once.
2. For file transfer, the target needs a shell and the base32 program.
   The file transfer implementation was chosen to be as generic as
   possible, so it could support as many targets as possible.
3. Transferring files is significantly slower. On a 115200 baud serial
   connection, the fastest observed speed was 30kbps. This is due to
   overhead in the implementation due to decisions documented in #2
   above.

(From OE-Core rev: d817b27d73d29ba2beffa2e0a4e31a14dbe0f1bf)

Signed-off-by: Andrew Oppelt <andrew.j.oppelt@boeing.com>
Signed-off-by: Matthew Weber <matthew.l.weber3@boeing.com>
Signed-off-by: Chuck Wolber <chuck.wolber@boeing.com>

--

Tested with core-image-sato on real hardware. TEST_SERIALCONTROL_CMD
was set to a bash script which connected with telnet to the target.

Additionally tested with QEMU by setting TEST_SERIALCONTROL_CMD to
"ssh -o StrictHostKeyChecking=no root@192.168.7.2". This imitates
a serial connection to the QEMU instance.

Steps:
1) Set the following in local.conf:
  - IMAGE_CLASSES += "testexport"
  - TEST_TARGET = "serial"
  - TEST_SERIALCONTROL_CMD="ssh -o StrictHostKeyChecking=no root@192.168.7.2"
2) Build an image
  - bitbake core-image-sato
3) Run the test export
  - bitbake -c testexport core-image-sato
4) Run the image in qemu
  - runqemu nographic core-image-sato
5) Navigate to the test export directory
6) Run the exported tests with target-type set to serial
 - ./oe-test runtime --test-data-file ./data/testdata.json --packages-manifest ./data/manifest --debug --target-type serial

Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-01 12:28:10 +01:00
Jon Mason
001eea1f7d oeqa/runtime/ssh: increase the number of attempts
Under high load, the ssh test is hitting the amount of retries.
Increase it to 20 to avoid this issue.  This would increase the maximum
failure time from 50 seconds (5 * 10) to 100 seconds.

(From OE-Core rev: c796438eec5dd6b4671b798f85506bc89ff402ab)

Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-22 16:17:41 +01:00
Bruce Ashfield
e7b915a3ad oeqa/runtime/parselogs: mips: skip sysctl warning
Upstream has shuffled the sysctl registration via the commit:

   commit d4ae80ffa64f87b9c355692b680b603add084e96
   Author: Zhen Ni <nizhen@uniontech.com>
   Date:   Tue Feb 15 19:46:03 2022 +0800

       sched: Move cfs_bandwidth_slice sysctls to fair.c

       move cfs_bandwidth_slice sysctls to fair.c and use the
       new register_sysctl_init() to register the sysctl interface.

       Signed-off-by: Zhen Ni <nizhen@uniontech.com>
       Signed-off-by: Luis Chamberlain <mcgrof@kernel.org>

The way that we have to configure our mips qemu platforms
results in an empty sysctl table registration and the following
harmless warnings:

  "failed when register_sysctl_sz sched_fair_sysctls to kernel"
  "failed when register_sysctl_sz sched_core_sysctls to kernel"

Adding them to our list of acceptable dmesg warnings.

(From OE-Core rev: 4cf678858ef6f2c3310ad8f26cac3e7e133d4f0a)

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-07 15:47:15 +01:00
Bruce Ashfield
12d0848bf2 oeqa/runtime/parselogs: update pci BAR ignore for kernel 6.10
The format of the pci BAR warnings we get on qemu boots has
changed in 6.10+ via the following kernel commit:

    commit dc4e6f21c3f844ebc1c52b6920b8ec5dfc73f4e8
    Author: Puranjay Mohan <puranjay@kernel.org>
    Date:   Sat Nov 6 16:56:06 2021 +0530

        PCI: Use resource names in PCI log messages

        Use the pci_resource_name() to get the name of the resource and use it
        while printing log messages.

        [bhelgaas: rename to match struct resource * names, also use names in other BAR messages]
        Link: https://lore.kernel.org/r/20211106112606.192563-3-puranjay12@gmail.com
        Signed-off-by: Puranjay Mohan <puranjay12@gmail.com>
        Signed-off-by: Bjorn Helgaas <bhelgaas@google.com>

Since it doesn't appear that we can do regex's in parselogs
and the bar number is now in the middle of the message, we
go with a slightly wider format of the message to ignore.

(From OE-Core rev: 0a7126604b6536868600d43aff000a426384995c)

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-07 15:47:15 +01:00
Jose Quaresma
d954ed621f oeqa/runtime/scp: requires openssh-sftp-server
The SCP protocol was deprecated in favor of the SFTP.
For the legacy SCP protocol scp should be run with "-O".
Instead of adding "-O" on the scp_options ssh oeqa we can
require the openssh-sftp-server to be instaled on the target.

This way the test will work more deterministic regardless of
the host machine client used.
For the old fashion clients still using legacy SCP protocol
the openssh-sshd server will be used, for the new ones using
the SFTP the openssh-sftp-server will be picked.

(From OE-Core rev: 2f43da91ba20d18bc419bca7651bb383a51f20af)

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-23 11:54:37 +01:00
Jon Mason
8d405ba864 oeqa/runtime/ssh: check for all errors at the end
With the retry for the -SIGTERM, it is possible to still see that error
after the 5th attempt and mark the run a success.  Check for any
non-zero status in the final check and error out to close the gap.
While there, make the error print match the one above and be a little
more verbose.  Also, I'm seeing it take roughly 6 attempts on my local
(very slow) system to pass.  So, increasing the number of attempts to
10.

(From OE-Core rev: 3c3ebe591eef6e0479d623ec2237cfea16db5c80)

Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-08 09:17:09 +01:00
Richard Purdie
a7d13a3cfa oeqa/runtime/ssh: In case of failure, show exit code and handle -15 (SIGTERM)
Ensure we show the failing exit code in case of failures.

We're seeing autobuilder failures with -15 (SIGTERM) which is probably from
slow boot/init. Retry in these cases for now.

(From OE-Core rev: 127d3bd8d5509ae17e359c1365859fd362ffc74f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-04 22:56:15 +01:00
Jon Mason
a65a3dd4aa oeqa/runtime/ssh: add retry logic and sleeps to allow for slower systems
On exceptionally slow systems, the ssh test can intermittently fail due
to a race between when ping works and the networking applications being
brought up.  To work around this issue, add some retry logic when ssh
fails to connect.  According to the man page of ssh, "ssh exits
with the exit status of the remote command or with 255 if an error
occurred."  So, only retry if the return code is 255, and limit the
number of retries to prevent it looping forever.

(From OE-Core rev: f0fe0b490d309cdf1c97754f85a61b5b948b7f28)

Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-02 22:29:13 +01:00
Richard Purdie
e881c552f0 oeqa/runtime/parselogs: Add some kernel log trigger keywords
During testing we're finding some kernel oops messages and other key kernel
errors such as irq issues are not triggering our log parsing.

Add those keywords to the list of things to scan for, making such
failures much more visible.

(From OE-Core rev: ef4623be60226e8caaf7813705aa4941ff354eac)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-27 13:03:35 +01:00
Etienne Cordonnier
d87082efa5 oeqa/runtime: fix race-condition in minidebuginfo test
Fix this error where 'coredumpctl info' warns that the coredump is still being
processed:

```
AssertionError: 1 != 0 : MiniDebugInfo Test failed: No match found.
-- Notice: 1 systemd-coredump@.service unit is running, output may be incomplete.
```

(From OE-Core rev: ad1ce64f5c1f22a7b10025d8cba20dc74354ac81)

(From OE-Core rev: ed562345d5a5f2edb649028553199f3f7966e19e)

Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-24 12:43:40 +01:00
Etienne Cordonnier
32ce07d3f5 oeqa/runtime: make minidebuginfo test work with coreutils
The test was failing with core-image-full-cmdline which uses sleep from coreutils instead of sleep from busybox.

(From OE-Core rev: 8497edf0c56da34ea7b7509b8fcd46fcba2fd0e3)

Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-06 11:56:03 +01:00
Richard Purdie
424c43070d oeqa/runtime/ltp: Drop groups that were removed in new version
The io, cap_bounds and filecaps groups were removed in latest version
of ltp. Tweak the test class to match.

(From OE-Core rev: d5460f15cbc56e52cbbc09785a777278208da7f7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-04 13:49:14 +01:00
Etienne Cordonnier
afe7a3a384 oeqa/runtime: fix regression in minidebuginfo test
The builtin command "sleep" was getting executed instead of the busybox binary.
Hence "pidof sleep" was empty and the test was failing.

(From OE-Core rev: 3188f3f0718b8a59bc9e3cae4cdf74e82bdb86c3)

Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-31 16:58:36 +01:00
Richard Purdie
0bd3234676 oeqa/runtime/login: Fix dbus-wait timeout and loop conditional
The dbus-wait command returns a timeout after 60s but reports "success", detect this.
Unfortunately it does effectively break the test as the signal is nearly never being
correctly detected since it was already sent.

For that reason comment out the code instead too.

Also fix the loop conditional as the logic was incorrect and it was looping
indefinitely when an image match didn't occur.

(From OE-Core rev: 89c930e9e4b38b116edcba59e88621a39f8bda67)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-07 17:27:51 +00:00
Eilís 'pidge' Ní Fhlannagáin
f6235fef62 oeqa/runtime/login: Add screenshot sample logic/timeout/dbus-wait
This patch uses dbus-wait to wait for matchbox to be up. Once that
happens, it sets a timeout of 60 seconds and takes a screenshot
and compares it, every 2 seconds. If diff=0 it passes. If the timeout
ends, it fails.

(From OE-Core rev: 287b4f0a8244f7214f6a1aaa84ef16cc528f8326)

Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-07 17:27:51 +00:00
Eilís 'pidge' Ní Fhlannagáin
59c7b5e706 oeqa/runtime/login: Exclude qemuriscv64
Excluding riscv64 due to mouse rather than a touchscreen which adds a
moving cursor, so the diff ends up > 0. Need to fix the image to use the
touchscreen rather than mouse input.

(From OE-Core rev: 7f7032c7613abd62ce510c98211c75fc7c5e7090)

Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-07 17:27:51 +00:00
Richard Purdie
96af410e8e oeqa/runtime/login: Mask out the mouse panel icon for now
The mouse panel icon can move when the time changes between 3 and 4 digits.
Ideally we'd fix the alignment of the clock on the panel but to get the
tests working, increase the size of the mask for now.

(From OE-Core rev: 8ec02142ab98741749281bdb29d5973c96b839d7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-07 17:27:51 +00:00
Richard Purdie
4d161405d5 oeqa/runtime/login: Various code improvements and fixes
* Allow tools to be found from the host PATH so that imagemagick from a buildtools
  tarball/sdk can work
* Reformat the code to have imports at the start of the file and have more standard
  formatting and whitespace
* Always save copies of the images, the space imapct is negligle compared to the
  debug win
* Write the images to ${T}
* Use bb.utils.mkdirhier() instead of more complex code
* Restrict the tests to images containing matchbox-desktop

(From OE-Core rev: d09989b49517830297654e4d1d150aaa8723c41a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-07 17:27:51 +00:00
Eilís 'pidge' Ní Fhlannagáin
b49d42ce90 oeqa/runtime/login: Proof of concept for screenshot testcases
This takes the work rburton did on image screenshot testing and
expands it.

Right now this works with most of the qemu based machines except for

- qemuppc64
- qemuarmv5
- qemuriscv32
- qemuloongarch64

See "Known Issues" further down.

This test takes a screendump of a qemu image, blanks out the clock
and compares it to an image we have on record. If the diff is exact,
the test passes. If not, it stores the image in build/failed-images and
fails out.

In order to enable this test, you will need meta-openembedded/meta-oe in
your bblayers.conf for imagemagick and the following in local.conf:

IMAGE_CLASSES += "testimage"
TEST_SUITES = "login"
TESTIMAGEDEPENDS:append:qemuall = " imagemagick-native:do_populate_sysroot "

Known Issues
------------

The main issue is that I've yet to find a gating factor that would allow
me to tell when the qemu instance is fully up and rendered. I've tried a
few tactics here, (dbus-wait, qmp) but for now a disgusting
time.sleep(30) is there.

You can replicate this by running qemumips. The screen load takes forever,
but you even see it on qemux86 where the Home and Workspace Switch icons
will sometimes take a while to fully load.

Eventually I'm going to have to take multiple screenshots and compare
them, but then you get into the issue where the question is, is the diff
greater than 0 because it hasn't fully loaded or something is actually
incorrect.

There are the issues I know about:

   - runqemu qemuppc64 comes up blank.
   - qemuarmv5 comes up with multiple heads but sending "head" to screendump.
     seems to create a png with a bad header.
   - qemuriscv32 and qemuloongarch64 don't work with testimage apparently?
   - qemumips64 is missing mouse icon.
   - qemumips takes forever to render and is missing mouse icon.
   - qemuarm and qemuppc return incorrect width
   - All images have home and screen flipper icons not always rendered fully at first.
     The sleep seems to help this out some, depending on machine load.

(From OE-Core rev: dc7cefbaccde50df6c4396e66d50659a45e00631)

Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Co-authored-by: Ross Burton <ross.burton@arm.com>
Co-authored-by: Eilís 'pidge' Ní Fhlannagáin <pidge@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-07 17:27:51 +00:00
Michael Opdenacker
6a9aab5654 oeqa/runtime/cases: fix typo in information message
(From OE-Core rev: 901ff496b97119add792912fbb2f1efb1152e7c2)

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-09 15:55:58 +00:00
Khem Raj
bd7a3fd0f7 buildcpio.py: Switch to using cpio-2.15
This helps in getting it building with newer architectures like riscv32
since it has upgraded gnulib over 2.14 which has the needed fixes.

Drop the -fno-common workaround as it is already applied to cpio

drop --disable-maintainer-mode

Fixes
configure: WARNING: unrecognized options: --disable-maintainer-mode

(From OE-Core rev: 18d303497089d3a7a893ee0eec5b0f0c78cca06d)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-08 10:59:06 +00:00
Richard Purdie
0513fd279c ltp: Enable extra test groups
Petr Vorel (upstream ltp maintainer) recommended some extra tests we should run.
Enable these for extra test coverage.

(From OE-Core rev: ebae56999dc2ed7abb65062a9abee53827c3bad3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-06 10:32:19 +00:00
Petr Vorel
b6abbf4402 ltp: Update to 20240129
Removed patch accepted upstream as
8c53cc947 ("testcases/kernel/syscalls/fcntl: define _LARGEFILE64_SOURCE")

Rebase runtest/mm patch.

[RP:
Drop connectors group since dropped upstream:
9b642d89c0

Drop fsx too:
fb2b6a0b3c
]

(From OE-Core rev: b2867cac5ac58862469f0c6c056607abeca0b0d0)

Signed-off-by: Petr Vorel <petr.vorel@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-05 14:06:10 +00:00
Alexander Kanavin
482ed7cd80 oeqa/runtime/rpm: fail tests if test rpm file cannot be found
Discovery of the test file was happening in a class initializer.
That block of code cannot fail (it's not a test), and so it
falls through to completion even if the needed file could not be found.

Then the tests themselves fail later due to class variables not
being set, but all information as to why is already lost at that point.

This converts the discovery to a helper function called from
the tests, so that the function can fail the tests precisely when the
problems occur.

(From OE-Core rev: 5d7a6ede105ea1efc9c324c7029f9d08dadf7255)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-27 08:20:02 +00:00
Mikko Rapeli
a703d8ddef oeqa systemd.py: settle() using "running" or "degraded" state
systemd boot has completed when system is in "running" or "degraded"
(some services failed) state. Check for that in the systemd settle()
function instead of listing all services and checking their activation
state since some services are in activation state even when whole
system is already in "running" state. Examples of services which can be
in activation state are rootfs auto mounting related generated services.

Without this patch systemd test_systemd_list (systemd.SystemdBasicTests) times
out on an image with dm-verity /usr partition and systemd generated
rootfs:

NOTE:  ... FAIL
Traceback (most recent call last):
  File "/home/builder/src/base/build/../poky/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
    return func(*args, **kwargs)
  File "/home/builder/src/base/poky/meta/lib/oeqa/runtime/cases/systemd.py", line 97, in test_systemd_failed
    self.assertTrue(settled, msg=msg)
AssertionError: False is not true : Timed out waiting for systemd to settle:
  UNIT                                                                                              LOAD   ACTIVE     SUB
     DESCRIPTION
  dev-disk-by\x2did-dm\x2dname\x2droot.device                                                       loaded activating tentativ
e    /dev/disk/by-id/dm-name-root
  dev-disk-by\x2did-dm\x2dname\x2dusr.device                                                        loaded activating tentativ
e    /dev/disk/by-id/dm-name-usr
  dev-disk-by\x2did-dm\x2duuid\x2dCRYPT\x2dLUKS2\x2df2b944f394174eb5918cb6af2c6b4cb2\x2droot.device loaded activating tentativ
e    /dev/disk/by-id/dm-uuid-CRYPT-LUKS2-f2b944f394174eb5918cb6af2c6b4cb2-root
  dev-disk-by\x2did-dm\x2duuid\x2dCRYPT\x2dVERITY\x2d3dd703c88f1946658697a6d57617473b\x2dusr.device loaded activating tentativ
e    /dev/disk/by-id/dm-uuid-CRYPT-VERITY-3dd703c88f1946658697a6d57617473b-usr
  dev-disk-by\x2duuid-bfbf856e\x2d3c65\x2d4eb2\x2d9ffb\x2d8e0b11641d85.device                       loaded activating tentativ
e    /dev/disk/by-uuid/bfbf856e-3c65-4eb2-9ffb-8e0b11641d85
  dev-dm\x2d0.device                                                                                loaded activating tentativ
e    /dev/dm-0
  dev-dm\x2d1.device                                                                                loaded activating tentativ
e    /dev/dm-1
...

Fix is to check for the systemd global "running" or "degraded" state.
Note that it would be possible to use a blocking call
"systemctl is-system-running --wait" to exit after system enters "running"
or "degraded" state but using the existing loop for a 2 minute timeout.

(From OE-Core rev: 3b013ae441d117adeda0d9950e02e9f7d0deba2f)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-07 12:24:57 +00:00
Ross Burton
9367835346 oeqa/parselogs-ignores-qemuarmv5: add comments and organise
Add a comment explaining why we need the jitterentropy ignore.

(From OE-Core rev: ed4b202b11fd18e18502fe4e90f421dce4db8b44)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-04 23:47:51 +00:00
Bruce Ashfield
d68492d6c5 oeqa/runtime/parselogs: add qemux86 ACPI ignore for kernel v6.6+
upstream commit 003e0694fcd3d5 [ACPI: processor: Refine messages
in acpi_early_processor_control_setup()] changes logging during
boot such that some of the informational messages are caught as
QA issues.

Adding the string to the ingore files so that 6.6+ will pass QA.

(From OE-Core rev: 463d40f2d4bf30becce431bfc1cbdf3b673e8e60)

Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-03 23:36:34 +00:00
Saul Wold
acf45e764c package.py: OEHasPackage: Add MLPREFIX to packagename
FIXES [YOCTO #12342]

When testing a Multilib image, the package manifest list contains
the fully qualified package name which includes the Multilib Prefix.
This patch adds the MLPREFIX to the package names that are passed
into the @OEHasPackage() decorator to ensure the set isdisjoint()
matches correctly.

(From OE-Core rev: ab87e4f92305b2a664cc473869e1615cf56e0936)

Signed-off-by: Saul Wold <sgw@bigsur.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-28 10:57:49 +00:00
Tim Orling
dfe9c45264 oeqa: add runtime 'maturin develop' test case
Similar to the sdk test case, build the "guessing-game" example from
https://maturin.rs/tutorial

This test case:
* creates a python3 venv
* echoes "nameserver 8.8.8.8" to /etc/resolv.conf as we need to have
  functional DNS to fetch the crates on target
* fetches crates, builds guessing-game crate and wheel

Put the following in your local.conf:

EXTRA_IMAGE_FEATURES += "tools-sdk"
SDK_INCLUDE_TOOLCHAIN = '1'
SDK_TOOLCHAIN_LANGS += 'rust'

IMAGE_INSTALL:append = " python3-maturin"

IMAGE_CLASSES += "testimage"
TEST_QEMUPARAMS ?= "-m 8192 -smp 4"
IMAGE_ROOTFS_EXTRA_SPACE = "10000000"
NOHDD="1"
NOISO="1"

TEST_SUITES = "ping ssh python maturin"

Test with:

bitbake core-image-full-cmdline
bitbake -c testimage core-image-full-cmdline

(From OE-Core rev: ca7e78c8be6aaa2780702eab54715a74fc0dac5e)

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-23 08:46:00 +00:00
Tim Orling
844e91049c oe-selfest: add maturn runtime (testimage) test
Basic smoke test for maturin to test the 'maturin list-python' case.

(From OE-Core rev: 47c948c3cf6e582abd12021ceeff2c20a3e81fb5)

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-17 19:07:22 +00:00
Richard Purdie
1b14962c3f oeqa/runtime/systemd: Ensure test runs only on systemd images
When enabled in DISTRO_FEATURES the test may run on a system without systemd.
Fix this.

(From OE-Core rev: c2b473390dec0f5132d5b4bff6d3c35214eb898b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-08 17:09:59 +00:00
Etienne Cordonnier
959b1f7de4 gdb/systemd: enable minidebuginfo support conditionally
Enabling minidebuginfo is not useful if gdb and systemd-coredump
are unable to parse it.

In order to parse it, gdb needs xz support. Systemd needs coredump enabled, as
well as elfutil enabled as well (systemd-coredump loads libdw which is part of elfutils using dlopen).

(From OE-Core rev: 0d2df803bebfd7e832ab7da54c4dacaaeeb424a9)

Signed-off-by: Etienne Cordonnier <ecordonnier@snap.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:50 +00:00
Ross Burton
9406c0d28f oeqa/runtime/parselogs: migrate ignores
Move the ignores from a huge dict in the parselogs.py module to .txt
files. This is just the common, tune, and qemu machine ignores; the
machine ignores that are not in oe-core will be added to the relevant
layers.

The list of ignores has not been reviewed in any meaningful way, this
should be done soon as I suspect a number of these are redundant.

(From OE-Core rev: bba243e1d18b954578afcdb3c727d8f687187ee8)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Ross Burton
41ffa16912 oeqa/runtime/parselogs: load ignores from disk
Instead of hardcoding the list of ignored errors/warnings in the test
itself, read them plain text files on disk.

This uses importlib to try to open a file called
oeqa.runtime.cases.parselogs-ignores-[candidate].txt, where the
candidate will be:

- "common"
- The TARGET_ARCH
- Each of the MACHINEOVERRDES

This allows the common and tune-specific ignores to be retained in
oe-core, and machine-specific ignores added to the layer where the
machine is defined.

[ YOCTO #14604 ]

(From OE-Core rev: 7a04063f7cff243fe2bee09664ad7979612110cb)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
luca fancellu
0f25c8aa77 oeqa/ssh: Handle SSHCall timeout error code
The current code in ssh.py is terminating the ssh process that
does not finish its computation in a given timeout (when timeout
is passed), the SSHCall function is returning the process error
code.

The Openssl ssh before version 8.6_p1 is returning 0 when it is
terminated, from commit 8a9520836e71830f4fccca066dba73fea3d16bda
onwards (version >= 8.6_p1) ssh is returning 255 instead.

So for version of ssh older than 8.6_p1 when the SSHCall time out,
the return code will be 0, meaning success, which is wrong.

Fix this issue checking if the process has timeout (hence it's been
terminated) and checking if the returned code is 0, in that case
set it to 255 to advertise that an error occurred.

Add a test case excercising the timeout in the SSHTest, test_ssh
test function.

(From OE-Core rev: 948fecca1db4c7a30fcca5fcf5eef95cd12efb00)

Signed-off-by: Luca Fancellu <luca.fancellu@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-10 17:44:27 +00:00
Michael Opdenacker
7ac5639812 oeqa/runtime/cases/parselogs: remove "edgerouter" case
The "edgerouter" machine has been removed since
https://git.yoctoproject.org/poky/commit/?id=0c64d0e4317e3749f7f7ed9ecd5d08bbb0cedc9e

(From OE-Core rev: b82514c821add181e141d12b0c1723760b445fea)

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-26 21:11:28 +01:00
Ross Burton
d44333b92c oeqa/runtime/_qemutiny: rewrite test to be functional
The _qemutiny is a small test case that was explicitly designed to do a
minimal level of testing for poky-tiny images.  These typically don't
have SSH servers so we need to assume that qemu is being used and access
the serial console directly.

(From OE-Core rev: 2245b2754d6f4798127ce85a2ab7cb48f458c1f7)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-11 09:43:45 +01:00
Ross Burton
f15f380650 oeqa/runtime/parselogs: parse the logs with Python, not grep
Instead of constructing huge grep statements, we can simply open the logs
in Python and do the relevant string operations directly.

The trick is to remember to casefold() all of the strings, so that the
"in" operator can be used.

Just one of the ignores needs to be adjusted because it uses a regular
expression and the new logic doesn't support that.  This is handled
by simply reducing the size of the ignore match.

(From OE-Core rev: 78ae254c4a78a025a712281ce9de373cdccf5472)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-26 10:25:42 +01:00
Ross Burton
048e56f3b3 oeqa/runtime/parselogs: select the correct machine-specific ignores early
This has no impact to the execution, but makes the following changes
neater.

(From OE-Core rev: 124dede2b20930d09a5cc319cb9333c5a912fa48)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-26 10:25:42 +01:00
Ross Burton
17dbdc677a oeqa/runtime/parselogs: move some variables out of global scope
errors and log_locations can be trivially set in the class directly,
instead of being defined in the module and then copied into the class.

(From OE-Core rev: dd1416b719a30b18f21916d50fa431a88503918f)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-26 10:25:42 +01:00