mirror of
git://git.yoctoproject.org/poky.git
synced 2025-07-19 12:59:02 +02:00
1eae53e277
resulttool regression module simply compare a base test status to a target test
result status. This approach raises many false positives since all XXX -> PASS
transitions (XXX being any status different from PASS) are flagged as
regression.
- Do not list XXX -> PASS transitions in regression report, instead count them
and print a summary of "newly passing tests"
- If an inspected pair has only "newly passing tests", do not print detailed
list and print it as "Improvement" instead of "Regression"
Updated output example looks like the following:
[...]
Improvement: oeselftest_fedora-37_qemux86-64_20230127010225
oeselftest_ubuntu-22.04_qemux86-64_20230226120516
(+1 test(s) passing)
[...]
Match: oeselftest_almalinux-8.7_qemuarm64_20230127015830
oeselftest_almalinux-8.7_qemuarm64_20230227015258
[...]
Regression: oeselftest_almalinux-9.1_qemumips_20230127000217
oeselftest_opensuseleap-15.4_qemumips_20230226130046
ptestresult.glibc-user.debug/tst-read-chk-cancel: PASS -> None
ptestresult.glibc-user.nptl/tst-mutexpi4: PASS -> FAIL
ptestresult.glibc-user.nptl/tst-mutexpi5a: PASS -> FAIL
Additionally, 44 previously failing test(s) is/are now passing
(From OE-Core rev: c335f96f687c73fde443ac330ca3e17113794d9e)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
||
|---|---|---|
| .. | ||
| build_perf | ||
| checklayer | ||
| devtool | ||
| recipetool | ||
| resulttool | ||
| wic | ||
| argparse_oe.py | ||
| buildstats.py | ||
| scriptpath.py | ||
| scriptutils.py | ||