Add a meta-clang builder that runs testimage on core-image-sato-sdk for
both qemux86-64 and qemuarm64, and add meta-clang to the nightly
check-layer builder.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This builder isn't run regularly, and is redundant now that
oe-selftest-armhost uses MACHINE=qemuarm64.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The "non-rpm" tests were not covering our builds particularly well and wasting
build resources. Update the tests to drop the "deb on rpm" and "rpm on deb" targets
which never really find issues.
Replace them with one set of good x86 and arm tests for both ipk and deb, adding in
SDK testing to the mix as well.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Most users are using 64 bit x86 now, update our remaining configuration
defaults to reflect that.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
edgerouter is no longer part of meta-yocto so we removed it from the
autobuilder configuration as well.
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
AUH itself already has an option to include the link into
its email reports; the option just needs to be enabled.
[YOCTO #15103]
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When available, expose tesresult-regressions-report.txt on non-release web page,
as it is done for many other artifacts currently
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some nightly builders are configured in yocto-autobuilder2 to run master builds.
Those build parameters currently skip all branches of
get_regression_base_and_target, which then return None, while the caller
expects a base and target tuple
Set default behaviour to return previous tag as comparison base and passed
branch as target for such builds
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the test assert is about a tag in Poky, the result will not be the same
depending on existing tags at the time of running tests.
Add a LAST_TAG marker to loosen constraints but still allow to tests for general
cases (e.g. : test that tag-depending tests does not return None)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a typo in BUILD_HISTORY_FORKPUSH, leading to failures on Autobuilder
when trying to generate regression reports:
Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/a-full/yocto-autobuilder-helper/scripts/send-qa-email", line 213, in <module>
send_qa_email()
File "/home/pokybuild/yocto-worker/a-full/yocto-autobuilder-helper/scripts/send-qa-email", line 117, in send_qa_email
basebranch, comparebranch = utils.getcomparisonbranch(ourconfig, repo, branch)
File "/home/pokybuild/yocto-worker/a-full/yocto-autobuilder-helper/scripts/utils.py", line 392, in getcomparisonbranch
comparerepo, comparebranch = base.split(":")
ValueError: not enough values to unpack (expected 2, got 1)
Observed on build a-full/5070
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add "abelloni/master-next" branch from poky-contrib in configuration so that
regression reports are generated when testing for patches
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
d6018b891a broke regression reporting for testing
branches (e.g: master-next in poky, ross/mut in poky-contrib) by ignoring the comparebranch returned by
utils.getcomparison branch
Fix regression reporting for those branches by using comparebranch again. The
fix also refactor/add a intermediary step to guess base and target for
regression reporting, to isolate a bit the logic and make it easier later to add
multiple base/target couples
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
is_non_release_version has an inverted logic which makes its reuse quite
confusing
Transform it as is_release_version and let caller do the negation if needed
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A new tool named yocto_testresults_query.py has been integrated in poky as a
thin wrapper between send-qa-email and resulttool. The new tool is in charge of
converting tags/branches names to SHA1 revisions and to call resulttool with
those revisions
Remove any code related to tag/branches conversions to SHA1 and use
yocto_testresults_qery.py instead
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Minor typo observed when cheking the "Prepared shared repository" step logs in
autobuilder web interface:
====================================================================================================
Intially fetching repo poky (1675810261.1)
====================================================================================================
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current regression reports do not contain information about versions compared
when generating reports. While it is still possible to get the information by
searching the autobuilder log, it is not convenient. Moreover, future
developments will allow to generate multiple reports (with different bases for
comparison) in a single build.
As a consequence, embed target and base revisions in the report header
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since we are now force-fetching base revisions and target revisions for
regression report generation, we can make testresults clone even more "shallow"
to increase clone speed in CI pipelines
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we try to run send-qa-email to simulate past releases (for example, for
development or debugging), the execution will very likely fail because the
target revision to examine (ie: the poky revision) is too old, and as a
consequence is not contained in the testresults shallow clone anymore (because
testsresults history keeps moving forward as builds are triggered on
autobuilder). As a consequence, force-fetch the "target" revision, as it is
already done for the "base" revision
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For the month of January 2023, the distribution of dd times has a long
tail that extends to 13 seconds with 2 events exceeding the current
limit of 30 seconds.
Reduce the timeout to 3 seconds based on the observed distribution of
dd times, which would result in the timout triggering about 20 times a month.
That's enough data to be useful but not so much that it's overwhelming the
logging or the people who will analyze it. It also avoids the rapid increase
in the tail of the distribution which starts to rise exponentially under 2 seconds.
It's sensible response time for people to expect the system to have.
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The "previous version" computation bring many edge cases depending on the
version under release. Add a basic test suite to validate currently implemented
computation strategy and to prevent mistakes when ediiting it in the future
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of only generating regressions reports against HEAD of relevant branch, compute
most relevant tag (ie : release) against which we can check for regressions. General rules
introduced are the following :
- milestone release is checked against previous milestone if possible, otherwise
against major release
- point release is checked against previous point release if possible,
otherwise against major release
- major release is checked against previous major release
- a non release build is checked against base branch
Examples :
- 4.1.2.rc1 is checked against yocto-4.1.1
- 4.1.2 is checked against yocto-4.1.1
- 4.1.1.rc1 is checked against yocto-4.1
- 4.1.1 is checked against yocto-4.1
- 4.1 is checked against yocto-4.0
- 4.1.rc4 is checked against yocto-4.0
- 4.1_M2.rc1 is checked against 4.1_M1
- 4.1_M2 is checked against 4.1_M1
- 4.1_M1.rc1 is checked against yocto-4.0
- 4.1_M1 is checked against yocto-4.0
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This wrapping allows importing send_qa_email.py in another script without
executing the QA emailing routine. This is useful for unit testing the script
internal functions
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Merely renaming send-qa-email to send_qa_email.py would imply to update
yocto-autobuilder2 scripts, which can be daunting. In order to preserve
compatibility with current configuration, add a symlink pointing to the
new script
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This rename fixes send-qa-email regarding python guidelines, which allows to
import the script content in another script (for unit testing purpose for
example)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The pub/non-release directories have aged out and no longer work.
Switch to a stable location and bump the version to 4.1.2 as well.
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Currently we use 56 or 24 cores on the various workers but this is
pulling machines over with OOM issues, particularly in oe-selftest.
Lower the parallelism for parsing to try and help avoid this as we
don't really need maximum speed for parsing on the autobuilder.
Keep the same numbers for the perf workers as we want to use all
cores there.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will be particularly useful for testing Y2038 issues,
as that is the only 32 bit target that can be run at native speed
on the AB cluster.
Whether a-full should get fast ptests or full ptests can be decided
after collecting AB execution times for both.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
