When the test assert is about a tag in Poky, the result will not be the same
depending on existing tags at the time of running tests.
Add a LAST_TAG marker to loosen constraints but still allow to tests for general
cases (e.g. : test that tag-depending tests does not return None)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
d6018b891a broke regression reporting for testing
branches (e.g: master-next in poky, ross/mut in poky-contrib) by ignoring the comparebranch returned by
utils.getcomparison branch
Fix regression reporting for those branches by using comparebranch again. The
fix also refactor/add a intermediary step to guess base and target for
regression reporting, to isolate a bit the logic and make it easier later to add
multiple base/target couples
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
is_non_release_version has an inverted logic which makes its reuse quite
confusing
Transform it as is_release_version and let caller do the negation if needed
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A new tool named yocto_testresults_query.py has been integrated in poky as a
thin wrapper between send-qa-email and resulttool. The new tool is in charge of
converting tags/branches names to SHA1 revisions and to call resulttool with
those revisions
Remove any code related to tag/branches conversions to SHA1 and use
yocto_testresults_qery.py instead
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Minor typo observed when cheking the "Prepared shared repository" step logs in
autobuilder web interface:
====================================================================================================
Intially fetching repo poky (1675810261.1)
====================================================================================================
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current regression reports do not contain information about versions compared
when generating reports. While it is still possible to get the information by
searching the autobuilder log, it is not convenient. Moreover, future
developments will allow to generate multiple reports (with different bases for
comparison) in a single build.
As a consequence, embed target and base revisions in the report header
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since we are now force-fetching base revisions and target revisions for
regression report generation, we can make testresults clone even more "shallow"
to increase clone speed in CI pipelines
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we try to run send-qa-email to simulate past releases (for example, for
development or debugging), the execution will very likely fail because the
target revision to examine (ie: the poky revision) is too old, and as a
consequence is not contained in the testresults shallow clone anymore (because
testsresults history keeps moving forward as builds are triggered on
autobuilder). As a consequence, force-fetch the "target" revision, as it is
already done for the "base" revision
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The "previous version" computation bring many edge cases depending on the
version under release. Add a basic test suite to validate currently implemented
computation strategy and to prevent mistakes when ediiting it in the future
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of only generating regressions reports against HEAD of relevant branch, compute
most relevant tag (ie : release) against which we can check for regressions. General rules
introduced are the following :
- milestone release is checked against previous milestone if possible, otherwise
against major release
- point release is checked against previous point release if possible,
otherwise against major release
- major release is checked against previous major release
- a non release build is checked against base branch
Examples :
- 4.1.2.rc1 is checked against yocto-4.1.1
- 4.1.2 is checked against yocto-4.1.1
- 4.1.1.rc1 is checked against yocto-4.1
- 4.1.1 is checked against yocto-4.1
- 4.1 is checked against yocto-4.0
- 4.1.rc4 is checked against yocto-4.0
- 4.1_M2.rc1 is checked against 4.1_M1
- 4.1_M2 is checked against 4.1_M1
- 4.1_M1.rc1 is checked against yocto-4.0
- 4.1_M1 is checked against yocto-4.0
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This wrapping allows importing send_qa_email.py in another script without
executing the QA emailing routine. This is useful for unit testing the script
internal functions
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Merely renaming send-qa-email to send_qa_email.py would imply to update
yocto-autobuilder2 scripts, which can be daunting. In order to preserve
compatibility with current configuration, add a symlink pointing to the
new script
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This rename fixes send-qa-email regarding python guidelines, which allows to
import the script content in another script (for unit testing purpose for
example)
Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The pub/non-release directories have aged out and no longer work.
Switch to a stable location and bump the version to 4.1.2 as well.
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
This will be particularly useful for testing Y2038 issues,
as that is the only 32 bit target that can be run at native speed
on the AB cluster.
Whether a-full should get fast ptests or full ptests can be decided
after collecting AB execution times for both.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It happened that the git repositories were dirty and resulted in
incorrect files being used. Let's use git clean -ffdx to force a
completely clean git repositories before and after checking out a branch
so that nothing is left from or to another branch build. Also git reset
--hard to undo changes to tracked files.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Reported-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> #https://lore.kernel.org/yocto-docs/e50abe3c777e4a23a752a3ec25ad0b2a@axis.com/
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We disabled meta-gplv2 for master, ensure the repo isn't used in the
collection or QA scripts.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the recent template config changes to oe-core its content is no longer
backwards compatible.
'-f' is to guarantee future compatibility with build setups where the file
is not present.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Archive the buildstats of every failed build and 1% of random builds. Convert
the time-stamped buildstats directory to a compressed tarball using the
hostname as a prefix (to the file name only) so that one can identify the
source machine. Move these tarballs to the directory:
testresults/<build_name>/buildstats/
The archiving is performed during the "collect results" step.
Signed-off-by: Aryaman Gupta <aryaman.gupta@windriver.com>
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Master branch of the yocto-docs now autogenerates the releases.rst file
and the file therefore does not exist anymore.
Signed-off-by: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
`git checkout <pathspec>` without <tree-ish> only undo the changes made
to pathspec compared to HEAD. It is not guaranteed that HEAD is on
latest commit of master branch from upstream when running this script,
therefore let's add the <tree-ish> argument to specify which branch to
use to make sure set_versions.py is the latest, otherwise getlatest
might return the wrong information.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We may not have fetched the master revision if a different one was being
built so using origin/master works consistently in all cases.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Fix a syntax error causing the script to fail
* Skip symlinks files (symlink directories were already skipped)
* Drop unneeded whitespace
* Abstract path join to a variable
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a script which adds banners to the old docs and outdated dunfell docs
so that the user knows they're using old documentation.
Signed-off-by: Abongwa Bonalais Amahnui <abongwabonalais@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For some reason, the old docs tarball includes many eclipse
subdirectories which are just cluttering the docs website up.
Therefore, let's just not extract eclipse directories from the tarball.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Keeping all the previous json files in tree has too much overhead
so drop the old ones. Honister doesn't support cve-check with json.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
