Commit Graph

239 Commits

Author SHA1 Message Date
Richard Purdie
098de71cbd scripts/generate-testresult-index.py: Ensure we're not always rerunning resulttool
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-21 19:30:41 +01:00
Richard Purdie
83c2052781 scripts/generate-testresult-index: Ensure backwards compatibility with older layout
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-21 19:23:49 +01:00
Richard Purdie
6d6c45cbda scripts/generate-testresult-index: Update after 'posttrigger' renaming broke the index generation
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-21 19:17:58 +01:00
Richard Purdie
3634672674 scripts/send-qa-email: Fix buildtools handling
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-24 12:27:58 +01:00
Richard Purdie
eb295f62f1 utils: Teach buildtools script handling about unset
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-23 08:53:19 +01:00
Alexander Kanavin
0f8534c747 auh: correct the SMTP server in config file
Michael Halstead has confirmed this.

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-17 12:31:52 +01:00
Richard Purdie
e47a9a26f5 scripts/send-qa-email,utils: Use buildtools if present
resulttool has python version requirements so use buildtools if present.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-17 07:44:46 +01:00
Richard Purdie
838be1a00c scripts: Add runqemu-renice.c for renicing runqemu
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-05 14:59:14 +01:00
Michael Halstead
01c16fa772 scripts/send-qa-email: Update to dedicated push capable domain
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-01 18:14:31 +01:00
Richard Purdie
a677305551 scripts/utils: Ensuring new logging is flushed from buffers
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-24 18:31:10 +01:00
Richard Purdie
f3892fac1e scripts/utils: Add more logging for fetchgitrepo
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-24 18:27:56 +01:00
Quentin Schulz
7571e44fc1 scripts: run-auh: update upgrade-helper name
Since https://lists.yoctoproject.org/g/yocto/message/50282 was merged in
autoupgrade-helper git repo, the python script isn't called
upgradehelper.py anymore but upgrade-helper.py for consistency sake.

Let's update the run-auh script so that it's still working.

Signed-off-by: Quentin Schulz <foss@0leil.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-17 11:38:32 +01:00
Richard Purdie
4c9358ac78 scripts/utils: Ensure buildtools tarball hash matches any supplied value
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-14 22:42:25 +01:00
Alexander Kanavin
d88728431a auh-config: send AUH update emails to oe-core list as well
Sadly, many maintainers do not act on AUH mails in a timely manner
(or do not act at all). Sending these emails to oe-core list as well
will allow other people to pick them up, and ease up the path
to doing recipe version maintenance.

Also, recently the amount of outdate recipes has been greatly reduced,
so there should not be too many of those emails each month.

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-30 17:33:41 +01:00
Vineela
3aefa5a763 scripts/utils.py: Convert md5sum to sha256sum for layer tarballs
Signed-off-by: Vineela Tummalapalli <vineela.tummalapalli@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-26 21:33:42 +01:00
Richard Purdie
eb86b7fd11 send-qa-email: Fix stable branch test result history issues
On stable release branch builds, it will clone "master" since comparebranch
is None. It will then fail to push since there is already a dunfell
branch upstream with changes on which it needs to rebase on top of.
The change to use shallow clones significantly increases this problem.

To fix this, fall back to cloning basebranch by name, then master
if it doesn't exist.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-09 16:51:23 +01:00
Tummalapalli, Vineela
835ac325a4 publish-artefacts: Improve sha256 checksum handling
The existing checksum files were encoding full paths. This patch changes
this to use just the filename which allows relocation of the directories
and is more appropriate for sharing over http.

Signed-off-by: Vineela Tummalapalli <vineela.tummalapalli@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-27 07:55:06 +01:00
Alexander Kanavin
cfa62a586a scripts: add a pair of scripts to set up and run Auto Upgrade Helper
This allows automating its setup and execution on all autobuilder worker machines;
previously there was a static setup on a dedicated machine, which wasn't
great from maintenance perspective.

To use:

scripts/setup-auh target_dir
scripts/run-auh target_dir

(run-auh can be run several times in a directory that
was previously set up)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-17 16:57:25 +01:00
Richard Purdie
80c53d03ea scripts/publish-artefacts: Convert md5sum to sha256sum
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-05 13:29:01 +01:00
Richard Purdie
033ae6ffbd scripts/run-config: Share buildtools setup code with shared-repo-unpack
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-27 19:05:11 +01:00
Richard Purdie
0a7bf5191e scripts/shared-repo-unpack: Accept workername parameter
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-27 17:09:03 +01:00
Richard Purdie
aa81bd61c5 scripts/run-config: Add extra debug about buildtools to show when active
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-27 17:07:28 +01:00
Steve Sakoman
00df7cbc02 scripts/send-qa-email: fix bug in git push logic for yocto-testresults
We were mistakenly doing a force push if the branch was in either
BUILD_HISTORY_FORKPUSH or BUILD_HISTORY_DIRECTPUSH.

Now we force push for branches in BUILD_HISTORY_FORKPUSH, regular push
for branches in BUILD_HISTORY_DIRECTPUSH, and no push if the branch is
in neither list.

Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-23 14:43:36 +01:00
Steve Sakoman
e79f6ddafd scripts/utils.py: Fix confusing naming of getcomparisonbranch() return value
This code was copy/pasted but in this case the variables should
be called "compare", not base.

Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-23 14:42:38 +01:00
Richard Purdie
a1ea9f6e60 scripts/run-config: Resolve path early for buildtools
The buildtools path contains "../" which may not resolve correctly by the
shell if the final directory doesn't exist. Avoid this by using abspath()
on the expression straightaway.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-07 08:33:51 +01:00
Richard Purdie
a9f4f21ee9 run-config: Fix lockfile loop handling
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-12 18:13:42 +00:00
Richard Purdie
531cdb168e scripts/run-config: Fix BB_LOGFILE missing /
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-12 12:44:49 +00:00
Richard Purdie
c41c1d0c74 scripts/run-config: Ensure racing for the lockfile doesn't traceback
Traceback (most recent call last):
  File "yocto-autobuilder-helper/scripts/run-config", line 116, in <module>
    fcntl.flock(fileno, fcntl.LOCK_EX)
OSError: [Errno 22] Invalid argument

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-12 12:41:32 +00:00
Richard Purdie
1622cb7b09 scripts/run-config: Add support for local filesystem buildtools urls
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-12 11:58:13 +00:00
Richard Purdie
0902ededff scripts/run-config: Handle quoting issues in buildtools environment files
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-12 11:58:05 +00:00
Richard Purdie
8e7a052f07 scripts/run-config: Handle multiple builds racing with lockfile for buildtools
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-12 11:57:55 +00:00
Richard Purdie
37955b4b8d send-qa-email: Limit results clone to shallow depth to avoid timeouts
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-10 22:22:36 +00:00
Richard Purdie
91bd587bf3 scripts/run-config: Set BB_LOGCONFIG on autobuilder
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-09 22:43:49 +00:00
Richard Purdie
d4a9248fd0 scripts/run-config: Fix buildtools path
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-07 10:02:20 +00:00
Richard Purdie
fbdaa169c3 run-config: Add support for wrapping builds in a buildtools tarball
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-06 15:51:42 +00:00
Richard Purdie
17386f43c2 run-config: Add workername parameter
Allow the workername to be passed into the script. This is to allow future
buildtools tarball support based upon the worker.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-06 14:37:38 +00:00
Richard Purdie
ee6fc35569 build-perf/send-qa-email: Fallback to master as a comparision branch
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-01 17:44:34 +00:00
Richard Purdie
89633facc4 run-config: Preface log output with auto.conf settings to aid debugging
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-19 16:47:09 +00:00
Richard Purdie
1b78039d41 scripts/send-qa-email: Allow force pushing of basebranch based comparisions
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 15:25:42 +00:00
Richard Purdie
5f423e3a26 scripts/send-qa-email: Handle older git clients
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08 14:32:10 +00:00
akuster
32218185dd yocto-autobuilder-helper: remove mpc8315 from configs
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-02 10:38:30 +00:00
Richard Purdie
8bae4b2955 run-config/upload-error-reports: Handle builddir not existing
This fixes failures on the buildperf workers which do early cleanup of build
directories.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-12 13:02:57 +00:00
Richard Purdie
79dd2f94aa send-qa-email/utils: Improve new branch regression script handling
There are several issues:

* New branches don't currently have git regression history
* The regression tool errors if there isn't anything to compare against

To fix this, create a branch with history and only generate a regression
report if there are commits to compare against.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-16 16:25:43 +00:00
Richard Purdie
014a1c4b89 scripts/publish-artefact: Ensure ptest images are published
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-31 11:14:24 +00:00
Richard Purdie
aa5f3f7b39 build-perf-test-wrapper: Cannot clean up helper at this point
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-04 08:39:56 +01:00
Richard Purdie
488882ac9a build-perf-test-wrapper: Ensure worker checkout and sstate is cleaned up
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-03 16:49:56 +01:00
Richard Purdie
f54b2f5104 publish-artefacts: Update to use -alt naming for release output
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-02 10:44:51 +01:00
Richard Purdie
28ee9be07d config.py: Update -lsb -> -alt naming
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-02 10:44:00 +01:00
Richard Purdie
81b3587a90 run-config: Fix selftest tag filtering command
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-07 16:50:32 +01:00
Richard Purdie
68c19d067d config.json/run-config: Add configuration for toolchain testsuites
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-06 17:55:24 +01:00