Commit Graph

60 Commits

Author SHA1 Message Date
Richard Purdie
142caae69e run-config/config.json: Add support for installing extra tools for specific targets
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-07 22:28:29 +00:00
Alexis Lothoré
44650c9681 scripts/send_qa_email: re-clarify base and target revisions
There are some inversions in words used to describe elements of comparison
for regression reporting: the main function of send_qa_email starts using
"base" to talk about the target revision and "compare" to talk about the
reference against which it is compared. Then later in the script, the
"base" is used as "base of comparison"/reference revision, while the
"target" branch/revision appears. This becomes quite confusing when we need
to update the script

Re-align wording to avoid confusion:
- always use "target" to talk about current branch/revision of interest
  (the newest)
- always use "base" to talk about the reference branch/revision  (the
  oldest), against which we want to compare the target revision

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-04 23:39:50 +01:00
Alexis Lothoré
c2afe19aba scripts/utils.py: replace BUILD_HISTORY_DIRECTPUSH with hardcoded condition
It has been observed that when a new release branch is created, it is quite
easy to forget to update the BUILD_HISTORY_DIRECTPUSH variable, which leads
to failures in autobuilder like test results not being pushed.
Replace the BUILD_HISTORY_DIRECTPUSH usage with a hardcoded condition which
validates any branch in poky representing a "main" branch, i.e all branches
not ending in "-next"

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-26 16:27:11 +01:00
Alexis Lothoré
d6018b891a scripts/send-qa-email: Generate regression reports against most relevant release
Instead of only generating regressions reports against HEAD of relevant branch, compute
most relevant tag (ie : release) against which we can check for regressions. General rules
introduced are the following :
- milestone release is checked against previous milestone if possible, otherwise
  against major release
- point release  is checked against previous point release if possible,
  otherwise against major release
- major release is checked against previous major release
- a non release build is checked against base branch
Examples :
- 4.1.2.rc1 is checked against yocto-4.1.1
- 4.1.2 is checked against yocto-4.1.1
- 4.1.1.rc1 is checked against yocto-4.1
- 4.1.1 is checked against yocto-4.1
- 4.1 is checked against yocto-4.0
- 4.1.rc4 is checked against yocto-4.0
- 4.1_M2.rc1 is checked against 4.1_M1
- 4.1_M2 is checked against 4.1_M1
- 4.1_M1.rc1 is checked against yocto-4.0
- 4.1_M1 is checked against yocto-4.0

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-01-26 22:40:29 +00:00
Richard Purdie
400e1e1b2b scripts/utils: Allow tags to be updated from upstream (meta-agl has force pushed them)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-29 16:44:03 +01:00
Richard Purdie
8702fdbb72 Clarify license and copyright information
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-17 10:29:45 +01:00
Richard Purdie
e420545f62 utils: Limit fetch optimisation to master for now
The depth code doesn't work if the branch doesn't exist in the main repo
mirror and only the remote.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-01 14:15:54 +00:00
Richard Purdie
5dc0559cc8 prepare-shared-repo/utils: Limit HEAD clones to shallow depth to save time/space
By not syncing all the history is is possible to save some time/space
in the checkout process since we never use this data. This reduces data
from 650MB to 400MB or with the tarball, 416MB to 55MB.

The logic for the commands needs to be tweaked to handle this and as
written it can't work in non-HEAD revision case but that isn't a commonly
used situation.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-31 13:03:22 +00:00
Yi Fan Yu
0d938352d7 scripts/utils.py: Add reporting for yocto-check-layer
The default behavior is to look for a bitbake command,
which fails and produces a confusing output of [-1:].

[YOCTO #14208]

Signed-off-by: Yi Fan Yu <yifan.yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-25 17:02:28 +01:00
Richard Purdie
23d65680f8 run-config: Adapt to two pass execution
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-20 23:32:26 +00:00
Richard Purdie
eb295f62f1 utils: Teach buildtools script handling about unset
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-23 08:53:19 +01:00
Richard Purdie
e47a9a26f5 scripts/send-qa-email,utils: Use buildtools if present
resulttool has python version requirements so use buildtools if present.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-17 07:44:46 +01:00
Richard Purdie
a677305551 scripts/utils: Ensuring new logging is flushed from buffers
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-24 18:31:10 +01:00
Richard Purdie
f3892fac1e scripts/utils: Add more logging for fetchgitrepo
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-24 18:27:56 +01:00
Richard Purdie
4c9358ac78 scripts/utils: Ensure buildtools tarball hash matches any supplied value
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-14 22:42:25 +01:00
Vineela
3aefa5a763 scripts/utils.py: Convert md5sum to sha256sum for layer tarballs
Signed-off-by: Vineela Tummalapalli <vineela.tummalapalli@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-26 21:33:42 +01:00
Richard Purdie
033ae6ffbd scripts/run-config: Share buildtools setup code with shared-repo-unpack
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-27 19:05:11 +01:00
Steve Sakoman
e79f6ddafd scripts/utils.py: Fix confusing naming of getcomparisonbranch() return value
This code was copy/pasted but in this case the variables should
be called "compare", not base.

Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-23 14:42:38 +01:00
Richard Purdie
79dd2f94aa send-qa-email/utils: Improve new branch regression script handling
There are several issues:

* New branches don't currently have git regression history
* The regression tool errors if there isn't anything to compare against

To fix this, create a branch with history and only generate a regression
report if there are commits to compare against.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-16 16:25:43 +00:00
Richard Purdie
becdbd3729 utils: Fix incorrect module usage
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-12 18:21:51 +01:00
Richard Purdie
f9bdcf888f config.json/utils: Allow buildhistory to be configurable and disable for oe-selftest in qemu-arch
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-12 16:23:54 +01:00
Richard Purdie
a0963045de scripts/utils: Simplify buildhistory path, no need for such deep paths
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-22 00:07:37 +00:00
Richard Purdie
5130ab9f21 utils: Move getcomparisonbranch() to common function library
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-21 17:13:16 +00:00
Richard Purdie
e5310650b0 scripts/utils: Ensure list filtering code doesn't match appends/prepends
We want all INHERIT += to apply for example

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-12 14:18:09 +00:00
Richard Purdie
5a2eddd6bd setup-config/utils: Improve extravars handling
Currently there was no way to override an existing extra vars
assignment, which meant BB_NUMBER_THREADS was impossible to change.

This adds code to allow the higher override of a value to override
the other settings which makes the code function more as a user
would expect.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-11 11:05:49 +00:00
Richard Purdie
7f6a3b7fba scripts/utils: Fix string/float combination error
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-09 14:17:10 +00:00
Richard Purdie
dabf9d6e20 scripts/utils: Fix variable name conflict
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-09 14:14:34 +00:00
Richard Purdie
8ce219dda5 run-config/utils: Allow headers to show the difference in time since the previous header
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-09 14:06:52 +00:00
Paul Eggleton
d7d1a2b345 scripts: use argparse for command line parsing
[RP: Minor tweaks made to adpat to buildbot and option naming

Fixed shared-repo-unpack for publishing when no cache-dir supplied]

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-30 14:36:07 +01:00
Richard Purdie
89b5f50dd2 utils: Fix previous commit
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-06 15:55:02 +01:00
Richard Purdie
579b732755 utils: Filter reponame to only use base repository name, not a full url
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-06 15:48:30 +01:00
Richard Purdie
621143cee4 scripts: Various buildhistory code fixes/improvements
* Remove the code disabling it
* Move to a non-shared directory by default (but still support absolute pathnames
  if wanted)
* Use shallow clones by default for speed
* Allow force pushing 'fork' mode branches

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-06 15:17:41 +01:00
Richard Purdie
696eacc94a utils: Ensure unspecified variables aren't expanded badly
We want ${BUILDDIR} to stay as ${BUILDDIR}, not become BUILDDIR
for example.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-03 14:57:56 +01:00
Richard Purdie
42416bdc9c utils: Ensure we don't infinitely recurse for variable expansion
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-03 14:08:40 +01:00
Richard Purdie
94cc0afb1c utils: Enhance loadconfig to better merge dicts
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-21 00:04:11 +01:00
Richard Purdie
b7cf6d8209 utils: Allow customisation using ABHELPER_JSON from the environment
Usage is documented in README and an example, local-example.json is
included.

Also clean up the parameter to loadconfig() as its actually not needed.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-20 23:16:10 +01:00
Richard Purdie
0ccbfc0e05 utils: Ensure list entries also are expanded
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-20 23:07:09 +01:00
Richard Purdie
547a3793fc utils: Fix for python 3.5 and before, remove debug
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-20 22:14:46 +01:00
Richard Purdie
d6253df2bc scripts: Convert to use data expansion pretty much everywhere
We want to be able to use data expansion/variable substitution
in most variables so convert to use the utils API for this.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-20 16:21:16 +01:00
Richard Purdie
d2c0dcd1d9 utils: Add data expansion handling/API
The code clearly won't scale without some kind of variable substitution/
variable expansion mechanism, so add one.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-20 16:20:08 +01:00
Richard Purdie
fde1417b5d utils: Ensure updates on the remote branch are applied for revison==HEAD
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-09 11:19:01 -08:00
Richard Purdie
c8e4cc8483 utils: Ensure fetchgitrepo fetches revisions as well as tags
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-01 23:29:59 +00:00
Richard Purdie
a37a62a15c utils: Fix typo
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-07 11:29:38 +00:00
Richard Purdie
0a6549a913 utils: Add timestamp to log headers
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-07 10:59:18 +00:00
Richard Purdie
53618a59cb utils: Handle oe-selftest in error reports
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-07 10:39:43 +00:00
Richard Purdie
eb0f56cece utils: Need to flush *after* the header, not before
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-05 23:25:12 +00:00
Richard Purdie
9948821070 utils: Ensure header flushes stdout/stderr first to order output correctly
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-05 22:33:21 +00:00
Richard Purdie
193ea5a9fe utils: Ensure revision has no whitespace
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-05 18:00:06 +00:00
Richard Purdie
477b75ac3c utils: Add missing output decoding
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-05 17:58:17 +00:00
Richard Purdie
a71ea84970 Add support for publishing the layer tarballs
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-05 17:40:49 +00:00