Commit Graph

143 Commits

Author SHA1 Message Date
Richard Purdie
f022d37f70 build-perf-test-wrapper/report-email: Tweak for ab-helper usage
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-22 23:01:57 +00:00
Richard Purdie
fbe172bbdf scripts: Import build-perf scripts from oe-core
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-22 22:58:11 +00:00
Richard Purdie
d36deb8543 run-config: Ensure log directory (builddir) exists
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-26 11:34:03 +00:00
Richard Purdie
d6049a972f publish-artefacts: Fix an incorrect target directory for qemux86-64-lsb
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-21 17:38:38 +00:00
Richard Purdie
a1e1525178 config.json/publish-artefacts: Split qemu machines from real hardware and drop nightly prefix+rename
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-21 15:32:38 +00:00
Richard Purdie
dfd07f06f8 scripts/run-config: Ensure we rename the build directory before exitting when failing
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-21 11:47:46 +00:00
Richard Purdie
e9b855fe8c scripts: Add results collection script and option in run-config 2018-11-20 23:32:09 +00:00
Richard Purdie
10b47e4928 run-config: Fix missing / typo
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-20 15:24:42 +00:00
Richard Purdie
37b3943ca7 scripts/run-config: Rename any completed build directory so that other builds can't reference paths within it
This should further protect us against contamination between different build directories
as old build directory paths should no longer exist.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-20 14:52:00 +00:00
Richard Purdie
9e181c3ebb run-config: Remove successful build directories straight away
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-20 14:50:35 +00:00
Richard Purdie
19bd8ebfee setup-config: Also set HELPERBUILDDIR
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-11 09:26:51 +00:00
Richard Purdie
3c57c77a77 run-config: Fix typo
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-10 14:45:26 +00:00
Richard Purdie
885d79c393 config.json/run-config: Add HELPERBUILDDIR and fix sstate reuse for nightly-bringup
There is no current mechanism to reference the 'builddir' within the config.json,
so add HELPERBUILDDIR for this purpose.

Use this to set SSTATEDIR for nightly-bringup, meaning sstate arefacts should be
shared for use between the different oe-selftest threads, speeding up builds.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-10 14:42:07 +00:00
Richard Purdie
7f6a3b7fba scripts/utils: Fix string/float combination error
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-09 14:17:10 +00:00
Richard Purdie
dabf9d6e20 scripts/utils: Fix variable name conflict
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-09 14:14:34 +00:00
Richard Purdie
8ce219dda5 run-config/utils: Allow headers to show the difference in time since the previous header
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-09 14:06:52 +00:00
Richard Purdie
df2e005030 scripts/setup-config: Allow SSTATEDIR to unset
The SSTATEDIR was accidentally using getconfiglist instead of getconfigvar which
meant it couldn't be unset. Fix to use var so the values in config.json work
as intended.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-08 21:22:01 +00:00
Richard Purdie
478eb6d78c scripts/run-config: Export SCRIPTSDIR to avoid scripts having to mess with paths
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-08 20:45:56 +00:00
Richard Purdie
5fe0e21b8a scripts/send-qa-email: Handle systems with sendmail in sbin
Many of the distros we use have sendmail in /sbin or /usr/sbin
which may not be in the path for non-interactive users. Handle
this in the script to avoid failures where tests work but the
real QA email fails.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-05 16:57:37 +00:00
Richard Purdie
e23d0b3ab2 scripts/setup-config: Ensure PACKAGE_CLASSES is set
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-10 11:40:41 +01:00
Richard Purdie
b27ef9a53d run-config: Renumber the log files to match step numbers
Also print a summary of the logfiles which contain failures.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-05 14:11:33 +01:00
Aaron Chan
842def0599 publish-artefacts: Add deployment BSP support on x86_64
Add in the support to publish images into the designated path.
BSP packages of the previous build will be cleaned up before
new BSP packages are copied over. This ensures the previous
image will not be retained and causes conflicts before image
is loaded into x86_64 (MTURBOT64) hardware.

Signed-off-by: Aaron Chan <aaron.chun.yew.chan@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-30 16:21:51 +01:00
Paul Eggleton
d7d1a2b345 scripts: use argparse for command line parsing
[RP: Minor tweaks made to adpat to buildbot and option naming

Fixed shared-repo-unpack for publishing when no cache-dir supplied]

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-30 14:36:07 +01:00
Richard Purdie
c41c2e0a5a run-config: Add a new option to control echoing of log output
The new buildbot code can directly monitor the logfiles itself so
we need to be able to optionally stop echoing onto the console.

Add missing flush() calls and print the command being used and logfile
so its clear what is happening on stdout too.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-30 13:25:05 +01:00
Richard Purdie
1cb54ada86 scripts/checkvnc: Add checkvnc script from old autobuilder codebase
Update the scripts to correctly reference the new script.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-30 13:22:16 +01:00
Aaron Chan
0be30e2b54 run-config: Reverse the oder of remove layers
This patch fixes the "Collection Error during parsing layer conf"
when a parent layer is accidentally added before depend layers.
When removing layers from bblayers, we should not be following
the same sequence of adding the layers, it should be done in a
reversed order. This is an assumption that the layers have their
dependent layers added before the parent layers are added.
In general, the parent layer require to be remove first before
its child dependencies layers.

Signed-off-by: Aaron Chan <aaron.chun.yew.chan@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-08 11:15:20 +01:00
Richard Purdie
a49b11979e run-config: Add a test mode using an enviornmental variable
Testing this code is becoming increasingly problematic. This allows
ABHELPERTEST to be set from the environment and means the script will
still create directories and configuration files but it will not execute
anything, just print what would be executed.

This allows faster validation of changes to the configurations/code.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-06 16:01:24 +01:00
Richard Purdie
89b5f50dd2 utils: Fix previous commit
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-06 15:55:02 +01:00
Richard Purdie
579b732755 utils: Filter reponame to only use base repository name, not a full url
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-06 15:48:30 +01:00
Richard Purdie
621143cee4 scripts: Various buildhistory code fixes/improvements
* Remove the code disabling it
* Move to a non-shared directory by default (but still support absolute pathnames
  if wanted)
* Use shallow clones by default for speed
* Allow force pushing 'fork' mode branches

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-06 15:17:41 +01:00
Richard Purdie
004d403b6a upload-error-reports: Shorted name since server has a 50 char limit
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-04 10:45:29 +01:00
Richard Purdie
696eacc94a utils: Ensure unspecified variables aren't expanded badly
We want ${BUILDDIR} to stay as ${BUILDDIR}, not become BUILDDIR
for example.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-03 14:57:56 +01:00
Richard Purdie
42416bdc9c utils: Ensure we don't infinitely recurse for variable expansion
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-03 14:08:40 +01:00
Richard Purdie
94cc0afb1c utils: Enhance loadconfig to better merge dicts
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-21 00:04:11 +01:00
Richard Purdie
b7cf6d8209 utils: Allow customisation using ABHELPER_JSON from the environment
Usage is documented in README and an example, local-example.json is
included.

Also clean up the parameter to loadconfig() as its actually not needed.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-20 23:16:10 +01:00
Richard Purdie
0ccbfc0e05 utils: Ensure list entries also are expanded
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-20 23:07:09 +01:00
Richard Purdie
547a3793fc utils: Fix for python 3.5 and before, remove debug
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-20 22:14:46 +01:00
Richard Purdie
d6253df2bc scripts: Convert to use data expansion pretty much everywhere
We want to be able to use data expansion/variable substitution
in most variables so convert to use the utils API for this.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-20 16:21:16 +01:00
Richard Purdie
d2c0dcd1d9 utils: Add data expansion handling/API
The code clearly won't scale without some kind of variable substitution/
variable expansion mechanism, so add one.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-20 16:20:08 +01:00
Richard Purdie
fcd4a3891f scripts/send-qa-email: Handle form enable/disable of email
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-10 23:17:51 +01:00
Richard Purdie
001353ea6f send-qa-email: Ensure we don't run when not publishing
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-10 14:10:29 +01:00
Richard Purdie
51054804fc send-qa-email: Resolve revisions into full hashes
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-16 16:03:42 +01:00
Richard Purdie
477d511aa0 send-qa-email: Allow execution (chmod)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-14 17:00:42 +01:00
Richard Purdie
25d574946a config.json: Add QA/publish info and script to mail QA
Add a script which allows QA to be emailed upon build completion
for QA test runs.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-14 13:04:50 +01:00
Ross Burton
b3eb077391 pre-commit-hook: add a pre-commit hook to verify the JSON
Signed-off-by: Ross Burton <ross.burton@intel.com>
2018-05-11 11:36:06 +01:00
Richard Purdie
74de885ad7 scripts/run-config: Add more detail when running EXTRA*CMDS
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-09 15:44:38 +01:00
Richard Purdie
7175195f7a publish-artefacts: Fix eclipse artefact publish dir
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-10 05:38:20 -08:00
Richard Purdie
fde1417b5d utils: Ensure updates on the remote branch are applied for revison==HEAD
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-09 11:19:01 -08:00
Richard Purdie
c8e4cc8483 utils: Ensure fetchgitrepo fetches revisions as well as tags
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-01 23:29:59 +00:00
Richard Purdie
ef9b1c8d47 run-config: Add eclipe build support
This means making the configuration generation conditional, the error
reporting conditional and allowing non bitbake commands.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-01 21:41:30 +00:00