Commit Graph

52 Commits

Author SHA1 Message Date
Richard Purdie
21ebf7df51 scripts/run-config: Add --build-type commandline option
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-06 14:24:04 +01:00
Richard Purdie
68f351ed83 run-config: Fix uninitalized variable problem
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-12 18:17:16 +01:00
Richard Purdie
f9bdcf888f config.json/utils: Allow buildhistory to be configurable and disable for oe-selftest in qemu-arch
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-12 16:23:54 +01:00
Richard Purdie
72b22dc785 config.json/run-config/build-perf-test-wrapper: Allow build performance branch comparisions
This ensures that release branch performance build tests work on the correct
branch and that comparisions between mut+master and master-next+master
start to work correctly. It also ensures reports for the current
revision are generated even if later revisions are built.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-09 17:24:16 +00:00
Richard Purdie
6fbc1321a0 config.json/run-config: Ensure build-perf publish-dir contains the target name
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-05 22:22:46 +00:00
Richard Purdie
efeb042a77 config.json/build-perf-test-wrapper.sh: Add support for publishing to the standard results location
This makes the build performance code behave similarly to the other
results publishing code using the standard location.

Also tweak the report names so they sort.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-01 15:18:57 +00:00
Richard Purdie
b480d77271 scripts/run-config: Allow command failures to be non-fatal
We need the final logging to show which command steps contain errors so print
failing commands but allow the code to continue and show the final log output.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-31 12:57:54 +00:00
Richard Purdie
81382c6c5d scripts/run-config: Always run bitbake with -k option
This ensures the builds get as far as they can when there is failure rather
that stopping at the first sign of one. This means caches are hotter and
allows for faster respins, at the cost of the failure being lost in longer
logs.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-31 12:53:00 +00:00
Richard Purdie
d36deb8543 run-config: Ensure log directory (builddir) exists
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-26 11:34:03 +00:00
Richard Purdie
dfd07f06f8 scripts/run-config: Ensure we rename the build directory before exitting when failing
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-21 11:47:46 +00:00
Richard Purdie
e9b855fe8c scripts: Add results collection script and option in run-config 2018-11-20 23:32:09 +00:00
Richard Purdie
10b47e4928 run-config: Fix missing / typo
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-20 15:24:42 +00:00
Richard Purdie
37b3943ca7 scripts/run-config: Rename any completed build directory so that other builds can't reference paths within it
This should further protect us against contamination between different build directories
as old build directory paths should no longer exist.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-20 14:52:00 +00:00
Richard Purdie
9e181c3ebb run-config: Remove successful build directories straight away
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-20 14:50:35 +00:00
Richard Purdie
3c57c77a77 run-config: Fix typo
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-10 14:45:26 +00:00
Richard Purdie
885d79c393 config.json/run-config: Add HELPERBUILDDIR and fix sstate reuse for nightly-bringup
There is no current mechanism to reference the 'builddir' within the config.json,
so add HELPERBUILDDIR for this purpose.

Use this to set SSTATEDIR for nightly-bringup, meaning sstate arefacts should be
shared for use between the different oe-selftest threads, speeding up builds.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-10 14:42:07 +00:00
Richard Purdie
8ce219dda5 run-config/utils: Allow headers to show the difference in time since the previous header
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-09 14:06:52 +00:00
Richard Purdie
478eb6d78c scripts/run-config: Export SCRIPTSDIR to avoid scripts having to mess with paths
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-08 20:45:56 +00:00
Richard Purdie
b27ef9a53d run-config: Renumber the log files to match step numbers
Also print a summary of the logfiles which contain failures.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-05 14:11:33 +01:00
Paul Eggleton
d7d1a2b345 scripts: use argparse for command line parsing
[RP: Minor tweaks made to adpat to buildbot and option naming

Fixed shared-repo-unpack for publishing when no cache-dir supplied]

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-30 14:36:07 +01:00
Richard Purdie
c41c2e0a5a run-config: Add a new option to control echoing of log output
The new buildbot code can directly monitor the logfiles itself so
we need to be able to optionally stop echoing onto the console.

Add missing flush() calls and print the command being used and logfile
so its clear what is happening on stdout too.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-30 13:25:05 +01:00
Richard Purdie
1cb54ada86 scripts/checkvnc: Add checkvnc script from old autobuilder codebase
Update the scripts to correctly reference the new script.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-30 13:22:16 +01:00
Aaron Chan
0be30e2b54 run-config: Reverse the oder of remove layers
This patch fixes the "Collection Error during parsing layer conf"
when a parent layer is accidentally added before depend layers.
When removing layers from bblayers, we should not be following
the same sequence of adding the layers, it should be done in a
reversed order. This is an assumption that the layers have their
dependent layers added before the parent layers are added.
In general, the parent layer require to be remove first before
its child dependencies layers.

Signed-off-by: Aaron Chan <aaron.chun.yew.chan@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-08 11:15:20 +01:00
Richard Purdie
a49b11979e run-config: Add a test mode using an enviornmental variable
Testing this code is becoming increasingly problematic. This allows
ABHELPERTEST to be set from the environment and means the script will
still create directories and configuration files but it will not execute
anything, just print what would be executed.

This allows faster validation of changes to the configurations/code.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-06 16:01:24 +01:00
Richard Purdie
621143cee4 scripts: Various buildhistory code fixes/improvements
* Remove the code disabling it
* Move to a non-shared directory by default (but still support absolute pathnames
  if wanted)
* Use shallow clones by default for speed
* Allow force pushing 'fork' mode branches

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-06 15:17:41 +01:00
Richard Purdie
b7cf6d8209 utils: Allow customisation using ABHELPER_JSON from the environment
Usage is documented in README and an example, local-example.json is
included.

Also clean up the parameter to loadconfig() as its actually not needed.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-20 23:16:10 +01:00
Richard Purdie
74de885ad7 scripts/run-config: Add more detail when running EXTRA*CMDS
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-09 15:44:38 +01:00
Richard Purdie
ef9b1c8d47 run-config: Add eclipe build support
This means making the configuration generation conditional, the error
reporting conditional and allowing non bitbake commands.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-01 21:41:30 +00:00
Richard Purdie
a979873199 run-config: Remove layers after EXTRACMDS to fix oe-selftest
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-07 10:35:22 +00:00
Richard Purdie
d8f1e3ec27 run-config: Replace tee usage
tee hides the exit code of the pipeline. Instead use subprocess to write out the logfile.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-06 12:58:41 +00:00
Richard Purdie
7d31d6cb84 run-config: Handle build directory being removed/recreated
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-05 22:32:57 +00:00
Richard Purdie
08410f5257 run-config: Fix step numbering printed
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-05 22:20:42 +00:00
Richard Purdie
38af6b0f6c run-config: Add missing parameter documentation
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-05 16:38:39 +00:00
Richard Purdie
297219ecc0 run-config: Clean up stepnum a bit
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-05 16:36:56 +00:00
Richard Purdie
f66abd6270 run-config: Print step x of y headers and summary of the number of failures
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-05 16:34:51 +00:00
Richard Purdie
4239708b85 Add error reporting server handling
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-05 13:16:41 +00:00
Richard Purdie
2808a9bc72 run-config: Add publish artefact support
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-31 10:12:52 +00:00
Richard Purdie
376ad36d4d Move buildhistory init to run-config (and factor common functionality to utils)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-23 13:49:38 +00:00
Richard Purdie
806957b392 scripts: Fix help text and program name
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-23 13:09:53 +00:00
Richard Purdie
45b14f6c80 utils: Create helper for loading config.json
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-22 15:20:00 +00:00
Richard Purdie
b47ea0eaa6 run-config: Also use templating support (move to utils)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-20 13:41:33 +00:00
Richard Purdie
40eb3f0820 run-config: Fix return value handling
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-20 13:40:01 +00:00
Richard Purdie
db8cf29820 run-config: Add support for adding/removing layers and extra commands in steps
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-20 11:14:58 +00:00
Richard Purdie
9996fff768 run-config: Ensure stdout/stderr is flushed and fix DISPLAY
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-19 12:26:00 +00:00
Richard Purdie
0fbdf0c27f run-config: Fix typo
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-18 23:02:48 +00:00
Richard Purdie
4ceb9e49d4 run-config: Fix off by one error
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-18 22:29:46 +00:00
Richard Purdie
307ce0c502 run-config: Print error message/exit code upon failure
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-18 18:30:34 +00:00
Richard Purdie
8cd1d0ccb8 Handle build-appliance srcrev
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-18 18:14:06 +00:00
Richard Purdie
0a045ed92d run-config: Cleanup/fix commands
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-18 17:52:30 +00:00
Richard Purdie
94460edf3b run-config: Use subprocess directly so we see the output
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-18 17:10:02 +00:00