Commit Graph

37 Commits

Author SHA1 Message Date
Aaron Chan
81273de5ac builders: Inherit build properties from nightly scheduler
This change enables the children jobs to inherit the build
property from nightly scheduler. Build property inherits
yocto release, milestone and rc number from parent job.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-07 10:42:06 +00:00
Richard Purdie
ad5baf5ce1 builders: Increase maxsteps to 9 to match current helper configuration maximums
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-10 11:12:25 +01:00
Richard Purdie
b1d2a6b8fb Revert "builders.py: Fix invalid option"
This reverts commit 85cd275481.
2018-09-05 20:48:32 +01:00
Richard Purdie
24085f2a34 builders: Improve log handling to match run-config step numbering
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-05 14:30:27 +01:00
Richard Purdie
b972e58722 builders/yocto_console_view: Add yp_build_branch property
This corresponds to the poky branch being used by default so its easier
for users to tell the builds apart and know which ones they are.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-05 11:50:56 +01:00
Richard Purdie
85cd275481 builders.py: Fix invalid option
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-05 11:50:56 +01:00
Richard Purdie
aed82199a1 builders.py: Fix invalid option
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-30 14:59:32 +01:00
Richard Purdie
772544ee27 builders: Adapt to autobuilder-helper argparse changes
In particular this lets us drop the horrible 'None' sentinals.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-30 14:43:55 +01:00
Richard Purdie
40fe66b97f Add custom log observer
We want seperate log files in the UI for each of the build steps. This
changes buildbot to monitor the command.log.X files which run-config
generates. It also searches the log output for errors and warnings and
summarises these neatly in the UI.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-30 14:41:44 +01:00
Richard Purdie
9200e30765 builders: Add an example of adding extra configuration to workers
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-24 09:22:12 +01:00
Richard Purdie
3ef80817ae builders: Connect in send_email form control to script
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-10 23:22:32 +01:00
Richard Purdie
223720737f schedulers/builders: Clean up deploy_artefacts/send_email/is_release into true boolean types
Lets use the proper BooleanParameter input type and boolean values for these.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-10 23:22:30 +01:00
Richard Purdie
2856c3283d builders: Drop got_revision hack and replace with yp_build_revision property
The got_revision trick, whilst quite neat, doesn't quite work and is a bit of
a hack breaking other buildbot internals. It gets et late in the builds leaving
a confusing build display in the meantime.

Use a dedicated property to store the "Yocto" build revision instead. This
property will be used by our own custom console UI.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-10 17:17:22 +01:00
Richard Purdie
ea05bf56ed builders: Pass sharedrepolocation to QA emails script
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-16 16:04:19 +01:00
Richard Purdie
a1596a5f29 builders: Fix basename version of get_publish_dest
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-15 15:54:32 +01:00
Richard Purdie
7e6b5d4119 builders: Add deploy_artefacts selection/property
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-14 15:12:56 +01:00
Richard Purdie
5b91d39dad builders: Add code to trigger QA email sscript in helper
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-14 13:05:13 +01:00
Richard Purdie
52bb871dd9 builders.py: Attempt to set revision earlier
This means the display will show the correct revision before triggering and waiting
for all the builds to complete making the display less confusing.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-09 17:22:29 +01:00
Richard Purdie
b5970d5b49 builders: Drop bitbake-selftest as moved into the helper
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-09 17:20:44 +01:00
Richard Purdie
b8e5905a07 builders.py: Ensure we use get_publish_dest, not config.publish_dest
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-21 23:03:40 +00:00
Graydon, Tracy
67a636ee44 /builders.py
Add the releases subdirectory to the release publishing destination path.

Signed-off-by: Graydon, Tracy <tracy.graydon@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-21 22:59:49 +00:00
Richard Purdie
5e41093974 builders.py: Don't use Interpolate inside functions
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-20 11:13:39 +00:00
Richard Purdie
e34df24042 builders: Fix syntax/typo in get_publish_dest()
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-20 10:57:28 +00:00
Richard Purdie
063c5798fd Wait for all triggers to complete before publishing
This means that when we stop builds, all triggered builds also stop.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-12 16:19:49 -07:00
Richard Purdie
04a134f2e7 builders.py: Tweak ordering of setproperty build revision
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-08 20:54:18 -08:00
Paul Eggleton
7bdc7c9dcd builders: set build revision to poky revision
By default, since the yocto-autobuilder-helper repository is the only
one that buildbot actually checks out in a step, the revision of that is
the one that gets associated with the build; however, it's much more
useful to have the revision set to the poky revision, so add a step that
does that.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-07 09:35:16 -08:00
Paul Eggleton
6c6772b908 builders: set early steps to halt on failure
If you can't clobber the build directory, check out the helper source
or unpack the repos successfully there's really not much point in
continuing, so stop the build immediately if they fail.

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-07 09:34:30 -08:00
Joshua Lock
d978426cd8 builders: fix get_publish_dest()
Various errors resulted from not having been able to test the initial
implmentation of get_publish_dest():
* incorrect logic to determine whether a build was a release
* passing a filename rather than a string representing file contents to
  json.loads()
* missing parameter on a call to buildbot's IProperties.setProperty()

Further we update the properties passed to the triggered builds via the
Trigger build steps to include the deploy_artefacts and publish_destination
properties.

This patch is able to succesfully supply required information for the
helper scripts to publish artefacts.

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2018-03-07 14:37:48 +00:00
Joshua Lock
f2b41bd937 builders: fix use of util.renderer decorated functions
These shouldn't be called at the time of buildstep creation, instead we
pass the method name and buildbot calls the function when the step starts.

http://docs.buildbot.net/current/manual/cfg-properties.html#renderer

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2018-03-07 11:06:17 +00:00
Joshua Lock
9540a56ffe builders: implement get_publish_dest()
Based on the SetDest step in the old codebase this function is used to
determine where the artefacts of a build should be published. In order to
avoid overwriting previously published content the function keeps a list of
destinations which have already been created and written to on the
controller and avoids reusing them.

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2018-03-07 10:58:17 +00:00
Joshua Lock
443ed86cec builders: implement get_sstate_release_number()
Uses the values submitted to the scheduler to determtine the major number
of the release for the purposes of publishing per-major release
shared-state artefacts.

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2018-03-07 10:18:04 +00:00
Joshua Lock
0e0fba1754 builders: fix clobber step for nightly builder
Copy/paste error resulted in incorrect factory variable name.

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2018-03-01 09:51:11 +00:00
Joshua Lock
7fabb26b63 builders: use yocto-autobuilder-helper's clobberdir script
Removing the build directory (with rm -fr) can take a significant amount
of time, longer than the standard buildbot timeout, which delays the
builds. Instead, we use the clobberdir script from the helper repository.

This script moves the directory contents out of the way to a location which
is reaped by a background process.
Unfortunately this assumes/requires a copy of the repository is kept in the
home directory of the user running buildbot.

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2018-02-28 21:58:18 +00:00
Joshua Lock
585ed37847 builders: pass all repo configuration properties to triggered builders
Ensure triggered builds have all of the branch_, commit_ and repo_
properties set. Most builders won't use them, but it's simpler to set them
all for all triggered builds than to special case specific builders.

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2018-02-28 12:17:31 +00:00
Joshua Lock
37a42c46fd builders: increase timeout for run-config invocations
run-config can run without producing output for longer than the default
1200s/20m timeout value. Increase the timeout to a generous 16200s/4.5h
in order to give the script plenty of time to run.

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2018-02-28 11:26:32 +00:00
Joshua Lock
20dcfb7159 Fix module name after rename
The repository was renamed yoctoabb from yoctoab, fix up module references
2018-02-22 14:05:19 +00:00
Joshua Lock
4fd189ab38 Initial prototype of using yocto-autobuilder-helper scripts
Initial prototype of using yocto-autobuilder-helper scripts from vanilla
buildbot to replicate yocto-autobuilder configuration.

* README.md is updated to describe goals and approach
* TODO contains known issues and work items, TODO: comments in the code
  point to specific locations of work

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2018-02-22 10:38:19 +00:00