Commit Graph

327 Commits

Author SHA1 Message Date
Ross Burton
91b05475ce config.py: add ubuntu2204-arm-1 (the Ampere Altra) to workers_arm
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-23 23:34:57 +01:00
Richard Purdie
b46908300d schedulers: Update meta-virt kirkstone config
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-04 16:13:36 +01:00
Richard Purdie
83f6475915 steps/writelayerinfo: Always rewrite the file as concat doesn't make sense
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-01 22:13:20 +01:00
Steve Sakoman
b074abdc83 config.py: define workers for kirkstone
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-29 16:25:17 +01:00
Richard Purdie
c46984fefa builders: Rather than use a central shared repo dir, use the local build directory
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-27 23:04:44 +01:00
Richard Purdie
e3e148cb4a builders: Avoid errors if sharedrepolocation is unset
Try and avoid errors like:

/bin/sh: 1: cd: can't cd to /poky
0631ddde57c27c25f5729bc0204476f8c7ff826f

for builds without a parent trigger and hence a shared checkout directory.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-27 22:49:19 +01:00
Richard Purdie
1f76afc03f builders: Allow bringup workload on any worker, ignore filters
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-21 20:35:52 +01:00
Anuj Mittal
c91b43c415 schedulers.py: dont schedule check-layer for honister
Honister is now EOL so remove the scheduled check.

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-01 09:16:17 +01:00
Anuj Mittal
798dd418fe config.py: fix quotes in workers_prev_releases list
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-27 22:31:37 +01:00
Anuj Mittal
969a227758 config.py: define workers for honister
Define the worker list for honister so we can test reliably now the
release support is ending.

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-27 22:31:37 +01:00
Richard Purdie
cd8beede08 schedulers: Run metrics at 7am daily
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-25 16:26:24 +01:00
Richard Purdie
525830f5ab config.py: Add bringup-fast and allow bringup to be tested on all workers
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-13 16:56:47 +01:00
Richard Purdie
64894cd8a1 config: Add metrics build target
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-13 15:59:42 +01:00
Richard Purdie
47ec6c73c1 config: Add other oe-core machines as potential targets
These are not default targets as they're not platforms we currently
cover in testing but this does allow us to see their status manually.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-13 09:44:55 +01:00
Steve Sakoman
d2b9e20151 schedulers: update branches for repos that now support kirkstone
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-03 07:31:14 +01:00
Ross Burton
1e9836486b config.py: add meta-virtualization to a-full
The meta-virtualization builder is reliable and useful now, so add it to
a-full.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-29 14:45:04 +01:00
Richard Purdie
b1e30acc32 schedulers: Add first pass at kirkstone support
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-31 21:15:33 +01:00
Richard Purdie
d63c3a4f66 reprodubile: Weight the same as other reproducible builds so it starts earlier
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-15 08:30:35 +00:00
Richard Purdie
2d60fe7103 config: Add buildperf-debian11
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-15 08:29:26 +00:00
Steve Sakoman
36dbd6b3c2 config.py: enable alma8, fedora35, and opensuse153 workers for dunfell
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-09 11:32:51 +00:00
Alexander Kanavin
6a8baf1221 config.py: run AUH only on alma8
According to halstead, only those workers are able to send mail
so that it reaches the mailing lists.

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-03 09:22:32 +00:00
Richard Purdie
c634385cb2 config.py: Update worker setup to match current autobuidler
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-02 22:45:25 +00:00
Richard Purdie
3df04cc44f config.py: Remove distro specific reproducible builds from a-full
The reproducible builds on each distro now show the same failures all the time
so their usefulness has reduced. They do add quite a load to the autobuilder
so reduce to a single random reproduicible build from a-full from now on
as we likely don't need it.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-26 00:08:14 +00:00
Richard Purdie
6fc8e7c3dc config.py: Update publish path to correct url in name
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-24 11:09:39 +00:00
Alexander Kanavin
7190192e2b schedulers.py: run AUH twice a month
Once a week has proven a bit too hectic: it's better to have
more time to test, submit, review and integrate the updates
before AUH runs again.

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-01 08:08:17 +00:00
Anuj Mittal
f924c793e0 config.py: enable debian11 for hardknott
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-01-20 13:03:07 +00:00
Richard Purdie
c4a41ad462 schedulers: Fix meta-ti branch references
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-01-17 16:59:18 +00:00
Richard Purdie
71aa271f3d schedulers: Add meta-virtualisation and mata-ti branch options
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-01-13 12:30:41 +00:00
Ross Burton
7bcd57f51d config.py: add qemuarm64-alt to a-quick and a-full
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-01-12 18:25:30 +00:00
Ross Burton
7fc1241f34 Run oe-selftest-armhost jobs on the Arm workers only
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-01-05 14:44:36 +00:00
Anuj Mittal
f50e334c65 config.py: enable fedora34 for hardknott
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-01-05 08:39:03 +00:00
Richard Purdie
137b41162a config.py: Fix OE urls
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-25 10:04:25 +00:00
Richard Purdie
fc4caf8573 config/schedulers: Use push.yoctoproject.org
If we don't do this, the mirrors may not have synced and it will break
expectations about being able to start builds immediately. Triggered
builds may also not see the correct code.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-24 23:27:00 +00:00
Ross Burton
7e1d76ccf9 schedulers: add deploy_artefacts to all builders
Instead of having a limited set of builders which can deploy artefacts,
let every builder have the ability to deploy. This makes it easier to
experiment with deploy steps.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-24 21:57:27 +00:00
Richard Purdie
de5263b7be builders: Tweak space limits
Practise is showing 60GB can still let the builders run out of space. Increase
to 100GB and also delay things more to allow better recovery, 2 minutes isn't
a long time when dealing with file deletion.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-12 12:11:57 +00:00
Richard Purdie
8829a85464 builders: Fix function call
Fix a function call typo in previous commit.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-31 12:53:36 +00:00
Richard Purdie
7353a843d5 builders: Fix quarantine handling
The way the canStartBuild code was written, it inserted a delay between
each build starting of 2 minutes unconditionally. We only want to do this
if the worker had run out of space so tweak the code accordingly.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-30 18:07:36 +01:00
Richard Purdie
17c7b35155 builders: Correct bonus application
The bonus is applied to the start time and needs to make the build
appear as if it started earlier than it did since earlier builds are
prioritised. This means the bonus should be subtracted, not added.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-30 17:54:07 +01:00
Trevor Gamblin
e21c1123a2 builders.py: fix bonus time calculation
Modify the builder_bonuses calculation so that it provides variable
bonus time based on the builder, rather than a constant value.

Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-29 16:19:33 +01:00
Trevor Gamblin
fda674dec6 builders.py: Add canStartBuild disk space check
We need a way to limit the builds for when a given worker has less than
a certain amount of disk space available. This implements a
canStartBuild method based on the example in the Buildbot docs and
blocks a build if the worker has less than 60GB of disk space available.
Unlike the example code, we want the stdout of the command so that we
can calculate the amount of disk space, rather than just relying on the
remote command's return code.

Docs: https://docs.buildbot.net/latest/manual/customization.html#canstartbuild-functions

[YOCTO #14591]

Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-29 16:19:18 +01:00
Steve Sakoman
77416bf3c0 config.py: debian 11 should have been enabled in dunfell, not warrior, fix it
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-28 23:47:14 +01:00
Steve Sakoman
88125e306c config.py: enable debian11 workers for dunfell
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-20 20:15:02 +01:00
Steve Sakoman
52c8be94b8 config.py: enable fedora34 workers for dunfell
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-20 20:15:02 +01:00
Richard Purdie
08020902be schedulers: Add honister release
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-11 11:12:11 +01:00
Richard Purdie
5f649e42cd builder: Rework builder prioritisation mechanism
The previous idea whilst looking good meant those builders were prioritised
over all others meaning multiple builds running in parallel which would
all be partly complete.

After discussion with upstream, add a bonus weighting mechanism to the
default sorting (which is based upon start time) so that our priority
builds should start first but builds started earlier should still be
prioritised.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-11 22:34:05 +01:00
Trevor Gamblin
b5a047f99f builders: Add prioritizeBuilders
This prioritizeBuilders function sorts builders by the length of their
associated worker lists, so that builders that can only be assigned to a
small number of workers are assigned to those workers before other
builds that don't have specific needs when resources are limited. An
example might be when a slot is available on an Ubuntu-based worker, and
"oe-selftest-ubuntu" and "genericx86-64" build requests exist in the
queue. Since oe-selftest-ubuntu requires an Ubuntu-based worker and
genericx86-64 does not, genericx86-64 will be assigned a higher value
(lower priority) so that oe-selftest-ubuntu is assigned to that worker
first.

[YOCTO #13644]

Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-09 14:24:24 +01:00
Richard Purdie
e0c1452d5a README: Fix mailing list email address
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-13 22:48:56 +01:00
Alexandre Belloni
6bc7226d43 schedulers: change default value for SWAT monitoring
meta-oe and auh are not for SWAT, change the default to be false

Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-13 22:48:12 +01:00
Richard Purdie
0fa2d40f05 schedulers: Add meta-aws/meta-openembedded branch mappings to all newer releases
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-13 22:47:34 +01:00
Steve Sakoman
c86fdefca4 schedulers.py: add dunfell branchdefaults for meta-aws and meta-openembedded
Both repos have been added to check-layer-nightly so we need to make
sure the dunfell branches of the repos are being used

Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-13 22:44:53 +01:00