Commit Graph

495 Commits

Author SHA1 Message Date
Mathieu Dubois-Briand
ea0077dce7 builders: Rework least loaded worker finding in nextWorker()
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
2025-03-07 14:13:01 +00:00
Mathieu Dubois-Briand
776469c3ae docker: Allow to add several local workers
Add more more workers to test more realistic setups.

The first one is still added as a Debian builder, additional ones are
used as generic ones, allowing to test worker selection.

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-03-06 14:30:07 +00:00
Mathieu Dubois-Briand
fab4b91312 docker: Use current git version
Use local git version instead of fetching remote master branch.

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-03-06 14:30:07 +00:00
Richard Purdie
754040f52c builders: Avoid giving jobs to workers that already have them
When giving jobs to workers, we want to prioritise idle ones and only
give them to the busiest ones when there are no others free.

This isn't entirely straight forward as there is no "idle" attribute
for workers but we can use a query and work it out.

Includes fixes from Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-03-06 12:50:27 +00:00
Richard Purdie
f05bf11157 config: Add metrics-gitstats builder
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-03-05 21:48:43 +00:00
Richard Purdie
20f12e3576 builders: Fix after reproducible changes
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-03-05 14:44:18 +00:00
Richard Purdie
f072fe7a28 config.py: Drop *-tc targets from a-quick
These were mostly skipped for a-quick anyway, simplify the code
to allow easier standalone triggering of the builds.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-03-05 14:37:51 +00:00
Richard Purdie
0e5934dda8 config: Drop distro specific reproducible targets
We don't use/need these any more.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-03-05 14:35:07 +00:00
Steve Sakoman
e8c69af5ec config.py: enable ubuntu2404 for kirkstone, not honister
A previous commit erroneously enabled ubuntu 2404 for honister instead
of kirkstone.  Fix this.

Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-03-05 14:06:44 +00:00
Richard Purdie
d8dd465877 config.py: Update the workers list
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-03-05 09:27:16 +00:00
Mathieu Dubois-Briand
a89c199e30 releaseselector: Fix page scroll on selector use
When using selectors, all fields modified by the selector will be
successively focused, inducing a potential page scroll. Save current
scroll before applying the release selector and set back this scroll
once the selector has been applied, setting back the page at the same
position.

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-02-24 10:31:49 +00:00
Mathieu Dubois-Briand
62d5d77167 releaseselector: Fix selector not applied in some conditions
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-02-21 11:34:44 +00:00
Mathieu Dubois-Briand
5da3f894a5 yocto_console_view: Override Force Build form CSS
Compact a bit the force build window, to look like what we had on
buildbot 3.8.

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-02-21 11:34:44 +00:00
Mathieu Dubois-Briand
93eaa4426a yocto_console_view: Fix release selector use with keyboard
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-02-21 11:34:44 +00:00
Steve Sakoman
e42afa8f88 config.py: add ubuntu2404 to workers_prev_releases for kirkstone and scarthgap
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-02-20 15:11:45 +00:00
Mathieu Dubois-Briand
1669ef72ce yocto_console_view: Fix output link on valkyrie
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-02-11 15:06:19 +00:00
Mathieu Dubois-Briand
2ab0fa45dd yocto_console_view: Keep known revision mappings
The yp_build_revision property disappears from the build instance once
the build is done. There is no reason for its value to change, so keep
the last seen value in revision mapping. This will prevent finished
builds to jump back to "Unresolved Revision".

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-02-11 15:06:19 +00:00
Mathieu Dubois-Briand
d9bcc41cf7 yocto_console_view: Group similar builder builds
Builds from similar builders have to be in the same <td>, compacting a
bit the whole view.

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-02-11 15:06:19 +00:00
Richard Purdie
dab2c9bb81 master.cfg: Tweak defaults to improve console build display
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-02-10 22:21:21 +00:00
Mathieu Dubois-Briand
58efe8c055 yocto_console_view: Add release selector
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
2025-02-10 18:22:57 +01:00
Mathieu Dubois-Briand
c8f1590596 yocto_console_view: create our own console
Modify existing console view plugin to recreate our own console view.

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
2025-02-10 18:22:57 +01:00
Mathieu Dubois-Briand
c2793d8bfd yocto_console_view: import console_view plugin from buildbot
Plugin changed a lot with the buildbot transition from angularjs to
reactjs: start our own plugin customisation from scratch.

This commit only copies sources from buildboot git, without any
modification.

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
2025-02-10 18:22:57 +01:00
Mathieu Dubois-Briand
421dd04863 builders: Fix configuration for buildbot 4
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
2025-02-10 18:22:57 +01:00
Mathieu Dubois-Briand
02661be391 builders: Add controller url in build tag
Make sure tags created on a-full/a-quick builds are unique, even across
different controller instances.

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-02-07 10:40:25 +00:00
Mathieu Dubois-Briand
cb639d12eb docker: Add docker file to create a local instance
Add a docker compose configuration, allowing to easily create a local
autobuilder instance.

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-02-06 11:56:14 +00:00
Mathieu Dubois-Briand
253d1f5bd7 schedulers: Fix missing parenthesis in field label
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-02-06 11:56:14 +00:00
Antonin Godard
8f6709cf0f schedulers: docs: pass properties to branch schedulers
Unlike the ForceScheduler, the AnyBranchScheduler does not take a list
of parameters but a simple dict. Add a repos_dict_for_scheduler() helper
that returns a dict containing the {repo,branch}_<repo> keys mapping to
the values in config.py. These are used in the factory definition of the
docs build (see the Git steps).

Also add the missing "yab-helper" to the bitbake-docs-changed scheduler.

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-01-10 15:43:12 +00:00
Antonin Godard
afa838d962 docs build: make it possible to override repo settings
In the current form when forcing a docs build, the different repo
parameters do nothing and are not propagated. The properties from
props_for_builder() are propagated, but weren't used in the factory
definition.

So use FixedParameters in the codebases definition: this removes the
useless entries in the form (we cannot simply remove the codebases arg
or it triggers another error "No sourcestamp found in build for
codebase"). Then in config.py, populate the "docs" entry in
buildertorepos, so that these show up in the form. Finally, in the docs
factory build, use Interpolate to get the property (I had issues using
util.Property directly).

Note: in latest buildbot version I hit a known bug
https://github.com/buildbot/buildbot/issues/8274. It shouldn't affect us
since we use an older version of Buildbot, but leaving this here just in
case.

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-01-10 15:43:12 +00:00
Richard Purdie
adb36c2213 config.py: Fix typo
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-12-20 15:16:37 +00:00
Richard Purdie
4ad0dc5539 config.py: Update with typhoon -> valkyrie changes
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-12-20 15:10:48 +00:00
Mathieu Dubois-Briand
2078359b30 builders: Create poky tags on a-full or a-quick builds
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-12-17 15:21:54 +00:00
Richard Purdie
7a517815e0 schedulers: Switch to a-full once a day
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-12-02 23:02:55 +00:00
Mathieu Dubois-Briand
1cc6527125 config.py: Update meta-qt6 from 6.6.0 to 6.7.2
meta-qt6 is only used for the webosose builder and webosose now needs
Qt6.7.

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-11-05 11:29:39 +00:00
Richard Purdie
29a5a12025 schedulers: Fix styhead for meta-ti
meta-ti will never have a styhead branch. Have the scheduler point to a valid branch.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-28 14:38:05 +00:00
Mathieu Dubois-Briand
15f37f7101 schedulers: Add codebase on all forced builds
Adding the code base with no name and a label on sub-builders, so it
adds the missing "yocto-autobuilder-helper" header on the build start
pop-up. This mimic the behaviour we already have for the a-full/a-quick
builds.

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-24 10:32:39 +01:00
Mathieu Dubois-Briand
f6a7a67425 builders/config: Use a shorter codebase name for docs
The introduction of codebases for docs builder changed the git clone
steps internal names, as they are now suffixed with the codebase name.
This new name for the yocto-autobuilder-helper seems to be too long for
the database used on our autobuilder and triggers an exception. Use a
shorter codebase name to fix this issue.

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-23 15:26:47 +01:00
Mathieu Dubois-Briand
b6a8b1eac1 schedulers: Add branch selector for Mathieu
As Alexandre is no longer using the autobuilder daily, remove his branch
selector and add mine.

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-23 11:35:33 +01:00
Mathieu Dubois-Briand
110cd04b3d README-Guide: Fix spaces in README-Guide
Remove any no-break-spaces in the file.

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-23 11:35:23 +01:00
Mathieu Dubois-Briand
06dfc4e43a builders/config: Fix docs builder with custom branches
While most of the builds on the autobuilder rely on the
shared-repo-unpack helper script and do not use the Buildbot codebases,
the "docs" builder use Buildbot traditional way to setup sources. As a
consequence, we have to use Buildbot codebases, otherwise the same
branch name is used for all cloned repos.

Add the codebase specifications for all three repos used by the docs
builder. Also remove them from the repos list, as these fields are not
used.

Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-23 11:34:55 +01:00
Richard Purdie
fc65d707b7 config.py: Add buildtools-docs builder with meta-oe as a depenedncy
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-14 15:42:03 +01:00
Richard Purdie
8c7217134a builders: Fix reversed branchname matching logic
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-09 23:50:54 +01:00
Richard Purdie
0094a385c8 builders: Add commented out debugging for bonuses
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-09 21:57:27 +01:00
Richard Purdie
48686d22b9 builders: Allow partial matching of helper branches
Helper branches with names like contrib/kirkstone-nut don't work with
worker restrictions currently. Try and fix this.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-07 22:27:30 +01:00
Richard Purdie
64cbb9b0ef schedulers: Add styhead
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-17 12:38:35 +01:00
Richard Purdie
3ba986a65a schedulers: Drop nanbield
The nanbield release is out of maintenance now, stop running metrics for it.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-23 10:26:01 +01:00
Richard Purdie
fdc1617271 builders: Reduce threshold for /tmp space to 1GB
buildperf-debian11 only has 1.8GB space on /tmp so this was blocking builds.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-08 11:06:13 +01:00
Richard Purdie
12f10fc494 builders: Handle different distro /tmp mount point output
Simplfy the space command to directly use findmnt and output in bytes to simplify the expression.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-05 23:26:51 +01:00
Richard Purdie
d809e29a23 builders: Improve disk space checking
We've been seeing increasing problems with disk space issues breaking builds.

Increase the check from 100GB to 200GB
Monitor /tmp for 10GB free as well
Refactor the code to make adding other mount point checks easy

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-04 12:58:31 +01:00
Richard Purdie
694d2a9bae config.py: Don't test toaster on fedora39/fedora40 as it doesn't work
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-01 16:28:40 +01:00
Richard Purdie
3b6febc312 config.py: Don't test toaster on fedora39/fedora40 as it doesn't work
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-01 16:27:58 +01:00