Commit Graph

834 Commits

Author SHA1 Message Date
Joshua Watt
e3c3e15749 config.json: Use buildtools for Ubuntu 18.04
Bitbake is going to upgrade the minimum python to 3.8, so Ubuntu 18.04
needs to use buildtools tarball to remain compatible

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-07 11:23:01 +00:00
Ross Burton
b6938a7373 config: set MACHINE=qemuarm64 for oe-selftest-armhost
Many of the oe-selftest test cases boot up an image in qemu, but if the
host can't use KVM to accelerate the guest then the tests take a lot
longer and may randomly timeout.

Now that oe-selftest can run for qemuarm64[1], switch the
oe-selftest-armhost to use qemuarm64 so that KVM can be used. This
should mean faster and more reliable tests, which also cover another
architecture.

[1] oe-core 87fcee9fed915ff3af528b976d7d44737f9f1a95

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-16 11:56:12 +00:00
Richard Purdie
ee25d25679 config.json: Add langdale buildhistory support
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-28 13:30:37 +01:00
Richard Purdie
5fee00855e yocto-metrics: Add langdale CVE monitoring
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-28 13:30:19 +01:00
Quentin Schulz
0c8395c947 scripts: run-docs-build: make the workdir pristine between tries
It happened that the git repositories were dirty and resulted in
incorrect files being used. Let's use git clean -ffdx to force a
completely clean git repositories before and after checking out a branch
so that nothing is left from or to another branch build. Also git reset
--hard to undo changes to tracked files.

Cc: Quentin Schulz <foss+yocto@0leil.net>
Reported-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> #https://lore.kernel.org/yocto-docs/e50abe3c777e4a23a752a3ec25ad0b2a@axis.com/
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-27 17:37:18 +01:00
Michael Halstead
1e0cd71f65 scripts/run-docs-build: make nested dirs for contrib/name/* branches
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
2022-10-18 13:46:53 -07:00
Richard Purdie
effb9b0fcd scripts: Ensure gplv2 revision isn't listed and repos aren't used
We disabled meta-gplv2 for master, ensure the repo isn't used in the
collection or QA scripts.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-26 13:01:37 +01:00
Richard Purdie
cc0005d364 scripts/patchmetrics-update: Improve debug output
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-20 15:56:25 +01:00
Alexander Kanavin
e18974a86c scripts/run-patchmetrics: remove conf/templateconf.cfg as well between builds
With the recent template config changes to oe-core its content is no longer
backwards compatible.

'-f' is to guarantee future compatibility with build setups where the file
is not present.

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-02 11:43:59 +01:00
Richard Purdie
b7ee72be92 scripts/archive_buildstats: Exit early if buildstats isn't present
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-24 20:56:12 +01:00
Aryaman Gupta
c054c03e4f scripts/archive_buildstats.py: archive buildstats to tar.zst
Archive the buildstats of every failed build and 1% of random builds. Convert
the time-stamped buildstats directory to a compressed tarball using the
hostname as a prefix (to the file name only) so that one can identify the
source machine. Move these tarballs to the directory:
	testresults/<build_name>/buildstats/

The archiving is performed during the "collect results" step.

Signed-off-by: Aryaman Gupta <aryaman.gupta@windriver.com>
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-23 23:35:47 +01:00
Aryaman Gupta
d6dd128808 config.json: enable CPU and IO pressure regulation
Prevent severe system overload by setting the pressure limits for CPU
and IO at 10000.

Signed-off-by: Aryaman Gupta <aryaman.gupta@windriver.com>
Signed-off-by: Randy Macleod <Randy.Macleod@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-23 23:35:47 +01:00
Richard Purdie
3032b524bc config.json: Enable rm_work for meta-oe only to assist space issues
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-22 18:08:03 +01:00
Richard Purdie
699fb6008b config.json: Use IMAGE_CLASSES for testimage bbclass
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-11 00:11:58 +01:00
Richard Purdie
aee716639f config.json: Test rust toolchain
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-09 22:42:41 +01:00
Ross Burton
2b01de08c2 config.json: add a PREEMPT_RT-rt test build
Build and test core-image-full-cmdline with the linux-yocto-rt kernel,
adding the new rt test to verify that the kernel is actually the
PREEMPT_RT version expected.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-20 14:26:22 +01:00
Quentin Schulz
a9f5454b30 scripts/run-docs-build: do not checkout releases.rst from master anymore
Master branch of the yocto-docs now autogenerates the releases.rst file
and the file therefore does not exist anymore.

Signed-off-by: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-20 14:25:19 +01:00
Quentin Schulz
695bd33b47 scripts: run-docs-build: use latest version of set_versions.py script
`git checkout <pathspec>` without <tree-ish> only undo the changes made
to pathspec compared to HEAD. It is not guaranteed that HEAD is on
latest commit of master branch from upstream when running this script,
therefore let's add the <tree-ish> argument to specify which branch to
use to make sure set_versions.py is the latest, otherwise getlatest
might return the wrong information.

Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-20 14:25:19 +01:00
Michael Halstead
b3629e0e88 config.json: update to dedicated domain
A dedicated domain for hashserv allows us to move it to dedicated
infrastructre.

Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-15 21:36:21 +01:00
Ross Burton
1975a86c46 config.json: remove non-gpl3 job
meta-gplv2 is deprecated, so remove it from the autobuilder in master.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-12 14:38:47 +01:00
Ross Burton
dc28281c52 config.json: remove meta-gplv2 from the check-layer job
The meta-gplv2 layer is deprecated in master, so remove it from the
check-layer job.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-12 14:38:47 +01:00
Ross Burton
65281a5645 config.json: Set SDKMACHINE to aarch64 for oe-selftest-armhost
Although bitbake.conf sets the default SDKMACHINE to the build
architecture, config.json resets that to i686.

As oe-selftest assumes that the SDKs it builds are usable on the host
machine, we should set SDKMACHINE=aarch64 in the oe-selftest-armhost
build.

A follow-up more invasive patch to clean up the SDKMACHINE assignments
is in progress, once it has been verified to not cause regressions.

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-12 14:38:07 +01:00
Richard Purdie
a263c0b0d3 srcipts/generate-chartdata: Drop dateutil dependency
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-30 16:33:54 +01:00
Richard Purdie
400e1e1b2b scripts/utils: Allow tags to be updated from upstream (meta-agl has force pushed them)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-29 16:44:03 +01:00
Richard Purdie
54630b14ce scripts/run-patchmetrics: Ensure cve data for last year is copied
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-29 07:45:46 +01:00
Richard Purdie
01f4920d78 scripts/generate-charts: Add json data for last year as well as multiyear
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-29 00:04:23 +01:00
Richard Purdie
fe0d12175f config.json: Disable memory resident bitbake for metrics and buildperf
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-26 09:08:10 +01:00
Richard Purdie
8f078d8790 config.json: Enable memres bitbake
We could do with wider exposure of testing of this so enable on the
autobuilder by default.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-25 21:17:18 +01:00
Richard Purdie
70cdf365ef run-docs-build: Switch to newer docs tarball
This updates docs generate to a newer version of sphinx.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-23 13:31:41 +01:00
Michael Opdenacker
a1575b063b run-docs-build: allow build warnings again
This allows to switch to a more recent of Sphinx
which will generate warnings with old versions of docs.

This way, it's not immediately necessary to patch
all such versions.

This commit reverts
https://git.yoctoproject.org/yocto-autobuilder-helper/commit/?id=8273124feb9da2ffe93fcee7c4529d5597e1684a
which originally reverted
https://git.yoctoproject.org/yocto-autobuilder-helper/commit/?id=931d409b255a85f2217ca093d8391a678ce00ddb

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-23 13:30:26 +01:00
Richard Purdie
1baf966db6 run-docs-build: Ensure origin is specified for master
We may not have fetched the master revision if a different one was being
built so using origin/master works consistently in all cases.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-23 13:29:56 +01:00
Steve Sakoman
929c4cb688 config.json: use buildtools for opensuse154
Getting failures due to make version 4.2.1

Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-20 14:03:38 +01:00
Richard Purdie
813105cdce config.json: Simplify sstate structure
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-15 17:05:23 +01:00
Richard Purdie
3921d4f3ec scripts: Ensure directories exist
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-14 22:09:16 +01:00
Richard Purdie
d5eec6e3c0 config.json: Enable buildinfo for buildtools
[YOCTO #10133]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-11 11:18:35 +01:00
Richard Purdie
c1169d514e config.json: Add quick test for OEQA core tests
[YOCTO #13677]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-09 09:43:58 +01:00
Richard Purdie
9d7820da27 scripts/docs_add_banner: Fix several issues
* Fix a syntax error causing the script to fail
* Skip symlinks files (symlink directories were already skipped)
* Drop unneeded whitespace
* Abstract path join to a variable

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-03 10:51:38 +01:00
Abongwa Amahnui Bonalais
5dd90ea43b scripts/docs_add_banner: Add a banner on the old documentation docs.
Add a script which adds banners to the old docs and outdated dunfell docs
so that the user knows they're using old documentation.

Signed-off-by: Abongwa Bonalais Amahnui <abongwabonalais@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-03 10:51:08 +01:00
Quentin Schulz
9ebeb61276 scripts: run-docs-build: do not extract eclipse directories from old docs
For some reason, the old docs tarball includes many eclipse
subdirectories which are just cluttering the docs website up.

Therefore, let's just not extract eclipse directories from the tarball.

Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-01 09:14:58 +01:00
Richard Purdie
a5d46c4a2f config.json: Don't show CVE warnings in metrics target
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-30 00:03:06 +01:00
Richard Purdie
3f14f4bdb4 config.json: Build make and docs buildtools
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-26 12:55:06 +01:00
Richard Purdie
d2aacfd94d config.json: Extend poky-tiny testing
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-26 12:52:57 +01:00
Richard Purdie
a772bf318d scripts/cve-generate-chartdata: No need to exlude db
These are already handled in all branches we care about so drop the exclusion.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-25 17:55:12 +01:00
Richard Purdie
60902768cc scripts/run-patchmetrics: Drop honister, remove oldder json files
Keeping all the previous json files in tree has too much overhead
so drop the old ones. Honister doesn't support cve-check with json.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-25 17:52:39 +01:00
Richard Purdie
47d8883222 scripts/run-patchmetrics: Continue CVE checks for honister for now
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-25 09:46:15 +01:00
Richard Purdie
10ecb12155 scripts: Add script to update CVE count data in json file
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-25 08:32:19 +01:00
Richard Purdie
6655466ea6 scripts/run-patchmetrics: Seperate out cve-check json files into direcotries
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-24 15:21:23 +01:00
Richard Purdie
94f5a02706 config.json: Clear metrics BB_DISKMON_DIRS setting so dunfell works
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-24 12:38:05 +01:00
Richard Purdie
573aee4421 scripts/patchmetrics-generate-chartdata: Remove unused data
When generating the chart specific data, remove elements which aren't
used to reuse the filesize which improves browser loading speed.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-24 11:05:17 +01:00
Richard Purdie
7234d37633 config.json: Update cve-check config in metrics target
Ensure we do try and update the database on each daily run by lowering
the check timeout to 6 hours and ensure json report generation is enabled
in older releases like dunfell.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-24 11:03:42 +01:00