Commit Graph

609 Commits

Author SHA1 Message Date
Richard Purdie
01f4920d78 scripts/generate-charts: Add json data for last year as well as multiyear
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-29 00:04:23 +01:00
Richard Purdie
fe0d12175f config.json: Disable memory resident bitbake for metrics and buildperf
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-26 09:08:10 +01:00
Richard Purdie
8f078d8790 config.json: Enable memres bitbake
We could do with wider exposure of testing of this so enable on the
autobuilder by default.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-25 21:17:18 +01:00
Richard Purdie
70cdf365ef run-docs-build: Switch to newer docs tarball
This updates docs generate to a newer version of sphinx.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-23 13:31:41 +01:00
Michael Opdenacker
a1575b063b run-docs-build: allow build warnings again
This allows to switch to a more recent of Sphinx
which will generate warnings with old versions of docs.

This way, it's not immediately necessary to patch
all such versions.

This commit reverts
https://git.yoctoproject.org/yocto-autobuilder-helper/commit/?id=8273124feb9da2ffe93fcee7c4529d5597e1684a
which originally reverted
https://git.yoctoproject.org/yocto-autobuilder-helper/commit/?id=931d409b255a85f2217ca093d8391a678ce00ddb

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-23 13:30:26 +01:00
Richard Purdie
1baf966db6 run-docs-build: Ensure origin is specified for master
We may not have fetched the master revision if a different one was being
built so using origin/master works consistently in all cases.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-23 13:29:56 +01:00
Steve Sakoman
929c4cb688 config.json: use buildtools for opensuse154
Getting failures due to make version 4.2.1

Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-20 14:03:38 +01:00
Richard Purdie
813105cdce config.json: Simplify sstate structure
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-15 17:05:23 +01:00
Richard Purdie
3921d4f3ec scripts: Ensure directories exist
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-14 22:09:16 +01:00
Richard Purdie
d5eec6e3c0 config.json: Enable buildinfo for buildtools
[YOCTO #10133]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-11 11:18:35 +01:00
Richard Purdie
c1169d514e config.json: Add quick test for OEQA core tests
[YOCTO #13677]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-09 09:43:58 +01:00
Richard Purdie
9d7820da27 scripts/docs_add_banner: Fix several issues
* Fix a syntax error causing the script to fail
* Skip symlinks files (symlink directories were already skipped)
* Drop unneeded whitespace
* Abstract path join to a variable

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-03 10:51:38 +01:00
Abongwa Amahnui Bonalais
5dd90ea43b scripts/docs_add_banner: Add a banner on the old documentation docs.
Add a script which adds banners to the old docs and outdated dunfell docs
so that the user knows they're using old documentation.

Signed-off-by: Abongwa Bonalais Amahnui <abongwabonalais@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-03 10:51:08 +01:00
Quentin Schulz
9ebeb61276 scripts: run-docs-build: do not extract eclipse directories from old docs
For some reason, the old docs tarball includes many eclipse
subdirectories which are just cluttering the docs website up.

Therefore, let's just not extract eclipse directories from the tarball.

Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-01 09:14:58 +01:00
Richard Purdie
a5d46c4a2f config.json: Don't show CVE warnings in metrics target
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-30 00:03:06 +01:00
Richard Purdie
3f14f4bdb4 config.json: Build make and docs buildtools
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-26 12:55:06 +01:00
Richard Purdie
d2aacfd94d config.json: Extend poky-tiny testing
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-26 12:52:57 +01:00
Richard Purdie
a772bf318d scripts/cve-generate-chartdata: No need to exlude db
These are already handled in all branches we care about so drop the exclusion.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-25 17:55:12 +01:00
Richard Purdie
60902768cc scripts/run-patchmetrics: Drop honister, remove oldder json files
Keeping all the previous json files in tree has too much overhead
so drop the old ones. Honister doesn't support cve-check with json.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-25 17:52:39 +01:00
Richard Purdie
47d8883222 scripts/run-patchmetrics: Continue CVE checks for honister for now
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-25 09:46:15 +01:00
Richard Purdie
10ecb12155 scripts: Add script to update CVE count data in json file
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-25 08:32:19 +01:00
Richard Purdie
6655466ea6 scripts/run-patchmetrics: Seperate out cve-check json files into direcotries
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-24 15:21:23 +01:00
Richard Purdie
94f5a02706 config.json: Clear metrics BB_DISKMON_DIRS setting so dunfell works
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-24 12:38:05 +01:00
Richard Purdie
573aee4421 scripts/patchmetrics-generate-chartdata: Remove unused data
When generating the chart specific data, remove elements which aren't
used to reuse the filesize which improves browser loading speed.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-24 11:05:17 +01:00
Richard Purdie
7234d37633 config.json: Update cve-check config in metrics target
Ensure we do try and update the database on each daily run by lowering
the check timeout to 6 hours and ensure json report generation is enabled
in older releases like dunfell.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-24 11:03:42 +01:00
Richard Purdie
7c7b96cf19 scripts/patchmetrics-update: Fix typo
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-23 23:18:37 +01:00
Richard Purdie
803bdecd62 scripts/run-patchmetrics: Ensure tmpdir is clean between build runs
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-23 22:56:23 +01:00
Richard Purdie
125f1dbea4 scripts: Add patch to filter the chart data to make it usable
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-23 22:54:54 +01:00
Richard Purdie
c491e01653 scripts/run-patchmetrics: Ensure environment works for dunfell
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-23 22:11:19 +01:00
Richard Purdie
e3beeb9328 scripts: Extend CVE testing to kirkstone and dunfell branches
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-23 16:22:35 +01:00
Richard Purdie
7eca1ae4d7 scripts: Add ability to add/process missing patch stats data
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-23 16:05:23 +01:00
Richard Purdie
ca67ed751e run-patchmetrics: Add very basic cve-check functionality
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-17 17:14:13 +01:00
Richard Purdie
5428ba491b Add license file
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-17 12:37:51 +01:00
Richard Purdie
8702fdbb72 Clarify license and copyright information
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-17 10:29:45 +01:00
Richard Purdie
036fd8b311 gitignore: Ignore __pycache__ directories
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-17 09:47:47 +01:00
Richard Purdie
348467c2c8 run-patchmetrics: Use -m for realpath for results dir as it doesn't exist
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-13 17:21:45 +01:00
Richard Purdie
5b73a9b6eb run-patchmetrics: Use full variable names
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-13 17:19:17 +01:00
Richard Purdie
6bd5d99ade config.py: Split bring up into a template and add a fast version
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-13 16:54:28 +01:00
Richard Purdie
fd1aa61d87 scripts/run-patchmetrics: Ensure paths are sanitized
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-13 16:39:59 +01:00
Richard Purdie
1887e59889 config/scripts: Allow publishing of patch metrics
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-13 16:37:10 +01:00
Richard Purdie
2765a6a314 config/scripts: Add initial patch metric tracking code
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-13 16:21:57 +01:00
Richard Purdie
47ef936ea5 config/run-config: Allow other OE-Core machines to be tested manually
These are not part of our standard testing but allow them to
be triggered manually to assess their status.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-13 15:17:54 +01:00
Ross Burton
a71a53025d config.json: also build meta-virt for qemuarm
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-29 14:43:29 +01:00
Richard Purdie
91f49091cc scripts/run-docs-build: Use set-versions.py to get the latest release branch name
The intent here is to use the latest release branch name instead of a tagged version.
For the 4.0 case, this means the full migration guide will be shown instead of
the reduced information in the 4.0 tag.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-26 11:35:28 +01:00
Richard Purdie
12ef538e12 config.json: centos7 and ubuntu1604 perf workers are being decommissoned, remove
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-25 21:52:15 +01:00
Quentin Schulz
f418022d8f scripts: run-docs-build: reuse logic to link to latest tag
Since the latest tag already has a specific handling in the forloop,
let's just move the symlink creation inside the forloop.

Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-20 13:13:15 +01:00
Quentin Schulz
1870fb3a6c scripts: run-docs-build: make latest documentation tag the default version
Currently, the master branch is the default branch being displayed by
default when reaching the Yocto docs website. When big changes are
implemented for the next release, these are shown immediately to the
user, even though there is currently no release available for those
changes. This is an issue when e.g. behaviors are changed, new features
are added, variables get renamed or some syntaxes change because the
user might try to use things that aren't available to them yet.

I believe more people are using released version of Yocto
Project/Bitbake than people working on latest master. So let's make the
default version of the docs the latest tag (in terms of version number,
not date of tagging) to avoid too much confusion.

The master branch of the docs is now available at /dev subpath.

Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-20 13:13:15 +01:00
Quentin Schulz
4cd985b23d scripts: run-docs-build: use set_versions.py and switchers.js.in from master
set_versions.py and switchers.js.in need to be up-to-date so that a
consistent behavior is kept between different branches and tags of the
documentation.

Right now, kirkstone branch is lagging behind master and therefore does
not have the latest changes from master (e.g. the new obsolete
algorithm, which obviously isn't an issue right now, but will be in two
years). Using master version for those scripts also lightens the
maintenance burden.

Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-19 22:07:09 +01:00
Quentin Schulz
2852909e55 scripts: run-docs-build: add option to skip upstream rsync
The rsync only makes sense for the Autobuilder as most people don't have
access to docs@docs.yoctoproject.org. Therefore, to allow for easier
contribution to this script, let's allow to skip the rsync so the script
can continue to run.

Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-19 22:07:09 +01:00
Quentin Schulz
2f972bb742 scripts: run-docs-build: allow to pass buildtools script and docbook tarball paths
Currently, the path to buildtools script and docbook tarball are
hardcoded to work on Yocto Project Autobuilder. However, this makes it
harder to contribute to this script because it is very unlikely those
paths exist on a developer PC.

Instead, let's allow to override variables by using the environment and
make the current hardcoded values the default ones.

Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-19 22:07:09 +01:00