These are not part of our standard testing but allow them to
be triggered manually to assess their status.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The intent here is to use the latest release branch name instead of a tagged version.
For the 4.0 case, this means the full migration guide will be shown instead of
the reduced information in the 4.0 tag.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since the latest tag already has a specific handling in the forloop,
let's just move the symlink creation inside the forloop.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, the master branch is the default branch being displayed by
default when reaching the Yocto docs website. When big changes are
implemented for the next release, these are shown immediately to the
user, even though there is currently no release available for those
changes. This is an issue when e.g. behaviors are changed, new features
are added, variables get renamed or some syntaxes change because the
user might try to use things that aren't available to them yet.
I believe more people are using released version of Yocto
Project/Bitbake than people working on latest master. So let's make the
default version of the docs the latest tag (in terms of version number,
not date of tagging) to avoid too much confusion.
The master branch of the docs is now available at /dev subpath.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
set_versions.py and switchers.js.in need to be up-to-date so that a
consistent behavior is kept between different branches and tags of the
documentation.
Right now, kirkstone branch is lagging behind master and therefore does
not have the latest changes from master (e.g. the new obsolete
algorithm, which obviously isn't an issue right now, but will be in two
years). Using master version for those scripts also lightens the
maintenance burden.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The rsync only makes sense for the Autobuilder as most people don't have
access to docs@docs.yoctoproject.org. Therefore, to allow for easier
contribution to this script, let's allow to skip the rsync so the script
can continue to run.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, the path to buildtools script and docbook tarball are
hardcoded to work on Yocto Project Autobuilder. However, this makes it
harder to contribute to this script because it is very unlikely those
paths exist on a developer PC.
Instead, let's allow to override variables by using the environment and
make the current hardcoded values the default ones.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The script is written in such a way that absolute paths are expected.
Instead of failing weirdly at some point in the script, let's just make
all paths passed to this script absolute by calling realpath on them.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This uses the set_versions.py script from the docs master branch to handle
creation of the swictchers.js version information as well as the data in
poky.yaml.
This allows patches to those sections of the docs to be dropped.
It is assumed the patch to use set_versions is applied to the docs transitions
branch so that it's switchers.js files no longer need to be tweaked.
This does lead to user visible changes on the website:
- Older versioned released docs gain their current version and the
latest version listed in the switcher rather than being unlisted
- The list of releases is normally filtered down to our active ones
(hardknott, honister and dev right now)
- 3.3's doc references to gatesgath are corrected to hardknott
- Docs for unversioned release branches (not linked on the website) now
use 3.1.999 versioning instead of confusing them with the last relased version
- 3.1.13 refers to 3.1.13 instead of 3.1.12
all of which seem to be reasonable improvements.
The big advantage of these changes is that with a single change to the script
in the master branch, the right thing should happen for all the current docs
and at release time, only a docs rebuild should need to be triggered after tags
are pushed.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Except patching which is specific to tags and yocto- tag prefix
stripping, the logic is identical, so let's merge both loops together.
Cc: Quentin Schuls <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The commit that introduced Sphinx support in yocto-docs is
01dd5af7954e24552aca022917669b27bb0541ed. Any tag containing this commit
is buildable by sphinx.
Dunfell tags don't all have Sphinx support. However, all tags containing
the introducing commit c25fe058b88b893b0d146f3ed27320b47cdec236 are
buildable by sphinx.
Therefore, let's just list all tags which contains either of those two
commits instead of the complex series of pipes and shell commands.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since commit 01dd5af7954e24552aca022917669b27bb0541ed, all later
releases of yocto-docs can be built with Sphinx. Instead of manually
updating this list, let's have git return the list of remote branches
which contains the commit.
dunfell branch was initially released without Sphinx support but was
later patched, hence why it's explicitly listed.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
master, master-next and transition only differ from other branches by
their output directory name. Let's put everything in common and only
have a check on whether the branch is master, master-next or transition
and modify the output dir in those cases.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since commit 84ccba0f4aff91528f764523fe1205a354c889ed, docs of all later
releases can be built with Sphinx. Instead of manually updating this
list, let's have git return the list of remote branches which contains
this commit.
1.46 branch was initially released without Sphinx support but was later
patched, hence why it's explicitly listed.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
master and master-next only differ from other branches by their output
directory name. Let's put everything in common and only have a check on
whether the branch is master or master-next and modify the output dir in
those cases.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The patch was meant as a quick temporary work-around to have the docs
built and published.
Now that releases where -W flag is set (turning warnings into errors)
are appropriately patched to make those warnings disappear (on Sphinx
v3.2.1 which is the one used on the builder), this patch can be reverted
so that next time a warning appears the doc building will fail but will
not destroy the doc website (because of commit
6a4e6ef18d "scripts: run-docs-build: make
the script fail hard ASAP when there's an error", since rsync will not
be run if any error happens before).
This reverts commit 931d409b25.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The patch was missing from yocto-3.4.1 release and only made it to
yocto-3.4.2 unfortunately so let's backport it.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Each Yocto Project release is validated for a specific Bitbake release.
By mistake, most release docs are pointing at the development branch of
Bitbake documentation instead of the one that was validated for the
Yocto Project release.
This fixes the issue by patching the conf.py of impacted releases.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There's no need to keep a list of tags requiring to be patched since the
tag is part of the path where patches are stored.
Therefore, let's only check if there's a patch directory for a given tag
and if so, apply all patches in there.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This approach didn't work reliably so switch to text based emails
pointing at the website version with graphs and data.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current setup only allows sending mail via localhost on alma workers.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add patch for version 3.3.5 and move patches into a dedicated
subdirectrory to keep the scripts directory tidy.
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
When current_version and all_versions are not updated before the release
build the tag will not build working docs. Make docs builds pull in
needed updates after the fact.
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Default flags for sphinx-build contain -W which turns warnings into
errors. This is really helpful during debugging and could be used for
continuous integration though we do not have such a thing separate from
the continuous delivery in place currently. Nowadays, the docs files
served at docs.yoctoproject.com are actually removed before being
updated from the newly built docs. If the `html` target, a dependency of
`publish` target is failing, the docs aren't copied over to the `final`
temporary directory. Therefore it'll be missing in the rsync upload to
docs.yoctoproject.org.
Instead, let's disable the turning of warnings into errors so that the
`html` make target can finish successfully and the `publish` make target
to finish successfully too.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are some intermittent issues with the script not publishing all
versions. So let's go extreme and fail the script if any error happens:
- a command returns a non-zero code, even if piped,
- a variable is used uninitialized,
This also makes the script print each and every command being run so we
have a better idea where the script struggles.
Cc: Quentin Schulz <foss+yocto@0leil.net>
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A wrong path was given given the working directory.
Also revert the changes with "git reset --hard" to
have a clean state before further branch switches.
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is done regularly by the overall AB maintenance tooling, and erasing
build artifacts makes it difficult to find out what went wrong
when it does.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Without this we'd see failures in the perf builds since it would be able to track
history correctly for commit counts.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows all versions of Bitbake and Yocto Project manuals
to see the manuals for the latest versions.
This also simplifies the release process, not having to update the
releases.rst file for all releases every time a new release is made.
Note that such synchronization is already done for the
switchers.js file (but in a different way). This way, advertised
releases are in sync with switchers.js.
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Publish the generic-arm64 binaries in a dedicated meta-arm/ directory so
it is clear this isn't from the core layers.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The depth code doesn't work if the branch doesn't exist in the main repo
mirror and only the remote.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The rysnc of 20,000 files (650MB) onto the nas is slow taking ~3 minutes
at idle and worse at load. This is due to the number of files which
is a pain point for NFS. This piece of the build is also a bottleneck
since the rest of a build depends on it happening.
If we switch to zstd compressed tar, it takes 2.49s. Other compression
methods were much slower but zstd seems 'accptable' and speeds things
up too.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By not syncing all the history is is possible to save some time/space
in the checkout process since we never use this data. This reduces data
from 650MB to 400MB or with the tarball, 416MB to 55MB.
The logic for the commands needs to be tweaked to handle this and as
written it can't work in non-HEAD revision case but that isn't a commonly
used situation.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Both releases are missing an important patch that changes the displayed
version from dev to the appropriate release number.
This is confusing to the user and probably breaks some assumptions in
some scripts.
Ideally, the tags should have been moved with those patches applied to
their respective branch but that is not a git best practice so we're
stuck with this "hack" instead.
3.3.x releases aren't impacted as they got the patch applied.
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
honister and 1.52 Bitbake branch were recently released, so let's build
those too.
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The auh.yoctoproject.org domain is no longer used and cannot be
validated.
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In addition to the testresult json, testimage class now also
provides the testimage task log and qemu console output log
which can be useful for debugging test failures or
even checking qemu test runs when failures did not happen.
Rather than duplicate specific file/folder names, let's copy all
that is available, and define what is published in the testimage
class itself (with appropriate folder structure if/when needed).
At the moment there's just three files, and they are copied into
folders named after image names, so there's no clutter or risk
of mixing them up with unrelated logs.
[YOCTO #14518]
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is not guaranteed to build or work otherwise, and causes
update failures in unrelated recipes when building for musl for instance.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Host data is collected on failure by default. List the collected files
in the index.
Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
data collection on failure is enabled by default.
Collect all host stats files created.
Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
summarize_top_output.py is used to summarize the top
output that is captured during autobuilder intermittent
failures.
Use the script to summarize the host top output and
publish the summary that is created instead of
the raw logfile.
Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make sure that https://docs.yoctoproject.org/current will always link
to the most recent release documentation.
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The default behavior is to look for a bitbake command,
which fails and produces a confusing output of [-1:].
[YOCTO #14208]
Signed-off-by: Yi Fan Yu <yifan.yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a column "Host Data" to publish the host data collected.
Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com>
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
collect-data template generates host_stats files which are collected.
All files are published, file names are determined by the contents:
- Files that contain "top" output are named host_stats_<number>_top.txt
- host_stats_<number>.txt otherwise
Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com>
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Build Sphinx docs for all versions newer than yocto-3.1.5 inclusive.
Integrate suggestions from Quentin Schulz and Nicolas Dechesne.
Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The branches exist now, let's publish the corresponding docs.
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use for loop to avoid repeating the same pattern over and over.
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Line buffering (bufsize=1) is unavailable with binary mode so use unbuffered
mode instead. This fixes python runtime warnings.
[YOCTO #14093]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
run-docs-build rebuilds the entire doc website, so when we rsync the
output, we need to delete files no longer needed, this is especially
important when we move/rename pages, to avoid stale content.
Using rsync with "wildcards" was a problem in case a file/folder is
removed in the output dir, it won't be deleted.
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Learn from the previous experiments and add meaninful shortnames and
descriptions to work around the 50 char name limit.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a script which from the exit code determines if a build target is present
in the configuration. The autobuilder can use this to determine whether to
execute the target at all or whether to skip it.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds systemd and pam related recipes to upstream checks and devtool-driven updates.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The script is suboptimal in many ways but is a start and gives us something to test
and improve upon.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Michael Halstead has confirmed this.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since https://lists.yoctoproject.org/g/yocto/message/50282 was merged in
autoupgrade-helper git repo, the python script isn't called
upgradehelper.py anymore but upgrade-helper.py for consistency sake.
Let's update the run-auh script so that it's still working.
Signed-off-by: Quentin Schulz <foss@0leil.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sadly, many maintainers do not act on AUH mails in a timely manner
(or do not act at all). Sending these emails to oe-core list as well
will allow other people to pick them up, and ease up the path
to doing recipe version maintenance.
Also, recently the amount of outdate recipes has been greatly reduced,
so there should not be too many of those emails each month.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
On stable release branch builds, it will clone "master" since comparebranch
is None. It will then fail to push since there is already a dunfell
branch upstream with changes on which it needs to rebase on top of.
The change to use shallow clones significantly increases this problem.
To fix this, fall back to cloning basebranch by name, then master
if it doesn't exist.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The existing checksum files were encoding full paths. This patch changes
this to use just the filename which allows relocation of the directories
and is more appropriate for sharing over http.
Signed-off-by: Vineela Tummalapalli <vineela.tummalapalli@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows automating its setup and execution on all autobuilder worker machines;
previously there was a static setup on a dedicated machine, which wasn't
great from maintenance perspective.
To use:
scripts/setup-auh target_dir
scripts/run-auh target_dir
(run-auh can be run several times in a directory that
was previously set up)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were mistakenly doing a force push if the branch was in either
BUILD_HISTORY_FORKPUSH or BUILD_HISTORY_DIRECTPUSH.
Now we force push for branches in BUILD_HISTORY_FORKPUSH, regular push
for branches in BUILD_HISTORY_DIRECTPUSH, and no push if the branch is
in neither list.
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This code was copy/pasted but in this case the variables should
be called "compare", not base.
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The buildtools path contains "../" which may not resolve correctly by the
shell if the final directory doesn't exist. Avoid this by using abspath()
on the expression straightaway.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow the workername to be passed into the script. This is to allow future
buildtools tarball support based upon the worker.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This fixes failures on the buildperf workers which do early cleanup of build
directories.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are several issues:
* New branches don't currently have git regression history
* The regression tool errors if there isn't anything to compare against
To fix this, create a branch with history and only generate a regression
report if there are commits to compare against.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous fixes requires the user to set "no-layer-add"
for a repo and then use ADDLAYER to insert the sub-repos
(e.g., meta-openmbedded/meta-oe) as a two-part process.
This means that you would also have to specify that flag
if a repo that is a layer with dependencies is in the
list so that it can be inserted in the correct order later
via ADDLAYER to avoid parsing problems. This fix allows
for specifying a NEEDREPOS with the subdirectory of the
target layer (e.g., meta-openembedded/meta-oe) so that
there is no need for the "no-layer-add" followed by
ADDLAYER combination. The entire meta-openembedded
repo would be moved into place, and the sublayer added
to bblayers.conf.
Signed-off-by: Thomas Goodwin <btgoodwin@geontech.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The preceeding errors that would be shown should be enough to
identify the problem since it would have been a problem
running bitbake-layers add-layer, indicating a misconfiguration
of config.json.
Signed-off-by: Thomas Goodwin <btgoodwin@geontech.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The return value from bitbakecmd was not being returned when
errors occurred which allowed shared-repo-unpack to succeed
despite the failure. This fix changes to check_call and a
try-catch when attempting to add repos that fail for whatever
reason during add-layer, like a missing conf/layer.conf at
the top level or a previously-added layer breaks parsing
because of missing dependencies.
Signed-off-by: Thomas Goodwin <btgoodwin@geontech.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Affects sumo since that doesn't have oe-selftest parallelism. thud/master create
these in a different location so don't see the issue.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since we now have functional buildhistory repository functionality,
generate buildhistory-diff output for builds where its enabled.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently there was no way to override an existing extra vars
assignment, which meant BB_NUMBER_THREADS was impossible to change.
This adds code to allow the higher override of a value to override
the other settings which makes the code function more as a user
would expect.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently it was being placed a directory level too high and hence
was being ignored.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This ensures that release branch performance build tests work on the correct
branch and that comparisions between mut+master and master-next+master
start to work correctly. It also ensures reports for the current
revision are generated even if later revisions are built.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The shell script can't access the config.json data easily, convert
from shell to python so that we prepare the way for further enhancements.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes the build performance code behave similarly to the other
results publishing code using the standard location.
Also tweak the report names so they sort.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need the final logging to show which command steps contain errors so print
failing commands but allow the code to continue and show the final log output.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This ensures the builds get as far as they can when there is failure rather
that stopping at the first sign of one. This means caches are hotter and
allows for faster respins, at the cost of the failure being lost in longer
logs.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This should further protect us against contamination between different build directories
as old build directory paths should no longer exist.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no current mechanism to reference the 'builddir' within the config.json,
so add HELPERBUILDDIR for this purpose.
Use this to set SSTATEDIR for nightly-bringup, meaning sstate arefacts should be
shared for use between the different oe-selftest threads, speeding up builds.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The SSTATEDIR was accidentally using getconfiglist instead of getconfigvar which
meant it couldn't be unset. Fix to use var so the values in config.json work
as intended.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Many of the distros we use have sendmail in /sbin or /usr/sbin
which may not be in the path for non-interactive users. Handle
this in the script to avoid failures where tests work but the
real QA email fails.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add in the support to publish images into the designated path.
BSP packages of the previous build will be cleaned up before
new BSP packages are copied over. This ensures the previous
image will not be retained and causes conflicts before image
is loaded into x86_64 (MTURBOT64) hardware.
Signed-off-by: Aaron Chan <aaron.chun.yew.chan@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[RP: Minor tweaks made to adpat to buildbot and option naming
Fixed shared-repo-unpack for publishing when no cache-dir supplied]
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The new buildbot code can directly monitor the logfiles itself so
we need to be able to optionally stop echoing onto the console.
Add missing flush() calls and print the command being used and logfile
so its clear what is happening on stdout too.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch fixes the "Collection Error during parsing layer conf"
when a parent layer is accidentally added before depend layers.
When removing layers from bblayers, we should not be following
the same sequence of adding the layers, it should be done in a
reversed order. This is an assumption that the layers have their
dependent layers added before the parent layers are added.
In general, the parent layer require to be remove first before
its child dependencies layers.
Signed-off-by: Aaron Chan <aaron.chun.yew.chan@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Testing this code is becoming increasingly problematic. This allows
ABHELPERTEST to be set from the environment and means the script will
still create directories and configuration files but it will not execute
anything, just print what would be executed.
This allows faster validation of changes to the configurations/code.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Remove the code disabling it
* Move to a non-shared directory by default (but still support absolute pathnames
if wanted)
* Use shallow clones by default for speed
* Allow force pushing 'fork' mode branches
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Usage is documented in README and an example, local-example.json is
included.
Also clean up the parameter to loadconfig() as its actually not needed.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We want to be able to use data expansion/variable substitution
in most variables so convert to use the utils API for this.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code clearly won't scale without some kind of variable substitution/
variable expansion mechanism, so add one.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
