Backport for dunfell since we have been running for ~1 week with this
patch in master and the same load limiting may help reduce build
failures for dunfell itself as well as when its builds are running
alongside other branches.
This adds the "-l" option to PARALLEL_MAKE in config.json with an
initial testing value of 50 (the previous iteration of this patch both
had it set to 100 and mistakenly stated that this was a percentage,
rather than an absolute representation of system load ala top/htop).
This option is supported by both Make and Ninja. However, we also
require the "--debug=j" option to be passed to Make in order for the
latter to report perceived system load in the do_compile logs, (which we
need in order to track test values), and since this option is not
supported by Ninja, also add EXTRA_OEMAKE to the EXTRAVARS so that we
can determine if the target load percentage needs to be adjusted.
Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
This adds locale information which should fix reproducibility issues with vim.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 345475b99a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
The autobuilders have 128GB memory, we don't want them using 50% which is
the default, 5% should be enough. Also limit the number of threads down
from 48 to something reasonable. This may be partly causing some of our
performance issues?
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b38664d2db)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 97e0979d6e)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Line buffering (bufsize=1) is unavailable with binary mode so use unbuffered
mode instead. This fixes python runtime warnings.
[YOCTO #14093]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit c21732937c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 19b7456b92)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
qa-extras and qa-extras2 were split incorrectly, fix this.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 80fe0ab069)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Learn from the previous experiments and add meaninful shortnames and
descriptions to work around the 50 char name limit.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit bceb63fb79)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 6e2825564c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2bb4804243)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ce63e8f024)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b1bc4d64c2)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 4a4c888f66)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 23d65680f8)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e183db413f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit afb6c5a887)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 290e1bc2ee)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Replace the hard-coded individual email addresses with
the mailing list created for this purpose:
qa-build-notification@lists.yoctoproject.org
Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a script which from the exit code determines if a build target is present
in the configuration. The autobuilder can use this to determine whether to
execute the target at all or whether to skip it.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The existing checksum files were encoding full paths. This patch changes
this to use just the filename which allows relocation of the directories
and is more appropriate for sharing over http.
Signed-off-by: Vineela Tummalapalli <vineela.tummalapalli@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 835ac325a4)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 80c53d03ea)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 385fdf2295)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 6876a5536c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit d1d56776fa)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
On stable release branch builds, it will clone "master" since comparebranch
is None. It will then fail to push since there is already a dunfell
branch upstream with changes on which it needs to rebase on top of.
The change to use shallow clones significantly increases this problem.
To fix this, fall back to cloning basebranch by name, then master
if it doesn't exist.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were mistakenly doing a force push if the branch was in either
BUILD_HISTORY_FORKPUSH or BUILD_HISTORY_DIRECTPUSH.
Now we force push for branches in BUILD_HISTORY_FORKPUSH, regular push
for branches in BUILD_HISTORY_DIRECTPUSH, and no push if the branch is
in neither list.
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This code was copy/pasted but in this case the variables should
be called "compare", not base.
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
