Fixes [YOCTO #15417]
When a layer adds a new dependency after it was added to a conf, it can not be
removed w/o this dependency in the setup. Even the dependent layer can not be
added, as the tinfoil setup will fail.
Adapt --force to not perform the tinfoil at all, the use will be at own risk,
i.e. the added layers might not parse properly afterwards.
This is not merged into the force option with -F as it even changes the loading of
plugins from other layers and is hence even more invasive as force. Instead
force can now be speciefied multiple times and is counted.
(Bitbake rev: 541fa7f582133949563e65f2d43c4b16e873e5c1)
Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the flush in serverlog() removed and a memory resident bitbake with a
60s timeout, the following could fail in strange ways:
rm bitbake-cookerdaemon.log
bitbake-layers add-layer ../meta-virtualization/
bitbake-layers add-layer ../meta-openembedded/meta-oe/
bitbake -m
specifically that it might error adding meta-oe with an error related to meta-virt.
This clearly shows that whilst bblayers.conf was modified, bitbake was not
recognising that. This would fit with the random autobuilder issues seen when
the serverlog flush() call was removed.
The issue appears to be that you have no way to "sync()" the inotify events with
the command stream coming over the socket. There is no way to know if there are
changes in the IO queue which bitbake needs to wait for before proceeding with
the next command.
I did experiment with os.sync() and fsync on the inotify fd, however nothing
addressed the issue. Since it is extremely important we have accurate cache data,
the only realistic thing to do is to switch to stat() calls and check mtime.
For bitbake commands, this is straightforward since we can revalidate the cache
upon new connections/commands. For tinfoil this is problematic and we need to
introduce and explict command "revalidateCaches" that the code can use to force
bitbake to re-check it's cache validity. I've exposed this through tinfoil with
a new "modified_files" function.
So, this patch:
a) drops inotify support within bitbake's cooker/server and switch to using mtime
b) requires a new function call in tinfoil when metadata has been modified
(Bitbake rev: da3ec3801bdb80180b3f1ac24edb27a698415ff7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By including the full path to bblayers.conf the remove-layer
command can be executed from any location, not only from the
build directory.
(Bitbake rev: 25cb4e17e8a4c0310df018b9df184b0155b267eb)
Signed-off-by: Pedro Baptista <pedro.miguel.baptista@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By including the full path to bblayers.conf the add-layer
command can be executed from any location, not only from the
build directory.
(Bitbake rev: 77aa230bf2a053fc0941723a6abbc798ebe53a19)
Signed-off-by: Pedro Baptista <pedro.miguel.baptista@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Where copyright headers were not present, add them to make things
clear.
(Bitbake rev: 1aa338a216350a2751fff52f866039343e9ac013)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In line with the inclusive language migration defined at:
https://wiki.yoctoproject.org/wiki/Inclusive_language
replace the remaining usage of "abort" in documentation, error
messages, and comments with halt/fail/exit as appropriate.
A couple of external Javascript API calls in Toaster remain, as
they cannot currently be changed.
(Bitbake rev: bc27762bf3ffb4a20b58eace5302438c4a526626)
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When adding a layer, parse error can occur, raising BBHandledException.
Catch this and error, aborting the layer add to meet user expectations.
[YOCTO #14054]
(Bitbake rev: ceddb5b3d229b83c172656053cd29aeb521fcce0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Splits the cooker to track a collection per multiconfig instead of a
single collection for all multiconfigs. Practically speaking, this
allows each multiconfigs to each have different BBMASKs that apply to it
instead of each one using the mask specified in the base configuration.
(Bitbake rev: dd6d8eca2027f8d9be8a734a493227b440075e49)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds the SPDX-License-Identifier license headers to the majority of
our source files to make it clearer exactly which license files are under.
The bulk of the files are under GPL v2.0 with one found to be under V2.0
or later, some under MIT and some have dual license. There are some files
which are potentially harder to classify where we've imported upstream code
and those can be handled specifically in later commits.
The COPYING file is replaced with LICENSE.X files which contain the full
license texts.
(Bitbake rev: ff237c33337f4da2ca06c3a2c49699bc26608a6b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When adding multiple layers in a chain of commands, reparsing all recipes
each time can be painfully slow. Instead just parse the base configuration
which gives some confidence things worked out correctly without as much
overhead.
(Bitbake rev: cfd1302031f3cca96300a0e445a47b1614ecd00c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you can add multiple layers at once, it stands to reason that you
should also be able to remove more than one at a time.
(Bitbake rev: 2f2033836a5ce4064d9e4f263788a563001bc008)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow specifying multiple layers with bitbake-layers add-layer so that
you can add more than one in a single command. This is not just useful,
it's actually pretty important if you need to add a layer and its
dependencies at the same time - since we now go through a parse process
when the layer is added, without this you have to add them all in just
the right order and wait for the parse each time which is somewhat
painful.
(Bitbake rev: ad6b14f01aa326a1c6baa31bfac33be238bce805)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the original implementation, "bitbake-layers add-layers <layer>"
succeeded without error checking. This will further introduce
failures in recipe parsing only when "bitbake" command is executed.
Adding a meta layer without its dependency layer(s) should failed
and exit the process gracefully.
Added extra argument "-F" to force add a layer without checking
layer dependency.
[YOCTO #10913]
(Bitbake rev: 705ab252e631903e6d2e46202b419a9e8adcd861)
Signed-off-by: Phoong Stanley Cheong Kwan <stanley.cheong.kwan.phoong@intel.com>
Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.
Search made with the following regex: getVar ?\(( ?[^,()]*), True\)
(Bitbake rev: 3b45c479de8640f92dd1d9f147b02e1eecfaadc8)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds the notion of supporting multiple configurations within
a single build. To enable it, set a line in local.conf like:
BBMULTICONFIG = "configA configB configC"
This would tell bitbake that before it parses the base configuration,
it should load conf/configA.conf and so on for each different
configuration. These would contain lines like:
MACHINE = "A"
or other variables which can be set which can be built in the same
build directory (or change TMPDIR not to conflict).
One downside I've already discovered is that if we want to inherit this
file right at the start of parsing, the only place you can put the
configurations is in "cwd", since BBPATH isn't constructed until the
layers are parsed and therefore using it as a preconf file isn't
possible unless its located there.
Execution of these targets takes the form "bitbake
multiconfig:configA:core-image-minimal core-image-sato" so similar to
our virtclass approach for native/nativesdk/multilib using BBCLASSEXTEND.
Implementation wise, the implication is that instead of tasks being
uniquely referenced with "recipename/fn:task" it now needs to be
"configuration:recipename:task".
We already started using "virtual" filenames for recipes when we
implemented BBCLASSEXTEND and this patch adds a new prefix to
these, "multiconfig:<configname>:" and hence avoid changes to a large
part of the codebase thanks to this. databuilder has an internal array
of data stores and uses the right one depending on the supplied virtual
filename.
That trick allows us to use the existing parsing code including the
multithreading mostly unchanged as well as most of the cache code.
For recipecache, we end up with a dict of these accessed by
multiconfig (mc). taskdata and runqueue can only cope with one recipecache
so for taskdata, we pass in each recipecache and have it compute the result
and end up with an array of taskdatas. We can only have one runqueue so there
extensive changes there.
This initial implementation has some drawbacks:
a) There are no inter-multi-configuration dependencies as yet
b) There are no sstate optimisations. This means if the build uses the
same object twice in say two different TMPDIRs, it will either load from
an existing sstate cache at the start or build it twice. We can then in
due course look at ways in which it would only build it once and then
reuse it. This will likely need significant changes to the way sstate
currently works to make that possible.
(Bitbake rev: 5287991691578825c847bac2368e9b51c0ede3f0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Various misc changes to convert bitbake to python3 which don't warrant
separation into separate commits.
(Bitbake rev: d0f904d407f57998419bd9c305ce53e5eaa36b24)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This uses bb.utils.load_plugins, based on the plugin handling in recipetool
and devtool in oe-core.
(Bitbake rev: 5e542df9b966a99b5a5b8aa7cf6100174aff54b2)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
