poky/bitbake
Richard Purdie cdb1cb2ed8 bitbake: runqueue: Improve setcene performance when encoutering many 'hard' dependencies
"bitbake world -n --setscene-only" shows poor performance as the numbers of tasks
increases. Analysys shows this is due to the "deferred" hard dependencies being
repeatedly processed which doesn't allow much forward progress in overall task
execution.

To avoid this, mark when it has been done and don't reprocess until dependencies
are updated. We have to be careful as we've seen bugs where these dependencies
aren't processed at the right time. They also have to be reproceseed during task
migration/rehashing so the code has to "self heal" the data structures.

(Bitbake rev: e5609bac06c17dabcf6286b47b1a3f19f5a1160f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-13 13:52:10 +00:00
..
bin bitbake: bitbake-diffsigs: fix walking the task dependencies and show better error 2024-02-08 10:59:32 +00:00
contrib bitbake: contrib/vim: Syntax improvements 2023-12-23 16:20:25 +00:00
doc bitbake: doc: bitbake-user-manual: remove reference to SSTATE_MIRRORS variable 2023-09-05 08:13:13 +01:00
lib bitbake: runqueue: Improve setcene performance when encoutering many 'hard' dependencies 2024-02-13 13:52:10 +00:00
.gitattributes bitbake: .gitattributes: Add to improve git diff for minified css/js files 2019-03-07 12:18:48 +00:00
AUTHORS misc: Update the email address to a working one. 2011-01-04 14:36:54 +00:00
ChangeLog *: Fix typo in documentation 2010-08-04 16:12:39 +01:00
LICENSE bitbake: bitbake: Add initial pass of SPDX license headers to source code 2019-05-04 10:44:04 +01:00
LICENSE.GPL-2.0-only bitbake: bitbake: Add initial pass of SPDX license headers to source code 2019-05-04 10:44:04 +01:00
LICENSE.MIT bitbake: bitbake: Add initial pass of SPDX license headers to source code 2019-05-04 10:44:04 +01:00
README bitbake: doc: README: simpler link to contributor guide 2024-02-10 14:13:51 +00:00
SECURITY.md bitbake: SECURITY.md: add file 2023-10-24 12:49:56 +01:00
toaster-requirements.txt bitbake: Update toaster-requirements to add django-log-viewer==1.1.7 2023-10-15 09:12:05 +01:00

Bitbake

BitBake is a generic task execution engine that allows shell and Python tasks to be run efficiently and in parallel while working within complex inter-task dependency constraints. One of BitBake's main users, OpenEmbedded, takes this core and builds embedded Linux software stacks using a task-oriented approach.

For information about Bitbake, see the OpenEmbedded website: https://www.openembedded.org/

Bitbake plain documentation can be found under the doc directory or its integrated html version at the Yocto Project website: https://docs.yoctoproject.org

Bitbake requires Python version 3.8 or newer.

Contributing

Please refer to our contributor guide here: https://docs.yoctoproject.org/contributor-guide/ for full details on how to submit changes.

As a quick guide, patches should be sent to bitbake-devel@lists.openembedded.org The git command to do that would be:

git send-email -M -1 --to bitbake-devel@lists.openembedded.org

If you're sending a patch related to the BitBake manual, make sure you copy the Yocto Project documentation mailing list:

git send-email -M -1 --to bitbake-devel@lists.openembedded.org --cc docs@lists.yoctoproject.org

Mailing list:

https://lists.openembedded.org/g/bitbake-devel

Source code:

https://git.openembedded.org/bitbake/

Testing

Bitbake has a testsuite located in lib/bb/tests/ whichs aim to try and prevent regressions. You can run this with "bitbake-selftest". In particular the fetcher is well covered since it has so many corner cases. The datastore has many tests too. Testing with the testsuite is recommended before submitting patches, particularly to the fetcher and datastore. We also appreciate new test cases and may require them for more obscure issues.

To run the tests "zstd" and "git" must be installed.

The assumption is made that this testsuite is run from an initialized OpenEmbedded build environment (i.e. source oe-init-build-env is used). If this is not the case, run the testsuite as follows:

export PATH=$(pwd)/bin:$PATH
bin/bitbake-selftest

The testsuite can alternatively be executed using pytest, e.g. obtained from PyPI (in this case, the PATH is configured automatically):

pytest