Introduce a new variable "OEQA_REPRODUCIBLE_TEST_LEAF_TARGETS".
It can be used like OEQA_REPRODUCIBLE_TEST_TARGET but will try to use
sstate for the dependencies.
This can be used to "quickly" test the reproducibility of a single recipe.
[YOCTO #15701]
(From OE-Core rev: e2440618bd12d7288a7f3f7cfb92d377ae61a3b4)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code line was moved in
commit 5a9a5ad6a7 ("reproducibility: continue testing in case of build failure")
(From OE-Core rev: b4b60941153ebeb5019b26eeed593d0a1cdb4012)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Printed message looks like this:
2025-01-12 00:40:08,326 - oe-selftest - ERROR - reproducibleA build failed. Trying to compute built packages differences but the test will fail.
2025-01-12 00:40:08,326 - oe-selftest - INFO - Failure log for reproducibleA will be copied to <snip>/oe-reproducible-20250112-mza9w940/bitbake-reproducibleA.log
Fixes [YOCTO #15655] (first half).
(From OE-Core rev: 1c2287849a2742445d80c81d6a1c6c3e3668765e)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are several problems with these paths. Firstly they contain full
system paths which depend upon where the test was run. These are pretty
pointless and just take up a lot of space making the results files large.
Secondly, they contain the same path twice. The reference and target path
will always be the same thing in two different locations.
Strip off the prefix and remove the duplication. This does change the output
data but that can't really be avoided. It does shrink the results data and makes
it more readable.
(From OE-Core rev: 81a44de36e864b08687451fd85aeba7c529fd7f7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The "rawlogs" data consists of a long string of results data which is
already in a structured data format. I can't see this is adding much
value in duplciating the data but it does create a huge string with a
lot of long problematic pathnames and inflates the results data size.
I suggest we drop this data as obsolete and not necessary.
(From OE-Core rev: 5b2c70fab2ffa409b861d83f048b65d458d03a90)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since [0], the reproducibility test tries to save logs of failed builds.
Using the memory persistent bitbake prevent this by concatenating logs in the
single persistent instance (See [YOCTO #15641]).
So, force a BitBake server restart for each build to ensure a new log
file is created.
This fixes this error (seen on AB [1]):
2024-11-02 08:16:00,816 - oe-selftest - ERROR - reproducibleA build failed. Trying to compute built packages differences but the test will fail.
2024-11-02 08:16:00,819 - oe-selftest - INFO - ... ERROR
2024-11-02 08:16:00,819 - oe-selftest - INFO - Traceback (most recent call last):
File "/srv/pokybuild/yocto-worker/reproducible-meta-oe/build/meta/lib/oeqa/selftest/cases/reproducible.py", line 321, in test_reproducible_builds
self.copy_file(variables["BB_CONSOLELOG"], os.path.join(save_dir, "bitbake-%s.log" % name))
File "/srv/pokybuild/yocto-worker/reproducible-meta-oe/build/meta/lib/oeqa/selftest/cases/reproducible.py", line 216, in copy_file
shutil.copyfile(source, dest)
File "/usr/lib/python3.11/shutil.py", line 256, in copyfile
with open(src, 'rb') as fsrc:
^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/srv/pokybuild/yocto-worker/reproducible-meta-oe/build/build/build-st-meta-python/build-st/tmp/log/cooker/qemux86-64/20241102071141.log'
[0]: https://git.openembedded.org/openembedded-core/commit/?id=c78cc753843d4199443052e8deb0c9c3b7e4b580
[1]: https://valkyrie.yoctoproject.org/#/builders/87/builds/17/steps/40/logs/stdio
(From OE-Core rev: e89bbc00ba16574d719b199c01ffbf37646f4f54)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current reproducibility test stops all build tasks when a single
task fails (default BitBake behavior). This means that a single build
failure prevents the reproducibility of other packages from being
tested, which is not ideal.
To address this, we now use the --continue option of BitBake during test
builds, allowing the build process to proceed even when some tasks fail.
The failure cases are handled as gracefully as possible.
In the event of a build failure:
- The entire reproducibility test will be considered a failure.
- The complete BitBake log will be saved in the "saved output" directory
to facilitate debugging. On the autobuilder, this log should be
available at https://autobuilder.yocto.io/pub/repro-fail/.
- The last 20 lines of the log, which should show the failing recipes,
will be displayed in the oe-selftest console.
- If BitBake fails to create the deployment directory, it will be
manually created to allow the comparison process to proceed.
Here is what the output looks like when testing reproducibility of bash,
a failing recipe (hello-fail) and a non-reproducible recipe
(hello-norepro):
<snip>
2024-10-01 23:09:33,977 - oe-selftest - INFO - test_reproducible_builds (reproducible.ReproducibleTests.test_reproducible_builds)
2024-10-01 23:10:39,093 - oe-selftest - INFO - Non-reproducible packages will be copied to <snip>/yocto/repro_output/oe-reproducible-20241001-ng43_hzd
2024-10-01 23:10:39,094 - oe-selftest - INFO - Building reproducibleA (sstate allowed)...
2024-10-01 23:18:12,378 - oe-selftest - ERROR - Bitbake failed! but keep going... Log:
2024-10-01 23:18:12,379 - oe-selftest - INFO - Command 'bitbake --continue bash hello-norepro hello-fail' returned non-zero exit status 1:
2024-10-01 23:18:12,379 - oe-selftest - INFO -
2024-10-01 23:18:12,379 - oe-selftest - INFO - ... (last 20 lines of output)
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: Running task 976 of 990 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_package_qa)
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_populate_sysroot: Started
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_qa: Started
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_populate_sysroot: Succeeded
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: Running setscene task 342 of 343 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_create_package_spdx_setscene)
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_package_spdx_setscene: Started
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_package_spdx_setscene: Succeeded
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: Running setscene task 343 of 343 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_create_spdx_setscene)
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_spdx_setscene: Started
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_spdx_setscene: Succeeded
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_qa: Succeeded
2024-10-01 23:18:12,379 - oe-selftest - INFO - NOTE: Running noexec task 979 of 990 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_build)
2024-10-01 23:18:12,380 - oe-selftest - INFO - NOTE: Tasks Summary: Attempted 979 tasks of which 841 didn't need to be rerun and 1 failed.
2024-10-01 23:18:12,380 - oe-selftest - INFO - NOTE: Generating JSON CVE summary
2024-10-01 23:18:12,380 - oe-selftest - INFO - Complete CVE JSON report summary created at: <snip>/yocto/poky/build-master-st/reproducibleA/tmp/log/cve/cve-summary.json
2024-10-01 23:18:12,380 - oe-selftest - INFO -
2024-10-01 23:18:12,380 - oe-selftest - INFO - Summary: 1 task failed:
2024-10-01 23:18:12,380 - oe-selftest - INFO - <snip>/yocto/poky/meta/recipes-core/hello-single/hello-fail_1.0.bb:do_compile
2024-10-01 23:18:12,380 - oe-selftest - INFO - log: <snip>/yocto/poky/build-master-st/reproducibleA/tmp/work/core2-64-poky-linux/hello-fail/1.0/temp/log.do_compile.3221257
2024-10-01 23:18:12,380 - oe-selftest - INFO - Summary: There was 1 ERROR message, returning a non-zero exit code.
2024-10-01 23:18:12,380 - oe-selftest - ERROR - reproducibleA build failed. Trying to compute built packages differences but the test will fail.
2024-10-01 23:18:12,382 - oe-selftest - INFO - Building reproducibleB-extended (sstate NOT allowed)...
2024-10-01 23:46:58,451 - oe-selftest - ERROR - Bitbake failed! but keep going... Log:
2024-10-01 23:46:58,463 - oe-selftest - INFO - Command 'bitbake --continue bash hello-norepro hello-fail' returned non-zero exit status 1:
2024-10-01 23:46:58,463 - oe-selftest - INFO -
2024-10-01 23:46:58,463 - oe-selftest - INFO - ... (last 20 lines of output)
2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_write_ipk: Started
2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_qa: Started
2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_spdx: Succeeded
2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: Running task 978 of 990 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_create_package_spdx)
2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_package_spdx: Started
2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_create_package_spdx: Succeeded
2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_qa: Succeeded
2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_write_deb: Succeeded
2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_write_ipk: Succeeded
2024-10-01 23:46:58,463 - oe-selftest - INFO - NOTE: recipe bash-5.2.32-r0: task do_package_write_rpm: Succeeded
2024-10-01 23:46:58,464 - oe-selftest - INFO - NOTE: Running noexec task 979 of 990 (<snip>/yocto/poky/meta/recipes-extended/bash/bash_5.2.32.bb:do_build)
2024-10-01 23:46:58,464 - oe-selftest - INFO - NOTE: Tasks Summary: Attempted 979 tasks of which 2 didn't need to be rerun and 1 failed.
2024-10-01 23:46:58,464 - oe-selftest - INFO - NOTE: Generating JSON CVE summary
2024-10-01 23:46:58,464 - oe-selftest - INFO - Complete CVE JSON report summary created at: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/log/cve/cve-summary.json
2024-10-01 23:46:58,464 - oe-selftest - INFO -
2024-10-01 23:46:58,464 - oe-selftest - INFO - Summary: 1 task failed:
2024-10-01 23:46:58,464 - oe-selftest - INFO - <snip>/yocto/poky/meta/recipes-core/hello-single/hello-fail_1.0.bb:do_compile
2024-10-01 23:46:58,464 - oe-selftest - INFO - log: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/work/core2-64-poky-linux/hello-fail/1.0/temp/log.do_compile.4136075
2024-10-01 23:46:58,464 - oe-selftest - INFO - Summary: There were 5 WARNING messages.
2024-10-01 23:46:58,464 - oe-selftest - INFO - Summary: There was 1 ERROR message, returning a non-zero exit code.
2024-10-01 23:46:58,467 - oe-selftest - ERROR - reproducibleB-extended build failed. Trying to compute built packages differences but the test will fail.
2024-10-01 23:46:58,481 - oe-selftest - INFO - Checking deb packages for differences...
2024-10-01 23:46:58,586 - oe-selftest - INFO - Reproducibility summary for deb: same=52 different=1 different_excluded=0 missing=0 total=53
unused_exclusions=[]
2024-10-01 23:46:58,588 - oe-selftest - INFO - Checking ipk packages for differences...
2024-10-01 23:46:58,658 - oe-selftest - INFO - Reproducibility summary for ipk: same=52 different=1 different_excluded=0 missing=0 total=53
unused_exclusions=[]
2024-10-01 23:46:58,659 - oe-selftest - INFO - Checking rpm packages for differences...
2024-10-01 23:46:58,691 - oe-selftest - INFO - Reproducibility summary for rpm: same=52 different=1 different_excluded=0 missing=0 total=53
unused_exclusions=[]
2024-10-01 23:46:58,692 - oe-selftest - INFO - Running diffoscope
2024-10-01 23:46:59,765 - oe-selftest - INFO - ... FAIL
2024-10-01 23:46:59,766 - oe-selftest - INFO - Traceback (most recent call last):
File "<snip>/yocto/poky/meta/lib/oeqa/selftest/cases/reproducible.py", line 380, in test_reproducible_builds
self.fail('\n'.join(fails))
AssertionError: Bitbake reproducibleA failure
Bitbake reproducibleB-extended failure
The following deb packages are different and not in exclusion list:
<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb
The following ipk packages are different and not in exclusion list:
<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/ipk/./core2-64/hello-norepro_1.0-r0_core2-64.ipk
The following rpm packages are different and not in exclusion list:
<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/rpm/./core2_64/hello-norepro-1.0-r0.core2_64.rpm
2024-10-01 23:46:59,769 - oe-selftest - INFO - ======================================================================
2024-10-01 23:46:59,770 - oe-selftest - INFO - FAIL: test_reproducible_builds (reproducible.ReproducibleTests.test_reproducible_builds)
2024-10-01 23:46:59,770 - oe-selftest - INFO - ----------------------------------------------------------------------
2024-10-01 23:46:59,770 - oe-selftest - INFO - Traceback (most recent call last):
File "<snip>/yocto/poky/meta/lib/oeqa/selftest/cases/reproducible.py", line 380, in test_reproducible_builds
self.fail('\n'.join(fails))
AssertionError: Bitbake reproducibleA failure
Bitbake reproducibleB-extended failure
The following deb packages are different and not in exclusion list:
<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb
The following ipk packages are different and not in exclusion list:
<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/ipk/./core2-64/hello-norepro_1.0-r0_core2-64.ipk
The following rpm packages are different and not in exclusion list:
<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/rpm/./core2_64/hello-norepro-1.0-r0.core2_64.rpm
2024-10-01 23:46:59,770 - oe-selftest - INFO - ----------------------------------------------------------------------
2024-10-01 23:46:59,770 - oe-selftest - INFO - Ran 1 test in 2246.039s
2024-10-01 23:46:59,770 - oe-selftest - INFO - FAILED
2024-10-01 23:46:59,770 - oe-selftest - INFO - (failures=1)
2024-10-01 23:47:03,200 - oe-selftest - INFO - RESULTS:
2024-10-01 23:47:03,200 - oe-selftest - INFO - RESULTS - reproducible.ReproducibleTests.test_reproducible_builds: FAILED (2245.79s)
2024-10-01 23:47:03,203 - oe-selftest - INFO - SUMMARY:
2024-10-01 23:47:03,203 - oe-selftest - INFO - oe-selftest () - Ran 1 test in 2246.040s
2024-10-01 23:47:03,203 - oe-selftest - INFO - oe-selftest - FAIL - Required tests failed (successes=0, skipped=0, failures=1, errors=0)
=> Test failed but hello-norepro is displayed as non-reproducible.
The testresult.json contains:
{
"oeselftest_debian-12_qemux86-64_20240930000424": {
"configuration": { <snip> },
"result": {
"reproducible": {
"files": {
"package_deb": {
"different": [
{
"reference": "<snip>/yocto/poky/build-master-st/reproducibleA/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb",
"test": "<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb"
}
],
"different_excluded": [],
"missing": [],
"same": [
<snip>
{
"reference": "<snip>/yocto/poky/build-master-st/reproducibleA/tmp/deploy/deb/./core2-64/bash_5.2.32-r0_amd64.deb",
"test": "<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/bash_5.2.32-r0_amd64.deb"
},
<snip>
]
},
"package_ipk": { <same as deb> },
"package_rpm": { <same as deb> }
}
},
"reproducible.ReproducibleTests.test_reproducible_builds": {
"duration": 2146.5671875476837,
"log": "Traceback (most recent call last):\n File \"<snip>/yocto/poky/meta/lib/oeqa/selftest/cases/reproducible.py\", line 380, in test_reproducible_builds\n self.fail('\\n'.join(fails))\nAssertionError: Bitbake reproducibleA failure\nBitbake reproducibleB-extended failure\nThe following deb packages are different and not in exclusion list:\n<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb\nThe following ipk packages are different and not in exclusion list:\n<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/ipk/./core2-64/hello-norepro_1.0-r0_core2-64.ipk\nThe following rpm packages are different and not in exclusion list:\n<snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/rpm/./core2_64/hello-norepro-1.0-r0.core2_64.rpm\n",
"status": "FAILED"
},
"reproducible.rawlogs": {
"log": "DIFFERENT: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/hello-norepro_1.0-r0_amd64.deb\nSAME: <snip>/yocto/poky/build-master-st/reproducibleB-extended/tmp/deploy/deb/./core2-64/bash_5.2.32-r0_amd64.deb\n<snip>"
}
}
}
}
=> "reproducible.ReproducibleTests.test_reproducible_builds".status is
correctly "FAILED" but the reproducibility of bash and hello-norepro is
tested.
(From OE-Core rev: c78cc753843d4199443052e8deb0c9c3b7e4b580)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The 'codegen-units' option split the crate into multiple compilation units for parallel compilation. Currently, this split is causing the rustdoc to generate differnt binary between the builds.
To fix this the codegen-units & the lto options are disabled.
More info about options:
https://doc.rust-lang.org/cargo/reference/profiles.html#codegen-unitshttps://doc.rust-lang.org/rustc/codegen-options/index.html#lto
(From OE-Core rev: 0c00875de10b171f4ff2990af351a8124ec7e972)
Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We do see a few reproducible issues are depending on the path length of the build directory.
https://bugzilla.yoctoproject.org/show_bug.cgi?id=15554
The current implementation of reproducible tests having different names for directories but with same length.
The build directory names are changed to have different length.
(From OE-Core rev: 2466ed34a8e5afd8b6f83173316821b78edf954b)
Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're seeing reproducibility failures where some packages don't appear
in the "from sstate" builds. The common factor is these are all recipes
with PROVIDES = "virtual/XXX".
In a full build from scratch, these are build but in a build from sstate,
there are situations where they aren't. For now, to try and keep builds
working, work around the problem until we can better look into the problem.
It is likely recent taskhash imrovements have caused this to occur more
regularly.
(From OE-Core rev: 68086a333acc54390e4e589ef928dc90da3edb48)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Setting it explicitly replaces rust's default choice which is rustdoc
(needed for example in selftests and otherwise expected to be present
in typical rust installations):
https://github.com/rust-lang/rust/blob/master/config.example.toml#L320
This addresses some of the rust selftest failures but not all. Help
is appreciate to restore the selftest.
Unfortunately, this also breaks rust reproducibility (or rather exposes
that it was never properly fixed, as explained here:
https://lists.openembedded.org/g/openembedded-core/message/199288
)
(From OE-Core rev: 58eaf2ee6c0809bf0a0d3c1d177e62bda7241651)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The PGO (Profile-guided Optimization) collect data about the typical execution of a program
and then use this data to inform optimizations such as inlining, machine-code layout,
register allocation, etc.
This optimization is by default disabled in rust sources but enabled in Yocto and causing
the reproducibility issue in rustdoc binary. To fix the issue this optimization is set to
it's default 'false'.
More about the optimization: https://doc.rust-lang.org/rustc/profile-guided-optimization.html
With the reproducibility issue fixed, we can enable the reproducibility tests again.
(From OE-Core rev: 189c266378c8c4a918cb205b3888577c7ce76856)
Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The 'rustdoc' is moved to a separate 'rust-rustdoc' package. This is a
workaround to test if the main rust binary is reproducible even if
rustdoc isn't.
(From OE-Core rev: dfed7e63e199009ecca8e2e34f6a823c6e1677e0)
Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES can be used to prevent known
not-reproducible packages to make the reproducible test fail.
For example, in local.conf:
OEQA_REPRODUCIBLE_EXCLUDED_PACKAGES = "example-not-reproducible"
To quote the original exclusion commit (4864764667)
> selftest/reproducible: add an exclusion list for items that are not yet reproducible
> [...]
> Non-reproducible excluded packages are not given to diffoscope and do not cause a
> failure, but still saved side-by-side with non-reproducible failing ones to make
> investigation easier.
NB: Patch was written by Richard with small fixes and a commit message
from Yoann.
(From OE-Core rev: bea0e3f5b8b383ae8590b874a5878f67832d6929)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
No other change.
Just cosmetic to avoid tripping the maximum line length of patchtest
when the next element will be added to the list.
(From OE-Core rev: 7c401fe421c2085f2e7fc14589ca44ec1791db03)
Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Unfortunately there is still an issue with rustdoc not being reproducible
so part of the problem has been fixed but not everything.
Add the exclusion back until this has been addressed to avoid autobuilder
failures and long diffoscope analysis time.
(From OE-Core rev: ab03b3416dda65e39a756c02d87492e1fe5e0fb0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The '--remap-path-prefix' option removes all references to build directory
structure in the debug information within the compiled output for Cargo
dependencies and the project's binary.
However, some references to build directories remains in the final binary
in .rustc section in the form of compressed metadata and this makes the
build output dependent on the folder structure of the computer it's compiled on.
So, for reproducible builds, use the configuration option
'remap-debuginfo = true' along with the '--remap-path-prefix'.
[YOCTO# 14875]
(From OE-Core rev: 6ae62259afbbe861ed74211dab18a27b8c8d8b7a)
Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the bb_get_var calls trigger multiple "bitbake -e" executions
which slow the start of oe-selftest for any test. Rework the code to avoid
these delays.
(From OE-Core rev: c9ea3570f5dbdaf8c61e445ae9de759a0b71e21f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We can't have systemd here any longer without usrmerge. We don't really
want to enable the latter since having separate usr will likely result in
a class of reproducibility and host contamination issues that enabling it
might hide.
Also drop INHIBIT_PACKAGE_STRIP since we generally don't build with that
and the debug binaries should be generated regardless. I suspect this
is legacy from older issues.
(From OE-Core rev: 7b7411788e805fa067dd672c9771dcaf2af918a0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a follow-up of 76e5fcb2 that also allow users to chose
the package manager using OEQA_REPRODUCIBLE_TEST_PACKAGE
(From OE-Core rev: 3d414d85b44077bac57aba36707b0fc699a73e97)
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow users to set different build reproducibility targets than
the defaults using OEQA_REPRODUCIBLE_TEST_TARGET and
OEQA_REPRODUCIBLE_TEST_SSTATE_TARGETS variables in local.conf.
Fixing all issues from "world" builds is not possible in some
complex build environments with lots of layers. Limiting the focus to
a smaller subset allows using this test to detect and fix build
reproduction issues incrementally.
(From OE-Core rev: c66bebbce5995e386a1a4d055a914a39b6ee518d)
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't compare reproducibility of the native/cross components, only the
target ones. With the long build times of rust-native, the test now takes
crazy lengths of time so this tweak should allow us to reuse native/cross
artefacts from sstate whilst still testing the target output is
reproducible.
(From OE-Core rev: b494d83c639a877cefeb7cbab6d37195e492f059)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we provide sstate_targets recipes they are reported as missing
which is not quite correct because they are proveided to cut build/debugging time.
(From OE-Core rev: fd70a564d6946fa460638dd04ce2daecf4566cf3)
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There looks to be a reproducibility issue left in one of the rust
libraries. It doesn't appear to be a string issue but some binary
problem. Disable rust from the reproducibility testing until we can
get to the bottom of the issue (allowing wider testing of all the
other improvements).
(From OE-Core rev: a261333f6591ea94afc567dee04a2e3c6d5059cf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The issue is reported upstream:
https://bugs.ruby-lang.org/issues/18456
Otherwise I do not feel that further investigation of rdoc's parser to find
out the source of non-determinism is worth the time.
(From OE-Core rev: 208021f7212a8a790c350ccca720695c5bcbb1ca)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Diffoscope can end up running for a very long time if there are a lot of
changes. To put a limit on how long it can run, cap the maximum report
size at 250 MB by default.
(From OE-Core rev: 52d5f76f54eac384f9480dffe96df089d9ee8f33)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With this changeset, go becomes reproducible \0/
(From OE-Core rev: 03ada343ecbbbe3822a8fd74e678b01be6b9511a)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Reproducibility is here to stay and needs to be part of our default workflow.
Move the remaining code to base.bbclass so it is always a first class citizen
and it is clear people need to be mindful of it.
(From OE-Core rev: abb0671d2cebfd7e8df94796404bbe9c7f961058)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
rust-llvm-liblto and rust-llvm-staticdev sometimes vary in contents. Exclude
them from the test for now until we can work on and resolve the issues.
(From OE-Core rev: 60c12da5aae2534c972df851f26e4523fed03afc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is the result of automated script conversion:
scripts/contrib/convert-overrides.py <oe-core directory>
converting the metadata to use ":" as the override character instead of "_".
(From OE-Core rev: 42344347be29f0997cc2f7636d9603b1fe1875ae)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The hardcoded path issue was fixed, drop the exclusion.
(From OE-Core rev: 58e02c83dcf4d6c40c206a0371ef5d7f8db2b272)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The meson determinism issue is the same frozenset issue we encountered with
python itself. Remove the problematic pyc file until upstream work out the
best way to address the issue (as with core python).
(From OE-Core rev: ad00107dc02b3211f5d1a7fe889f538ee2bc064d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
go-dep was an effort for dependency management before go modules, which
since 2020 has been deprecated in favor of go modules. Since its not
developed any longer and go mdules is officially supported, this should
be retired from OE-core as well.
(From OE-Core rev: 1e7ed44d87034446f1d07692c9378c3b0a8a9dd3)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Cc: Otavio Salvador <otavio.salvador@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have fixes for perf reproducibility, so we can drop it from the
exclusion list.
(From OE-Core rev: 2989779fa244ec2c2c714c27f67753eefeeaf4af)
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are build paths which are in generated sources injected
into ${PN}-src. These are in generated files and don't influence the
binary output. As such we can correct them at do_package time by
tweaking in an appropriately injected function. This fixes
the reproducbility of the recipe.
(From OE-Core rev: cc1de818247c9ae329fdc0536658b3968a4caea2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a patch adding sorting to a couple of points in the Makefiles
which removes most of the determinism issues in ltp.
Build swapon before the main build to ensure libswapon.o is built
deterministically as it races with swapoff.
All issues reported on the upstream mailing list.
(From OE-Core rev: 0f51f9a37e5d058bce28cfe7b9a32a895f83c091)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This change adds core-image-weston to the reproducible build test.
For this we also need to update the static-group and static-passwd.
Besides the world package build which includes the wayland package,
this adds a wayland/weston-based image to the reproducible build test.
(From OE-Core rev: ffc4de47988ccf7568eecc8a27e8964beeaaacfb)
Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a hack to hardcode in specific rpaths which we then remove,
allowing the build to be reproducible.
Strip build patches out of one of the test scripts too.
(From OE-Core rev: 18299a114c66280ba2d00becc3fae235d3a6cbd7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a patch to address two determinism issues and allow reproducible
builds.
Also strip full paths we don't need out of some ptest files,
we can use the installed binaries.
(From OE-Core rev: 14e7cc6c6040b983024bec56fbd32d9101507ff0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When debugging why a single recipe doesn't reproduce, its a pain
to wait for the world to rebuild from scratch. Update the selftest
to allow this to be configured, for example you could set
targets as ['perf'] and sstate_targets as ['virtual/kernel']
and then it should only be rebuilding perf in the test rather than
things like the toolchain (parts of the kernel may be unavoiable
as they're not in sstate).
Can be run as:
OEQA_DEBUGGING_SAVED_OUTPUT=/tmp/perf-diffoscope oe-selftest -r reproducible.ReproducibleTests.test_reproducible_builds
to save diffoscope output.
(From OE-Core rev: 132a17d02f29711572e14a2f38a841323fbb6df6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
LTO likely doesn't buy us much here, disable it to allow the binaries
to be reproducible.
(From OE-Core rev: b4b5e52b5bd98deb0855700e4aae6228d52e5a83)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add sorting to the globbing within the Makefile to make the output
reproducible.
(From OE-Core rev: c43ae151f572786818fe048233b4bbfd6b0ba2cf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove hardcoding the build configuration into the help/version output
from swig to make the binaries reproducible.
(From OE-Core rev: 7ed28ae9717ea9dad4e131012186d5f08e8f0bec)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add sorting to wildcard expansion in the makefile to make builds
reproducible.
(From OE-Core rev: 5541ba76ccc0c416f315bc0dc14a20a33059bd5f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The library hardcodes paths to the python library internally and currently
these are build paths. Fix this to use the correct target path and
fix reproducibility in the process.
(From OE-Core rev: 4c190e8c9a2b53de3def1065fb6e479b72d040dc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to use CFLAGS with the correct WORKDIR in them, replace those
in the sysroot file with the ones appropriate to the current recipe.
(From OE-Core rev: 45edf189961aff1858be9bb7b63116073c0a0c10)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
