poky/scripts
Paul Barker 36575a9493 wic: Copy rootfs dir if fstab needs updating
By default, wic updates the /etc/fstab in the rootfs to include details
of additional partitions described in the selected wks file. If this
modification is performed in place, other tasks which create an image
file from the rootfs directory (e.g. do_image_tar and do_image_ext4)
will pick up the modified fstab file which would not be appropriate for
those images as they do not include the additional partitions described
in the wks file. wic does undo modifications to the fstab file once it
has finished creating the filesystem image, however this leaves open a
race condition if one of the other tasks reads the contents of the fstab
file from the rootfs directory between the point where wic modifies the
fstab file and the point where wic restores the files original content.

This could be solved by adding a lockfile for tasks which use the rootfs
directory to ensure that no other such task is reading the rootfs
directory while do_image_wic is running. This would serialize several
do_image_* tasks and result in slower builds, especially for large
images. Another drawback of this solution is that it is hard to
selectively optimise - adding lockfiles to do_image_* tasks would result
in these tasks always being serialized even if no fstab modification
will take place.

An alternative solution is to copy the rootfs directory when fstab needs
to be modified. The code to do this in wic already exists as it is
needed when including or excluding content in the rootfs. This still
results in an impact on build times but the copy uses hardlinks if
possible (so little data is actually copied) and we can make selective
optimisations to improve things. The rootfs copy will only take place if
fstab modification is required (or if it was already needed to include
or exclude rootfs content). We can also follow up with further
optimisations after this commit. So this second solution is chosen.

Fixes [Yocto #13994]

(From OE-Core rev: ce682a73b7447652f898ce1d1d0416a456df5416)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-20 22:46:18 +00:00
..
contrib graph-tool: add filter subcommand 2020-06-16 23:39:28 +01:00
lib wic: Copy rootfs dir if fstab needs updating 2021-01-20 22:46:18 +00:00
native-intercept scripts/native-intercept: Add chgrp intercept 2019-11-14 13:21:00 +00:00
postinst-intercepts update_udev_hwdb: clean hwdb.bin 2020-10-31 08:54:12 +00:00
pybootchartgui scripts/pybootchartgui: Fix to work with python 3.8 2020-03-15 22:23:19 +00:00
tiny scripts/tiny/ksize: Fix for more recent kernels 2020-05-09 18:57:21 +01:00
autobuilder-worker-prereq-tests meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
bitbake-prserv-tool meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
bitbake-whatchanged meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
buildall-qemu buildall-qemu: automate build testing for qemu MACHINEs 2020-04-24 14:10:08 +01:00
buildhistory-collect-srcrevs buildhistory-collect-srcrevs: sort directories 2020-06-17 11:10:28 +01:00
buildhistory-diff buildhistory-diff: use BUILDDIR to know where buildhistory is 2020-09-17 09:47:46 +01:00
buildstats-diff scripts/buildstats-diff: Add option to filter tasks 2019-07-17 09:36:35 +01:00
combo-layer scripts/combo-layer: Fix deprecation warning 2020-02-21 09:39:00 +00:00
combo-layer-hook-default.sh meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
combo-layer.conf.example combo-layer: partial import for '--history init' 2015-03-20 11:21:24 +00:00
cp-noerror meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
create-pull-request scripts/create-pull-request: fix putting subject containing / into cover letter 2019-07-31 13:50:46 +01:00
crosstap meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
devtool meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
gen-lockedsig-cache scripts/gen-lockedsig-cache: Only look at new format sstate objects 2020-01-06 23:54:39 +00:00
gen-site-config meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
install-buildtools scripts/install-buildtools: Update to 3.2 M3 buildtools 2020-10-20 11:11:45 +01:00
lnr meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
multilib_header_wrapper.h multilib_header_wrapper: Drop using __MHWORDSIZE 2020-05-18 06:33:36 +01:00
oe-build-perf-report oe-build-perf-report: Ensure correct data is shown for multiple branch options 2020-10-10 13:37:56 +01:00
oe-build-perf-test scripts/oe-build-perf-test: Use python3 from the environment 2019-12-15 09:10:46 +00:00
oe-buildenv-internal scripts/oe-buildenv-internal: Add pointer to scripts/install-buildtools 2020-04-01 11:44:24 +01:00
oe-check-sstate meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
oe-depends-dot meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
oe-find-native-sysroot meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
oe-git-archive meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
oe-git-proxy oe-git-proxy: fix dash "Bad substitution" 2019-09-16 23:02:44 +01:00
oe-gnome-terminal-phonehome meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
oe-pkgdata-browser oe-pkgdata-browser: Select a matching package when a recipe is selected 2020-01-10 21:18:22 +00:00
oe-pkgdata-browser.glade oe-pkgdata-browser: Make the window 1200x900 by default 2020-01-10 21:18:22 +00:00
oe-pkgdata-util scripts/oe-pkgdata-util: Enable list-pkgs to print ordered packages 2019-11-04 13:39:05 +00:00
oe-publish-sdk oe-publish-sdk: add --keep-orig option 2020-09-08 16:43:40 +01:00
oe-pylint meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
oe-run-native scripts: oe-run-native, fix *-native directories 2021-01-15 13:39:25 +00:00
oe-selftest oe-selftest: Implement console 'keepalive' output 2019-09-06 08:15:45 +01:00
oe-setup-builddir meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
oe-test meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
oe-trim-schemas meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
oepydevshell-internal.py meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
opkg-query-helper.py meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
pythondeps meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
README Further cleanup of various poky references 2011-04-21 12:56:16 +01:00
recipetool meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
relocate_sdk.py meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
resulttool meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
rootfs_rpm-extract-postinst.awk meta/classes/rootfs_rpm.bbclass: re implement base on rpm5 2010-07-24 00:46:57 +01:00
rpm2cpio.sh rpm2cpio.sh: fix to handle newline character 2020-02-19 11:24:15 +00:00
runqemu runqemu: Add support for VHD/VHDX rootfs 2021-01-20 00:45:05 +00:00
runqemu-addptable2image meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
runqemu-export-rootfs meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
runqemu-extract-sdk runqemu-extract-sdk: fix install debugfs on rootfs 2020-04-06 16:45:11 +01:00
runqemu-gen-tapdevs meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
runqemu-ifdown meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
runqemu-ifup meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
runqemu.README scripts/runqemu.README: fix typo 2017-05-18 14:01:47 +01:00
send-error-report meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
send-pull-request meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
sstate-cache-management.sh meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
sstate-diff-machines.sh meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
sstate-sysroot-cruft.sh meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
sysroot-relativelinks.py meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
task-time meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
test-reexec meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
test-remote-image meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
verify-bashisms meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
wic wic: Add workdir argument 2021-01-13 23:13:45 +00:00
yocto-check-layer checklayer: check layer in BBLAYERS before test 2020-07-27 10:37:58 +01:00
yocto-check-layer-wrapper meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00

This directory contains Various useful scripts for working with OE builds