poky/scripts
Trevor Woerner 35807e8f63 wic: do not ignore ROOTFS_SIZE if the rootfs is modified
If the *.wks file contains a "--source rootfs" then
lib/wic/plugins/source/rootfs.py will be invoked to generate (what is assumed
to be) the rootfs partition. If the rootfs partition needs to be tweaked or
modified, the "rootfs.py" plugin will make a copy of the filesystem and then
perform the changes on that copy. In other words, if the "--source rootfs"
line of the *.wks file also contains any of:

	--exclude-path
	--include-path
	--change-directory
	--use-label (i.e. modify etc/fstab)

then the rootfs will be copied first, then the copy is modified.

If, for example, the unmodified IMAGE_ROOTFS is:

	.../tmp/work/qemuarm64_secureboot-oe-linux/core-image-base/1.0/rootfs

then the copy would be made at:

	.../tmp/work/qemuarm64_secureboot-oe-linux/core-image-base/1.0/tmp-wic/rootfs${LINENO}

where ${LINENO} is the line number where this "--source rootfs" line appears
in the *wks file.

When it comes time to make an actual partition of a specific filesystem type,
lib/wic/partition.py::prepare_rootfs() is called. It is in this function that
wic figures out if any extra size needs to be added. The bitbake variable used
to specify the ultimate rootfs size is ROOTFS_SIZE, and since this variable is
only valid for the rootfs (and not any other partitions), the code also
verifies that the partition being created is ${IMAGE_ROOTFS}:

	rsize_bb = get_bitbake_var('ROOTFS_SIZE')
	rdir = get_bitbake_var('IMAGE_ROOTFS')
	if rsize_bb and rdir == rootfs_dir:
		<use rsize_bb>
	else:
		<calculate the partition size using "du -ks $p">

As noted above, if lib/wic/plugins/source/rootfs.py has made a copy, then the
"rdir == rootfs_dir" clause will fail and the code will assume this partition
is not a rootfs since the strings do not compare equal.

Therefore, in order to determine if this is a rootfs, retain the existing
"rdir == rootfs_dir" comparison, but also add another one to check whether or
not this is a wic-generated copy of the rootfs.

STEPS TO REPRODUCE:
	- start with the following *wks file:
		bootloader --ptable gpt
		part /boot --size=100M --active --fstype=ext4 --label boot
		part /     --source rootfs      --fstype=ext4 --label root
	- and the following extra variable in conf/local.conf:
		IMAGE_ROOTFS_EXTRA_SPACE = "500000"
	- build an image
	- run it in qemu
		$ runqemu slirp nographic serial
	- verify the root partition has extra space:
		root@qemuarm64-secureboot:~# df -h
		Filesystem                Size      Used Available Use% Mounted on
		/dev/root               721.5M     67.4M    600.6M  10% /
		devtmpfs                477.7M         0    477.7M   0% /dev
		tmpfs                    40.0K         0     40.0K   0% /mnt
		tmpfs                   489.3M     92.0K    489.2M   0% /run
		tmpfs                   489.3M     68.0K    489.2M   0% /var/volatile
		/dev/vda1               120.4M     19.9M     91.4M  18% /boot
	- modify the "/" line of the *wks file to be:
		part /     --source rootfs      --fstype=ext4 --label root --exclude-path boot/
	- build image

	when it fails:
		root@qemuarm64-secureboot:~# df -h
		Filesystem                Size      Used Available Use% Mounted on
		/dev/root                73.4M     41.9M     25.8M  62% /
		devtmpfs                477.7M         0    477.7M   0% /dev
		tmpfs                    40.0K         0     40.0K   0% /mnt
		tmpfs                   489.3M     92.0K    489.2M   0% /run
		tmpfs                   489.3M     68.0K    489.2M   0% /var/volatile
		/dev/vda1               120.4M     19.9M     91.4M  18% /boot

	after this fix:
		root@qemuarm64-secureboot:~# df -h
		Filesystem                Size      Used Available Use% Mounted on
		/dev/root               721.5M     47.4M    620.6M   7% /
		devtmpfs                477.7M         0    477.7M   0% /dev
		tmpfs                    40.0K         0     40.0K   0% /mnt
		tmpfs                   489.3M     92.0K    489.2M   0% /run
		tmpfs                   489.3M     68.0K    489.2M   0% /var/volatile
		/dev/vda1               120.4M     19.9M     91.4M  18% /boot

Doing the math we see that the /boot partition is ~20MB and in the first image
the / partition contains this ~20MB in addition to the rest of the rootfs.
This ~20MB is completely wasted since it is used in the / partition, but then
the /boot partition is mounted on top of it, making the /boot directory of /
inaccessible. After the fix the / partition has an additional ~20MB since the
/boot portion is excluded.

Fixes [YOCTO #15555]

(From OE-Core rev: 1c690aa046ebca13d7b29de50d42b5d8a4a8486c)

Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-04-17 11:03:22 +01:00
..
contrib scripts/contrib: Add oe-image-files-spdx script 2025-02-18 11:56:03 +00:00
cross-intercept native/cross: Add ar wrapper for determinism 2021-11-26 17:01:08 +00:00
esdk-tools scripts/esdk-tools: use a dedicated, static directory for esdk tools 2023-11-05 08:44:06 +00:00
lib wic: do not ignore ROOTFS_SIZE if the rootfs is modified 2025-04-17 11:03:22 +01:00
native-intercept native/cross: Add ar wrapper for determinism 2021-11-26 17:01:08 +00:00
nativesdk-intercept nativesdk-intercept: Fix bad intercept chgrp/chown logic 2023-09-08 22:59:47 +01:00
postinst-intercepts update_gtk_icon_cache: Fix for GTK4-only builds 2023-12-08 16:58:34 +00:00
pybootchartgui pybootchartgui.py: python 3.12+ regexes 2024-07-22 16:53:06 +01:00
tiny scripts/tiny/ksize: Fix for more recent kernels 2020-05-09 18:57:21 +01:00
.oe-layers.json oe-setup-build: add a tool for discovering config templates and setting up builds 2024-02-19 11:47:53 +00:00
autobuilder-worker-prereq-tests scripts: Add copyright statements to files without one 2022-08-12 11:58:01 +01:00
b4-wrapper-poky.py b4-wrapper-poky.py: output errors to stderr 2025-03-25 21:20:41 +00:00
bblock scripts/bblock: add a script to lock/unlock recipes 2023-10-09 15:58:47 +01:00
bitbake-prserv-tool migrate_localcount: Drop long obsolete code 2024-10-08 13:26:39 +01:00
buildall-qemu buildall-qemu: automate build testing for qemu MACHINEs 2020-04-24 14:10:08 +01:00
buildhistory-collect-srcrevs buildhistory-collect-srcrevs: Adapt to the new variable override syntax 2021-08-26 22:09:44 +01:00
buildhistory-diff scripts/buildhistory-diff: drop use of distutils 2022-01-04 23:14:05 +00:00
buildstats-diff scripts: fix buildstats diff/summary hard bound to host python3 2023-05-05 11:07:26 +01:00
buildstats-summary buildstats-summary: look for buildstats if not specified 2024-10-25 15:25:32 +01:00
clean-hashserver-database scripts: Add clean-hashserver-database script 2025-03-18 11:25:36 +00:00
combo-layer scripts: python 3.12 regex 2024-02-13 13:51:41 +00:00
combo-layer-hook-default.sh scripts: Add copyright statements to files without one 2022-08-12 11:58:01 +01:00
combo-layer.conf.example combo-layer: partial import for '--history init' 2015-03-20 11:21:24 +00:00
cp-noerror scripts: Add copyright statements to files without one 2022-08-12 11:58:01 +01:00
create-pull-request scripts/create-pull-request: update URLs to git repositories 2023-08-14 12:51:21 +01:00
crosstap scripts: Various typo/grammar/punctuation fixes 2022-03-02 18:43:24 +00:00
cve-json-to-text.py cve-check: add support for cvss v4.0 2024-10-25 15:25:33 +01:00
devtool devtool: remove unused 'config' param from '_create_workspace' method 2025-01-20 13:38:59 +00:00
gen-lockedsig-cache scripts: Add copyright statements to files without one 2022-08-12 11:58:01 +01:00
git scripts: Add copyright statements to files without one 2022-08-12 11:58:01 +01:00
install-buildtools scripts/install-buildtools: write download URLs to files for traceability 2025-03-17 17:09:22 +00:00
lz4c scripts: wrap lz4c and convert to lz4 2024-12-17 11:41:53 +00:00
makefile-getvar scripts/makefile-getvar: add script to get values from Makefiles 2024-06-13 09:11:17 +01:00
multilib_header_wrapper.h multilib_header_wrapper: Drop using __MHWORDSIZE 2020-05-18 06:33:36 +01:00
oe-build-perf-report oe-build-perf-report: Use commit_time if available 2025-02-11 11:44:19 +00:00
oe-build-perf-test scripts/oe-build-perf-test: Use python3 from the environment 2019-12-15 09:10:46 +00:00
oe-buildenv-internal oe-buildenv-internal: update required Python version 2023-08-21 16:15:35 +01:00
oe-check-sstate scripts: python 3.12 regex 2024-02-13 13:51:41 +00:00
oe-debuginfod oe-debuginfod: add option for data storage 2024-04-30 22:20:16 +01:00
oe-depends-dot oe-depends-dot: improve '-w' behavior 2023-09-02 18:23:05 +01:00
oe-find-native-sysroot scripts/oe-find-native-sysroot: use bitbake-getvar 2023-09-07 07:53:51 +01:00
oe-git-archive meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
oe-git-proxy oe-git-proxy: fix dash "Bad substitution" 2019-09-16 23:02:44 +01:00
oe-gnome-terminal-phonehome scripts: Add copyright statements to files without one 2022-08-12 11:58:01 +01:00
oe-pkgdata-browser scripts: Add copyright statements to files without one 2022-08-12 11:58:01 +01:00
oe-pkgdata-browser.glade oe-pkgdata-browser: Make the window 1200x900 by default 2020-01-10 21:18:22 +00:00
oe-pkgdata-util scripts: python 3.12 regex 2024-02-13 13:51:41 +00:00
oe-publish-sdk scripts/oe-publish-sdk: Disable git gc to avoid build errors 2021-09-23 13:49:11 +01:00
oe-pylint scripts: Add copyright statements to files without one 2022-08-12 11:58:01 +01:00
oe-run-native scripts: oe-run-native, fix *-native directories 2021-01-15 13:39:25 +00:00
oe-selftest oe-selftest: drop redundant imports 2025-03-11 11:20:34 +00:00
oe-setup-build oe-setup-build: raise exceptions on errors 2024-12-18 11:11:55 +00:00
oe-setup-builddir scripts/oe-setup-builddir: add support for configuration summaries 2024-02-19 11:47:53 +00:00
oe-setup-layers oe-setup-build: add a tool for discovering config templates and setting up builds 2024-02-19 11:47:53 +00:00
oe-setup-vscode oe-init-build-env: generate .vscode from template 2024-02-19 11:34:33 +00:00
oe-test meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
oe-time-dd-test.sh scripts: Add copyright statements to files without one 2022-08-12 11:58:01 +01:00
oe-trim-schemas scripts: Add copyright statements to files without one 2022-08-12 11:58:01 +01:00
oepydevshell-internal.py scripts: Add copyright statements to files without one 2022-08-12 11:58:01 +01:00
opkg-query-helper.py scripts: python 3.12 regex 2024-02-13 13:51:41 +00:00
patchtest scripts/patchtest: split out result messages 2024-09-30 17:00:50 +01:00
patchtest-get-branch scripts: python 3.12 regex 2024-02-13 13:51:41 +00:00
patchtest-get-series patchtest: update SPDX identifiers 2023-10-17 22:53:30 +01:00
patchtest-send-results patchtest-send-results: add --debug option 2024-02-16 15:14:26 +00:00
patchtest-setup-sharedir patchtest: update SPDX identifiers 2023-10-17 22:53:30 +01:00
patchtest.README patchtest: Fix minor README typo 2025-03-13 11:00:35 +00:00
pull-sdpx-licenses.py scripts/pull-spdx-licenses.py: Add script 2024-07-02 22:29:13 +01:00
pythondeps scripts: Add copyright statements to files without one 2022-08-12 11:58:01 +01:00
README Further cleanup of various poky references 2011-04-21 12:56:16 +01:00
recipetool meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
relocate_sdk.py script/relocate_sdk.py: check dynamic loader arch before relocating interpreter 2025-02-11 11:44:19 +00:00
resulttool resulttool: Add support to create test report in JUnit XML format 2024-08-29 21:58:19 +01:00
rootfs_rpm-extract-postinst.awk meta/classes/rootfs_rpm.bbclass: re implement base on rpm5 2010-07-24 00:46:57 +01:00
rpm2cpio.sh rpm2cpio.sh: update to the last 4.x version 2023-06-21 18:38:25 +01:00
runqemu runqemu: also set GBM_BACKENDS_PATH 2025-04-11 11:44:22 +01:00
runqemu-addptable2image image_types/runqemu-addptable2image: Fix a minor typo 2022-05-15 08:57:15 +01:00
runqemu-export-rootfs qemu-helper: depend on unfs3 and pseudo directly 2022-11-29 10:29:58 +00:00
runqemu-extract-sdk qemu-helper: depend on unfs3 and pseudo directly 2022-11-29 10:29:58 +00:00
runqemu-gen-tapdevs runqemu-gen-tapdevs: remove only our taps 2023-06-24 12:21:48 +01:00
runqemu-ifdown runqemu-ifdown: catch up with ifup 2023-06-28 07:56:34 +01:00
runqemu-ifup runqemu: Fix automated call to runqemu-ifup 2023-06-28 07:56:34 +01:00
runqemu.README scripts/runqemu.README: fix typos and trailing whitespaces 2022-08-31 12:25:44 +01:00
send-error-report meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
send-pull-request meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
sstate-cache-management.py sstate-cache-management: add dry-run argument 2025-04-03 11:06:20 +01:00
sstate-diff-machines.sh scripts: Add copyright statements to files without one 2022-08-12 11:58:01 +01:00
sstate-sysroot-cruft.sh scripts: Add copyright statements to files without one 2022-08-12 11:58:01 +01:00
sysroot-relativelinks.py scripts: Add copyright statements to files without one 2022-08-12 11:58:01 +01:00
task-time scripts: Add copyright statements to files without one 2022-08-12 11:58:01 +01:00
test-reexec meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
test-remote-image meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00
verify-bashisms scripts: Add copyright statements to files without one 2022-08-12 11:58:01 +01:00
wic wic: fix WicError message 2022-06-22 22:40:28 +01:00
yocto_testresults_query.py scripts/yocto_testresults_query: add option to change display limit 2023-10-23 10:49:19 +01:00
yocto-check-layer yocto-check-layer: Allow OE-Core to be tested 2022-12-09 13:18:41 +00:00
yocto-check-layer-wrapper meta/lib+scripts: Convert to SPDX license headers 2019-05-09 16:31:55 +01:00

This directory contains Various useful scripts for working with OE builds