Commit Graph

511 Commits

Author SHA1 Message Date
Joshua Watt
4f3aca1a56 wic: Fix --extra-space argument handling
467f84e12b ("wic: Add --offset argument for partitions") broke the
--extra-space argument handling in wic. Fix the option and add a unit
test for the argument.

(From OE-Core rev: 87722a92c18f94917c8f70afc8cd0763462a5c25)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-12 08:25:04 +01:00
Joshua Watt
ba9be4bfa5 wic: Add --offset argument for partitions
Add support for an --offset argument when defining a partition. Many
SoCs require that boot partitions be located at specific offsets. Prior
to this argument, most WKS files were using the --align attribute to
specify the location of these fixed partitions but this is not ideal
because in the event that the partition couldn't be placed in the
specified location, wic would move it to the next sector with that
alignment, often preventing the device from booting. Unlike the --align
argument, wic will fail if a partition cannot be placed at the exact
offset specified with --offset.

Changes in V2:
* Fixed a small typo that prevented test_fixed_size_error from passing

(From OE-Core rev: 467f84e12b96bc977d57575023517dd6f8ef7f29)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-04 13:27:32 +01:00
Ricardo Ribalda Delgado
31b14186e9 wic: rootfs: Combine path_validation in one function
Combine all the common path validation in a function to avoid code
duplication.

Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: ebd12b10d17db0b4176b0188407d7e9b8420eab1)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Ricardo Ribalda Delgado
a293c76c53 wic: root: Add an opt. destination on include-path
Allow specifying an optional destination to include-path and make the
option aware of permissions and owners.

It is very useful for making a partition that contains the rootfs for a
host and a target Eg:

/ -> Roofs for the host
/export/ -> Rootfs for the target (which will netboot)

Although today we support making a partition for "/export" this might
not be compatible with some upgrade systems, or we might be limited by
the number of partitions.

With this patch we can use something like:

part / --source rootfs --fstype=ext4 --include-path core-image-minimal-mtdutils export/ --include-path hello

on the .wks file.

Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: e8c21c6ebaebde88151697381bdb2452f1171090)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Ricardo Ribalda Delgado
fc2589384b wic: misc: Do not find for executables in ASSUME_PROVIDED
Executables like tar won't be available on the native sysroot, as they
are part of the ASSUME_PROVIDED variable.

Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: 2f574d535f8665b26dab65c14668cf8fc7b751c0)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Ricardo Ribalda Delgado
c81d5c6243 wic: Avoid creating invalid pseudo directory
If the source of the rootfs is not a bitbake cooked image, or it is not
pointing to the root of one, we call pseudo again, which will produce
a new pseudo folder at rootfs/../pseudo

Eg:

part /etc --source rootfs --rootfs-dir=${IMAGE_ROOTFS}/home --fstype=ext4

Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: 799a24ae78655f7a3eda7456b1a0ffaf3e43ec16)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Ricardo Ribalda Delgado
19e778fe1b wic: Continue if excluded_path does not exist
If an excuded path does not exist, continue without an error.
This allows to seamleasly reuse .wks among different projects.

Eg:

part / --source rootfs --fstype=ext4 --exclude-path=opt/private_keys

Where /opt/private_keys in only populated by some of the image.bb files.

Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: 2dbd692b8e563cf991fb4ae1ef6129fda0d7e3c4)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Ricardo Ribalda Delgado
d19004da2f wic: Add --change-directory argument
This option allows to specify which part of a rootfs is going to be
included, the same way the -C argument on tar.

Thanks to this option we can make sure the permissions and usernames
on the target partition are respected, and also simplify the creation of
splitted partitons, not neeting to invoke external vars or using .wks.in
files. Eg:

part / --source rootfs --ondisk sda --fstype=ext4 --exclude-path=etc/   
part /etc --source rootfs --fstype=ext4 --change-directory=etc

Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: 2265d089a58e1f78f26d623ee667c420cb1c3bd4)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Ricardo Ribalda Delgado
c58711f0ea wic: Fix multi images .wks with bitbake
In order to support .wks files with multiple images inside bitbake we
need to explicitly set the pseudo database in use.

Eg: If we try this .mks:
part / --source rootfs --ondisk sda --fstype=ext4
part /export --source rootfs --rootfs=core-image-minimal-mtdutils --fstype=ext4

The username for all the files under /export will be set to the runner
of bitbake (usually UID 1000).

Before we run wic, we need to make sure that the pseudo database will be
flushed, and contains all the data needed.

Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: dde90a5dd2b22a539095d1bac82acc15c6380ac8)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Ricardo Ribalda Delgado
90164f0f3d wic: Fix permissions when using exclude or include path
When parameters include_path or exclude_path are passed to the rootfs
plugin, it will copy the partition content into a folder and make all
the modifications there.

This is done using copyhardlinktree(), which does not take into
consideration the content of the pseudo folder, which contains the
information about the right permissions and ownership of the folders.

This results in a rootfs owned by the user that is running the wic
command (usually UID 1000), which makes some rootfs unbootable.

This bug can be easily triggerd with the following .wks

part / --source rootfs --fstype=ext4 --exclude-path=home

And this sequence:

$ wic create test-permissions -e core-image-minimal -o test/
$ sudo mount test/test-permissions-202004080823-sda.direct.p1 /mnt
$ ls -la /mnt/etc/shadow

To fix this we copy the content of the pseudo folders to the new folder
and modify the pseudo database using the "pseudo -B" command.

If the rootfs is not a rootfs generated by bitbake a warning is shown
making the user aware that the permissions on the target might not match
what he expects.

WARNING: /tmp/test/../pseudo folder does not exist. Usernames and permissions will be invalid

Cc: Paul Barker <pbarker@konsulko.com>
(From OE-Core rev: fd739c15cdba221d9d497d3402b7d929c0be2ca4)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Lee Chee Yang
df77ee4723 wic: use Filesystem UUID when expand swap partition
part.get("uuid") return the Partition UUID from the partition
table instead of Filesystem UUID. This lead to swap partition
UUID not match/change when wic write expand swap partition.

change it to read the filesystem UUID using blkid. The output
from blkid should looks like this:
wic-partvzhiwq3s: LABEL="swap1" UUID="04e55c19-3f3f-4491-9e32-44eea8daa827" VERSION="1" TYPE="swap" USAGE="other"

[YOCTO #13313]

(From OE-Core rev: 683297eefebe83f848daad9927871242ab28ef91)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26 14:00:50 +01:00
Richard Purdie
f4e0b26568 Revert "wic: Fix permissions when using exclude or include path"
This reverts commit 36993eea89d1c011397b7692b9b8d61b499d0171.

After discussion on the mailing list it was felt these changes
were not ready yet.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-07 21:57:16 +01:00
Richard Purdie
157a12d499 Revert "wic: Add --embed-rootfs argument"
This reverts commit efdcf94801f6abe8e4099e324d9a3deccd8d4384.

After discussion on the mailing list it was felt these
changes were not ready yet.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-07 21:57:16 +01:00
Ricardo Ribalda Delgado
242412656b wic: Add --embed-rootfs argument
This option adds the content of a rootfs on a specific location on the
rootfs.

It is very useful for making a partition that contains the rootfs for a
host and a target Eg:

/ -> Roofs for the host
/export/ -> Rootfs for the target (which will netboot)

Although today we support making a partition for "/export" this might
not be compatible with some upgrade systems, or we might be limited by
the number of partitions.

With this patch we can use something like:

part / --source rootfs --embed-rootfs target-image /export --embed-rootfs target-image2 /export2

on the .wks file.

(From OE-Core rev: efdcf94801f6abe8e4099e324d9a3deccd8d4384)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-06 16:45:11 +01:00
Ricardo Ribalda Delgado
6bac089383 wic: Fix permissions when using exclude or include path
When parameters include_path or exclude_path are passed to the rootfs
plugin, it will copy the partition content into a folder and make all
the modifications there.

This is done using copyhardlinktree(), which does not take into
consideration the content of the pseudo folder, which contains the
information about the right permissions and ownership of the folders.

This results in a rootfs owned by the user that is running the wic
command (usually UID 1000), which makes some rootfs unbootable.

To fix this we copy the content of the pseudo folders to the new folder
and modify the pseudo database using the "pseudo -B" command.

(From OE-Core rev: 36993eea89d1c011397b7692b9b8d61b499d0171)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-06 16:45:11 +01:00
Lee Chee Yang
b5a3625e59 wic: align layer plugin path
wic look for external layer source plugin under scripts/lib/
while other tools look for lib/.

allow wic to check for source plugin at both scripts/lib/ and
lib/ to align with other tools while avoid breaking any existing
source plugin in external layer.

[YOCTO #13056]

(From OE-Core rev: 986baff26bd96a6265f5fe2d631818fff9f66374)

Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-02 15:24:59 +01:00
Chee Yang Lee
590555e3b8 wic/direct: reserve 2 sector for extended partition
extended partition should reserve at least 2 unallocated sectors
between the start of the extended partition and the start of a
logical partition.

[YOCTO #13658]

(From OE-Core rev: 884c8a0e50a49bdca5e048197a9dc0ff37cc8d1a)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-24 16:39:40 +00:00
Armin Kuster
26525f2f4f wic/engine: lets display an error not a traceback
If the requested partition does not exist in this request "wic ls {path}:pnum"
display a nice message not a trackback

Also fix displaying the pnum and not "%s"

(From OE-Core rev: 15d1722950a22649905cf8a5789d3cfe48a2a892)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-02 21:00:54 +00:00
Pierre-Jean Texier
0c8892fa1d wic/engine: dump the partition table in JSON format only
since commit 03154d2cf25c1d5ce908da9e72d324004b8d5722 ("sfdisk: add -J between mutually exclusive options")
in util-linux [1], the '-J' can't be used with the '-d' option anymore.

So, since we're using json format output, drop the '-d' option.

Fixes:

ERROR: _exec_cmd: <snip>/usr/sbin/sfdisk -dJ <snip>/core-image-minimal-qemux86-64.wic returned '1' instead of 0
output: sfdisk: mutually exclusive arguments: --list-free --json --dump

[1] - 03154d2cf2 (diff-31d5cc325380dbdafe64d0e019846f12)

(From OE-Core rev: 49e7f3e8e3a775510d8b0e2826a06d000934706b)

Signed-off-by: Pierre-Jean Texier <pjtexier@koncepto.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-01 10:40:11 +00:00
Alexander Kanavin
e88fe83014 qemux86: do not add vga=0 to kernel parameters
This was added ages ago to enable GL passthrough with
vmware driver, and is no longer relevant, as std or virgl is used
instead nowadays.

Original commit:

commit 072545b111
Author: Richard Purdie <rpurdie@linux.intel.com>
Date:   Wed Jan 21 17:40:51 2009 +0000

    scripts/poky-qemu-internal: Add support for GL passthrough in qemux86 images

(From OE-Core rev: 857078ba8eda153f4a097683db551a7d310ecc01)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-18 23:53:54 +00:00
Kevin Hao
dd372f09fc wic: Honor APPEND
APPEND is used to add additional parameters to kernel by features, such
as read-only-rootfs. So we should honor this variable when we compose
the kernel parameter in wic. I know we also can resolve this kind of
issue by using the .wks.in template introduced by commit 42e870c5ed
("image_types.bbclass: support template .wks.in files for wic"), but the
APPEND is needed by all the wks, it would seems pretty ridiculous to me
to change all the .wks to .wks.in and then foist the APPEND into them.
So the APPEND is definitely deserved to export to the wic directly.

[Yocto #12809]

(From OE-Core rev: 18981b8a457104391dfd94938c247eac04e4ed50)

Signed-off-by: Kevin Hao <kexin.hao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-15 22:42:14 +00:00
Maxim Uvarov
8290a50fe2 wic: fix images build in parallel
OE wic plugins create temporary file with the index of the line
tmp file name. This causes race in case several builds run in time.
If source_params['file'] is an absolute path, the cr_workdir prefix
is not applied by os.path.join(). So instead it writes to a ".1"
file next to the original image - this is outside the WORKDIR
and at risk of collision.

(From OE-Core rev: c68d8a37ba4348fe1c0e75c63b5668187d326ec2)

Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
Suggested-by: Ilias Apalodimas <ilias.apalodimas@linaro.org>
Suggested-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-19 13:24:39 +00:00
Paul Barker
2c81446248 wic: Add --include-path argument
This option adds the contents of the given path to a partition built
with the rootfs source plugin. The path is relative to the directory in
which wic is running not the rootfs itself so use of an absolute path
is recommended. This option is most useful when multiple copies of the
rootfs are added to an image and it is required to add extra content to
only one of these copies. This option only has an effect with the
rootfs source plugin.

(From OE-Core rev: d4cd27a9837426e809190548a83c6c7c76505114)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10 21:18:22 +00:00
Paul Barker
ab6e8dae1e wic: Preserve intermediate files
When debugging issues or trying to add new features in wic it's useful
to examine the intermediate files created by the tool such as images of
the individual partitions. Previously these files were created in a
subdirectory of IMGDEPLOYDIR and, deleted by wic itself and then the
working subdirectory completely removed at the end of IMAGE_CMD_wic.
Instead we should work in a staging directory named build-wic and leave
behind intermediate files for analysis if necessary.

This has the added benefit of allowing individual partition images to
be preserved by adding commands to IMAGE_CMD_wic_append() in an image
recipe if needed. This is useful where an update system requires exact
sized and pre-formatted partition images for deployment.

(From OE-Core rev: 07b6b284204e912b5d08bdd0ca51b766ab7dd9c1)

Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-10 21:18:22 +00:00
Kalle Lampila
b14588a778 wic/filemap: If FIGETBSZ iotctl fail, failback to os.stat
Some file systems don't support fetching the block size (notably the
file system Docker uses for containers), so if iotctl() fail, try to use
failback via os.stat() to get block size.

(From OE-Core rev: e219f5175177a640dd62833082ea19adc1c13d42)

Signed-off-by: Kalle lampila <kalle.lampila@lempea.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-01-02 16:39:13 +00:00
Chee Yang Lee
89288705c6 wic: 'wic cp' to copy from image
currently 'wic cp' only works for copy file from local storage to
wic image.

enhance 'wic cp' to copy file/directory from wic image to local storage.

include selftest and 'wic help' updates.

[YOCTO#12169]

(From OE-Core rev: bd669c1809a378f93580eb9e0679a26ec6746cb8)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-27 13:25:18 +00:00
Chee Yang Lee
3555766ea8 wic: rm with -r flag support
wic currently unable to remove non-empty directory in ext* partition.
enable wic rm to remove non-empty directory and all the sub-content
with -r flag.
update help documents for 'wic rm'.
[YOCTO #12404]

(From OE-Core rev: 5cb7a329d0aaac8fe5328eb2001692c540aa5ade)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21 23:08:19 +00:00
Chee Yang Lee
621b10f007 wic: beautify 'wic help'
The Wic help returned to the user is unreadable.

Use a custom ArgumentParser to override argparse help message.

change help message as suggest in
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12205
[YOCTO #12205]

changes applies to 'wic help', 'wic -h', 'wic --h' and 'wic --help'

(From OE-Core rev: d29d553cc096f4e1208c44dc595e1cf365c3dff3)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14 13:20:59 +00:00
Haris Okanovic
a98c7c7aa3 isoimage-isohybrid.py: Parameterize ESP partition size
Add "esp_extra_blocks" plugin parameter so that caller may change
ESP's free space from the default 100 blocks.

(From OE-Core rev: db4be094b0d6741ca4f9e2cbb499fc9c224fa690)

Signed-off-by: Haris Okanovic <haris.okanovic@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-10 14:06:10 +00:00
Haris Okanovic
36bb63c3d2 isoimage-isohybrid.py: Parameterize ESP label
Add "esp_label" plugin parameter so that caller may override default
ESP partition label 'EFIimg'.

(From OE-Core rev: 6b3b9042a55d47205b16ef792f93a9bb240fd7cb)

Signed-off-by: Haris Okanovic <haris.okanovic@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-10 14:06:10 +00:00
Chee Yang Lee
a94e0e14e1 wic/engine: use 'linux-swap' for swap file system
[YOCTO #13312]
see https://bugzilla.yoctoproject.org/show_bug.cgi?id=13312

wic/engine.Disk._get_part_image was looking at variable fstypes for
supported fstype which is 'swap' but image build with 'linux-swap'.
supported fstype should be 'linux-swap'.

(From OE-Core rev: 7e6da22fe4faf841bcec02e55f376b4dae04d6a8)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-23 16:30:36 +01:00
Chee Yang Lee
8ec9acb55d wic/help: change 'wic write' help description
update the help desciption for 'wic write' command to
match behaviour.

see commit:
https://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/scripts/wic?id=74b44e59be89c738f702948f9e16e367c0ba43ba

(From OE-Core rev: 521942bde1d49d987c85bf071275e8e7150d012a)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-23 16:30:36 +01:00
Eugene Smirnov
8d619931ee wic/rawcopy: Support files in sub-directories
If the source file is located in a subdirectory of DEPLOY_DIR
rawcopy will currently fail in sparse_copy function on
open(dst_fname, 'wb'), as the parent directory for destination
file does not exist.

This patch helps to avoid that by recursively creating
parent directories.

(From OE-Core rev: 073c435644091c2801e45c6d02afa917de575082)

Signed-off-by: Eugene Smirnov <evgenii.smirnov@here.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-19 23:18:33 +01:00
Michael Cooper
7a1d68d136 wic/direct: Partition numbering is broken for MBR primary partition #4
When wks describes extra partitions that aren't in the partition
table (e.g. boot loader) and exactly four primary MBR partitions, the
last partition gets added to fstab as partition #5 instead of #4.

[YOCTO #13560]

(From OE-Core rev: 7537580b3dd21bd512fb26e56e92b6553c549fa8)

Signed-off-by: Michael Cooper <michaelcooper81@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-08 11:22:24 +01:00
Alessio Igor Bogani
b404c0cd3c wic: Using the right rootfs size during prepare_rootfs
The commit 8e48b4d6c4 makes wic ignores IMAGE_ROOTFS_SIZE for rootfs
size and makes it uses the computed one only. Re-add support for
IMAGE_ROOTFS_SIZE variable and compute roots size only if the former
is not defined. Moreover the size of a provided directory with
--rootfs-dir="" in the .wks file should always be computed on the fly,
else every partition will be constrained to be the same size as what
ever value was in ROOTFS_SIZE.

(From OE-Core rev: 0217a0392b8cc534fcc0d760b7663a1845b306f5)

Signed-off-by: Alessio Igor Bogani <alessio.bogani@elettra.eu>
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27 13:02:16 +01:00
Alistair Francis
b8c2575002 qemuriscv: Generate a wic rootFS with a larger filesystem
This allows us to generate a rootFS with a large filesystem for use with
QEMU.

(From OE-Core rev: e06439200e44999c1e2f88d7d6c651da13698ca7)

Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-03 09:55:50 +01:00
Jason Wessel
7365a605ae wic: Make disk partition size consistently computed
When using different root directories with a wks file wic is using the
value from the original ROOTFS_SIZE which is not correct.  Example:

Number  Start   End     Size    File system  Name       Flags
 1      20.5kB  318MB   318MB   fat16        otaefi     legacy_boot, msftdata
 2      318MB   636MB   318MB   ext4         otaboot
 3      636MB   1709MB  1074MB  ext4         otaroot
 4      1709MB  2027MB  318MB   ext4         otaboot_b
 5      2027MB  3101MB  1074MB  ext4         otaroot_b
 6      3101MB  5249MB  2147MB  ext4         fluxdata

The partitions 1, 2, and 3 incorrectly inherit the size, instead of using
a computed size.  With the patch applied it is working properly:

Number  Start   End     Size    File system  Name       Flags
 1      20.5kB  14.5MB  14.5MB  fat16        otaefi     legacy_boot, msftdata
 2      14.5MB  65.3MB  50.8MB  ext4         otaboot
 3      65.3MB  1139MB  1074MB  ext4         otaroot
 4      1139MB  1190MB  50.8MB  ext4         otaboot_b
 5      1190MB  2264MB  1074MB  ext4         otaroot_b
 6      2264MB  4411MB  2147MB  ext4         fluxdata

As for the removal of the bb.warn, if the size is not specified, it is
not something to warn the end user about.  Some of my default images
make use of the head room + a computed directory size or while generating
images.

(From OE-Core rev: 8e48b4d6c4d0ed213089a7449fea63aa0656e786)

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-12 16:23:57 +01:00
Jason Wessel
351a10ccd6 wic: Add partition type for msdos partition tables
In order to create a msdos partition table disk image that can auto
expand after the image is copied to an SD card, wic needs the ability
to have a primary partition as the last entry.  The desired use case
is to be able to create an A/B update partition image scheme with a
/var volume that can be auto expanded to the remainder of the SD card
at run time.

The typical .wks file will look similar to the following:

bootloader --ptable msdos
part / --source rawcopy --sourceparams="file=u-boot.imx" \
   --ondisk mmcblk --no-table --align 1 --size 1
part /boot --source bootimg-partition \
   --ondisk mmcblk --fstype=vfat --label boot --active --align 4 --size 16
part / --source rawcopy --sourceparams="file=imx6_boot.otaimg" \
   --ondisk mmcblk --fstype=ext4 --label otaboot --align 4 --type logical
part / --source rawcopy --sourceparams="file=imx6.otaimg" \
   --ondisk mmcblk --fstype=ext4 --label otaroot --align 4 --type logical
part / --source rawcopy --sourceparams="file=imx6_boot.otaimg" \
   --ondisk mmcblk --fstype=ext4 --label otaboot_b --align 4 --type logical
part / --source rawcopy --sourceparams="file=imx6.otaimg" \
   --ondisk mmcblk --fstype=ext4 --label otaroot_b --align 4 --type logical
part /var --source rawcopy --sourceparams="file=imx6_var.otaimg" \
    --ondisk mmcblk --fstype=ext4 --label fluxdata --align 4

Without the patch applied, wic will generate an SD card image that looks like:

Disk boot.img: 890940s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start    End      Size     Type      File system  Flags
 1      2056s    48001s   45946s   primary   fat16        lba
 2      48008s   132467s  84460s   primary   ext4
 3      132472s  454467s  321996s  primary   ext4
 4      454471s  890939s  436469s  extended               lba
 5      454472s  538931s  84460s   logical   ext4
 6      538936s  860931s  321996s  logical   ext4
 7      860936s  890939s  30004s   logical   ext4         boot

With the patch applied a primary partition can be created at the end
of the image which can be expanded to fill the free space on the media
where the image has been copied, which looks like:

Disk boot.img: 890940s
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags:

Number  Start    End      Size     Type      File system  Flags
 1      2056s    48001s   45946s   primary   fat16        lba
 2      48007s   860931s  812925s  extended               lba
 5      48008s   132467s  84460s   logical   ext4
 6      132472s  454467s  321996s  logical   ext4
 7      454472s  538931s  84460s   logical   ext4
 8      538936s  860931s  321996s  logical   ext4
 3      860936s  890939s  30004s   primary   ext4         boot

(From OE-Core rev: 56add7cc547e0113cdf980579d1421b14cc233e5)

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-12 16:23:57 +01:00
William Bourque
def9b64e27 wic/plugins: Source that support both EFI and BIOS
Add a source plugin that support both EFI and legacy PC-Bios.
While using this plugin, both bootloaders configurations reside
in the same /boot partitions.
This plugin has very little code : to avoid code duplication,
we simply re-import bootimg-pcbios and bootmg-efi source and
call both their SourcePlugin methods.

(From OE-Core rev: c8545d54139c6c48bffd1dd1d39d79891626c6f7)

Signed-off-by: William Bourque <wbourque@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-23 22:26:28 +01:00
Ricardo Ribalda Delgado
58589bc538 wic: Fix (again) partition files UIDs on multi rootfs images
Commit 450335ba5e73a375eb9932b4c4cf37979640dbfc copies the pseudo
database to the working directory in order to have ownership information
when the filesystem is generated.

Unfortunately this does not work anymore. The filenames on the database
are absolute and there is no information about the new directory.

Instead of fixing the database, we could redo a bit the way we patch the
fstab file. Now I am saving the old contents of fstab, modifying the
file and then reverting the changes on exit.

This is faster than the previous approach, although it can cause
indeterminism if the application is killed before finishing.

(From OE-Core rev: dcbf7b864dd1713b54a172d8714ce1508482f086)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-19 08:41:40 +01:00
Chee Yang Lee
b697aba61e wic: add support for kernel with initramfs bundled
when INITRAMFS_IMAGE_BUNDLE and INITRAMFS_IMAGE are set, wic should
look for kernel with initramfs image bundled.

Include required variable MACHINE, INITRAMFS_IMAGE_BUNDLE,
INITRAMFS_IMAGE, INITRAMFS_LINK_NAME and KERNEL_IMAGETYPE in WICVARS.
No longer require default value for variable kernel as KERNEL_IMAGETYPE
is not optional variable and included in WICVARS.
image_types_wic to inherit kernel-artifact-names to obtain default
INITRAMFS_LINK_NAME when INITRAMFS_IMAGE_BUNDLE are set.

update wic.Wic2.test_image_env test case to filter optional
variable INITRAMFS_LINK_NAME, INITRAMFS_IMAGE and INITRAMFS_IMAGE_BUNDLE.

(From OE-Core rev: bac984fbb2d5ad5d13ba3275c8a3e878d8753c58)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-17 19:25:02 +01:00
Robert Yang
b19ce5d2e8 wic: python2 -> python3
(From OE-Core rev: 9303f92599343adf645fee5d2434fadd97e7febb)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-30 22:40:52 +01:00
Chee Yang Lee
fc0c36a647 wic/bootimg-efi: allow multiple initrd
Allow plugin bootimg-efi to configure with multiple initrd
through source parameter.

Uses ; to separate each initrd.

e.g:
--sourceparams="loader=${EFI_PROVIDER},initrd=initrd1;initrd2"

(From OE-Core rev: c7b0823f9ab6a9959aaa809b8c3f70d199feb64d)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-19 22:13:39 +01:00
Chee Yang Lee
97feebdf99 wic/plugins: kernel image refer to KERNEL_IMAGETYPE
replaced hardcoded kernel image with KERNEL_IMAGETYPE.
set kernel image to "bzImage" incase KERNEL_IMAGETYPE not set.

(From OE-Core rev: 88a9fef761c5e67b2964fedc85a7e8ad37067564)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-11 13:26:55 +01:00
Ross Burton
ca4cdfcd18 wic/filemap: handle FIGETBSZ failing
Some file systems don't support fetching the block size (notably the file system
Docker uses for containers), so handle the iotctl() failing and raise the
expected error.

(From OE-Core rev: 3757073726a00c5250556aae3d0daac76b88085e)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-08 16:01:41 +01:00
Chee Yang Lee
cc083399ce wic/engine: include .wks.in in wic search and list
allow wic to list and search for kickstart file in .wks.in extension.
basename show by wic list images to fully exclude extension.

(From OE-Core rev: 2c0a292a790ad069648e37b1b29fcea656fcf3e4)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-07 09:11:49 +01:00
Chee Yang Lee
01543538d1 wic: bootimg-efi: add label source parameter
Add new source parameter label to allow custom boot.conf/grub.cfg label,
so far it's hardcoded to "Boot".

Default label to "Boot" for systemd-boot and blank for grub-efi when source
parameter label are not set.

(From OE-Core rev: 7a0aab1aa31e66e6bc94c04c2f6c1043b64a8967)

Signed-off-by: Chee Yang Lee <chee.yang.lee at intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-30 12:37:03 +01:00
Chee Yang Lee
92cc79f4e6 wic/bootimg-efi: replace hardcoded volume name with label
volume name should refer to --label in .wks.
Replace the hardcoded volume name  with label.
set "ESP" as default name when no lable specified.

(From OE-Core rev: 5621aceaf39ef0dc097b16c83e73b9882c987a7c)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-15 17:53:29 +01:00
Richard Purdie
ffae400179 meta/lib+scripts: Convert to SPDX license headers
This adds SPDX license headers in place of the wide assortment of things
currently in our script headers. We default to GPL-2.0-only except for the
oeqa code where it was clearly submitted and marked as MIT on the most part
or some scripts which had the "or later" GPL versioning.

The patch also drops other obsolete bits of file headers where they were
encoountered such as editor modelines, obsolete maintainer information or
the phrase "All rights reserved" which is now obsolete and not required in
copyright headers (in this case its actually confusing for licensing as all
rights were not reserved).

More work is needed for OE-Core but this takes care of the bulk of the scripts
and meta/lib directories.

The top level LICENSE files are tweaked to match the new structure and the
SPDX naming.

(From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-09 16:31:55 +01:00
Angus Lees
97f7d39702 Revert "wic: Set a miniumum FAT16 volume size."
This reverts commit f7dfb4d43247d3c13a4e0a3853007d63b9512b83.

FAT16 volumes do not have a minimum size of 8250 blocks
(== slightly over 4MB).  Exhibit A: floppy disks.

The original commit message suggests this was a workaround for a bug in
parted - in which case we should fix it there, or use the wic
`--fixed-size` option to pass down an enforced minimum from a more
context-aware point in the callstack.

(From OE-Core rev: e0a1ae3863bc229512d43e4a5248551bf25950e5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-26 10:09:08 +01:00