While calling wic ls/cp/rm/write in bitbake task along with do_image_wic,
it hung without return.
Due to commit [2255f28b57 wic: add WIC_SECTOR_SIZE variable][1] applied,
It calls get_bitbake_var in `wic ls|cp|rm|write' to define sector size.
By default, get_bitbake_var starts a `bitbake -e' to get variables which
triggers nested bitbake in this situation
Refer `wic create', adds option --vars and --image-name to support to
read bitbake variables from <image>.env files
NOTE: This commit does not add -e for `wic write' to avoid confliction
with existed option -e/--expand
[1] 2255f28b57
(From OE-Core rev: 793732a6ac2b3788d6c6635e5a496b117bd60584)
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* add missing % to print the values instead of:
| INFO: Build artifacts not found, exiting.
| INFO: (Please check that the build artifacts for the machine
| INFO: selected in local.conf actually exist and that they
| INFO: are the correct artifacts for the image (.wks file)).
|
| ERROR: ("The artifact that couldn't be found was %s:\n %s", 'kernel-dir', '/OE/build/deploy/images/qemux86-64')
(From OE-Core rev: e104c2b1273d8c5bd97893f318bf2a2699ef7f2d)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the custom kernel path is provided in options, then
use that path instead of the default path.
(From OE-Core rev: 1068102216a894c467f71f6046fdb37d5577545c)
Signed-off-by: Bill Pittman <bill.pittman@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use shutil.which to find the executable instead to silence the below warning:
$ cat tmp/work/intel_x86_64-poky-linux/core-image-base/1.0-r5/temp/log.do_image_wic
[snip]
DEBUG: Executing shell function do_image_wic
/path/layers/oe-core/scripts/wic:27: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
from distutils import spawn
INFO: Creating image(s)...
[snip]
[RP: Added conversion for missed function reference]
(From OE-Core rev: 3966cbf5c8a2dbc3a4f0f3eefdbeeb83f522bf87)
Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows extra space to be added after the last partition and is
especially useful when free space is needed for ex: adding partitions on
first boot with ex: systemd-repart[1] and the image is tested in QEMU.
[1] https://www.freedesktop.org/software/systemd/man/systemd-repart.html
(From OE-Core rev: f81b188bcf5aa18746fd622eb7b5c0dcb0b5c93d)
Signed-off-by: Kristian Klausen <kristian@klausen.dk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows the path for the temporary workdir used by wic to be set
when running wic from bitbake or directly from the command line.
(From OE-Core rev: 2e7314ac3a3cab89de93c932e8efc75de0feb0a7)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
currently 'wic cp' only works for copy file from local storage to
wic image.
enhance 'wic cp' to copy file/directory from wic image to local storage.
include selftest and 'wic help' updates.
[YOCTO#12169]
(From OE-Core rev: bd669c1809a378f93580eb9e0679a26ec6746cb8)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wic currently unable to remove non-empty directory in ext* partition.
enable wic rm to remove non-empty directory and all the sub-content
with -r flag.
update help documents for 'wic rm'.
[YOCTO #12404]
(From OE-Core rev: 5cb7a329d0aaac8fe5328eb2001692c540aa5ade)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The Wic help returned to the user is unreadable.
Use a custom ArgumentParser to override argparse help message.
change help message as suggest in
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12205
[YOCTO #12205]
changes applies to 'wic help', 'wic -h', 'wic --h' and 'wic --help'
(From OE-Core rev: d29d553cc096f4e1208c44dc595e1cf365c3dff3)
Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds SPDX license headers in place of the wide assortment of things
currently in our script headers. We default to GPL-2.0-only except for the
oeqa code where it was clearly submitted and marked as MIT on the most part
or some scripts which had the "or later" GPL versioning.
The patch also drops other obsolete bits of file headers where they were
encoountered such as editor modelines, obsolete maintainer information or
the phrase "All rights reserved" which is now obsolete and not required in
copyright headers (in this case its actually confusing for licensing as all
rights were not reserved).
More work is needed for OE-Core but this takes care of the bulk of the scripts
and meta/lib directories.
The top level LICENSE files are tweaked to match the new structure and the
SPDX naming.
(From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a global --debug option to assist debugging.
(From OE-Core rev: 4e23b22bdf7b0e1fd16c7e53b6a7232c9c92ce95)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The documentation says that --expand takes a comma-separated list of
partition:size pairs, but the code was splitting on hyphens.
Hyphens are not a transitional separator for a list of items, so change the code
to reflect the documentation.
(From OE-Core rev: a210e28bb3fd5433ebecf50e218fc548013b35dc)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Whilst its tempting to call bitbake_main(), the script also directly executes
bitbake. By doing both this creates all kinds of races. Therefore
change to use subprocess execution in all cases.
(From OE-Core rev: 0507ec4c6069490c9a503e9e2b65f0e7b1962d34)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(From OE-Core rev: f24c3538a27388d282a94e2d8a70cd9108d0919c)
Signed-off-by: Michael Davis <michael.davis@essvote.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we run wic within eSDK:
$ wic create mkefidisk -e core-image-minimal
ERROR: BUILDDIR not found, exiting. (Did you forget to source oe-init-build-env?)
In order to figure out variable values, one must have sourced
the OE build environment setup script. However, when we are in
within the eSDK environment which isn't initialised like the
normal OE build environment, we can't use wic utility with eSDK.
Reference:
https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#wic-requirements
While wic ought to be fixed to be able to run without bitbake
& native tools [YOCTO #11281], but this is a workaround to set
BUILDDIR in the environment so that bitbake environment is setup
for wic to build its required native tools.
(From OE-Core rev: 03fa13a269d2887cc5d13fd474fb39a2be037f2c)
Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wic modules in scripts/lib/ are needed for wic to work, but path to
the python module is not exported in eSDK environment and we were
using an absolutized path of wic script within the sysroots.
We now changed to use real script path instead, where the wic modules
are located. This will also resolved the tracebacks found when running
wic from within the eSDK environment.
Traceback (most recent call last):
File "/tmp/deploy/sdk/poky_sdk/sysroots/x86_64-pokysdk-linux/usr/bin/wic", line 58, in <module>
from wic import WicError
ImportError: No module named 'wic'
(From OE-Core rev: dcea30b885797ece3439cf1201795a975628d664)
Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wic needs a set of tools to be available from sysroots.
wic will find bitbake executable within the environment,
and wic was unable to locate bitbake executable within eSDK
because it wasn't setup with the OE build environment script.
Hence, we need to add bitbake file path into the environment
PATH for wic to be able to discover it and import bb modules.
(From OE-Core rev: 89df0d31c9dd22ceba4c95a2a56ca78e58d871a8)
Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the scriptpath module in order to standardize the adding of
bitbake and meta/lib path to sys.path.
(From OE-Core rev: 8aba1fd023ce3c6767bf42b9faf9ec14fd7c4d02)
Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create an option to wic doesn't change fstab file, the final
fstab file will be same that in rootfs and wic doesn't update
file, e.g adding a new mount point.
Users can control the fstab file content in base-files recipe.
This is useful if you want to only create an partition but not
add fstab mount point or add new mount point using label e.g:
LABEL=recovery /recovery auto defaults 0 1
(From OE-Core rev: 00420ec42140c1b752132bda190dede85756d157)
Signed-off-by: Fabio Berton <fabio.berton@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This command writes image to the media or another file with
the possibility to expand partitions to fill free target space.
[YOCTO #11278]
(From OE-Core rev: ac5fc0d691aad66ac01a5cde34c331c928e9e25a)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When wic can't find native sysroot it tries to build wic-tools
However, it's not possible when wic is run from bitbake.
Moreover, it's not even feasible anymore as wic-tools should be
used only when wic is run manually.
Checked if wic is run manually before building wic-tools.
(From OE-Core rev: 522ac21831944b06fbcc372ac61762576d792c22)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
At the moment, when building images with IMAGE_FSTYPES=wic one ends up
depending on wic-tools and thus syslinux and grub-efi even when not
using those at all. Ideally, building an image with wic should only
build the tools and components really needed.
The problem is that "wic-tools" is needed also for the manual
invocations of wic, in which case everything that might be needed has to
be built in advance.
Replaced dependency on wic-tools with dependency to a much shorter set
of tools that wic uses almost for any image: 'parted', 'gptfdisk',
'dosfstools' and 'mtools'.
[YOCTO #11552]
(From OE-Core rev: 33ca15b94dbe7204c556c4b5526edd529f6d85f4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moved misc.py from wic/utils/ to wic/
Removed wic/utils directory
(From OE-Core rev: df906f3caa0721756f5ed48fa657e62e05ae2aa3)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added empty 'wic rm' command that does nothing.
The functionality will be added by the next commits.
[YOCTO #11283]
(From OE-Core rev: f8e42c13609c482359fbdb225fb16a45101ae9ae)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added empty 'wic cp' command that does nothing.
The functionality will be added by the next commits.
[YOCTO #11283]
(From OE-Core rev: f0dcf39d52185430422cb0c94c7fe99c12764acd)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add parser for 'wic cp' subcommand and a custom argument type.
(From OE-Core rev: df5152c834ef58036b41ad51b19eda6abecd9543)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added empty 'wic ls' command that does nothing.
The functionality will be added by the next commits.
(From OE-Core rev: ba4613469cc2c3d3433be2e2f520f4fff6b3b333)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* optparse is deprecated and will not be developed further
(see: https://docs.python.org/2/library/optparse.html)
* argparse supports subcommands, which simplifies definition of
arguments and options
* reimplement help mechanism through sub-subcommands
[YOCTO #9636]
(From OE-Core rev: 4fe9635c72a6ee1becc47d832ab54724176ee811)
Signed-off-by: Andreas Reichel <andreas.reichel.ext@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
Signed-off-by: Daniel Wagner <daniel.wagner@siemens.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
roofs -> rootfs
(From OE-Core rev: 131629ca6238ea0596f5dd11d659ce2ca63067a4)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Default value for bootimg_dir was not set in main wic script
unlike the rest of artifacts: kernel_dir, roofts_dir and
native_sysroot. Set it to the value of STAGING_DATADIR for
consistency and to avoid confusion of wic plugin developers.
(From OE-Core rev: afc486b6316f1118c8dcc74a5e217bb217dd1f85)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Switched from using wic-tools recipe sysroot to using
image sysroot in wic script. This way is more logical
and consistent with the way wic uses other artifacts.
[YOCTO #11017]
(From OE-Core rev: fc94feda4f4785bc020413943c80050adea13dc0)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed unused exceptions from error.py
Moved definition of WicError to lib/wic/__init__.py
(From OE-Core rev: 15442d072bb6d93bd9b941726f93262503053da5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replaced sys.exit with raising WicError in main module.
(From OE-Core rev: b7c19f1ee8b850806c73ccd6f99c42d94a725fc9)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moved from lib/wic/utils/ to lib/wic as this is a core module.
(From OE-Core rev: 808c70b81de5c3cfc3dcb01f08213e2ea33b7252)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Set up wic logger using standerd logging module.
This is going to replace custom msger module.
(From OE-Core rev: f7d9e33aa129d8ab98dd1971154c29c275d103b0)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It will omit the given path from the resulting partition, and if the
given path ends in a slash, it will only delete the content, and keep
the directory.
Since mkfs only accepts whole directories as input, we need to copy
the rootfs directory to the workdir so that we can selectively delete
files from it.
Since we want to use the copyhardlinktree() function, we need to put
the generic oe lib in the module search path.
(From OE-Core rev: 6602392db3d391d926dead49fcc54326015cfe35)
Signed-off-by: Kristian Amlie <kristian.amlie@mender.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no need to pass scripts_path from main wic module
down the stack as it's not used there.
Removed scripts_path argument from DirectPlugin class
and wic_create function.
(From OE-Core rev: f9f72c506befdff13260f37ded0beaea3aa30fad)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wic code is hard to follow due to deep and twiggy class
inheritance tree.
Flatten imager tree:
wic -> wic_create -> Creator -> DirectPlugin -> DirectImageCreator
to
wic -> wic_create -> DirectPlugin
by
removing Creator class and creator module
merging DirectImageCreator into DirectPlugin
Changed APIs to use the same parameters names.
Passed parsed command line options as an object down the stack.
(From OE-Core rev: 1e28d512341ce470c7afb256a01e597ab87170ca)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used options.outdir instead of image_output_dir.
There is no sense to use extra variable for this.
(From OE-Core rev: c6013dcb158a84d48cc2677f1509681cf9e0a3cb)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Set default output directory to current dir.
[YOCTO #10783]
(From OE-Core rev: ad116c4d02ccf36e22fbf3e45e45bc508849a833)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rebuild wic-tools if its native sysroot doesn't exist to ensure
that all required tools are available.
(From OE-Core rev: 0d005d099a2b8ee1303b98710cdc78e06e14ab39)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wic used STAGING_DIR_NATIVE variable as a path to native sysroot.
This doesn't work with recipe specific sysroots as STAGING_DIR_NATIVE
points to the native sysroot of the current recipe.
Used RECIPE_SYSROOT_NATIVE variable of wic-tools recipe
to fix the issue.
(From OE-Core rev: de9d7d14cd03e4dfc5812890a53c79b706b56537)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Short variant of wic command line option --skip-build-check
is incorretly named -p. It's named -s in wic help and Yocto
documentation.
(From OE-Core rev: 8d4121459334349f10bfb0f4cda00a8461aab245)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a typo in the logging parameters, "filname" is being used instead of "filename" for yocto-kernel, yocto-layer and wic scripts. This didn't cause issues before since python 2 didn't validate unused parameters but with python >= 3.4.3 an exception is thrown. This patch fixes this parameter name.
[YOCTO #9834]
(From OE-Core rev: 5d123a6ec0e97652b2ec4295428797a336ef357a)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed print_function and with_statement imports from __future__
as they're supported by python 3 by default.
(From OE-Core rev: 77ae2daad5d775d710b953cf0c623ce74cb2c274)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Switched scripts/wic to use python3 as a default
python interpreter.
(From OE-Core rev: ea6245d2383e2ba905ef9f1ba210e5dadc779ad8)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This option enables generation of <image>.bmap file for the
result image using native bmaptool.
[YOCTO #9413]
(From OE-Core rev: d64c7b37c40b052510419b4d6629b83319c833e4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
