Commit Graph

32 Commits

Author SHA1 Message Date
Ioan-Adrian Ratiu
436df4de37 wic/isoimage-isohybrid: remove do_stage_partition()
The purpouse of this function was to check dependencies for building a
hybrid iso and build them using bitbake if not found. Calling bitbake in
this context means this wic plugin itself cannot be instrumented inside
bitbake recipes which is undesirable, the benefits of this are clear:
there is no need to maintain outside scripts to generate an iso using wic
and the isohybrid building logic can be further abstracted away into an
isohybrid.bbclass in the future which can be easily inherited or something
similar.

So remove the function and add all dependencies to NATIVE_RECIPES so that
wic can print useful errors when they're not built.

To automate building the isohybrid image dependencies, add the following
somewhere in your image build inheritence hierarcy (or maybe create a
bbclass in the future to do these sort of things automatically):

DEPENDS += "syslinux syslinux-native cdrtools-native e2fsprogs-native \
            parted-native dosfstools-native mtools-native grub-efi-native"

(From OE-Core rev: ba4346069ab87f1cf942d1928f911eca6a9d65cd)

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:12 +00:00
Ed Bartosh
5651e8da60 wic: fix parsing of 'bitbake -e' output
Current parsing code can wrongly interpret arbitrary lines
that are of 'key=value' format as legitimate bitbake variables.

Implemented more strict parsing of key=value pairs using
regular expressions.

(From OE-Core rev: f0ec387ad40fb9c098ac8d761993bc2bacc76e65)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:12 +00:00
Ed Bartosh
a49d279b50 wic: add bmaptool to the list of utilities
Added bmaptool -> bmap-tools pair to the dictionary
executable -> recipe as bmaptool is going to be used by wic
to generate .bmap file.

[YOCTO #9413]

(From OE-Core rev: 4684ac5f4d823a6afedc57aad0aa417bb17ba950)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-19 09:05:19 +01:00
Ed Bartosh
bc89dc4225 wic: use find_executable in favor of bb.utils.which
As bitbake is not ported to Python 3 yet it's better to
avoid using its APIs as much as possible to be able to
test wic under Python 3 at least partially.

Used distutils.spawn.find_executable API in favor of
bb.utils.which to get path of the command to run.

(From OE-Core rev: 9658956bf8a5da779e06f71941de9b3e89415cdc)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-14 23:05:14 +01:00
Ed Bartosh
0c57dd96c8 wic: replace print statements with print function
Print statements have been replaced with print function in
Python 3. Replaced them in wic code to be able to run it
under both Python 2 and Python 3.

[YOCTO #9412]

(From OE-Core rev: ee6979a19c77931c3cf6368e695e370d46192fef)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-14 23:05:13 +01:00
Ed Bartosh
e6a523286e wic: add sfdisk to the list of utilities
Added sfdisk -> util-linux pair to the dictionary
executable -> recipe as sfdisk is going to be used by wic
to set partition system id.

[YOCTO #9096]

(From OE-Core rev: 398aafa185acbc7239505f7107735e93a502f6d2)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-29 07:58:44 +01:00
Ed Bartosh
67af6d6402 wic: exec_native_cmd: implement support for pseudo
Wic runs some tools through pseudo, which makes exec_native_cmd
to fail and throw cryptic error message when tool is not baked:
For example:
  Error: exec_cmd: 'export
  PSEUDO_PREFIX=/media/ssd/poky-build/tmp/sysroots/x86_64-linux/usr;export
  PSEUDO_LOCALSTATEDIR=/media/ssd/poky-build/tmp/work/qemux86-poky-linux/
...
  PSEUDO_PASSWD=/media/ssd/poky-build/tmp/work/qemux86-poky-linux/ ...
  PSEUDO_NOSYMLINKEXP=1;/media/ssd/poky-build/tmp/sysroots/ ...
  mkfs.ext4 -F -i 8192 /var/tmp/wic/build/rootfs_platform.7.ext4 -L
  platform -d
  /media/ssd/poky-build/tmp/work/qemux86-poky-linux/core-image-minimal/...
  returned '1' instead of 0

Made exec_native_cmd aware of pseudo and properly report
errors when command is not found.

(From OE-Core rev: 04bab58809c63c9114feb0aadc9b6115be10fcc4)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:02 +00:00
Randy Witt
0f75740830 wic/utils/oe/misc.py: Preserve PATH when running native tools
Previously exec_native_cmd() would remove all items from PATH except for
the native sysroot. This can cause issues for the tools that are created
using create_wrapper().

Now instead of wiping out the PATH, run a sanity check to check if the
command is in the native sysroot.

(From OE-Core rev: ba127370e621b5b683d6f454596c3d0c60c13df7)

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-27 07:24:29 +00:00
Ed Bartosh
2e2ac0cb7a wic: fix typo
Fixed typo in tool name: mkswqp -> mkswap.

(From OE-Core rev: e21d4b442437fac450070922ec74029ee1d10208)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-03 12:43:26 +01:00
Ed Bartosh
5856f8ff96 wic: implement getting variables from .env files
Added functionality of getting variables from <image>.env files to
BitbakeVars class. env files will be parsed if the directory with
env files is known, i.e. when vars_dir attribute is set.
Otherwise 'bitbake -e' output will be parsed.

(From OE-Core rev: d21e4c1e56cab750ed4f6031d7f3dc5775a2c2cc)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-30 21:38:22 +01:00
Ed Bartosh
95c6749af8 wic: add default_image attribute to BitbakeVars
New attribute is used when bitbake variable is requested without
specifying image name. The attribute should be set from outside,
for example when wic is called with '-e <image>' option.

(From OE-Core rev: 38d0b3744b5829333148ac4fc97dc45a8d0ba3a5)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-30 21:38:21 +01:00
Ed Bartosh
e1fe3479a6 wic: create new method _parse_line
Moved code that parses one line of 'bitbake -e' output
to separate method _parse_line.

This method will be also used later to parse lines of .env files.

(From OE-Core rev: 49ef04d3c9eeb76cbbc89b27b4dd1570b7a2552b)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-30 21:38:21 +01:00
Ed Bartosh
58c393022f wic: add BitbakeVars class
Moved code of getting bitbake variables into separate class.

Created singleton object of this class in the module namespace.

Preserved existing API get_bitbake_var.

(From OE-Core rev: 3229d37993e315c9ca1902849746b9f50f35845c)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-30 21:38:21 +01:00
Ed Bartosh
b41de121c6 wic: remove undescore from function name
Renamed __exec_cmd -> _exec_cmd as double underscores cause
strange behaviour when function is called in class method.
Python complains that __exec_cmd method(!!!) of the same class
doesn't exist.

(From OE-Core rev: 3b400a06d755e3d437967f60842b1d9bd94f0814)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-30 12:36:08 +01:00
Ed Bartosh
8ce8328901 wic: code cleanup: pylint misc.py
Fixed pylint warnings.
Increased pylint score from 8.02 to 9.40.

(From OE-Core rev: 26d8c70fb8a7cc8f6473ad1779b20b00616740c0)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-09 00:14:02 -07:00
Ed Bartosh
8421e0e29f wic: code cleanup: remove unused code
Removed unused global variable wks_vars and 2 unused functions
get_wks_var and add_wks_var.

(From OE-Core rev: 7d87c821da6a5020e8dde3f1907fb8d6a023b110)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-09 00:14:01 -07:00
Ed Bartosh
87a846d588 wic: Report recipe name for native commands
exec_native_cmd was modified to report recipe to build
native programs.

Pairs executable->recipe are hardcoded as it's not possible
to obtain this information automatically.

[YOCTO #7631]

(From OE-Core rev: 1274379c91ee8e2fb9fbb34a6445cd5767eb4a35)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-09 00:14:01 -07:00
Ed Bartosh
8f8d3362b8 wic: Set default set of bitbake variables
Set default set of bitbake variables to the set of variables
for the first parsed image.

This allows wic to find proper bitbake varibale values
if it's called with '-e <image>' even without specifying
image in the call of get_bitbake_var.

(From OE-Core rev: 18cc6d2ec4dc289bb0333dddc96df5a645ea53d0)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-10 13:35:40 +01:00
Ed Bartosh
992f78939e wic: Turn off debug output for 'bitbake -e'
Switched debug level to 'normal' to prevent huge 'bitbake -e'
output to go into wic debug output. This should help to make
wic debug info much more clean and easier to read.

(From OE-Core rev: 71510c32d78ba24bf1172548f8eb4adfe621d2de)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-02 23:08:36 +01:00
Ed Bartosh
aa470a1fcd wic: Refactor getting bitbake variables
Wic gets bitbake variables by parsing output of 'bitbake -e' command.

This implementation improves this procedure as it runs 'bitbake -e' only
when API is called and does it only once, i.e. in a "lazy" way. As parsing
results are cached 'bitbake -e' is run only once and results are parsed
only once per requested set of variables.

get_bitbake_var became the only API call. It replaces find_artifacts,
find_artifact, find_bitbake_env_lines, get_bitbake_env_lines,
set_bitbake_env_lines and get_line_val calls making API much more clear.

(From OE-Core rev: 3abe23bd217315246ec2d98dc9c390b85cfe6a92)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-02 23:08:36 +01:00
Ed Bartosh
a4da21d92f wic: code cleanup: superfluous-parens
Removed unncecessary parents after 'if' 'del' and 'print' keywords.
Fixed pyling warning: Unnecessary parens after 'xxx' keyword

(From OE-Core rev: a64604d11f75973b4c2347fa2669da9889e44013)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23 11:47:36 +01:00
Ed Bartosh
dec9156ce8 wic: move checks to exec_native_cmd
Checked for return code and output of native commands
inside exec_native_cmd.

Removed similar code from a lot of places where
exec_native_cmd is called.

(From OE-Core rev: 9e3e933321d58c04619a585326fb291dbf2748f5)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-11 23:59:11 +01:00
Ed Bartosh
77e9243210 wic: removed exec_cmd_quiet and exec_native_cmd_quiet
These functions are not used anywhere.

(From OE-Core rev: 7467fd446d08704881325577b7035b6128db6151)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-11 23:59:11 +01:00
Ed Bartosh
e9fcc57071 wic: Print bitbake error output
Added bitbake error output to wic error message to make it
more descriptive.

[YOCTO #6911]

(From OE-Core rev: f68576d4c9cecbe7122325d7fc78c9b27ff07a8e)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-14 12:38:50 +01:00
Ed Bartosh
d0d0ab85c4 wic:code cleanup: No space allowed
Fixed pylint warning
  'No space allowed around keyword argument assignment'

(From OE-Core rev: e07dd9b9c71960fbeded162ed52fbce06de620e9)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-13 22:33:20 +01:00
Ed Bartosh
16d8c895a4 wic: make error message more informative
Included name of the missing program into the error message produced
by exec_native_cmd when program is not found.

Removed mentioning of the host programs as this API is not running
them anymore.

(From OE-Core rev: 7d1599cf6a55d3624d2b90c1f55e14c4df73e570)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-10 18:10:26 +01:00
Ed Bartosh
024514037d wic: don't use host paths when looking for native tools
exec_native_cmd should not use host paths in $PATH to
avoid finding and using host tools.

[YOCTO: #6204]

(From OE-Core rev: fa263f238bbddb00c9953994fb69cc358170e2ec)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-10 18:10:26 +01:00
Alexandre Belloni
cf3e21e71e wic: allow to configure overhead factor per partition
Introduce a new option --overhead-factor to replace
IMAGE_OVERHEAD_FACTOR.

(From OE-Core rev: 20fe0c7202724187dbe80eb2101d8ef69e86b94e)

Signed-off-by: Alexandre Belloni <alexandre.belloni@free-electrons.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-14 08:40:58 +00:00
Alexandre Belloni
d84742c39f wic: allow to configure extra space per partition
Introduce a new option --extra-space instead of using IMAGE_EXTRA_SPACE.
This is useful for boot partitions where the extra space is often
useless or for huge partition where 10MiB may not be enough.

(From OE-Core rev: 9f7fe71a10bcdd1864d2f838f3510e96810ef42e)

Signed-off-by: Alexandre Belloni <alexandre.belloni@free-electrons.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-14 08:40:58 +00:00
Tom Zanussi
005d81da2b wic: Use overhead factor when creating partitions from rootfs directories
When creating partitions sized to given rootfs directories, filesystem
creation could fail in cases where the calculated target partition
size was too small to contain the filesystem created using mkfs.  This
occurred in particular when creating partitions to contain very large
filesystems such as those containing sdk image artifacts.

This same limition is present in the oe-core image creation classes,
which can be readily see by changing IMAGE_OVERHEAD_FACTOR from the
default 1.3 to 1.0 and building a sato-sdk image.

It should be possible to calculate required sizes exactly given the
source rootfs and target filesystem types, but for now, to address the
specific problem users are hitting in such situations, we'll just do
exactly what oe-core does and define and use an IMAGE_OVERHEAD_FACTOR
or 1.3 in those cases.

Fixes [YOCTO #6863].

(From OE-Core rev: bbaef3ff5833fc1d97b7b028d7770834f62789da)

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-10-30 13:03:40 +00:00
Tom Zanussi
ef700ea11f wic: Add utility function for parsing sourceparams
Parses strings of the form key1=val1[,key2=val2,...] and returns a
dict.  Also accepts valueless keys i.e. without =.

(From OE-Core rev: 36f258ee6e60c26fd44b9bc71c318363cec71f42)

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-15 18:21:52 +01:00
Tom Zanussi
d8f9d05bae wic: Rename /mic to /wic
As well as any other stray instances of mic in the codebase that can
be removed.

We don't really need to carry around legacy naming, and the history is
in git.

(From OE-Core rev: 598b120406dc1d2b7e377bd1ab6f0acbef034b22)

Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-11 10:53:12 +01:00