Commit Graph

330 Commits

Author SHA1 Message Date
Mike Crowe
ecd8dad98f image.bbclass: Only append to IMAGE_LINK_NAME if it was already set
create_symlinks does not create any links if IMAGE_LINK_NAME is empty.
Unfortunately, setup_debugfs_variables unconditionally appends '-dbg' which
results in a previously-empty IMAGE_LINK_NAME containing just '-dbg'. Let's
check that it's not empty before appending.

(From OE-Core rev: e529c45f29bd9a1de21f31fef7acb23eb6e8ebdd)

Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-10 09:56:22 +01:00
Chen Qi
da263d9c7a image.bbclass: fix systemd_preset_all
Check the existence of systemd before using systemctl to preset units.
This is because even if 'systemd' is in DISTRO_FEATURES, it's possible
that systemd is not even installed. e.g. container-test-image in
meta-selftest layer.

As systemd DEPENDS on systemd-systemctl-native, the existence of systemd
also ensures the existence of systemd-systemctl-native.

This would fix the following test case when using systemd as the init
manager.

  containerimage.ContainerImageTests.test_expected_files

Also remove the IMAGE_EXTRADEPENDS setting, as nothing references this
variable.

(From OE-Core rev: c9854a4ab6af9e60b1a588a87b9a062624af6fae)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-02 08:13:07 +01:00
Alex Kiernan
7e5124a44c systemd: Default to non-stateless images
When creating images, for anything other than the explicitly stateless
case, touch /etc/machine-id so that the images can be booted without an
initramfs and with `ro` set on the kernel command line, otherwise system
refuses to start:

  [    7.222134] systemd[1]: No hostname configured.
  [    7.227266] systemd[1]: Set hostname to <localhost>.
  [    7.232622] systemd[1]: System cannot boot: Missing /etc/machine-id and /etc is mounted read-only.
  [    7.241750] systemd[1]: Booting up is supported only when:
  [    7.247362] systemd[1]: 1) /etc/machine-id exists and is populated.
  [    7.253752] systemd[1]: 2) /etc/machine-id exists and is empty.
  [    7.259757] systemd[1]: 3) /etc/machine-id is missing and /etc is writable.

If IMAGE_FEATURES includes `stateless-rootfs` then systemctl-native is
not run on the image leaving the image for population at runtime by
systemd.

(From OE-Core rev: c5fb399f5894c16cf8eeadd507dc38c29b0fd657)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-09 16:31:55 +01:00
Alex Kiernan
665fd0ae23 image: call systemctl preset-all for images
Rather than rely on systemd's default invocation of preset-all at
runtime, we pre-populate the symlink tree as part of of the image. This
is done late so any overrides of presets during rootfs construction
should already have happened.

Whilst we don't strictly need this for the read-write root case, it
avoids boot time churn; for read-only root we have to do it here.

(From OE-Core rev: a57678a076109c42fbdf7531e90ff4491b387d53)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-03 06:11:57 +01:00
Khem Raj
9d973d3b4e default-distrovars: Drop DISTRO_FEATURES_LIBC
After eglibc was merged into glibc, Kconfig support was also dropped so
these libc features therefore are not effective anymore and can be
removed

(From OE-Core rev: c62b1cc06613a4cdddf53290e6203559f43fc62d)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-28 13:21:54 +00:00
Kai Kang
3a439047bf image.bbclass: respect PACKAGE_NO_GCONV
It installs locale-base-* packages according to var IMAGE_LINGUAS.
Packages locale-base-* are split in libc-package.bbclass if variable
PACKAGE_NO_GCONV is not set. When none of ditro features libc-charsets
libc-locales and libc-locale-code is set, PACKAGE_NO_GCONV is set. Then
no locale-base-* is created and fails to create image.

Clear IMAGE_LINGUAS in such situation.

(From OE-Core rev: 85240094175a8ea726bfba19c00d4556a62862fc)

Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-05 22:30:19 +00:00
Ming Liu
12e30bd897 image.bbclass: fix a wrong position blank
A flaw was introduced by commit c5fa6034:
[ image.bbclass: use prependVarFlag for postfuncs ]

it changed to use prependVarFlag instead of appendVarFlag, then the
blank also needs change to adapt it.

(From OE-Core rev: 6085023158ffbfbaf0f3d65ef18054c003d3f463)

Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-27 22:13:11 +00:00
Ming Liu
e5a43f6386 image.bbclass: add a missing blank
When calling d.appendVarFlag, a blank is needed or else it could mess
up the later appended variables.

Reported-by: Stefan Agner <stefan.agner@toradex.com>
(From OE-Core rev: 8b2df6e5874b932b356847c5bde10d5b31d67213)

Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-16 11:46:07 +00:00
Andrej Valek
1a2ae051ae rootfs: ensure that packages in IMAGE_INSTALL_DEBUGFS are existing
Ensure that additional packages in IMAGE_INSTALL_DEBUGFS are available
before installation.

(From OE-Core rev: 06a1bbea67a7233deff80ebe3681b4e0cca6817a)

Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-14 11:14:39 +00:00
Andrej Valek
5f30534c47 rootfs: possibility to add packages to debug rootfs
Static libraries are not included in rootfs, it means, that sources are
not going into debug rootfs. This option enables to install additional
packages even if the standard package is not installed.

(From OE-Core rev: 708fe1df3d3e22dd693ae7bbfdd3e5af1a1b0bdc)

Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-18 10:56:55 +01:00
Christopher Larson
aa9d8e73bb image.bbclass: inherit and use image-postinst-intercepts
(From OE-Core rev: 73cccdb6942404961415e5939263686719b24061)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-16 09:49:39 +01:00
Alex Kiernan
7f93c8bc4a image: Use ${COREBASE}/meta for timestamp, fallback to bitbake.conf
To handle the case where ${COREBASE} isn't the git directory, avoid
erroring out when the git command fails. If we don't have a timestamp
after this, fall back to the timestamp from conf/bitbake.conf.

(From OE-Core rev: 97b439469a45a089431ca9c31893288c855045f4)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-15 21:45:58 +01:00
Richard Purdie
65c8154f21 image: Add locale archive optimisation
Refactor the locale archive function from the SDK to also make it work during
general image creation. This reduces the size of the locales from 900MB to 220MB
in core-image-lsb-sdk.

The exception handling around subprocess was dropped as the standard subprocess
exception printing is better handled than the catchall exception.

(From OE-Core rev: 8ffd93bdb09b0a4a84b27dafcd684c6abba392ed)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-15 09:44:33 +01:00
Richard Purdie
ce5cd33a8a license_image: Fix race
The current code pokes into do_deploy manifests from do_image_complete when the
do_image_complete task may or may not depend upon the do_deploy tasks in question.
Often it gets lucky, sometimes it results in build failures.

To fix this, split the functionality to its own task which can have the correct
task dependencies. This means the data in BB_TASKDEPDATA is definitive, the other
code can be dropped, as can the IMAGE_EXTRATYPES do_populate_lic dependencies
from image.bbclass.

This fixes bugs which show up as:

NOTE: recipe linux-yocto-4.14.48+gitAUTOINC+d64aec9793_97c8063d2d-r0: task do_deploy: Started
ERROR: core-image-minimal-1.0-r0 do_image_complete: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:write_deploy_manifest(d)
     0003:
File: '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/classes/license.bbclass', lineno: 33, function: write_deploy_manifest
     0029:        'w+').write(output)
     0030:}
     0031:
     0032:python write_deploy_manifest() {
 *** 0033:    license_deployed_manifest(d)
     0034:}
     0035:
     0036:python license_create_manifest() {
     0037:    import oe.packagedata
File: '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/classes/license.bbclass', lineno: 191, function: license_deployed_manifest
     0187:        # It is necessary to mark this will be used for image manifest
     0188:        man_dic[dep]["IMAGE_MANIFEST"] = True
     0189:        man_dic[dep]["PN"] = dep
     0190:        man_dic[dep]["FILES"] = \
 *** 0191:            " ".join(get_deployed_files(dep_dic[dep]))
     0192:        with open(os.path.join(lic_dir, dep, "recipeinfo"), "r") as f:
     0193:            for line in f.readlines():
     0194:                key,val = line.split(": ", 1)
     0195:                man_dic[dep][key] = val[:-1]
File: '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/classes/license.bbclass', lineno: 289, function: get_deployed_files
     0285:    """
     0286:
     0287:    dep_files = []
     0288:    excluded_files = []
 *** 0289:    with open(man_file, "r") as manifest:
     0290:        all_files = manifest.read()
     0291:    for f in all_files.splitlines():
     0292:        if ((not (os.path.islink(f) or os.path.isdir(f))) and
     0293:                not os.path.basename(f) in excluded_files):
Exception: FileNotFoundError: [Errno 2] No such file or directory: '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build-st-730/tmp/sstate-control/manifest-qemux86_64-linux-yocto.deploy'

ERROR: core-image-minimal-1.0-r0 do_image_complete: Function failed: write_deploy_manifest
ERROR: Logfile of failure stored in: /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build-st-730/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/temp/log.do_image_complete.50537
NOTE: recipe core-image-minimal-1.0-r0: task do_image_complete: Failed
ERROR: Task (/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/recipes-core/images/core-image-minimal.bb:do_image_complete) failed with exit code '1'
NOTE: recipe linux-yocto-4.14.48+gitAUTOINC+d64aec9793_97c8063d2d-r0: task do_deploy: Succeeded

(From OE-Core rev: b54cdaea7844ee3bf0c39eb97cc7c4c17ed5818c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-25 16:48:27 +01:00
Jackie Huang
7c1adcc6d7 rootfs-postcommands: split ssh_allow_empty_password
"allow root login" should not be bundled in ssh_allow_empty_password,
because some distro may want only one of "allow root login" and "allow
empty password", so split it out into ssh_allow_root_login and add new
imagefeature allow-root-login so they can be controlled separately,
debug-tweaks will still include both of them.

(From OE-Core rev: 1ab494f06a12548a902298afabd0a842161ef10d)

Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-09 22:48:38 +01:00
Alex Kiernan
3c46673100 image: Set COREBASE as the git directory for timestamp
When REPRODUCIBLE_TIMESTAMP_ROOTFS is unset and we want to parse one
from git, use COREBASE as the base for the git command so we have a
known repository which we're using. Without this the build may fail
if the current directory is not part of a git repository.

(From OE-Core rev: 1c2197f96d69547e10b74dc722d9a569d9a2b2b6)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-28 09:22:35 +01:00
Richard Purdie
fae0a9a439 image/testimage: Rework auto image test execution
The TEST_IMAGE interface has never particularly worked and image testing currently
gets configured manually.

This reworks the interface to better serve the needs of its users, replacing it
with TESTIMAGE_AUTO.

This does away with the need for the separate class due to better bitbake APIs for
changing tasks.

TESTIMAGE_AUTO will automatically boot under qemu any image that is built. It
also adds in dependencies so that any SDK for which testing is requested will
automatically be built first.

The code in bitbake.conf was error prone (e.g. testsdk wasn't considered), this
improves it to standardise on IMAGE_CLASSES as the standard configuration mechanism.

(From OE-Core rev: b34d44f3dfea8254826a46701a4fe3769a900434)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-21 09:34:40 +01:00
Richard Purdie
83d791089f image/image-live: Improve handling of live/iso/hddimg types (drop NOISO/NOHDD)
The logic can be improved and the historical NOISO/NOHDD variables
moved into the class and out of common code.

The variables are also then removed in favour of directly controlling
the behaviour from IMAGE_FSTYPES in line with all the other image types.

(From OE-Core rev: a052caed563a09a01f5a3ea1f0477f379c05bee0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-21 09:34:40 +01:00
Richard Purdie
25a2244265 image: Combine all the class handling code into a single section
Right now the code handling class inherits is spread all over and
its hard to get an idea of what is happening overall. Combine all
the code together to make it clearer. There shoould be no functionality
changes.

(From OE-Core rev: 9b6cda7ff443eebfc5a5a8c9442c93a881807dab)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-21 09:34:40 +01:00
Alexander Kanavin
bdfca89b80 image.bbclass, populate_sdk_base.bbclass: make all variants of qemuwrapper-cross available in sysroots
The variants are needed in particular when executing postinst_intercepts as
those may require running binaries built for different architectures and
against different sets of library paths, when multilib is in use (or nativesdk host
packages are installed), so a single global variant of the script was not working.

I do understand expanding PATH and DEPENDS in this manner is hackish, however
every other approach I could think of is worse.

(From OE-Core rev: 2f31eecc40ea4d0865aa28d65a0ba7d5a629393a)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-18 11:07:58 +01:00
Peter Kjellerstedt
144faeb10c classes: Use ${MACHINE_ARCH} instead of ${MACHINE} for stamp-extra-info task flag
Without this change, there will be two sstate index files in
tmp/sstate-control for any machine that contains a dash in the
name.

(From OE-Core rev: 29e7799bdb3773c40492e01448e0c614ed44583d)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-10 09:10:36 +01:00
Juro Bystricky
98e852983c classes/recipes: Use expanded BUILD_REPRODUCIBLE_BINARIES value
Replace the occurences of BUILD_REPRODUCIBLE_BINARIES with expanded
values ${BUILD_REPRODUCIBLE_BINARIES} so the variable does not need to be
exported.

(From OE-Core rev: 27f87bbc8395a2481ef808465a62d213a6b678ac)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-12 15:06:32 -07:00
Richard Purdie
49f7c2cd87 image: Remove the do_package_write_* tasks
Now we're filtering tasks in the rpm indexing code so that tasks can only
see the packages they really depend upon, having noexec package_write tasks
around is causing problems since the tasks exist but don't have manifests.

Removing the tasks entirely solves this problem and streamlines the
task execution graph too.

(From OE-Core rev: 027445cd88a4e706bdfe83bb4ff2c21e2186982a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-04 11:35:42 +00:00
Ross Burton
4ae1f081a5 classes: don't scan for CVEs in images or packagegroups
There's no point even looking in the database for these, so unset CVE_PRODUCT.

(From OE-Core rev: f47da3e91541d75e1213dd9cf1f89ed16f21141a)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-16 18:05:40 +00:00
Ross Burton
563a020870 meta: don't use deprecated functions from utils.bbclass
These functions were moved to meta/lib/oe in 2010 and the base_* functions in
utils.bbclass were intended to be a short-term compatibility layer.  They're
still used in a few places, so update the callers to use the new functions.

(From OE-Core rev: c97acbd034532895ce57c6717ed1b3ccc7900b0d)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-30 11:50:11 +00:00
Richard Purdie
9e18114927 image: Expand PV to avoid AUTOREV parsing failures
Currently, setting PV to include SRCPV for build-appliance results in:

bb.data_smart.ExpansionError: Failure expanding variable SRCPV, expression was
${@bb.fetch2.get_srcrev(d)} which triggered exception FetchError: Fetcher
failure: Fetch command export ftp_proxy="http://proxy.yocto.io:5187/"; export
FTP_PROXY="http://proxy.yocto.io:5187/"; export PATH="${TMPDIR}/work/
qemux86_64-poky-linux/build-appliance-image/fetcheravoidrecurse-r0/
recipe-sysroot-native/usr/bin/python3-native:${TMPDIR}/work/
qemux86_64-poky-linux/build-appliance-image/fetcheravoidrecurse-r0/
recipe-sysroot-native/usr/bin/python3-native:${TMPDIR}/work/
qemux86_64-poky-linux/build-appliance-image/fetcheravoidrecurse-r0/
recipe-sysroot-native/usr/bin/python3-native:/home/pokybuild/
yocto-autobuilder/yocto-worker/buildtools/build/scripts:${TMPDIR}/
work/qemux86_64-poky-linux/build-appliance-image/fetcheravoidrecurse-r0/
recipe-sysroot-native/usr/bin/x86_64-poky-linux:${TMPDIR}/work/
qemux86_64-poky-linux/build-appliance-image/fetcheravoidrecurse-r0/
recipe-sysroot/usr/bin/crossscripts:${TMPDIR}/work/qemux86_64-poky-linux/
build-appliance-image/fetcheravoidrecurse-r0/recipe-sysroot-native/usr/sbin:
${TMPDIR}/work/qemux86_64-poky-linux/build-appliance-image/
fetcheravoidrecurse-r0/recipe-sysroot-native/usr/bin:${TMPDIR}/work/
qemux86_64-poky-linux/build-appliance-image/fetcheravoidrecurse-r0/
recipe-sysroot-native/sbin:${TMPDIR}/work/qemux86_64-poky-linux/
build-appliance-image/fetcheravoidrecurse-r0/recipe-sysroot-native/bin:/home/
pokybuild/yocto-autobuilder/yocto-worker/buildtools/build/bitbake/bin:${TMPDIR}
/hosttools"; export HOME="/home/pokybuild"; git -c core.fsyncobjectfiles=0
ls-remote git://git.yoctoproject.org/poky  failed with exit code 127, output:
/bin/sh: 1: git: not found

This is because PV is being expanded when TMPDIR is unset.

Expand PV in advance to avoid this problem.

(From OE-Core rev: 9ca2fad2e569597f460e6bcbbd96077c8b8cfce9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-11 10:26:07 +00:00
Saul Wold
370483fce1 image.bbclass: Add additional bb.debug to help track 12304
We actually caught the ext4 size issue in the wild with the debug
output in the oe_mkext234fs() code, but it did not help.  What that
showed was that the get_rootfs_size was returning a default size of
8192, where as the actual rootfs was more like 10572, thus too large
to fit in the created sparse file.

This additional temporary debug code should help us determine where
the failure might be.

More debug for
[YOCTO #12304]

(From OE-Core rev: 978472c58629d1448399207873bbead96b27102e)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-12-18 18:03:58 +00:00
Ming Liu
ae1a9a87b5 image.bbclass: let do_image depend on do_populate_lic of EXTRA_IMAGEDEPENDS
The licenses of EXTRA_IMAGEDEPENDS recipes are being referenced in
image postcommand write_deploy_manifest, but a dependency is missing
between do_image and do_populate_lic of EXTRA_IMAGEDEPENDS recipes,
this leads some license files not present when write_deploy_manifest
runs, hence will cause build errors.

Fixed by letting do_image depend on do_populate_lic of
EXTRA_IMAGEDEPENDS recipes.

(From OE-Core rev: 276ff1f22d00edd6c9728e44ed5d6692a6c9e04f)

Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-08 22:24:03 +00:00
Igor Romanov
3e5cf0c92f image.bbclass: Fix 'vardepsexclude' mechanism for image_cmd_${FSTYPE}
Current mechanism doesn't allow to use any non-determenistic variable, except 'DATE' and 'DATETIME', inside IMAGE_CMD_${FSTYPE} prototype.

Passing 'vardepsexclude' values from IMAGE_CMD_${FSTYPE}, so users will be able to avoid taskhash mismatch problems.

(From OE-Core rev: cb19166c540969f56959e25c1068934bdc40b013)

Signed-off-by: Igor Romanov <i.romanov@inango-systems.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-08 22:24:03 +00:00
Gerson Fernando Budke
69e97240b3 image.bbclass: Sorted ctypes to avoid basehash error
When selected multiple subimages a similar error could happend:
  Variable do_image_cpio[subimages] value changed \
    from 'cpio.gz.u-boot cpio.gz' to 'cpio.gz cpio.gz.u-boot'
To avoid this, 'ctypes' should be sorted at 'gen_conversion_cmds'.

This garantee that 'CONVERSION_CMD_xxx' are always written in tha same
order and consequently 'do_image_cpio' have the same hash.

(From OE-Core rev: 271f1a5f65b8685a1e3645026876251122ef3974)

Signed-off-by: Gerson Fernando Budke <nandojve@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21 09:24:25 +01:00
Ross Burton
dc6a45889d classes/image: move image_qa to between rootfs and image
It was noticed that do_image_qa is useless when rm_work is enabled as the rootfs
directory is deleted before image_qa is called.

This indicates that image_qa is incorrectly scheduled as it failing should mean
images don't get generated, so move it between do_rootfs and do_image.

Also, add a little bit more documentation to the comments.

(From OE-Core rev: 62ce334e583ecdf1f93619f4131c0fa5d88d5b02)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-21 09:24:25 +01:00
Ming Liu
45d7f5485a image.bbclass: drop initramfs bundle related code
The original purpose of this code snippet was to repackage initramfs
bundled kernel images before do_image_complete, to be able to be
included by rootfs, but it's not going to achieve that since the
initramfs bundled kernel images are not even installed to ${D}/boot
after commit a49569e3a7534779bbe3f01a0647fd076c95798d:
[ kernel.bbclass: do not copy bundled initramfs to /boot ]

So there is not a initramfs bundled kernel package at all, we should
drop the code, because it is leading kernel do_initramfs_bundle
unnecessarily rerun and it's very time consuming and hence is impacting
the performance a lot.

(From OE-Core rev: eca501aeb4f2cc9255fabab14c68f6910367aaf9)

Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-17 00:21:15 +01:00
Juro Bystricky
22463d2256 image.bbclass: support binary reproducibility
Added a new task "reproducible_final_image_task".
If binary reproducibility is desired ($BUILD_REPRODUCIBLE_BINARIES" = "1"),
then recursivley modify mtimes of all files to a reproducible vale.
The value is obtained via REPRODUCIBLE_TIMESTAMP_ROOTFS.
This task is executed as the very last step in image creation, once all
the files in the image have been finalized.

[YOCTO#11176]

(From OE-Core rev: df8df5f907736c3e5b7f15ea11898a1d49726ca5)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-16 00:03:15 +01:00
Stefan Agner
a3ae69755b image.bbclass: delete DATE variable too
When creating a custom image which uses the DATE variable the basehash
seems to change every day and lead to errors such as:
ERROR: console-tdx-image-2.7.6-r0 do_image_customimg: Error executing a python function in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:set_image_size(d)
...

Add DATE to the variables which should not get expanded early and to the
vardepsexclude list for the image task.

(From OE-Core rev: 4af13a4855c74cea9cf6c168fd73165d7094bf93)

Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-16 00:03:14 +01:00
Richard Purdie
a089f7b735 image: Make do_image_qa a dummy sstate task to match do_image_complete
Similar to do_image_complete, make do_image_qa a dummy sstate task
so that rm_work doesn't cause image generation to repeat on every
new build command.

(From OE-Core rev: ca61f5437cb2421a67f002785e644ba215e04aaf)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-31 15:13:31 +01:00
Richard Purdie
c6958bbd87 image/rm_work: Promote do_image_complete to be more sstate like
We relied on the missing do_image_complete_setscene task to ensure the dummy
sstate tarball that was created would never be used. This lead to its own
issues and a better fix for SSTATE_SKIP_CREATION has now been merged.

We can therefore make do_image_complete look like a more standard sstate
task which means image generation doesn't keep rerunning when using rm_work.

We do need to turn do_image_complete's stamp into an sstate version to
handle this (it otherwise matches the do_image_* glob).

(From OE-Core rev: 2ff9d40dc88d43567472218cf3d3faf414398c71)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-31 15:13:31 +01:00
Richard Purdie
7b37b0ac07 image: Drop before do_build from do_rootfs and do_image
do_image_complete has the do_build dependency, these other two tasks
don't need it. If this is present, it complicates the dependency tree
and convinces rm_work that the tasks need to rerun even if they
already did since the stamps are removed.

Drop the unneeded dependency to help resolve this issue.

(From OE-Core rev: f61bcae0926a1ad79032119cc5656c0e862b13af)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-31 15:13:31 +01:00
Tom Rini
9d07c736e3 image: Convert vmdk/vdi/qcow2 to strict CONVERSION_CMD types
The vmdk/vdi/qcow2 IMAGE_FSTYPEs predate wic.  As such, they provide
some similar underlying functionality in order to produce a "disk" image
that in turn can be converted into different formats that various
hypervisor types work with.  They do not however provide the ability for
other disk image types to be converted into these same output types.
Furthermore, they are less flexible than what wic does provide.  This
drops the old style vmdk/vdi/qcow2 types and re-introduces them under
the CONVERSION_CMD framework.  The equivalent of vmdk is now wic.vmdk
and so forth for the other types.

(From OE-Core rev: 929ba563f1bc7195c4981b8e139c432b2cc388ea)

Signed-off-by: Tom Rini <trini@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-30 08:46:20 +01:00
Tom Rini
a035d6563e image: Fix "metadata is not deterministic" when chaining 2+ CONVERSION_CMDs
When we have more than one CONVERSION_CMD being used, for example
ext4.gz.sha256sum we will see errors about "metadata is not
deterministic".  This is because we do not have a stable order of
intermediate files that will be removed in the generated shell command.
We fix this by calling sorted() on the set of rm_tmp_images so that we
will have a stable hash again.

Cc: Patrick Ohly <patrick.ohly@intel.com>
(From OE-Core rev: 98a2afeb3a53bec7a72a4a9846e1dba636cc6f3d)

Signed-off-by: Tom Rini <trini@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-25 23:15:53 +01:00
Tom Rini
e18cec750b image_types.bbclass: Make u-boot signed images more versatile
With the introduction of chaining compression/conversion support we can
convert the old image_types_uboot.bbclass code that did a hand-chaining
of a set of ${filesystem}.${compression} into generic and arbitrary
support to sign whatever the user wants to sign for their image.

This, for the record, does remove setting a valid compression type in
the record in favour of just saying none.  This is not a generally
useful feature in U-Boot and I believe being versatile in terms of being
able to pass in arbitrary compressions is more important.

(From OE-Core rev: 65f27122950a35a67ce39ae4cfe93d0dca6b0dab)

Signed-off-by: Tom Rini <trini@konsulko.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-25 15:59:13 +01:00
Tom Rini
6fbbdc626a image.bbclass: Correct chaining compression support
When chaining of compression/conversion types was added, we had a new
way to handle doing things like "ext4.bz2.sha256sum" or
"ext2.gz.u-boot".  However, because the U-Boot image class isn't
included normally, it wasn't properly converted at the time.  After the
support was added the "clean" argument that the .u-boot code uses no
longer functions.  The fix for this inadvertently broke chaining
compression/conversion.  First, correct the u-boot conversion code.

Fixes: 46bc438374de ("image.bbclass: do exact match for rootfs type")
Cc: Zhenhua Luo <zhenhua.luo@nxp.com>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: Patrick Ohly <patrick.ohly@intel.com>
(From OE-Core rev: 0a7ce0b971a208956cb895ba5a869ec8c5d94703)

Signed-off-by: Tom Rini <trini@konsulko.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-25 15:59:13 +01:00
Amarnath Valluri
96e3848ba0 image.bbclass: create root symlinks in nativesdk target sysroot
To match with the target image, the generated nativesdk's target sysroot also
should have the root symlinks(/lib*, /bin/, /sbin). So, extended the
'create_merged_usr_symlinks' to reuse symlink creation part even for nativesdk
using POPULATE_SDK_PRE_TARGET_COMMAND.

(From OE-Core rev: 2d7a58ef7a9597fde868a0582153d1f9a3007f1e)

Signed-off-by: Amarnath Valluri <amarnath.valluri@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-06 14:38:15 +01:00
Amarnath Valluri
66cf94e740 image: create symlinks needed for merged /usr
Prepare the symlinks required for merged /usr at the time of rootfs creation.

The links created in rootfs are:
/bin --> /usr/sbin
/sbin --> /usr/sbin
/lib --> /usr/lib
/lib64 --> /usr/lib64

We cannot make these symlinks as part of 'base-files' or some other package.
Because at rootfs creation, installation of the package(say kernel) that depends
on these root folders/links fails, if package manager installs this package
prior to base-files.

These symbolic links in top level folder should be present as long as
 - kernel tools use /lib/{module,firmware}
 - shell scripts uses "#!/bin/sh"

(From OE-Core rev: 3f978f24ae2cf831bd2b82270f6353edda6364a5)

Signed-off-by: Amarnath Valluri <amarnath.valluri@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-06 14:38:15 +01:00
Ed Bartosh
6fa6099b49 image.bbclass: use prependVarFlag for postfuncs
It would be possible to achieve any order of calling functions if
prefuncs are added with appendVarFlag and postfuncs with prependVarFlag.
Then image_X.bbclass can add code with either pre/post-funcs or
do_image_x_append or _prepend.

The execution order would be:

image_X prefuncs
image prefuncs
do_image_X_prepend from image_X.bbclass
do_image_X from image.bbclass
do_image_X_append from image_X.bbclass
image postfuncs
image_X postfuncs

[YOCTO #11372]

Thanks to Ola Nillsson for the idea.

(From OE-Core rev: c5fa6034708b344e184bb6fa361d5ea41371008f)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-28 20:55:08 +01:00
Richard Purdie
f09b074de8 meta: Drop further remnants of uclibc support
uclibc support was removed a while ago and musl works much better. Start to
remove the various overrides and patches related to uclibc which are no longer
needed.

uclibc support in a layer would still be possible. I have strong reasons to
believe nobody is still using uclibc since patches are missing and I doubt
the metadata even parses anymore.

(From OE-Core rev: ec03023d2165b49a52b83bac1ea2f0bfded7b852)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-22 09:16:00 +01:00
Ed Bartosh
bb8ad58dd6 image.bbclass: fix setting of vardeps flag
Added leading space to vardeps to avoid flag value to be
added to the existing value without a separator.

(From OE-Core rev: f582773c2b1e8db441f397867d3c9665fd265cec)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-14 14:53:57 +01:00
Ed Bartosh
201e4acefb image.bbclass: improve setup of flags
Replaced setVarFlag calls to appendVarFlag to allow
modification of prefuncs, postfuncs and subimages flags
in inherited image classes.

[YOCTO #11372]

(From OE-Core rev: 6690f3ab43c04fa7cff7215d4a5d8d639e41aed8)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-14 14:53:57 +01:00
Ed Bartosh
daaed7b28a image.bbclass: cleanup: add new variable "task"
Replaced repeated expression "do_image_%s" % t with
a variable 'task' to simplify the code and increase
readability.

(From OE-Core rev: d24dd95f2c1c7f773875454bee3f2016c4e7553c)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-14 14:53:57 +01:00
Pascal Bach
66ecc9354f image.bbclass: allow override of image LICENSE
Currently the LICENSE of every image is hard set to MIT.
This allows this to be overriden in derived images.

(From OE-Core rev: 62454568c12d4fd19bb69b1b679e9c7b6dc95927)

Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-16 14:08:28 +01:00
David Vincent
6ea5de4433 image.bbclass: Set ROOTFS_RO_UNNEEDED correctly
Use a weak assignment for ROOTFS_RO_UNNEEDED to let users define their
own list overriding defaults.

(From OE-Core rev: aeec0b2ccdf8566dd07961f8c4c44fcff13b70c8)

Signed-off-by: David Vincent <freesilicon@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-16 14:08:28 +01:00