When copying the package database files for the debugfs, add individual
file copy as well as tree copying. After the debug rootfs has been
created, cleanup the package files.
This then allows us to avoid a problem where (for rpm at least)
extraneous files in the debug rootfs would cause failures during
oe-selftest because some files existed in both regular and debugfs
images.
(From OE-Core rev: ce49ea435ce55eb5b6da442c12e03a806534c38d)
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The _exec_shell_cmd function is used by a couple of other functions that
are ultimatelly called from the create_rootfs function. The latter is
used in image bbclass' do_rootfs which is running using the fakeroot
support in bitbake. This makes the fakeroot support in _exec_shell_cmd
redundant and never actually used.
(From OE-Core rev: d1439c41049e3cf201939bbc2b1fee58a46137a1)
Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We run depmod (through depmodwrapper) at the end of the rootfs
generation process. This part of the process assumes in its current
implementation that the kernel package name is always 'kernel' and that
there is only one set of kernel modules for which we need to generate
the modules.dep and map files.
The kernel package name can be configured via a variable
(KERNEL_PACKAGE_NAME) and becomes a namespace that enables the build
system to deal with multiple compiled kernel recipes. This patch checks
for all the depmod pkgdata and runs depmod for each of the detected
kernel versions/kernel package name.
(From OE-Core rev: efa88e1c227d695319197f511701e0230d301f39)
Signed-off-by: Andrei Gherzan <andrei.gherzan@huawei.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Theres not a rootfs for baremetal images, hence we should avoid
trying to list rootfs packages for them.
This fixes an issue where some classes (e.g. license_image) rely on
rootfs functionality when included for baremetal images even if its
nonexistent
(From OE-Core rev: 5a7e13f46083ce3b08aa762238c1e93b7626dda4)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandro@enedino.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Where there isn't a copyright statement, add one to make it explicit.
Also add license identifiers as MIT if there isn't one.
(From OE-Core rev: bb731d1f3d2a1d50ec0aed864dbca54cf795b040)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With CONFIG_MODULE_COMPRESS_ZSTD enabled, kernel modules will have a
.ko.zst extension. This fixes depmod not being run.
Fixes: 1b696a45ddb ("rootfs.py: Add check for kernel modules before running depmod")
(From OE-Core rev: 425efac7110f0f42d70643e0a448e834d0f01a7a)
Signed-off-by: Sean Anderson <sean.anderson@seco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add ${IMAGE_ROOTFS} and ${IMGDEPLOYDIR} to do_rootfs[dirs] and
do_rootfs[cleandirs], this ensures do_rootfs run from a clean
workspace, with this change, we can now remove two bb.utils.mkdirhier
lines from meta/lib/oe/rootfs.py.
(From OE-Core rev: cb8b6f7eee4e059bb311330c57068e11bc477366)
Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Repo-wide replacement to use newer variable to represent systemd
system unitdir directory.
(From OE-Core rev: 5ace3ada5c54500c71becc8e0c6eddeb8bc053e3)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removes the /var/cache/ldconfig auxiliary cache directory from
the rootfs when:
- read-only-rootfs is in IMAGE_FEATURES,
- ldconfig is not in DISTRO_FEATURES.
In both cases the /var/cache/ldconfig/aux-cache is useless.
(From OE-Core rev: e7a18904582b27fd12a437ff4d8d25c625af7696)
Signed-off-by: Damian Wrobel <dwrobel@ertelnet.rybnik.pl>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(From OE-Core rev: 2621dbbc1181808f18ca4ae79408d0d5b557670f)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Incremental build in Docker fails with:
OSError: [Errno 18] Invalid cross-device link
when source and destination are on different overlay filesystems.
Rather than adding fallback code to every call site, use a new wrapper
in bitbake which detects this case and falls back to shutil.move
which is slower but will handtle the overlay docker filesystems correctly.
[YOCTO #14301]
(From OE-Core rev: 656a65b2b84e7d529b89cf5de7eb838f902d84a2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* with xz PACKAGECONFIG enabled in kmod and xz module compression enabled in kernel
the do_rootfs task doesn't run depmod in the image, because it thinks there are no modules:
NOTE: No Kernel Modules found, not running depmod
(From OE-Core rev: 9c13ce05eae0f126eb150e48709e9bd06e9280fa)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Christophe Chapuis <chris.chapuis@gmail.com>
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The run-postinsts package runs post installation scripts
on target if packages request delayed post installations. When
no delayed post installations are found the sysV style scripts
are disabled for the package and hence it did not run on sysV
based systems. However, the package provides systemd service
as well which still ran on systems based on systemd even when
no post installations were found.
Rather than disabling/masking scripts for different initialization
managers we now simply remove/uninstall the run-postinsts package
when no post installations are found to be delayed till runtime.
This is also more aligned with the function (_uninstall_unneeded)
this functionality is triggered through.
(From OE-Core rev: 627fb3181edd71502fbdf96549c41b2dea027250)
Signed-off-by: Awais Belal <awais_belal@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Example use case in OpenBMC: rootfs is squashfs and the system has either
overlayfs for whole rootfs or for some parts (e.g. /etc).
This option will allow to create migration one-shot postinsts using
"pkg_postinst_ontarget_${PN}" routines defined in recipes to fix
files under upper workdir in overlayfs.
(From OE-Core rev: 0977204e16279b117811b5d5cdac5918287e95ac)
Signed-off-by: Anton D. Kachalov <rnouse@google.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Dynamic loading of package managers will allow other layers to simply
add their package manager code in package_manager/ and have bitbake find
it according to the package manager configuration. This is useful for
adding new (faster) package managers to Open Embedded while not increasing the
test scope or require Open Embedded to support more package managers.
How this is tested:
* Build core-image-minimal with all three package managers
* Build the sdk with all three package managers. dpkg fails, but
it fails on master as well.
* Run the complete test suite, all tests passed except 16
* Run those 16 tests on master and verify that they fail there as well
* Fix errors making tests works on master but not with this patch.
(From OE-Core rev: 02670501dea192879ddf9f8048eea57a94719fc1)
Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using rpm as package manager and trying to install a file called
'/usr/share/doc/What to when an Error occurs.txt'
log_check falsely errors out on the build, because used regex match on
'DEBUG: Removing manifest: /path/usr/share/doc/What
to when an Error occurs.txt'.
To handle such cases introduce IMAGE_LOG_CHECK_EXCLUDES, to allow
user to add custom exclude regex to log_check exclude list
(From OE-Core rev: 7c7555a79b8bdef0a8d7fbd57e9ddf059066de76)
Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.
RP: Fixes to parse/build
(From OE-Core rev: 510d5c48c0496f23a3d7aede76ea8735da2d371d)
Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.
RP: Fixes to parse/build
(From OE-Core rev: 3ef5a3c885e1010cddfe7eba1cd3728f15270d78)
Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.
RP: Fixes to parse/build
(From OE-Core rev: 8b776ed9ed291dd8e112621561762449c7eb5ee2)
Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.
(From OE-Core rev: abadf053371ce863bf21b4a9474eb61761545de1)
Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.
RP: Fixes to parse/build
(From OE-Core rev: 67fa086589bae484a9beca50b627b007766dcb93)
Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.
(From OE-Core rev: e4f07444276a88f336d5fe5e60688fcfb2f22df7)
Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.
(From OE-Core rev: f8ee8bc737a982001b9fd0ad441495a52f12e9b4)
Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.
(From OE-Core rev: 405cd8560fed2e05fc82919d728c42516793cc0f)
Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a part of a refactor that will split the package manager
code so that it's possible to use other package managers in other
layers.
(From OE-Core rev: 87a1c8ee406f73e53888df3b682e8a5f0f610c2f)
Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds -X option to ldconfig to not create symlinks
as part of the ldconfig cache creation process.
It is much better to keep all needed symlinks as part of
of the package then let ldconfig do the job.
In the case symlink was created by ldconfig and later on the
package got removed the dangling link will remain on the
filesystem. Not to mention that such a symlink is orphaned
(e.g. rpm -qf <file-path> will not show the package the symlink
is part of).
It also align the behaviour with libc-musl where the ldconfig
is not invoked by default.
(From OE-Core rev: ad4ff7c18cc6a5c15a0525149dbe2dbd5ae39129)
Signed-off-by: Damian Wrobel <dwrobel@ertelnet.rybnik.pl>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are cases where one doesn't want ldconfig on target (e.g. for
read-only root filesystems, it's rather pointless), yet one still
needs ld.so.conf to be present at image build time:
When some recipe installs libraries to a non-standard location, and
dutifully drops in a file in /etc/ld.so.conf.d/foo.conf, we need the
ld.so.conf containing the
include /etc/ld.so.conf.d/*.conf
stanza to get those other locations picked up.
So change the packaging logic so that there's always an ld.so.conf
present when the build-time ldconfig runs.
The ld.so.conf and ld.so.conf.d/*.conf files don't take up much
room (at least not compared to the 700K binary ldconfig), and they
might be needed in case ldconfig is installable, so leave them
alone.
In case of a read-only rootfs, one could add some logic to remove them
if one really wants to shave those few dozens of bytes off.
While here, fix typos in the bb.note (add spaces) so one can just
copy-paste the line from the log-file and redo the command.
(From OE-Core rev: a4cdda012f613d8d80203b9f5fc737d8511d16ce)
Signed-off-by: Rasmus Villemoes <rasmus.villemoes@prevas.dk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Unless cachedpath is used correctly then it's just a glorified clone of
os.walk, but without any of the recent optimisations in os.walk. In
this codepath there is no point to using cachedpath.
(From OE-Core rev: 765e0c3410d6bf3b4a50c4a036f555eae760acd2)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds SPDX license headers in place of the wide assortment of things
currently in our script headers. We default to GPL-2.0-only except for the
oeqa code where it was clearly submitted and marked as MIT on the most part
or some scripts which had the "or later" GPL versioning.
The patch also drops other obsolete bits of file headers where they were
encoountered such as editor modelines, obsolete maintainer information or
the phrase "All rights reserved" which is now obsolete and not required in
copyright headers (in this case its actually confusing for licensing as all
rights were not reserved).
More work is needed for OE-Core but this takes care of the bulk of the scripts
and meta/lib directories.
The top level LICENSE files are tweaked to match the new structure and the
SPDX naming.
(From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, BAD_RECOMMENDATIONS on the opkg backed relies on editing the
opkg status file (it sets BAD_RECOMMENDATIONS pkg want state to
deinstalled and pinned). This is brittle, and not consistent across the
different solver backends. Use new --add-ignore-recommends flag instead.
(From OE-Core rev: 0d11e813ba9b4e8de9e6e5099ff85f5d914243bc)
Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use prelink only when image-prelink class is inherited. Inheriting this
class adds prelink-native as dependency to do_rootfs, which makes prelink
binary and configuration available.
|Exception: FileNotFoundError: [Errno 2] No such file or directory:
|'/home/.../1.0-r0/recipe-sysroot-native/etc/prelink.conf'
|
| if not os.path.exists(prelink_cfg):
| shutil.copy(self.d.expand('${STAGING_DIR_NATIVE}${sysconfdir_native}/prelink.conf'),prelink_cfg)
(From OE-Core rev: 9e30e72920b3c661279499f2e94d5dc0087a1a8e)
Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix handling of escape characters in regexs and hence fix python
Deprecation warnings which will be problematic in python 3.8.
Note that some show up as:
"""
meta/classes/package.bbclass:1293: DeprecationWarning: invalid escape sequence \.
"""
where the problem isn't on 1293 in package.bbclass but in some _prepend to a
package.bbclass function in a different file like mesa.inc, often from
do_package_split() calls.
(From OE-Core rev: 4b1c0c7d5525fc4cea9e0f02ec54e92a6fbc6199)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
While parsing opkg package status, last package status was not
properly handled, resulting in final image without postinst and
pkg depends
(From OE-Core rev: 0d3ca08347eb0c8b9615a0197c213a32f52033c8)
Signed-off-by: Raul Martins <raul.martins@alta-rt.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Static libraries are not included in rootfs, it means, that sources are
not going into debug rootfs. This option enables to install additional
packages even if the standard package is not installed.
(From OE-Core rev: 708fe1df3d3e22dd693ae7bbfdd3e5af1a1b0bdc)
Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the source packages are created, they need to be included in the
debug image created when IMAGE_GEN_DEBUGFS = "1" for it to be usable for
debugging.
[YOCTO #12930]
(From OE-Core rev: 5fc7fbc10bbd3a54df16db275ecd6080bc58f19e)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous logic assumed that if $BUILD_IMAGES_FROM_FEEDS=1 then a
complete set of ipk feeds from which to build the image is already
present under $IPK_FEED_URIS at do_rootfs runtime.
$IPK_FEED_URIS usually contains "file://${DEPLOY_DIR_IPK}" which
renders the above assumption bad because some recipes in the current
build can contain code like do_install[nostamp] = "1" which will cause
rebuilds bumping $PR and invalidating the index.
Even when the index is manually re-created before an image build
("bitbake package-index"), the nostamp will cause failures because the
dependency gets rebuilt before do_rootfs in the "bitbake <image>" call.
So make the opkg rootfs index logic the same as for rpm/deb, to always
update the index in $DEPLOY_DIR_IPK to fix the above nostamp failure.
Feeds outside $DEPLOY_DIR_IPK added to $IPK_FEED_URIS continue to work
as usual, for eg. by using a http:// URI.
(From OE-Core rev: bce90f48d1cc136fdfdf98b3830f5d99e381271b)
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Similar to rpm, use copies of the ipk/deb directories for rootfs construction.
This means the image creation code can no longer "see" recipes wich aren't in its
dependency chain which is good for a variety of reasons including determinism,
incompatible recipe (e.g. systemd/sysvinit) package conflicts and locking
performance.
(From OE-Core rev: c7c5f4065c102fde4e11d138fb0b6e25bffe0379)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If package management is present opkg/dpkg will bring the original
copy of the postinsts scripts with the metadata and will be able to
handle postinsts just fine. In fact, it is preferred to let package
management handle the postinsts scripts in this case since it will
keep the package managers database up-to-date too. The run-postinsts
scripts will make sure the package manager gets invoked instead of
the scripts directly.
Note: Before commit 5aae19959a44 ("rootfs.py: Change logic to
unistall packages") rootfs.py did not install /etc/$pm-postinsts
too. It is not clear whether that change was intentionally or just
a bug. This commit fixes/reverts that aspect of the commit.
(From OE-Core rev: 4b571c59e3e6bd8b22a63f547a95757aeba5b638)
Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If package management has been disabled then we've already removed all the
state, and running 'dnf clean all' again will simply recreate a lot of the
files.
(From OE-Core rev: 4524068ad2248b37fb08a24828d018e2f7e6a761)
(From OE-Core rev: f23332dc6a0c5b49246b4d17ec4e6fcad0cb7d47)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows running the intercepts when creating SDKs, which previously
wasn't possible, as SDK code does not use the rootfs class, and calls
into PackageManager methods directly.
(From OE-Core rev: f830388c5e9125f385a42acd7365d1235967b57c)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will allow handling postinst_intercepts when populating SDKs (which
use PackageManager class directly, and do not utilize RootFS class).
(From OE-Core rev: 9454fd328040fd58c981d028a74fcf181bde8e89)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that the first boot deferral needs to be requested explicitly,
it's not really something to be concerned about.
(From OE-Core rev: 19f3f4c60060ff5ed3c1c9871e18fcd4d128c5a3)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
That hook is empty, and doesn't need to be executed; it merely indicates that packages
that have used it are requesting to defer their postinst scripts to first boot
unconditionally.
(From OE-Core rev: 939f7f1a06cd2db05aeb5e75a66322314e10aa6d)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With "read-only-rootfs" in IMAGE_FEATURES, packages in ROOTFS_RO_UNNEEDED
are removed when building the rootfs. The list of packages to remove is
passed to the package manager and the list is sorted so that
update-alternatives provider is the last entry. This is with the
assumption that the last entry on the list/command line is removed last.
However, it turns out rpm does not care about "last on the command
line" and update-alternatives provider is removed before other the
packages get to run their %preun scripts for update-alternatives.
This leaves broken alternative symlinks in rootfs.
The fix is to first remove all but update-alternatives provider and
after that update-alternatives provider in its own remove() call.
(From OE-Core rev: 0a8639593c11ef0cfb3a3a514b17e36318b8e8f2)
Signed-off-by: Mikko Ylinen <mikko.ylinen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It contains cached metadata for a transient repository that is used
only when creating images on the host, and so is of no use on target
images. Dnf will recreate the cache on target when needed.
(From OE-Core rev: 89ddf2516b713e6622df95ab391423b033d4271a)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure we capture stderr, always write the output to the log, and don't be so
emotional in the warning message.
(From OE-Core rev: 87110e971c548d9d3eb7296ea6772d18e0226e53)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* when OPKGLIBDIR doesn't have the default /var/lib value it will
silently fail to copy package database from normal rootfs to debugfs
rootfs and then when trying to install *-dbg complimentary packages
it won't install anything, because installed_pkgs.txt file generated
from debugfs is empty
(From OE-Core rev: bebdb056c8bd0efc056f07b275a154e5d5a3aa2b)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
