Commit Graph

117 Commits

Author SHA1 Message Date
Ricardo Ribalda Delgado
931ed3e4f8 rootfs: Fix dependency for every dpkg run
Avoid getting a warning on do_rootfs.

Fixes:

NOTE: Installing the following packages: libgdk-pixbuf-2.0-locale-en-gb glibc-locale-en-gb libatspi-locale-en-gb gstreamer1.0-locale-en-gb gtk+3-locale-en libatk-1.0-locale-en-gb gtk+3-locale-en-gb gstreamer1.0-plugins-good-locale-en-gb gstreamer1.0-plugins-base-locale-en-gb gstreamer1.0-plugins-bad-locale-en-gb libglib-2.0-locale-en-gb avahi-locale-en-gb vte-locale-en-gb xkeyboard-config-locale-en-gb
WARNING: Unable to install packages. Command '/workdir/build/tmp/work/qt5122-poky-linux/bottlecam-image/1.0-r0/recipe-sysroot-native/usr/bin/apt-get  install --force-yes --allow-unauthenticated libgdk-pixbuf-2.0-locale-en-gb glibc-locale-en-gb libatspi-locale-en-gb gstreamer1.0-locale-en-gb gtk+3-locale-en libatk-1.0-locale-en-gb gtk+3-locale-en-gb gstreamer1.0-plugins-good-locale-en-gb gstreamer1.0-plugins-base-locale-en-gb gstreamer1.0-plugins-bad-locale-en-gb libglib-2.0-locale-en-gb avahi-locale-en-gb vte-locale-en-gb xkeyboard-config-locale-en-gb' returned 100:
Reading package lists...
Building dependency tree...
Reading state information...

(From OE-Core rev: 4e315dd23c52b75fd64dca89e7bffe25a0b0e6f9)

Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-19 22:13:42 +01:00
Richard Purdie
ffae400179 meta/lib+scripts: Convert to SPDX license headers
This adds SPDX license headers in place of the wide assortment of things
currently in our script headers. We default to GPL-2.0-only except for the
oeqa code where it was clearly submitted and marked as MIT on the most part
or some scripts which had the "or later" GPL versioning.

The patch also drops other obsolete bits of file headers where they were
encoountered such as editor modelines, obsolete maintainer information or
the phrase "All rights reserved" which is now obsolete and not required in
copyright headers (in this case its actually confusing for licensing as all
rights were not reserved).

More work is needed for OE-Core but this takes care of the bulk of the scripts
and meta/lib directories.

The top level LICENSE files are tweaked to match the new structure and the
SPDX naming.

(From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-09 16:31:55 +01:00
Alejandro del Castillo
d8e12f2bfc OpkgPM: use --add-ignore-recommends to process BAD_RECOMMENDATIONS
Currently, BAD_RECOMMENDATIONS on the opkg backed relies on editing the
opkg status file (it sets BAD_RECOMMENDATIONS pkg want state to
deinstalled and pinned). This is brittle, and not consistent across the
different solver backends. Use new --add-ignore-recommends flag instead.

(From OE-Core rev: 0d11e813ba9b4e8de9e6e5099ff85f5d914243bc)

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-08 10:57:19 +00:00
Andrej Valek
391d4ee036 lib/oe/rootfs: prelink only when image-prelink is inherited
Use prelink only when image-prelink class is inherited. Inheriting this
class adds prelink-native as dependency to do_rootfs, which makes prelink
binary and configuration available.

|Exception: FileNotFoundError: [Errno 2] No such file or directory:
|'/home/.../1.0-r0/recipe-sysroot-native/etc/prelink.conf'
|
| if not os.path.exists(prelink_cfg):
|     shutil.copy(self.d.expand('${STAGING_DIR_NATIVE}${sysconfdir_native}/prelink.conf'),prelink_cfg)

(From OE-Core rev: 9e30e72920b3c661279499f2e94d5dc0087a1a8e)

Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-06 08:29:06 +00:00
Richard Purdie
cd4b8a8553 meta: Fix Deprecated warnings from regexs
Fix handling of escape characters in regexs and hence fix python
Deprecation warnings which will be problematic in python 3.8.

Note that some show up as:

"""
meta/classes/package.bbclass:1293: DeprecationWarning: invalid escape sequence \.  

"""

where the problem isn't on 1293 in package.bbclass but in some _prepend to a
package.bbclass function in a different file like mesa.inc, often from
do_package_split() calls.

(From OE-Core rev: 4b1c0c7d5525fc4cea9e0f02ec54e92a6fbc6199)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-16 15:35:07 +00:00
Raul Martins
f7ef271db5 oe: Fix opkg status list parse - Missing postinst
While parsing opkg package status, last package status was not
properly handled, resulting in final image without postinst and
pkg depends

(From OE-Core rev: 0d3ca08347eb0c8b9615a0197c213a32f52033c8)

Signed-off-by: Raul Martins <raul.martins@alta-rt.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-15 17:10:52 +00:00
Andrej Valek
5f30534c47 rootfs: possibility to add packages to debug rootfs
Static libraries are not included in rootfs, it means, that sources are
not going into debug rootfs. This option enables to install additional
packages even if the standard package is not installed.

(From OE-Core rev: 708fe1df3d3e22dd693ae7bbfdd3e5af1a1b0bdc)

Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-18 10:56:55 +01:00
Joshua Watt
364c4c7d3f rootfs.py: Install -src packages when PACKAGE_DEBUG_SPLIT_STYLE is "debug-with-srcpkg"
When the source packages are created, they need to be included in the
debug image created when IMAGE_GEN_DEBUGFS = "1" for it to be usable for
debugging.

[YOCTO #12930]

(From OE-Core rev: 5fc7fbc10bbd3a54df16db275ecd6080bc58f19e)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-25 23:15:49 +01:00
Ioan-Adrian Ratiu
48ad67e485 rootfs: always update the opkg index
The previous logic assumed that if $BUILD_IMAGES_FROM_FEEDS=1 then a
complete set of ipk feeds from which to build the image is already
present under $IPK_FEED_URIS at do_rootfs runtime.

$IPK_FEED_URIS usually contains "file://${DEPLOY_DIR_IPK}" which
renders the above assumption bad because some recipes in the current
build can contain code like do_install[nostamp] = "1" which will cause
rebuilds bumping $PR and invalidating the index.

Even when the index is manually re-created before an image build
("bitbake package-index"), the nostamp will cause failures because the
dependency gets rebuilt before do_rootfs in the "bitbake <image>" call.

So make the opkg rootfs index logic the same as for rpm/deb, to always
update the index in $DEPLOY_DIR_IPK to fix the above nostamp failure.

Feeds outside $DEPLOY_DIR_IPK added to $IPK_FEED_URIS continue to work
as usual, for eg. by using a http:// URI.

(From OE-Core rev: bce90f48d1cc136fdfdf98b3830f5d99e381271b)

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-25 23:15:49 +01:00
Richard Purdie
26a786f869 package_manager/sdk: Use filtered copies of the deploy ipk/deb directories
Similar to rpm, use copies of the ipk/deb directories for rootfs construction.
This means the image creation code can no longer "see" recipes wich aren't in its
dependency chain which is good for a variety of reasons including determinism,
incompatible recipe (e.g. systemd/sysvinit) package conflicts and locking
performance.

(From OE-Core rev: c7c5f4065c102fde4e11d138fb0b6e25bffe0379)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-08-15 09:44:33 +01:00
Stefan Agner
261ee18968 rootfs.py: for dpkg/opkg, don't install postinsts if package management is present
If package management is present opkg/dpkg will bring the original
copy of the postinsts scripts with the metadata and will be able to
handle postinsts just fine. In fact, it is preferred to let package
management handle the postinsts scripts in this case since it will
keep the package managers database up-to-date too. The run-postinsts
scripts will make sure the package manager gets invoked instead of
the scripts directly.

Note: Before commit 5aae19959a44 ("rootfs.py: Change logic to
unistall packages") rootfs.py did not install /etc/$pm-postinsts
too. It is not clear whether that change was intentionally or just
a bug. This commit fixes/reverts that aspect of the commit.

(From OE-Core rev: 4b571c59e3e6bd8b22a63f547a95757aeba5b638)

Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-18 11:07:58 +01:00
Ross Burton
99745a4ddb rootfs: only use dnf to cleanup if package management is enabled
If package management has been disabled then we've already removed all the
state, and running 'dnf clean all' again will simply recreate a lot of the
files.

(From OE-Core rev: 4524068ad2248b37fb08a24828d018e2f7e6a761)

(From OE-Core rev: f23332dc6a0c5b49246b4d17ec4e6fcad0cb7d47)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-22 13:13:32 +01:00
Alexander Kanavin
ae66103939 package_manager.py: move intercept running logic from rootfs class to PackageManager class
This allows running the intercepts when creating SDKs, which previously
wasn't possible, as SDK code does not use the rootfs class, and calls
into PackageManager methods directly.

(From OE-Core rev: f830388c5e9125f385a42acd7365d1235967b57c)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 14:49:07 +01:00
Alexander Kanavin
19cd7a1776 package_manager.py: move postinst_intercept dir initialization from RootFS to PackageManager class
This will allow handling postinst_intercepts when populating SDKs (which
use PackageManager class directly, and do not utilize RootFS class).

(From OE-Core rev: 9454fd328040fd58c981d028a74fcf181bde8e89)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 14:49:07 +01:00
Alexander Kanavin
ff4835470b rootfs.py: change the notice about deferring scripts to first boot from warning to a note
Now that the first boot deferral needs to be requested explicitly,
it's not really something to be concerned about.

(From OE-Core rev: 19f3f4c60060ff5ed3c1c9871e18fcd4d128c5a3)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-30 11:50:12 +00:00
Alexander Kanavin
32c500b8ea meta/lib/oe/rootfs.py: do not execute defer_to_first_boot when processing postinst_intercept hooks
That hook is empty, and doesn't need to be executed; it merely indicates that packages
that have used it are requesting to defer their postinst scripts to first boot
unconditionally.

(From OE-Core rev: 939f7f1a06cd2db05aeb5e75a66322314e10aa6d)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-30 11:50:12 +00:00
Alexander Kanavin
4c808e31ea meta/lib/oe/rootfs.py: separate first boot deferral logic into a separate function
(From OE-Core rev: 4612291411ad788df88d5fc6dde98ff53fd91002)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-30 11:50:12 +00:00
Mikko Ylinen
3833ec59bf rootfs.py: remove update-alternatives correctly
With "read-only-rootfs" in IMAGE_FEATURES, packages in ROOTFS_RO_UNNEEDED
are removed when building the rootfs. The list of packages to remove is
passed to the package manager and the list is sorted so that
update-alternatives provider is the last entry. This is with the
assumption that the last entry on the list/command line is removed last.

However, it turns out rpm does not care about "last on the command
line" and update-alternatives provider is removed before other the
packages get to run their %preun scripts for update-alternatives.

This leaves broken alternative symlinks in rootfs.

The fix is to first remove all but update-alternatives provider and
after that update-alternatives provider in its own remove() call.

(From OE-Core rev: 0a8639593c11ef0cfb3a3a514b17e36318b8e8f2)

Signed-off-by: Mikko Ylinen <mikko.ylinen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-10-07 23:20:39 +01:00
Alexander Kanavin
141bfb844e meta/lib/oe/rootfs.py: clean up dnf cache after creating an image
It contains cached metadata for a transient repository that is used
only when creating images on the host, and so is of no use on target
images. Dnf will recreate the cache on target when needed.

(From OE-Core rev: 89ddf2516b713e6622df95ab391423b033d4271a)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-22 17:15:30 +01:00
Ross Burton
aa0c24f351 rootfs: improve postinst intercept logging
Ensure we capture stderr, always write the output to the log, and don't be so
emotional in the warning message.

(From OE-Core rev: 87110e971c548d9d3eb7296ea6772d18e0226e53)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-11 17:30:28 +01:00
Martin Jansa
da8369b670 rootfs.py: Respect OPKGLIBDIR variable
* when OPKGLIBDIR doesn't have the default /var/lib value it will
  silently fail to copy package database from normal rootfs to debugfs
  rootfs and then when trying to install *-dbg complimentary packages
  it won't install anything, because installed_pkgs.txt file generated
  from debugfs is empty

(From OE-Core rev: bebdb056c8bd0efc056f07b275a154e5d5a3aa2b)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-24 23:43:32 +00:00
Alexander Kanavin
67615e0175 rootfs_rpm.bbclass: migrate image creation to dnf
To properly look at this patch, you probably need a side-by-side diff viewing tool.

(From OE-Core rev: 65581c68d130fa74d703f6c3c92560e053857ac7)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-14 14:42:17 +00:00
Alexander Kanavin
b9c550dd6e package_manager.py: improve the API for insert_feed_uris()
No need to store the configuration as class members,
just pass it directly into the method.

(From OE-Core rev: a5cc38481be3c5e6ccbecf951f9fdc049e5101d5)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-14 14:42:17 +00:00
Mike Crowe
0b8d85a094 kernel, license, sstate, rootfs.py: Remove deploy directory README
It isn't clear that the README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt
file in the deploy directory warrants the complexity it brings elsewhere.
Let's just remove it entirely.

In particular, if two do_image_complete tasks run in parallel they risk
both trying to put their image into ${DEPLOY_DIR_IMAGE} at the same time.
Both will contain a README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt
file. In theory this should be safe because "cp -alf" will just cause one
to overwrite the other. Unfortunately, coreutils cp also has a race[1]
which means that if one copy creates the file at just the wrong point the
other will fail with:

 cp: cannot create hard link ‘..../tmp-glibc/deploy/images/pantera/README_-_DO_NOT_DELETE_FILES_IN_THIS_D.txt’ to
+‘..../tmp-glibc/work/rage_against-oe-linux-gnueabi/my-own-image/1.0-r0/deploy-my-own-image-complete/README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt’: File exists

[1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25680

(From OE-Core rev: 71e9e88847d7000781642ea6187ebd8f40dfdcfe)

Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01 11:17:45 +00:00
Richard Purdie
6ab4c5033a classes/oeqa: Replace subprocess.check_call() with check_output()
If you use subprocess.check_output() the traceback will contain the output
when the command fails which is very useful for debugging. There is no
good reason not to use this everywhere.

(From OE-Core rev: ad750dd1cc9d789abe723daddd098ce41d8547f5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 23:30:57 +00:00
André Draszik
08fd7c4fc2 lib/oe/rootfs: reliably handle alternative symlinks
When removing unneeded packages from a (read-only) rootfs
during rootfs creation, alternative symlinks from those
packages may or may not be removed.

The reason is as follows:

update-alternatives(-native) is used during package
installation as part of the image creation. It uses
a database which contains entries for all the
alternative symlinks possible, and the -native version
uses the target's database by means of $OPKG_OFFLINE_ROOT,
i.e. the rootfs we're in the process of creating.

Once the rootfs has been created, OE removes certain
packages because we have a read-only rootfs - in
particular ROOTFS_RO_UNNEEDED which includes
VIRTUAL-RUNTIME_update-alternatives, i.e. the
update-alternatives. Recently, a change was made in
OE, where uninstallation of update-alternatives from the
rootfs causes removal of its database, too, to save space
(700KiB (uncompressed) in a busybox system)
  b24a63d71b517af701dfedbc7f7b541d25af708f
  http://git.openembedded.org/openembedded-core/commit/meta/recipes-devtools/opkg-utils/opkg-utils_git.bb?id=b24a63d71b517af701dfedbc7f7b541d25af708f

Following from that, if update-alternatives is removed
from the target file system, update-alternatives-native
has no database anymore, meaning it can't manage any of
the alternative symlinks anymore.

Because the order of packages to uninstall is
non-deterministic, and update-alternatives could well
be removed before any packages that use the mechanism
provided, sometimes the extra symlinks are removed,
sometimes not.

By sorting the list of packages to be removed such that
update-alternatives is removed last, we can ensure that
that tings work reliably. (Certainly opkg seems to
uninstall packages in the order given on the command
line.)

[YOCTO #10916]

(From OE-Core rev: 5263dd3eac9d9fbdb7ef654d0cd532c192baed16)

Signed-off-by: André Draszik <adraszik@tycoint.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:14 +00:00
Ross Burton
1800b1ba7a rootfs: don't put /usr/lib/ssl and /etc into debugfs
The /etc and /usr/lib/ssl directories were only put into the opkg-generated
debugfs because of a bug in opkg which means that a conffile has to exist if
we're running 'opkg status'.  This is now fixed, so the workaround can be
reverted.

(From OE-Core rev: 7267b1f6fa25e290eac070263355aa7f30b2ebcb)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:13 +00:00
Joshua Lock
c4e2c59088 meta: remove True option to getVar calls
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.

Search made with the following regex: getVar ?\(( ?[^,()]*), True\)

(From OE-Core rev: 7c552996597faaee2fbee185b250c0ee30ea3b5f)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Paul Eggleton
af867199a5 classes/image: suppress log_check mechanism for warnings/errors logged through BitBake
If you printed a warning through bb.warn() / bbwarn or an error through
bb.error() / bberror, this was also being picked up by our log_check
mechanism that was designed to pick up warnings and errors printed by
other programs used during do_rootfs. This meant you saw not only the
warning or error itself, you saw it a second time through log_check,
which is a bit ugly. Use the just-added BB_TASK_LOGGER to access the
logger and add a handler that we can use to find out if any warning or
error we find in the logs is one we should ignore as it has already been
printed.

Fixes [YOCTO #8223].

(From OE-Core rev: fb37304d27857df3c53c0867e81fbc8899b48089)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Paul Eggleton
13916a4fab lib/oe/rootfs: fix log_check warnings being printed twice with RPM packaging
We were calling _log_check() in the RPM-specific rootfs class as well as
in the base class; this is unnecessary and resulted in any errors/warnings
generated during the actual package installation time triggering two warnings
instead of one. Drop the call from RpmRootfs._create() to fix this.

(From OE-Core rev: 541c56d755ba0354297673e857628026ad9e4df2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:49 +00:00
Ed Bartosh
9cc4492732 image: Deploy images to IMGDEPLOYDIR
Changed deployment directory from DEPLOY_DIR_IMAGE to
IMGDEPLOYDIR to make sstate machinery to do final deployment and
generate manifest.

Renamed variable deploy_dir to deploy_dir_image in selftest code
to avoid confusion with DEPLOYDIR variable.

Updated the code of rootfs.py:Rootfs class to use IMGDEPLOYDIR variable
as it's now used as a new deployment destination.

(From OE-Core rev: 6d969bacc718e21a5246d4da9bf9639dcae29b02)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-04 00:07:28 +01:00
Stephano Cetola
d45a3449bc rootfs.py: allow removal of unneeded packages
Current functionality allows for the removal of certain packages
based on the read-only image feature. This patch extends this
functionality by adding the FORCE_RO_REMOVE variable, which will
remove these packages regardless of any image features.

[ YOCTO #9491 ]

(From OE-Core rev: cfb869ffd4c37c3cc8e6b3eb732c1a7b7cfc3cb0)

Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-25 23:03:48 +01:00
Paul Eggleton
40712171b2 classes/image: implement progress support for do_rootfs
Use the new task progress functionality to report progress during
do_rootfs. This is a little coarse and ideally we would have some
progress within the installation section, but it's better than
nothing.

(From OE-Core rev: 370f08d434480c1790950e40db8f7687da78cb14)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:23 +01:00
Richard Purdie
3b39971748 classes/lib: Complete transition to python3
This patch contains all the other misc pieces of the transition to
python3 which didn't make sense to be broken into individual patches.

(From OE-Core rev: fcd6b38bab8517d83e1ed48eef1bca9a9a190f57)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:01 +01:00
Richard Purdie
44e9a0d2fa classes/lib: Update to explictly create lists where needed
Iterators now return views, not lists in python3. Where we need
lists, handle this explicitly.

(From OE-Core rev: caebd862bac7eed725e0f0321bf50793671b5312)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:00 +01:00
Anders Darander
e73a85be3e lib/oe/rootfs: Fix DEBUGFS generation, without openssl
In commit 20ea6d274bb0a9a5addb111f32793de49b907865, debugfs generation
for images using opkg, which included openssl was fixed.

However, that broke the generation of the opkg-based images, that lacks
openssl. The error is a python stack trace, showing that shutil.copytree
tries to copy a non-existing directory.

This relates to [YOCTO #9040].

(From OE-Core rev: f6b0b260ce18a30d04edfb0afb7942b9f9a5480b)

Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-20 10:20:56 +01:00
Peter Kjellerstedt
18eccfa383 rootfs.py: Unify _log_check_warn() and _log_check_error()
Use a common _log_check_common() function (based on the old
_log_check_warn() function) to implement the logic for both
_log_check_warn() and _log_check_error().

The main benefit of this is that now all error messages will be
reported again, not just the first one found. Additionally the output
will now look the same for both error and warning messages.

This removes the context for the error messages. However, since there
was no indication in the output that some of the lines were context,
they were more confusing than helping.

(From OE-Core rev: dd73dcac36b80b3b886a2e9bf575b91c4f60d039)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-19 22:32:07 +01:00
Peter Kjellerstedt
9948e0d3cb rootfs.py: Reduce spam from _log_check_warn()
For each warning found in the log, _log_check_warn() would output a
line stating that it had found a warning, then the actual warning and
finally an empty line. This is quite excessive when there are many
warnings in the log.

With this change the output is instead a line stating how many
warnings were found, followed by the warnings. This makes the output
much more compact and actually much more readable.

(From OE-Core rev: d6e3477749b1d09d40a773e0ac857a24d5851984)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-19 22:32:07 +01:00
Peter Kjellerstedt
79d177822b rootfs.py: Exclude lines in _log_check_warn() as well
This will make _log_check_warn() exclude the same lines as
_log_check_error() does.

(From OE-Core rev: 85f64c68278f797c6f73f002f63d7f46fe80aef4)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-19 22:32:07 +01:00
Peter Kjellerstedt
e3e8d500e2 rootfs.py: Use one way to exclude lines in _log_check_error()
Before there were three different ways to exclude a line from being
searched for error messages in _log_check_error(). Now there is only
one: an array of regular expressions. This should make it easy to add
more excludes if nedded.

(From OE-Core rev: 321df88088fbfa657b61b2bae32751f03daec46f)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-19 22:32:07 +01:00
Peter Kjellerstedt
a653f36814 rootfs.py: Simplify the regular expression used in _log_check_warn()
In commit 0387d095 lines with "NOTE:" in them were excluded from the
log check for warnings. However, those lines were only there in the
first place since the regular expression that is used to find warning
messages explicitly included those lines...

(From OE-Core rev: 865ab39f18a52ed84217df56d0e65113e2894d02)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-19 22:32:06 +01:00
Peter Kjellerstedt
d416a4e16e rootfs.py: Remove _log_check_error() from the RpmRootfs class
The fact that this function was overridden in the RpmRootfs class
seems to have led to a number of misstakes when changes have been made
to the base function in the Rootfs class. E.g., this change will
properly solve ticket 7789, which was supposedly solved in 38871dc0,
but that change had no effect in practice as the log_check_regex that
was modified for RpmRootfs class was not used by the RpmRootfs version
of _log_check_error()...

The only thing _log_check_error() in RpmRootfs did that the base
function in Rootfs did not do was to skip lines in the log that begin
with a + sign. This has now been moved to the base function instead.

[YOCTO #7789]

(From OE-Core rev: 1eb0a46502fca4b2ee30ccd2508f4e21a40c25ca)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-19 22:32:06 +01:00
Peter Kjellerstedt
c60134f4c8 Revert "rootfs.py: add more info to the warning message"
The reverted commit added a warning with the log line that triggered
the log check for error messages before the warning that states that
an error has been found in the log. However, the error line is output
by the call to bb.fatal() that follows immediately after the original
warning, which makes it redundant. Additionaly, having two warnings
contradicts the intent of commit 8dfdd329 where the log warnings were
tidied up.

This reverts commit f9cf31525fc885e1a0f65bd55654631257f87078.

(From OE-Core rev: e509a83c8eb5a572aad7e7451d019fa2469d70c3)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-19 22:32:06 +01:00
Anders Darander
c1ce0d9a9e lib/oe/rootfs: Fix DEBUGFS generation for opkg & openssl-cnf
When enabling extra DEBUGFS image generation with opkg, the following error is
seen when openssl-cnf is included in the image.

Collected errors:
 * file_md5sum_alloc: Failed to open file /mnt/cs-builds/anders/oe-build/build-ccu/tmp/work/ccu-oe-linux-gnueabi/ccu-image/1.0-r0/rootfs/usr/lib/ssl/openssl.cnf: No such file or directory.

Lots of similar issues was fixed by an earlier commit in oe-core,
5084ed9401250ed269a49d27b303806ab173c5d5, but openssl-cnf is outside of that fix.

Followup to [YOCTO #9490]

(From OE-Core rev: 20ea6d274bb0a9a5addb111f32793de49b907865)

Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11 10:33:41 +01:00
Richard Purdie
69d3df9169 lib/oe/rootfs: Fix DEBUGFS generation when using opkg
When enabling extra DEBUGFS image generation with opkg, errors are seen like:

ERROR: core-image-minimal-1.0-r0 do_rootfs: Cannot get the installed packages list. Command '/media/build1/poky/build/tmp/sysroots/x86_64-linux/usr/bin/opkg -f /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/opkg.conf -o /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs  --force_postinstall --prefer-arch-to-version   status' returned 0 and stderr:
Collected errors:
 * file_md5sum_alloc: Failed to open file /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/etc/syslog-startup.conf.busybox: No such file or directory.
 * file_md5sum_alloc: Failed to open file /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/etc/fstab: No such file or directory.

basically for all CONFFILES in the image. This is due to the file rearranging
the rootfs generation code does. If we preserve the /etc directory,
the avoids the problem.

We need to tell copyfile to preserve symlinks since some are present in /etc.

[YOCTO #9490]

(From OE-Core rev: 5084ed9401250ed269a49d27b303806ab173c5d5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-29 07:41:42 +01:00
Bill Randle
967bc74220 rootfs.py: apply ROOTFS_POSTINSTALL_COMMAND to all package formats
Previously, ROOTFS_POSTINSTALL_COMMAND was run only after the opkg rootfs
install post processing phase. This patch makes it generic so it is run fter
any package manager specific rootfs post processing, but before _run_intercepts().

[YOCTO #9248]

(From OE-Core rev: 8f2571881cd01ace88d282a63ad802cab4ab6940)

Signed-off-by: Bill Randle <william.c.randle@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-14 10:58:33 +01:00
Peter Kjellerstedt
aa13b97286 image.bbclass: Make unneeded packages for a read-only rootfs configurable
Previously the list of packages that are considered unneeded for a
read-only rootfs was hardcoded. This made it impossible to, e.g., have
shadow installed on a system with a read-only rootfs, but where /etc
is mounted writable.

This also lists ${VIRTUAL-RUNTIME_update-alternatives} rather than
update-alternatives (as was previously the case) since this should
actually remove the intended package.

(From OE-Core rev: e3b881d4168e5b02ff00f5c470ba472ab8bbc747)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-31 23:01:37 +01:00
Stefan Agner
db81fc967a lib/oe/package_manager: remove package feed lists
Remove opkg package feed lists after generating the rootfs. The
lists have been generated by the local feed but are no longer
necessary after image generation. This still leaves the package
management fully usable (and hence is different from omitting
package-management from IMAGE_FEATURES).

(From OE-Core rev: 25477874ef737777cedc623a8e1c5aedf2f4bae1)

Signed-off-by: Stefan Agner <stefan.agner@toradex.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-31 13:29:48 +00:00
Mariano Lopez
6ebda8e659 lib/oe/rootfs: Use list_pkgs() instead of list()
This patch changes the use list_pkgs() instead of list()
from class RpmPkgsList. The change is in two functions,
image_list_installed_packages from rootfs.py and
sdk_list_installed_packages from sdk.py.

With this change the functions calling the functions
listed above, must format the output as they required.
The formatting can be done using format_pkg_list() from
oe.utils.

The classes calling the afected functions are changed too
with this patch, to keep the same functionality using the
new data structure.

[YOCTO #7427]

(From OE-Core rev: 983ea373362514e5888bd1d7d9c4f136c94b00f2)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-20 17:07:15 +00:00
Aníbal Limón
b8ebac9794 DpkgRootfs: Fix logcheck_error false-positive when use multilib
Rootfs with dpkg was failing due to false-positive in logcheck_error
because current logic of DpkgPM handles missing dependencies failure
using apt-get -f install [1][2].

This support was broken due to addition of logcheck and don't take into
account dpkgpm cases, in order to fix add an attr for specify expected
errors regex'es by package manager.

[1] http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oe/rootfs.py#n659
[2] http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/lib/oe/package_manager.py#n2038

(From OE-Core rev: fa7a5ebef87883755491b847c2f4e1a7b021d585)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:31 +00:00