Commit Graph

167 Commits

Author SHA1 Message Date
Alexander Kanavin
cd005e2d5b meta/lib/oe/package_manager.py: warn about failing scriptlets for all package types
Previously this was done only for rpm packages; now also ipk/deb scriptlet
failures are reported.

In the future this will become a hard error, but it can't yet happen
due to the legacy 'exit 1' way of deferring scriptlet execution to first boot which
needs a deprecation period.

(From OE-Core rev: a36671faf6e0b7623185b0e22814a786d5444592)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-15 06:27:18 -07:00
Niko Mauno
0d4459e708 package_manager.py: Explicit complementary fail
When running bitbake -c populate_sdk <image_name>, it is expected that
packages matching SDKIMAGE_INSTALL_COMPLEMENTARY name mask (unless
declared in PACKAGE_EXCLUDE_COMPLEMENTARY) are installed to resulting
SDK. Underlying mechanism issues a package manager install call for set
of complementary packages. However the mechanism doesn't seem to inform
the user all too obviously in case the package manager command behind
install_complementary() method fails -- and since it is combined with
attempt_only=True option, user might end up wondering why several *-dev,
*-dbg packages are missing from resulting SDK.

Improve associated install() method behaviour in affected OpkgPM and
DpkgPM classes so that a problematic state of affairs becomes directly
obvious for bitbake user, resulting in shell output like:

  WARNING: someimage-1.0-r0 do_populate_sdk: Unable to install packages.
  Command '...' returned 1:
  Collected errors:
   * Solver encountered 1 problem(s):
   * Problem 1/1:
   *   - package somepkg-dev-1.0-r0.x86 requires somepkg = 1.0-r0, but
         none of the providers can be installed
   *
   * Solution 1:
   *   - allow deinstallation of someotherpkg-1.1-r1.x86

   *   - do not ask to install a package providing somepkg-dev

   * Solution 2:
   *   - do not ask to install a package providing somepkg-dev

(From OE-Core rev: 2502bd591c37bf532d02dc6b37fc1e8b5224fb0a)

Signed-off-by: Niko Mauno <niko.mauno@vaisala.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-08 10:39:33 -08:00
Jason Wessel
22318836f6 package_manager.py: Print offending package instead of non-sense trace
If you have a package that does not generate a manifest due to using a
noexec rule, the package name should be printed so the problem can be
tracked down.  With out the patch you get an error that makes it look
more like the package_manager is broken as shown below.

oe-core/meta/lib/oe/package_manager.py', lineno: 534, function: create_packages_dir
     0530:
     0531:    for dep in rpmdeps:
     0532:        c = taskdepdata[dep][0]
     0533:        manifest, d2 = oe.sstatesig.find_sstate_manifest(c, taskdepdata[dep][2], taskname, d, multilibs)
 *** 0534:        if not os.path.exists(manifest):
     0535:            continue
     0536:        with open(manifest, "r") as f:
     0537:            for l in f:
     0538:                l = l.strip()
File: '/usr/lib/python3.5/genericpath.py', lineno: 19, function: exists
     0015:# This is false for dangling symbolic links on systems that support them.
     0016:def exists(path):
     0017:    """Test whether a path exists.  Returns False for broken symbolic links"""
     0018:    try:
 *** 0019:        os.stat(path)
     0020:    except OSError:
     0021:        return False
     0022:    return True
     0023:
Exception: TypeError: stat: can't specify None for path argument

(From OE-Core rev: 21924fdba286e5962b1680601664dc0491527e25)

Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-07 06:34:51 -08:00
Ross Burton
8994767745 package-manager: add install_glob()
(From OE-Core rev: 8d1b530c82de386d4183f5673c060b9d416a3835)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-01 22:18:47 +00:00
Ross Burton
b194e2f15f package_manager: improve install_complementary
- No need to use bb.utils.which() as subprocess will search $PATH
- Clarity flow by moving the install inside the try/except

(From OE-Core rev: f4d22b7195dd8f08fe26dd353c7e860208e87d6a)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-01 22:18:47 +00:00
Richard Purdie
3eff72cc1f sstatesig/staging/package_manager: Create common sstate manifest code
Create a common function for locating task manifest files rather than
several implementations with missing pieces.

(From OE-Core rev: 68150bac7444f089f19c789e9f6602d59f605d7a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-01 22:13:55 +00:00
Richard Purdie
6a07697a6d package_manager: Filter to only rpms we depend upon
Currently do_rootfs gets to see all rpms in the deploy directory. This filters
that view to only rpms which the image recipe has actual depends upon which
potentially removes some sources of confusion in the image construction.

This makes builds more reproducibile and also fixes contamination issues
where dnf picks up packages it shouldn't be able to 'see'.

[YOCTO #12039]

(From OE-Core rev: 85e72e129362db896b0d368077033e4a2e373cf9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-01 22:13:55 +00:00
Alexander Kanavin
7bc55b2960 meta/lib/oe/package_manager.py: deprecate 'exit 1' as a way to defer to first boot
'exit 1' is not optimal for two reasons:

1) Code is hard to read; it is not obvious that it means 'defer what follows to first boot'.
2) Worse, this hides actual errors in the scriptlets; there is no difference between scriptlet
failing because it's intended to be run on target and scriptlet failing because there's a bug or
a regression somewhere.

The new, supported way is to place the code that has to run on target into pkg_postinst_ontarget(),
or, if a more fine-tuned control is required, call 'postinst-intercepts defer_to_first_boot' from
pkg_postinst() to explicitly request deferral to first boot.

(From OE-Core rev: d12cf56e9ff2a4f13dfbef9290ea5647b52b3f6d)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-30 11:50:12 +00:00
Richard Purdie
902b77bf91 lib/oe/package_manager/sdk: Ensure do_populate_sdk_ext and do_populate_sdk repos don't conflict
The repository indexes updated during do_populate_sdk_ext and do_populate_sdk
can conflcit. Add the missing lockfile calls for deb/ipk and in the rpm case,
ensure different directories are used for the index for the two sdk cases.

(From OE-Core rev: 5e5569c962c9ebc898eeb5044214e95117b190e1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-22 10:39:10 +00:00
Richard Purdie
0d0984e1e6 nativesdk/sdk: Update sdk dummy providers
When we migrated rpm v5 -> v4, we lost the ability to drop "per file"
dependencies from the rpm backend for things like "/bin/bash" and
"/usr/bin/env" which meant the sdks were becomming 'bloated'.

This restores the functionality using a dummy package, similarly to
the way the buildtools perl issue was addressed. It also removes
the non-functional old code so as not to confuse people in future.

I ran into this problem trying to filter dependencies to only rpms
a build directly depends upon and it turns out we have some determinism issues
in this area so this is something key to fix.

(From OE-Core rev: 9d490dc01dcedb216129b22cbe17a6c99efc4f5c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-11 10:26:06 +00:00
Robert Yang
c60a2c3ffe package_manager.py: remove obsolete MULTILIB_ARCHS
It had been removed since 2011:

commit b774bf44ef004276da12a83ebd69715c00b596ac
Author: Lianhao Lu <lianhao.lu@intel.com>
Date:   Tue Aug 16 16:26:49 2011 +0800

    package(_ipk).bbclass: opkg using ALL_MULTILIB_PACKAGE_ARCHS

(From OE-Core rev: e03cfb5a04e359c0bacb002dc80f3348301445d3)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-06 22:32:22 +00:00
Robert Yang
70c122402f package_manager.py: print running dnf command
This can make debug easier, for example, makes it easy to run the
command mannually.

(From OE-Core rev: 220b912237da78d257c5310bd07db89489afede1)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-06 22:32:22 +00:00
Ross Burton
eeb79f4e18 package_manager: force dnf to refresh the cache
DNF has a time-based cache policy (and a great sense of humour) so it's possible
that 'dnf makecache' won't actually refresh any caches.  Force the cache updates
by passing --refresh.

(From OE-Core rev: a6e34b6aa194a27db9667af1fb0195ac8fb563f5)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-12-02 11:25:34 +00:00
Otavio Salvador
48d2dc6711 lib/oe/package_manager.py (rpm): Allow use of non-signed packages
When we wish to use the package feed for local development, it does
not uses GPG signed feeds by default but dnf uses package signature
check. We need to configure the GPG signature check out so it works
out of box.

With this patch, installing non-signed packages works:

$: dnf install <package>

(From OE-Core rev: bceafe23cbe12a1ab0628b70865cb8867c7d7ee1)

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-10-07 23:20:40 +01:00
Mikko Ylinen
edb4d9984b package_manager: rpm: improve logging
To be able to better debug remove() behaviour, add more logging
to rpm calls via bb.note(). The change also makes remove() logging
more consistent with other package managers' remove() (e.g., opkg).

(From OE-Core rev: 1a90c97616698050529e169a83e6f07a255687df)

Signed-off-by: Mikko Ylinen <mikko.ylinen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-10-07 23:20:39 +01:00
Ming Liu
307f25c23e meta: drop True option to getVar calls
Search made with the following regex: getVar ?\((.*), True\).

(From OE-Core rev: dbc0eaf478feb3f752ae22fd184984494fc85d0a)

Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-26 11:05:01 +01:00
Richard Purdie
91f9752cf1 package_manager: Clean up horrible 'None' return values
If this fails the exception will now be raised. Lets use that and drop
all this 'None' return value ugliness.

(From OE-Core rev: b3c63b3b816179b96f1ed9b5baaf6e1f1c3c7b80)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-02 00:52:10 +01:00
Juro Bystricky
a8b9b3a5ae package_manager.py: improve reproducibility
When creating Packages.gz, do not save the time stamp in the compressed
file metadata.

(From OE-Core rev: 693fc377aa0788825b921d1b7804f5b2eafec76f)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31 17:41:41 +01:00
Alexander Kanavin
f9c451b71a signing-keys: do not use DISTRO_VERSION in key names
DISTRO_VERSION may contain the current date, and so is prone to mismatches
particularly when keys are created on one date, and dnf is configured to use the keys
on another date.

[YOCTO #11983]

(From OE-Core rev: b8f33f3f0a1e24f89151743d5c0b82efe45a9df1)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-30 11:14:26 +01:00
Markus Lehtonen
efcecc318b package_manager.py: enable dnf's repo_gpgcheck if feed signing is enabled
If package feed signing is enabled enable repo gpg signature check for
rpm repositories added via PACKAGE_FEED_URIS. This has the implication
that all repositories added via this mechanism must be signed with the
same key.

[YOCTO #11209]

(From OE-Core rev: f7716f1de0791dfe778bb70f1769a7e1e83c7a54)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-16 00:03:15 +01:00
Markus Lehtonen
d27f45a692 lib/oe/package_manager: re-implement rpm feed signing
[YOCTO #11209]

(From OE-Core rev: 7a33426561ddc72296e33c5e7b8ca38763a7ab07)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-16 00:03:15 +01:00
Richard Röjfors
0a3bb8b827 package_manager: Fix support for NO_RECOMMENDATONS
When support for dnf was introduced the check of the
no NO_RECOMMENDATIONS variable got broken.

This fixes the issue by compairing to the string "1"
rather than the number 1.

(From OE-Core rev: 1849ce3bd7c0af055f3e849a6508e746b6a0dca5)

Signed-off-by: Richard Röjfors <richard@puffinpack.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-09 09:24:16 +01:00
Alexander Kanavin
ee7c7150f0 package_manager.py: set dnf's releasever setting from DISTRO_CODENAME
So that:

1) dnf does not complain anymore about releasever not being set and then fail
for the same reason;

2) it's possible to refer to $releasever in dnf package feed configuration
(repo paths in particular) without hardconding the release name (pyro, morty, etc.)

(From OE-Core rev: 789e3fc225adbb61f10aaa3bbc3677856f5f0238)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-23 11:44:14 +01:00
Martin Jansa
1460e8aa99 package_manager: flush installed_pkgs file before oe-pkgdata-util uses it
* since this commit:
  commit f5a1013ffa9815f22e13989e2bcb83f966e7ce2c
  Author: Ross Burton <ross.burton@intel.com>
  Date:   Tue Apr 18 16:19:12 2017 +0100

    package_manager: don't race on a file when installing complementary
    packages

  the file isn't closed before oe-pkgdata-util uses it and this
  temporary file might look empty to oe-pkgdata-util, because it
  wasn't flushed yet. Which resulted in almost empty debugfs tarballs
  and no locale packages in regular rootfs.
* without this change:
  124K May 30 07:41 core-image-full-cmdline-raspberrypi3-64-20170530054003-dbg.rootfs.tar.gz
* with this change:
  173M May 30 07:29 core-image-full-cmdline-raspberrypi3-64-20170530052715-dbg.rootfs.tar.gz

(From OE-Core rev: 877d38db08aa7060d16405443cf70539c559fe82)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-14 14:53:58 +01:00
Peter Kjellerstedt
3a9a5e85e8 package_manager.py: Generate correct RPM package names again
During the transition to RPM4, the package names returned by
RpmPM.list_installed() changed from the expected names of the packages
that were installed into the image to some fictitious source RPM
names.

This restores the original functionality so that the
installed-packages.txt files produced by inheriting buildhistory yet
again contains a list of the names of the installed packages.

(From OE-Core rev: 25a36bcea4c02002d4895ba17f49ced09ae821c2)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-12 23:01:22 +01:00
Jussi Kukkonen
adea8003ab package_manager.py: Reverse rpm arch order
The architecture list used by dnf/libsolv was in the wrong order.
As a result, the images were built with wrong and unpredictable
packages.

$ MACHINE=intel-corei7-64 bitbake core-image-sato
$ MACHINE=qemux86-64 bitbake core-image-sato
$ MACHINE=intel-corei7-64 bitbake -ccleansstate core-image-sato
$ MACHINE=intel-corei7-64 bitbake core-image-sato

The first image had 0 core2_64 packages in it, but the last one had
583 core2_64 packages (which were built for the qemu image in
between).

Reverse the arch order in etc/dnf/vars/arch.

Fixes [YOCTO #11384].

(From OE-Core rev: 4a82433de42943f8219beca3286f40b67157172f)

Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-20 07:55:25 +01:00
Ross Burton
ae5d643114 package_manager: don't race on a file when installing complementary packages
PackageManager.install_complementary() uses WORKDIR/installed_pkgs.txt as a
temporary file but if two tasks are executing for the same recipe which uses
this file (e.g. bitbake my-image my-image:do_populate_sdk) then it's possible
for the file to be overwritten or deleted.

Instead of using a static filename, use tempfile to generate a unique name and
ensure it is cleaned up when finished.

Also move the glob generation/expansion earlier in the function as if there are
no globs to install, we don't need to generate a package list.

(From OE-Core rev: f5a1013ffa9815f22e13989e2bcb83f966e7ce2c)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-19 10:18:43 +01:00
Ian.Arkver
2bc2e67dd0 package_manager.py: Generate separate repo entries per arch
dnf requires a serparate repo for each architecture. This patch
writes one config file per PACKAGE_FEED_URIS entry with an entry
for each architecture, if any.

It also uses a space separated version of the repo id as the repo
name instead of just the id again.

(From OE-Core rev: 3eed822b5c5661aa9f43af6582c1481bacf0d39a)

Signed-off-by: Ian.Arkver <ian.arkver.dev@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-14 09:47:08 +01:00
Ian.Arkver
f2e41a197f package_manager.py: Add a name for the generated dnf repo.
This adds repo_name as the name, which is the same as the
repo ID and hence a bit pointless, but it stops dnf from
complaining that the repo doesn't have a configured name.

(From OE-Core rev: c92cff23893e87cfb7ef9f77b16761310104543a)

Signed-off-by: Ian.Arkver <ian.arkver.dev@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-12 15:09:58 +01:00
Ian.Arkver
ae65b95354 package_manager.py: Split feed_archs for RPM repo URIs
By default the feed_archs variable is split into single characters
resulting in very many broken short repo_uris. Add a split() to
split the string into words first.

(From OE-Core rev: 7bd6a4d56c08bf7f9ced7c07327e6163216fee08)

Signed-off-by: Ian.Arkver <ian.arkver.dev@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-11 18:10:17 +01:00
Martin Jansa
fcbad38193 package_manager.py: respect OPKGLIBDIR
* respect it for incremental rootfs generation
* add lists_dir option to opkg.conf
* also fix setting info_dir and status_file when they use default value, the
  problem is that self.opkg_dir is already prefixed with rootfs directory,
  comparing it with /var/lib/opkg always returned false and the options were
  appended to config file unnecessary
* with opkg 0.3.4 we can use VARDIR prefix added in:
  commit d2a8e23dc669adc398f4bb8bcfcabfcf925708f7
  Author: Florin Gherendi <floring2502@gmail.com>
  Date:   Mon Dec 19 12:25:38 2016 +0200
  libopkg: make the /var and /etc directories configurable at compile time.

(From OE-Core rev: b14c11d062872c3dcf95e03b61017005dea5b754)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-24 23:43:32 +00:00
Alexander Kanavin
4581e2c3e6 sign_rpm.bbclass: do not set/use RPM_GPG_PUBKEY
This is entirely unnecessary (we can ask the signer backend to export the
key to a file when needed), and was causing confusing selftest failures
due to the variable being set from two different places.

[YOCTO #11191]

(From OE-Core rev: 74ea979044368dc28c24325e7e77471b70aa8fe8)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-23 13:19:49 +00:00
Paul Gortmaker
52d2faad9f package_manager: fix "exlcude" vs. "exclude" typo
Which results in:

 --------------------------------
      0557:        package_exclude = self.d.getVar('PACKAGE_EXCLUDE')
  *** 0558:        exclude_pkgs = (bad_recommendations.split() if bad_recommendations else []) + (package_exlcude.split() if package_exclude else [])
      0559:
      0560:        output = self._invoke_dnf((["--skip-broken"] if attempt_only else []) +
      0561:                         (["-x", ",".join(exclude_pkgs)] if len(exclude_pkgs) > 0 else []) +
      0562:                         (["--setopt=install_weak_deps=False"] if self.d.getVar('NO_RECOMMENDATIONS') == 1 else []) +
 Exception: NameError: name 'package_exlcude' is not defined
 ERROR: cube-builder-initramfs-1.0-r0 do_rootfs: Function failed: do_rootfs
 ---------------------------------

Cc: Alexander Kanavin <alexander.kanavin@linux.intel.com>
(From OE-Core rev: cee1bdc09f4bbfedcd7cac06b48ba9d195c29e62)

Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-17 16:53:04 +00:00
Alexander Kanavin
aa66e8782e dnf: add /usr/bin/dnf symlink that points to /usr/bin/dnf-2
All documentation refers to dnf binary as 'dnf' yet make install
does not create one - it's done by Fedora's spec file when building
the rpm. Let's replicate this behavior.

(From OE-Core rev: 456c4a8ffc9a292d7a3e036d92baf4a8f14d1f45)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-17 16:53:04 +00:00
Markus Lehtonen
9a548edb0f lib/oe/package_manager: import rpm signing key to rpmdb
Import the gpg key used in rpm signing into rpmdb. This makes it
possible again to create images when rpm signing is enabled.

Also, instruct dnf to enforce signature check if rpm signing is enabled.

(From OE-Core rev: f30c1653cc5ef9daf594cbd3faad329b9fa08ab7)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-17 16:53:04 +00:00
Alexander Kanavin
2b77735c72 rpm: add support for remote package feeds via PACKAGE_FEED_URIS variable
I've used a previous patch (which was never merged) by
Humberto Ibarra <humberto.ibarra.lopez@intel.com> as a model
for how to do runtime testing of this feature (e.g. we need to boot
an image, run dnf on it, and check that it is indeed able to
access the remote repo over http). Here's his original commit message:

=====
Testing that feeds specified with PACKAGE_FEED_URIS var are set
correctly has two parts. First a build with this var set is required,
and then smart update needs to be issued in the running taget.

The previous is not a common selftest practice because this is a
simple test, but requires building and running a specific image,
which takes a lot of time. testimage is not a good fit either,
since the images tested there do not have the PACKAGE_FEED_URIS
var set.

For this test, the runtime-test module is being used, which is a
selftest module but runs a testimage command. The var and test
environment were set in runtime-perf.py and the actual test is
done in a new testcase added to meta-selftest layer.
=====

[YOCTO #10872]

(From OE-Core rev: 3a9e2fdef9316e24b52ce99ac355fc2b09786c72)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-17 16:53:04 +00:00
Alexander Kanavin
67615e0175 rootfs_rpm.bbclass: migrate image creation to dnf
To properly look at this patch, you probably need a side-by-side diff viewing tool.

(From OE-Core rev: 65581c68d130fa74d703f6c3c92560e053857ac7)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-14 14:42:17 +00:00
Alexander Kanavin
b9c550dd6e package_manager.py: improve the API for insert_feed_uris()
No need to store the configuration as class members,
just pass it directly into the method.

(From OE-Core rev: a5cc38481be3c5e6ccbecf951f9fdc049e5101d5)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-14 14:42:17 +00:00
Richard Purdie
bfd0a39bdf classes: Drop now unneeded update_data calls
Now that the datastore works dynamically we don't need the update_data calls
so we can just remove them. They're not actually done anything at all for
a while.

(From OE-Core rev: 8de0c5d3bd01919e2bf0394f9c485936d6098cec)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 09:29:55 -08:00
Chen Qi
2a64701acf package_manager: default to have scriptlet output captured in log
We need to have scriptlet output captured in log. If we don't do so,
some useful information from scriptlets (especially postinstall script)
would be missing. In case a script has a warning message but it does not
necessarily have to fail, the message should be captured.

Opkg has already done that. Change for rpm and dpkg so that scriptlet
output is captured and no warning message is missing.

(From OE-Core rev: 0e52e0c619e02327602d83999a61d978d3a9240e)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:22 +00:00
Mariano Lopez
55aa669750 lib/oe/package_manager.py: Fix extract for ipk and deb
With the move to use lists instead of strings in subprocess
calls, package extraction was broken for ipk and deb. This
fixes this issue.

(From OE-Core rev: 3e1d8e5c7ac3238eda85ee95dfef044bef2a6411)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:20 +00:00
Richard Purdie
022a8b58c8 meta/scripts: Various getVar/getVarFlag expansion parameter fixes
There were a few straggling expansion parameter removals left for
getVar/getVarFlag where the odd whitespace meant they were missed
on previous passes. There were also some plain broken ussages such
as:

d.getVar('ALTERNATIVE_TARGET', old_name, True)
path = d.getVar('PATH', d, True)
d.getVar('IMAGE_ROOTFS', 'True')

which I've corrected (they happend to work by luck).

(From OE-Core rev: 688f7a64917a5ce5cbe12f8e5da4d47e265d240f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-09 13:39:11 +00:00
Ross Burton
de43650c29 lib/oe/package_manager: bail if createrepo can't be found
If createrepo isn't found then the errors later are mysterious, so explicitly
check and error out early if it isn't there.

(From OE-Core rev: e09636bbb3ea8ec58984197fd9c691bb908efe00)

(From OE-Core rev: c87361fc886432a9db584712bf3e41ecd0541960)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-17 09:57:04 +00:00
Joshua Lock
c4e2c59088 meta: remove True option to getVar calls
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.

Search made with the following regex: getVar ?\(( ?[^,()]*), True\)

(From OE-Core rev: 7c552996597faaee2fbee185b250c0ee30ea3b5f)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Stephano Cetola
d8fbaebc48 package_manager: remove strings and migrate to direct arrays
When using subprocess call and check_output, it is better to use arrays
rather than strings when possible to avoid whitespace and quoting
problems.

[ YOCTO #9342 ]

(From OE-Core rev: b12cec9a5ef14ecb02be7feec65508cf5d65c795)

Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-08 10:31:30 +00:00
Linus Wallgren
b91e47fccb lib/oe/package_manager: .deb pre/postinst args
The debian policy manual and MaintainerScripts wiki page states that the
postinst script is supposed to be called with the `configure` argument
at first install, likewise the preinst script is supposed to be called
with the `install` argument on first install.

https://www.debian.org/doc/debian-policy/ch-maintainerscripts.html
https://wiki.debian.org/MaintainerScripts

(From OE-Core rev: 3d9c3aae54589794ce3484fa1b21d1af2bd32661)

Signed-off-by: Linus Wallgren <linus.wallgren@scypho.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:11 +00:00
Samuli Piippo
b18901d9bc package_manager.py: correctly remove all dependent packages
Do not use --force-depends when trying to remove all dependent packages,
as it removes only the selected package and not the dependent packages.

(From OE-Core rev: a82e8725902086dab785a0b14305927dae1e4e8d)

Signed-off-by: Samuli Piippo <samuli.piippo@qt.io>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-06 23:35:33 +00:00
Peter Kjellerstedt
d067b0e1e4 package_manager.py: Allow multiple regexps in PACKAGE_EXCLUDE_COMPLEMENTARY
The PACKAGE_EXCLUDE_COMPLEMENTARY variable can currently only contain
one regular expression. This makes it hard to add to it from different
configuration files and recipes.

Allowing it to contain multiple, whitespace separated regular
expressions should be backwards compatible as it is assumed that
whitespace is not used in package names and thus is not used in any
existing instances of the variable.

After this change, the following three examples should be equivalent:

  PACKAGE_EXCLUDE_COMPLEMENTARY = "foo|bar"

  PACKAGE_EXCLUDE_COMPLEMENTARY = "foo bar"

  PACKAGE_EXCLUDE_COMPLEMENTARY = "foo"
  PACKAGE_EXCLUDE_COMPLEMENTARY += "bar"

(From OE-Core rev: a5f7e98a94e96d40b1276c85249619aa8d7be847)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-06 07:51:00 +01:00
Peter Kjellerstedt
2334201960 package_manager.py: Allow a leading - in PACKAGE_EXCLUDE_COMPLEMENTARY
This allows a regular expression specified in
PACKAGE_EXCLUDE_COMPLEMENTARY to have a leading dash. Without this,
the dash was treated by oe-pkgdata-util as the beginning of a command
line argument. E.g., if PACKAGE_EXCLUDE_COMPLEMENTARY = "-foo$", it
resulted in an error like:

  ERROR: <imagename>-1.0-r0 do_populate_sdk: Could not compute
  complementary packages list. Command '<topdir>/scripts/oe-pkgdata-util -p
  <builddir>/tmp/sysroots/<machine>/pkgdata glob
  <workdir>/installed_pkgs.txt *-dev *-dbg -x -foo$' returned 2:
  ERROR: argument -x/--exclude: expected one argument
  usage: oe-pkgdata-util glob [-h] [-x EXCLUDE] pkglistfile glob [glob ...]

(From OE-Core rev: ac4ca41d3a27356d46c0c39053e74d3519b24c44)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-06 07:51:00 +01:00
Mark Hatle
0a04692279 package_manager.py: Change diagnostic messages per IRC
Based on a discussion with IRC user: Ulfalizer

It was suggested that removing the diagnostic list, and replacing it with a
simple hint to what might be causing the problem was a better solution.

(From OE-Core rev: ca78313665b23bd7fee85f034acfe1eb1009bd65)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:08 +01:00