Commit Graph

400 Commits

Author SHA1 Message Date
Andreas Oberritter
349f3ab519 package.bbclass: Add SYSTEMD_SERVICE to list of package specific variables
Changes to SYSTEMD_SERVICE should change the sstate checksum. To make that
happen, it needs to be listed in the list of package specific variables,
therefore add it.

(From OE-Core rev: 0a51f024e8212f011bb8b03d81f18bcd9aa6aa1f)

Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-25 12:39:37 +00:00
Chen Qi
0aa5ea1251 packaging: allow globs in CONFFILES
Allow globs in CONFFILES.

This patch changes the way of CONFFILES handling. After this  change,
the CONFFILES can take the same form as FILES. That means, we don't
have to list a bunch of files for CONFFILES. It will just be expanded
like the FILES variable.

We don't assume default value for CONFFILES in OE. But distro vendors could
provide a default value for CONFFILES in their distro configuration file
like below.

    CONFFILES = "${sysconfdir}"

In this way, files under /etc are treated as configuration files by
default. Of course, setting CONFFILES in recipes take precedence over
the CONFFILES. For example, if the recipe author decides that package A
should only treat files under ${sysconfdir}/default/ as config files,
he/she can write like this.

    CONFFILES_A = "${sysconfdir}/default"

[YOCTO #5200]

(From OE-Core rev: 0d446ef0e5bbca7058eec7259e34f2a1637dfab1)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-19 07:51:38 +00:00
Mike Looijmans
a9102e5bf9 package.bbclass: Let PR server update PKGV, not PV
PV is the package version as we need it to be during the build. PKGV is the
final version as it ends up in the package, and defaults to PV.

The packager handled builds without PR-server by replacing the AUTOINC string
in PKGV, but when the PR-server is being used, the script replaces the contents
of PKGV with the PV if the PV contains "AUTOINC". Thus the packager overrides
any change to PKGV the recipe might have made.
This breaks classes like gitpkgv that provide a correctly numbered PKGV, the
number as calculated by that class will simply be replaced with a 0-based index
from the PR-server.

This patch makes the packager look at the PKGV version instead of the PV, and
update the PKGV only based on the PKGV contents as set by the recipe.

See also the discussion here:
http://lists.openembedded.org/pipermail/openembedded-core/2015-January/100329.html

From investigating the history of the code and changes in the past year, the
use of "pv" instead of "pkgv" appears to be just an oversight, introduced in:
commit b27b438221e16ac3df6ac66d761b77e3bd43db67 "prs: use the PRServer to replace the BB_URI_LOCALCOUNT functionality"
A later commit 865d001de168915a5796e5c760f96bdd04cebd61 "package/prserv: Merge two similar functions into one"
silently fixed this only for the case without PR-server by using pkgv there.

(From OE-Core rev: 7895c0a67d381ff66668fca5207bd196f36c91db)

Signed-off-by: Mike Looijmans <mike.looijmans@topic.nl>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-23 11:36:31 +00:00
Martin Jansa
013de192d7 package.bbclass: Fix support for private libs
* n is a tuple since this commit:
  commit d3aa7668a9f001044d0a0f1ba2de425a36056102
  Author: Richard Purdie <richard.purdie@linuxfoundation.org>
  Date:   Mon Jul 7 18:41:23 2014 +0100
  Subject package.bbclass: Improve shlibs needed data structure

  since then 'n in private_libs' was always false and private libs
  were always processed
* this is bad when we have libfoo in private libs, but also some package
  providing libfoo, that way we ship own libfoo.so, but together with
  runtime dependency on package providing libfoo

(From OE-Core rev: ec1d379683cedca4be1c252475d02c8041227142)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-16 23:08:20 +00:00
Paul Eggleton
de8730ce2d classes/package: move read_shlib_providers() to a common unit
This allows us to use this function elsewhere in the code.

(From OE-Core rev: 657cff8a0f0e5db171b2ed9388a790ee0b135842)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-21 12:37:56 +00:00
Peter Seebach
244cb9b5c3 package.bbclass: do variable fixups even when FILES was set
A number of settings (DESCRIPTION, SUMMARY, postinst, postrm,
and appends to RDEPENDS) were made only if FILES_foo was not
set for a given package. If you had a modified glibc packaging
setup that was defining FILES_glibc-gconv-somelocale, this would
prevent the automatic append of glibc-gconv as a dependency,
because extra_depends was ignored.

I think the assumption may have been that if FILES_foo was set,
DESCRIPTION_foo and SUMMARY_foo would also be set, but it seems
to me that the right answer is probably to set them if they aren't
already set, and leave them alone if they are.

(From OE-Core rev: 7e59b0c7e03fc08a6eaf9c8ccb6bfa72b4604cc5)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-19 18:08:00 +00:00
Khem Raj
ec4781194c package.bbclass: Create empty key/value if not there for shlib_provider
When we use ASSUME_SHLIBS,e.g.

ASSUME_SHLIBS = "libEGL.so.1:libegl-implementation"

then we end up with errors like below when using shlibs2 (dizzy+)

File: 'package_do_shlibs', lineno: 216, function: package_do_shlibs
     0212:            dep_pkg = dep_pkg.rsplit("_", 1)
     0213:            if len(dep_pkg) == 2:
     0214:                lib_ver = dep_pkg[1]
     0215:            dep_pkg = dep_pkg[0]
 *** 0216:            shlib_provider[l][libdir] = (dep_pkg, lib_ver)
     0217:
     0218:    libsearchpath = [d.getVar('libdir', True),
d.getVar('base_libdir', True)]
     0219:
     0220:    for pkg in packages.split():
Exception: KeyError: 'libEGL.so.1'

This is because the entry which is being populated does not exist
so lets create it if its not already there.

Change-Id: I9e292c5439e5d1e01ea48341334507aacc3784ae
(From OE-Core rev: a64f81fcef42172f788cec7a63bb4672eac99f94)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-03 12:23:58 +00:00
Richard Purdie
593f14b2e3 package/prserv: Merge two similar functions into one
Having these two separate functions handling PR values seems pointless,
and worse, there are impossible code branches mixed within them.

Merge them into one function and tweak comments so at least you
don't have to read both functions to figure out what is going on.

This does restructure the conditionals to try and aid readability.

(From OE-Core rev: 865d001de168915a5796e5c760f96bdd04cebd61)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-11-09 10:21:19 +00:00
Khem Raj
b2f1250f6d package_do_shlibs: Look for provider in the path thats in shlib_provider dictionary
shlib2 code puts the information about path where a provider
will be found. e.g.

{'/usr/lib/llvm3.3': ('libllvm3.3-llvm-3.3', '3.3')}

This is obtained from new shlib2 pkgdata from

llvm3.3/3.3-r0/pkgdata/shlibs2/libllvm3.3-llvm-3.3.list

However when we search for NEEDED libraries we ignore the
key above which is the path where the provider library is installed
and instead just seach in libdir and base_libdir and hence
libraries which are not in above standard search paths gets
ignored even if they appear in DT_NEEDED sections

and a note is emitted

NOTE: Couldn't find shared library provider for libLLVM-3.3.so, used by
files: ....

IMO this note should actually become an error since if we do
not have all DT_NEEDED libraries in image the system is dysfunctional.

This patch extracts this libpath from key and add it to seach paths
when looing for a provider of a shared library

[YOCTO #6798]

Change-Id: Ie5f08632e37ba8d3439c8aaae33bc68b8996792f
(From OE-Core rev: bf8472274ac1e9a35f8cbc82357da6c95b396759)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-10-10 22:40:45 +01:00
Otavio Salvador
533a852edd package.bbclass: Reverse runtime symlinks should be tied to package generation
In case a package is not generated (is empty and does not has allow
empty flag set) the package data regarding reverse runtime dependency
shouldn't be done.

This were causing a false-positive in the meta-fsl-arm layer, when
building mesa, as:

,----[ Error during build of MX53 in meta-fsl-arm ]
| ERROR: The recipe mesa is trying to install files into a shared area
|  when those files already exist. Those files and their manifest
|  location are:
|    /.../build/build/tmp/sysroots/imx53qsb/pkgdata/runtime-reverse/libopenvg-dev
|    Matched in manifest-imx53qsb-amd-gpu-x11-bin-mx51.packagedata
| Please verify which recipe should provide the above files.
`----

Fixes [YOCTO: #6795]

(From OE-Core rev: 9ef8728514b02dd2e18e87645298d9ec2e8a785a)

Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-10-06 15:15:52 +01:00
Peter Kjellerstedt
9f36a3c445 package: Correct two typos in a comment
This quite coincidentally invalidates the sstate for do_package which
is needed due to the correction of oe.utils.multiprocess_exec().

(From OE-Core rev: 9972f0686794a01582fd1a15889dcbd89bc5cf72)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-29 23:44:37 +01:00
Richard Purdie
1a76648ec9 package: Disable runtime mapping of RPROVIDES/RCONFLICTS/RREPLACES
It doesn't really make sense to run the remapping code on these
variables. If specific renaming is needed, it should be applied
manually. This means that the debian RPROVIDES of the original package
name can be preserved. There was also a bug report about this on the
OE-Core mailing list recently where someone else ran into this problem
too.

(From OE-Core rev: a0cd18ff5405b63b8296b8ef1fc2a488b906f576)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-25 08:57:45 +01:00
Richard Purdie
ff6ab056c7 package: Don't remap names in packagegroup recipes
For package groups, use the original package names and don't use the
remapped (e.g. debian) naming.

(From OE-Core rev: afde71e8784144f05cebb0bdeeff6b3184819362)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-25 08:57:44 +01:00
Richard Purdie
9412254f23 package.bbclass: Fixup for using common function
We may as well use the common function for this rather than
duplicating the code.

(From OE-Core rev: 20979854c52607212c16e0b01cf610c80d8b524e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-23 09:31:41 +01:00
Richard Purdie
b52132122c package: Fix darwin shlibs code
We need to scan binaries as well as libraries for dependencies.

Also ensure if its not an object file (as found by otool), we handle
this case.

(From OE-Core rev: 3a26862b7a7e0ff247bd1441d670443e83fca391)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-04 16:25:46 +01:00
Richard Purdie
53eaed5c15 package: Convert dylib handling from .la to otool
Currently, the darwin shlibs detection is done by parsing the .la file
dependency fields. This is very old code and is incomplete in some
cases so convert to using otool -l and otool -L to correctly load
the rpath and dependency information.

(From OE-Core rev: e27573b6c3562662e4b2f5d8543eb7d150c3bc92)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-03 12:40:41 +01:00
Richard Purdie
c70203998b package: Fix pkgdest for darwin shlibs code
When the code was rewritten we forgot to strip the pkg that is present
in the pkgdest path. This was fixed in the linux version of the code
but not the darwin one, this matches the fix.

Without this, the provider paths are broken.

(From OE-Core rev: bcada055f89c8722024faf20088943ea006cfe36)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-03 12:40:41 +01:00
Ross Burton
360171c843 package.bbclass: add a stub implementation of package_name_hook
do_package() calls package_name_hook so that e.g. debian-style renaming through
debian.bbclass can happen.  If there is no class providing a package_name_hook
then this causes "WARNING: Function package_name_hook doesn't exist" every time
do_package() is executed.

Silence this warning by providing an empty package_name_hook in package.bbclass.

(From OE-Core rev: f6eb8e4e44b66217e4ada9f830a058c5ba120932)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-10 17:38:33 +01:00
Richard Purdie
e35d6269ba package.bbclass: Generate runtime-rprovides data
Currently, given a list of rdepends its near impossible to figure out
which ones are potentially invalid and which might be an RPROVIDES of a
package.

This problem is simple to solve, we can write lookup data into the
pkgdata. This patch does that, accounting for the fact that multiple
packages can RPROVIDE the same thing (particularly with locales). Its
done with symlinks so the performance overhead is minimal.

(From OE-Core rev: 30b4bff944c4625696e9be97ce4912a2710d13e9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-10 17:38:32 +01:00
Richard Purdie
3ead62a578 package.bbclass: Rewrite shlib_provider handling to include RPATH
Change the do_package shlibs code to account for RPATHS. This means that
for library dependency purposes, only libraries in system paths or in
any declared RPATH will be seen. This is important to resolve problems
people have been having where similarly named libraries in "private"
paths were conflicting, e.g. with gstreamer.

For now this code assumes the default search path is libdir and
base_libdir and places ASSUME_SHLIBS in libdir so they are searched by
default.

(From OE-Core rev: 48b9ecd5335f63fb2bcc47c4b23d067122734fe9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-10 17:38:31 +01:00
Richard Purdie
db788ac702 package.bbclass: Improve shlibs pkgdata file format
Instead of having a .list file and a .ver file, place the version
information into the .list file in a ":" delimited string.

Also place the path to the library here, this can then be used to
evaluate RPATHs in the shlib dependency code.

Since the disk format has changed, the easiest way to avoid build
failures in the same TMPDIR is to change the shlibs directory to
shlibs2. sstate dependency code with ensure everything rebuilds.

(From OE-Core rev: 8dd09b3ddc572a338b4a6f1d859b68b2b4482d00)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-10 17:38:31 +01:00
Richard Purdie
a20cc90cdc package.bbclass: Rewrite sonames data structure to include library path
In order to do more advanced processing of the shared libraries, we need
to know where a given library is located on disk so we can know whether
its a system path or a private directory for example.

This patch adds this information into the 'sonames' data structure.

(From OE-Core rev: 5f48d53c5ed94b5c6aad9f73ff177aa35af90397)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-10 17:38:31 +01:00
Richard Purdie
adb89b1589 package.bbclass: Improve shlibs needed data structure
Improve the shlibs 'needed' data structure to include the file and
any rpath information. This allows various cleanups to the data structure
and moves us closer to being able to resolve shlibs providers issues
based on path in due course.

This commit doesn't change any stored data, just cleans up internal data
structures (for example dropping the needed_from dict).

(From OE-Core rev: d3aa7668a9f001044d0a0f1ba2de425a36056102)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-10 17:38:31 +01:00
Richard Purdie
4db0d5f563 package.bbclass: Add CONFFILES to list of package specific variables
Changes to CONFFILES should change the sstate checksum. To make that happen,
it needs to be listed in the list of package specific variables, therefore
add it.

(From OE-Core rev: 9db71fa03b9d5f5307b2d09e7aa89f46f622aa09)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-08 13:59:59 +01:00
Richard Purdie
458496829b package: Drop do_package_write task
The reasons this task was introduced are lost in the mists of time. It
allowed for the a single "package_write" task instead of spelling out
the explicit package backends, however in all but one case we do that
anyway.

As such as might as well give in and delete the task, converting that
single reference into explicit dependencies.

This gives bitbake a bit less work to to when processing the runqueue
since there are less tasks (but more dependencies in some cases).

(From OE-Core rev: cf70e15f063716f3227d467ab1f4bfc0018286f6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-25 08:01:09 +00:00
Josua Mayer
a204c1e1be package.bbclass: do_split_packages should always return something.
So far, when do_split_packages is passed a non-existing folder, it will return nothing.
While building Mesa with PACKAGECONFIG="egl gles" for qemux86, do_package threw a nice exception on a line reading "pkgs += do_split_packages":
"Exception: TypeError: 'NoneType' object is not iterable"
I think a function should always return something, and in this specific case, returning an empty list should be right.
This patch simply fixes the return statement to do just that.

(From OE-Core rev: 39737e00e85bd4a6053f63f0c959fd424aa009be)

Signed-off-by: Josua Mayer <josua.mayer97@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-17 15:28:59 +00:00
Richard Purdie
716557ae35 package: Remove the before do_build constraint of the do_package task
Currently, if you bitbake xxxx, it will restore the package_write_,
packagedata, populate_sysroot *and* the package data from sstate.

We generally care about the packagedata and the actual packages, we
no longer need/care about the do_package data itself unless something
is going to repackage.

By relaxing this constraint (and relying on the packagedata target), we
can save things being installed from sstate and hence speedup build times.

(From OE-Core rev: 4fda89477a2d7c1b3186bf4896297338ed64c448)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-13 11:41:21 +00:00
Koen Kooi
afc4431250 package.bbclass: fix strip and split logic
Marks original commit message and variable documentation state that stripping and splitting are independent of eachother, but package.bbclass ANDs the two INHIBIT flags to see which files can be stripped and/or split.

Original behaviour:

INHIBIT_PACKAGE_STRIP: no strip, no debug split
INHIBIT_PACAKGE_DEBUG_SPLIT: no strip, no debug split

Behaviour after this patch:

INHIBIT_PACKAGE_STRIP: no strip, no debug split
INHIBIT_PACKAGE_DEBUG_SPLIT: strip, no split

(From OE-Core rev: 8ea3cc2c45d4e34bb68bd3e0bc359204c772133c)

Signed-off-by: Koen Kooi <koen.kooi@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11 11:56:56 +00:00
Martin Jansa
3f8f70bc15 package.bbclass: Show which files require given dependency in debug output
* when log.do_package shows some unexpected dependency, people usually
  need to grep package directory to find which binary was creating that
  dependency, show it directly in the debug output

(From OE-Core rev: cf0696a39b811b13bb6e7dd06a2dad607e93a643)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-29 17:38:50 +00:00
Martin Jansa
ee045eeb38 package.bbclass: Don't search for providers of PRIVATE_LIBS
* split PRIVATE_LIBS and don't use find(), so that libfoo cannot be
  found in PRIVATE_LIBS = "libfoobar"

(From OE-Core rev: 2dec075478f977b554061dd9a4b2b8ff4af3597a)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-29 17:38:50 +00:00
Leonid Borisenko
be77f3138c package.bbclass: add getter of additional metadata
Two variables are searched for value of additional package metadata:

  * PACKAGE_ADD_METADATA_<PKGTYPE>
  * PACKAGE_ADD_METADATA

First found variable with defined value wins.

<PKGTYPE> is a parameter of getter and expected to be a distinct name
of specific package type. For example: 'DEB' or 'RPM'.

Variable can contain multiple [one-line] metadata fields, separated by
literal sequence '\n'. Separator can be redefined through variable flag
'separator'. Getter returns found value with separator replaced with
newline character.

As side-effect, searched variables acquired flags 'type' (equals to
'list') and 'separator'.

(From OE-Core rev: 98ea2fc35a3ef609a944929e21e0f9be2889036d)

Signed-off-by: Leonid Borisenko <ive.found@gmail.com>
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-28 00:52:32 +00:00
Martin Jansa
1180ec37e4 package.bbclass: show warning when package is providing already provided shlib
* move read_shlib_providers before registering package as provider
  and show warning when different package tries to provide something
  already provided.

[YOCTO #4628]

(From OE-Core rev: 8141e3f61f12c8901b990496bcf6b76a9db95a57)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-21 10:20:13 +00:00
Martin Jansa
bd938fe510 package.bbclass: move reading shlibs providers to separate function
* prepare for reading shlibs providers only from dependency tree of
  current recipe

[YOCTO #4628]

(From OE-Core rev: c5076f33ac27c0c2b0743bf6dc4edc983254c467)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-21 10:20:12 +00:00
Phil Blundell
fef870956c package: Keep global pkgfiles[] up to date when snapping library symlinks
Since ca86603607a69a17cc5540d69de0e242b33382d3 we are now calling stat() on
all the pkgfiles[] during emit_pkgdata().  If symlink snapping has removed
some of the files then we will blow up trying to stat a path that no longer
exists.  Fix that by ensuring that pkgfiles[] is updated when we process the
list of library renames.

(From OE-Core rev: bdad58c51dfdda5253933c3e371a7d526cdbb67e)

Signed-off-by: Phil Blundell <pb@pbcl.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-10 15:16:51 +00:00
Paul Eggleton
17aa506450 classes/package: set SUMMARY in do_split_packages()
do_split_packages() is commonly used to split out plugin/module packages
dynamically within recipes. If it doesn't set SUMMARY for each of these
packages then they get a generic SUMMARY from the recipe, which isn't
particularly useful; so add a parameter to set this and default it from
the current description parameter (it ought to have been the other way
around, but the description parameter is what we currently have in use
by all recipes that use this function.)

Fixes [YOCTO #5406].

(From OE-Core rev: 2b3080831042ffab26d70c3feee232fc31aa591a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-02 22:39:24 +00:00
Richard Purdie
480bf037ae sstate: Get rid of crazy name mapping
When originally developed, it was thought a task may have more than one associated
sstate archive. The way the code has grown that idea is now not possible or needed.
We can therefore assume one sstate archive per task and drop the crazy name
mapping code. Simpler is better in this case.

The downside is that various sstate archives will change name so this forces a cache
rebuild. Given the other sstate changes going in at this time, this isn't really
a bad thing as things would rebuild anyway.

(From OE-Core rev: 5afe86a6854b21692fd97c5fc7fab50dbc068acb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-18 17:08:05 +00:00
Paul Eggleton
c36c1989c1 classes/package: write PE and PKGE out to pkgdata
These are important parts of the version for every package, so we should
include them in PKGDATA just as we include PV/PR/PKGV/PKGR.

(From OE-Core rev: 5ceed97ba02a698f1c260c3f56cdf2cc156e6d8b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:49 +00:00
Paul Eggleton
f4e91404f8 classes/package: record PKGSIZE as total file size in pkgdata
We were using "du -sk" to collect the total size of all files in each
package for writing out to PKGSIZE in each pkgdata file; however this
reports the total space used on disk not the total size of all files,
which means it is dependent on the block size and filesystem being used
for TMPDIR on the build host. Instead, take the total of the size
reported by lstat() for each packaged file, which we are already
collecting for FILES_INFO in any case.

Note: this changes PKGSIZE to be reported in bytes instead of kilobytes
since this is what lstat reports, but this is really what we should be
storing anyway so that we have the precision if we need it.

Fixes [YOCTO #5334]

(From OE-Core rev: 29615b36fca696822a715ece2afbe0bf9a43ed61)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:49 +00:00
Paul Eggleton
4a75e83b29 classes/package: fix FILES_INFO serialisation in pkgdata
The FILES_INFO entry in each pkgdata file stores the list of files for
each package. Make the following improvements to how this is stored:
* Store paths as they would be seen on the target rather than
  erroneously including the full path to PKGDEST (which is specific to
  the build host the package was built on)
* For simplicity when loading the data, store complete paths for each
  entry instead of trying to break off the first part and use it as the
  dict key
* Record sizes for each file (as needed by Toaster)
* Serialise the value explicitly using json rather than just passing it
  through str().

Fixes [YOCTO #5443].

(From OE-Core rev: ca86603607a69a17cc5540d69de0e242b33382d3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:49 +00:00
Felipe F. Tonello
8f84cc99bd package.bbclass: Fix split_and_strip_files when file has single quote (')
Fix false error report when a file that has a single quote by escaping
the single quote. Some packages might install files with quotes, such
as music files and other types, that will cause the problem.

(From OE-Core rev: 212471f81d210e596798db5e5d927418090a63a2)

Signed-off-by: Felipe F. Tonello <eu@felipetonello.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-05 22:04:27 +00:00
Paul Eggleton
567754acb8 classes/package: handle filenames containing wildcards
It is uncommon, but it is possible for upstream sources to contain files
that have wildcard characters in their names (Webmin is an example).
Because we were running glob.glob() on every entry in the list of
entries in FILES and then adding the result to the files list to be
processed, the process would loop infinitely if files whose names
contained wildcard characters were present. Fix this by avoiding
re-processing the output of glob.glob() with itself, and also "escape"
wildcard characters in FILES entries added automatically from
do_split_packages().

Fixes [YOCTO #1676].

(From OE-Core rev: 1aa3fbb547b0e21455f0dcc9b72ded08dc0efd67)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-14 16:55:24 +01:00
Richard Purdie
d3846f06d1 classes: tar 1.27 fixes
tar version 1.27 returns:

tar: --same-order option cannot be used with -c

with the commandlines we have been using. We can remove the -s option (which
is --same-order) to remove the error.

(From OE-Core rev: 3d5a6d0a480a0fa98260a3b3ffc71b8d9e3e58af)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-14 16:55:21 +01:00
Richard Purdie
2f8ce2c784 package.bbclass: Clear umask when using os.mkdir
We switched to using os.mkdir with the file creation mode specified as the
second parameter. Python masks this with umask behind the scenes which isn't
what we want, we really want the permissions we specify.

To avoid this we zero the umask beforehand and restore afterwards. Other
solutions are possible but would not perform as well which is why
we're using os.mkdir in the first place.

Martin Jansa deserves the credit for debugging where the problem was.

(From OE-Core rev: f91226553e39439bfd17ab2b06c56cb8bf41061b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-26 16:37:56 +01:00
Richard Purdie
8ebe7be3d9 bitbake.conf/package: Collapse PKGDATA_DIR into a single machine specific directory
Currently we have a hierarchy of pkgdata directories and the code has to put together
a search path and look through each in turn until it finds the data it needs.

This has lead to a number of hardcoded paths and file globing which
is unpredictable and undesirable. Worse, certain tricks that should be
easy like a GL specific package architecture become problematic with the
curretn search paths.

With the modern sstate code, we can do better and construct a single pkgdata
directory for each machine in just the same way as we do for the sysroot. This
is already tried and well tested. With such a single directory, all the code that
iterated through multiple pkgdata directories and simply be removed and give
a significant simplification of the code. Even existing build directories adapt
to the change well since the package contents doesn't change, just the location
they're installed to and the stamp for them.

The only complication is the we need a different shlibs directory for each
multilib. These are only used by package.bbclass and the simple fix is to
add MLPREFIX to the shlib directory name. This means the multilib packages will
repackage and the sstate checksum will change but an existing build directory
will adapt to the changes safely.

It is close to release however I believe the benefits this patch give us
are worth consideration for inclusion and give us more options for dealing
with problems like the GL one. It also sets the ground work well for
shlibs improvements in 1.6.

(From OE-Core rev: 1b8e4abd2d9c0901d38d89d0f944fe1ffd019379)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-14 08:21:00 +01:00
Khem Raj
6bf6708138 package.bbclass: skip already-stripped QA test if asked for
Some packages like grub have already stripped binaries e.g.

ERROR: QA Issue: File '/boot/grub/kernel.img' from grub was already
stripped, this will prevent future debugging!
ERROR: QA run found fatal errors. Please consider fixing them.

We would like to have a possibility to skip it using something like

INSANE_SKIP_${PN} = "already-stripped"

This adds the logic to do so

it acts at PN level and not at package level. so something like

INSANE_SKIP_${PN}-misc = "already-stripped" wont work.

(From OE-Core rev: 765982f4c050d9cd3eb608d630312da482c737c7)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-06 23:56:27 +01:00
Richard Purdie
6d41d0dab7 package.bbclass: Fix darwin shlib handling
shlibs dependency calculations on darwin we not functioning correctly, we
need to process the filename without the complete path. If we don't,
"." characters in the path cause problems.

(From OE-Core rev: 07e697d651178a84007123181fca38e4d98ae0e9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-04 14:15:57 +01:00
Richard Purdie
694f034ec0 package.bbclass: Fix handling of symlinks in debug packages
When copying the sources for the debug source package we use cpio -Ll
which means to copy files as hardlinks and to dereference symlinks.
It appears there is a bug in cpio since -Ll will copy symlinks and
not dereference them. We therefore do a second pass over copied symlinks
resolving them into files. Ideally we would copy these as hardlinks as well
however it doesn't seem worth the extra code and effort for what amounts
to a corner case for a minor space improvement.

This means that the -dbg packages no longer contain broken symlinks.

[YOCTO #5020]

(From OE-Core rev: 2ca2c4747f645a0d478c2171fff4c65752188285)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-23 16:20:11 +01:00
Andreas Müller
054b932f7b package.bbclass: avoid packing debug sources from other packages
The error case was gcc-dbg packing files from libgcc-dbg reported in [1]

[1] http://lists.openembedded.org/pipermail/openembedded-core/2013-July/080728.html

(From OE-Core rev: fd2c0af422d67da870a5a11720d06a871e660215)

Signed-off-by: Andreas Müller <schnitzeltony@googlemail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-07-29 13:09:00 +01:00
Paul Eggleton
74a1426551 classes/package: print command output when commands fail
When external commands such as debugedit fail, it can be useful to see
their output, so use oe.utils.getstatusoutput() instead of
subprocess.call() to capture this and print it on failure.

(From OE-Core rev: 34179cc78b730ecb8ff3f4feb4beee2d17498ce3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-07-02 22:26:57 +01:00
Richard Purdie
1cc028840d package: Ensure we iterate all the pool objects
There is the possibility that if we don't iterate through the multiprocessing
pool objects we might not catch return codes and this could lead to hung/zombie
processes either temproarily or on a wider scale. Adding this certainly doesn't
hurt anything and is better practise so we might as well do it.

Its not 100% clear if this fixes some issues or not.

(From OE-Core rev: 89c8493d4d85044cd72af2756569d15e87cd5947)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-28 10:01:27 +01:00
Robert Yang
829d6bf005 insane/package: refactor packaging sanity tests
Refactor packaging sanity tests from package.bbclass to insane.bbclass
so that the message can respect WARN_QA (print the warning message and
go on the task) and ERROR_QA (print the error message and fail the
task).

- For the bb.warn(), give it a message name and add it to WARN_QA, then
  use package_qa_handle_error() to handle it.

- For the bb.error(), give it a message name and add it to ERROR_QA,
  then use package_qa_handle_error() to handle it.

- All the bb.warn() and bb.error() have been replaced in
  package.bbclass.

- A few bb.warn() and bb.error() in insane.bbclass have been kept since
  they can not be replaced or doesn't have to, for example the
  bb.error() in package_qa_check_license(), it will print the error
  message and then invoke bb.fatal() to fail the task, I think that we
  don't have to replace it with package_qa_handle_error().

- Put all the WARN_QA and ERROR_QA in one line, so that they can be
  redefined by the user easily.

[YOCTO #3190]
[YOCTO #4396]

(From OE-Core rev: 2f117ee615b703db07d3274ac592e2bd653743dd)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-07 16:48:27 +01:00
Robert Yang
43713b9e17 insane/package: let package.bbclass inherit insane.bbclass
RP's comment:
"What we're trying to do is move everything to use a standard mechanism
for reporting issues of this type (do_package). With insane.bbclass, you
can elect whether a given type of error is a warning or error and fails
the task."

* The package.bbclass had used package_qa_handle_error() which is from
  insane.bbclass, and we will use it for handling other warnings and
  errors, so let package.bbclass inherit insane.bbclass, this change will
  make the insane as a requirement (always included).

* Change the "PACKAGEFUNCS ?=" to "+=", otherwise there would be an
  error like:
  Exception: variable SUMMARY references itself!

  This is because we let package.bbclass inherit insane.bbclass, and
  PACKAGEFUNCS has been set in insane.bbclass, so the "PACKAGEFUNCS ?="
  will set nothing, then the "emit_pkgdata" doesn't run which will
  cause this error.

* Add a QA_SANE variable in insane.bbclass, once the error type
  is ERROR_QA, it will fail the task and stop the build.

[YOCTO #3190]
[YOCTO #4396]

(From OE-Core rev: 852dead71387c66ec0cba7c71e3814a74e409560)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-07 16:48:26 +01:00
Robert Yang
51c3e9df28 bbclass: bb.fatal() clean up
The bb.fatal() is defined as:

def fatal(*args):
    logger.critical(''.join(args))
    sys.exit(1)

So anything after bb.fatal() in the same code block doesn't have any
effect, e.g.:

    bb.fatal("%s_%s: %s" % (var, pkg, e))
    raise e

The "raise e" should be removed.

I searched all the files which use bb.fatal(), only the following 3
classes have this issues:

  insane.bbclass
  package.bbclass
  package_rpm.bbclass

[YOCTO #4461]

(From OE-Core rev: 4c47525c5a171b1282615c9fbc7d84addef85f92)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-16 00:09:47 +03:00
Richard Purdie
efb8a460d2 package.bbclass: Fix sources contents
http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=c376f1f49cea182a2887945840ab97a20970a373
fixed a valid issue where the sources file was accumulating information
and subsequent task runs of do_packacge were not cleaning it. The
fix is wrong however since we're removing the file within a loop.
This fix removes the file outside the loop ensuring it is not truncated
and contains the correct information.

(From OE-Core rev: a015881f2207aded601459ba3eebbefb0002b3c5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-14 08:52:47 +03:00
Richard Purdie
566628d8cd class/lib: Fix up various file access methods
There are various bits of cruft that have built up around our file accesses. This patch
cleans some of them up, specifically:

 * Remove pointless "from __builtin__ import file"
 * Use open(), not file()
 * Wrap file usage in a with container to ensure files are closed
 * Add missing .close() calls in some cases

(From OE-Core rev: a43e0a8ecd0441131e929daf998c3cd454d9c8f3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09 22:28:04 +01:00
Richard Purdie
b54339d633 classes/lib: Fix getcmdstatus breakage
I mistakenly thought subprocess had getcmdstatus in python 2. It doesn't so lets
add a wrapper and have this work in both worlds.

(From OE-Core rev: 2253e9f12734c6e6aa489942b5e4628eca1fa29d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09 16:05:08 +01:00
Richard Purdie
e16faa55da sanity/patch.py: Remove commands module usage
The commands module is removed in python3. Use the subprocess module instead
and the pipes module to replace the mkargs usage.

(From OE-Core rev: e2e1dcd74bc45381baccf507c0309dd792229afe)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09 14:06:51 +01:00
Richard Purdie
2ac4f8b397 clases/lib: Use modern exception syntax
Update older code to use modern exception handling syntax which
is the form accepted by python 3.

(From OE-Core rev: b010501cd089e649a68f683be0cf4d0aac90fbe3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09 14:06:50 +01:00
Christopher Larson
824962fcf2 package.bbclass: ensure license excluded files aren't packaged
An excluded package left its files behind, which meant they could end up in
another package instead, meaning we could ship GPLv3 binaries even with GPLv3
in INCOMPATIBLE_LICENSE. Skip the files belonging to the excluded packages to
prevent this from occurring.

(From OE-Core rev: c045bfe6b991006ac80f0e2d06a8917ae58d9262)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-04-22 14:45:04 +01:00
Christopher Larson
fd4f354853 package.bbclass: add LICENSE_EXCLUSION to vardeps
Ensure that changes to INCOMPATIBLE_LICENSE re-run do_package for affected
recipes.

(From OE-Core rev: b9b187f8a0f513e676cd3d18f71d1055445f29fa)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-04-22 14:45:03 +01:00
Richard Purdie
8e74a9795c package.bbclass: Add useradd variables to PACKAGEVARS
THe USERADD_PARAMS and similar group variables are package specific variables
which should get added to the vardeps of the packaging process. This
change also ensures they get remapped correctly by the multilib code.

(From OE-Core rev: 805add18e7b917aaef61c82c0b5f2e9682af1d0d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-04-18 21:12:19 +01:00
Richard Purdie
79a205be0d package.bbclass: Fix populate_packages for glob expansion issues
If we put a valid glob like "*/foo/*" into FILES, populate_packages
breaks with a "file exists" message. This is because the glob expansion
does not have "./" prefix however there may already be an entry in
the seen list which does have such a prefix. The easiest/simplest fix
right now is to add the prefix if it doesn't exist which only happens
for certain globs.

(From OE-Core rev: 138c5f3af892e33f576fc7dd268e122b179f82a1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-04-18 00:37:57 +01:00
Richard Purdie
093dec12e6 package/image.bbclass: Fix multilib rprovides
allarch multilib recipes are meant to provide a list of different multilib variants.
Unfortunately since the pkgdata also has mappings for these, they get mapped back to
the original package name which means the effect is undone at package creation time
when the remapping code is called.

This patch adds in a conditional to break that chain meaning the packages get
the correct RPROVIDES and image builds work correctly with opkg.

[YOCTO #3453]

(From OE-Core rev: 1a1927f8a04fe0a2b3b853ebdd33ccb807f00b59)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-04-13 23:49:28 +01:00
Andrei Dinu
4dbdcf9462 Add file information to package information window
Removed the package files parsing routine from the
packageinfo.bbclass file and added it to the
package.bbclass file.

(From OE-Core rev: 225e7826b0d082f43db82201e826b98b3a95cd57)

Signed-off-by: Andrei Dinu <andrei.adrianx.dinu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-04-04 14:10:56 +01:00
Richard Purdie
3a78a23e76 package.bbclass: Correctly handle /usr/src/debug file ownership
After the recent cached path operation improvements, we need to tell
the cache about directory changes. /usr/src/debug was was cached as
not created and this was leading to bugs in file ownership. This
change updates the cache and avoids this issue.

(From OE-Core rev: 553a8010e81448629a94ab04fd989e6e49066f2b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-03-26 17:50:08 +00:00
Richard Purdie
c376f1f49c package.bbclass: Ensure debug source file is generated correctly
The command for generating debug sources appends to the file, therefore we need
to ensure it does not exist before we run the command else we end up doing
more work that we should have to.

(From OE-Core rev: 48dfd61901ce07491ef913cf04dbd9de9a52759f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-03-25 17:42:58 +00:00
Richard Purdie
7a8f3f7a01 package.bbclass: Handle subprocess errors correctly
If an error occurs in subprocess.call() we currently don't catch it. In particular
we have issues where debugedit is segfaulting unnoticed. This fixes up
various code paths to catch the errors.

[YOCTO #4089]

(From OE-Core rev: 262a69ffd33e9d001a7a15fc73671a015e3b5dd1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-03-25 17:42:58 +00:00
Richard Purdie
add11fa1ab package: Add cachedpath optimisation
Currently, various standard library operations like os.walk(),
os.path.isdir() and os.path.islink() each call stat or lstat which
involves a syscall into the kernel. There is no caching since they could
conceivably have changed on disk. The result is that for something like
the do_package task of the kernel we're spending over two minutes making
868,000 individual stat calls for 23,000 files. This is suboptimal.

This patch adds lib/oe/cachedpath.py which are a set of replacement
functions for these operations which use cached stat data rather than
hitting the kernel each time. It gives a nice performance improvement
halving the build time of the kernel do_package.

(From OE-Core rev: 556dee0c4d6d8a87c0cddbd2f60fe5917d009f18)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-03-18 21:26:40 +00:00
Martin Jansa
92281f9998 package.bbclass: add debug-without-src PACKAGE_DEBUG_SPLIT_STYLE
* same as original and default version, but does not package source files in PN-dbg

(From OE-Core rev: 3c8452c3abae74a42989c0fbd5ba303788528750)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-03-15 01:53:03 +00:00
Richard Purdie
81ac275984 package.bbclass: Ensure all .so files get stripped
It was realised that .so files which were not marked as executable were not
gettings stripped. This was wasting space in images. This patch ensures
they do get processed by the code correctly.

[YOCTO #3973]

(From OE-Core rev: 725354886ae3650a7a4875d4c0bffcfab7e8cc40)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-03-05 13:14:58 +00:00
Richard Purdie
1122a417de package.bbclass: Fix do_package variable dependencies
When the do_package function was split into different sections, the vardeps
were not updated to match meaning some function changes weren't reflected
in the sstate checksums. Fix this.

[YOCTO #3980]

(From OE-Core rev: 2e7ddbcca3f613c3b5725dcc3d457985d41e5f16)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-03-05 13:14:57 +00:00
Song.Li
4878b3ced8 pakcage.bbclass:use a better way to cut -dev/-dbg suffix
In pakcage.bbclass,function replace('-dev', '') is
used to cut '-dev' suffix. but it will cause the
name "lib32-device-dev" to "lib32ice",
so change it to [:-4].

(From OE-Core rev: 91a3a48798029ca88a46a528200f4c5252f245e9)

Signed-off-by: Song.Li <Song.Li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-22 06:09:19 -08:00
Richard Purdie
069a332d1c classes: Drop none package specific packaging variable accesses
(From OE-Core rev: 24cbe88429ba021d7c461e0271b67106d11d5eb2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-17 22:32:03 +00:00
Enrico Scholz
d518019d6c package.bbclass: support dangling path components
Commit ec2aab09769f4b6817d74d2175afa2b7c7598750 introduced a regression
on packages which contain symlinks with unresolvable path components
(e.g. lsof-dbg).  While assigning a variable, an exception was raised
and the exception handler accessed this variable.

Patch deals both with the dangling path components by assuming them as
valid directories and by avoiding the broken assignment.

(From OE-Core rev: 579369b0fb27fad6d628746a50b9b798078500f6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-11 22:45:25 +00:00
Enrico Scholz
f2e16c6552 package.bbclass: use oe.path.realpath()
oe.path.realpath() provides are common and more correct implementation
for resolving symlinks within sysroot. Use it.

Old implementation suffered from lot of problems; e.g.

* redundant code

* calls 'os.stat()' which references files on host; this can give wrong
  results about existing/non-existing and can cause EPERM (instead of
  the catched ENONENT) exceptions

* does not deal with special cases like '..' leaving the sysroot.

(From OE-Core rev: ec2aab09769f4b6817d74d2175afa2b7c7598750)

Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-11 14:46:11 +00:00
Khem Raj
43bc21462b package.bbclass: Replace undefined 'src' with valid variable
After rewrite of split_and_strip_files src var is no longer
available here where it is used so we should replace it with
a valid value

(From OE-Core rev: b07e79ea049d341a5a0a7e4bb72151f49336c5bc)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06 23:37:10 +00:00
Richard Purdie
7980773d90 package.bbclass: Ensure package_get_auto_pr is run at the correct point
This needs to run before PKGR and other variables using PRAUTO are used,
including the expansion optimisations.

(From OE-Core rev: eb09ebc9680eb50e23778dcc1c2697d13cf073ae)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06 16:52:20 +00:00
Martin Jansa
451f2d083d package.bbclass: return list of packages created in do_split_packages
* sometimes it's useful to do something with packages created
  by do_split_packages later in do_package_prepend, e.g. in:
  http://lists.linuxtogo.org/pipermail/openembedded-devel/2013-February/043824.html
  I have .bbclass which adds some postinst, postrm and RDEPENDS,
  but ttf-mplus is using do_split_packages to create those packages

(From OE-Core rev: 5aa52c6882d2929b2e530c4fa297c6d3e97d4f9f)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06 13:16:51 +00:00
Richard Purdie
5b7c822609 package.bbclass: Pre-expand some variables to save time
(From OE-Core rev: fc5bff0145d8f5db1c09be61f5de209ac4016ed1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06 13:13:01 +00:00
Richard Purdie
78955b837c package.bbclass: Better document the different phases of operation
Add headers to document the different phases of do_package and
make the steps clearer.

(From OE-Core rev: b6438c94035a014902ec89af63ff3787cd8c67f6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06 13:13:01 +00:00
Richard Purdie
4cae28e309 package.bbclass: Add PACKAGESPLITFUNCS variable
Prepending to populate_packages is rather ugly and means its hard to trace
errors and also profiling informaiton is summed together in one function.

This patch starts to split out the prepends to become separate functions
to avoid these issues. This is generally a neater way to write functions
than prepending to where there can sometimes be variable scope issues
and we've been bitten by whitespace issues in the past.

(From OE-Core rev: 4f9963d1d82ee896fe9491d6a8b32be42cd06f14)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06 13:13:00 +00:00
Richard Purdie
1c906e7643 package.bbclass: Simplify empty directory removal
Rather than an exec() per directory, we might as well exec one command and
be done with it.

(From OE-Core rev: 82ae9cfb09ee5c0aa6402c972d71e2b64d1ce8bc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06 13:13:00 +00:00
Richard Purdie
48f8eece27 package.bbclass: Various minor performance tweaks
* Call getVar outside the loop
* Drop unneeded PATH export (bitbake does this already)
* Drop unused variable
* Simplify if statement nesting
* Simplify variable expandion to a getVar call (expand would just call getVar)

(From OE-Core rev: 52b506145bcddc133ca93a8c9f7343de69d10907)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06 13:13:00 +00:00
Richard Purdie
ee5e443dba package.bbclass: Make use of cleandirs and dirs function flags
We can use the cleandirs and dirs flags for the fuctions to handle
directory cleaning and creation at the bitbake level rather than
using these calls within the functions

(From OE-Core rev: 4b31d6f6f0a2a6b9e504ffae0d3b2099cbd7dddc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06 13:13:00 +00:00
Richard Purdie
85b4891a1c package.bbclass: Rewrite split_and_strip_files
The split_and_strip_files funciton was hard to follow and its usage of prefixes
to strings was unusual. This rewrites it to use a list of hardlinks, symlinks and
elffiles where each list is iterated over at the correct point.

This means we can avoid creating dandling symlinks for example so we can simply
delete the cleanup code for this.

The isfile() check is also removed which gives a significant improvement in speed.
Its uneeded since os.walk will have already checked things in files are files.

(From OE-Core rev: 0cd295d8cdc8cc39d6b6c7d26ea8a2a10a979d7c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06 13:13:00 +00:00
Richard Purdie
f2ee5b48d9 package.bbclass: Fix up bb.mkdirhier/bb.copyfile usage
These are in bb.utils so lets the correct function and avoid the overhead
of the fixup/warning for the deprecated usage.

(From OE-Core rev: d17329db4842c50af1a3d7f5f20e692c89913fba)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06 13:13:00 +00:00
Richard Purdie
bf25dd25c2 package.bbclass: use the multiprocess pool from bitbake
(From OE-Core rev: 7e880a95840db82f4035959b03630ba2a96c0311)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06 13:12:59 +00:00
Richard Purdie
b34fd60d8f package: Process package stripping in parallel
(From OE-Core rev: 981fed49ee80560fb067b3f47aeada1fdee792ca)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06 13:12:59 +00:00
Richard Purdie
e45db24bbe package: Don't export PATH
PATH is already exported, we don't need to do this each time we run
something, its just noise and overhead.

(From OE-Core rev: 060f617cea4ea0a5af28d31ea19c0387e9773fce)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06 13:12:59 +00:00
Richard Purdie
d81aa06ece package.bbclass: Multithread per file dependency generation code
(From OE-Core rev: b659eb0f2070149d9516c129b3853b41fbbd1033)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06 13:12:59 +00:00
Peter Seebach
4872ba3ee4 package.bbclass: Allow per-package SKIP_FILEDEPS
The existing check for SKIP_FILEDEPS can be overridden per recipe
using SKIP_FILEDEPS_pn-${PN}. However, there's no mechanism for
letting a single package within a recipe use SKIP_FILEDEPS.

This patch adds SKIP_FILEDEPS_<pkg>, by analogy to FILES_<pkg>.
Note that it only works one way; if the recipe has SKIP_FILEDEPS = 1,
the checks for individual packages will never be reached.

(From OE-Core rev: 94557b500ad38a49aec40629015ed0b24e167f76)

Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-04 13:18:28 +00:00
Richard Purdie
26ea367852 package.bbclass: Use expanded RDEPENDS in read_shlibdeps
We may as well expand the RDEPENDS when reading and writing as this function does.
if we don't do this, we could accidentally duplicate data and it also turns out
to be much less efficient.

(From OE-Core rev: d62a3151e7b252911c172a605b3089de355663c5)

(From OE-Core rev: 8f602e84e6ae4e346acef7cf5473343039fa352a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-01 15:54:02 +00:00
Richard Purdie
244f107c1f package: Create global package file list and use throughout PACKAGEFUNCS
Currently we do a signficant amount of tree traversal in many different places
which in inefficient. We can assume that the files don't change and cache the
file list which gives an efficiency improvement which this patch does using
a global variable.

(From OE-Core rev: 2d7608842d2dab07065e60aab729a5c8fd6b7907)

(From OE-Core rev: 5c7c8347eb1bc25d194be6f4be142ba0924e2600)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-01 15:54:02 +00:00
Richard Purdie
ed931d0314 package.bbclass: Improve package_fixsymlinks
Improve package_fixsymlinks so we don't handle RDEPENDS for every single package
in PACKAGES.

(From OE-Core rev: 20ff8feb95d54e4db646f8c0cb006ce187e288af)

(From OE-Core rev: ca2ee871f82dd0ba4122a8373e4efd21cec5722b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-01 15:54:02 +00:00
Richard Purdie
398d62fd74 package.bbclass: Tweak PACKAGEFUNCS
Move package_rename_hook call into PACKAGEFUNCS and also move package_get_auto_pr
to a more appropriate execution point, grouping package metadata handling
functions together.

(From OE-Core rev: a3f41cfbc8923e54198d10db292a11ef2edda4d7)

(From OE-Core rev: 96d268166ab0d4640767bf3a0886358dcb663d0f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-01 15:54:02 +00:00
Richard Purdie
b369a3619e package.bbclass: Split out package_fixsymlinks function
The symlink handling code doesn't need to being part of populate_packages
and is logically separate so split it out into a separate function,
package_fixsymlinks.

(From OE-Core rev: fc964ed2b558b08327e2bc8c93ada80ca9d97337)

(From OE-Core rev: 08a194c13857b02a2febf9b1484b3cdd67a18160)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-01 15:54:02 +00:00
Richard Purdie
f955e4e482 package.bbclass: Move PKG handling code to emit_pkgdata
We only use the PKG variable in emit_pkgdata so we might as well move the
fallback code there, allowing restructuring of other parts of the metadata.

(From OE-Core rev: 0136ae9a5f719f0e7ba6e00bfd366c0e61b3b3f2)

(From OE-Core rev: df38695f0fbea10289e85fdcb7c2bdf566990577)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-01 15:54:01 +00:00
Richard Purdie
8a0b4578ec package.bbclass: Avoid copying the datastore for FILES handling
There is no real point in adjusting overrides and creating a copy of the datastore,
just to access a single variable. We can do this just as easily with a slightly
more complicated getVar call. This improves performance.

(From OE-Core rev: 69f4351809359a0c7c38e8f233f3e3f7680ed2e2)

(From OE-Core rev: b5d65f5d5e5b26d3a2c673e899114c90bcaf6bc8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-01 15:54:01 +00:00
Richard Purdie
b0a2f55ccc package.bbclass: Check FILES once, not once per loop iteration
There is no need to check FILES in each loop iteration, we can just check it once
at the start when we read the variable.

(From OE-Core rev: 1bcc5cba12cbb1b846b433719d1b0820f5a97105)

(From OE-Core rev: 40542f7a46bfb8431e31eca9af06adb4b343d810)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-01 15:54:01 +00:00
Richard Purdie
ae8586c952 package.bbclass: Add a shortcut exit from the mkdir function
The mkdir function iterates over strings with many different operations,
even if ultimately the target already exists. This adds a check to the start
of the function so we don't waste time when the target already exists.

(From OE-Core rev: 06e188e673313f1eb9ae7d85ae64467c8d2a94b3)

(From OE-Core rev: 9509627e06ab4cf18c5bbf3f405ecf19a9f40287)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-01 15:54:01 +00:00
Richard Purdie
afb1b3fac8 package.bbclass: Improve kernel module handling
Currently the kernel module handling consists of several special cases
and has its own path walking. This refactors the code to handle them in
a more standardised way which is also a bit more efficient.

(From OE-Core rev: ad51b54f0afe8c56033137b7cf9ba398877b2651)

(From OE-Core rev: cb24a72e97afb43de5e1e79ff807fd9e184df6a2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-01 15:54:01 +00:00
Richard Purdie
fed8dfff1f package: Rename splitfile functions to something more descriptive
The splitfile and splitfile2 function names are confusing and the comments
are also misleading, hard to understand or plain incorrect. This tries to
improve things.

(From OE-Core rev: 46f3050a1f46f814e2d031e7e71600b2932d631c)

(From OE-Core rev: bc6f7085b42d8137957795e800e8b46d2f1eddac)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-01 15:54:01 +00:00
Andreas Müller
b71a16e8ac package.bbclass: remove RRECOMMENDS on base packages for locale packages
As long as we support only one locale package per recipe this leads to unwanted
packages in images. This patch was motivated by samba (meta-oe): only wanting
libsmbclient one gets full samba server into the image. For my standard image
the follwing package were removed by this patch:

-acl_2.2.51-r3_armv7a-vfp-neon.ipk
-alsa-states_0.2.0-r3_armv7a-vfp-neon.ipk
-alsa-utils_1.0.25-r3_armv7a-vfp-neon.ipk
-alsa-utils-aconnect_1.0.25-r3_armv7a-vfp-neon.ipk
-alsa-utils-alsaconf_1.0.25-r3_armv7a-vfp-neon.ipk
-alsa-utils-alsactl_1.0.25-r3_armv7a-vfp-neon.ipk
-alsa-utils-alsaloop_1.0.25-r3_armv7a-vfp-neon.ipk
 alsa-utils-alsamixer_1.0.25-r3_armv7a-vfp-neon.ipk
-alsa-utils-alsaucm_1.0.25-r3_armv7a-vfp-neon.ipk
-alsa-utils-amixer_1.0.25-r3_armv7a-vfp-neon.ipk
-alsa-utils-aplay_1.0.25-r3_armv7a-vfp-neon.ipk
-alsa-utils-aseqdump_1.0.25-r3_armv7a-vfp-neon.ipk
-alsa-utils-aseqnet_1.0.25-r3_armv7a-vfp-neon.ipk
-alsa-utils-iecset_1.0.25-r3_armv7a-vfp-neon.ipk
-alsa-utils-midi_1.0.25-r3_armv7a-vfp-neon.ipk
-alsa-utils-speakertest_1.0.25-r3_armv7a-vfp-neon.ipk
-attr_2.4.46-r4_armv7a-vfp-neon.ipk
-avahi_0.6.31-r7.1_armv7a-vfp-neon.ipk
-gtk-engines_2.20.2-r3_armv7a-vfp-neon.ipk
-pulseaudio_2.1-r15_armv7a-vfp-neon.ipk
-samba_3.6.8-r6_armv7a-vfp-neon.ipk
-vte_0.28.2-r5_armv7a-vfp-neon.ipk
-xz_5.1.2alpha-r0_armv7a-vfp-neon.ipk

Strange effect: For samba buildhistory's depends.dot says:
-samba -> libc6;
-samba -> libgcc1;
-samba -> libreadline6;
-samba -> libtalloc2;
-samba -> libtdb1;
-samba -> libwbclient0;
-samba -> libz1;
-samba -> update_rc_d;
-samba_locale_de -> samba [style=dotted];
 samba_locale_de -> virtual_locale_de [style=dotted];
-samba_locale_fr -> samba [style=dotted];
-samba_locale_fr -> samba [style=dotted];

This is conflicts to buildhistory's result on samba package not mentioning
any changes for samba package.

(From OE-Core rev: 50354ebece43a6c13f9ca592e7d230765a26fdf8)

Signed-off-by: Andreas Müller <schnitzeltony@googlemail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-01-28 12:29:30 +00:00
Richard Purdie
3247292683 Split do_packagedata task from do_package
Currently, do_rootfs has a dependency on all the do_package output being present
due to its usage of the pkgdata directories. This means that if you run:

bitbake xxxx-image -c rootfs

you end up having to fetch and unpack all the do_package data which is usually
large and inefficient. It also means rm_work has to leave all the do_package
data lying around so rootfs works.

This patch splits the actual creation of the pkgdata directory off into a separate
task, "packagedata" which happens immediately after do_package. We can then remap
the dependencies so this task is depended upon, not do_package. Sstate can then be
programmed not to require do_package at the appropriate times.

Whilst this patch doesn't do so, it opens the possibility of rm_work wiping
out the do_package output from WORKDIR as long as it also removed the do_package
stamp (both normal and setscene variants) and allowing more space savings
with rm_work which has been regularly requested.

(From OE-Core rev: 6107ee294afde395e39d084c33e8e94013c625a9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-01-25 12:42:47 +00:00
Constantin Musca
8aa508970a prserv: change PRSERV_HOST semantics
- remove PRSERV_PORT variable
- use 'hostname:port' as PRSERV_HOST format
- remove USE_PR_SERV variable
- one can activate PRS by setting PRSERV_HOST

[YOCTO #3744]

(From OE-Core rev: e00f49de8b1f79c3e07b887d257bd75a46052fa0)

Signed-off-by: Constantin Musca <constantinx.musca@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-01-25 12:42:43 +00:00
Christopher Larson
f63f387b83 package.bbclass: drop unnecessary and broken packages.remove()
(From OE-Core rev: 5480371d0473482437ce4350e6593f006101ef40)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-01-21 10:22:08 +00:00
Richard Purdie
eabd8606f9 package.bbclass: Fix shlibs cross package-arch contamination
I found that if I had two packages architectures built (say core2 and i586),
that the shlibs data from i586 was being used in the core2 build. This
lead to odd dependency issues if the i586 build was out of date and core2
was being targetted.

When loading shlibs files, the last loaded wins so we need to iterate the
lists from least to strongest weighting, not the other way around. With this
patch applied, the contamination issue is resolved.

(From OE-Core rev: acd7b1c2ee9f3ff84c45a723dba5e0bc5f6a100c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-01-20 12:58:58 +00:00
Enrico Scholz
116fb4a72f package.bbclass: replace AUTOINC directly in PKGV
b27b438221e16ac3df6ac66d761b77e3bd43db67 introduced a regression when
PKGV is modified by the recipe.  E.g. when having something like

| PV = "1.7.33+gitr${SRCPV}"
| PKGV = "1.7.33+gitr${GITPKGV}"
| RDEPENDS_${PN}-subpkg += "${PN} (= ${EXTENDPKGV})"
| inherit gitpkgv

The resulting core package has

| Version: 1.7.33+git234+cd01e8c-r11

and because the subpackage has

| Depends: vdr (= 1.7.33+git0+cd01e8c18d494150730214f033f84e5282ce7639-r11)

image build fails with

|  * satisfy_dependencies_for: Cannot satisfy the following dependencies ...
|  *    vdr (= 1.7.33+git0+cd01e8c18d494150730214f033f84e5282ce7639-r11)

Patch avoids the 'PKGV = "cleanup(${PV})"' operation and replaces
AUTOINC directly within PKGV.  This might introduce a regression too
when 'AUTOINC' is added to PKGV intentionally but impact is small and
can be fixed trivially by using another string.

(From OE-Core rev: 6e46664085fa49bc204fc944ca23ed6383e2ec51)

Signed-off-by: Enrico Scholz <enrico.scholz@informatik.tu-chemnitz.de>
Cc: Constantin Musca <constantinx.musca@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-01-20 12:58:57 +00:00
Robert P. J. Day
faf94a9d6d package.bbclass: Skip testing "packages" a second time.
Since the "packages" variable is already tested a few lines above in
the same routine:

    if len(packages) < 1:
        bb.debug(1, "No packages to build, skipping do_package")
        return

there is no point testing it again, and the error message doesn't even
refer to "packages", anyway.

(From OE-Core rev: 371fbacff47828cfff99f1152f9e269e27c474da)

Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-01-07 11:09:44 +00:00
Constantin Musca
94516ced8d package.bbclass: don't prepend MLPREFIX to LOCALEBASEPN
- all the recipes that overwrite LOCALEBASEPN must consider
also the MLPREFIX
- if the LOCALEBASEPN variable is not overwritten then it will
have the correct prefix (LOCALEBASEPN ??= "${PN}")

(From OE-Core rev: 855d64a3091c5358e2e1a7f4247929515b068708)

Signed-off-by: Constantin Musca <constantinx.musca@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-01-07 11:09:37 +00:00
Constantin Musca
1674541ed8 multilib: fix allarch/kernel/module-base multilib issues
- skip the non-packagegroup allarch recipes in multilib_virtclass_handler
- extend PROVIDES/RPROVIDES for allarch recipes which are not packagegroups
- use variants from MULTILIB_GLOBAL_VARIANTS (lib32 lib64 libx32) to create
additional pkgdata files for multilib allarch: ${pkgdatadir}/${variant}-${PN}
and ${pkgdatadir}/runtime/${variant}-${pkg}
- use variants from MULTILIB_VARIANTS to create additional pkgdata files
for multilib kernel/module-base recipes
- add a sanity check to determine if the current multilib is in
MULTILIB_GLOBAL_VARIANTS

[YOCTO #2918]
[YOCTO #3440]
[YOCTO #3565]
[YOCTO #3568]

(From OE-Core rev: bc4da2573dfb59ea2fc4af359701818df20f7663)

Signed-off-by: Constantin Musca <constantinx.musca@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-12-31 09:43:25 +00:00
Constantin Musca
9240695462 package.bbclass: prepend MLPREFIX to LOCALEBASEPN
We need to prepend MLPREFIX to LOCALEBASEPN in order to
fully enable multilib

[YOCTO #3440]

(From OE-Core rev: 91fe09a9a8b317d27145efff4201f1138d6a45d4)

Signed-off-by: Constantin Musca <constantinx.musca@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-12-11 15:58:35 +00:00
Ross Burton
3488973587 package.bbclass: make do_split_packages handle non-existent root directories
This function has different behaviour if the split directory doesn't exist
depending on the recursive argument: non-recursive uses os.listdirs which throws
an exception, recursive uses os.walk which doesn't.

do_split_packages should silently handle non-existent directories because it's
mainly used for plugin directories, which may end up being empty though changing
the distro configuration (for example, connman without wifi distro feature).

So, add an early exit if the split root doesn't exist.

(From OE-Core rev: 937101e3fdd6afd00f6f8a8be411a67110c4ae78)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-12-11 15:54:36 +00:00
Constantin Musca
59a704679c prs: use the PRServer to replace the BB_URI_LOCALCOUNT functionality
- adapt the PRServer to generate an incremental number for every revision set
associated with a (PN, PV)
- use one incrementing value rather than several
- PV becomes 0.1+gitAUTOINC+deadbeefdecafbad_decafbaddeadbeef
- if PV contains 'AUTOINC' set PKGV to the same thing with AUTOINC replaced
with the "auto PR" value
- if the PRServer isn't running replace AUTOINC with 0 (no package feeds, but usual
image creation would work)
- use the PR table with "AUTOINC-${PV-without-SRCREVs}" as key
- modify prexport/primport to address the generated srcrev entries
- this patch addresses the following proposal:
http://lists.linuxtogo.org/pipermail/bitbake-devel/2012-November/003878.html

(From OE-Core rev: b27b438221e16ac3df6ac66d761b77e3bd43db67)

Signed-off-by: Constantin Musca <constantinx.musca@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-12-06 12:31:16 +00:00
Richard Purdie
212397c87b package.bbclass: Adapt debugsrc code to workdir layout changes
With the change to WORKDIR layout, splutting PN and PV into two directories,
the debugsrc splutting code layout became suboptimal. This changes things to
include the information as it was before. Ideally this code would be written
to more generically support other layouts buts it not clear that the tools
would even support that right now so this is the best immediate fix.

(From OE-Core rev: 432cfbb403f0e864d1fad383c2bbb6f9bdb80770)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-11-21 09:10:02 +00:00
Richard Purdie
d59d4306fc package: Hardlink debug source to improve performance
When copying the source files needed for the -dbg package, use hardlinks
where possible. This saves some disk space and hence helps performance
of the builds.

(From OE-Core rev: 6775feb9fe935ab01fd9cae2b2d3fce5824a9a72)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-10-23 12:28:31 +01:00
Richard Purdie
92b44ec00e package.bbclass: Exclude the PKGTRIPLETS variable
Without this, we'd rerun packaging for every machine since this variable contains
a machine specific component.

(From OE-Core rev: 61131828c59178c923b3d5b5fcacf0dbcba275a5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-10-22 14:59:29 +01:00
Richard Purdie
a68c7c792e package.bbclass: Switch shlibs to pkgdata directory and make package non-machine specific
Currently, do_package is machine specific since the shlibs data is installed
into each machine specific sysroot. This change moves the shlibs data to the
pkgdata structure, at the expense of having to iterate over a set of shlibs
directories instead of a single one.

It turns out this isn't any particular hardship for the code and as a result,
do_package stops being machine specific leading to optimisations for builds
that use a common PACKAGE_ARCH.

(From OE-Core rev: cc088489d70fb27d460c3dbe35d6ea382123c134)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-10-22 09:47:47 +01:00
Richard Purdie
a6f6d4faba package: Recommend virtual-locale-*, don't depend on it
The virtual-locale-* packages are provided by libc which may or
may not have a matching locale for any given recipes's provided locales.
Certainly, we shouldn't get a failure if the locale package isn't
available.

This patch therefore makes the dependency optional rather than
required, allowing the locale packages in question to install. This
resolves errors like:

	virtual-locale-eo is needed by bash-locale-eo-4.2-r5.i586
	virtual-locale-en+boldquot is needed by bash-locale-en+boldquot-4.2-r5.i586
	virtual-locale-en+quot is needed by bash-locale-en+quot-4.2-r5.i586

(From OE-Core rev: 2be67f95abaa7e8655a1ca8f4ffe66b3e099a650)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-10-18 12:13:12 +01:00
Richard Purdie
71030c6b37 package.bbclass: Fix regression in -dbg packages introduced by explode_dep_versions change
We need to iterate over the dictionary pulling out the values, not take the top
level keys. If we don't do this, we end up with dependencies on the values of
PACKAGES, not library dependencies.

(From OE-Core rev: 7219bca11f554fbe2ed30f1537491987d65e9316)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-10-03 10:02:22 +01:00
Richard Purdie
5fdbda6922 classes: Update to use corrected bb.utils.explode_dep_versions2 API
The bb.utils.explode_dep_versions function has issues where dependency information
can be lost. The API doesn't support maintaining the correct information so this
changes to use a new function which correctly handles the data.

This patch also fixes various points in the code to ensure that we do not have any
duplicates in things that use explode_dep_versions.

A new sanity test to test the contents of the R* variables is also added.

[Some changes from Mark Hatle <mark.hatle@windriver.com>]

(From OE-Core rev: 16a892431d0c0d03f8b561b92909cf2f11af4918)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-10-02 11:40:53 +01:00
Paul Eggleton
272cb74d70 Rename task to packagegroup
"Package group" is a much more appropriate name for these than task,
since we use the word task to describe units of work executed by
BitBake.

(From OE-Core rev: 424dcf7046e4ad09dcc664eb1992201195247fcf)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-09-04 12:52:55 +01:00
Tomas Frydrych
4759a71d3b package.bbclass: fix stripping for kernel modules
runstrip(path, elftype, d) does bitwise & on the elftype parameter, so it has
to be passed an integer. Passing None fails with: TypeError: unsupported
operand type(s) for &: 'NoneType' and 'int'.

(From OE-Core rev: 67f21c6c698464959c27c7e65ee537ab4c378944)

Signed-off-by: Tomas Frydrych <tomas@sleepfive.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-09-02 05:52:09 -07:00
Paul Eggleton
bf64b6ec39 classes/package: document do_packages_split arguments
This function takes quite a number of arguments and can be tricky to use
properly; this is not made easier if it is undocumented, so document all
of the arguments. (No functional changes, comments only.)

(From OE-Core rev: 324fbcc047819b6948d13eda627ec898cb1f1eb8)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-22 13:59:54 +01:00
Richard Purdie
48619958d5 meta/classes: Various python whitespace fixes
It was pointed out we have a number of weird indentations in the python functions.
This patch cleans up 3, 7 and other weird indentations for the core bbclass files.

It also fixes some wierd (odd) shell function indentation which my searches picked up.

(From OE-Core rev: 8385d6d74624000d68814f4e3266d47bc8885942)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-21 12:15:30 +01:00
Laurentiu Palcu
21049ec964 package.bbclass: change RPATHs for cross-canadian binaries
When building the meta-toolchain, the binaries didn't get relocatable
RPATHs. They were hardcoded to the default path. Hence, if one had
already installed one SDK in the default path and one in another
location, the later toolchain's binaries would search and load libraries
from the first location, ending in a "Segmentation Fault".

[YOCTO #2927]

(From OE-Core rev: b40a03c43d5d9d738a9aa5b43b2ecfe74fc95018)

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-17 13:06:42 +01:00
Paul Eggleton
06ad59cca9 classes/package: don't make locale pkgs depend on PN pkg if not in PACKAGES
If ${PN} is not in PACKAGES then don't add it as an RDEPENDS for each
locale package. This fixes the installation of gcc-runtime locale
packages, for example.

(From OE-Core rev: d36c3235b3022c07f064929f55114f808a7634f6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-06 15:35:45 +01:00
Laurentiu Palcu
e3a13b8d93 package.bbclass: change RPATHs for nativesdk packages
Change binaries RPATHs, to include $ORIGIN, to make them relocatable.

(From OE-Core rev: f6362c5248914485fc8bf3c3f0ca6cc962bf753e)

Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-02 15:28:39 +01:00
Martin Jansa
5ab3a11149 package.bbclass: fix TypeError in runstrip
* some packages have .ko files which are not elf, without this change
  it fails with TypeError, with this change only runstip fails and
  reports where:
  ERROR: runstrip: ''arm-oe-linux-gnueabi-strip'  '/OE/shr-core/tmp-eglibc/work/armv4t-oe-linux-gnueabi/emacs-23.4-r0/package/usr/share/emacs/23.4/etc/tutorials/TUTORIAL.ko'' strip command failed

(From OE-Core rev: a834ab8a6d53cae504fa112a89bab93d726539ec)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-31 11:59:05 +01:00
Paul Eggleton
22965c4715 classes/package: ensure pkgdata is cleaned out first
If do_package needs to be re-run and packages are removed/renamed as a
result, we want the corresponding stale pkgdata files to no longer be
present; for that we need to ensure all of the old pkgdata files are
cleaned out prior to creating the new ones. This is particularly
important now we rely on pkgdata more heavily (for complementary
packages, license manifest, etc.).

(From OE-Core rev: 5dee47b476a6f715ab02af5e758441a2a934eea3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-29 10:16:16 +01:00
Richard Purdie
754d240d2c package.bbclass/eglibc-locale: Ensure the correct PN is used for locale packages
(From OE-Core rev: fc4b138948656d482603b80f8c8e86fdbb185df4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-28 09:20:53 +01:00
Paul Eggleton
a5227e205b classes/package: reduce dbg package dependencies
Make dbg package dependencies added via DEPCHAIN_POST less aggressive -
only add dependencies on dbg packages for shared library dependencies.
This avoids dragging in extraneous packages (such as eglibc-dbg forcing
bash-dbg to be installed) whilst preserving the ability to easily debug
into shared libraries in use by an application by just requesting the
installation of the single dbg package for that application.

For task recipes however we preserve the previous behaviour, since when
you install task-xxx-dbg you expect the dbg packages for every package
in the task to be installed. Unfortunately not all of our tasks inherit
from task.bbclass so we just use a name match - this should be tidied up
later.

Fixes [YOCTO #2599].

(From OE-Core rev: 352522d474cb75992d7865545b6fbe4e157a5f99)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-28 09:20:52 +01:00
Paul Eggleton
e95738004f classes/package: do not add self-dependencies to dev/dbg packages
Avoid adding dependencies to dev and dbg packages on themselves via the
DEPCHAIN_* handling code. These don't actually cause any problems, but
they do complicate dependency graphs.

(From OE-Core rev: ecdf7874470cff24cb23b7fd2723096b15aead31)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-26 18:53:04 +01:00
Andrei Gherzan
5334a93d26 package.bbclass: Create symlinks for packages with different packaged name
While generating license.manifest package information is searched in:
filename=`ls ${TMPDIR}/pkgdata/*/runtime/${pkg}| head -1`
This is ok as long as package name is the same as the package name
after packaging.
For example dbus is packaged as dbus-1. So, searching
ls ${TMPDIR}/pkgdata/*/runtime/dbus-1 will fail because there is no file
with this package name.

Create a symlink to the pkgdata file in a runtime-reverse directory so
that these reverse lookups are possible.

Fixes [YOCTO #2638].

(From OE-Core rev: 76ef414b8a017557cf5f3ad7b3ca8a324f7aad29)

Signed-off-by: Andrei Gherzan <andrei@gherzan.ro>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-26 14:40:47 +01:00
Richard Purdie
73cf0335c4 Remove a number of unneeded import os/bb calls
The bb and os modules are always imported so having these extra import calls
are a waste of space/execution time. They also set a bad example for people
copy and pasting code so clean them up.

(From OE-Core rev: 7d674820958be3a7051ea619effe1a6061d9cbe2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-19 12:24:50 +01:00
Richard Purdie
bfd279de32 Convert tab indentation in python functions into four-space
(From OE-Core rev: 604d46c686d06d62d5a07b9c7f4fa170f99307d8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-19 10:24:50 +01:00
Peter Seebach
07a9e9ee48 package.bbclass: Allow overriding of debugedit starting path
In the usual case, the computed path used for debugedit to fix
up path references for the target filesystem is correct. However,
prebuilt binary components, such as prebuilt toolchains, can
have debug paths that do not reflect paths within the local
build directory. Providing an override lets us continue to use
the standard debugedit logic in package.bbclass.

(From OE-Core rev: 32b1fde106bf423873a4a56e1111f230494e2d4a)

Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-17 10:53:54 +01:00
Amy Fong
b1c2b57b3e package install failure on systems without /sbin/ldconfig
Package install failures due to issues in post install rules.

package_do_shlibs only looks for libraries in a directory "lib", this should be
modified to the variable baselib.

ldconfig_postinst_fragment failure observed on systems without /sbin/ldconfig
        [ -x /sbin/ldconfig ] && /sbin/ldconfig
results in the post install rule returning a failure. Modify to
        if [ -x /sbin/ldconfig ]; then /sbin/ldconfig; fi

(From OE-Core rev: 3f85ab6d0d05a4e34fe73fa343d2b3b942fc69b7)

Signed-off-by: Amy Fong <amy.fong@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-17 10:53:53 +01:00
Saul Wold
0a1d5f5977 package.bbclass: enable the use of package_qa_handle_error
This will allow the reporting of these errors as either WARNINGs (default)
or ERRORs if installed_vs_shipped is added to the ERROR_QA of the policy
file (such as a <distro_name>.conf file.

V2: found the code I had intended to send instead of that other junk,
    was just not watching what I pushed on that one, sorry. (this is edit in
    no in the actual commit message)

(From OE-Core rev: 20d4205a4e408b6a99db392d4df458156113106a)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-04 17:40:35 +01:00
Mark Hatle
63c6f46b22 package.bbclass: Allow a package to skip per file dependency generation
Most of the time skipping per file dependency generation is a bad idea, but
when building a nativesdk or similar you may be required to pickup host
dependencies.  These host dependencies can not always be reconciled within
the scope of other nativesdk components, so if we skip them we can facilitate
this unique situation.

(From OE-Core rev: a1ca125de55a8e7cab402e2ba5737a56d4e890c2)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-03 14:55:00 +01:00
Richard Purdie
58977a94dd package.bbclass: Add missing PKGR, PKGV, PKGE and ALTERNATIVE variables from PACKAGEVARS
These variables all take package name extensions and should be considered in sstate
dependency processing.

This may also fix some multilib alternatives issues (unconfirmed).

[YOCTO #2056]

(From OE-Core rev: 64422f7c5da160050a5454817c8fa9d070104b34)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-02 16:47:45 +01:00
Richard Purdie
b3e76b33f3 package.bbclass: Add warning about FILES containing '//'
'//' in a FILES variable causes hard to track down issues with
packaging. This adds a warning and attempts to auto-correct the issue to
try and make the problem more user friendly.

[YOCTO #2448]

(From OE-Core rev: 0e33e314b1d2d3779658696f13a274b20d698667)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-05-30 17:32:31 +01:00
Richard Purdie
0d9e893711 Revert "meta: replace os.popen with subprocess.Popen"
This reverts commit e83d8e58a6b107eea87df0ec233a1bc932b2c6e as the conversion
is not correct. Its replacing readlines() calls which generate an array with
what are effectively strings. There are split("\n") calls missing in many
cases so this needs to be reverted until it gets fixed.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-05-30 14:21:29 +01:00
Robert Yang
5996b2b58e meta: replace os.popen with subprocess.Popen
Replace os.popen with subprocess.Popen since the older function would
fail (more or less) silently if the executed program cannot be found

There are both bb.process.run() and bb.process.Popen() which wraps the
subprocess module, use it for simplifying the code.

Note: We don't need the "2>/dev/null" or "2>&1" since bb.process.run()
can handle it, it will raise exception when error occurs, we should
handle the exception ourselves if we want to ignore the error.

More info:
http://docs.python.org/library/subprocess.html#subprocess-replacements

[YOCTO #2454]

(From OE-Core rev: e83d8e58a6b107eea87df0ec233a1bc932b2c6ea)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-05-30 12:04:45 +01:00
Robert Yang
e40995e569 meta: replace os.system with subprocess.call
Replace os.system with subprocess.call since the older function would
fail (more or less) silently if the executed program cannot be found

More info:
http://docs.python.org/library/subprocess.html#subprocess-replacements

[YOCTO #2454]

(From OE-Core rev: a07d03cc6f67c88feb9813ae7deb6e4a93552dfe)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-05-30 12:04:45 +01:00
Mark Hatle
6ac7e9b598 update-alternatives.bbclass: Refactor the implementation
Refactor in order to:
  * Deprecate the old interfaces, but keep them for compatibility
  * Provide a new, interface -- capable of working with split packages
  * Each update-alternative will now set proper "per-file" provides

Note: this adds a warning message when the older deprecated behavior is
used.  The older behavior has been fully tested using oe-core.

(From OE-Core rev: 309117d26de6a87b16406a44a0cefcbaaf7b5d7a)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-05-30 12:04:37 +01:00
Mark Hatle
d03e29fec2 package.bbclass: Add additional debugging for dependencies
When trying to understand why a QA wanring such as:

ERROR: QA Issue: foo rdepends on bar-dev

it is very difficult to figure out where the bar-dev dependency
comes from, since many of them are added dynamically.

This adds a debug statement that says which dependency adds an
rdepends to the system.

Also, while doing this work, it was noted that the same dependencies
were being scanned for over and over.  Instead we shorten the list
by only added to the dep list if the dependency was not already there.

(From OE-Core rev: 257b9e371143421b78a6991ef5401e564918c164)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-05-25 11:18:03 +01:00
Lianhao Lu
8de572f95a (libc-)package.bbclass: Added MLPREFIX to locale packages.
Added multilib prefix to the locale related package names/dependencies.

(From OE-Core rev: a52f7cf2cc90d918e3250c410995dcc1f3bfd5ee)

Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-05-18 15:25:10 +01:00
Richard Purdie
c73779d150 package.bbclass: Drop dubious use of packages[0] from do_split_packages and do_split_locales
As the comment says, using packages[0] is rather broken and can be incorrect,
as demonstrated by the recent gdk-pixbuf change. Replacing it with PN is
a bit more correct and more likely to do what was originally intended
by this function.

(From OE-Core rev: 5d0144fe6bd1febc2d1e651bbcff891c80f20b88)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-05-12 11:23:35 +01:00
Richard Purdie
a246af9bf7 package.bbclass: Drop EXPORT_FUNCTIONS use against mapping_rename_hook
The usage of this function renaming and it being called using
bb.build.exec_func() causes needless indirection loops, confusing log
files and seems generally pointless.

This simplification makes the process much simpler and faster. I can't
come up with a good reason why the export_functions functionality is
needed for this function.

(From OE-Core rev: 74daad03ca29a03b0005f7d2b90a0347d5b583a5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-05-11 18:03:31 +01:00