Commit Graph

6348 Commits

Author SHA1 Message Date
Manjukumar Matha
c71dc290b0 kernel-fitimage.bbclass: Fix the dependency issue while generating fitimage_initramfs
When building fitimage_initramfs, the correct depedency is to build
after do_bundle_initramfs. We can run into the following dependency
issue

DEBUG: Python function extend_recipe_sysroot finished
DEBUG: Executing shell function do_assemble_fitimage_initramfs
aarch64-xilinx-linux-objcopy: 'vmlinux': No such file

This happens because initramfs renames vmlinux to vmlinux.bak while
generating vmlinux.initramfs, there is a chance that fitimage_initramfs
can also start during this process and create the above issue.

This patch resolve the dependency issue by running fitimage_initramfs
task after do_bundle_initramfs

(From OE-Core rev: 8f0bece39a634fce5bd882cbd9e289ea905a0b17)

Signed-off-by: Varalaxmi Bingi<varalaxm@xilinx.com>
Signed-off-by: Manjukumar Matha <manjukumar.harthikote-matha@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-18 10:33:13 +00:00
Richard Purdie
eea86c4f0a native: Enable RDEPENDS handling
Native recipes don't currently honour their RDEPENDS. In the case of
some python scripts this has started causing problems since whilst they're
not needed at build time (DEPENDS), they are needed at runtime.

We put off making this change due to circular dependency issues. I believe
the three such problems in OE-Core are now fixed, as is the dependency loop
identfication code in bitbake so its time to improve this situation.

[YOCTO #10113]

(From OE-Core rev: c62520b63284927e177831c351fafa4d2768cb1f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-17 10:19:59 +00:00
Richard Purdie
050aab7f45 package: Rework debug source file handling
Currently we parallel process the files we install running dwarfsrcfiles over
each one in parallel threads but requiring a lock to write the results to one
file. This is not ideal for performance and means we can't then use per file
data for other purposes such as source code license processing.

Rework the code so that the list of source files is generated per installed
file and is reusable.

The code still generates a null separated debugsources.list file since this
is used by a shell pipeline but it no longer needs locking.

(From OE-Core rev: 95de93988eb725c14102f642ebabff3920ae194f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-17 10:19:59 +00:00
Anuj Mittal
bd5c9ca7c8 meson.bbclass: point to llvm-config
Allow packages using llvm-config to find it using meson's dependency
interface.

(From OE-Core rev: e2e733c813e1eabaaff795a30b82e703dfeecfe7)

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-15 16:05:37 +00:00
Joshua Watt
8d47356ad2 classes/license_image.bbclass: Fix rootfs license file permissions
Fixes up the permissions on the license files when they are put on the
target file system so that they are readable by everyone. Previously,
they would have inherited whatever permissions the file had in the
recipe, which may not have been appropriate.

[YOCTO #13175]

(From OE-Core rev: 8190d192fceb9b0969385507d3d4bca7be75c810)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-15 08:17:49 +00:00
Robert Yang
407a02b057 base.bbclass: Check BuildStarted for HOSTTOOLS
There might be no bb.event.ConfigParsed event if bitbake server is running, so
check bb.event.BuildStarted too to make sure HOSTTOOLS_DIR exists.

Fixed:
$ export BB_SERVER_TIMEOUT=-1
$ bitbake quilt-native
$ rm -fr tmp
$ bitbake quilt-native
ERROR: Error running gcc  --version: /bin/sh: gcc: command not found

This error is caused by enable_uninative(), it runs twice (ConfigParsed and
BuildStarted), the error would happen when there is no ConfigParsed event
(no hosttools is created), but BuildStarted. This patch can fix the problem.

[YOCTO #13022]

(From OE-Core rev: da798db0a48282e3d4f58890a7aec42c3deff0b8)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-15 08:17:49 +00:00
Khem Raj
c6a243dbb2 image_types.bbclass: Set memory usage limit and CPU threads for xz
when building with opkg backend and huge packages e.g. chromium/llvm all
going in parallel, memory pressure causes xz to catapult with

do_package_write_ipk: Failed to create package, opkg-build failed with: xz: (stdin): Cannot allocate memory

since there are many tasks going on in parallel, xz adds to memory pressure
and it wants it all, put an upper limit for memory xz can use

We add a variable XZ_MAXRAM with 30% of RAM limit and can be customized
if builders have more memory one can set it like

XZ_DEFAULTS = "-M 0 -T 0"

(From OE-Core rev: 28b277a93a34bba033d9d0d9f3227c9453efd384)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-14 11:20:55 +00:00
Silvio Fricke
1689d590e5 image_types: add base64 conversion
Sometimes it is useful to have a base64 representation of an image.

(From OE-Core rev: 9d00b146c4125b195fac5a0743fe7d8e59edef7f)

Signed-off-by: Silvio Fricke <silvio.fricke@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-12 14:04:31 +00:00
Alexander Kanavin
e2c3247c23 python3: upgrade to 3.7.2
I took the same approach as the recent perl upgrade: write recipe from scratch,
taking the pieces from the old recipe only when they were proven to be necessary.

The pgo, manifest and ptest features are all preserved.

New features:

- native and target recipes are now unified into one recipe

- check_build_completeness.py runs right after do_compile() and verifies that
all optional modules have been built (a notorious source of regressions)

- a new approach to sysconfig.py and distutils/sysconfig.py returning values
appropriate for native or target builds: we copy the configuration file to a
separate folder, add that folder to sys.path (through environment variable
that differs between native and target builds), and point python to the file
through another environment variable.

There were a few other patches where it was difficult to decide if the patch
is still relevant, and how to test that it works correctly; please add those
as-needed by testing the new python.

(From OE-Core rev: 02714c105426b0d687620913c1a7401b386428b6)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-08 10:57:19 +00:00
Richard Purdie
251ded90d4 archiver/package_rpm: Fix the worst src.rpm generation race
The package_rpm code is writing outside the task's sstate directory into
the sstate of do_deploy_archives. This is "out of spec" since if the
task is installed from sstate, the files are not restored. This means
the files may appear/disappear, things are not deterministic and there are
races.

Extend the do_package_write_rpm code to handle writing the src.rpm into
place to avoid these issues. There are other problems but this avoids races
around this file.

(From OE-Core rev: c6e151ba7fe0f14044537cf0ab2cac436f1496e3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-06 08:29:07 +00:00
André Draszik
d05823086d update-alternatives: correctly escape PATHs when updating FILES_${PN}
The recently added support for updating FILES based on the file renames
that are happening here is using a regex replace, but failed to
properly escape the search pattern (the full path). This manifests itself
in FILES not being updated as soon as the full path contains any
character that has a special meaning, e.g. '+'.

In other words an original path (alt_target in the code) like
    /opt/poky/2.6+snapshot/sysroots/i686-pokysdk-linux/sbin/losetup
can't be matched, and hence we fail to update FILES with the new value,
causing packaging errors.

Fix by using re.escape() on the original path before passing into re.sub()

Fixes: 5c23fe378732 ("update-alternatives: try to update FILES_${PN} when
renaming a file"), or bcb3e7b7f8 in poky.git

[YOCTO #13058]

(From OE-Core rev: 126743162397e4145902b3f127f2dafd80a8a49b)

Signed-off-by: André Draszik <andre.draszik@jci.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-06 08:29:06 +00:00
Richard Purdie
52faf80b7c package_rpm/archiver: Apply bandaid to src.rpm creation
| error: create archive failed on file /media/build1/poky/build/tmp/work/all-poky-linux/xcursor-transparent-theme/0.1.1+gitAUTOINC+23c8af5ba4-r0/deploy-sources/allarch-poky-linux/xcursor-transparent-theme-0.1.1+gitAUTOINC+23c8af5ba4-r0/xcursor-transparent-theme-0.1.1+git0+23c8af5ba4-r0.src.rpm: cpio: read failed - No such file or directory
| Building target platforms: noarch-poky-linux
| Building for target noarch-poky-linux

This is caused by:

$ cat log.task_order

do_cleansstate (24289): log.do_cleansstate.24289
do_deploy_archives_setscene (24395): log.do_deploy_archives_setscene.24395
do_fetch (24407): log.do_fetch.24407
[..]
do_package_write_rpm (25448): log.do_package_write_rpm.25448
do_package_qa (25451): log.do_package_qa.25451

So do_deploy_archives can run from sstate, created a .src.rpm in WORKDIR/deploy-sources,
then it was removed when rpm was running. This leads to a broken Source line in the
spec file as the original file was found by the os.listdir().

This fix is just a bandaid over much more fundamental problems sadly.

(From OE-Core rev: a10020ace4c3cd863c782760f7cbecea557ec6e7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-05 13:53:48 +00:00
André Draszik
bcb3e7b7f8 update-alternatives: try to update FILES_${PN} when renaming a file
When using update-alternatives, FILES_${PN} must be
referencing the new name after update-alternatives has
renamed files.

This is more or less OK when having static lists of files to
be packaged into a package, but makes it quite hard to
dynamically generate FILES_${PN}, e.g. using do_split_packages(),
as in that case we can not easily modify what goes into
FILES_${PN}, because that list is based on filenames as seen
at the time do_split_packages() is executing.

Of couse one could explicitly specify the (renamed) file(s)
in the recipe, but that contradicts the intended usage of
do_split_packages().

Instead, if FILES_${PN} contains the file name as it was pre
renaming, we here modify this to reflect the new name.

This will allow usage of do_split_packages() to populate
FILES_${PN}.

[YOCTO #13058]

(From OE-Core rev: 5c23fe378732038643a450cbf916334d24764b70)

Signed-off-by: André Draszik <andre.draszik@jci.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-02 11:05:59 +00:00
André Draszik
0b782c39b9 update-alternatives: convert file renames to PACKAGE_PREPROCESS_FUNCS
At the moment, the update-alternatives file renaming is
happening right after copying into PKGD during packaging
time using an _append OVERRIDE to the copy function
perform_packagecopy().

This is not really readable and hard to maintain.

Additionally, this makes it impossible to e.g. populate
PACKAGES dynamically using do_split_packages() and still
do update-alternatives - update-alternatives file renaming
requires the PACKAGES variable to have been fully populated
to work correctly. On the other hand, do_split_packages()
can only execute after perform_packagecopy(), as it needs
PKGD populated; so it's impossible to insert
do_split_packages() early enough in a deterministic way in
this use-case.

As there doesn't seem to be a reason not to, convert
this to a proper function and use PACKAGE_PREPROCESS_FUNCS
instead - after all, that's what this is meant for.

No other classes or recipes in oe-core or meta-openembededd
seem to have a hard requirement on update-alterantives
executing before any other PACKAGE_PREPROCESS_FUNCS, so
this should be perfectly fine.

The only implication is that if compress_doc is inherited,
compressed man-page file names will end up being, e.g.
    eject.1.util-linux.gz
  or
    eject.1.gz.util-linux
based on the include of compress_doc.bbclass vs.
update-alternatives.bbclass order, but the symlink created
(alternative name) will always be correct.

This solves both problems:
* the code is easier to read / follow
* the above described use-case can be accomodated easily

[YOCTO #13058]

(From OE-Core rev: 8c3db2fa735e5933f842f80321bb3ed38753812d)

Signed-off-by: André Draszik <andre.draszik@jci.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-02 11:05:59 +00:00
Alexander Kanavin
99fc8d54d6 pixbufcache.bbclass: remove a dependency loop introduced by enabling gtk+3-native
(From OE-Core rev: f99921554fddef82eb80568125aa646a27859a4b)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-28 23:14:10 +00:00
Joshua Watt
3c34a844fc classes/icecc: Add ICECC_REMOTE_CPP variable
Newer versions of Icecream (1.2) add support for remotely pre-processing
files, controllable with the ICECC_REMOTE_CPP environment variable.

(From OE-Core rev: 8fe6728536bbee814c421afbbaa2022e0da0f65c)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-28 23:14:10 +00:00
Joshua Watt
ba3aa53112 Change default debug split to make separate source packages
Changes the default PACKAGE_DEBUG_SPLIT_STYLE to generate separate
source and debug packages. SDKIMAGE_FEATURES is updated to include the
source packages so that there is not change for the SDK contents.

[YOCTO #12931]

(From OE-Core rev: 8df14b678e44cc749b361224af05ccbcfa9ae9b5)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-28 23:14:09 +00:00
Richard Purdie
55ab45436d package: Add pkg_postinst_ontarget to PACKAGEVARS
Changes to pkg_postinst_ontarget were not triggering rebuilds, this fixes
that.

[YOCTO #13127]

(From OE-Core rev: 432d5a5481bd8efb848b95fbe6500a72fba9ac65)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-26 13:39:37 +00:00
Robert Yang
043d6ac1f6 native.bbclass/cross.bbclass: No strip sysroot when DEBUG_BUILD
This makes dbg work for native tools, and makes debug native tools problem
easier, otherwise, there is no symbol since trippped.

(From OE-Core rev: 1d903485da26fec991b4a940182e32934220e19b)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-26 13:39:37 +00:00
Alexander Kanavin
f7718fa8ae testimage.bbclass: add support for passing runqemu params
This is particularly useful when setting up GL tests.

(From OE-Core rev: 167a46775059b782c6f82ce8c5a47b27262e95d4)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-26 13:39:37 +00:00
Alexander Kanavin
b8473eb822 fontcache: fix postinst for nativesdk case
Both installing the binary into the correct place, and passing that place
to postinst_intercept were missing.

(From OE-Core rev: 9ac7415464b7817aa5cc0c2167ff61f4344660dd)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-26 13:39:37 +00:00
Robert Yang
62dd6c2eff packagegroup.bbclass: Set INHIBIT_DEFAULT_DEPS
It doesn't need them since no compile happens.

(From OE-Core rev: c08478c06d541086bd358c318fce213910c7312d)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-24 17:45:25 +00:00
Alexander Kanavin
cc4e883a54 sstate.bbclass: make sure changes to SSTATE_SCAN_FILES are not ignored
When changing the SSTATE_SCAN_FILES variable in a recipe it doesn't cause a rebuild,
so if there's a sstate-cache available with "bad" sstate data in it that will still
be used even though the recipe is updated to address this.

[YOCTO #13144]

(From OE-Core rev: ea3526961920a229e0bb5fb459952be89fce2255)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-24 17:45:25 +00:00
Andreas Müller
ba0b32ebc3 remove unused distutils-tools.bbclass
* As far as I can see nothing uses it
* From commit history it seems distutils-tools came in accidentally

(From OE-Core rev: a4edfa4cf451bf412525887b5b24b9db6486ae97)

Signed-off-by: Andreas Müller <schnitzeltony@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-23 07:57:07 +00:00
Joshua Watt
24ff4ee9fe classes/populate_sdk_base: Add src-pkgs to COMPLEMENTARY_GLOB
Add a COMPLEMENTARY_GLOB for source packages (*-src) so that they can be
included when PACKAGE_DEBUG_SPLIT_STYLE is "debug-with-srcpkg"

[YOCTO #12931]

(From OE-Core rev: d6d2175032c6c950a0d8d63dad0734e1a9f80e12)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-22 14:35:58 +00:00
Joshua Watt
1d86f65ff5 classes/sstate: Update output hash
Updates the output hash calculation for determining if tasks are
equivalent. The new algorithm does the following based on feedback:
 1) The output hash function was moved to the OE library.
 2) All files are printed in a single line tabular format
 3) Prints the file type and mode in a user-friendly ls-like format
 4) Includes the file owner and group (by name, not ID). These are only
    included if the task is run under pseudo since that is the only time
    they can be consistently determined.
 5) File size is included for regular files

(From OE-Core rev: 4bd297dfe92851f3b44f6b5560bac9d8f9ccf9f2)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-22 14:35:58 +00:00
Khem Raj
d5edd81f89 musl,glibc,newlib: Drop redundant STAGINGCC
We do not have initial phase of bootstrapping toolchains anymore

(From OE-Core rev: 75a2c15bbabf4df14631c822b20ce6d31098a5c8)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-21 23:44:22 +00:00
Peter Kjellerstedt
0ac56322d9 extrausers.bbclass: Suggest to use IMAGE_CLASSES rather than INHERIT
Since this class is only useful for image recipes, it is better to
suggest adding it via IMAGE_CLASSES instead of INHERIT in the example.

Also make the example a bit more readable by indenting the variable
values.

(From OE-Core rev: ee91a464f2cad6b3801e18fee28e639e1c957d41)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-18 16:24:40 +00:00
Robert Yang
024b5af548 sstate.bbclass: remove dependencies of SSTATE_EXTRAPATHWILDCARD
There might be do_cleansstate errors sometimes:
ERROR: When reparsing
/path/to/meta/recipes-devtools/quilt/quilt-native_0.65.bb.do_cleansstate,
the basehash value changed from
b4dbcb956a32ed4c3f58b7971717907bfc03bb21f3b140fa97f7765ee695f4d0 to
c8307418a671686349b73efbd51c5c82c897a88707a759ddb22fd95baa5df2ba. The metadata
is not deterministic and this needs to be fixed.

The stable reproducer is:
- Initial a fresh build, this is a must, otherwise we may can't reproduce it
  $ . oe-init-build-env build
  $ bitbake quilt-native -ccleansstate

This is because uninative.bbclass resets NATIVELSBSTRING from distro (e.g.,
ubuntu) to universal, remove dependencies of SSTATE_EXTRAPATHWILDCARD as
SSTATE_EXTRAPATH did can fix the problem.

(From OE-Core rev: 781117f9f02c0080dadc8797a8f8f9377a99b164)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-16 15:35:07 +00:00
André Draszik
46bd059869 classes/compress_doc: add missing import subprocess
Enabling compress_doc gives the following stack trace:

File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:package_do_compress_doc(d)
     0003:
File: '/scratch/yocto/swupd.bb/poky/meta/classes/compress_doc.bbclass', lineno: 50, function: package_do_compress_doc
     0046:    mandir = os.path.abspath(dvar + os.sep + d.getVar("mandir"))
     0047:    if os.path.exists(mandir):
     0048:        # Decompress doc files which format is not compress_mode
     0049:        decompress_doc(mandir, compress_mode, decompress_cmds)
 *** 0050:        compress_doc(mandir, compress_mode, compress_cmds)
     0051:
     0052:    infodir = os.path.abspath(dvar + os.sep + d.getVar("infodir"))
     0053:    if os.path.exists(infodir):
     0054:        # Decompress doc files which format is not compress_mode
File: '/scratch/yocto/swupd.bb/poky/meta/classes/compress_doc.bbclass', lineno: 180, function: compress_doc
     0176:                    _collect_hardlink(hardlink_dict, file)
     0177:                # Normal file
     0178:                elif os.path.isfile(file):
     0179:                    cmd = "%s %s" % (compress_cmds[compress_mode], file)
 *** 0180:                    (retval, output) = subprocess.getstatusoutput(cmd)
     0181:                    if retval:
     0182:                        bb.warn("compress failed %s (cmd was %s)%s" % (retval, cmd, ":\n%s" % output if output else ""))
     0183:                        continue
     0184:                    bb.note('compress file %s' % file)
Exception: NameError: name 'subprocess' is not defined

Fix by adding the missing import in two places.

(From OE-Core rev: 539f65d2533a277233d83d085cb78bdf56a6e16c)

Signed-off-by: André Draszik <andre.draszik@jci.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-16 15:35:07 +00:00
Richard Purdie
e1b639bd5e meta: Fix python code quoting issues
python 3.8 will be stricter about python quoting. Fix up several misquoted
expressions and fix Deprecation warnings like:

Var <do_compile>:1: DeprecationWarning: invalid escape sequence \$

(From OE-Core rev: 3ba6cee84de89f8eb200e4c93d446f6cdeeaa4be)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-16 15:35:07 +00:00
Richard Purdie
cd4b8a8553 meta: Fix Deprecated warnings from regexs
Fix handling of escape characters in regexs and hence fix python
Deprecation warnings which will be problematic in python 3.8.

Note that some show up as:

"""
meta/classes/package.bbclass:1293: DeprecationWarning: invalid escape sequence \.  

"""

where the problem isn't on 1293 in package.bbclass but in some _prepend to a
package.bbclass function in a different file like mesa.inc, often from
do_package_split() calls.

(From OE-Core rev: 4b1c0c7d5525fc4cea9e0f02ec54e92a6fbc6199)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-16 15:35:07 +00:00
André Draszik
181e3a8446 classes/buildhistory: display modified git branches
When collecting the git revision of repositories, also take
note of whether or not that repository has uncommited
changes.

This makes it a bit clearer what went on when looking at
diffs.

(From OE-Core rev: b9d780c7eeda0fefb13edde8bdba4f1d91e7823c)

Signed-off-by: André Draszik <andre.draszik@jci.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-14 17:38:24 +00:00
André Draszik
dfd833725d meta: remove True option to getVar calls (again)
A couple have still been missed in the past despite multiple
attempts at doing so (or simply have re-appeared?).

Search & replace made using the following command:
    sed -e 's|\(d\.getVar \?\)( \?\([^,()]*\), \?True)|\1(\2)|g' \
        -i $(git grep -E 'getVar ?\( ?([^,()]*), ?True\)' \
             | cut -d':' -f1 \
             | sort -u)

(From OE-Core rev: 9f551d588693328e4d99d33be94f26684eafcaba)

Signed-off-by: André Draszik <andre.draszik@jci.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-14 11:35:56 +00:00
Jacob Kroon
889d9e037f package.bbclass: Make package output files more deterministic
Observing depsig.do_package for packages inbetween rebuilds indicated
that the following variables/files content was changing order randomly.
Make them deterministic by sorting the output:

 RDEPENDS_<pkg>
 RRECOMMENDS_<pkg>
 FILERDEPENDSFLIST_<pkg>

 packages-split/<pkg>.shlibdeps

The following variable was not observed to change, but it is
assumed that the same situation can occur, so do the same
sorting for consistency:

 FILERPROVIDESFLIST_<pkg>

(From OE-Core rev: c99cb0bbb78089d1d15c4c8563a71db0df1cb0da)

Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-14 11:35:56 +00:00
Robert Yang
3be36b0d27 cmake.bbclass: Make it work with ccache
This can make the following recipes work with cmake:
cmake libdnf libcomps librepo createrepo-c llvm dnf libsolv assimp waffle
libjpeg-turbo taglib libproxy libical

And the following 3 recipes don't:
webkitgtk vulkan piglit
Now cmake.bbclass doesn't disble ccache any more, disable it in the recipes if
needed.

(From OE-Core rev: d014c8c11fb663f131d3a860ddeda17d604b2dd3)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-14 11:35:55 +00:00
Robert Yang
7a713b0fbf goarch.bbclass: Set CCACHE_DISABLE
Go can't be built with ccache.

(From OE-Core rev: cf64c9413a2264aa67e26c6302342ff4aa99a575)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-14 11:35:55 +00:00
Robert Yang
e96d0200ca ccache.bbclass: Set CCACHE_CONFIGPATH to fix race issues
Fixed race issues when parallel build:
ccache: error: /path/to/ccache/i586-poky-linux/mmc-utils/ccache.conf: No such file or directory
ccache: error: /path/to/ccache/i586-poky-linux/mmc-utils/ccache.conf: No such file or directory

This is because we set CCACHE_DIR for earch recipe, and ccache will create a
ccache.conf for each CCACHE_DIR when CCACHE_CONFIGPATH is not set, but there
might be a race issue in parallel build:

ccache gcc file1.c
ccache gcc file2.c

If the two ccache processes use fopen(path, "w") to create ccache.conf at the
same time, the error would happen. Set CCACHE_CONFIGPATH to
meta/conf/ccache.conf can fix the problem, and we can add other configs to the
file when needed.

And also set cache_dir_levels to 1 (default is 2) since each recipe has a cache
dir, thus we don't have too many files in one dir.

(From OE-Core rev: 2abbc4d0cd571e82ed6188d3b2d84b4cd6be25e8)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-14 11:35:55 +00:00
Robert Yang
7953d6a8bc ccache.bbclass: Add task do_cleanccache
(From OE-Core rev: 36cead66fbadd8c3827aec4b67ea124ee3c2ff94)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-14 11:35:55 +00:00
Robert Yang
e1cad77fa7 ccache.bbclass: Make it can be shared between different builds
CCACHE_BASEDIR: ccache removes this from file path, so that hashes will be
                the same in different build dirs.

CCACHE_TOP_DIR: Set it to a shared location for different builds.

(From OE-Core rev: 35d7fe73bba15de16d2eb0a4b12ef03b57b23306)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-14 11:35:55 +00:00
Robert Yang
0c98ff2c31 ccache.bbclass: Refactor it to make it more reliable
The previous ccache.bbclass has the following problems:
- It uses host's ccache for native recipes, but this may not work on some
  hosts, for example, it nerver works on my Ubuntu 14.04.4, there are always
  build failures (m4-native failed at do_configure, and others will also be
  failed if I disable CCACHE for m4-native)

- native/nativesdk/cross/crosssdk recipes use host's ccache, but target uses
  ccache-native, this may confuse user.

- The target recipes may use both host's ccache and ccache-native, this may
  cause unexpected problems and be hard to debug. This is because ccache-native is
  in SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS, so ccache-native may not be present when
  rebuild target recipes, and then it would use hosttools/ccache, but the
  previous ccache files were generated by ccache-native.

- Target recipes can't use ccache when no ccache is installed on the host:
  CCACHE = "${@bb.utils.which(d.getVar('PATH'), 'ccache') and 'ccache '}"

After refactored:
All types recipes (native, target and others) will use ccache-native except
ccache-native itself, host's cache won't be used any more. It is more
reliable now, which will work everywhere when ccache-native can be built.

And now we need use "CCACHE_DISABLE = '1'" to disable ccache for the recipe
rather than "CCACHE = ''" since we set CCACHE in anonymous function, and
d.getVar('CCACHE') works after "CCACHE ??=" which is set in bitbake.conf, so we
can't check whether CCACHE is set or not in anonymous function since it is
always set. Use CCACHE_DISABLE to disable it would be more clear.

(From OE-Core rev: b25271b65262f70d849a4861da216c9be6c54d53)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-14 11:35:55 +00:00
Ross Burton
c62ad6939f gsettings: allow the schemas to be in any package
Instead of assuming that the schemas are located in ${PN}, add a
GSETTINGS_PACKAGE variable for the package name and default it to ${PN}.

For recipe that can conditionally ship schemas, support GSETTINGS_PACKAGE being
empty gracefully by doing nothing.

(From OE-Core rev: c2b9c34ce4af151cc0422e14af775c6c962de051)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-11 10:39:09 +00:00
Robert Yang
c856ed29df oeqa: make it work for multiple users
There are failures when multiple users run oe-selftest on the same
host:

PermissionError: [Errno 13] Permission denied: '/tmp/oe-saved-
tests/201812250324_qemu'

This is because /tmp/oe-saved-tests was created by user A, while user B tries
to write data in it, then the error will happen. This patch can fix the
problem.

Move the dumped data to ${LOG_DIR}/runtime-hostdump/ rather than
/tmp/oe-saved-tests/ to fix the problem.

(From OE-Core rev: e219fe5329599cd6c3682f521eaee3852a2c8980)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-11 10:39:09 +00:00
Joshua Watt
14800d7361 classes/icecc: Remove trailing whitespace
(From OE-Core rev: d065c592977ad742515121e07ed3b7698db55f25)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-11 10:39:09 +00:00
Joshua Watt
f5cc622aab classes/icecc: Check blacklist for BPN
If a given PN is listed in the icecream blacklist, there is a very good
chance that the native, nativesdk, and multilib variants should also be
skipped. Check the blacklist entries against BPN to cover them.

[YOCTO #13128]

(From OE-Core rev: 1863f695a1411e95e7e547a3eb3e7ef6604a93bf)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-11 10:39:09 +00:00
Jacob Kroon
18c1b63cb2 buildhistory: simplify buildhistory_list_files()
Avoid duplicating shell code for the two cases, fakeroot/non-fakeroot.

(From OE-Core rev: c4a931df28f45f95f19a13062b8dc38db60da342)

Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-11 10:39:09 +00:00
Robert Yang
f476905b9c package.bbclass: Do not hide cpio's error
We use subprocess.check_output() to run the command, which means that we need care
about the error, so the 2>/dev/null should not be used, otherwise it is hard to
debug when the error happens.

I guess it was copied from previous lines, but that command's error can be
ignored (excpet: pass):
   try:
       subprocess.check_output(cmd, shell=True, stderr=subprocess.STDOUT)
   except subprocess.CalledProcessError:
       # Can "fail" if internal headers/transient sources are attempted
       pass

But we don't do this in the current location, so remove "2>/dev/null"

(From OE-Core rev: 017a53d2743be2b5d4965a39b4e126fb74f700ad)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-11 10:39:09 +00:00
Alex Kiernan
1c0473fe4b uboot-sign.bbclass: silence warnings when UBOOT_DTB_BINARY is empty
When UBOOT_DTB_BINARY is set to "", the keys for signed booting are
expected to be already present in U-Boot's DTB, so don't issue warnings
for this.

(From OE-Core rev: 04656f5df326a72747fc5878ce201b636a2a419c)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-11 10:39:09 +00:00
Kai Kang
24d8aa6ec2 distro_features_check.bbclass: show all error info at one time
In distro_features_check.bbclass it checks whether items in
REQUIRED_DISTRO_FEATURES and CONFLICT_DISTRO_FEATURES exist in
DISTRO_FEATURES. But it only shows one required or conflict distro
feature when error occurs. Update to show them all at one time.

(From OE-Core rev: d0441c40afdba119a65189d6a5aca5c533f68279)

Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-11 10:39:09 +00:00
Jacob Kroon
348f6b0b61 package.bbclass: Sort FILES_INFO by key
Observing depsig.do_package for a package inbetween rebuilds indicated
that FILES_INFO was changing content order randomly. Force it to be
deterministic by sorting with respect to the keys when serializing.

Suggested-by: Joshua Watt <jpewhacker@gmail.com>
(From OE-Core rev: b0364be65bf0b3dee44f81379f4062e9f707c128)

Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-11 10:39:09 +00:00
Alexander Kanavin
d460892f32 meta/classes: adjust perl-related classes to the new recipes
This mostly means tweaking the paths to match upstream defaults.

get_perl_arch() functions are taken from the patch by Jens Rehsack:
http://lists.openembedded.org/pipermail/openembedded-core/2018-November/276546.html

(From OE-Core rev: d6b36b1babb4d3e8d41278111e71c71fde9af39e)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-11 10:39:09 +00:00
Douglas Royds
c3a244b792 patch: reproducibility: Fix host umask leakage
Some patch files create entirely new files, so their permissions are subject to
the host umask. If such a file is later installed into a package with no change
in permissions, it breaks the reproducibility of the package.

This was observed on libpam, for instance: The patch file
pam-security-abstract-securetty-handling.patch creates a new file
(tty_secure.c). This file is later copied into the -dbg package with no change
in permissions.

(From OE-Core rev: 2a2bbd755b330cd63f7f6e2f2b374a3ae065b37a)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-11 10:39:08 +00:00
Ross Burton
3085647e0c toolchain-scripts: run post-relocate scripts for every environment
SDKs for multilib configurations have multiple environment scripts, so
re-arrange the post-relocate hook invocation so that it runs the post-relocate
hooks after sourcing each environment script.

(From OE-Core rev: c4897001cb3eeda1f4f11197b28c09c950bdcf02)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 11:16:44 +00:00
Robert Yang
dda8084d7a oeqa: Fix for QEMU_USE_KVM
Fixed:
MACHINE = "qemux86"
QEMU_USE_KVM = "qemux86"
IMAGE_CLASSES += "testimage"

$ oe-selftest -r runqemu.RunqemuTests.test_boot_rootfs

[snip]
  File "/buildarea1/lyang1/poky/meta/lib/oe/types.py", line 122, in boolean
    raise ValueError("Invalid boolean value '%s'" % value)
ValueError: Invalid boolean value 'qemux86'

Now QEMU_USE_KVM can only be boolean, can not contain MACHINE any more, kvm
will be enabled if target_arch == build_arch or both of them are x86 archs.

(From OE-Core rev: 7c1a8a624cad8d967635c6cb5f99cf655bde3d44)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 11:16:44 +00:00
Joshua Watt
adc37721a8 sstate: Implement hash equivalence sstate
Converts sstate so that it can use a hash equivalence server to
determine if a task really needs to be rebuilt, or if it can be restored
from a different (equivalent) sstate object.

The unique hashes are cached persistently using persist_data. This has
a number of advantages:
 1) Unique hashes can be cached between invocations of bitbake to
    prevent needing to contact the server every time (which is slow)
 2) The value of each tasks unique hash can easily be synchronized
    between different threads, which will be useful if bitbake is
    updated to do on the fly task re-hashing.

[YOCTO #13030]

(From OE-Core rev: d889acb4f8f06f09cece80fa12661725e6e5f037)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 11:16:44 +00:00
Joshua Watt
cbdfa37663 classes/sstate: Handle unihash in hash check
Handles the argument that passes task unique hash in the hash check
function, as it is now required by bitbake

[YOCTO #13030]

(From OE-Core rev: 1c14b6969e58b51a325c1c1acf9c96e55675035c)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-08 11:16:44 +00:00
Manjukumar Matha
6a080d90e5 kernel.bbclass: Fix incorrect deploying of fitimage.initramfs
When kernel-fitimage and initramfs is enabled using
INITRAMFS_IMAGE_BUNDLE = "1", kernel do_deploy tries to deploy
fitImage.initramfs with following error

| install: cannot stat 'arch/arm64/boot/fitImage.initramfs': No such
file or directory

Skip deploying fitimage.initramfs, since fitimage does not
create fitimage.initramfs

(From OE-Core rev: 0f87f8522a2c1b5a7042738887952b98060fbd9d)

Signed-off-by: Manjukumar Matha <manjukumar.harthikote-matha@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-07 10:54:13 +00:00
Clemens Lang
caf55cf9b4 update-alternatives.bbclass: Stabilize iteration order
The use of a dictionary for link_rename causes problems for higher-order
alternatives, i.e. when an alternative link points to another
alternative link, since these links must be processed in the order in
which they were originally added for symlink correction to work.

Switch from a dict to a list of tuples to ensure processing happens in
the original order.

(From OE-Core rev: 326220267ffc43ec1f507ad0cc47ac59caafd5b7)

Signed-off-by: Clemens Lang <clemens.lang@bmw-carit.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-07 10:54:13 +00:00
Jacob Kroon
2b9c40e56b classes/buildhistory: avoid git diff churn by sorting more dictionaries
Sort 'srcrevs' and 'tag_srcrevs' before iterating over them in order
to avoid unnecessary changes in the build history.

(From OE-Core rev: 031d708aedda11d65ba9746af4f01b91264a1f86)

Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-07 10:54:13 +00:00
Chen Qi
ea915ed419 package.bbclass: fix python unclosed file ResourceWarning
Fix the following warning.

  ResourceWarning: unclosed file <_io.TextIOWrapper name='/.../systemd/1_239-r0/debugsources.list' mode='a' encoding='UTF-8'>

(From OE-Core rev: 91810a57f0edd8b37c5f3f989a5aca69d9a40b37)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-03 21:16:00 +00:00
Michael Ho
aa83cb5264 sstate: add support for caching shared workdir tasks
The sstate bbclass uses workdir as a hardcoded string in path
manipulations. This means that the sstate caching mechanism does
not work for the work-shared directory which the kernel uses to
share its build configuration and source files for out of tree
kernel modules.

This commit modifies the path manipulation mechanism to use the
work-shared directory if detected in the paths when handling the
sstate cache packages.

(From OE-Core rev: 27642449f95e38598f9c83948ce109c5891e5877)

Signed-off-by: Michael Ho <Michael.Ho@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-03 21:16:00 +00:00
Aníbal Limón
fed984dc8e meta/classes/testimage.bbclass: Only validate IMAGE_FSTYPES when is QEMU
When use simpleremote target the flash/boot process is executed
manually, the IMAGE_FSTYPES validation is only needed when execute
testimage against qemu.

The supported_fstypes comes from oeqa.core.target.qemu module.

(From OE-Core rev: e7dc5963adbacc091fe8943119262166977623ad)

Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-03 12:35:54 +00:00
Lei Maohui
d7d492af8e Added armeb into meson.bbclass.
(From OE-Core rev: b0021e3d33ba0fb60a340d4553267b18d296b2be)

Signed-off-by: Lei Maohui <leimaohui@cn.fujitsu.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-03 12:35:53 +00:00
Martin Jansa
08eb8565b6 sstate.bbclass: catch ValueError from incorrectly written sstate manifests
* in some rare cases the file format might be broken, e.g. when you run
  out of disk space while writing to it (hint: make sure to use
  BB_DISKMON_DIRS on _all_ the builds, to make sure that they can stop
  gracefully before this happens).

* will show error like this:
  ERROR: Invalid line '/OE/build/luneos-thud/webos-ports' in sstate manifest '/OE/build/luneos-thud/webos-ports/tmp-glibc/sstate-control/index-allarch'
  when the file isn't complete, like in my case:
  $ tail -n 2 /OE/build/luneos-thud/webos-ports/tmp-glibc/sstate-control/index-allarch
  /OE/build/luneos-thud/webos-ports/tmp-glibc/stamps/all-webos-linux/org.webosports.app.calculator/0.1.1+gitrAUTOINC+9e9eb67c28-r0 /OE/build/luneos-thud/webos-ports/tmp-glibc/sstate-control/manifest-allarch-org.webosports.app.calculator /OE/build/luneos-thud/webos-ports/tmp-glibc/work/all-webos-linux/org.webosports.app.calculator/0.1.1+gitrAUTOINC+9e9eb67c28-r0
  /OE/build/luneos-thud/webos-ports

* instead of much longer exception which doesn't really show what's
  wrong to selectively fix that (other than removing while TMPDIR):

  ERROR: Command execution failed: Traceback (most recent call last):
  File "/OE/build/luneos-thud/webos-ports/bitbake/lib/bb/command.py", line 113, in runAsyncCommand
    self.cooker.updateCache()
  File "/OE/build/luneos-thud/webos-ports/bitbake/lib/bb/cooker.py", line 1562, in updateCache
    bb.event.fire(event, self.databuilder.mcdata[mc])
  File "/OE/build/luneos-thud/webos-ports/bitbake/lib/bb/event.py", line 225, in fire
    fire_class_handlers(event, d)
  File "/OE/build/luneos-thud/webos-ports/bitbake/lib/bb/event.py", line 134, in fire_class_handlers
    execute_handler(name, handler, event, d)
  File "/OE/build/luneos-thud/webos-ports/bitbake/lib/bb/event.py", line 106, in execute_handler
    ret = handler(event)
  File "/OE/build/luneos-thud/webos-ports/openembedded-core/meta/classes/sstate.bbclass", line 1083, in sstate_eventhandler2
    (stamp, manifest, workdir) = l.split()
  ValueError: not enough values to unpack (expected 3, got 1)

(From OE-Core rev: 870ba5d80e76e0f989971532fc2adc0ebe811ab6)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-03 12:35:53 +00:00
Pau Espin Pedrol
48c0ed2aec kernel-module-split.bbclass: Fix modinfo decoding in old kernels
Before linux commit d36b691077dc59c74efec0d54ed21b86f7a2a21a, some
strings contained invalid utf-8 character which made
split_kernel_module_packages unhappy when parsing ums-isd200.ko:

Exception: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf6 in position 21: invalid start byte

(From OE-Core rev: 503b977acf6984120818cbc5cfd35ff2cffb39cd)

Signed-off-by: Pau Espin Pedrol <pespin@sysmocom.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-01-03 12:35:53 +00:00
Kai Kang
a32c80e6f0 testimage.bbclass: remove boot parameter systemd.log_target
Boot parameter systemd.log_target=console affects command journalctl's
output and causes oe selftest case test_systemd_boot_time fail to pass.

| Error at obtaining the boot time from journalctl
| RESULTS:
| RESULTS - systemd.SystemdJournalTests.test_systemd_boot_time - Testcase -1: SKIPPED (0.74s)

systemd.log_target=console was introduced by oe-core commit a0bb649 and
work with parameter systemd.log_level to enable systemd debug.
systemd.log_level has been removed already, so remove systemd.log_target
too to make case test_systemd_boot_time pass.

(From OE-Core rev: caa776bdcf8ea34c857f45970370bf771075f4bc)

Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-27 22:52:58 +00:00
Alejandro del Castillo
46c9a32230 package_ip.bbclass: add xz threading
Before, threading was enabled via a patch to opkg-build. Now that
opkg-build provides a hook for extra arguments, use that.

(From OE-Core rev: b288f4e825a036b9cc15b27301d264fe695dba65)

Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-27 22:52:58 +00:00
Khem Raj
dc07845321 base.bbclass, classextend.py: Drop catering to gcc-initial
(From OE-Core rev: 5d0307fbefbc8e6667edfa93d527166059a30100)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-26 11:19:54 +00:00
Richard Purdie
46cb012491 crosssdk/cross-canadian: Set LIBCOVERRIDE correctly
(From OE-Core rev: 94bece16c23c9ec8850fd497aea37e6a265da30a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-26 11:12:12 +00:00
Khem Raj
801745d918 populate_sdk_ext.bbclass: Include site.conf in parsing for contents for local.conf
Some distros use site.conf to emit certain variables which are important
for eSDK e.g. DISTRO with out which eSDK will not be able to ger right
metadata when it tries to build

(From OE-Core rev: 95659bed3f6f3216b346f70cfc9ffae9788c0fc1)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-20 14:53:55 +00:00
Douglas Royds
4f2414afd0 icecc: patchelf is needed by icecc-create-env
Although we could potentially build patchelf and all its dependencies,
they would all have to be blacklisted to avoid trying to build them with
icecc. We use the host patchelf instead.

(From OE-Core rev: 0860216a81092fba084d29696db6d0c65a226c85)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-20 14:53:55 +00:00
Douglas Royds
f6c639f578 icecc: Don't generate recipe-sysroot symlinks at recipe-parsing time
The python function icecc_path() was being invoked inline by set_icecc_env(),
meaning that it was being invoked at recipe-parsing time.
As a side-effect, icecc_path() was creating the recipe-sysroot directory and
symlinking icecc into it. Because this was done at parsing time (rather than
configure time), we were generating otherwise-empty WORKDIRs for *all* parsed
recipes, and for all virtual classes (-native, -nativesdk).
In my build, this generated more than 800 of these otherwise-empty WORKDIRs.

I have simplified icecc_path() to return only the intended path to the icecc
symlinks in the recipe-sysroot, with no side-effect.
We then create the directory and the icecc symlinks at configure time.

Because get_cross_kernel_cc() is still invoked at parse-time,
it needs a guard-clause for the non-kernel case.
We are now finding the host icecc at do_configure time,
so icecc needs to be in the HOSTTOOLS. I have made this non-fatal,
so that we can still inherit icecc without icecc installed.

(From OE-Core rev: d2fcaeb153fdc3f8d7143ea823139f1537055ff1)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-20 14:53:55 +00:00
Douglas Royds
45a59ffbd6 icecc: Syntax error meant that we weren't waiting for tarball generation
If two bitbake processes try to generate the ICECC_VERSION tarball at the same
time, the thread that fails to get the lock will wait 30 sec for the first to
finish. A syntax error meant that this was not happening, and in particular, if
tarball generation failed (eg. for lack of patchelf), it did so silently.

(From OE-Core rev: 4455c46bf7f49c2b44f250aa89b63b342368e2e3)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-20 14:53:55 +00:00
Douglas Royds
7e80de311c icecc: Trivial simplification
(From OE-Core rev: fa6bae80bc3392bf99ce53bbe19a0e05dbd0c4ee)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-20 14:53:55 +00:00
Douglas Royds
40fd208847 icecc: readlink -f on the recipe-sysroot gcc/g++
We were accidentally doing a readlink -f on simply 'gcc', for instance

(From OE-Core rev: 6d6788a3ea45d9693743d9b3319fb0368a4d5d33)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-20 14:53:55 +00:00
Richard Purdie
739c420caa distrodata: Remove
The code in the tasks within this class are old and there are much better
ways to do these things now such as through tinfoil rather than writing csv
files incrementally with start/stop events and lock files.

Existing users of the code (mainly the do_checkpkg task) have been ported
over to tinfoil. Drop the class, any other users should also make use
of the tinfoil functionality.

(From OE-Core rev: 6e2a488f01e5e8bc6f06b6fac1d336bf5372a82b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-18 16:19:34 +00:00
Richard Purdie
42cc9e6c99 upstream-tracking.inc: Remove the long deprecated file
(From OE-Core rev: 22447eba250656489a6ed636fe58cb304d74e975)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-18 16:19:34 +00:00
Richard Purdie
be4d74c509 defaultsetup: Include maintainers.inc by default
There is little harm in including this file rather than having a
rather obscure and soon to be obsolete bbclass adding this information.

(From OE-Core rev: 98536be8d1a5dcf620ee6d07135f23126a09d98f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-18 16:19:34 +00:00
Robert Yang
2e8f8d7eb3 base.bbclass: Add comments for gcc links to ccache
(From OE-Core rev: ce6bf125aba7344d56368885605949e373b06393)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-15 17:10:52 +00:00
Erik Botö
217035f098 testimage: Add possibility to pass parmeters to qemu
Add a variable called TEST_QEMUPARAMS in testimage.bbclass to make it
possible to pass parameters to qemu. This can be useful for e.g.
increasing the amount of RAM available during testimage runs.

(From OE-Core rev: 1a9163f5779d233c884c8fd50e0812eabab4fdf3)

Signed-off-by: Erik Botö <erik.boto@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-15 17:10:51 +00:00
Michael Ho
bc25bb645e insane.bbclass: add package specific skips to sstate hash
The bbclass currently adds INSANE_SKIP to the sstate hash dependencies
however the package specific skips such as INSANE_SKIP_${PN} are
not added automatically because of how the class references them.

This causes the problem that modifying INSANE_SKIP_${PN} does not
invalidate the sstate cache and can mask build breaking warnings.

Add an anonymous python snippet to explicitly include these additional
relevant skips to the sstate hash.

Singed-off-by: Michael Ho <Michael.Ho@bmw.de>
(From OE-Core rev: 8690332183f10a5e5689da7ec030567dfd1ac091)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-14 17:10:58 +00:00
Michael Ho
f510f79859 cmake.bbclass: append includedir to implicit include dirs
This resolves issues with paths being marked as system includes that
differ from /usr/include but are considered implicit by the toolchain.
This enables developers to add directories to system includes
to supress compiler compiler warnings from them.

(From OE-Core rev: 9c2227d5d960f93e00791157354f0c920fbecf39)

Signed-off-by: Michael Ho <Michael.Ho@bmw.de>
Cc: Pascal Bach <pascal.bach@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-14 17:10:58 +00:00
Khem Raj
a34555a54d insane.bbclass: BPF objects may mismatch in endianness and bitness too
This ensures that bitness and endianness is ignored for BPF objects
Fixes QA issues like

Bit size did not match (32 to 64) kernel-selftest on
/work/qemumips-yoe-linux/kernel-selftest/1.0-r0/packages-split/kernel-selftest/usr/kernel-selftest/bpf/test_btf_nokv.o

(From OE-Core rev: 26722e59ca5df14bd90fc09306d97ff0fd078f32)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-13 16:32:21 +00:00
Andrea Adami
3c6751ee7a meson.bbclass: map mips64el TARGET_ARCH to mips64 for the cross file
Meson uses 'mips64' for both big- and little-endian MIPS64 machines,
so map mips64el to mips64.

(From OE-Core rev: 8d1023f222f32ef64d37c3cb15b7dcf9c3e5990d)

Signed-off-by: Andrea Adami <andrea.adami@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-08 17:17:02 +00:00
Robert Yang
67628ea66b uboot-sign.bbclass: fix signature and deployment
Fixed:
MACHINE = "beaglebone-yocto"
KERNEL_CLASSES += "kernel-fitimage"
KERNEL_IMAGETYPE_beaglebone-yocto = "fitImage"
UBOOT_MACHINE_beaglebone-yocto = "am335x_boneblack_vboot_config"
UBOOT_MKIMAGE_DTCOPTS = "-I dts -O dtb -p 2000"
UBOOT_SIGN_KEYDIR = "${TOPDIR}/conf"
UBOOT_SIGN_KEYNAME = "dev"
UBOOT_SIGN_ENABLE = "1"
IMAGE_INSTALL_remove = "kernel-image-zimage"

$ cd conf
$ openssl genrsa -F4 -out dev.key 2048
$ openssl req -batch -new -x509 -key dev.key -out dev.crt
$ cd ../
$ bitbake u-boot linux-yocto
$ grep signature tmp/deploy/images/beaglebone-yocto/*.dtb
Binary file tmp/deploy/images/beaglebone-yocto/u-boot-beaglebone-yocto-2018.07-r0.dtb matches
Binary file tmp/deploy/images/beaglebone-yocto/u-boot-beaglebone-yocto.dtb matches
Binary file tmp/deploy/images/beaglebone-yocto/u-boot.dtb matches

And there would be no signature info when rebuild from sstate:
$ bitbake u-boot linux-yocto -cclean
$ bitbake u-boot linux-yocto
$ grep signature tmp/deploy/images/beaglebone-yocto/*.dtb
No result

This s because kernel directly edit ${DEPLOY_DIR_IMAGE}/u-boot.dtb, (Note, it
is global ${DEPLOY_DIR_IMAGE}, not recipe's DEPLOYDIR), so that the modified
info is not in sstate, and would be lost when rebuild from sstate.

There are other problems in previouse code:
- The u-boot.dtb is provided by u-boot, but edited by kernel during signing, so
  it should be deployed by kernel rather than u-boot.

- The u-boot.do_concat_dtb directly install files to global ${DEPLOY_DIR_IMAGE},
  this is incorrect, the ${DEPLOY_DIR_IMAGE} should be installed by do_deploy.

- It seems that it assumes do_deploy depends on do_install according the comments,
  but they have no relationships:
  # do_concat_dtb is scheduled _before_ do_install as it overwrite the
  # u-boot.bin in both DEPLOYDIR and DEPLOY_IMAGE_DIR.

- The do_concat_dtb should be run after do_compile, but it doesn't have this
  dependency.

Make u-boot install u-boot.dtb to ${datadir}, kernel copies u-boot.dtb from
${STAGING_DATADIR} to ${B} and deploy it can fix the problem.

[YOCTO #12112]

Reported-by: Christian Andersen <c.andersen@kostal.com>
(From OE-Core rev: 493f70cfb177f1d452a13329647a38642bf2b161)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-08 17:17:01 +00:00
Federico Sauter
6aef6271e1 kernel: don't assign the build user/host
The KBUILD_BUILD_USER and KBUILD_BUILD_HOST variables were
assigned at the kernel class level, which made it impossible
to override them in the local configuration.

By setting only the default values of those variables in the
kernel class, it is now possible to override them as expected.

(From OE-Core rev: a3e8cdf9c3ba966fa4b5a21235540eb0b00fb487)

Signed-off-by: Federico Sauter <federico.sauter@ableton.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-08 17:17:01 +00:00
Joshua Watt
0fc667d695 image-buildinfo: Remove unused function argument
Removes the listvars argument to image_buildinfo_outputvars(). It
doesn't appear that this argument ever did anything.

(From OE-Core rev: 66ff9408291f3df98e8a6cb3e6e348d7ec8f401d)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-08 17:17:01 +00:00
Alex Kiernan
85bad95a08 kernel-fitimage: Replace tabs with spaces in python
Fix:

  WARNING: python should use 4 spaces indentation, but found tabs in kernel-fitimage.bbclass, line 24

(From OE-Core rev: 4ec42465e9df8cef20a97be11243726aa7dfb8c1)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-08 17:17:01 +00:00
Richard Purdie
7843548285 classes: Correctly markup regex strings
There are various escape characters in these stings which python warns
about so use the correct regex markup for them.

(From OE-Core rev: 252b69c9f2abe3258366c540f56b156ed63e5437)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-08 17:17:01 +00:00
Kai Kang
3a439047bf image.bbclass: respect PACKAGE_NO_GCONV
It installs locale-base-* packages according to var IMAGE_LINGUAS.
Packages locale-base-* are split in libc-package.bbclass if variable
PACKAGE_NO_GCONV is not set. When none of ditro features libc-charsets
libc-locales and libc-locale-code is set, PACKAGE_NO_GCONV is set. Then
no locale-base-* is created and fails to create image.

Clear IMAGE_LINGUAS in such situation.

(From OE-Core rev: 85240094175a8ea726bfba19c00d4556a62862fc)

Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-05 22:30:19 +00:00
Kai Kang
7291426ed5 glibc-locale: fix installed-vs-shipped qa issue
Variable PACKAGE_NO_GCONV is set in libc-package.bbclass when none of
libc-charsets, libc-locales and libc-locale-code set in DISTRO_FEATURES.
Then it causes installed-vs-shipped qa issue of glibc-locale:

| ERROR: glibc-locale-2.28-r0 do_package: QA Issue: glibc-locale:
| Files/directories were installed but not shipped in any package:
|   /usr/share/i18n
|   /usr/share/i18n/charmaps
|   /usr/share/i18n/locales
|   /usr/share/i18n/charmaps/CP737.gz
|   ...
|   /usr/share/i18n/locales/ru_RU
|   ...
|   /usr/lib64/gconv/gconv-modules
| Please set FILES such that these items are packaged. Alternatively if they
| are unneeded, avoid installing them or delete them within do_install.
| glibc-locale: 843 installed and not shipped files. [installed-vs-shipped]

So check PACKAGE_NO_GCONV during do_install and not copy those files if
PACKAGE_NO_GCONV has been set.

Simplify call of bb.utils.contains() in libc-package.bbclass as well.

(From OE-Core rev: 39840ed10af559bbcc306b378baa4723921668f5)

Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-05 22:30:19 +00:00
Joshua Watt
794a60191b classes/waf: Fix builds when B != S
Waf requires that the current working directory be ${S} (the location of
the wscript) when building. Most of the time, this was true only because
B defaults to S. However, anything that changed that behavior (notably,
using externalsrc) would break the recipe. Remedy this by explicitly
changing cwd to ${S} when running waf commands. As a happy side effect,
B can be set up for "out of tree" builds to keep the source directory
clean.

(From OE-Core rev: 62dffb71ce22222c635bd90eaa47dd01f70f9c0f)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-05 22:30:19 +00:00
Robert Yang
eff9942d30 uboot-sign.bbclass: Remove tab indentations in python code
Use 4 spaces to replace a tab.

(From OE-Core rev: 790828a65316fee57cd7659ad8b7544de5d2610c)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-05 13:04:23 +00:00
Joshua Watt
645493af37 classes/icecc.bbclass: Fix ccache disable
The ccache disable flag was misspelled, preventing it from being
disabled.

(From OE-Core rev: 33fba601a7365aced9f4b206c1fadda997076d4c)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-05 12:37:02 +00:00
Ross Burton
be0b9a3a81 insane: Clarify GNU_HASH warning
We have a fatal error if ELF objects don't have GNU_HASH segments but it
doesn't explain what the problem is.  At least give a hint to users by
suggesting that LDFLAGS wasn't passed to the compiler.

(From OE-Core rev: 5d4da6713b40e10e853eb746f700096307ffe158)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-05 12:37:02 +00:00
Alexander Kanavin
185f1d99fd gobject-introspection.bbclass: add libdir to XDG_DATA_DIRS
Otherwise .gir files installed in the sysroot will not be found when multilib is in use
(multilib configurations use libdir rather than datadir).

(From OE-Core rev: cfae171964ba85298d883e70635324a33b8539ac)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-05 12:37:02 +00:00
Joshua Watt
9775741a20 classes/update-alternatives: Skip alternatives when disabled
Skips the update alternative steps for recipes that shouldn't have them
enabled.

Fixes errors like:

 nativesdk-bzip2-1.0.6-r5 do_package: bzip2: alternative target
 (/opt/poky/2.5+snapshot/sysroots/i686-pokysdk-mingw32/usr/bin/bunzip2
 or
 /opt/poky/2.5+snapshot/sysroots/i686-pokysdk-mingw32/usr/bin/bunzip2.bzip2)
 does not exist, skipping...

When building mingw SDKs

[YOCTO #12962]

(From OE-Core rev: d11576f569ee618a8e638eee3b1f17bf8a9d4264)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-03 12:20:01 +00:00
Douglas Royds
fa424823b6 ptest: Optionally strip build host tool info from installed files
Several packages (eg. acl, attr, openssl) install Makefiles or other build host
configuration files for ptest so that they can run `make check`, for instance.
These build host files commonly include paths and versions of build host tools,
(eg. rpm or perl) whose presence and version depends on the build host distro.

Optionally strip lines from installed PTEST_BUILD_HOST_FILES using an
extended regex pattern defined in PTEST_BUILD_HOST_PATTERN. This has no
effect on the on-target ptest.

(From OE-Core rev: a3b657470eeea9f1954bc3a3e593a5c4907b6f17)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-03 12:20:01 +00:00
Douglas Royds
d3d09bd064 ptest: Reproducibility: Strip build host paths from any installed Makefile
A common pattern for ptest is to install a Makefile that provides a make check
target. These generated Makefiles are normally full of build host paths, to
hosttools, and to files and scripts within the source or build tree. They also
commonly include the CFLAGS and CPPFLAGS. None of these build host paths
can possibly work on-target, so stripping them has no effect on the success of
the resulting ptests.

Stripping the HOSTTOOLS_DIR has this effect, for instance:

    -MKDIR_P = /home/douglas/workspace/upstream/build/tmp/hosttools/mkdir -p
    +MKDIR_P = mkdir -p

The curious WORKDIR/*= pattern avoids non-parseable -fdebug-prefix-map patterns
in the CFLAGS by adding an arbitrary ".", eg:

    -fdebug-prefix-map=.=/usr/src/debug/rsyslog/8.37.0-r0

In all other cases, we simply remove the WORKDIR, eg:

    CPP = arm-tait-linux-gnueabi-gcc -E --sysroot=recipe-sysroot ...

(From OE-Core rev: 137e38dbae42e9d4d413cd2f9460fb707ca77f9c)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-03 12:20:01 +00:00
Ross Burton
22fe4e27ec binconfig: only try to mangles files
If a file matches the glob that is an absolute symlink then sed will try to
alter files on the host, so restrict the search to just actual files.

(From OE-Core rev: 5556b5ca9efac132754bd82af52a0f68bc4ce85a)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-01 11:38:37 +00:00
Alexander Kanavin
b8c3449613 libc-package: fix postinst error when ENABLE_BINARY_LOCALE_GENERATION = "0"
[YOCTO #13028]

(From OE-Core rev: f630da64b010795482e013362c3fe184dcbd8d25)

Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-01 11:38:37 +00:00
Ross Burton
66dadb754f image_types: use cpio-native to build cpio images
As per the previous commit, upstream cpio has a bug which means it crashes on
append. If the image being built has already had testimage ran then cpio-native
will be in the sysroot.  It's also possible that some distributions are shipping
this broken CVE patch too.

Now that our cpio-native is fixed, until we can be sure that the host cpio isn't
broken depend on cpio-native if building a cpio image.

[ YOCTO #13042 ]

(From OE-Core rev: c3b9aedcbe538d7fa74bd814644b4899769dec46)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-01 11:38:36 +00:00
Kai Kang
8d53253d0a multilib_script: fix packages split
In multilib_script.bbclass it renames script file which listed in
MULTILIB_SCRIPTS. It may mix up packages split. Take package curl as
example, ${bindir}/curl-config is packaged to curl-dev originally. But
it is renamed to curl-config-${MULTILIB_SUFFIX} and packaged to curl
when multilib is enabled.

And expand 'pkg' to fix QA warning:

| WARNING: Variable key FILES_${PN}-dev (
| ${bindir}/curl-config-${MULTILIB_SUFFIX}) replaces original key
| FILES_curl-dev (${includedir} ${FILES_SOLIBSDEV} ... ${bindir}/*-config)

Insert a necessary space to the argument 'value' of d.appendVar() as well.

(From OE-Core rev: 841bcbe429dcab54de3b89a927394750f9ccae60)

Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-27 22:13:11 +00:00
Adrian Bunk
2cba73f2b1 archiver.bbclass: Fix COPYLEFT_LICENSE_{IN, EX}CLUDE
Setting them to empty before inheriting copyleft_filter
made them not getting their documented values there.

(From OE-Core rev: ee1f86ca7affc65265f219f0718f89406c9181a5)

Signed-off-by: Adrian Bunk <bunk@stusta.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-27 22:13:11 +00:00
Ming Liu
12e30bd897 image.bbclass: fix a wrong position blank
A flaw was introduced by commit c5fa6034:
[ image.bbclass: use prependVarFlag for postfuncs ]

it changed to use prependVarFlag instead of appendVarFlag, then the
blank also needs change to adapt it.

(From OE-Core rev: 6085023158ffbfbaf0f3d65ef18054c003d3f463)

Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-27 22:13:11 +00:00
Robert Yang
871fcd2c15 sstate.bbclass: Only remove sstate file when task is existed
This can improve the performance a lot for "bitbake <recipe-native/cross/crosssdk>
-ccleansstate" when there are a lot of sstate files.

For example:
* Before
  $ bitbake quilt-native -ccleansstate
  - Check log.do_cleansstate:
  Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package.tgz*
  Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package_qa.tgz*
  Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package_write_rpm.tgz*
  Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_packagedata.tgz*
  Removing /sstate-cache/*/sstate:quilt-native::0.65:r0::3:*_populate_lic.tgz*
  Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_populate_sysroot.tgz*

  There are no package tasks for quilt-native, so the first 4 lines doesn't
  make any sense, but the glob pattern "sstate-cache/*/*" is very time
  consuming when there are no disk caches. E.g., I have more than 600,000
  sstate files:
  - Without disk caches
  # echo 3 >/proc/sys/vm/drop_caches
  $ time python3 -c 'import glob; glob.glob("/sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package.tgz*")'
    real    4m32.583s
    user    0m5.768s
    sys     0m12.892s

  - With disk caches (e.g., run it in the second time)
  $ time python3 -c 'import glob; glob.glob("/sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package.tgz*")'
    real    0m5.128s
    user    0m2.772s
    sys     0m2.308s

  So the 4 removing *package* commands cost more than 20s or 272s in theory.

* After
  $ bitbake quilt-native -ccleansstate
  - Check log.do_cleansstate:
  Removing /sstate-cache/*/sstate:quilt-native::0.65:r0::3:*_populate_lic.tgz*
  Removing /sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_populate_sysroot.tgz*

  We can see that it saved 20s or 272s in theory.

(From OE-Core rev: bb2d6349ea87f090c58001f0d4348b24c2982cde)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-23 23:35:19 +00:00
Robert Yang
cffefd2eae bugzilla.bbclass: Remove it since obsoleted
It is a still python2 bbclass, so it has been broken since bitbake changed to
python3 which was 2 years ago. No one reported/fixed it for python3 in recent 2
years. So we can assume that no one uses it anymore.

(From OE-Core rev: 7f6da5fb54cbcf8e358e988382f45839a8b80019)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-23 23:35:19 +00:00
Robert Yang
00379e9589 classes: Remove tab indentations in python code
Use 4 spaces to replace a tab.

(From OE-Core rev: 55eaf8779170b9396e94dc4a44667824c4f36363)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-23 23:35:19 +00:00
Douglas Royds
3518cdde5b ptest: Reproducibility: Take control of umask
The build host umask was leaking into the thing-ptest packages
at do_install_ptest() time.

(From OE-Core rev: 891343e8ba6490ca3e1876c892269b611ddc7877)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-23 23:35:19 +00:00
Alexey Brodkin
c0f89c609d u-boot: Add mkenvimage tool
This utility is used for creation of images containing
usable in run-time U-Boot environment.

As of today this utility is added per-board like here [1]
for Intel Edison board.

[1] http://git.yoctoproject.org/cgit/cgit.cgi/meta-intel-edison/tree/meta-intel-edison-bsp/recipes-bsp/u-boot/u-boot-tools_2014.04.bb

Given there're quite some U-Boot tools that we may want to add later
this recipe name switch from "u-boot-mkimage" to generic "u-boot-tools"
still for compatibility we provide "u-boot-mkimage" with help
of PROVIDES as well as proposed "u-boot-mkenvimage".

(From OE-Core rev: 314885b16e5d26d27d46a4bfb0d581b27a03b8fe)

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Cc: Richard Purdie <richard.purdie@linuxfoundation.org>
Cc: Otavio Salvador <otavio@ossystems.com.br>
Cc: Martin Jansa <martin.jansa@gmail.com>
Cc: Ross Burton <ross.burton@intel.com>
Cc: Marek Vasut <marex@denx.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-23 23:35:19 +00:00
Douglas Royds
721c000b00 reproducible: Don't look for youngest file when no source tarball
Some packages (eg. init-ifupdown) take their source files entirely from
openembedded-core, that is, they download no source tarball.
These recipes either don't use S at all (ie. it is empty at unpack time),
or they set S = WORKDIR (as in init-ifupdown).
Looking at the file timestamps in the WORKDIR causes a non-reproducible
SOURCE_DATE_EPOCH, as files taken from file:// URIs do not have
reproducible timestamps.

If S == WORKDIR, we are better to assume that there is no source tarball,
and to fall back to a fixed timestamp for the SOURCE_DATE_EPOCH.
This makes the init-ifupdown build reproducible.

(From OE-Core rev: d395bad0179037eb5d0fa4d921985c87ae13f3a4)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-23 23:35:19 +00:00
Douglas Royds
59c8937095 reproducible: Refactor: Break out fixed_source_date_epoch() function
(From OE-Core rev: 4eb6def4fe82959c2a348142b9eada27d3354aef)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-23 23:35:19 +00:00
Robert Yang
76276d90ce sstate.bbclass: set SSTATE_EXTRAPATHWILDCARD explicitly
The glob.glob("/sstate/*/*/") is very time consuming, set
SSTATE_EXTRAPATHWILDCARD explicity to avoid that. This can save a lot of time
when there are many sstate files.

For example, I have more than 600,000 sstate files:
* Before
  - Without disk caches
  $ time python3 -c 'import glob; glob.glob("/sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_populate_sysroot.tgz*")'
    real    4m32.583s
    user    0m5.768s
    sys     0m12.892s

  - With disk caches
  $ time python3 -c 'import glob; glob.glob("/sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_populate_sysroot.tgz*")'
  real    0m4.111s
  user    0m2.348s
  sys     0m1.756s

* After
  $ time python3 -c 'import glob; glob.glob("/sstate-cache.bak/universal/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_populate_sysroot.tgz*")'
  - Without disk caches:
  real    0m7.928s
  user    0m0.172s
  sys     0m0.124s

  - With disk caches:
  real    0m0.131s
  user    0m0.088s
  sys     0m0.044s

We can see that it saves about 3.8s with disk caches, and saves about 264s
without disk caches.

(From OE-Core rev: 8b31c919814b8bdf25b3381053656523c001ae0d)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-23 23:35:18 +00:00
Yeoh Ee Peng
fea124ae44 testimage: Add support for slirp
Enable testimage to support qemu slirp. Configure "QEMU_USE_SLIRP"
& "TEST_SERVER_IP" variables to enable slirp.

[YOCTO#10713]

(From OE-Core rev: 3df9ee85ce7fe52f0893fd33aea3bf1fcc6ead0a)

Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-23 23:35:18 +00:00
Joshua Watt
5dd8df08c5 meta/icecc.bbclass: Update system blacklists
Updates the system blacklists to include packages that are known to have
problems compiling under icecream

(From OE-Core rev: fc5418e7bbdecfb27bafe595084e0fd0f991a388)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-21 11:54:14 +00:00
Joshua Watt
1da297d354 meta/icecc.bbclass: Move system blacklist to variables
The system blacklists are moved to variables which are ignore when
hashing. This prevents changes to the blacklists from causing all
taskhashes to change (and thus rebuild).

(From OE-Core rev: f5be9f6e9180ace3362bba52c7ced3b039441d7d)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-21 11:54:14 +00:00
Joshua Watt
1554c7c3a6 classes/testsdk: Split implementation into classes
Splits the SDK test implementation into configurable Python classes. The
classes used for the normal and extensible SDKs are
${TESTSDK_CLASS_NAME} and ${TESTSDKEXT_CLASS_NAME} respectively.

This allows SDK machines to override the classes used to implement the
tests. For the traditional SDK, a common "run()" function is provided by
the class (oeqa.sdk.testsdk.TestSDK), with several hook member functions
that can be overridden in child classes, making it easier to have
consistent behavior. The extensible SDK class
(oeqa.sdkext.testsdk.TestSDKEXT) also has a common "run()" function, but
no hooks have yet been added as there is not currently a known use case
for create derived classes.

These changes should be purely organizational; no functional changes
have been made to either the standard SDK or extensible SDK tests.

[YOCTO #13020]

(From OE-Core rev: a06d53928b22d5f88276023c4d57b206db2f27f9)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-21 11:54:14 +00:00
Richard Purdie
9c55d8d8fe sanity: Add check for WSL
Users are starting to expect OE to work under WSL which it doesn't. Add a warning to
tell them about this up front and manage expectations.

(From OE-Core rev: 62fe04c2451f612b9ee5eba469c7724b63484499)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-16 11:46:07 +00:00
Richard Purdie
fabffda42e testimage: Enable autorunning of the package manager testsuites
Now that the hangs in httpservice are fixed we can let these tests
auto skip as appropriate.

(From OE-Core rev: 42a0d70291d551578e21f590fcb85ca72a78ccb5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-16 11:46:07 +00:00
Richard Purdie
99f0e68973 testimage/testsdk/selftest: Avoid platform.distro_identifier deprecation warnings
Use our own lsb function instead as used elsewhere by the codebase.

(From OE-Core rev: acac45a6fd604d28ef7c23d67482af3d7e8bcfe3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-16 11:46:07 +00:00
Ming Liu
e5a43f6386 image.bbclass: add a missing blank
When calling d.appendVarFlag, a blank is needed or else it could mess
up the later appended variables.

Reported-by: Stefan Agner <stefan.agner@toradex.com>
(From OE-Core rev: 8b2df6e5874b932b356847c5bde10d5b31d67213)

Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-16 11:46:07 +00:00
Wes Lindauer
10270f4222 module.bbclass: Add INSTALL_FW_PATH for out-of-tree modules
This same variable was added to the make line in kernel.bbclass in
0decf1cc1c. With the addition of the
usrmerge feature, out-of-tree modules could use this same variable.

(From OE-Core rev: 1d24d1ac5b9c772d84104396438e5d1308332785)

Signed-off-by: Wes Lindauer <wesley.lindauer@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-16 11:46:07 +00:00
Andreas Müller
1c7bc2ff9d scons.bbclass: fix indention
(From OE-Core rev: ae12c5f86601a81f4208fd371ce8803464aedaa0)

Signed-off-by: Andreas Müller <schnitzeltony@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-14 11:14:40 +00:00
Andreas Müller
11af905ac7 scons.bbclass: Force rebuild if necessary
(From OE-Core rev: dd462c3b3f449805e104be402b23fbfabcaefe8f)

Signed-off-by: Andreas Müller <schnitzeltony@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-14 11:14:40 +00:00
Richard Purdie
0577d02fae license_image: Fix rootfs file ownership issues with COPY_LIC_DIRS
We need to set ownership of the license files after copying into the rootfs.
We also need to ensure we don't overwrite the rootfs files outside rootfs
generation, hence only write there when the new rootfs function parameter
is True.

[YOCTO #12961]

(From OE-Core rev: 6aa0eeaa4d31a440700f7695174de2a1dcc54866)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-14 11:14:40 +00:00
Alexey Brodkin
ef63723965 locale: Allow usage of cross-localedef for ARC
With this it's possible to build locale data for ARC
and not do it instead on the first boot.

(From OE-Core rev: f13c303491dc8850126ea14baedc7b63b7b5ecf4)

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-14 11:14:40 +00:00
Richard Purdie
8291cd25d8 testimage: Further cleanup DEFAULT_TEST_SUITES
Now the test markup of the development tools tests is complete, this
can be further tweaked to auto run the correct tests.

(From OE-Core rev: bd4f8d12fe1f9f2643ee9e68fa2bb981134294fb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-14 11:14:40 +00:00
Richard Purdie
8c4ba1afdd testimage: Simplfy DEFAULT_TEST_SUITES logic
Now that the tests have correct markup to automatically determine which images
they can run against, clean up the default test suites logic to be simpler
and not image specific.

Some cleanup of the compiler tests still needs to be completed but this
is a good first step.

The only downside to this is more noise during testing as we now see
many skipped messages for simple images like core-image-minimal.

The auto type is being removed since it currently breaks badly due to the
socat mandatory inclusion from the meta-selftest layer which is a problem
which needs to be addressed seperately.

(From OE-Core rev: 4966bc33845752eb0aeae54b72e8ba0146a7ed52)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-14 11:14:40 +00:00
Richard Purdie
7ed0af48df testimage: Remove duplicate dependencies
(From OE-Core rev: 16d4368e7b85d5179374adb0a2d1ac215793e104)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-14 11:14:39 +00:00
Nathan Rossi
97c9fb7ba0 base.bbclass: Display name of licenses which caused SkipRecipe
Display the name of the restricted licenses which caused the recipe to
be skipped. This makes it easy to determine which license or licenses
are missing and need to be checked and whitelisted.

(From OE-Core rev: b71cd1ec45e247db688b784697829c1b485ca9ca)

Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-14 11:14:39 +00:00
Jens Rehsack
62c15051ce cpan.bbclass: avoid newer MakeMaker run perl.real
In newer versions of ExtUtils::MakeMaker, the location of the perl
binary to run is determined differently which results in finding
${bindir}/perl.real instead of ${bindir}/perl

(From OE-Core rev: 362bd36cbec052d2a37710498c901b699dcf5ca7)

Signed-off-by: Jens Rehsack <sno@netbsd.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-14 11:14:39 +00:00
Andrej Valek
1a2ae051ae rootfs: ensure that packages in IMAGE_INSTALL_DEBUGFS are existing
Ensure that additional packages in IMAGE_INSTALL_DEBUGFS are available
before installation.

(From OE-Core rev: 06a1bbea67a7233deff80ebe3681b4e0cca6817a)

Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-14 11:14:39 +00:00
Richard Purdie
f702509017 metadata_scm: Drop orphaned monotone functions
Its been 'a while' since we used monotone, drop these revision
function remnants.

(From OE-Core rev: f1c77ab87b3c16d14deff801f48292ed348da637)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-14 11:14:39 +00:00
Robert Yang
a0618f5726 metadata_scm.bbclass: simplify base_detect_revision()
The previous code looks strange because of historical reason, simply return
base_get_metadata_git_revision/branch() is enough, the later functions has
handled everything like '<unknown>' and strip().

(From OE-Core rev: 8a6e70960e021e10a1d02e20e0519fdbdeac8616)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-14 11:14:39 +00:00
Bruce Ashfield
d2aaa2614a kernel: use olddefconfig as the primary target for KERNEL_CONFIG_COMMAND
As was warned by commit 312ee68752fa [kconfig: announce removal of
oldnoconfig if used], oldnoconfig has been removed from the 4.20 kernel.

So we switch our default mode to olddefconfig.

commit fb16d8912 [kconfig: replace 'oldnoconfig' with 'olddefconfig',
and keep the old name as an alias] introduced olddefconfig in the 3.10
kernel, we shuffle oldnoconfig to the fallback target.

The fallback mode allows kernels between 3.10 and the currently listed
oldest kernel of 3.2 to continue to configure.

(From OE-Core rev: 8593dcb7e8c938530ff00ffedf7f3d02d26c3bad)

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-14 11:14:38 +00:00
Richard Purdie
d3b55a3f67 image-buildinfo,oeqa/selftest/containerimage: Ensure image-buildinfo doesn't break tests
Having image-buildinfo enabled causes containerimage.ContainerImageTests.test_expected_files
to fail due to the presence of an unexpected file:
  ['./',
   './etc/',
-  './etc/build',
   './etc/default/',
   './etc/default/postinst',

Tweak the class to allow it to be disabled and disable it from the test just in
case it was enabled.

(From OE-Core rev: af67bf422a4df5b7e07894512ff73a5f493682ab)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-09 15:27:34 +00:00
Matthias Schiffer
f4de5d85b8 base.bbclass: avoid 'find -ignore_readdir_race -delete'
Due to a bug in find [1], -ignore_readdir_race does not work correctly with
-delete. This can lead to spurious build failures when files disappear
while such a command is running; specifically this was seen in the case of
do_configure and do_populate_lic running concurrently for packages
with ${B} == ${WORKDIR}:

   find: '.../sstate-build-populate_lic': No such file or directory

While the issue is fixed in the findutils git master, the find command of
the host system is called here, so we can't ensure that the used version
contains the fix. Many common distros have not updated to a recent enough
findutils version yet (Ubuntu 18.10 contains the fix, while 18.04 is still
affected).

Work around the issue by passing the output of find to 'rm -f' instead of
using -delete.

[1] https://savannah.gnu.org/bugs/?52981

(From OE-Core rev: 8079e2d62e23f7c274f46185e6dad64fa95394c1)

Signed-off-by: Matthias Schiffer <matthias.schiffer@ew.tq-group.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-09 15:27:34 +00:00
Fabien Lahoudere
82da3463c8 archiver: Drop unwanted directories
In sources directory we can find patches/ and temp/.
The first one is filled with symbolic link unusable on another
machines.
The second contains yocto logs to create this archives and are
typically copied when 'S = "${WORKDIR}"'

(From OE-Core rev: 3904f98851c6a63dd9377e38f1432be6b1c0a94d)

Signed-off-by: Fabien Lahoudere <fabien.lahoudere@collabora.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-07 23:08:55 +00:00
Richard Purdie
8929ffd67c oe-init-buildenv/base: Relax python version checks in favour of HOSTTOOLS manipulation
Several distros are now shipping "python" as python v3 contra to the original
python guidelines. This causes users confusion/pain in trying to use our tools.

We can just force "python" to "python2" within HOSTTOOLS to avoid this issue
and hide the complexity from the user.

(From OE-Core rev: b06a6cde5c5503f456f260c773cf126085e18c8d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-07 23:08:55 +00:00
Richard Purdie
d21cf7ca12 crosssdk: Remove usage of host flags for cross-compilation
Similarlly to OE-Core rev 4b936cde58ca0a6f34092ce82640a02859110411 for
cross.sdk, BUILD_* flags can't be used as TARGET_* flags

gcc-crosssdk buils leaks config.log's through "gcc-stashed-builddir" and
TARGET_* flags to libgcc cross-build through "gcc/libgcc.mvars" file
on "gcc-stashed-builddir". This means that if BUILD_CFLAGS contains
host-specific flags like "-isystem/usr/include" libgcc build will
fail "do_qa_configure" and "do_package_qa" checks.

Remove host-related flags from TARGET_* flags for gcc-crosssdk builds.

[YOCTO #11874]

(From OE-Core rev: 6e162e619b6f5173c073cd9bedbcadf205017e30)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-07 23:08:55 +00:00
Richard Purdie
e06de80a61 go: Change from TARGET_ARCH to TUNE_PKGARCH
Right now go-cross is changing signatures when you change TUNE for a given
architecture. In particular this breaks layer tests like:

yocto-check-layer ../meta-yocto-bsp/ --machines qemuarm beaglebone-yocto

This changes the PN addtion to something containing the tune rather than
the arch which avoids these kinds of errors. If go-cross can be tune
independent that would be nice but currently that isn't the case.

[YOCTO #12586]

(From OE-Core rev: e3c7e1703499e6a5332d9ab8a941671ec8235c4f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-07 23:08:55 +00:00
Douglas Royds
87b140abdb reproducible_build: update do_deploy_source_date_epoch commment
Once the value of SOURCE_DATE_EPOCH is determined, it is stored in the recipe's SDE_FILE.
If none of the existing mechanisms are suitable, replace the do_deploy_source_date_epoch task
with recipe-specific functionality to write the appropriate SOURCE_DATE_EPOCH into the SDE_FILE.

(From OE-Core rev: e7b891b76954c784f5a93bd0a1c91315673ce40d)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-07 23:08:54 +00:00
Victor Kamensky
4f7deccda2 meson: map powerpc64 TARGET_ARCH to ppc64 for the cross file
Meson uses 'ppc64' for 64 bit powerpc. Issue came up while
building systemd for MACHINE that uses ppc64e5500 tune.

(From OE-Core rev: eccd5414c37be26df63a90154c1808f6f5618b7d)

Signed-off-by: Victor Kamensky <kamensky@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-29 17:26:47 +00:00
Richard Purdie
1fd2d4acfc testsdk: Improvements to the json logging
Tweak the preceeding commit to:
* Add STARTTIME to the identifier to make it unique
* Add MACHINE to the identifier
* Use LOG_DIR
* Store the layer config in a more natural json format
* Drop '_' function prefixes

(From OE-Core rev: c272079d662237b1b9f87940c88cbadaeda9a9d1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-29 17:26:47 +00:00
Yeoh Ee Peng
fc16be1787 testsdk.bbclass: write testresult to json files
As part of the solution to replace Testopia to store testresult,
OEQA sdk and sdkext need to output testresult into json files, where
these json testresult files will be stored into git repository
by the future test-case-management tools.

By default, json testresult file will be written to "oeqa"
directory under the "WORKDIR" directory.

To configure multiple instances of bitbake to write json testresult
to a single testresult file at custom directory, user will define
the variable "OEQA_JSON_RESULT_DIR" with the custom directory for
json testresult.

(From OE-Core rev: cdbedba7da9c3f97a642d79a5e8ec363ba938e35)

Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-29 17:26:47 +00:00
Richard Purdie
8daaae4206 testimage: Improvements to the json logging
Tweak the preceeding commit to:

* Add STARTTIME to the identifier to make it unique
* Log DISTRO
* Use LOG_DIR
* Store the layer config in a more natural json format
* Drop '_' function prefixes

(From OE-Core rev: dda94cfa708cf3c6e5408d6dbcece360b4e8cd27)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-29 17:26:47 +00:00
Yeoh Ee Peng
e73bcdd059 testimage.bbclass: write testresult to json files
As part of the solution to replace Testopia to store testresult,
OEQA testimage need to output testresult into json files, where
these json testresult files will be stored into git repository
by the future test-case-management tools.

By default, json testresult file will be written to "oeqa"
directory under the "WORKDIR" directory.

To configure multiple instances of bitbake to write json testresult
to a single testresult file at custom directory, user will define
the variable "OEQA_JSON_RESULT_DIR" with the custom directory for
json testresult.

(From OE-Core rev: 5d135d4769b6bb60d575eb6ed196367f9e077cc4)

Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-29 17:26:47 +00:00
Grygorii Tertychnyi
e93457c151 cve-check: Allow multiple entries in CVE_PRODUCT
There are both "curl" and "libcurl" CPEs in NVD.
All "curl" CVEs are currently missing in the reports.

Hence, switch "CVE_PRODUCT" to a space separated list.
It is useful for recipes generating several packages,
that have different product names in NVD.

(From OE-Core rev: 404f75e026393ddc55da87f6f04fb1201cff4e11)

Signed-off-by: Grygorii Tertychnyi <gtertych@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-29 17:01:45 +00:00
Armin Kuster
3be5331986 testimage.bbclass: define ptest suite variable
Ptest has dependencies on other tests so contain them in a
variable. This allows layers to not worry about maintaining the depends
themselves.

(From OE-Core rev: c5c5ad9030ec7ddaa6387e2ba52e9e37b9ea76a2)

Signed-off-by: Armin Kuster <akuster@mvista.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-25 14:53:44 +01:00
Ioan-Adrian Ratiu
519aaef791 kernel.bbclass: allow custom KERNEL_IMAGEDEST
Some distros want to install the kernel in a custom location other
than /boot and have it properly packaged, so it's useful to use a
weaker assignment.

(From OE-Core rev: cadd91655a336eb6d65ca43dc1231bbe5271e1cf)

Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-25 14:53:44 +01:00
Jef Driesen
3a184fdfa2 npm.bbclass: allow overriding NPM_INSTALL_DEV
Commit b1f10b18ea added the NPM_INSTALL_DEV
variable. But due to the use of simple assignment, a recipe can't override its
value.

(From OE-Core rev: b53d5094d5e05f9c1955c8565d777ad74f668f7e)

Signed-off-by: Jef Driesen <jef.driesen@niko.eu>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-25 14:53:44 +01:00
Richard Purdie
e9270af429 npm: change install directory to upstream default
The node binary searches for packages in a number of locations, the last
of which is $PREFIX/lib/node (here: /usr/lib/node) from the list of
GLOBAL_FOLDERS [1]. So change the installation directory for all
packages depending on npm.bbclass to that location. This removes the
need to define the NODE_PATH variable to the non-standard
/usr/lib/node_modules value.

While the Tips for Package Managers [2] discusses installing packages to
/usr/lib/node_modules/<name>/<version>, this has several drawbacks:

 * it does not work for the REPL as mentioned in the documentation
 * it also does not work for any code _not_ installed as a global
   package under /usr/lib/node_modules (e.g. /usr/share/foo.js will not
   find any packages below /usr/lib)
 * using the non-default location and then having to set NODE_PATH
   barely saves any time: there are only two file-system lookups (to the
   legacy $HOME/.node_modules and $HOME/.node_libraries) directories
   before the library would be found

And the suggestion was made in the context of deduping the node_modules
tree by installing all packages in a flat hierarchy and using symlinks
to the correct version of each dependency. This is not what OpenEmbedded
does, so none of those benefits (deduping, cleaner packages) are being
had by shifting the installation directory to /usr/lib/node_modules.

The choice of a "proper" installation path is not helped by npm
installing to /usr/lib/node_modules if asked to install globally. Still,
using the location expected by nodejs (/usr/lib/node) seems the right
choice.

[1]: https://nodejs.org/api/modules.html#modules_loading_from_the_global_folders
[2]: https://nodejs.org/api/modules.html#modules_addenda_package_manager_tips

(From OE-Core rev: c73bc49038effd64f2c2542c1f4da8b6a4168477)

Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-25 14:53:44 +01:00
Olaf Mandel
2713d9bcc3 npm: change install directory to upstream default
The node binary searches for packages in a number of locations, the last
of which is $PREFIX/lib/node (here: /usr/lib/node) from the list of
GLOBAL_FOLDERS [1]. Change the installation directory for all packages
depending on npm.bbclass to that location. This removes the need to
define the NODE_PATH variable to the non-standard /usr/lib/node_modules
value.

While the Tips for Package Managers [2] discusses installing packages to
/usr/lib/node_modules/<name>/<version>, this has several drawbacks:

 * it does not work for the REPL as mentioned in the documentation
 * it also does not work for any code _not_ installed as a global
   package under /usr/lib/node_modules (e.g. /usr/share/foo.js will not
   find any packages below /usr/lib)
 * using the non-default location and then having to set NODE_PATH
   barely saves any time: there are only two file-system lookups (to the
   legacy $HOME/.node_modules and $HOME/.node_libraries) directories
   before the library would be found

And the suggestion was made in the context of deduping the node_modules
tree by installing all packages in a flat hierarchy and using symlinks
to the correct version of each dependency. This is not what OpenEmbedded
does, so none of those benefits (deduping, cleaner packages) are being
had by shifting the installation directory to /usr/lib/node_modules.

[1]: https://nodejs.org/api/modules.html#modules_loading_from_the_global_folders
[2]: https://nodejs.org/api/modules.html#modules_addenda_package_manager_tips

(From OE-Core rev: 2036137151929b541293154ff529475071cd92b0)

Signed-off-by: Olaf Mandel <o.mandel@menlosystems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-20 22:40:16 +01:00
Olof Johansson
32914a1780 devtool-source.bbclass: Only create each patch branch once
For conditonally applied patches based on SRC_URI overrides, the
devtool-source class would try to create a new branch for each override
assignment as a postfunc to do_patch, but if the same override was used
multiple times, it would try to create the same branch multiple times,
causing errors like

> Exception: bb.process.ExecutionError: Execution of
\   'git checkout f0f0f0f0f0ff0f0f0f0f0f0f0f0f0f0f0ff0f0f0 -b devtool-override-foo'
\   failed with exit code 128:
> fatal: A branch named 'devtool-override-foo' already exists.

This change makes sure that the devtool-source bbclass will only create
one branch per override.

(From OE-Core rev: 95a921959d340f74b5604df57737c1eeaad0023e)

Signed-off-by: Olof Johansson <olofjn@axis.com>
Reviewed-by: Peter Kjellerstedt <pkj@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-20 22:40:16 +01:00
Richard Purdie
72a92f6e42 image_types: Fix a shell syntax error in do_image_ubi
| DEBUG: Executing shell function do_image_ubi
| /home/pokybuild/yocto-worker/nightly-oe-selftest/build/build-st-31289/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/temp/run.do_image_ubi.7928: 123: [: missing ]

(From OE-Core rev: e1c6442872c9361b6b61a83adcce9cade2f2ecd2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-20 22:40:16 +01:00
Aníbal Limón
07fab73745 meta/classes/testexport: Remove __pycache__ from exported tests
The __pycache__ is created based on python3 version in the build machine
and isn't a garantee to be useful on the system that runs exported tests.

(From OE-Core rev: 405e822e9da4b53ec9ee8019364fdcb13ccc2587)

Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-18 23:26:35 +01:00
Andrej Valek
5f30534c47 rootfs: possibility to add packages to debug rootfs
Static libraries are not included in rootfs, it means, that sources are
not going into debug rootfs. This option enables to install additional
packages even if the standard package is not installed.

(From OE-Core rev: 708fe1df3d3e22dd693ae7bbfdd3e5af1a1b0bdc)

Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-18 10:56:55 +01:00
Changqing Li
603f808f59 update-rc.d: add -f into postinst OPT
add -f into postinst OPT to fix run postinst scriptlet fail
problem during do rootfs, while INITSCRIPT_PARAMS set to remove.

+ type update-rc.d
+ '[' -n qemux86_64-poky-linux/core-image-sato/1.0-r0/rootfs ']'
+ OPT='-r qemux86_64-poky-linux/core-image-sato/1.0-r0/rootfs'
+ update-rc.d -r work/qemux86_64-poky-linux/core-image-sato/1.0-r0/rootfs keepalived remove
update-rc.d: work/qemux86_64-poky-linux/core-image-sato/1.0-r0/rootfs/etc/init.d/keepalived exists during rc.d purge (use -f to force)
D: %post(keepalived-1.4.2-r0.core2_64): waitpid(305974) rc 305974 status 100

(From OE-Core rev: 45224526edbe65012a846c066d2b539a360f0c51)

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-16 20:35:43 +01:00
Hongxu Jia
4e1c5499fe gobject-introspection.bbclass: fix gir installed but not shipped in any package while multilib enabled
Since commit [9524330 gobject-introspection: fix multilib install
file conflicts] applied in oe-core, while multilib enabled, gir
files will be installed to `${libdir}'.

Refer above commit, modify gobject-introspection.bbclass to
split gir to package correctly.

(From OE-Core rev: e0cc1dc6f1c14babe29a29a7211dc1c69d782b95)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-16 20:35:43 +01:00
Peter Kjellerstedt
0cdcce13bc useradd_base.bbclass: Make perform_groupmems handle overlapping usernames
If the name of the last user being part of the group had a name that
was a prefix of the user being added, then perform_groupmems() would
treat it as if the user already existed in the list of users and not
add it.

Reported-by: Peter Henricsson <peter.henricsson@axis.com>
(From OE-Core rev: 3bab0416f20366e75444be9b65fb1369643f103a)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-16 20:35:43 +01:00
Mark Asselstine
6dd32df4b4 meson.bbclass: Fix build issues with /tmp mounted with noexec
Since commit d297f7ebf3f6 [fribidi: use Meson instead of autotools]
build failures have been observed with this package. The immediate
issue was related to improperly named #defines per
https://github.com/fribidi/fribidi/commit/46f52d588ab5, however, the
root cause was FRIBIDI_SIZEOF_INT getting a value of "-1".

After searching the meson logs the following was found:

  Could not run: /tmp/tmp2fxe6ha1/output.exe (error: [Errno 13] Permission denied)
  Checking for size of "int": -1

Which pointed to the real root cause being /tmp mounted with noexec, a
common configuration on Redhat and other distros. This issues has been
raised in the meson community:
https://github.com/mesonbuild/meson/issues/2972 but is yet to be
addressed.

Using the discussion from issue#2972 and the fact that the underlying
code makes use of python 'tempfile' we can simply create a 'tmp'
directory and make use of TMPDIR to avoid this issue.

(From OE-Core rev: 9800daf59d2235bc492d1aeb600e46ad62303510)

Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-12 16:57:21 +01:00
Richard Purdie
d91a282947 testimage: Ensure opkg/apt test suites are run
Currently only the dnf package manager tests are automatically added to the
list of tests to run. Improve the code to handle automatic addition of
the apt and opkg tests too.

(From OE-Core rev: c9169e4d1fa67289d5beb0e1f8fb5c54153c4242)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-12 16:57:21 +01:00
Richard Purdie
64ca57fe52 testimage: Ensure package-index is triggered for opkg/apt tests
If the opkg/apt tests are run without a package-index they will fail.
Trigger this here for now as a dependency until the code can be improved.

(From OE-Core rev: 7805eec0f522da2d2b6a08c8a07a9243e26c014a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-11 23:06:35 +01:00
Richard Purdie
3ad7ccc21d testimage: Ensure TESTIMAGEDEPENDS is appended to for qemuall override
The current qemuall override overwrites other dependencies unintentionally.
Tweak the code to avoid this by appending to the variable.

(From OE-Core rev: fc8df45e6b27dd7a26b7bf88a06ffa831fcf075c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-11 23:06:35 +01:00
grygorii tertychnyi via Openembedded-core
bc8f229366 lib/oe/utils: add eol to format_pkg_list()
Append '\n' to the non-empty formatted string before return. If you
write it to the (manifest) file, it will ensure file ends with a newline.

Many GNU utilities have problems processing the last line of a file
if it is not '\n' terminated. E.g. if the last line is not terminated
by a newline character, then "read" will read it but return false,
leaving the broken partial line in the read variable(s).
It can also break or adversely affect some text processing tools,
that operate on the file.

(From OE-Core rev: ee4d0c879713ba50dc6cc3300f44647faebee2e0)

Signed-off-by: grygorii tertychnyi <gtertych@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-11 23:06:35 +01:00
Ross Burton
f997e13d03 archiver: remove left-over debugging
(From OE-Core rev: 6fe833611f985198e3ca038e5095f0d03249ce82)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-10 12:47:35 +01:00
Jens Rehsack
2499589fb6 populate_sdk_ext.bbclass: avoid populating tools twice
When running `bitbake -c populate_sdk_ext ...` for an image
recipe inheriting populate_sdk_qt5, `nativesdk-qemu-helper` is
deployed via `nativesdk-packagegroup-sdk-host`, which already
contains some of the tools (by name: "oe-find-native-sysroot
runqemu*").

To avoid error like:
    Configuring nativesdk-packagegroup-qt5-toolchain-host.

    NOTE: Running intercept scripts:
    NOTE: > Executing update_gio_module_cache-nativesdk intercept ...
    NOTE: + [ True = False ]
    + qemu-x86_64 -r 3.2.0 -E LD_LIBRARY_PATH=/home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/fischer-poky-linux/updatable-app-image/1.0-r0/sdk-ext/image/opt/mops/2.5+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib:/home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/fischer-poky-linux/updatable-app-image/1.0-r0/sdk-ext/image/opt/mops/2.5+snapshot/sysroots/x86_64-pokysdk-linux/lib -L /home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/fischer-poky-linux/updatable-app-image/1.0-r0/sdk-ext/image /home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/fischer-poky-linux/updatable-app-image/1.0-r0/sdk-ext/image/opt/mops/2.5+snapshot/sysroots/x86_64-pokysdk-linux/usr/libexec/nativesdk-gio-querymodules /home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/fischer-poky-linux/updatable-app-image/1.0-r0/sdk-ext/image/opt/mops/2.5+snapshot/sysroots/x86_64-pokysdk-linux/usr/lib/gio/modules/

    NOTE: Executing copy_buildsystem ...
    DEBUG: Executing python function copy_buildsystem
    NOTE: Generating sstate task list...
    NOTE: Generating sstate-cache...
    NOTE: Generating sstate-cache...
    DEBUG: Python function copy_buildsystem finished
    NOTE: Executing install_tools ...
    DEBUG: Executing shell function install_tools
    Traceback (most recent call last):
      File "/home/sno/gpw-community-bsp/sources/poky/scripts/lnr", line 21, in <module>
	os.symlink(target, linkname)
    FileExistsError: [Errno 17] File exists: '../../../../layers/poky/scripts/oe-find-native-sysroot' -> '/home/sno/gpw-community-bsp/mops-yocto-platform/tmp/work/fischer-poky-linux/updatable-app-image/1.0-r0/sdk-ext/image//opt/mops/2.5+snapshot/sysroots/x86_64-pokysdk-linux/usr/bin/oe-find-native-sysroot'
    WARNING: exit code 1 from a shell command.
    DEBUG: Python function do_populate_sdk_ext finished

only add those tools unpackaged to the deployment which are
still missing.

(From OE-Core rev: 3274819ce3bb64316853c6f0bb7840c38635c26c)

Signed-off-by: Jens Rehsack <sno@netbsd.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-10 12:47:35 +01:00
Andrej Valek
38a4f6e4f8 populate_sdk_base: move template files location to variables
Let users to use custom toolchain template files instead of overloading
the whole function.

(From OE-Core rev: d4d5fdadaf2f671559326ab594fb596995846ca4)

Signed-off-by: Andrej Valek <andrej.valek@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-10 12:47:35 +01:00
Tom Rini
6006d4dd3f rootfs_*: Also depend on do_package_qa
Make sure that we don't create images that have QA problems with their
package files by adding do_package_qa to do_rootfs[recrdeptask].

(From OE-Core rev: 4ee2f87f0c9540c7e2c866dbc9739e21a618bddf)

Signed-off-by: Tom Rini <trini@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-09 19:04:02 +01:00
Ola x Nilsson
244ba49784 externalsrc.bbclass: Set BB_DONT_CACHE for non-target recipes
BB_DONT_CACHE was not set for non-virtual recipes where PN != BPN, such
as quilt-native.  Recipes that do not set BBCLASSEXTEND should always
have BB_DONT_CACHE set by externalsrc.

(From OE-Core rev: 4eff427a0ee629a1541a420a9591411648569a97)

Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-08 14:13:54 +01:00
Angelo Dureghello
5c8874f322 kernel-arch.bbclass: add m68k to kernel arch map
(From OE-Core rev: 69be35a58ce4122763c833e13cdbcc8e7fb9c52d)

Signed-off-by: Angelo Dureghello <angelo@sysam.it>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-08 14:13:54 +01:00
Angelo Dureghello
8d5f59f2da siteinfo: define data for m68k
(From OE-Core rev: d8c2a5fa7d59580dd70b7ec57450e3b7c9a7d048)

Signed-off-by: Angelo Dureghello <angelo@sysam.it>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-08 14:13:54 +01:00
Robert Yang
bdb4964787 go.bbclass: Add -buildmode=pie for non mips arch
Fixed QA issue like:
WARNING: runc-docker do_package_qa: QA Issue: ELF binary '/path/to/runc-docker/usr/bin/runc' has relocations in .text [textrel]

(From OE-Core rev: 8dcd4e6e791c3a8b8bf5e69dca9bb9c887231d69)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-01 13:04:50 +01:00
Ross Burton
4771a7c2ed archiver: generalise task running when in configured mode
When in configured mode the archive will re-run configure itself, but that
doesn't work if there are other tasks that need to run such as cmake's
generate_toolchain_file.

Instead of hard-coding a list of classes and tasks, obtain the list of tasks
preceeding do_configure, filter out ones we don't want to re-run such as
do_patch and do_prepare_recipe_sysroot, and run those too.

(From OE-Core rev: fe26382f94d8cb7d2453d9937aee451b757252b8)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-01 13:04:50 +01:00
Ross Burton
e85f567f96 archiver: check tasks exist before adding dependencies
Some recipes don't have do_configure, or have them but are marked as noexec.
Check for their existance before hooking up dependencies.

[ YOCTO #12868 ]

(From OE-Core rev: 5c8aa1f41f04519a8ba3997c0d8e3a71fe270fae)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-01 13:04:50 +01:00
Robert Yang
a1fd97e1c7 multilib: avoid expanding grub and grub-efi to multilib
It doesn't make much sense to expand them to multilib, and there is an error on
qemuarm64 since grub-efi supports arm64, but doesn't support armv7a or armv7ve:

* Fixed:
  MACHINE = "qemuarm64"
  require conf/multilib.conf
  MULTILIBS = "multilib:lib32"
  DEFAULTTUNE_virtclass-multilib-lib32 = "armv7a"
  MACHINE_FEATURES_append = " efi"

  $ bitbake lib32-core-image-minimal

Also introduced a variable NON_MULTILIB_RECIPES in multilib.conf, so that we
can easily add other recipes, such as syslinux if needed.

(From OE-Core rev: 25f7c6c329038b443d36074fff45a30ba3712f7a)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-01 13:04:49 +01:00
Changqing Li
ad2a0cd5b0 mime.bbclass: fix do_package_qa_multilib error of lib32-fltk
fltk.bb inherit mime.bbclass, mime.bbclass add RDEPEND during
populate_package, so need add mlprefix manually.

ERROR: QA Issue: lib32-fltk package lib32-fltk-dev - suspicious
values 'shared-mime-info-data-dev' in RRECOMMENDS [multilib]
ERROR: QA Issue: lib32-fltk package lib32-fltk-bin - suspicious
values 'shared-mime-info-data' in RDEPENDS [multilib]

(From OE-Core rev: 5a26b03c9f73822554ea969e42bca82217dfd3f8)

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-01 13:04:49 +01:00
Changqing Li
4e93a76c1e multilib.bbclass: fix do_package_qa_multilib error
lib32-packagegroup-anaconda-support have RDEPENDS to kernel-image,
but kernel-image don't have lib32, so skip it.

ERROR: QA Issue: lib32-packagegroup-anaconda-support package
lib32-packagegroup-anaconda-support - suspicious values 'kernel-image'
in RDEPENDS [multilib]

(From OE-Core rev: 24b8c61bf7dd13f7f371d3a910947a1fac062c6b)

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-01 13:04:49 +01:00
Max Kellermann
652e3ccd53 kernel-yocto.bbclass: fix "referenced before assignment" error
If "scc --configs" fails, do_kernel_configcheck() crashes like this:

      0338:    try:
      0339:        configs = subprocess.check_output(['scc', '--configs', '-o', s + '/.kernel-meta'], env=env).decode('utf-8')
      0340:    except subprocess.CalledProcessError:
  *** 0341:        bb.fatal( "Cannot gather config fragments for audit: %s" % configs)
      0342:
      0343:    try:
      0344:        subprocess.check_call(['kconf_check', '--report', '-o',
      0345:                '%s/%s/cfg' % (s, kmeta), d.getVar('B') + '/.config', s, configs], cwd=s, env=env)
 Exception: UnboundLocalError: local variable 'configs' referenced before assignment

This crash bug was introduced by commit
21de5cc43cfedc703e5bc0515507a6dae36afb74

(From OE-Core rev: 9602c03e8701c972f544001415efa92877622f57)

Signed-off-by: Max Kellermann <max.kellermann@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-27 23:41:41 +01:00
Chen Qi
2d83a6c579 nativesdk.bbclass: set consistent staging dirs regardless of multilib
For now, the RECIPE_SYSROOT of nativesdk recipes is ${WORKDIR}/recipe-sysroot
if multilib is disabled and ${WORKDIR}/nativesdk-recipe-sysroot if multilib
is enabled. And it's causing chaos. Problems I've met include:
1) 'File Exists' error when doing extend_recipe_sysroot
2) Rebuilding failure about cmake based nativesdk recipes if toggling multilib

In nativesdk.bbclass, We've set MULTILIBS to be "", and we've changed MLPREFIX
to be 'nativesdk-', I think we should also set consistent RECIPE_SYSROOT to be
${WORKDIR}/recipe-sysroot.

Below is an example showing why previous settings will cause do_prepare_recipe_sysroot
failure.

e.g.
A -> C
B -> C
A's RECIPE_SYSROOT is .../recipe-sysroot and B's RECIPE_SYSROOT is
.../nativesdk-recipe-sysroot.
As extend_recipe_sysroot function uses shared manifest, i.e., the same
manifest of C for both A and B, then there must be one of them having
the wrong manifest. And the wrong manifest results in RECIPE_SYSROOT
not cleaned up before installing new components, thus the following error.

  Exception: FileExistsError: [Errno 17] File exists: xxx -> xxx

This happens when toggling multilib and also between nativesdk recipes and
crosssdk, cross-canadian recipes. The latter situation also explains
why choosing ${WORKDIR}/recipe-sysroot instead of ${WORKDIR}/nativesdk-recipe-sysroot.
If we use 'nativesdk-recipe-sysroot', we still need to modify the extend_recipe_sysroot
function to treat crosssdk and cross-canadian as special cases. Using
'recipe-sysroot' does not have this problem.

(From OE-Core rev: 665934a506cc560bfbc469f5ed095e7d54e353a5)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-27 23:41:41 +01:00
Ross Burton
f0c59c8876 siteinfo: generalise siteinfo
Add a function siteinfo_data_for_machine to look up the data for any arch/os
pair, not just HOST_ARCH-HOST_OS.

(From OE-Core rev: 7429ed8d2a6e48cb3415113ef62348ae211aa189)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-25 23:15:49 +01:00
Ross Burton
feb5e7ed50 meson: stop Meson using target CFLAGS in native builds
With the goal of autoconf-compatibility Meson respects $CFLAGS et al in builds.
In cross-compiled build the cross file is the one true source of flags and the
environment isn't used, but in a native build the environment will still be
respected.

As this can lead to target flags being used in the build for native binaries
(including a single native binary inside a target recipe), export
CFLAGS=${BUILD_CFLAGS) et al.

(From OE-Core rev: 4ca0002860dca771836c0ce1c7a92b79a5f2db3f)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-25 23:15:49 +01:00
Ross Burton
16729efc18 meson: pass correct endian in the cross file
Meson doesn't care for the value of the endian field, but packages may want to
use it and Meson master now validates the value.

Use siteinfo to obtain the endianism and write the correct value.

(From OE-Core rev: 2f9adf05efdddf8dae9c58976ae56cf32d9e57f0)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-25 23:15:49 +01:00
Alex Kiernan
ce80e2572c kernel.bbclass: Avoid iterating over KERNEL_IMAGETYPES twice
During deploy we iterate over KERNEL_IMAGETYPES twice, with no consumer
of our changes in the interim. Collapse these two loops into one.

(From OE-Core rev: 8d3139640ed4cd4e196dc2d1f6ae02d612fb9564)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-22 15:52:12 -07:00
Alex Kiernan
b492b08a4d kernel: Fix modules tarball generation with usrmerge
When usrmerge is enabled the kernel modules aren't under /lib, but /usr/lib;
ensure we fetch the modules from the correct location so we don't have a
tarball which consists of a single symlink.

(From OE-Core rev: 62d2d7d12dda69d6019a1a2d5492450487e41bfd)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-22 15:52:12 -07:00
Alex Kiernan
f5603cac26 kernel-fitimage: Fix dangling symlink to image tree source
When deploying the its file, the target of the symlink is missing the its
extension, add it here.

(From OE-Core rev: df72761760d453b2dfc7cc2fa1344bb016df9712)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-22 15:52:12 -07:00
Alex Kiernan
4a49f5f617 Revert "kernel.bbclass: do not deploy fitImage; kernel-fitimage.bbclass does that."
This reverts commit 65cded55fa96b1ba72149a31689c081ac27af7f7.

The fitImage deployed here is the one generated in kernel-fitimage.bbclass,
whereas the one deployed by kernel-fitimage.bbclass is a raw image, so we
need the one deployed here keeping.

(From OE-Core rev: 798d2acc0141b13da7ddd18875a417b2df9d57b3)

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-22 15:52:12 -07:00
Reto Schneider
8e5a7658a0 kernel-uimage.bbclass: Dependency on u-boot-mkimage-native for all uImages
u-boot-mkimage-native is needed to build any type of uImages, be it
the uImage target itself or for example uImage.lzma. The dependency
however gets only added when at least one of KERNEL_IMAGETYPE and
KERNEL_ALT_IMAGETYPE is exactly uImage. Building uImage.bin and uImage.lzma is
not possible this way.

This patch adds a dependency to u-boot-mkimage-native for all imagetypes which
contains the string uImage.

(From OE-Core rev: ac4ce0b07b2742c64d44f988ecb27e7200b5a0bc)

Signed-off-by: Reto Schneider <code@reto-schneider.ch>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-22 15:52:12 -07:00
Nathan Rossi
4aa1f858ef siteinfo.bbclass: Add LatticeMico32 architecture site info
Add the basic architecture site info targets for the LatticeMico32
architecture. This architecture does not target Linux.

(From OE-Core rev: 216f8ea907c8ac7e580e33dcbd12c0f30608bf74)

Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-21 18:45:47 -07:00
Alexey Brodkin
bdbb6dad55 kernel-arch.bbclass: Add ARC to kernel arch map
(From OE-Core rev: d53b0b885fcf038d26d3f8aa8c5b46ec86649cea)

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-21 18:45:47 -07:00
Alexey Brodkin
28de528302 siteinfo: Define data for ARC
(From OE-Core rev: a66834f63e190377c3464ea180ccdc0262226049)

Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-21 18:45:47 -07:00
Ross Burton
f7fd5de4cf clutter: simplify SRC_URI
The Clutter class's ability to switch between tarballs and git isn't really
useful, so remove it.  If it comes back, it should use the devupstream class.

(From OE-Core rev: 6150ec737bad895b9fb62f711449a259887ebd1b)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-21 18:45:47 -07:00
Jens Rehsack
ef753fc7b3 distutils{, 3}.bbclass: improve error messages
For non-python developers it's difficult to identify where start
searching in case of errors. Fixing and marking the string to
grep for might help finding some root causes of issues slightly
quicker.

(From OE-Core rev: 44fff749ee7bfe47e88286aec1e8a22e63079ca7)

Signed-off-by: Jens Rehsack <sno@netbsd.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-21 18:45:46 -07:00
Douglas Royds
fdc0181062 reproducible: Consistent debug logging
Log both the source of the source_date_epoch and the resulting
source_date_epoch in all cases.

Also, now that we are determining the source_date_epoch successfully for both
yocto and non-yocto kernels, remove the inherits_class('kernel') exception.
We will log a failure to find a source_date_epoch for kernels as well.

(From OE-Core rev: 48fe0e83435f9fb53b1e5b37d5d9f1c2caccd22d)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-21 18:45:46 -07:00
Douglas Royds
d6f2ffb5c7 reproducible: Don't enforce existence of ${S} dir
Unnecessary.

(From OE-Core rev: 75d5a9d72566aca7a9f08bea21523b3c66bddeae)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-21 18:45:46 -07:00
Douglas Royds
73ba6051be reproducible: Find the git repo in WORKDIR/git or S first
Change the search regime for find_git_folder():

1. WORKDIR/git: This is the default git fetcher unpack path
2. ${S}
3. Go looking for .git/ under the WORKDIR as a last resort.

linux-yocto:
We had an existing (silent) defect. The linux-yocto recipes all specify
two git SRC_URIs, one for the kernel source itself, the other for the
kmeta data (config fragments and friends). find_git_folder() was finding
the git checkout for the kmeta data, but due to a typo in the git log -1
--pretty=%ct line, we were (silently) reading the source_date_epoch from
the ${S} directory = STAGING_KERNEL_DIR, which is empty. If your
build/ happened to be inside a git checkout, git would walk up the
directory tree, and silently read the commit timestamp from this other
git checkout. The correct path to read the git commit timestamp from is
the "gitpath", being that found by find_git_folder(), though this
function was incorrectly finding the kmeta data checkout, not the kernel
source tree.

Non-kernel git recipes:
The default git fetcher clones and checks out the sources at
WORKDIR/git/ regardless of the setting of S (unless subpath or
destsuffix is set). find_git_folder() now looks for the
WORKDIR/git/.git/ directory first.

Non-yocto linux kernels:
Kernel recipes that don't inherit kernel-yocto should always set
S = ${WORKDIR}/git, so that when base_do_unpack_append() in
kernel.bbclass moves the checkout down to the STAGING_KERNEL_DIR and
symlinks it as WORKDIR/git, the build can still work by following the
symlink. We were previously failing to follow the symlink in the
os.walk(), but we now look first for WORKDIR/git/.git/, and find it due
to the symlink.

If none of the above mechanisms work for finding the git checkout,
perhaps there was a subpath or destsuffix specified in the SRC_URI.
We go looking for the git checkout under the WORKDIR as a last resort.

(From OE-Core rev: b0ddb141d36853447f85ecaac07dbc9c5779627f)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-21 18:45:46 -07:00
Douglas Royds
526276a74d reproducible: Rename ambiguous "path" var to sourcedir or workdir as appropriate
(From OE-Core rev: 01358d6a89623a38e66969daa431d2eecb1ce8a2)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-21 18:45:46 -07:00
Douglas Royds
d872b0e51e reproducible: Refactor: get_source_date_epoch_from_youngest_file()
(From OE-Core rev: 2c8baf6c50e5a266a1e9ac939415bd481668e17e)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-21 18:45:46 -07:00
Douglas Royds
1bf7337bf8 reproducible: Tidy class description
Tidy whitespace, delete commented-out lines

(From OE-Core rev: 254eb5a2a12fd6b5cf7427818ea9fa886ea407a6)

Signed-off-by: Douglas Royds <douglas.royds@taitradio.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-21 18:45:46 -07:00
Khem Raj
244caa1dbf linuxloader.bbclass: Refactor to have seprate functions for musl/glibc loader
this makes it possible to use this for musl where we are trying to
create a glibc compat package

Add missing aarch64 loader definition for glibc function

(From OE-Core rev: 0a02ea79ec522582b46138a027eb166819e5647d)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-09-21 18:45:46 -07:00