Commit Graph

74 Commits

Author SHA1 Message Date
Richard Purdie
251ded90d4 archiver/package_rpm: Fix the worst src.rpm generation race
The package_rpm code is writing outside the task's sstate directory into
the sstate of do_deploy_archives. This is "out of spec" since if the
task is installed from sstate, the files are not restored. This means
the files may appear/disappear, things are not deterministic and there are
races.

Extend the do_package_write_rpm code to handle writing the src.rpm into
place to avoid these issues. There are other problems but this avoids races
around this file.

(From OE-Core rev: c6e151ba7fe0f14044537cf0ab2cac436f1496e3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-06 08:29:07 +00:00
Adrian Bunk
2cba73f2b1 archiver.bbclass: Fix COPYLEFT_LICENSE_{IN, EX}CLUDE
Setting them to empty before inheriting copyleft_filter
made them not getting their documented values there.

(From OE-Core rev: ee1f86ca7affc65265f219f0718f89406c9181a5)

Signed-off-by: Adrian Bunk <bunk@stusta.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-27 22:13:11 +00:00
Fabien Lahoudere
82da3463c8 archiver: Drop unwanted directories
In sources directory we can find patches/ and temp/.
The first one is filled with symbolic link unusable on another
machines.
The second contains yocto logs to create this archives and are
typically copied when 'S = "${WORKDIR}"'

(From OE-Core rev: 3904f98851c6a63dd9377e38f1432be6b1c0a94d)

Signed-off-by: Fabien Lahoudere <fabien.lahoudere@collabora.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-07 23:08:55 +00:00
Ross Burton
f997e13d03 archiver: remove left-over debugging
(From OE-Core rev: 6fe833611f985198e3ca038e5095f0d03249ce82)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-10 12:47:35 +01:00
Ross Burton
4771a7c2ed archiver: generalise task running when in configured mode
When in configured mode the archive will re-run configure itself, but that
doesn't work if there are other tasks that need to run such as cmake's
generate_toolchain_file.

Instead of hard-coding a list of classes and tasks, obtain the list of tasks
preceeding do_configure, filter out ones we don't want to re-run such as
do_patch and do_prepare_recipe_sysroot, and run those too.

(From OE-Core rev: fe26382f94d8cb7d2453d9937aee451b757252b8)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-01 13:04:50 +01:00
Ross Burton
e85f567f96 archiver: check tasks exist before adding dependencies
Some recipes don't have do_configure, or have them but are marked as noexec.
Check for their existance before hooking up dependencies.

[ YOCTO #12868 ]

(From OE-Core rev: 5c8aa1f41f04519a8ba3997c0d8e3a71fe270fae)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-01 13:04:50 +01:00
Tom Hochstein
a6567a4c21 archiver: Fix archive for linked kernel source
When archiving a kernel, if S is ${WORKDIR}/git, then
${WORKDIR}/git is in fact a soft link into work-shared,
and the archive contains just the soft link. Fix by
archiving the real directory.

(From OE-Core rev: 564204dd6b73697ef4b6c17243067892876cc323)

Signed-off-by: Tom Hochstein <tom.hochstein@nxp.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 14:49:07 +01:00
Tom Hochstein
c2eec907cb archiver: Fix build errors for kernels that don't use kernel-yocto.bbclass
Special handling for kernel source in work-shared was
incorrectly limited to kernel-yocto.bbclass instead
of kernel.bbclass.

(From OE-Core rev: b8c5680a67888d43540366a40bd0c501e926ac31)

Signed-off-by: Tom Hochstein <tom.hochstein@nxp.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-05 14:49:07 +01:00
Ross Burton
4239d9d2db archiver: remove deploy_all_archives task as --runall replaces it
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-16 18:05:40 +00:00
Ross Burton
ffa0eb5fdc archiver: don't require that images are built using RPM to build SRPMs
The requirement for images being built using RPM is too strict to have access to
the RPM functions, simply checking that package_rpm is in PACKAGE_CLASSES is
sufficient.

Also fail if SRPMs are requested but package_rpm isn't enabled, instead of
silently not doing what we were asked.

(From OE-Core rev: cdae5c41e4f67a8757efc5ce1ef11d730edc3af1)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-29 08:49:50 +00:00
Hongxu Jia
75f3ce2d47 archiver: preserve sysroot paths in configured mode for multilib
In the following commit, the archiver expanded RECIPE_SYSROOT
to preserve sysroot paths in configured mode:
...
commit aa2240657b015d46e9ba4bcb6264709a82313d83
Author: Ross Burton <ross.burton@intel.com>
Date:   Tue Jun 6 15:23:18 2017 +0100

    archiver: preserve sysroot paths in configured mode
...

In meta/conf/multilib.conf, it overrides the variables
of STAGING_DIR_HOST, STAGING_DIR_TARGET and RECIPE_SYSROOT
with "${WORKDIR}/${MLPREFIX}recipe-sysroot".

So the archiver should also expand STAGING_DIR_HOST and
STAGING_DIR_TARGET to preserve sysroot paths in configured mode for
multilib.

[YOCTO #11584]

(From OE-Core rev: b129af957a9fd8f603d1aebb9fca3adcb4e6b341)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-10-16 23:52:43 +01:00
Ross Burton
84b3d8b33c archiver: preserve sysroot paths in configured mode
do_ar_configured alters WORKDIR but also expects to be able to run do_configure,
so forcibly expand the paths to the sysroots as otherwise they'll point to a
non-existant directory in the temporary WORKDIR.

[ YOCTO #11584 ]

(From OE-Core rev: aa2240657b015d46e9ba4bcb6264709a82313d83)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-06 19:52:25 +01:00
Dengke Du
de1b5983c7 archiver.bbclass: fix do_ar_original error for matchbox-desktop
Error:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: matchbox-desktop-2.1-r0 do_ar_original: Can not determine archive names
for original source because 'name' URL parameter is unset in more than one URL.
Add it to at least one of these: git://git.yoctoproject.org/matchbox-desktop-2
file://vfolders/%2A

ERROR: matchbox-desktop-2.1-r0 do_ar_original: Function failed: do_ar_original
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In function do_ar_original, when recipes have more than one source, it added the
"name" URL parameter as suffix to identify the created tarball.

But the URL type "file://" that we always used to represent a series of patches,
it didn't have "name" parameter, so it failed.

So set "name" to the folder name to identify the created tarball, for example:

In matchbox-desktop bb file, the SRC_URI contains:

	file://vfloders/*

We set "name" to "vfolders" to identify the created tarball.

In connman-gnome bb file, the SRC_URI contains:

	file://images/*

We set "name" to "images" to identify the created tarball.

(From OE-Core rev: 0af636c635391b30c987dedeffe597ef4f8a1ed8)

Signed-off-by: Dengke Du <dengke.du@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-06 19:52:25 +01:00
Mikko Ylinen
069b125914 archiver.bbclass: adapt do_unpack_and_patch to RSS
do_unpack_and_patch was not correctly run until recently
("archiver.bbclass: various fixes for original+diff mode") but
with the fix applied, the errors we get indicate the function
is not adapted to work with recipe specific sysroots.

do_unpack_and_patch sets WORKDIR to ARCHIVER_WORKDIR which
affects all path settings relative to WORKDIR, inluding the paths
to recipes' sysroots. IOW, when do_unpack and do_patch are run, they
cannot find the necessary native tools and files located in the
sysroot (e.g., quiltrc) because the paths point to ARCHIVER_WORKDIR.

Adapt do_unpack_and_patch to RSS by restoring the original
STAGING_DIR_NATIVE after WORKDIR is changed to ARCHIVER_WORKDIR.

(From OE-Core rev: db7d2cc5a4df3c2077ba874c7ae395c73fd9ed13)

Signed-off-by: Mikko Ylinen <mikko.ylinen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-06 19:52:25 +01:00
Joshua Watt
1a4494b2e6 archiver: Escape recipe name in regex
The recipe name needs to be escaped when using it in a regular expression so
that and special characters are treated literally

(From OE-Core rev: 53c8cceb744adda1bf899d62071d11b20a5dea98)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-03 23:46:05 +01:00
Zhenbo Gao
4ef31feab1 archiver: avoid archiving source for glibc-locale
Avoid archiving source for glibc-locale as its tasks
do_fetch do_unpack and do_patch have already been deleted.

(From OE-Core rev: 50ed224ebc8d88a900febdc78013fa0c791d71cf)

Signed-off-by: Zhenbo Gao <zhenbo.gao@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-18 14:01:46 +01:00
Patrick Ohly
23ecae6d67 archiver.bbclass: do not cause kernel rebuilds
Adding or removing archiver.bbclass from a build configuration causes
rebuilds of linux-yocto-based kernels because of the
do_kernel_configme->do_unpack_and_patch task dependency.

This particular dependency can be ignored for the do_kernel_configme
sstate signature calculcation. Idea for the fix from Richard Purdie.

Note that building the kernel and adding archiver.bbclass later to
archive sources leads to do_unpack_and_patch running after
do_kernel_configme (because that already ran in the first build),
which might be problematic. This is independent of the change here.

The use case in YOCTO #11441 is to removed archiver.bbclass between a
production build with archiving enabled and builds via oe-selftests
without archiving. That direction is fine.

Fixes: YOCTO #11441

(From OE-Core rev: fed0ed82928e6a7846fbad233ac657bd17bcefc7)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-18 14:01:46 +01:00
Patrick Ohly
3208824ae6 archiver.bbclass: various fixes for original+diff mode
The diff.gz gets created in do_unpack_and_patch, but
do_deploy_archives did not depend on it, so there was a race
condition. For example, "bitbake linux-intel:do_deploy_archives"
without a prior "bitbake linux-intel:do_kernel_configme" did not
deploy the diff.gz.

When do_unpack_and_patch ran first, it failed because the output
directory didn't exist yet and the error was not detected because the
result of the diff command wasn't checked.

Changing the current working directory in create_diff_gz() without
returning to the original directory caused warnings like this:
   WARNING: linux-intel-... do_unpack_and_patch: Task do_unpack_and_patch changed cwd to .../tmp-glibc/work-shared/intel-corei7-64

(From OE-Core rev: 18aac553ca35049c80b6cc82ff0e69ce8a7a03a9)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-18 14:01:46 +01:00
Patrick Ohly
c2221c4ca1 archiver.bbclass: enhance do_ar_recipe task signature
Previously, do_ar_recipe ran again unnecessarily when adding or
removing classes like buildhistory.bbclass, because that changes the
BBINCLUDED variable which do_ar_recipe uses to find .bbappend files.

This is both extra work and also sometimes triggered "basehash
changed" errors (seen under oe-selftest, which adds machine.inc and
bblayers.inc) because BBINCLUDED is special and does not cause
the basehash to be recalculated.

The file *content* already was not considered in the task signature,
instead relying indirectly on PF (which includes the revision assigned
by a PR server) to ensure that a new versioned source archive gets
created each time there is a rebuild.

Therefore it makes sense to use the same mechanism and also ignore the
file *list*, i.e. exclude BBINCLUDED from the task signature.

(From OE-Core rev: 9666f0e0b02efc14226c77497fd38f79fc372f98)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-18 14:01:46 +01:00
Joshua Lock
cc2744267f meta: replace uses of bb.data.expand(VARNAME, d) with d.expand(VARNAME)
bb.data.expand(x, d) is deprecated API.

[YOCTO #10678]

(From OE-Core rev: a361babe443da635aed83be46679067457fd6a58)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22 11:35:19 +00:00
Joshua Lock
3c59b1bf93 meta: remove True option to getVarFlag calls
getVarFlag() now defaults to expanding by default, thus remove the
True option from getVarFlag() calls with a regex search and
replace.

Search made with the following regex:
getVarFlag ?\(( ?[^,()]*, ?[^,()]*), True\)

(From OE-Core rev: 2dea9e490a98377010b3d4118d054814c317a735)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Joshua Lock
c4e2c59088 meta: remove True option to getVar calls
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.

Search made with the following regex: getVar ?\(( ?[^,()]*), True\)

(From OE-Core rev: 7c552996597faaee2fbee185b250c0ee30ea3b5f)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Ross Burton
113828ff86 archiver: don't change directory when generating tarball
There's no need to chdir() as tarfile.add() can be told what name to use in the
archive.

(From OE-Core rev: d0b282bce34db44dde4dd7f53a64dfaafe6789de)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:20 +00:00
Saul Wold
45f7221915 archiver: fix gcc-source handling
The source archiver was not handling the gcc-source target correctly, since it uses the
work-shared directory, we don't want to unpack and patch it twice, just as the comments
say, but the code was not there to check for the gcc-source target.

[YOCTO #10265]

(From OE-Core rev: bbac0699ceadb7a25a60643fb23dffce8b4d23d0)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-11 08:31:41 +01:00
Patrick Ohly
b5a00339d9 archive.bbclass: fix do_ar_original archiving of multiple source repos
When a recipe uses more than one source which isn't a plain file (for
example, multiple git repos), then do_ar_original created the source
archives using the same filename and thus only archived one source.

The "name" parameter is used as file suffix to create unique names for
each source, leading to archives following this pattern:
deploy/${TARGET_SYS}/${PF}/${PF}[-<name>].tar.gz.

The ${PF} part is a bit redundant, which may or may not be
desirable. The patch is more localized this way (no need to modify
create_tarball()).

For example, meta-oic's iotivity_1.1.1.bb uses:

   url_iotivity = "git://github.com/iotivity/iotivity.git"
   branch_iotivity = "1.1-rel"
   SRC_URI = "${url_iotivity};destsuffix=${S};branch=${branch_iotivity};protocol=http;"

   url_tinycbor = "git://github.com/01org/tinycbor.git"
   SRC_URI += "${url_tinycbor};name=tinycbor;destsuffix=${S}/extlibs/tinycbor/tinycbor;protocol=http"

   url_hippomocks = "git://github.com/dascandy/hippomocks.git"
   SRC_URI += "${url_hippomocks};name=hippomocks;destsuffix=${S}/extlibs/hippomocks-master;protocol=http"
   SRC_URI += "file://hippomocks_mips_patch"

   url_gtest = "http://pkgs.fedoraproject.org/repo/pkgs/gtest/gtest-1.7.0.zip/2d6ec8ccdf5c46b05ba54a9fd1d130d7/gtest-1.7.0.zip"
   SRC_URI += "${url_gtest};name=gtest;subdir=${BP}/extlibs/gtest"

   url_sqlite = "http://www.sqlite.org/2015/sqlite-amalgamation-3081101.zip"
   SRC_URI += "${url_sqlite};name=sqlite3;subdir=${BP}/extlibs/sqlite3;unpack=false"

These now get archived in deploy/sources/*/iotivity-1.1.1-r2/ as:

gtest-1.7.0.zip                      iotivity-1.1.1-r2-recipe.tar.gz    sqlite-amalgamation-3081101.zip
hippomocks_mips_patch                iotivity-1.1.1-r2.tar.gz
iotivity-1.1.1-r2-hippomocks.tar.gz  iotivity-1.1.1-r2-tinycbor.tar.gz

(From OE-Core rev: 5c63ffc706c0fff8cfb797a238f4f0e73ee2813d)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-28 10:16:03 +01:00
Patrick Ohly
58866d6e9a archiver.bbclass: ignore unpack sub-directories in do_ar_original
Support for absolute paths in the "subdir" parameter was recently
added (bitbake rev: c3873346c6fa). The git fetcher has supported
absolute paths in "destsuffix" already before.

When the path is absolute as in destsuffix=${S}/foobar, the tmpdir
used by do_ar_original gets ignored, which breaks:
- source code archiving (tmpdir is empty)
- compilation due to race conditions (for example, ${S} getting
  modified by do_ar_original while do_compile runs)

To solve this, these parameters get removed from URLs before
instantiating the fetcher for them.

This is done unconditionally also for relative paths, because these
paths are not useful when archiving the original source (upstream
source does not have them, they only get used by the recipe during
compilation).

(From OE-Core rev: c27c464e267db3f4b08cbd966412d19b0e756d28)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-28 10:16:03 +01:00
Richard Purdie
5c8124df2e archiver: Improve debug output
Make the output from the task more informative about which directories
are being referenced.

(From OE-Core rev: 0e306a53c460302ec20192fc35930983781b7a2e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-13 20:58:41 +01:00
Richard Purdie
380004b855 archiver: Ensure sstate-inputdir directory is created
Without this, do_archive_sources will fail for the gcc tasks where there is
common source and the other tasks are not added as dependencies. The failure
happens when trying to restore these tasks from sstate.

To fix this ensure the sources directory is created as expected by the
sstate code.

[YOCTO #9433]

(From OE-Core rev: b3afe7d9bd25a943e947de3ec064ea245173c5a8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-12 22:50:21 +01:00
Richard Purdie
5602f64ab8 archiver: Fix ASSUME_PROVIDED issues
Currently the dependencies injected by this class for recipes in ASSUME_PROVIDED
e.g. tar-native or chrpath-native are ignored. We need to use their "replacement"
names, e.g. tar-replacement-native.

This avoids broken archives and sstate failures with these recipes.

(From OE-Core rev: 2279c6d20c2a33283ce9a8d1ef91a8acdad0a20e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-11 09:03:58 +01:00
Robert Yang
71db079bc5 archiver.bbclass: addtask do_deploy_archives_setscene
There was a do_deploy_archives_setscene, but no addtask.

(From OE-Core rev: 81db27c208468dc460e9d8f0063ad24b2b37dc97)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-30 21:24:02 +01:00
Mariano Lopez
9ee403b310 archiver.bbclass: Just archive gcc-source for all gcc recipes
It is pointless to archive several times the gcc source.
This change will archive gcc source once, for the gcc-source-{PV}
recipe.

[YOCTO #9327]

(From OE-Core rev: 2f53fb2ae561e2eb599682ba95ce83ad1b4b7ada)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-28 15:55:50 +01:00
Mariano Lopez
d7cf2c3230 archiver.bbclass: Fix tar name for git repositories
When archiving the original source, the git repositories have the name as
they are in the $DL_DIR plus the source revision; i.e.
"git.yoctoproject.org.linux-yocto-4.4.git.89419d8b90_dadb436904.tar.gz".
This change set the tar name to $PF.tar.gz instead, to have consistency with
the others archives created by the class.

(From OE-Core rev: 3f903cb767150e316337929d72559cad6931039a)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-25 10:29:17 +00:00
Mariano Lopez
2cb4cb7a6c archiver.bbclass: Fix gcc-source corner case
Bitbake couldn't add the task ar_configured when
trying to archive the configured source for
gcc-source-${PV} recipes. This is because the task
depended in the do_configure and this task doesn't
exist for gcc-source.

This fix allows to archive configured gcc-source recipe.

(From OE-Core rev: 5be5eae2e1630a151a9f5866d60325d439150ced)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-25 10:29:17 +00:00
Mariano Lopez
c29eea0df8 archiver.bbclass: Fix use of ARCHIVER_WORKDIR and ARCHIVER_OUTDIR
Currently do_unpack_and_patch() and do_ar_configured() are using
the ARCHIVER_OUTDIR as the ARCHIVER_WORKDIR, this lead to have
duplicated files inside the tars when using the archiver class
for patched and configured source.

(From OE-Core rev: 09dfc99a813090043262529beb6d900900e983dc)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-25 10:29:17 +00:00
Mariano Lopez
8b7ee6ebeb archiver.bbclass: Don't expand python functions in dumpdata
Currently the dumpdata task expands python data in the datastore, in
some functions this causes a silent error and the task will fail.

The change also rewrite the function to make a bit clearer.

(From OE-Core rev: e3239ade430ff48e00dce47066abe8fcb990af61)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-25 10:29:17 +00:00
Maxin B. John
a205c4ce1d bbclass: fix spelling mistakes
Fix some spelling mistakes in bbclass files

(From OE-Core rev: ed484c06f436eea62c5d0b1a2964f219f3e5cb61)

Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-24 15:50:27 +00:00
Jian Liu
4415dc5cdd archiver.bbclass: add bbappend when do_ar_recipe kernel and gcc packages
bbappend files are not included in the archiver during do_ar_recipe.
Find and put them into the tarball.

(From OE-Core rev: 5c84057de5b31c5d6d9abfcca3078bf766a21d88)

Signed-off-by: Jian Liu <jian.liu@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-24 15:50:26 +00:00
Alejandro Hernandez
2f0ff3abb4 archiver.bbclass: fix previous issue regarding work-shared for linux-yocto
A previous patch intended to fix and improve the archiver class, and while it
did for gcc packages, with it some issues ended up being hidden, the kernel
tasks taken from kernel.bbclass and kernel-yocto.bbclass specifically expect the
kernel to use work-shared, which either ended up causing issues or wasting time
doing unnecessary work, this patch fixes these issues by performing the right
tasks within the archiver in the right order

[YOCTO #8378]

(From OE-Core rev: d643e43622eb3e43fbb2e21fa33580e2fcdf42be)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-24 15:50:26 +00:00
Alejandro Hernandez
92b26ad938 archiver.bbclass: Fixes and improves archiver class for kernel and gcc packages
gcc packages use a shared source directory, this causes an issue since the archiver will
try to patch the same source several times (one for each gcc package), producing an error,
the archiver class used stamp-base to check this, nonetheless our gcc packages no longer
use stamp-base, they use gcc-shared instead, which is what broke this functionality.
This patch adds a check to see whether or not the source should be patched,
avoiding patching the source when it shouldn't.

Also, we dont need to create multiple identical tarballs for all gcc packages,
this patch fixes this and creates a single source tarball for gcc.

When requesting patched sources, a race condition is created for linux-yocto tasks,
unpack_and_patch is executed along with kernel_configme, which most of the time
causes errors during configure, since kernel_configme task is specific to the kernel,
simply modifying the tasks order by creating a dependency to kernel_configme was impossible,
causing errors on all other packages that didnt use kernel_configme, this is fixed by
creating a special case for the kernel, adding tasks with correct dependencies,
avoiding the race condition and behaving the way it should for all other packages as well.

[YOCTO #8378]

(From OE-Core rev: aecaa0e8739db1c228a6db78225a717d9f348a5b)

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-21 22:59:27 +01:00
Richard Purdie
99fc78653a archiver: stamp-base is dead, remove it
stamp-base was only ever used by the shared workdir code in gcc. This
turned out to be problematic and has been replaced by other approaches
which don't need specialist bitbake knowledge.

stamp-base will likely get removed from bitbake but for now, remove it
from archiver to simplfy the code since gcc no longer uses it.

This stops people getting confused by the obsolete code paths which I'm
getting a lot of questions about.

(From OE-Core rev: 4b9f70f03cbefe9db758500e3ad89f7a8ff8226e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-24 17:54:31 +01:00
Clemens Lang
76d7e8e9c9 archiver.bbclass: Run deploy_archives in $WORKDIR
In recipes that are exempt from source code archiving due to
COPYLEFT_LICENSE_EXCLUDE, do_deploy_archives does not have a transitive
dependency on do_unpack. Given enough parallelism, this means
do_deploy_archives can run at the same time or before do_unpack.

Because do_deploy_archives did not specify a working directory, its
working directory was ${B}, which defaults to ${S}, which may be set by
a recipe to a directory that is created by do_unpack.

In this case, do_deploy_archives can fail because do_unpack deletes and
re-creates the directory and do_deploy_archives cannot change into the
non-existent directory. Avoid this problem by explicitly specifying
a working directory for do_deploy_archives (and for
do_deploy_all_archives as well for good measure).

(From OE-Core rev: e22685ff11af6d54c939aa8f327a0aaa1557fbbc)

Signed-off-by: Clemens Lang <clemens.lang@bmw-carit.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19 18:05:36 +01:00
Patrick Ohly
95083b2699 archiver.bbclass: add do_deploy_all_archives
When working specifically on source archiving, it is useful
to trigger that for all recipes required by something like an
image recipe, without actually having to build that.

"bitbake -c deploy_all_archives <target>" does that now.

(From OE-Core rev: 30b109729a81097cc6cfa65148e0e9ae1a564d2c)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:55:41 +01:00
Mariano Lopez
ae9102bda3 copyleft_filter.bbclass: Allow to filter on name
The archiver uses a license based filter to provide the source code.
This patch allows to search on name based on two new variables (COPYLEFT_PN_INCLUDE,
COPYLEFT_PN_EXCLUDE). Both variables are empty by default.
The filter by name has higher priority than the license filter.

[YOCTO # 6929]

(From OE-Core rev: 04066239e9cd6a8461fb2c18e826289469ac1240)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23 11:46:57 +01:00
Joe Slater
1a0485040c archiver: fix truncation of src_rev
In trying to eliminate AUTOINC+ from revision strings,
we accidently truncated the strings to almost guarantee
information from SRCREV_FORMAT, when supplied, would be
lost.  So, we now only delete any AUTOINC+'s from the string.

(From OE-Core rev: 83c6930cdfac5a7759c4786172a5c70226e6adc4)

Signed-off-by: Joe Slater <jslater@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-10-24 17:36:15 +01:00
Joe Slater
a9b080405e archiver.bbclass: add revision to git tarfile name
When archiving original source, insert the first 10
characters of srcrev, after stripping off any
AUTOINC+, into the filename for tarfiles created
from directories.

(From OE-Core rev: 36726049a6a452f85d7cc849d57ad6af05d6d3ea)

Signed-off-by: Joe Slater <jslater@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-27 12:12:30 +01:00
Jian Liu
181bcbe2c7 archiver: delete the tail slash in directory name
If directory names in SRC_URI ended with "/", function
do_ar_original() in layers/oe-core/meta/classes/archiver.bbclass
will generate a tar file whose name is ".tar.gz".
So delete the "/" at the tail of the directory names before use the names.

(From OE-Core rev: a539e823d002fefe129e3045f893d1237fadb87f)

Signed-off-by: Jian Liu <jian.liu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-23 09:31:41 +01:00
Hongxu Jia
f2a104d777 archiver: create dir 'B' explicitly while WORKDIR changed
In the do_unpack_and_patch task of archiver, the changed 'WORKDIR' also
casued 'B' changed, create dir 'B' for the possibly requiring of the
tasks which executed in do_unpack_and_patch task.

Such as cut-ecgl's do_patch required 'B' existed:
...
| cp: target 'tmp/work/x86_64-wrs-linux/cut-ecgl/1.0-r0/archiver-work
/cut-ecgl-1.0' is not a directory
...
| ERROR: Function failed: do_patch (log file is located at tmp/work/
x86_64-wrs-linux/cut-ecgl/1.0-r0/temp/do_unpack_and_patch/
log.do_unpack_and_patch.11886)
...

The 'B' dir is implicitly created at the task executing time while the
task tag 'dirs' is not assigned. In the above cut-ecgl example, the 'B'
was created at the wrl_buildlink executing time which is the prefuncs of
task do_unpack. So the following do_patch could use 'B' correctly. But
wrl_buildlink is unnecessary and not invoked in the do_unpack_and_patch
task.

(From OE-Core rev: 8baefb49d0bf9d3dd757d9b2359e0a9f4f33dd60)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-06-17 08:59:04 +01:00
Robert Yang
21169c3042 archiver.bbclass: emit patch series files for original src
Emit patch series files for original src so that the user knows how to
apply the patch orderly.

[YOCTO #5113]

(From OE-Core rev: 053631bc3cf7c7c2d090decaa3b5e5690963e64a)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-25 09:55:35 +00:00
Robert Yang
4a91e1449b archiver.bbclass: make it can filter the license
* Filter the license (default: no), the recipe whose license in
  COPYLEFT_LICENSE_INCLUDE will be included, and in
  COPYLEFT_LICENSE_EXCLUDE will be excluded.

* The user can set the recipe type that would be archived (native,
  target, and so on), deafult to all.

The copyleft_filter.bbclass is come from copyleft_compliance.bbclass,
which is used by both copyleft_compliance.bbclass and archiver.bbclass.

[YOCTO #5740]

(From OE-Core rev: 0e798d5cbcf585535e19633828dc540a282261fc)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-25 09:55:35 +00:00
Robert Yang
1f44904dae archiver.bbclass: move a few code to copyleft_compliance.bbclass
Move the code which is only used by copyleft_compliance.bbclass from
archiver.bbclassc, and remove the "inherit archiver" from
copyleft_compliance.bbclass.

The archiver.bbclass is used for archiving various types of sources, but
the copyleft_compliance.bbclass is used for analysing the license, they
don't have much relationships.

[YOCTO #4986]
[YOCTO #5113]

(From OE-Core rev: 578830fe2ff279ea620916ea711b80dc1b29a275)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-21 17:15:27 +00:00