The current code had broken exception handling due to the use of a
"traceback" variable as well as an import. Use the new library code
for this instead which reduces code duplication and has fixed/improved
exception handling.
The chdir code can be dropped since any directory changes are in other
processes now so there is no need for it here and the code no longer
changes directory.
(From OE-Core rev: bcd47389f4b1fc69d2bb4da01933bfa1fdcae092)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We can end up with / in dependency names from file dependencies but the
deb format doesn't allow this. Filter the names to allow such dependencies
to work. Names have to start with an alphanumeric digit so also handle this.
This allows for future handling of "per file" dependencies similarly to
the rpm backend, bring parity to the functionality of the backends.
(From OE-Core rev: fc08972688d784f561c8be88d3100d6baaf22070)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current implementation does not handle possible exceptions coming from child
processes, the latter responsible for creating packages. With the aim to have more
control, use pipes to communicate exceptions and stop package creation in case
of failure.
Helps to debug [YOCTO #12012].
(From OE-Core rev: 11350a67ba137f560d04aa643ff500a7ff112c73)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Redirecting stderr to stdout helps debugging issues, i.e instead of just
getting the return code, get also the error log from the pkg manger
This commit is in the way to figure out the root cause of [YOCTO #12012],
where dpkg-deb fails with a 2 return code and according to the man page,
there are multiple issues leading to the same code.
(From OE-Core rev: 9ff023fb26f5f0ce19e757beda00ccc32c009b21)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Perl dependencies may look as "Perl(Foo::Bar)", but dpkg does not
support the non-alphanumeric characters. There was already special
handling present for turning '(' and ')' into '__'. This change does
the same for ':'.
(From OE-Core rev: a34e397095a9c2f8d0af1168ceab295af659242d)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These comments/variables appear to be long dead, remove them.
(From OE-Core rev: a50c8fa7c6c9d40279724a04fb616462b1b491ff)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow the creation of debs to happen in parallel, making best use of resources
on multiprocessor systems.
(From OE-Core rev: dd540fba6c65fb74df014f5d9d2965078314a790)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This uses more modern formatting to handle the lockfiles and control
file cleanup with try/finally, taking advantage of the previous
extra indentation.
(From OE-Core rev: 9cd7c2631b0840a57b9ed6c201bcb4fc80094f71)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This prepares the way to parallelise deb generation and splits the iteration
over packages and the package generation into separate functions. Whitespace
indentation is unchanged deliberately and is fixed in a followup patch. There
should be no functional change.
Some checks on variables are removed as they were pointless when you looked
at the code.
(From OE-Core rev: 5054f66f8fbaaa422f74a4b5d0e61e68de6ffe91)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Avoids parser errors if PACKAGE_ADD_METADATA_DEB is set to an
empty value.
(From OE-Core rev: f0959c0908dfb386d29f13fcd3e57b2b004c6c14)
Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In deb control files, each line of a long description starts with
a single space. Empty lines are represented by a single space
followed by a single full stop character.
(From OE-Core rev: f66278f471c0bf9421ce2c55a56a144a0f9332bf)
Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Unlike rpm, debian packaging does not allow file paths in 'Provides:' field.
When 'usrmerge' distro feature enabled bash/busybox packages adds '/bin/sh' to
it's RPROVIDES to satisfy build dependencies, this entry should be filtered out.
(From OE-Core rev: a36165ea8bf8c7a061a0173076c83ae6806d65b6)
Signed-off-by: Amarnath Valluri <amarnath.valluri@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that the datastore works dynamically we don't need the update_data calls
so we can just remove them. They're not actually done anything at all for
a while.
(From OE-Core rev: 8de0c5d3bd01919e2bf0394f9c485936d6098cec)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It was discovered that buildtools-tarball can't be built with Debian packaging:
E: Unable to locate package nativesdk-buildtools-perl-dummy
The package exists but dpkg doesn't see it because the Architecture field isn't
valid: for this package the Architecture field is 'allarch' which isn't in the
list of valid architectures we define.
package_deb already has a mapping of OE architectures (PACKAGE_ARCH) to dpkg
architectures (DPKG_ARCH), for example our x86_64 is amd64 in dpkg. However
allarch.bbclass sets an invalid TARGET_ARCH of 'allarch' (to break anything
which attempts to use the target architecture) and package_deb relied on the
PACKAGE_ARCH check later turning the Architecture field into 'all'.
This usually works, but nativesdk-buildtools-perl-dummy inherits allarch but
then changes PACKAGE_ARCH to buildtools-dummy-nativesdk to isolate the packages
in a separate feed. We can handle this by explictly checking for allarch in the
TARGET_ARCH check, as that everything that inherits allarch will want to have
Architecture: all in their packages.
[ YOCTO #10700 ]
(From OE-Core rev: 58a5771f20f7851816b5f39df0ada8f16438065c)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The exception handling in this function seemed mildly crazy. Python will
given perfectly good or in several cases better information if we let its
standard traceback/exception handling happen. Remove the pointless code
along with the duplicated key checking which was broken in the inner loop
by usage of the wrong variable.
(From OE-Core rev: f755b07b528e828618141eda402399d791efba4a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently if the dpkg-deb command fails you see an error message like this:
ERROR: nss-3.27.1-r0 do_package_write_deb: dpkg-deb execution failed
ERROR: nss-3.27.1-r0 do_package_write_deb: Function failed: do_package_deb
which is pretty much useless. If we use subprocess.check_output, we see a
traceback and then:
Exception: subprocess.CalledProcessError: Command '<cmd>' returned non-zero exit status 1
Subprocess output:
<output>
which is much easier to debug from.
(From OE-Core rev: 18160442869f56ee71538bc2dc60d7cb6c08c8a2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a new variable to allow markup of postinstall (and preinst)
script dependnecies on native/cross tools.
If your postinstall can execute at rootfs creation time rather than on
target but depends on a native tool in order to execute, you need to
list that tool in PACKAGE_WRITE_DEPENDS.
(From OE-Core rev: aff8ca95b8303a4a2a5600c0d8ec0a50ad677258)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.
Search made with the following regex: getVar ?\(( ?[^,()]*), True\)
(From OE-Core rev: 7c552996597faaee2fbee185b250c0ee30ea3b5f)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This sets a good example and avoids unnecessarily contributing to
perceived complexity and cargo culting.
Motivating quote below:
< kergoth> the *original* intent was for the function/task to error via
whatever appropriate means, bb.fatal, whatever, and
funcfailed was what you'd catch if you were calling
exec_func/exec_task. that is, it's what those functions
raise, not what metadata functions should be raising
< kergoth> it didn't end up being used that way
< kergoth> but there's really never a reason to raise it yourself
FuncFailed.__init__ takes a 'name' argument rather than a 'msg'
argument, which also shows that the original purpose got lost.
(From OE-Core rev: 5a074e8a26d27ea9c4f31e2b75b2b14f6e0641d3)
Signed-off-by: Ulf Magnusson <ulfalizer@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The dict.fromkeys() creates a dict without order, there might be a
problem when build the same recipe again, for example:
- First build of make:
Provides: es-translation, make-locale
- Second build of acl:
Provides: make-locale, es-translation
They are exactly the same Provides, but tools like "diff" doesn't think
so. Sort RPROVIDES will fix the problem.
(From OE-Core rev: 3506172d7d9f8d92362b6ebb75582b7c3e662dae)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Don't modify an OrderedDict while walking its keys.
(From OE-Core rev: eb7f08c4c01313afc8350200eeb63daefde8a6f6)
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
dpkg-build needs to be executed in the root of the package, so save and restore
the current directory so this task doesn't modify the state.
(From OE-Core rev: c294f4ed5a02b055916cfc26a2fca672edee1208)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In python3, strings are unicode by default. We need to encode/decode
from command pipelines and other places where we interface with the
real world using the correct locales. This patch updates various
call sites to use the correct encoding/decodings.
(From OE-Core rev: bb4685af1bffe17b3aa92a6d21398f38a44ea874)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The syntax for octal values changed in python3, adapt to it.
(From OE-Core rev: 737a095fcde773a36e0fee1f27b74aaa88062386)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a package depends on itself apt-get fails to install it with the
error attached to this patch.
This patch checks for this conditions and notifies the user so the
recipe maintainer can fix his RDEPENDS variable.
root@qt5022:~# apt-get install perl-module-cpan
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
perl-module-cpan
0 upgraded, 640 newly installed, 0 to remove and 0 not upgraded.
Need to get 0 B/5964 kB of archives.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n]
E: Internal error: MaxLoopCount reached in SmartUnPack (2) for
perl-module-cpan:amd64, aborting
E: Internal error, packages left unconfigured. perl-module-cpan:amd64
root@qt5022:~# apt-get install perl-modules
(From OE-Core rev: d8bf148c3b740254ed23d52cf244f0f6e9c7a4ab)
Signed-off-by: Ricardo Ribalda Delgado <ricardo.ribalda@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The dictionary cannot be used for iteration if its size may change
during the loop.
ERROR: Error executing a python function in
/home/ricardo/curro/qt5022/build-qt5022/repo/yocto/meta/recipes-devtools/perl/perl_5.22.0.bb:
Exception: RuntimeError: dictionary changed size during iteration
Acked-by: Aníbal Limón <anibal.limon@linux.intel.com>
(From OE-Core rev: 04aa40be126e00e2eb97311510e4d3ea90bfe7eb)
Signed-off-by: Ricardo Ribalda Delgado <ricardo.ribalda@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The use of TUNE_FEATURES was making do_package_write_deb of allarch
packages target specific.
To avoid this, only use the end value of DPKG_ARCH for its checksum,
not intermediate values or variables.
(From OE-Core rev: 65955cf1c7d5c59f29d769a8244ae7c156a43f38)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This tells APT that it can use such packages to resolve dependencies
from packages of any architecture in a multilib build.
(From OE-Core rev: 7158c79a70e3d820c9701dacfa7206d13f95845a)
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Have DPKG_ARCH set by directly invoking a mapping function, rather
than using an anonymous Python function modify the variable under
the hood, so we can have proper handling of overrides.
Also bring in some additional mappings to Debian architecture names
that weren't being handled.
(From OE-Core rev: 8d042ea4e755cb0bb28b88333e10e04ec4e86a36)
Signed-off-by: Matt Madison <matt@madison.systems>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix some spelling mistakes in bbclass files
(From OE-Core rev: ed484c06f436eea62c5d0b1a2964f219f3e5cb61)
Signed-off-by: Maxin B. John <maxin.john@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Changes to OVERRIDES (e.g. from changing MACHINE) should not change
the sstate signatures of do_package_write_*. Exclude the variable
explicitly in the package classes to avoid this.
(From OE-Core rev: 5826a9260138c437f87ba1a9f84d5c08442b997d)
(From OE-Core rev: 59ccabdfb59a3e0917585dafab7a1e7c71330996)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow UTF-8 characters on control files. Also handle an expection
in case of invalid characters (non UTF-8).
[YOCTO #6693]
(From OE-Core rev: 4096f3c5d309161999adc996fdfa7526e5504366)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In some scenarios rpm needs version information from RPROVIDES. We can
add this to the metadata where needed however we need to stop it entering
the ipk/deb packages. This data is not needed due to the way opkg/dpkg
handle the data.
This patch ensures that data isn't used.
(From OE-Core rev: c68b975693c10899ce50b4d8c2aa3985ca890ce3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than just use d.getVar(X), use the more explict d.getVar(X, False)
since at some point in the future, having the default of expansion would
be nice. This is the first step towards that.
This patch was mostly made using the command:
sed -e 's:\(getVar([^,()]*\)\s*):\1, False):g' -i `grep -ril getVar *`
(From OE-Core rev: ab7c1d239b122c8e549e8112c88fd46c9e2b061b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow globs in CONFFILES.
This patch changes the way of CONFFILES handling. After this change,
the CONFFILES can take the same form as FILES. That means, we don't
have to list a bunch of files for CONFFILES. It will just be expanded
like the FILES variable.
We don't assume default value for CONFFILES in OE. But distro vendors could
provide a default value for CONFFILES in their distro configuration file
like below.
CONFFILES = "${sysconfdir}"
In this way, files under /etc are treated as configuration files by
default. Of course, setting CONFFILES in recipes take precedence over
the CONFFILES. For example, if the recipe author decides that package A
should only treat files under ${sysconfdir}/default/ as config files,
he/she can write like this.
CONFFILES_A = "${sysconfdir}/default"
[YOCTO #5200]
(From OE-Core rev: 0d446ef0e5bbca7058eec7259e34f2a1637dfab1)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Trying to upgrade busybox removing symlinks but update-alternatives
need these links (sed, cut, tail, etc) in order to work.
Adding test to avoid this scripts on upgrade fix the problem, same
solution are found in package_rpm class.
[YOCTO #6768]
(From OE-Core rev: 7b9161dd0c475cca6ea7eb507f7c3c51869eb493)
Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The package_write task was previously removed. Remove a remaining superfluous
reference to it.
(From OE-Core rev: 76bbf9e8f07f3e6f20c890dd4c82c72641e2ca88)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Without this patch packages are generated as x86_64. Which cannot be
installed by default.
root@qt5022:~# dpkg -i alsa-utils_1.0.27.2-r0_x86-64.deb
dpkg: error processing alsa-utils_1.0.27.2-r0_x86-64.deb (--install):
package architecture (x86-64) does not match system (amd64)
Errors were encountered while processing:
alsa-utils_1.0.27.2-r0_x86-64.deb
(From OE-Core rev: a08eacc6d821d6946b23a99bca5abf785875b1cf)
Signed-off-by: Ricardo Ribalda Delgado <ricardo.ribalda@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The reasons this task was introduced are lost in the mists of time. It
allowed for the a single "package_write" task instead of spelling out
the explicit package backends, however in all but one case we do that
anyway.
As such as might as well give in and delete the task, converting that
single reference into explicit dependencies.
This gives bitbake a bit less work to to when processing the runqueue
since there are less tasks (but more dependencies in some cases).
(From OE-Core rev: cf70e15f063716f3227d467ab1f4bfc0018286f6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Package indexing is done in python and package-index.bb uses the new
routines.
(From OE-Core rev: 2ab1a2bccfbb4ed90fe3b877d1be80817ba32099)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The default value for HOMEPAGE of "unknown" has been in place since the
early OE-Classic days, but it doesn't really make sense - "unknown" is
not a valid URL and it just means we have to explicitly check for this
hardcoded string if we're displaying the value in some form of UI, such
as Toaster.
This has required some changes to the packaging classes as they
previously did not expect the value to be blank.
(From OE-Core rev: 244e1d73ef58e92d73c098044c66bd784644b933)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the packages are created from cache, we need to remove the stamp so
that we re-generate the index files at do_rootfs time.
(From OE-Core rev: dc06a91144b79a152eb481f6d36f6c328321b7c4)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit will:
* remove old bash code common to all backends;
* create a new do_rootfs() python function that will use the new
rootfs/image creation routines;
* allow creation of dpkg based images;
* fail for rpm/opkg (not implemented yet);
(From OE-Core rev: a83144bac8d67704ff66f5dc0fc56f5b63979694)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Additional metadata from user-defined variable is written into
control/spec file of binary package.
Three variables are searched for adiitional package metadata:
* PACKAGE_ADD_METADATA_<PKGTYPE>_<PN>
* PACKAGE_ADD_METADATA_<PKGTYPE>
* PACKAGE_ADD_METADATA
First found variable with defined value wins.
<PN> is a package name. <PKGTYPE> is a distinct name of specific
package type:
* IPK for .ipk packages
* DEB for .deb packages
* RPM for .rpm packages
Variable can contain multiple [one-line] metadata fields separated by
literal sequence '\n'. Separator can be redefined through variable flag
'separator'. In package control/spec file separator is replaced by
newline character.
(From OE-Core rev: 773d7352309241e15ef5acadcbe416bdd7d45c18)
Signed-off-by: Leonid Borisenko <ive.found@gmail.com>
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When originally developed, it was thought a task may have more than one associated
sstate archive. The way the code has grown that idea is now not possible or needed.
We can therefore assume one sstate archive per task and drop the crazy name
mapping code. Simpler is better in this case.
The downside is that various sstate archives will change name so this forces a cache
rebuild. Given the other sstate changes going in at this time, this isn't really
a bad thing as things would rebuild anyway.
(From OE-Core rev: 5afe86a6854b21692fd97c5fc7fab50dbc068acb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you Ctrl+C ppackage_write_{deb/ipk} control files can get left lying around
and make it into another packaging format. This ensures we cleanup all known
control files before starting packaging. We can simplify some of the globbing
as a result.
(From OE-Core rev: 7f8c728c82a17621461be2114c1afab3713a808e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The changes in this commit, dc5f6c3898,
moved apt config directory from native SYSROOT to WORKDIR.
Unfortunately, Dir::Etc in apt.conf was not changed accordingly and
sources.list file could not be found during do_rootfs().
This commit fixes this issue.
[YOCTO #5241]
(From OE-Core rev: 65a2a1e1d0b0e3dcc34d47f46185f24d753b53dd)
Signed-off-by: Laurentiu Palcu <laurentiu.palcu@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These have been deprecated for a long time, convert the remaining
references to the correct modules and prepare for removal of the
compatibility support from bitbake.
(From OE-Core rev: 6a39835af2b2b3c7797fe05479341d71a3f3aaf6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support for NO_RECOMMENDATIONS and PACKAGE_EXCLUDE. Also add a
warning that ensures users know that BAD_RECOMMENDATIONS support is
not implemented in the debian package/rootfs classes.
(From OE-Core rev: 42b115b6d65c8205acb77b96db481f3e5172266b)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
