Where copyright headers were not present, add them to make things
clear.
(Bitbake rev: 1aa338a216350a2751fff52f866039343e9ac013)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We should not redirect stderr to stdout if we need to get separated
stdout and stderr contents from Popen.communicate() later.
(Bitbake rev: 1ecc1d9424877df89fcda2f23c306998998a65ff)
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the caller is piping the logs, they likely don't want them in the error exception
as well. This removes duplicate output from the build output allowing the UI level
controls on whether to show logs to work correctly.
(Bitbake rev: fc58ad84a9deb2620ad90611684dad65dafedb11)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Correctly import, and inherit functions, and variables.
Also fix some typos and remove some Python 2 code that isn't recognised.
(Bitbake rev: b0c807be5c2170c9481c1a04d4c11972135d7dc5)
Signed-off-by: Frazer Clews <frazerleslieclews@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Leverage the comments that emit_var writes and the backtrace that
the shell func writes to generate an additional metadata-relative
backtrace. This will help the user troubleshoot shell funcs much
more easily.
Example:
| WARNING: /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955:171 exit 1 from 'exit 1'
| WARNING: Backtrace (BB generated script):
| #1: myclass_do_something, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 171
| #2: do_something, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 166
| #3: actually_fail, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 153
| #4: my_compile_extra, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 155
| #5: do_compile, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 141
| #6: main, /home/laplante/repos/oe-core/build/tmp-glibc/work/core2-64-oe-linux/libsolv/0.7.14-r0/temp/run.do_compile.68955, line 184
|
| Backtrace (metadata-relative locations):
| #1: myclass_do_something, /home/laplante/repos/oe-core/meta/classes/myclass.bbclass, line 2
| #2: do_something, autogenerated, line 2
| #3: actually_fail, /home/laplante/repos/oe-core/meta/recipes-extended/libsolv/libsolv_0.7.14.bb, line 36
| #4: my_compile_extra, /home/laplante/repos/oe-core/meta/recipes-extended/libsolv/libsolv_0.7.14.bb, line 38
| #5: do_compile, autogenerated, line 3
ERROR: Task (/home/laplante/repos/oe-core/meta/recipes-extended/libsolv/libsolv_0.7.14.bb:do_compile) failed with exit code '1'
NOTE: Tasks Summary: Attempted 542 tasks of which 541 didn't need to be rerun and 1 failed.
Summary: 1 task failed:
/home/laplante/repos/oe-core/meta/recipes-extended/libsolv/libsolv_0.7.14.bb:do_compile
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
(Bitbake rev: ae1aa4ea79826c32b20e1e7abdf77a15b601c6f2)
Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds the SPDX-License-Identifier license headers to the majority of
our source files to make it clearer exactly which license files are under.
The bulk of the files are under GPL v2.0 with one found to be under V2.0
or later, some under MIT and some have dual license. There are some files
which are potentially harder to classify where we've imported upstream code
and those can be handled specifically in later commits.
The COPYING file is replaced with LICENSE.X files which contain the full
license texts.
(Bitbake rev: ff237c33337f4da2ca06c3a2c49699bc26608a6b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a process terminates, some messages may still remain in stdout or
stderr and do not make it into the log file.
In addition, the messages that do make it to the log file may end up in
the log file in incorrect order.
This patch flushes all messages into the log file after the
process terminates. Some additional log flushing is also needed
to keep the various messages showing up in the log file in proper order.
[YOCTO#10785]
(Bitbake rev: 1f6e6aa8262369eafc3bbf9f01f8d981f90becdf)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python3 regards b"" as False so it is not being converted to a string by
d0f904d407f57998419bd9c305ce53e5eaa36b24. This confusingly causes three
different potential types for each member of the returned tuple.
Let's just assume that everything that's not None is a bytes object and
convert it to a string.
(Bitbake rev: 0cf5589b7fb3582a6caca5014c4d8152347df545)
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Various misc changes to convert bitbake to python3 which don't warrant
separation into separate commits.
(Bitbake rev: d0f904d407f57998419bd9c305ce53e5eaa36b24)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With python you should not assign a list as the default value of a
function parameter - because a list is mutable, the result will be that
the first time a value is passed it will actually modify the default.
Reference:
http://docs.python-guide.org/en/latest/writing/gotchas/#mutable-default-arguments
(Bitbake rev: 7859f7388f2e3f675d0e1527cfde18625f36f637)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're calling select() to find the fds that have data available, so we
really ought to check the return to see if the extra file is in there
before trying to read from it. This is part of the fix for the
performance regression that this code introduced (5-10 minutes extra in
a reasonable size OE build); the rest is down to an issue in the way
that pseudo currently handles FIFOs and will need to be addressed there,
see: https://bugzilla.yoctoproject.org/show_bug.cgi?id=7993
Solution suggested by Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: 3e2db8d7eaa0f14813213d1c95d3ee9efd97dc9d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create a fifo under ${T} for each running task that can be used by the
task to send events back to BitBake. The purpose of this is to allow
OpenEmbedded's logging.bbclass (which provides shell function
equivalents for bb.warn(), bb.note() etc.) to have those functions
trigger the appropriate events within BitBake so that messages are shown
through the BitBake UI rather than just in the task log; to do that we
just call the python functions.
Part of the fix for [YOCTO #5275].
(Bitbake rev: bae330a1f8a59a912eca71177b3c6ba7c92a2f38)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
On some machines, a hang has been noticed where the system sits in
the select call despite the task having completed.
The exact reasons for this as unknown but adding a timeout unblocked
the builds and resolved the hangs in question.
(Bitbake rev: 5223ffb5b6a46d8b3f6ac3362bd2672e2edf2691)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python 3 is stricter about how files are accessed. Specficially:
* Use open(), not file()
* Use binary mode for binary files (when checksumming)
* Use with statements to ensure files get closed
* Add missing file close statements
(Bitbake rev: 9f08b901375ba640f47596f1bcf43f98a931550f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are two problems with the _logged_communicate that are both
caused as a result of buffering I/O issues:
1) log truncation when python fails
2) While a bitbake task is running it is impossible to see what is
going on if it is only writing a small incremental log that is
smaller than the buffer, or you get only a partial log, up until
the task exists. It is worse in the case that stderr and stdout
are separate file handles, because previous code blocks on the read
of stdout and then stderr, serially.
The right approach is simply to use select() to determine if there is
data available and also flush the log before exiting.
This is based on a patch from Jason Wessel <jason.wessel@windriver.com>
with some changes to flush upon exit, abstract the non blocking file
descriptor setup and drop the buffer size parameter.
(Bitbake rev: 361fb71e907aa84c28ecec79fefc6ca39c39172f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update exception handling syntax to use the modern style:
except ExcType as localvar
(Bitbake rev: dbf5f42b06bef81749b13aa99945cc1292a6676d)
Signed-off-by: Scott Garman <scott.a.garman@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake/lib/bb/process.py:15: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using a logfile, we weren't sending input to the child process.
(Bitbake rev: 5ec4ca7e45bdf6d259503fc67155395e89ba6329)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
The previous attempt was incorrect. The issue isn't that subprocess fails to
set PWD, it's that PWD is in the metadata, inherited from the environment, and
is re-exported, overwriting the actual accurate one in the shell environment
with the old one from the metadata. So, ensure that PWD in the metadata is
not exported.
We can ditch this when the environment handling is reworked (e.g. poky's
commit to do so).
(Bitbake rev: 2c8683234acf514706b2b69f5b29405485e664dd)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
