Commit Graph

61 Commits

Author SHA1 Message Date
Joshua Watt
4fd9ee4670 bitbake: bitbake: Add --skip-setscene option
Adds an option to skip _setscene only if they would normally be
executed, without ignoring sstate completely.

Previously, '--no-setscene' would allow a build that completely ignored
sstate and _setscene tasks, and '--setscene-only' would allow a build
that only ran _setscene tasks, but there was no option do a build that
would respect tasks previously restored from sstate and build everything
else. Now one can run:

 bitbake --setscene-only IMAGE; bitbake --skip-setscene IMAGE

which is functionally equivalent to:

 bitbake IMAGE

The indented use is to allow a build to complete successfully in the
presence of _setscene task failures by splitting apart the two phases
e.g.:

 (bitbake -k --setscene-only IMAGE || true) && bitbake --skip-setscene IMAGE

(Bitbake rev: 813ba5b7c13b573a0b813b628a819bdbf0627540)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-03 17:00:57 +01:00
Richard Purdie
97e19bf73b bitbake: main: Alter EOFError handling
If the server shuts down 'cleanly' due to some issue, the socket will close.
A recently reported example was an invalid PRSERV being set. Doing this silently
and without changing the retries count will case the server startup to loop
infinitely.

Change the code so it triggers the usual retries note messages and times
out eventually pointing the user at the cooker log file.

[YOCTO #12984]

(Bitbake rev: bb696636ef0c59f9e9640bb9460e7cce323cc785)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-03 17:00:57 +01:00
Richard Purdie
53138165bc bitbake: main: Ensure log messages are printed when no UI starts
If the UI hasn't started, no messages are printed to the console
until the server starts. This is confusing, particularly if the server
never starts. Flush the UI queue through the simply handler upon connection
retry so the user sees the messages they're supposed to be seeing.

Also point the user at the logfile for hints about why this may be.

(Bitbake rev: 4b9ab675cebb427ab8ad0c56c7b37eed50a2a39e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-03 17:00:57 +01:00
Robert Yang
5161799993 bitbake: bitbake: lib: Cleanup /usr/bin/env python
(Bitbake rev: cc712f3257904960247a7532cfc4611f3dccd36c)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-28 13:29:04 +01:00
Richard Purdie
cc67ce63d5 bitbake: main: Fix error message typo
(Bitbake rev: a1330b8a15a3f12516e3e293ffb5d840f362375d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-30 12:37:03 +01:00
Richard Purdie
9501864db8 bitbake: bitbake: Strip old editor directives from file headers
There are much better ways to handle this and most editors shouldn't need this
in modern times, drop the noise from the files. Its not consitently applied
anyway.

(Bitbake rev: 5e43070e3087d09aea2f459b033d035c5ef747d0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-04 10:44:10 +01:00
Richard Purdie
cf9c0be3f6 bitbake: bitbake: Drop duplicate license boilerplace text
With the introduction of SPDX-License-Identifier headers, we don't need a ton
of header boilerplate in every file. Simplify the files and rely on the top
level for the full licence text.

(Bitbake rev: 695d84397b68cc003186e22f395caa378b06bc75)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-04 10:44:10 +01:00
Richard Purdie
79834a7144 bitbake: bitbake: Add initial pass of SPDX license headers to source code
This adds the SPDX-License-Identifier license headers to the majority of
our source files to make it clearer exactly which license files are under.

The bulk of the files are under GPL v2.0 with one found to be under V2.0
or later, some under MIT and some have dual license. There are some files
which are potentially harder to classify where we've imported upstream code
and those can be handled specifically in later commits.

The COPYING file is replaced with LICENSE.X files which contain the full
license texts.

(Bitbake rev: ff237c33337f4da2ca06c3a2c49699bc26608a6b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-04 10:44:04 +01:00
Richard Purdie
c263f31e39 bitbake: main: When retrying the connection, show the attempt number
The current bitbake output makes it hard to know which retry is being
attempted. Add this information to the output to make it clearer.

(Bitbake rev: 0774e6e03d27adb7aca6fa9c47ab6ad426c937de)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-26 11:10:33 +00:00
Richard Purdie
5806b4cbd8 bitbake: main: Don't use print() directly, use logger
Avoid failures like:

2018-12-04 03:30:56,033 - oe-selftest - INFO - ======================================================================
2018-12-04 03:30:56,033 - oe-selftest - INFO - FAIL: pkgdata.OePkgdataUtilTests.test_find_path (subunit.RemotedTestCase)
2018-12-04 03:30:56,033 - oe-selftest - INFO - ----------------------------------------------------------------------
2018-12-04 03:30:56,033 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
[...]
AssertionError: 'Previous bitbake instance shutting down?,[77 chars]xist' != 'ERROR: Unable to find any package produci[14 chars]xist'
- Previous bitbake instance shutting down?, waiting to retry...
  ERROR: Unable to find any package producing path /not/exist

We need to use the logger so output is correctly handled in such cases.

(Bitbake rev: b5686fc07d06e1f0a55161b5ebcec6552d74ceff)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-12-05 13:11:27 +00:00
Richard Purdie
4806d53304 bitbake: main: Fix environment handling for UI module imports
The environment was being cleared before the UI imports occurred which
caused problems for graphical UIs like taskexp. The full environment was
intended to be available to UI clients and it was only meant to be cleared
for the server/cooker, so tweak the code order so this is the case.

This fixes problems reported for taskexp.

[YOCTO #12670]

(Bitbake rev: ac15028391fdcc3fec2dd0e64a4f763ce63e5449)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-10-10 12:47:48 +01:00
Richard Purdie
c75985f40a bitbake: main/runqueue: Rework runall task and add runonly option
The runall commandline option was confusing people. There are in fact two
different behaviours people may want.

a) For a given target (or set of targets) look through the task graph and
run task X only if its present and would have been built.

b) For a given target (or set of targets) look through the task graph and
run task X if any recipe in the taskgraph has such a target even if it wasn't
in the original task graph.

I've decided to interpret the existing "runall" option as b), even if right
now if behaves like a). For a), which is a valid use case, this patch adds
a "runonly" option.

With both behaviours present, I'm hoping we can then kill off the "fetchall",
"checkuriall" and other tasks from OE metadata and replace them with this
option. This would significantly speed up task graph processing.

(Deleting the checkuriall and fetchall tasks takes "bitbake core-image-sato -g"
from 22s to 8s).

(Bitbake rev: 546a662c877b2d3af35e3996950582ed2df41fe4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-02-14 15:26:03 +00:00
Richard Purdie
5dacc55167 bitbake: main: Give a user readable error if we can't locate topdir
Currently if you run bitbake in an invalid directory, the user experience
is poor:

birbake/lib/bb/main.py", line 427, in setup_bitbake
    topdir, lock = lockBitbake()
  File "./bitbake/lib/bb/main.py", line 494, in lockBitbake
    lockfile = topdir + "/bitbake.lock"
TypeError: unsupported operand type(s) for +: 'NoneType' and 'str'

This ensures we exit straight away with a better error message.

[YOCTO #12163]

(Bitbake rev: 562f9ee674a8b392437096422b9cceab9c3cba2e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-10-31 09:09:21 +00:00
Richard Purdie
2355a9b6f0 bitbake: cookerdata/taskdata/runqueue: Drop remaining tryaltconfigs code and commandline option
I can't actually see how this was working, nothing connected the commandline option
to the data in TaskData(). Drop the remaining pieces of this option, it was a relic
from a decade ago and we want deterministic builds, not random tries until something
might work.

(Bitbake rev: 767c7ba8fc76ec667ac1567de1c971c3575f2ecd)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-02 00:52:10 +01:00
Richard Purdie
4152c47243 bitbake: main: Attempt to gain bitbake.lock rather than just waiting
Rather than just waiting for 5s, try and get the lockfile. If we gain
the lock, we know we're ready to retry and can skip any remaining timeout.

(Bitbake rev: 8a60106c6f7d586c793b965c5e9460b6016fab15)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-24 13:48:40 +01:00
Richard Purdie
2b453483d4 bitbake: process: Clean up connection retry logic
Its possible for a connection to connect to the server as its shutting down
but before its removed the socket file. This patch:

a) Removes the socket file earlier to avoid connections.
b) Handles EOFError in initial connections gracefully. These occur if the
   socket is closed during the server shutdown.
c) Ensure duplicate events aren't shown on the console. This makes debugging
   these issues very very confusing.

With these changes the backtrace that was concerning users is hidden and the
server works as expected with a reconnect when it catches it in a bad state.

(Bitbake rev: f45196cf84669723382730944dddc7eaf50826f2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-24 13:48:40 +01:00
Robert Yang
803b97a32e bitbake: main: Handle BB_SERVER_TIMEOUT = -1 for no server timeout
Make BB_SERVER_TIMEOUT = -1 mean no unload forever.

(Bitbake rev: 923d5567be6fd9969ca74c166f36817ec09305e3)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-09 09:24:56 +01:00
Robert Yang
896243f4fe bitbake: main: Remove unneeded float()
There is already a type=float, so the float() is not needed, which also makes
the error clearer:

$ export BB_SERVER_TIMEOUT=10000AA
With float():
$ bitbake quilt-native
[snip]
ValueError: could not convert string to float: '10000AA'

Without float():
$ bitbake quilt-native
[snip]
optparse.OptionValueError: option --idle-timeout: invalid floating-point value: '10000AA'

The second one tells clearly where is wrong.

(Bitbake rev: 7a4ea1e6a10f20d80009a78f4f0aebf8f90095fd)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-09 09:24:56 +01:00
Paul Eggleton
46e2251625 bitbake: Revert "tinfoil: fix duplication of log messages"
In combination with the recent server reworking, this change actually
prevents messages sent from tasks from being logged properly. This will
of course give us the duplicated messages back, and I really hate to do
that effectively a second time, but that's better than seeing no error
at all in the case of a failure - we'll have to find the proper way of
avoiding the duplication that doesn't result in some messages going
missing.

This reverts commit 8a5bae76f91f2411187c638a42fa3c762052cf11.

(Bitbake rev: 645c8dd15762516ae5ab64a1df47fadb95d072d1)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-09 09:24:56 +01:00
Richard Purdie
e4c9a92cb4 bitbake: main: Always return 0 for bitbake --kill-server
If the server isn't running return 0 as to do otherwise complicates
scripts which do cleanup of bitbake servers which would potentially
be memory resident.

(Bitbake rev: 0f85c54385f325f3ebbf911b35bbac11b8863b6b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-30 08:43:36 +01:00
Robert Yang
579b811233 bitbake: bb/main.py: fix logic for --observe-only
Fixed:
$ bitbake --observe-only
FATAL: '--observe-only' can only be used by UI clients connecting to a server.

And even:
$bitbake --observe-only -B localhost:-1
FATAL: '--observe-only' can only be used by UI clients connecting to a server.

This was beucase the brackets were in a wrong position.

(Bitbake rev: 4ffc91a2b3eb13e98078e6b1913f056a0c1797bc)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21 08:41:11 +01:00
Robert Yang
c276e38bf8 bitbake: bb/main.py: avoid starting server when not needed
Fixed 1:
$ . ../poky/oe-init-build-env-memres .
$ bitbake -m
$ bitbake -m # shutdown it again
$ bitbake -m
Starting bitbake server...
Terminated bitbake server.

It starts the server and kill it which is redundant.

$ bitbake -m
$ bitbake --status-only
Starting bitbake server...

It starts the server which is not what we need.

(Bitbake rev: 335fd0106359dddfe1eeb0c7d0bc03f8454b895c)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21 08:41:11 +01:00
Robert Yang
38af7fef51 bitbake: bb/main.py: fix infinite loop for --server-only
Fixed:
$ bitbake --server-only -B localhost:-1
Infinite loop
Reconnecting to bitbake server...
[snip]

Break the loop when server-only can fix the problem.

(Bitbake rev: c4d5c3dab5b35f3c6713efad1c8a23229ce4e56b)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21 08:41:11 +01:00
Richard Purdie
4602408c69 bitbake: server: Rework the server API so process and xmlrpc servers coexist
This changes the way bitbake server works quite radically. Now, the
server is always a process based server with the option of starting
an XMLRPC listener on a specific inferface/port.

Behind the scenes this is done with a "bitbake.sock" file alongside
the bitbake.lock file. If we can obtain the lock, we know we need
to start a server. The server always listens on the socket and UIs
can then connect to this. UIs connect by sending a set of three file
descriptors over the domain socket, one for sending commands, one for
receiving command results and the other for receiving events.

These changes meant we can throw away all the horrid server abstraction
code, the plugable transport option to bitbake and the code becomes
much more readable and debuggable. It also likely removes a ton of
ways you could hang the UI/cooker in weird ways due to all the race
conditions that existed with previous processes.

Changes:

 * The foreground option for bitbake-server was dropped. Just tail
   the log if you really want this, the codepaths were complicated enough
   without adding one for this.
 * BBSERVER="autodetect" was dropped. The server will autostart and
   autoconnect in process mode. You have to specify an xmlrpc server
   address since that can't be autodetected. I can't see a use case
   for autodetect now.
 * The transport/servetype option to bitbake was dropped.
 * A BB_SERVER_TIMEOUT variable is added which allows the server
   to stay resident for a period of time after the last client
   disconnects before unloading. This is used if the -T/--idle-timeout
   option is not passed to bitbake.

This change is invasive and may well introduce new issues however I
believe the codebase is in a much better position for further
development and debugging.

(Bitbake rev: 72a3dbe13a23588e24c0baca6d58c35cdeba3f63)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21 08:41:11 +01:00
Paul Eggleton
d44d5a2e79 bitbake: tinfoil: fix duplication of log messages
Adding an additional logger in setup_bitbake() interacts poorly with the
logger we have added by default in tinfoil's constructor, with the
result that messages may be doubled or even tripled in tinfoil-using
scripts. Disable adding this one when calling setup_bitbake() from
tinfoil to avoid this problem.

Part of the fix for [YOCTO #11275].

(Bitbake rev: 8a5bae76f91f2411187c638a42fa3c762052cf11)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-12 15:07:39 +01:00
Diana Thayer
71e48a3617 bitbake: main: Improve -v and -D option documentation
Expanded and clarify documentation for the -v, --verbose and -D, --debug
options.

[YOCTO #9962]

(Bitbake rev: 921592026c69287cdb40ffd90944d5944f28e2c3)

Signed-off-by: Diana Thayer <garbados@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-27 20:23:19 +01:00
Matthew McClintock
404f2aef94 bitbake: runqueue: add option to run all tasks in specific build target
For example:

$ bitbake core-image-minimal --runall unpack
...
NOTE: Tasks Summary: Attempted 326 tasks of which 88 didn't need to be rerun and all succeeded.

$ bitbake core-image-minimal --runall patch
NOTE: Tasks Summary: Attempted 453 tasks of which 332 didn't need to be rerun and all succeeded.

This can replace fetchall as well:
$ bitbake core-image-minimal --runall fetch
NOTE: Tasks Summary: Attempted 135 tasks of which 119 didn't need to be rerun and all succeeded.

(Bitbake rev: 7c0fa6ba66cdb956b37d94055307cde857633df9)

Signed-off-by: Matthew McClintock <msm-oss@mcclintock.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-13 09:43:40 +00:00
Aníbal Limón
8a3221e6e5 bitbake: lib/bb/main.py: Shutdown cooker when server isn't foreground
The cooker is added to the BBServer and then detached creating
a copy of the cooker in the process, if the server isn't in
foreground it cause fd leaks on inotify this can be see using
many instances of tinfoil.

Example:

from bb.tinfoil import Tinfoil
while True:
    with Tinfoil() as tinfoil:
        input("Pre\n")
        tinfoil.prepare(config_only=True)
        input("Post\n")

[YOCTO #10873]

(Bitbake rev: 2ada2efb057a291eb34c66d975eb339232137956)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 22:38:49 +00:00
Paul Eggleton
7d5c9860de bitbake: tinfoil: rewrite as a wrapper around the UI
Rewrite tinfoil as a wrapper around the UI, instead of the earlier
approach of starting up just enough of cooker to do what we want. This
has several advantages:

* It now works when bitbake is memory-resident instead of failing with
  "ERROR: Only one copy of bitbake should be run against a build
  directory".

* We can now connect an actual UI, thus you get things like the recipe
  parsing / cache loading progress bar and parse error handling for free

* We can now handle events generated by the server if we wish to do so

* We can potentially extend this to do more stuff, e.g. actually running
  build operations - this needs to be made more practical before we can
  use it though (since you effectively have to become the UI yourself
  for this at the moment.)

The downside is that tinfoil no longer has direct access to cooker, the
global datastore, or the cache. To mitigate this I have extended
data_smart to provide remote access capability for the datastore, and
created "fake" cooker and cooker.recipecache / cooker.collection adapter
objects in order to avoid breaking too many tinfoil-using scripts that
might be out there (we've never officially documented tinfoil or
BitBake's internal code, but we can still make accommodations where
practical). I've at least gone far enough to support all of the
utilities that use tinfoil in OE-Core with some changes, but I know
there are scripts such as Chris Larson's "bb" out there that do make
other calls into BitBake code that I'm not currently providing access to
through the adapters.

Part of the fix for [YOCTO #5470].

(Bitbake rev: 3bbf8d611c859f74d563778115677a04f5c4ab43)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:25:07 +00:00
Paul Eggleton
568409faa1 bitbake: knotty: make quiet option a level option
Allow you to specify -q / --quiet more than once to reduce the messages
even further. It will now operate as follows:

 Level  Option  Result
 -----  ------  ----------------------------------------
 0              Print usual output
 1      -q      Only show progress and warnings or above
 2      -qq     Only show warnings or above
 3+     -qqq    Only show errors

(Bitbake rev: 6cf2582e17c28ca04f5cfb59858c4a9778c700d4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:25:06 +00:00
Benjamin Esquivel
b9d6a7cc23 bitbake: main: Check bitbake server-only port is a number
Either using the memres script or the bitbake call with --server-only
if the port is a string instead of a number then the process hangs
indefinitely causing a loop that never ends.

Add a check at the beginning for the port being a number otherwise
show an error message and exit cleanly.

[YOCTO #10397]

(Bitbake rev: 35927a98daeeb854ef5782e900206af6cd96b3d7)

Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-09 12:33:26 +01:00
Ed Bartosh
317708235b bitbake: bitbake: implement idle timeout for xmlrpc server
Idle timeout can be specified either by -T/--idle-timeout option or
by sessing BBTIMEOUT environment variable. Bitbake xmlrpc server
will unload itself when timeout exprired, i.e. when server is idle
for more than <idle timeout> seconds.

[YOCTO #5534]

(Bitbake rev: 5fa0b3a19e7d0f40790f737abeddf499d53f1f6a)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-21 07:48:52 +01:00
Ed Bartosh
4cba010529 bitbake: bitbake: implement --foreground command line option
This option makes bitbake xmlrpc server to run in foreground.
It should be useful for debugging purposes.

(Bitbake rev: 9d4254be5853a546a346bf0d19919dcfba12773d)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-21 07:48:52 +01:00
Ed Bartosh
63ff759627 bitbake: bitbake: main: implement server autostart feature
If environment variable BBSERVER == 'autostart' bitbake will
automatically load server if it's not running yet.

If host and port are in bitbake.lock then bitbake tries to check
if server is running and responses to commands and starts new
server only if this check fails.

[YOCTO #5534]

(Bitbake rev: 89c6e625d47303b2aad8e6645762f17aee01b2d4)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:23:46 +01:00
Paul Eggleton
76feac37ce bitbake: knotty: add quiet output mode
Quiet output mode disables printing most messages (below warnings) to
the console; however these messages still go to the console log file.
This is primarily for cases where bitbake is being launched
interactively from some other process, but where full console output is
not needed.

Because of the need to keep logging all normal events to the console
log, this functionality was implemented within the knotty UI rather
than in bb.msg (where verbose mode is implemented). We don't currently
have a means of registering command line options from the UI end, thus
the option actually has to be registered in main.py regardless of the
UI, however I didn't feel like it was worth setting up such a mechanism
just for this option.

(Bitbake rev: db95cdef08e339dec7462bfde3ad7d75c1c60dd8)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:27 +01:00
Mariano Lopez
40e789d1fe bitbake: lib/bb/main.py: Fix use of BBPOSTCONF and BBPRECONF
The functionality of BBPOSTCONF and BBPRECONF was added in
commit 21b314d4d1 but there
was a typo in the variable name that raises an exception
in bitbake.

[YOCTO #9235]

(Bitbake rev: 6d1379c8818400e5cdc442e6142f08a110fd5b95)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-09 18:00:58 +01:00
Richard Purdie
0f2c59367a bitbake: bitbake: Convert to python 3
Various misc changes to convert bitbake to python3 which don't warrant
separation into separate commits.

(Bitbake rev: d0f904d407f57998419bd9c305ce53e5eaa36b24)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:02 +01:00
Richard Purdie
c0db739f35 bitbake: main: Ensure exceptions are correctly displayed
If the cooker fails to start, ensure a correct exception is displayed to the
user. After handling any queued events simply re-raise the original exception
else the output can be unclear.

(Bitbake rev: 9a4db1aa608c17d31bf5ea1cab5a99beb565dd83)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-13 13:41:33 +01:00
Richard Purdie
ef4a7c30ca bitbake: main: Change warn() -> warning()
This avoids a deprecation warning in python 3.

(Bitbake rev: bf1a92d0c002d73e8a34472dced1343dc4a4251a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-13 13:41:32 +01:00
Ed Bartosh
be49d0ad57 bitbake: bitake: main: fix line-too-long pytling warnings
Wrapped long lines to fix "Line too long" pylint warnings.

(Bitbake rev: e329a932e14d002a561245b5026f974897f64598)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-06 10:12:16 +01:00
Ed Bartosh
b5af50e024 bitbake: bitbake: main: fix bad-continuation warnings
Fixed pylint warning 'Wrong continued indentation'.

(Bitbake rev: 5bfc8c0dc70abd6462a6df4ce905173d9f2cdc07)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-06 10:12:16 +01:00
Ed Bartosh
3437266a08 bitbake: bitbake: main: fix bad-witespace pylint warnings
Fixed pylint warnings:
 No space allowed around keyword argument assignment
 No space allowed after bracket
 No space allowed before bracket

(Bitbake rev: c39770239f7b61217501782b9c5e9d3211355d42)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-06 10:12:16 +01:00
Ed Bartosh
24f5c32b80 bitbake: bitbake: main: set defaults from env variables
Environment variables BBSERVER, BBTOKEN and BBEVENTLOG silently
overwrite bitbake command line arguments. This is confusing and
can cause issues that are difficult to debug. It's better to use
them as default values instead.

Used environment variables BBSERVER, BBTOKEN and BBEVENTLOG to set
default values for command line arguments.

Changed setting default value of --ui command line argument from
BITBAKE_UI to look similar way.

(Bitbake rev: 87040be4ff54cd460961318224deef8f2ea4c85a)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-06 10:12:16 +01:00
Ed Bartosh
21b314d4d1 bitbake: bitbake: main: add 2 environment variables
Bitbake uses set of environment variables to set command line
options, e.g. seeting BBTOKEN variable has the same effect
as using --token command line option.

Added new environment variables BBPRECONF and BBPOSTCONF that
are equivalents of --read and --postread command line options.
They can be used by high level scripts to append or prepend
configuration files to conf/local.conf

[YOCTO #9235]

(Bitbake rev: bf604ec1ca4eb4d0b22bcc72249963e6d7445f34)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-06 10:12:16 +01:00
Ed Bartosh
c7980b6089 bitbake: main: fix processing of BBEVENTLOG
Fixed typo that caused incorrect processing of BBEVENTLOG
environment variable. Even if variable is set it was ignored
by bitbake.

(Bitbake rev: 2705b5f59aef4a070e2df2752d27bd04ea747057)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-19 17:31:01 +01:00
Ed Bartosh
b58e5b1fb3 bitbake: bitbake: xmlrpc: set single use mode differently
Currently xmlrpc server implicitly sets itself into single use mode
when bitbake server is started with anonymous port (0) or no port is
provided in command line. In this mode bitbake shuts down xmlrpc server
after build is done. This assumption is incorrect in some cases.
For example Toaster uses bitbake in this mode and expects xmlrpc server
to stay in memory.

Till recent changes single use mode was always unset due to the bug.
When the bug was fixed it broke toaster builds as Toaster couldn't
communicate with bitbake server in single use mode.

Reimplemented logic of setting single use mode. The mode is explicity
set when --server-only command line parameter is not provided to bitbake.
It doesn't depend on the port number anymore.

[YOCTO #9275]
[YOCTO #9240]
[YOCTO #9252]

(Bitbake rev: afc0dd5c532684f6201b1e12bbf4c226ea19062d)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-24 12:31:50 +00:00
Richard Purdie
46731daa5f bitbake: main/runqueue: Add --setscene-only option to bitbake
Its turning out that we really need a way to have bitbake just run
the setscene tasks but not any real tasks, particularly for SDK
operations.

Add an option for this since its pretty straight forward. This allows
various nasty workarounds in OE-Core to be removed.

(Bitbake rev: e4a2aafa1650a227a04d92a8a0b31efaed2c310e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:32 +00:00
Ed Bartosh
a178c5a163 bitbake: main: kill server without queue setup
Setup of event queue includes registering of UI handler.
This operation can fail when cooker is busy. However, there is
no need in registering UI handler for terminating the server.

Moved the call of connection.terminateServer before setting up
of the event queue. This should make terminating server to work
more reliably as it doesn't depend on setting up the event queue
and registering UI handler anymore.

This should also help Toaster backend to restart bitbake server
and observer without getting "Could not register UI event handler"
errors.

[YOCTO #8776]

(Bitbake rev: 0c5a9349f797d05c282c2ada1893e187e05f0576)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06 15:27:35 +00:00
Ed Bartosh
773700dbc3 bitbake: xmplrpc: split connect method
Current code in connect method sets up event queue, which requires
registering UI handler. This functionality may not be needed for
some operations, e.g. for server termination.

Moved functionality of setting up event queue in from 'connect'
method to 'setupEventQueue' in BitBakeXMLRPCServerConnection class.

(Bitbake rev: 4429871da76d6bd29e023ff42740fe7daa6b40fa)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-06 15:27:35 +00:00
Ed Bartosh
4ab7202c31 bitbake: cooker: preserve pre and post configs
Additional config files passed to bitbake server with --read and
--postread options are rewritten by client bitbake even if
it doesn't use those options.

This is a show stopper for toaster as toaster command line
builds are based on the assumption that server is aware of
toster configs, provided by --postread option.

This behaviour is fixed by preserving values of --read and
--postread options when bitbake server starts and restoring
them if client bitbake doesn't explicitly specify them.

(Bitbake rev: 02c64f7487ca8ec5d32c440f5002c4b8f64b76a6)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-27 07:24:30 +00:00