Commit Graph

139 Commits

Author SHA1 Message Date
Alexandru DAMIAN
4aebdecdd7 bitbake: xmlrpc: removes the event servers controller
XMLRPC API ran a XMLRPC server at port+2 in order
to provide endpoints to register an event server
in which to dump the events.

This is no longer used, so we remove it.

(Bitbake rev: e171a363913a86e56266f4c9d107110c7f5221e3)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-14 14:34:59 +01:00
Alexandru DAMIAN
f549cdd712 bitbake: xmlrpc: Allow server to remain memory resident
This patch enables the XMLRPC server to remain
resident in memory after a task is run, and to accept
a new controlling client. To check the server after
task completion, do

lsof bitbake.lock

in the build directory. Kill the server with kill.

(Bitbake rev: e823e1f0675ff3794eb39ef0b4df2d7a220f4013)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-07 14:13:18 +01:00
Alexandru DAMIAN
748e3c13c8 bitbake: bitbake server: create common server infrastructure
In an attempt to minimize code duplication, create
clear interfaces, and maximize code reuse through OOP,
bb.server adds base classes for the BitBakeServer,
BitBakeServerConnection and actual server implementations
instructed in particular server types.

These classes document the minimum interfaces that the
derived classes must implement, and provide boilerplate code.

Changes to None, Process and XMLRPC servers as to use
the common server infrastructure.

(Bitbake rev: 6db4a64cef20f8d0aba804db4c4e1eec7b112b46)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-07 14:13:18 +01:00
Alexandru DAMIAN
d0861b7a12 bitbake: bitbake: xmlrpc remote server
Added code in XMLRPC server that creates a stub local server
for a client-only connection and is able to connect to
a remote server, and receive events from the remote server.

Added the option to start a client with a remote server in
bitbake.

Original code by Bogdan Marinescu <bogdan.a.marinescu@intel.com>

(Bitbake rev: 25b2af76104d5aaf6435de8c158e0407512f97ce)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-30 10:44:00 +01:00
Alexandru DAMIAN
0fc3a1eddf bitbake: bitbake: xmlrpc transport has identification token
In order to be able to identify different clients over a
stateless XMLRPC connection, we add a custom header named
Bitbake-token, which identifies each client.

We refactor the rest of the code to use the new transport.

Based on a patch by Bogdan Marinescu <bogdan.a.marinescu@intel.com>

(Bitbake rev: a00c2186bffe848a7cedf31969b904f8f7322ae6)

Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-30 10:44:00 +01:00
Richard Purdie
3fb25ef24b bitbake: server/bitbake: Remove launchUI method
With the removal of the none server type, we can remove the launchUI
method and simplify the code slightly.

(Bitbake rev: 9bef2f2dd0bcaa59528ebcb3c1ce053b7dff1ec6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-30 09:54:35 +01:00
Richard Purdie
e118ee3ad8 bitbake: server: Remove none server
The process server backend has been serving well as the default for a long
time now and the UI model is much better thought out that it used to be. With
the move to make bitbake a memory resident process, the none server is now
looking rather pointless and complicates the code needlessly. Lets therefore
now remove it.

(Bitbake rev: 9af03a89605e3db9bce3cea1e0f2d0b6cfaa6fe1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-30 09:54:34 +01:00
Richard Purdie
e5d077d57f bitbake: lib: print is a function in moderm python, covert remaining users
(Bitbake rev: d0b180d868390a464b6799ad90db6bbe0350e158)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09 22:28:24 +01:00
Richard Purdie
bdfc5207a0 bitbake: bitbake: Always use separate process for PR Service
Using the threading module interacts badly with multiprocessing used elsewhere
in bitbake under certain machine loads. This was leading to bitbake hanging on
Ctrl+C when the PR Server was being used.

This patch converts it to always use the daemonize code which
then means the threading code isn't required.

[YOCTO #3742]

(Bitbake rev: 2d0bbd9398ab839bd2d1e29e50b25d52efb1ce2a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-02-06 23:45:36 +00:00
Richard Purdie
ce6199fd98 bitbake: xmlrpc: Unbreak the transport by adding a missing logfile name
I appreciate a fixed value is suboptimal but this is better than not
working at all!

(Bitbake rev: d338a1ad14680af41d218772decd03e98eb2cad5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-12-04 18:02:02 +00:00
Richard Purdie
dee77eca39 bitbake: server/process.py: Change timeout error handling
In normal usage, we never hit the timeout issue. If we do, it becomes obvious
that the current error handling is not good enough. The request may have made it
to the server and the answer will get queued. This means the next command may get
the return value from the previous command with suitably puzzling results.

Without rewriting large sections of code, its not possible to avoid this problem.
It is better to increase the timeout to several seconds giving the server a chance
to respond and if it does timeout, hard exit since recovery is not possible with the
code base today.

I'd be happy to see the structure of this code improved but this quick fix at least
stops corrupted builds from happening which has to be a good thing.

(Bitbake rev: 410c11dd10736873f2dc587fbe9119c38831e693)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-11-21 09:22:22 +00:00
Christopher Larson
c1c20c02a0 bitbake: command: add error to return of runCommand
Currently, command.py can return an error message from runCommand, due to
being unable to run the command, yet few of our UIs (just hob) can handle it
today. This can result in seeing a TypeError with traceback in certain rare
circumstances.

To resolve this, we need a clean way to get errors back from runCommand,
without having to isinstance() the return value. This implements such a thing
by making runCommand also return an error (or None if no error occurred).

As runCommand now has a method of returning errors, we can also alter the
getCmdLineAction bits such that the returned value is just the action, not an
additional message. If a sync command wants to return an error, it raises
CommandError(message), and the message will be passed to the caller
appropriately.

Example Usage:

    result, error = server.runCommand(...)
    if error:
        log.error('Unable to run command: %s' % error)
        return 1

(Bitbake rev: 717831b8315cb3904d9b590e633000bc897e8fb6)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-11-01 11:46:22 +00:00
Richard Purdie
e450b10c4c bitbake: bitbake/server: Remove dead console log code
This code is dead and doesn't do anything so lets remove it.

(Bitbake rev: 8d45739f49618757a5d7d79782deda355e3981ec)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-11-01 11:46:21 +00:00
Dongxiao Xu
c42f973180 bitbake: Add client socket info for BitBakeServerConnection
In server/client split model, the client will bind to a specific address
and port. We need to pass the values to BitBakeServerConnection().

(Bitbake rev: c8e19c5c389efc06696084c6f9439ba75472c5b7)

Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-23 22:52:16 +00:00
Dongxiao Xu
19089aca83 bitbake: add -B option to bind with interface
When start bitbake as a server only process, we need to assign certain
interface to it.

(Bitbake rev: 95b97d2dc6466ea3d99371f5b5bd68f6f3c99074)

Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-23 22:52:16 +00:00
Dongxiao Xu
e69e5a7e0b xmlrpc: Change BitbakeServerInfo init function
Pass host and port to BitbakeServerInfo class instead of the "server"
instance. With this change, GUI can connect with server individually
by host address and port.

(Bitbake rev: 5124351d6c287185723c98e6e4400c5402a5c8a8)

Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-01-06 15:55:56 +00:00
Richard Purdie
7e122178db bitbake/process.py: Ensure queued UI events are queued right before we add our own handler
(Bitbake rev: c7a9ef70ba91d47d53074e8d78cbc52f396144a7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-07 10:57:43 +01:00
Richard Purdie
90dab9783d process.py: Fix issue where early errors weren't making it to the console
(Bitbake rev: d97f7d762e3d2f1b0da038d4d99f2531b2490670)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-16 22:14:01 +01:00
Scott Garman
62d538fbe6 make exception handling syntax consistent
Update exception handling syntax to use the modern style:
except ExcType as localvar

(Bitbake rev: dbf5f42b06bef81749b13aa99945cc1292a6676d)

Signed-off-by: Scott Garman <scott.a.garman@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-15 11:13:13 +01:00
Richard Purdie
5ded108567 bitbake/server/xmlrpc: Fix typo
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 13:11:58 +01:00
Richard Purdie
83c3f872cb bitbake/server/process: Implement getEvent()
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 12:04:23 +01:00
Richard Purdie
658ba779ac bitbake/server/process: Update to new server API
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 11:38:47 +01:00
Richard Purdie
8aabfed148 bitbake: Add process server from upstream bitbake
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 11:38:43 +01:00
Richard Purdie
b34d66225f bitbake none/xmlrpc servers: Only send pickled events to the xmlrpc server
Only the xmlrpc server needs pickled events. Use the function names
to signify this requirement.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 11:38:15 +01:00
Richard Purdie
cd3c4292e7 bitbake: Cleanup bitbake server init process to be clearer to follow
Create a standard format server class instance with method calls
for each step in the server setup. There should be enough hooks
for each of the different server types.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 11:37:26 +01:00
Joshua Lock
7da9f27c37 bitbake/xmlrpc: only use BBTransport for affected Python versions
Upstream have fixed the xmlrpclip.Transport() bug from Python #8194 for
the Python 2.7.2 release, therefore as we know which versions of the
standard library are affected we can only use our copy/paste class when
it's needed.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2011-03-14 20:48:37 +00:00
Paul Eggleton
edd64c7e86 bitbake/server/none: fix getEvent() to return events
In the none server, events don't get processed unless the idle_commands
function gets called, which previously wasn't happening with getEvent();
thus UIs that use this to get events were not working.

Fixes [BUGID #561]

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2011-02-24 15:53:58 +00:00
Paul Eggleton
febb7f2632 bitbake/server/none: remove leftover XMLRPC bits from none server
Remove some comments, imports etc. to do with XMLRPC (inherited from xmlrpc.py
which this file was based upon.)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2011-02-24 14:32:01 +00:00
Richard Purdie
359a4d8bf7 bitbake/runqueue.py: Move SIGCHLD handler to wrap none server sleep call
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-18 11:54:28 +00:00
Chris Larson
26eda93337 Queue up events before the UI is spawned
- Queue up any events fired to the UI before the UI exists
- At exit, check if UIs exist, and if not, flush the queue of LogRecords to
  the console directly.
- When establishing a connection from the UI to the server, flush the queue of
  events to the queue in the server connection, so the UI will receive them
  when it begins its event loop.

(Bitbake rev: 73488aeb317ed306f2ecf99cc9d3708526a5933c)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:37 +00:00
Joshua Lock
b9617bf6e1 bitbake/xmlrpc: Modify xmlrpc server to work with Python 2.7
Python 2.7's library changes some of xmlrpclib's internal implementation such
that interacting with a proxy to BitBakes SimpleXMLRPCServer would cause
BitBake to crash.

The issue was traced to changes in the xmlrpclib.Transport implementation and
Python bug #8194 (http://bugs.python.org/issue8194).

This patch introduces a workaround by create a subclass of
xmlrpclib.Transport, which overrides the offending methods with the Python
2.6.6 implementation copy and pasted from the Python 2.6.6 xmlrpclib, and
using this BBTransport implementation for both xmlrpclib.Server objects we
create.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-07 12:51:59 +00:00
Richard Purdie
d4125bcac0 bitbake: Simplfy pre and post cooker hooks
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-28 13:54:31 -08:00
Richard Purdie
05ba6fc7cb bitbake: Rewrite profiling code so its functional for both none and xmlrpc backends
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-13 21:23:54 +08:00
Richard Purdie
9b8ae6ba45 bitbake/server: Raise a shutdown event if we're seeing exceptions
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-20 09:42:31 +01:00
Chris Larson
1180bab54e Apply some 2to3 transforms that don't cause issues in 2.6
(Bitbake rev: d39ab776e7ceaefc8361150151cf0892dcb70d9c)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:33 +01:00
Chris Larson
ad543e2e41 Apply the 2to3 print function transform
(Bitbake rev: ff2e28d0d9723ccd0e9dd635447b6d889cc9f597)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:33 +01:00
Chris Larson
7acc132cac Formatting cleanups
(Bitbake rev: 2caf134b43a44dad30af4fbe33033b3c58deee57)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:32 +01:00
Richard Purdie
b9d958c186 bitbake: Don't import xmlrpc server unless needed
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-01-22 16:15:50 +00:00
Richard Purdie
22c29d8651 bitbake: Switch to bitbake-dev version (bitbake master upstream)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-01-20 18:46:02 +00:00