Commit Graph

464 Commits

Author SHA1 Message Date
Richard Purdie
0ba9a9fffe bitbake: Overhaul environment handling
Currently, anything whitelisted in the environment makes it into the worker
processes. This is undesireable and the worker environment should be as
clean as possible. This patch adapts bitbake sosme variables are loaded into
bitbake's datastore but not exported by default. Any variable can be exported
by setting its export flag.

Currently, this code only finalises the environment in he worker as doing so
in the server means variables are unavailable in the worker. If we switch
back to fork() calls instead of exec() this code will need revisting.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-09 13:16:14 +00:00
Joshua Lock
c538ef33aa bitbake/cooker: fix idle command processing in servers
idle command processing in each of the servers does not handle an explicit
None return value, which means the goggle UI ends up repeatedly adding
"Tasks Summary:" rows to the list.

This patch modifies BBCooker.buildTargets.buildTargetsIdle to return False
when BuildCompleted is fired, as is done in BBCooker.buildFile.buildFileIdle.
It may be that the correct way to fix this is to change the idle command
processing in the servers.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-12-09 12:41:44 +00:00
Richard Purdie
d4125bcac0 bitbake: Simplfy pre and post cooker hooks
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-28 13:54:31 -08:00
Richard Purdie
05ba6fc7cb bitbake: Rewrite profiling code so its functional for both none and xmlrpc backends
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-11-13 21:23:54 +08:00
Richard Purdie
9747616600 bitbake/siggen: Ensure full signature data is not held unless needed, reducing memory consumption
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-29 10:47:17 +01:00
Richard Purdie
43595fabbe bitbake: Implement signatures
Includes functionality to find out what changes between two different singature data dumps.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-03 16:08:32 +01:00
Richard Purdie
9897b81e56 bitbake/codeparser: Implement persistent cache
For a given input to this code, the output doesn't change to implement a persistent
cache of the data to speed up parsing.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-31 14:49:14 +01:00
Richard Purdie
ca09a6f08c bitbake/cooker.py: Don't init the fetcher in the worker case
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-20 11:26:14 +01:00
Richard Purdie
b6bfe14205 bitbake: Switch to use subprocess for forking tasks and FAKEROOTENV to run shell and python under a fakeroot environment
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-19 22:36:25 +01:00
Richard Purdie
96ec9f8a60 bitbake/cooker.py: Allow idle handlers to pass through a true value
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-19 11:41:44 +01:00
Richard Purdie
5d9f37873d bitbake: Split Runqueue into two classes, a data processor and the execution part
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-18 11:49:58 +01:00
Richard Purdie
36e81a844b bitbake/cooker: Make sure no locks are held before we start forking workers
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-12 14:41:32 +01:00
Richard Purdie
16d299c6ce cooker.py: Fix case of -b option with a full filepath
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-04 16:12:46 +01:00
Richard Purdie
63e6ba8567 bitbake: Add support for .bbappend files (see mailing lists for detais)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-16 15:10:40 +01:00
Richard Purdie
90db9f3074 bitbake: Remove two layers of unnecessary functions from task execution
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-16 12:31:19 +01:00
Richard Purdie
14df571486 bitbake: Fix handling of empty variables in local config files
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:37 +01:00
Chris Larson
04b2a78837 Add a warning if a BBFILE_PATTERN doesn't match any bb files
Likely cause is, of course, typo in the pattern or incorrect BBFILES, so we
should warn the user about this.

(Bitbake rev: b781317b5006bc047a59e7fa3c93344115e78ccb)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:37 +01:00
Chris Larson
ef1de9ecaf Apply some 2to3 refactorings
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:37 +01:00
Chris Larson
b9f8af16f4 Until the interactive mode is fixed, kill it from the valid options, to avoid confusion
(Bitbake rev: 65a51bb20677148a29b9e472f6d9d0a97798f024)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:37 +01:00
Chris Larson
56f083ad13 Switch from our own 'dummywrite' class to StringIO
(Bitbake rev: f5b7e16adf86950d91a88a343031e71beb0f08a6)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:36 +01:00
Chris Larson
39f6ec5f63 cooker: Kill __getitem__ usage on exception objects (python 3 prep)
(Bitbake rev: 1239df8b82ddcd607630da5a888ca83051a22fa8)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:36 +01:00
Bernhard Reutner-Fischer
186f857466 cooker: use time.strftime() instead of spawing sh with date
time.strftime() defaults to using time.localtime(), use that instead
of spawning "sh -c 'date +<FMT>'"

(Bitbake rev: ec1f2773b4811b3beb52710120bb5a9b215c174e)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:35 +01:00
Chris Larson
4802a9d07a Move a var definition in cooker.parse_next
(Bitbake rev: 1b378ddf3dc9b2aa5384e8b28945b63fc623a6a4)

Signed-off-by: Chris Larson <clarson@kergoth.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:34 +01:00
Chris Larson
c3eae29efa Don't try to expand non-string values
(Bitbake rev: fe36a726b9f930bbd6fd758c0aee78559e95f02b)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:34 +01:00
Chris Larson
c04cee7ce8 Search up the directory tree for bblayers.conf
(Bitbake rev: 1c6b31c649474b4c2b63ef9d9311e61de20bc8c2)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:33 +01:00
Chris Larson
3238125138 Add a hack to avoid the req that all vars using LAYERDIR be immediately expanded
See the comment in the code for details.

(Bitbake rev: 849dbd63244cbc4eaca0f1beedbb67baca024629)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:33 +01:00
Chris Larson
dc39ebe91e Move the unset BBPATH sanity check after the bblayers processing
This way we can fully utilize bblayers, you can do everything in bblayers.conf
and avoid setting any environment variables at all.

(Bitbake rev: 5def1c8c31432968349f9b29d6333d7962260a8b)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:33 +01:00
Chris Larson
ad543e2e41 Apply the 2to3 print function transform
(Bitbake rev: ff2e28d0d9723ccd0e9dd635447b6d889cc9f597)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:33 +01:00
Chris Larson
1c74fd768f Import fixups
(Bitbake rev: 4fa052f426e3205ebace713eaa22deddc0420e8a)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:32 +01:00
Chris Larson
7acc132cac Formatting cleanups
(Bitbake rev: 2caf134b43a44dad30af4fbe33033b3c58deee57)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:32 +01:00
Chris Larson
87f62d7c8d Ensure we always utilize the correct messaging domains
(Bitbake rev: 92a2e2e90981c0615171abe03645a772d84f6986)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:31 +01:00
Enrico Scholz
ca9a956abc Made '-b' work with BBCLASSEXTEND
When BBCLASSEXTEND is set, '-b' builds usually failed with messages
like

| ERROR: Parsing error data_fn virtual:native:<recipe>.bb and fn
<recipe>.bb don't match
|
|   File ".../bb/providers.py", line 47, in sortPriorities
|     priority = dataCache.bbfile_priority[f]
| KeyError: 'virtual:native:<recipe>.bb'

This patch fixes it and allows to specify the alternative class in a
way like

| ./bitbake -b virtual:native:<recipe>.bb

This patch was written to be so minimal as possible; variables should be
probably renamed to reflect their new meaning.

(Bitbake rev: f1c7fe9fc12161ceb3fe201cde370b929b208729)

Signed-off-by: Enrico Scholz <enrico.scholz@informatik.tu-chemnitz.de>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:31 +01:00
Jeff Dike
f9d7e9e3be bitbake: Move the logger initialization from cooker to utils
In order to move the environment cleaning, which wants to log a
message, before cooker creation, the logging facility initialization
needs to happen earlier.  So, it is now in init_logger in utils.py and
called from bitbake before the creation of the data object.

It also seems more correct to initialize a global facility like this
from a more global context than the creation of an object, of which
there could theoretically be many.

Signed-off-by: Jeff Dike <jdike@linux.intel.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-06-29 13:32:04 +01:00
Richard Purdie
871f731e57 bitbake/cooker.py: Finishing the command needs to happen after the BuildCompleted event else the cooker can shutdown first
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-25 17:31:23 +00:00
Chris Larson
19d5f2b067 BBFILES: use a set to remove duplicates when collecting.
(Bitbake rev: b1b06133da4ca379a60775552d481f7fbf77e999)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-25 17:26:02 +00:00
Richard Purdie
666c753d4b bitbake/cooker.py: Add support for a bblayers.conf file
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-22 17:48:31 +00:00
Bernhard Reutner-Fischer
04263dc426 cooker: bail out early if no files to build
This prevents a misleading backtrace:
ERROR: no files to build.
Command execution failed: Traceback (most recent call last):
  File ".../bitbake/build/lib/bb/command.py", line 83, in runAsyncCommand
    self.cooker.updateCache()
  File ".../bitbake/build/lib/bb/cooker.py", line 779, in updateCache
    if not self.parser.parse_next():
  File ".../bitbake/build/lib/bb/cooker.py", line 969, in parse_next
    cooker.bb_cache.sync()
UnboundLocalError: local variable 'cooker' referenced before assignment

(Bitbake rev: 060ef3d957615b7eb1209dc0d01ebeb53f8c4edc)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Chris Larson <clarson@kergoth.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-22 15:03:46 +00:00
Chris Larson
ada2a8494a Avoid unnecessary calls to keys() when iterating over dictionaries.
dict objects provide an __iter__ method for the iteration which gives you the
keys, so calling keys directly is unnecessary, and isn't really a best
practice.  The only time you really need to call the keys is if there's a
danger of the dict changing out from underneith you, either due to external
forces or due to modification of the iterable in the loop.  Iterations over
os.environ are apparently subject to such changes, so they must continue to
use keys().

As an aside, also switches a couple spots to using sorted() rather than
creating a temporary list with keys() and sorting that.

(Bitbake rev: 5b6ccb16c6e71e23dac6920cd2df994d67c2587b)

Signed-off-by: Chris Larson <clarson@mvista.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-22 15:01:59 +00:00
Richard Purdie
7e8025459e bitbake/cooker: Drop unneeded import
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-01-22 00:04:18 +00:00
Richard Purdie
22c29d8651 bitbake: Switch to bitbake-dev version (bitbake master upstream)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-01-20 18:46:02 +00:00
Richard Purdie
cb8684f9bc bitbake/cooker.py: Fix handling of extra configuration files
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-12-11 12:48:49 +00:00
Richard Purdie
483f85802a bitbake: Sync various functions with those from bitbake-dev and bitbake upstream
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-11-13 12:15:21 +00:00
Richard Purdie
64b04685b6 bitbake: Add a --revisions-changed commandline option to indicate when floating srcrevs have changed
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-07-29 14:33:14 +01:00
Richard Purdie
81ff3a90f8 bitbake: Update to work without warnings with python 2.6
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-05-12 16:53:22 +01:00
Richard Purdie
107a9da006 bitbake: Merge further fixes from upstream 1.8 branch
* Make the test functionality work
* Optimise BBPATH handling when changing directory
* Optimise file globing for BBFILES

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-05-11 22:41:17 +01:00
Richard Purdie
ade351e2f4 bitbake: Add in code to support the BBCLASSEXTEND variable. Virtual native/sdk recipes then become possible 2009-01-03 16:25:20 +00:00
Richard Purdie
b6b37f065c bitbake: Improve error message for missing class files 2008-11-06 00:12:31 +00:00
Richard Purdie
b296ae263c bitbake: Add tryaltconfigs option to disable the alternative configuration attempts and make the 'continue' more aggresive 2008-10-28 22:15:06 +00:00
Robert Bradford
696d4f01d2 bitbake: Refactor the environment variable handling
Refactor the environment variable handling so that it more closes matches the
changes that have been made in bitbake-dev.

git-svn-id: https://svn.o-hand.com/repos/poky/trunk@5527 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-10-17 09:46:35 +00:00
Richard Purdie
221ac2b25f bitbake/bitbake-dev: Sync with upstream
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@5346 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-09-30 20:46:17 +00:00
Richard Purdie
f1e89b2bb9 bitbake: Sync with upstream
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@4504 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-05-19 20:41:58 +00:00
Richard Purdie
853280f161 bitbake: Update with changes from bitbake 1.8 branch
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@4006 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-03-14 11:44:34 +00:00
Richard Purdie
ab191d21e2 bitbake: Update to bitbake 1.8 branch head
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@3892 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-03-03 22:01:45 +00:00
Richard Purdie
bd837ac673 cooker.py: Fix -g option breakage from previous commits
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@3415 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-01-06 23:06:42 +00:00
Richard Purdie
7821f2250d bitbake: Sync with bitbake upstream for various fixes
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@3411 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-01-06 16:51:51 +00:00
Richard Purdie
e223238b1b bitbake: Update to latest bitbake-1.8 branch
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2651 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-09-02 14:10:08 +00:00
Marcin Juszkiewicz
a75b8d8b36 bitbake: use Psyco only on x86 (no support for other archs)
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2541 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-23 07:36:58 +00:00
Richard Purdie
d8bfa5c6ef bitbake: Sync with 1.8.8 release
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2513 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-20 07:48:43 +00:00
Richard Purdie
ce800d3aea bitbake: Sync with upstream 1.8 branch for fixes
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2484 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-12 23:06:49 +00:00
Richard Purdie
bfc70eb24e bitbake: Update along 1.8 branch
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2345 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-03 13:40:52 +00:00
Richard Purdie
1ad7c7b546 bitbake: Exit if parsing errors occur
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@1832 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-05-31 11:31:44 +00:00
Richard Purdie
d05bcee7f2 bitbake: Merge bugfixes from bitbake-1.8 svn
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@1759 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-05-22 11:50:37 +00:00
Richard Purdie
7371e6323c bitbake: Update to 1.8.1 (inc. various bug fixes, epoch support)
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@1419 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-04-01 15:04:49 +00:00
Richard Purdie
f5665d5bfc bitbake: Sync with upstream.
* File licence headers were sanitised causing most of the diff. 
 * cooker.py was created from bin/bitbake. 
 * cvs fetcher port option was added
 * The -f force option was fixed to work correctly
 * Multiple entries in rrecrdeps are now handled correctly
   (allows adding do_deploy to image depends)
 


git-svn-id: https://svn.o-hand.com/repos/poky/trunk@1129 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-01-08 23:53:01 +00:00