Commit Graph

32 Commits

Author SHA1 Message Date
Christopher Larson
f7c0488bd5 persist_data: survive if sqlite3 couldn't enable the shared cache
(Bitbake rev: e870fd57bb84a17a871a5b5de3bf5202fe9b135f)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-02-22 20:25:29 +00:00
Lianhao Lu
1fedd166b7 bitbake/persist_data: Reconnect when DB is locked
[YOCTO #1761]
Reconnect to the backend Sqlite DB in 'database is locked' exception so
the timeout can be leveraged in each time retry.

(Bitbake rev: b310382764367b573c84f33d847c6eb821266f9e)

Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-01-19 00:15:38 +00:00
Richard Purdie
4cd9671078 bitbake: Update users of getVar/setVar to use the data store functions directly
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-11-27 10:35:30 +00:00
Richard Purdie
f1c0c97f4d bitbake/persist_data: Sync file with upstream bitbake
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-05-06 18:07:23 +01:00
Chris Larson
754d1c6983 persist_data: implement comparison, same as dict
(Bitbake rev: 1190406c526c7bb7cf415867be83e0403812a7dd)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-05-06 18:05:11 +01:00
Chris Larson
824acff967 persist_data: don't allow non-string keys/values
(Bitbake rev: 28958cd55e592853c68f5f2ba79381d1b8dcfb8f)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-05-06 15:50:29 +01:00
Chris Larson
d43e7a91f2 persist_data: add has_key
(Bitbake rev: 7942833ca0685cf4f3b243dde6203499ef97420c)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-05-06 15:50:14 +01:00
Chris Larson
7fc020aa15 persist_data: raise KeyError on missing elements
(Bitbake rev: a4f62433845c29f98c6a9746d5d2847bf9506ea5)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-05-06 15:49:17 +01:00
Chris Larson
88dbb0523c Initial work on getting bitbake working under pypy
- use os.chmod, not os.fchmod, as the latter is missing under pypy
- rearrange our imports a bit
- don't die if sqlite3 is missing shared cache support

(Bitbake rev: f229824dc9c453adf6067500e2bf6761536e4f2f)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-05-06 15:42:03 +01:00
Chris Larson
c68c5f6e31 persist_data: kill unreachable break line
(Bitbake rev: 7486b38603f2766adaf976a9f95e9276c83abe31)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-10 22:35:12 +00:00
Chris Larson
1385162cd2 persist_data: loop on database lock for table creation
(Bitbake rev: d93fcbd64ab5d806288424170f55323b4297e7d6)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-10 22:32:12 +00:00
Chris Larson
478677fb62 persist_data: resurrect the lock wait for selects
Think this got inadvertantly dropped when switching to the new API.

(Bitbake rev: 628c5159d1151b89f2b7210c8819489e8dc9a84d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-05 00:58:23 +00:00
Chris Larson
2671bb4197 Rework the persist_data API
Rather than having to run .addDomain() and then .getValue(domain, key),
.setValue(domain, key), etc, now it just works as mappings.

As an example:

    setValue(domain, key) -> persist[domain][key] = value

It also arranges things so we can construct objects of this type using any
arbitrary filename/path for the sqlite3 database, rather than being so
tightly bound to the metadata.

(Bitbake rev: d9e8b8af308ae871efdc8ef0782be30af8c1f894)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-05 00:58:23 +00:00
Chris Larson
3069c0b258 Revert "persist_data: cache connection and use cursor"
Caching the database connection can cause serious issues if it results in
multiple processes (e.g. multiple tasks) simultaneously using the same
connection.

This reverts commit 8a6876752b90efd81d92f0947bfc9527d8260969.

(Bitbake rev: 60b9b18eafad5ac46c7cf1048d749d673c2ee0ad)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-05 00:58:23 +00:00
Chris Larson
3a2176fe73 persist_data: handle locked db for SELECT
Parallel processes interacting with the persist_data db can quite easily
explode without this.

(Bitbake rev: b3d5432cff0ff28f4c8a5bcf10efa3e383b4fd4d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:39 +00:00
Chris Larson
ecc68fa4fb Switch bitbake internals to use logging directly rather than bb.msg
We use a custom Logger subclass for our loggers

This logger provides:
- 'debug' method which accepts a debug level
- 'plain' method which bypasses log formatting
- 'verbose' method which is more detail than info, but less than debug

(Bitbake rev: 3b2c1fe5ca56daebb24073a9dd45723d3efd2a8d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:33 +00:00
Richard Purdie
fac06aaf2a bitbake/persist_data: Attempt to fix locking issues
It appears the timeout sometimes has no effect and we see database access failures. Combat
this by wrapping the execute function in all cases and retrying manually ourselves.

Thanks to Kevin Tian for help debugging this.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-20 11:24:02 +01:00
Bernhard Reutner-Fischer
726802b980 persist_data: cache connection and use cursor
Store database connection to persistent database in fetcher.

(Bitbake rev: 8a6876752b90efd81d92f0947bfc9527d8260969)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:35 +01:00
Chris Larson
1180bab54e Apply some 2to3 transforms that don't cause issues in 2.6
(Bitbake rev: d39ab776e7ceaefc8361150151cf0892dcb70d9c)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:33 +01:00
Chris Larson
bcd9c57a96 Switch some references to moved functions
(Bitbake rev: ddea2978cb969cf1381cfee5c055de0c9e56a4e3)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:33 +01:00
Chris Larson
7acc132cac Formatting cleanups
(Bitbake rev: 2caf134b43a44dad30af4fbe33033b3c58deee57)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:32 +01:00
Richard Purdie
433c2d2318 bitbake: Remove persist_data domain renaming code as it appears to expose sqlite bugs and data consistency issues
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-07-30 21:24:31 +01:00
Richard Purdie
ef9c095334 bitbake: Make sure tables exist when renaming
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-07-30 18:03:51 +01:00
Richard Purdie
231b5f6784 bitbake: Add renameDomain and getKeyValues functions to the persistent data code
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-07-29 14:10:11 +01:00
Richard Purdie
e14d7dcbee bitbake: Sync with 1.8 branch
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@4463 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-05-13 07:53:18 +00:00
Richard Purdie
05c49eaac2 bitbake: Update to 1.8.10 release
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@3555 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-01-20 14:58:00 +00:00
Richard Purdie
6883189d5f bitbake persist_data: Fix for python 2.4
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2366 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-04 23:58:56 +00:00
Richard Purdie
ce4ba3c2b7 bitbake persist_data: Fix for python 2.4
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2365 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-04 23:54:31 +00:00
Richard Purdie
e33eaf13d5 bitbake persist_data: Fix module reference
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2364 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-04 22:45:37 +00:00
Richard Purdie
71068e7cee bitbake persist_data: Add own retry logic
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2363 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-04 22:12:42 +00:00
Richard Purdie
9858ddb9c5 bitbake persist_data: Use immediate connection mode
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2362 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-04 21:15:47 +00:00
Richard Purdie
bfc70eb24e bitbake: Update along 1.8 branch
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2345 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-03 13:40:52 +00:00