layerindex-web/layerindex
Robert Yang 3ae517091b update.py: update layers in dependency order
* Problems
The update.py couldn't handle new (not only new branch, in fact, but also
existing branches, see below for more info) branch well, for example, there are
3 layers: layer_A, layer_B and layer_C, and create new branch "branch_1" for
them, and they have depends:

layer_A -> layer_B -> layer_C

The "->" means depends on.

Then run "update.py -b branch_1", there would be errors like:

ERROR: Dependency layer_B of layer_A does not have branch record for branch branch_1

Though update.py runs "update_layer.py" twice, but it didn't help since
layerbranch was None when it was failed to create in the first run.

The reason is if update.py updates layer_A firstly, it would fail since it
can't find layer_B:branch_1 in database (not added to database yet), similarly,
if add layer_B before layer_C, it would also fail. Only layer_C can be added
(assume it has no dependencies). So we have to re-run update.py again and again
to make it work, here we may have to run update.py 3 times, and more runs are
needed if the dependency chain is longer.

* Solutions:
Make update.py pass layers orderly to update_layer.py according to dependencies
can fix the problem, we can get intial dependencies from tinfoil, add an option
"-i, --initial" to update_layer.py to get them.

Not only new branch, but also existing branches may have the problem, because
collections and dependencies maybe changed in the coming update, so we can't
trust database when the layer is going to be updated, for example, if there are
10 layers in database, and 3 of them will be updated (-l layer1,layer2,layer3),
then we can not use the 3 layers' collections data from database, we need get
them from tinfoil.

* Performance improvement:
  It should be the same as before in theory, I have tested it with 97 layers:
  - Before: 11m6.472s, but only 75 layers were added, 22 ones were failed, I
    have to re-run update.py again and again (maybe 4 times to make all of
    them added). So:
    (11 * 60 + 6)/75*97/60 = 14m35s

  - Now 12m10.350s, all the layers are added in the first run.

   So about 2 minutes are saved.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
2017-07-21 08:15:53 +02:00
..
migrations layerindex: add "No update" status to LayerItem 2017-03-14 15:19:53 +13:00
static editlayer: allow selecting web interface type 2016-10-25 20:33:42 +13:00
templatetags Support (and require) Python 3 2016-06-12 11:33:08 +12:00
tools layerconfparse.py: remove unused layerbranch from parse_layer() 2017-07-21 08:14:57 +02:00
__init__.py Initial commit of layerindex-web 2013-02-13 12:09:57 +00:00
admin.py Record and display update logs 2016-11-16 15:31:46 +13:00
bulkchange.py layerindex: Update tinfoil to call shutdown method 2017-01-11 21:56:35 +13:00
context_processors.py Use Django sites framework to specify website title 2013-12-28 11:03:29 +00:00
forms.py layerindex: add "No update" status to LayerItem 2017-03-14 15:19:53 +13:00
layerconfparse.py layerconfparse.py: remove unused layerbranch from parse_layer() 2017-07-21 08:14:57 +02:00
models.py layerindex: add "No update" status to LayerItem 2017-03-14 15:19:53 +13:00
querysethelper.py Use functools.reduce instead of reduce 2016-07-04 09:51:21 +12:00
recipedesc.py layerindex: Update tinfoil to call shutdown method 2017-01-11 21:56:35 +13:00
recipeparse.py recipeparse.py: restore cwd when the parsing is done 2017-07-21 08:15:19 +02:00
restperm.py expose REST API for layerindex 2014-09-24 11:32:34 +01:00
restviews.py layerindex: add "No update" status to LayerItem 2017-03-14 15:19:53 +13:00
simplesearch.py Improve recipe search 2013-02-19 17:22:45 +00:00
update_layer.py update.py: update layers in dependency order 2017-07-21 08:15:53 +02:00
update.py update.py: update layers in dependency order 2017-07-21 08:15:53 +02:00
urls_branch.py Record and display update logs 2016-11-16 15:31:46 +13:00
urls.py Record and display update logs 2016-11-16 15:31:46 +13:00
utils.py update.py: update layers in dependency order 2017-07-21 08:15:53 +02:00
views.py layerindex/views: support querying by layer name 2017-06-15 16:04:14 +02:00