The rysnc of 20,000 files (650MB) onto the nas is slow taking ~3 minutes
at idle and worse at load. This is due to the number of files which
is a pain point for NFS. This piece of the build is also a bottleneck
since the rest of a build depends on it happening.
If we switch to zstd compressed tar, it takes 2.49s. Other compression
methods were much slower but zstd seems 'accptable' and speeds things
up too.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous fixes requires the user to set "no-layer-add"
for a repo and then use ADDLAYER to insert the sub-repos
(e.g., meta-openmbedded/meta-oe) as a two-part process.
This means that you would also have to specify that flag
if a repo that is a layer with dependencies is in the
list so that it can be inserted in the correct order later
via ADDLAYER to avoid parsing problems. This fix allows
for specifying a NEEDREPOS with the subdirectory of the
target layer (e.g., meta-openembedded/meta-oe) so that
there is no need for the "no-layer-add" followed by
ADDLAYER combination. The entire meta-openembedded
repo would be moved into place, and the sublayer added
to bblayers.conf.
Signed-off-by: Thomas Goodwin <btgoodwin@geontech.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The preceeding errors that would be shown should be enough to
identify the problem since it would have been a problem
running bitbake-layers add-layer, indicating a misconfiguration
of config.json.
Signed-off-by: Thomas Goodwin <btgoodwin@geontech.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[RP: Minor tweaks made to adpat to buildbot and option naming
Fixed shared-repo-unpack for publishing when no cache-dir supplied]
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Usage is documented in README and an example, local-example.json is
included.
Also clean up the parameter to loadconfig() as its actually not needed.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We want to be able to use data expansion/variable substitution
in most variables so convert to use the utils API for this.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As the mv commands use a filename wildcard we need to tell check_call to
execute through the shell so that the wildcard is expanded, rather than
treat as a file name.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
