Commit Graph

311 Commits

Author SHA1 Message Date
Paul Eggleton
eda77b320d devtool: upgrade: fix accidentally swapped parameters
It appears that when fixing the signature unlocking in OE-Core commit
4e9a0be32fc30fb87d65da7cd1a4015c99533aff I swapped the parameters here
and did not test it within the eSDK (it does nothing outside of the
eSDK) resulting in a TypeError when devtool upgrade was used in the
eSDK. Swap the parameters around to the correct ordering.

Fixes [YOCTO #12285].

(From OE-Core rev: 05e2c4ada7083f40866846a21fe76c852f1dfefe)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-11-11 12:14:26 +00:00
Peter Kjellerstedt
ede79dea7c devtool: standard: Expand SRCREV before using it in _update_recipe_srcrev
If SRCREV contains a variable reference, any devtool command that
would try to update it would fail. E.g., if SRCREV = "R${PV}", then
devtool finish without having committed any changes would fail with:

  oe.patch.CmdError: Command Error: 'sh -c 'git format-patch R${PV} -o
  /tmp/oepatchb_doareb -- .'' exited with 0 Output:
  fatal: bad revision 'R'

(From OE-Core rev: 094499c819722ad698ccb64ec65dd439b211c31c)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-10-16 23:52:44 +01:00
Ming Liu
5c67faff20 scripts: drop True option to getVar calls
Search made with the following regex: getVar ?\((.*), True\).

(From OE-Core rev: b848c3cb495905605283c57c79f2ed8ca17758db)

Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-26 11:05:01 +01:00
Paul Eggleton
3fde63363a devtool: ensure recipes devtool is working on are unlocked within the eSDK
Alongside reworking the way devtool extracts source, we now need to
ensure that within the extensible SDK where task signatures are locked,
the signatures of the tasks for the recipes being worked on get unlocked
at the right time or otherwise we'll now get taskhash mismatches when
running devtool modify on a recipe that was included in the eSDK such as
the kernel (due to a separate bug). The existing mechanism for
auto-unlocking recipes was a little weak and was happening too late, so
I've reimplemented it so that:
(a) it gets triggered immediately when the recipe/append is created
(b) we avoid writing to the unlocked signatures file unnecessarily
    (since it's a global configuration file) and
(c) within the eSDK configuration we whitelist SIGGEN_UNLOCKED_RECIPES
    to avoid unnecessary reparses every time we perform one of the
    devtool operations that does need to change this list.

Fixes [YOCTO #11883] (not the underlying cause, but this manifestation
of the issue).

(From OE-Core rev: 4e9a0be32fc30fb87d65da7cd1a4015c99533aff)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-18 11:07:29 +01:00
Paul Eggleton
10af6d86b3 devtool: rework source extraction so that dependencies are handled
Since it was first implemented, devtool's source extraction (as used by
the devtool modify, extract and upgrade subcommands) ignored other recipe
dependencies - so for example if you ran devtool modify on a recipe that
fetches from svn or is compressed using xz then it would fail if those
dependencies hadn't been built first. Now that we can execute tasks in
the normal way (i.e. tinfoil.build_targets()) then we can rework it to
use that. This is slightly tricky in that the source extraction needs to
insert some logic in between tasks; luckily we can use a helper class
that conditionally adds prefuncs to make that possible.

Some side-effects / aspects of this change worth noting:
* Operations are a little slower because we have to go through the task
  dependency graph generation and other startup processing. There's not
  really any way to avoid this though.
* devtool extract didn't used to require a workspace, now it does
  because it needs to create a temporary bbappend for the recipe. (As
  with other commands the workspace be created on the fly if it doesn't
  already exist.)
* I want any existing sysroot files and stamps to be left alone during
  extraction since we are running the tasks off to the side, and
  especially devtool extract should be able to be used without touching
  these. However, this was hampered by the automatic removal process in
  sstate.bbclass triggered by bb.event.ReachableStamps when the task
  signatures change, thus I had to introduce a way to disable this
  removal on a per-recipe basis (we still want it to function for any
  dependencies that we aren't working on). To implement this I elected
  to use a file written to tmp/sstate-control which gets deleted
  automatically after reading so that there's less chance of stale files
  affecting future sessions. I could have used a variable but this would
  have needed to be whitelisted and I'd have to have poked its value in
  using the setVariable command.

Fixes [YOCTO #11198].

(From OE-Core rev: 830dbd66992cbb9e731b48d56fddf8f220349666)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-18 11:07:29 +01:00
Juan M Cruz Alcaraz
688e0894aa devtool/standard: set a preferred provider when adding a new recipe with devtool
A recipe added with "devtool add" requires to be able to take precedence on recipes
previously defined with PREFERRED_PROVIDER.

By adding the parameter "--provides" to "devtool add" it is possible to specify
an element to be provided by the recipe. A devtool recipe can override a previous
PREFERRED_PROVIDER using the layer configuration file in the workspace.

E.g.
    devtool add my-libgl git@git://my-libgl-repository --provides virtual/libgl

[YOCTO #10415]

(From OE-Core rev: adeea2fe6895898a5e6006e798898f0f5dabd890)

Signed-off-by: Juan M Cruz Alcaraz <juan.m.cruz.alcaraz@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-13 22:07:42 +01:00
Paul Eggleton
6426136655 devtool: upgrade: check that user has configured git properly
If user.name or user.email haven't been set then git rebase can't really
work properly. Check that the user has set these and error out if not.
(Elsewhere we are relying on OE's git patch functionality which forces
a dummy OE value - that's OK there as it's completely under OE's control
and therefore it's OK for a dummy OE user to be the committer, but here
the rebase may require intervention so it's reasonable to have the
user's actual name and email on the operation.)

Fixes [YOCTO #11947].

(From OE-Core rev: 129a3be07e272013be2db17552c13b4d8cc2cf6e)

(From OE-Core rev: 802829f1c38d8c5eee11ba1d9ddd37cf02597f6e)

Signed-off-by: paul <paul@peggleto-mobl.ger.corp.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-11 17:30:30 +01:00
Ola x Nilsson
9bf6b1127a devtool: status: Sort entries before printing
Sorted entries are easier to read.

(From OE-Core rev: d0a123ec564f6d36977e472f8bc63f9c050ee616)

Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-09-11 17:30:28 +01:00
Paul Eggleton
e97cd0017b devtool: add: add explicit srcrev/branch options
At the moment when fetching source from a git repository you have to
know that you can specify the revision and branch in the URL with
';rev=' and ';branch=' respectively, and you can also get thrown off by
the shell splitting on the ; character if you forget to surround the URL
in quotes. Add explicit -S/--srcrev and -B/--srcbranch options
(consistent with devtool upgrade) to make this easier for the user to
discover and use. (The rev and branch URL parameters will continue to
work, however.)

(From OE-Core rev: 2d86cac853d6daa496c0315a5cb0662ebf1165b0)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31 23:30:02 +01:00
Paul Eggleton
dd5ceaefc0 devtool: edit-recipe: fix regression
OE-Core commit 5a16b3c804c5eca331a1c08a7ce31a54909af105 attempted to use
the same function to get the path to a recipe as the new "find-recipe"
command it implemented, except that cannot work because (a) it didn't
return anything and (b) event if it had tried, a command function can
only return an exit code and we don't want that for find-recipe if it
succeeded. Split out a separate reusable function for both commands.

(From OE-Core rev: d5191840212adbf480961ba6fc68e1ab17e5a77a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31 23:30:02 +01:00
Paul Eggleton
69d50eb9ec devtool: upgrade: workaround for recipes which apply patches conditional upon class
If we're upgrading a recipe that appends additional patches for, say,
class-native, and we're just upgrading the target variant, then when we
copied the recipe into the workspace we skipped copying the additional patches
for the native variant. This caused warnings because the workspace
recipe is preferred. Look at SRC_URI for all variants when copying files
to work around this.

More work is needed to make it easier to work with recipes that use
BBCLASSEXTEND where you need to build more than one variant at once, but
this at least fixes the immediate ugliness.

(From OE-Core rev: 56bf5e93358187e31160d7893f57906bb3dc7ad7)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31 23:30:02 +01:00
Paul Eggleton
34580ac287 devtool: upgrade: fix handling of non-absolute paths
If your BBLAYERS has non-absolute paths in it (e.g.
"${COREBASE}/../something") then none of the paths matched in
copy_recipe_files() with the result that no files got copied and you
ended up with an error later on because the recipe file couldn't be
found at the destination. Fix this as well as adding an explicit check
to see if no files got copied - error out earlier if so.

Fixes [YOCTO #10981].

(From OE-Core rev: 3861486ad06f90c8644ebab119bbc5ddb9e693ca)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31 23:30:02 +01:00
Paul Eggleton
24cfac5263 devtool: update-recipe: ensure patches get deleted in srcrev mode
Patches that we identify as having been "deleted" (i.e. patches in
SRC_URI that no longer appear in the git tree) need to be dropped even
if we're updating in srcrev mode. This fixes the case where HEAD of the
git tree is valid upstream (i.e. no extra commits), but there are
patches left over in the recipe, e.g. when we do devtool upgrade and
then all of the commits rebased on top of the new branch get skipped.

Fixes [YOCTO #11972].

(From OE-Core rev: 350f83dc1e317aeb93539f13966caca6d894f569)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31 23:30:02 +01:00
Tobias Hagelborn
cd26fc143b devtool: deploy-target: Support stripped libs and execs
New devtool deploy-target option --strip which enables deploying
stripped binaries, saving some space on target.

* Copies the files of ${D} into a new directory and strips them in place
* Used oe.package.strip_execs for stripping directory
* Added devtool.conf option "strip" for changing default behavior

Config example:
[Deploy]
strip = true

[YOCTO #11227]

(From OE-Core rev: 7f10c5118793da6ded59ae6e60e796152dbd7ca3)

Signed-off-by: Tobias Hagelborn <tobiasha@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-27 22:32:00 +01:00
Leonardo Sandoval
e798b4e980 devtool: import: new plugin to import the devtool workspace
Takes a tar archive created by 'devtool export' and imports (untars) it
into the workspace. Currently the whole tar archive is imported, there
is no way to limit what is imported.

https://bugzilla.yoctoproject.org/show_bug.cgi?id=10510

[YOCTO #10510]

(From OE-Core rev: 2de8ba89ef10fefcc97246dfeb4b8d1e48ee8232)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-23 08:47:02 +01:00
Leonardo Sandoval
ee21e81cff devtool: append md5sum only if not already present
In case the proposed md5sum to be appended to the .devtool_md5 file
is already present, do not append it.

(From OE-Core rev: f958c5cba3b0d24ca696b2b707857009c9a7b5b8)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-23 08:47:02 +01:00
Leonardo Sandoval
42a5894a3c devtool: export: new plugin to export the devtool workspace
By default, exports the whole workspace (all recipes) including the source code.
User can also limit what is exported with --included/--excluded flags. As
a result of this operation, a tar archive containing only workspace metadata
and its corresponding source code is created, which can be properly imported
with 'devtool import'.

https://bugzilla.yoctoproject.org/show_bug.cgi?id=10510

[YOCTO #10510]

(From OE-Core rev: f9bc3b5101b554a72298266519dbdd1497f262a6)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-23 08:47:02 +01:00
Chang Rebecca Swee Fun
b47a7b848b devtool: upgrade: enable branch checking when revision is provided
When devtool upgrade is run on a recipe with revision specified
that is not on master branch, and branch isn't set by --srcbranch or -B,
then we should get the correct branch and append the branch to the URL.

If the revision was found on multiple branches, we will display error
to inform user to provide a correct branch and exit.

[YOCTO #11484]

(From OE-Core rev: 29ced7387a92aed17b7fe93b1654790a981734c1)

Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-23 08:47:02 +01:00
Andrea Galbusera
6346d12d8d devtool: sdk-update: fix pulling updates from git
Commit 4657bc9d165e51981e034e73e7b92552e873eef7 replaced the git pull logic with
the git fetch + git reset --hard combo, but resetting to HEAD does not really
pull in new commits from remote... Replace with resetting to the upstream branch
instead.

(From OE-Core rev: 0dcdb146f59a184419bffd4f24cdf8343a43c0ea)

Signed-off-by: Andrea Galbusera <gizero@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-09 09:24:16 +01:00
Paul Eggleton
1cce7f885c devtool: fix handling of errors during task execution
* If an error is logged while executing a task, we need to ensure we
  exit instead of assuming everything went OK.
* If we receive CookerExit, the server is shutting down and we need to
  stop waiting for events and probably exit (knotty does this). This
  will occur if an exception or bb.fatal() happens during an event
  handler.

This fixes a couple of issues highlighted when using devtool upgrade or
modify on a non-supported recipe with intel-iot-refkit together with
bitbake master, but I'd be very surprised if it were hard to reproduce
in other scenarios.

(From OE-Core rev: 65e644368fc9c294af96906528ee0cf30305e0a6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-09 09:24:15 +01:00
Chang Rebecca Swee Fun
c2318ef416 devtool: standard: enable options for PREMIRRORS and MIRRORS
Since we have provide an option to manually enable PREMIRRORS and MIRRORS
in recipetool, we need to make sure devtool is having the same options
as devtool uses recipetool in creating new recipes.

(From OE-Core rev: 198bddeb928a318c3ad168bcb4f83b5cd9a604fb)

Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21 22:51:37 +01:00
Chang Rebecca Swee Fun
819179d775 devtool: find-recipe: enable new subcommand for devtool
devtool find-recipe will prints out the path to the recipe
in a workspace.

This subcommand can also help to find recipe outside of
current workspace using "-a" or "--any-recipe" option.

This enhancement helps developer to get the recipe path
when working with devtool.

[YOCTO #11434]

(From OE-Core rev: 5a16b3c804c5eca331a1c08a7ce31a54909af105)

Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21 22:51:37 +01:00
Chang Rebecca Swee Fun
5778e35378 scriptutils: pass in logger as parameter
logger was not defined in scriptutils.py based on the
observation in python traceback.

Traceback (most recent call last):
  File "/workdir/poky/scripts/devtool", line 351, in <module>
    ret = main()
  File "/workdir/poky/scripts/devtool", line 338, in main
    ret = args.func(args, config, basepath, workspace)
  File "/workdir/poky/scripts/lib/devtool/utilcmds.py", line 55, in
edit_recipe
    return scriptutils.run_editor(find_recipe(args, config, basepath,
workspace))
  File "/workdir/poky/scripts/lib/scriptutils.py", line 141, in
run_editor
    logger.error("Execution of '%s' failed: %s" % (editor, exc))
NameError: name 'logger' is not defined

We pass in logger as parameter to run_editor() from where it has
been called (devtool/utilcmds.py and recipetool/newappend.py),
which both modules already has logger setup.

(From OE-Core rev: 21f04b61973dd9029f0e6bff5445e31cd762bf32)

Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21 22:51:37 +01:00
Richard Purdie
821cb0b819 devtool: Setup logging before calling prepare
This avoids test failures like:

INFO - ======================================================================
INFO - FAIL [1.755s]: test_devtool_layer_plugins (devtool.DevtoolTests)
INFO - ----------------------------------------------------------------------
INFO - Traceback (most recent call last):
  File "/media/build1/poky/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f
    return func(*args, **kwargs)
  File "/media/build1/poky/meta/lib/oeqa/selftest/cases/devtool.py", line 1354, in test_devtool_layer_plugins
    self.assertEqual(result.output, s[::-1])
AssertionError: "NOTE: Starting bitbake server...\noY senu[36 chars]rciM" != "oY senuZ s'enoynA morF tiforP oN edaM tfosorciM"
- NOTE: Starting bitbake server...
  oY senuZ s'enoynA morF tiforP oN edaM tfosorciM
INFO - ----------------------------------------------------------------------

since there is corruption in the output. Setting the logging up before
calling tinfoil.prepare() resolves this.

(From OE-Core rev: 3c479fb17ae4d3e7e5f0889af0f68257ef66475c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21 08:44:25 +01:00
Paul Eggleton
0d72748e81 recipetool: create: refactor code for ensuring npm is available
Across devtool and recipetool we had an ugly set of code for ensuring
that we can call an npm binary, and much of that ugliness was a result
of not being able to run build tasks when tinfoil was active - if
recipetool found that npm was required and we didn't know beforehand
(e.g. we're fetching from a plain git repository as opposed to an npm://
URL where it's obvious) then it had to exit and return a special result
code, so that devtool knew it needed to build nodejs-native and then
call recipetool again. Now that we are using real build tasks to fetch
and unpack, we can drop most of this and move the code to the one place
where it's still needed (i.e. create_npm where we potentially have to
deal with node.js code in a plain source repository).

(From OE-Core rev: 8450de16ddb02d863204b411a94c6d84e0f88817)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21 08:44:25 +01:00
Paul Eggleton
e4346e8be5 recipetool: create: reimplement fetching with normal fetch/unpack tasks
Now that we have the ability to run the tasks in a more standard context
through tinfoil, change recipetool's fetching code to use that to fetch
files using it. This has the major advantage that any dependencies of
do_fetch and do_unpack (e.g. for subversion or npm) will be handled
automatically. This also has the beneficial side-effect of fixing a
recent regression that prevented this fetch operation from working with
memory resident bitbake.

Also fix devtool's usage of fetch_uri() at the same time so that we can
completely replace it.

Fixes [YOCTO #11710].

(From OE-Core rev: 9a47a6690052ef943c0d4760630ee630fb012153)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21 08:44:25 +01:00
Paul Eggleton
a9848fdd46 devtool: extract: refactor kern-tools-native handling
When extracting linux-yocto kernel source, we don't need to dance around
shutting down and starting up tinfoil anymore, we can just execute the
tasks as needed when needed using tinfoil's new build_targets()
function. This allows us to tidy up the code structure a bit.

(From OE-Core rev: 5c7f5031023fb74b5f2f26d6b3c829981f2f54d2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21 08:44:25 +01:00
Aníbal Limón
dfefd671f3 devtool/standard: Fix lock in _prep_extract_operation
If for any reason the parse_recipe fail in extract command
the process gets locked because Cooker is expecting the
finish event by tinfoil.

For example:

$ devtool extract remake /tmp/remake

ERROR: remake is unavailable:
  remake was skipped: PREFERRED_PROVIDER_virtual/make set to make, not remake

(From OE-Core rev: 2c0062b59178fa668b26487b6d2f1e81a0d868e0)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-12 15:08:30 +01:00
Daniel Lublin
693878a3e9 devtool: deploy-target: Let script continue even if rm fails
The remote script is run with `set -e`, so doing rm without -f on a
$file that is already gone will exit the whole script, failing any
redeployment. Assume a use case where packages sometimes produces
certain test binaries stored on volatile media (tmpfs), and where the
system is occasionally rebooted.

(From OE-Core rev: db54c9a22a9b66c673df8e836de5e47fc9edda0b)

Signed-off-by: Daniel Lublin <daniel@lublin.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-03 23:46:06 +01:00
Ola x Nilsson
1deb4a00ba devtool: Compare abspath of both B and S
Either both or none of the paths must be passed through
os.path.abspath or things like 'A//B', 'A/./B/', and 'A/B/' in S will
cause unintentional mismatches even when B = "${S}".

Using os.path.abspath for both seems more likely to be correct as that
will also handle the case where ${B} != ${S} but the abspaths are
equal.

(From OE-Core rev: 061f2aab40fecbfe0dcb928baa95d6b3a6b45eed)

Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-03 23:46:06 +01:00
Daniel Lublin
dd2f423525 devtool: deploy-target: Don't use find -exec
find may be provided by busybox, which might be compiled without support
for -exec.

(From OE-Core rev: 404e8e3661469175e1ea087ebfaf3a7867bf4df2)

Signed-off-by: Daniel Lublin <daniel@lublin.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-03 23:46:06 +01:00
Saul Wold
ab0747e7c6 devtool/standard.py: Enable perf to be buildable
Perf is a tool build from the kernel source, which is normally available
in /work-shared/..., but when devtool is used to modify the kernel
source code, perf is not buildable since it gets an error about being unable
to add a depends to a non-exisit task do_patch.

This patch removes do_patch from the SRCTREECOVEREDTASKS and creates an empty
do_patch task to enable the VarFlags code to have someplace to attach depends
information to.

[YOCT #11120]

(From OE-Core rev: 86c793595e560e7bc52e3cd2a2752746e6adcb76)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-25 23:59:32 +01:00
Paul Eggleton
bce0b509bf devtool: extract: drop erroneous bb.event.TaskStarted
This is a non-existent event - we already have the actual
bb.build.TaskSucceeded further down in the list hence why it wasn't
noticed earlier.

(From OE-Core rev: 4e059a5ceb6f44401154e89e37f56de1d664a7cb)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-14 09:47:08 +01:00
Paul Eggleton
02c39d3fad devtool: extract: fix handling of failed tasks
If a task such as do_fetch fails when we're extracting source for a
recipe (within devtool modify / upgrade / extract / sync) then we should
naturally stop processing instead of blundering on; in order to do that
we need to be listening for the TaskFailed event. Thanks to Richard
Purdie for noticing and fixing this.

(From OE-Core rev: 9174b845bf6a6be7753bf6b921959b1f3f2dcbc0)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-14 09:47:08 +01:00
Paul Eggleton
2338032d0b devtool: modify: add --keep-temp option for debugging
Most of the other extract-based commands have this option but oddly I
left it out for modify - I guess because if I was debugging an issue here
I just used devtool extract to do so, but there's no reason why we can't
have it here and it is useful.

(From OE-Core rev: 98fbc46e1a51237213bd7825a922389d3ab2ad9b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-14 09:47:08 +01:00
Paul Eggleton
48f8d1201a devtool: add: prevent repeatedly running recipetool
If recipetool returns with exit code 14 this means devtool needs to
build nodejs-native and then call it again. If recipetool returns exit
code 14 again then clearly something has gone wrong and we should just
quit with an error.

(From OE-Core rev: 8d7cced6e06d7c2037f5ab75ac859f501129532e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
Paul Eggleton
bb8f141d0c devtool: add: fix node.js/npm handling with recipe specific sysroots
The change over to recipe specific sysroots means that we can no longer
get a known location simply from configuration for the npm binary - we
need to get the recipe sysroot for nodejs-native, look there for npm if
we need to check it's present, and add that to PATH when calling out to
npm. Unfortunately this means anywhere we need to get that path we have
to have parsed all recipes, otherwise we have no reliable way of
resolving nodejs-native. Thus we have to change recipetool create to
always parse all recipes (the structure of the code does not allow us to
do this conditionally).

In the worst case, if npm hasn't already been added to its own sysroot
and we are fetching from a source repository rather than an npm
registry, this gets a bit ugly because we end up parsing recipes three
times:
1) recipetool startup, which then fetches the code and determines it's
   a node.js module, finds that npm isn't available and then exits with
   a specific error to tell devtool it needs to build npm
2) when we invoke bitbake -c addto_recipe_sysroot nodejs-native
3) when we re-invoke recipetool

This code is badly in need of refactoring, but now is unfortunately not
the time to do that, so we're going to have to live with this ugliness
for now.

Fixes [YOCTO #10992].

(From OE-Core rev: acfdbd796c99882b8586023c8c6b848716105c8d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-13 10:54:10 +01:00
brian avery
0ffc7b8d2e devtool: point runqemu to correct native bindir
devtool/runqemu.py was relying on STAGING_BINDIR_NATIVE to find the host
tools it needed like qemu-system-<arch>.  In the post RSS world, this no
longer exists. This patch points it to
{STAGING_DIR}/{BUILD_ARCH}/{bindir_native}.

[YOCTO #11223]

(From OE-Core rev: 1910f9e9336bfedc8278a3bc02e7e7f934a4fc86)

Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:12 +01:00
Stephano Cetola
243693be54 devtool: open kconfig fragment in binary mode
When devtool writes to the kconfig fragment, it writes the output of
the diff command returned from pipe.communicate(). This function
returns binary objects. We should open the kconfig fragment file in
binary mode if we expect to write binary objects to it.

[YOCTO #11171]

(From OE-Core rev: 72bec63ab0e78753fb6ed1794d11beef9485c014)

Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-28 08:43:12 +01:00
Paul Eggleton
995b02b00c devtool: tidy up handling of parse failures
Since the tinfoil2 refactoring, if an error occurred during parsing, we
were showing a traceback and not correctly exiting (since we weren't
calling shutdown()). Fix both of these issues.

(From OE-Core rev: 18304036e1b513fd12c049dbf549ba75c503ed84)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22 11:35:21 +00:00
Anders Darander
6fa0fda116 devtool/recipetill: npm install of devDependencies
Web applications built using e.g. angular2, usually requires that the
packages in devDependencies are available.

Thus, add an option '--fetch-dev' to both devtool add and recipetool, to
add npm packages in devDependencies to DEPENDS.

(From OE-Core rev: f246f820d53b459596fde6758a09f7a0d7db7c4c)

Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22 11:35:20 +00:00
Richard Purdie
f73172e64e devtool/sdk: Run build-sysroots after installing new things
After running sdk-install we need to ensure that the standalone sysroots are
updated as done when the eSDK is originally built. Add such a call so this
happens automatically and the envrionment scripts in the SDK work correctly
after updates.

(From OE-Core rev: 4f422071d6f9a074986f399d9e648977bd2e0a68)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01 23:27:11 +00:00
Tim Orling
ea8b9d5d7b scripts/lib/devtool/deploy.py: add --port/-P argument for target connection
Enable using, e.g. host port 2222 for connection to qemu target.
Defaults to 22 for standard ssh/scp port.

[YOCTO #11079]

(From OE-Core rev: a2bfa2cc9ee19f617f7d3b6447896e45eb855d2e)

Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01 23:27:06 +00:00
Luck Hoang
1d133ec021 devtool: upgrade: fix error of import recipeutils
upgrade.py imports oe.recipeutils in meta/lib/ but path to oe.recipeutils
is not provided. This fails populate_sdk_ext.

(From OE-Core rev: 5f140359f859fea9cfe8c8d9c9584bceec875adb)

Signed-off-by: Luck Hoang <huyht1205@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-23 12:49:49 -08:00
Paul Eggleton
e8f4e06e76 devtool: improve parse failure handling
With the move to tinfoil2, the behaviour when parsing failed has changed
a bit - exceptions are now raised, so handle these appropriately.
Specifically when if parsing the recipe created when running devtool add
fails, rename it to .bb.parsefailed so that the user can run bitbake
afterwards without parsing being interrupted.

(From OE-Core rev: b9592bd3c1ab2155b9f62d18ce593b9c88eff279)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-07 14:50:09 +00:00
Paul Eggleton
9b3e808624 devtool: fix source extraction to work with recipe specific sysroots
When extracting source for a recipe within devtool (for extract, modify
or upgrade) We need to redirect WORKDIR, STAMPS_DIR etc. under a
temporary directory so that:
 (a) we pick up all files that get unpacked to the WORKDIR, and
 (b) we don't disturb the existing build
However, with recipe-specific sysroots the sysroots for the recipe will
be prepared under WORKDIR, and if we used the system temporary directory
i.e. usually /tmp) as used by mkdtemp by default, then our attempts to
hardlink files into the recipe-specific sysroots will fail on systems
where /tmp is a different filesystem, and we'd have to fall back to
copying the files which is a waste of time. Put the temp directory under
the WORKDIR to prevent that from being a problem.

(From OE-Core rev: e10a973cd9390eacb13bdb99693a0622bd3695f5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
[RP: Add needed mkdirhier call]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-20 11:53:49 +00:00
Richard Purdie
022a8b58c8 meta/scripts: Various getVar/getVarFlag expansion parameter fixes
There were a few straggling expansion parameter removals left for
getVar/getVarFlag where the odd whitespace meant they were missed
on previous passes. There were also some plain broken ussages such
as:

d.getVar('ALTERNATIVE_TARGET', old_name, True)
path = d.getVar('PATH', d, True)
d.getVar('IMAGE_ROOTFS', 'True')

which I've corrected (they happend to work by luck).

(From OE-Core rev: 688f7a64917a5ce5cbe12f8e5da4d47e265d240f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-09 13:39:11 +00:00
Paul Eggleton
358befb76e devtool: modify: fix usage on the kernel
When using devtool modify on the kernel, we have to do a bit of a dance
with tinfoil instances because we only find out that we're working on a
kernel recipe after tinfoil is initialised, but then we need to build
kern-tools-native which we're doing just by running bitbake directly.
With the tinfoil2 changes, a datastore for the recipe that we were
keeping around across the opening and closing of tinfoil is no longer
able to be used. Re-parse the recipe to avoid this problem.

(In future this whole thing will be able to be done in the same tinfoil
instance thanks to tinfoil2, but that refactoring is yet to be done.)

(From OE-Core rev: 06127d0115ba449bf04e2579cd1010065e0ed6e3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-22 08:50:16 +00:00
Joshua Lock
c0f2890c01 scripts: remove True option to getVar calls
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.

Search made with the following regex: getVar ?\(( ?[^,()]*), True\)

(From OE-Core rev: 0a36bd96e6b29fd99a296efc358ca3e9fb5af735)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
Paul Eggleton
4448b2a122 devtool: extract: disable basehash mismatch errors
Using the setVariable commands here followed by buildFile will result in
"basehash mismatch" errors, and that's expected since we are deviating
*at runtime* from what was previously seen by changing these variable
values. Set BB_HASH_IGNORE_MISMATCH to turn off the errors.

(From OE-Core rev: b0169796f294bbec0397b7eae86454a46b68cdc5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14 12:30:50 +00:00