Commit Graph

291 Commits

Author SHA1 Message Date
Paul Eggleton
90f925cd41 recipetool: create: support git short form URLs
In keeping with making recipetool create / devtool add as easy to use as
possible, users shouldn't have to know how to reformat git short form ssh
URLs for consumption by BitBake's fetcher (for example
user@git.example.com:repo.git should be expressed as
git://user@git.example.com/repo.git;protocol=ssh ) - instead we should
just take care of that automatically. Add some logic in the appropriate
places to do that.

(From OE-Core rev: 78c672a72f49c4b6cfd8c247efcc676b0ba1681a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:07 +01:00
Paul Eggleton
39d3aa2828 devtool: update-recipe: support files with subdir=
It's rare but there are recipes that have individual files (as opposed
to archives) in SRC_URI using subdir= to put them under the source tree,
the examples in OE-Core being bzip2 and openssl. This broke devtool
update-recipe (and devtool finish) because the file wasn't unpacked into
the oe-local-files directory and thus when it came time to update the
recipe, the file was assumed to have been deleted by the user and thus
the file was erroneously removed. Add logic to handle these properly so
that this doesn't happen.

(We still have another potential problem in that these files become part
of the initial commit from upstream, which could be confusing because
they didn't come from there - but that's a separate issue and not one
that is trivially solved.)

(From OE-Core rev: 9069fef5dad5a873c8a8f720f7bcbc7625556309)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-08 00:36:48 +01:00
Paul Eggleton
e616beba1c scripts: ensure tinfoil is shut down correctly
We should always shut down tinfoil when we're finished with it, either
by explicitly calling the shutdown() method or by using it as a
context manager ("with ...").

(From OE-Core rev: 5ec6d9ef309b841cdcbf1d14ac678d106d5d888a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03 23:45:54 +01:00
Paul Eggleton
4b4387455c lib/oe/patch: commit with a dummy user/email when PATCHTOOL=git
When using PATCHTOOL = "git", the user of the system is not really the
committer - it's the build system itself. Thus, specify "dummy" values
for username and email instead of using the user's configured values.
Various parts of the devtool code that need to make commits have also
been updated to use the same logic.

This allows PATCHTOOL = "git" and devtool to be used on systems where
git user.name / user.email has not been set (on versions of git where
it doesn't default a value under this circumstance).

If you want to return to the old behaviour where the externally
configured user name / email are used, set the following in your
local.conf:

PATCH_GIT_USER_NAME = ""
PATCH_GIT_USER_EMAIL = ""

Fixes [YOCTO #8703].

(From OE-Core rev: 765a9017eaf77ea3204fb10afb8181629680bd82)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-03 23:45:52 +01:00
Ola x Nilsson
07ebdbebdf devtool: build_image: Fix recipe filter
The missing split() causes dev and dbg packages to match.

(From OE-Core rev: bf83e0f0a3d52958c4380599f1afc4b8e058afd7)

Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-25 23:03:45 +01:00
Ola x Nilsson
d97aaac2d5 devtool: Use the wildcard flag in update_recipe_patch
The --wilcard-version flag was only used in the srcrev variant of the
update-recipe command.

(From OE-Core rev: d3057cba0b01484712fcee3c52373c143608a436)

Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-25 23:03:45 +01:00
Richard Purdie
189371f839 devtool/recipetool/meta: Adapt to bitbake API changes for multi-configuration builds
Unfortunately to implenent multiconfig support in bitbake some APIs
had to change. This updates code in OE to match the changes in bitbake.
Its mostly periperhal changes around devtool/recipetool

[Will need a bitbake version requirement bump which I'll make when merging]

(From OE-Core rev: 041212fa37bb83acac5ce4ceb9b7b77ad172c5c3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-18 10:07:23 +01:00
Paul Eggleton
d62fd7711d devtool: add finish subcommand
Add a subcommand which will "finish" the work on a recipe. This is
effectively the same as update-recipe followed by reset, except that the
destination layer is required and it will do the right thing depending
on the situation - if the recipe file itself is in the workspace (e.g.
as a result of devtool add), the recipe file and any associated files
will be moved to the destination layer; or if the destination layer is
the one containing the original recipe, the recipe will be overwritten;
otherwise a bbappend will be created to apply the changes. In all cases
the layer path can be loosely specified - it could be a layer name, or
a partial path into a recipe. In the case of upgrades, devtool finish
will also take care of deleting the old recipe.

This avoids the user having to figure out the correct actions when
they're done - they just do "devtool finish recipename layername" and
it saves their work and then removes the recipe from the workspace.

Addresses [YOCTO #8594].

(From OE-Core rev: fa550fcb9333d59b28fc0e4aebde888831410f5c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:49 +01:00
Paul Eggleton
10a5af5eb4 devtool: upgrade: record original recipe files
This provides us with the information we need to remove the original
version recipe and associated files when running "devtool finish" after
"devtool upgrade".

(From OE-Core rev: 92eb42c347af919cd9f8739515fdf806c12b5ba8)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:49 +01:00
Paul Eggleton
7af3295b0b devtool: update-recipe: refactor to allow calling separately
This will be called by "devtool finish" to allow it to update the recipe
or create the bbappend depending on the destination.

(From OE-Core rev: 5067cdc73483b53d46d9bf584723e41957c7ec54)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:49 +01:00
Paul Eggleton
5d78525a2d devtool: reset: refactor to allow calling separately
This will be called by "devtool finish" to allow it to reset the recipe
at the end.

(From OE-Core rev: b8d398516556eaf97679e28ad58448f570984b52)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:49 +01:00
Paul Eggleton
c4297f3e22 devtool: update-recipe: don't copy local files that haven't changed
If there are files in the oe-local-files directory which are identical
to the original version, then we shouldn't be copying them to the
destination layer. This is particularly important when using the -a
option to create a bbappend.

(From OE-Core rev: 9230bfcc839eb35630949f0a8ed058ca1fa944b1)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:49 +01:00
Paul Eggleton
8be2077a38 devtool: update-recipe: find and use existing files directories
devtool update-recipe was defaulting to the ${BPN} named directory when
adding patches next to a recipe, but that meant if you already had files
in a ${BP} named directory (i.e. name and version) or "files" then you'd
end up with two directories next to the recipe, which is usually not
what you want. To avoid this, look through FILESPATH and take the first
one that's the same level or one level down from the recipe and already
exists, if any.

(From OE-Core rev: c7a8190cf8bdf86ba850b6780b8e951e90232c06)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:49 +01:00
Paul Eggleton
f465039737 recipetool: create: support specifying a file as the local source
It is currently possible to specify a file (e.g. a tarball) on the local
disk as the source, but you have to know to put file:// in front of it.
There's really no need to force users to jump through that hoop if they
really want to do this so check if the specified source is a file and
prefix it with file:// if that's the case.

Also ensure the same works for "devtool add" at the same time.

(From OE-Core rev: 71350003790c38e84b0e525a71a2fe5d24e3d083)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-12 23:10:15 +01:00
Paul Eggleton
5c91537ab2 devtool: return specific exit code for incompatible recipes
Certain recipes cannot be used with devtool extract / modify / upgrade -
usually because they don't provide any source. Return a specific exit
code (4) so that scripts such as scripts/contrib/devtool-stress.py know
the difference between this and a genuine failure.

(From OE-Core rev: ffd295fed4ab81fc0bd00bb145ef4d72c49584bf)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-12 23:10:15 +01:00
Paul Eggleton
d56caf3408 devtool: reset: allow reset to work if the recipe file has been deleted
We were attempting to open the recipe file unconditionally here - we
need to account for the possibility that the recipe file has been
deleted or moved away by the user.

(From OE-Core rev: 47822a2aff56fd338c16b5ad756feda9f395a8a1)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-12 23:10:15 +01:00
Paul Eggleton
117c80f9dc devtool: update-recipe: fix --initial-rev option
In OE-Core revision 7baf57ad896112cf2258b3e2c2a1f8b756fb39bc I changed
the default update-recipe behaviour to only update patches for commits
that were changed; unfortunately I failed to handle the --initial-rev
option which was broken after that point. Rework how the initial
revision is passed in so that it now operates correctly.

(From OE-Core rev: b2ca2523cc9e51a4759b4420b07b0b67b3f5ac43)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-12 23:10:15 +01:00
Paul Eggleton
5c1b9fd541 devtool: reset: allow specifying multiple recipes
Allow specifying more than one recipe on the devtool reset command line.

Also tweak the help text slightly.

(From OE-Core rev: ad92ed8e4f7f48a3d212962531d596b36f6b284f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-15 08:35:01 +01:00
Ed Bartosh
cdff6bc0c1 scripts: python3: change python to python3 in shebang
(From OE-Core rev: 4b544ff388497cac82b0585f237900595523e1cb)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-03 13:13:29 +01:00
Ed Bartosh
07c97db272 scripts: python3: convert iterables to lists
Converted return value of items() keys() and values() to
lists when dictionary is modified in the loop and when
the result is added to the list.

(From OE-Core rev: 874a269eb1d70060c2f3b3f8b70800e2aea789f4)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-03 13:13:28 +01:00
Ed Bartosh
3ee70cb725 scripts: Fix encoding errors for python3
Moved call of decode('utf-8') as close as possible to
call of subprocess API to avoid calling it in a lot of
other places.

Decoded binary data to utf-8 where appropriate to fix devtool
and recipetool tests in python 3 environment.

(From OE-Core rev: 30d02e2aa2d42fdf76271234b2dc9f37bc46b250)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:01 +01:00
Ed Bartosh
ed7abe6b9a scripts: Replace basestring -> str for python3
Python 3 doesn't have basestring type as all string
are unicode strings.

(From OE-Core rev: e8cfab060f4ff3c4c16387871354d407910e87aa)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:01 +01:00
Ed Bartosh
7eab022d4b scripts: Fix deprecated dict methods for python3
Replaced iteritems -> items, itervalues -> values,
iterkeys -> keys or 'in'

(From OE-Core rev: 25d4d8274bac696a484f83d7f3ada778cf95f4d0)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:01 +01:00
Richard Purdie
642a997ade classes/lib: Update xrange -> range for python3
xrange() no longer exists in python 3, use range()

(From OE-Core rev: d022b4335100612d6596cc4c4956cb98ed5873cc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:00 +01:00
Richard Purdie
44e9a0d2fa classes/lib: Update to explictly create lists where needed
Iterators now return views, not lists in python3. Where we need
lists, handle this explicitly.

(From OE-Core rev: caebd862bac7eed725e0f0321bf50793671b5312)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:00 +01:00
Paul Eggleton
49557a5e9d recipetool / devtool: set a fixed SRCREV by default when fetching from git
If fetching source from a git repository, typically within OpenEmbedded
we encourage setting SRCREV to a fixed revision, so change to do that by
default and add a  -a/--autorev option to use "${AUTOREV}" instead.

(From OE-Core rev: 000480c42797dd2f03ebc3bc6d1dabfc6a7b75f5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-01 12:38:41 +01:00
Paul Eggleton
2d0a5c8361 devtool: upgrade: clarify help text for --srcrev option
The -S / --srcrev option must be specified if fetching from a git
repository, so spell that out in the help text.

(From OE-Core rev: 1465f205c235a1688a85844ebf5259e8971038ae)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-25 07:50:16 +01:00
Paul Eggleton
93a1c01666 devtool: upgrade: tweak conflict handling
Make a couple of changes to the rebase operation:

1) Only wrap the actual rebase command in try...except since a failure
   in any of the other commands should be an error, not a warning
2) If it's a conflict (which unfortunately we can only tell by checking
   for the keyword "conflict" since git doesn't return error codes based
   on the type of error) then print a message clarifying that the user
   needs to resolve the issue themselves to finish the upgrade.

(From OE-Core rev: 2a76be958432a35a0de30e9a5433089a54a06cad)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-25 07:50:16 +01:00
Paul Eggleton
ab227452c7 devtool: upgrade: handle upgrading recipes with a versioned inc file
The gdb recipe in OE-Core has an inc file with the version in it;
since the inc file is pulled in with a "require ${PV}.inc", when
upgrading the recipe we need to also rename the inc file it will fail to
parse and the upgrade itself will fail.

Fixes [YOCTO #9574].

(From OE-Core rev: 3c623aac9333d20a62475279c72b6b6ec3d7dd6b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-25 07:50:16 +01:00
Paul Eggleton
02986886fc devtool: sdk-update: drop support for local updates
Having two code paths here makes maintenance difficult, and it doesn't
seem likely that you would use the local case in real usage anyway, so
drop the local support entirely.

This should allow us to resolve [YOCTO #9301].

(From OE-Core rev: 7a4c9c96fee4fb514c2b69b52e811c4f896a16f1)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-21 22:24:57 +01:00
Randy Witt
832029356f devtool: Fix build-sdk when pn doesn't match filename
If an image with the filename foo.bb could be built using the name "bar"
instead, then build-sdk would fail to create the derivative sdk.

This was because the code assumed that the file name matched the target,
which is not necessarily the case.

(From OE-Core rev: d58a326b6960be14b8a049253559aec9582b7d0d)

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11 10:33:43 +01:00
Paul Eggleton
519600c607 devtool: sdk-update: fix handling of UNINATIVE_CHECKSUM changes
If UNINATIVE_CHECKSUM changes over an SDK update, bitbake within the
extensible SDK will be broken because it will see that the matching
uninative tarball doesn't exist and if there is a default value of
UNINATIVE_URL it will attempt to download the file and will then fail
because the checksums don't match up; alternatively if no UNINATIVE_URL
is set then it'll also fail with an error about misconfiguration. To fix
this, add some logic to devtool sdk-update to download the matching
uninative tarball(s) for the checksum(s) in the newly fetched SDK
configuration.

Fixes [YOCTO #9301].

(From OE-Core rev: 14ff58ad98a5afac08db77068d80f152d8875766)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-19 21:11:24 +01:00
Stephano Cetola
dee47ad159 devtool: sdk-update: reset git metadata on update
Replace git pull with fetch and reset to avoid the merge logic in the
event that the layers repo in the published SDK we're updating to isn't
fast-forward merge from the local repo.

Also add gitignore and committer info during publish to avoid errors and
to be sure that the first commit has a dummy user in it.

[ YOCTO #9368 ]

(From OE-Core rev: 4657bc9d165e51981e034e73e7b92552e873eef7)

Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-18 16:28:22 +01:00
Paul Eggleton
60a8719e6e devtool: upgrade: handle recipes where source is not first entry in SRC_URI
It is unusual but not impossible to find recipes whose first entry is
not the main source URL but instead some patch or other local file, for
example python-cryptography in meta-python (which sets SRC_URI before
inheriting pypi). There's nothing inherently wrong with this, and we
shouldn't assume that the first entry is the main source URL, so just
take the first non-local entry instead.

(From OE-Core rev: c868198c1f6006789707b497c2ae34d7cc5e706f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-15 06:59:43 +01:00
Paul Eggleton
4b7790915f devtool: modify: call shutdown on tinfoil when done
Strictly speaking we ought to explicitly shut down a tinfoil instance
when we're done with it. This doesn't affect modify's operation but is
important if you want to be able to call into modify() from another
plugin (though anyone doing so should be advised that the function is
by no means a stable API and is subject to change in future releases).

(From OE-Core rev: 626dbadf22b57a22a8f8b9d1957937120f4ba4d5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-31 23:01:37 +01:00
Paul Eggleton
0523378082 devtool: add: create git repository if URL specified as positional argument
When we add from a fetched URL we are supposed to turn the resulting
source tree into a git repository (if it isn't already one). However, we
were using the older deprecated option name here instead of the
positional argument, so "devtool add -f <url>" resulted in the repo
being created but "devtool add <url>" didn't, which was wrong.

Also update the oe-selftest tests to check that this worked.

(From OE-Core rev: a7b6b1f8cc1c096724f794ac9dee312b0f771f66)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-25 10:29:13 +00:00
Paul Eggleton
11c1d30152 devtool: add: delete externalsrc files on npm recipe do_install
The npm class just installs whatever is in ${S}; if you're using
externalsrc in conjunction with it the symlinks (oe-workdir and oe-logs
by default) plus singletask.lock will end up in the final package, which
isn't really right. Introduce a variable so we know the path the files
will be installed into within npm.bbclass, and append to do_install
within the workspace bbappend to delete the files.

(From OE-Core rev: 766845e06db9d7d595e836ea1364c16fa132a413)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-25 10:29:13 +00:00
Paul Eggleton
552a68ad1c devtool: configure-help: fix error if do_configure not already run
The code here for running do_configure if it hadn't already been run was
using the wrong string substitution parameters; fix it and test it.

(From OE-Core rev: b2677a4448dbc42e523c731b953b44006749252c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-25 10:29:13 +00:00
Paul Eggleton
06845723d4 devtool: sdk-install: add option to allow building from source
By default the sdk-install subcommand expects to restore the requested
items from sstate and fails if it can't. If the user is OK with building
from source, add a -s/--allow-build option to allow them to do that. In
the process, ensure we show the status output while we're installing.

Also add the missing header to the top of the file.

(From OE-Core rev: a86b426cdd465ec5cb08bb5fa7729e4e673d94bb)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-25 10:29:13 +00:00
Markus Lehtonen
51464e7b89 devtool: change config symlink name to .config.new
Otherwise (if the symlink is named .config) kernel build considers
source tree as dirty and fails.

[YOCTO #9270]

(From OE-Core rev: d36699b28c661880957d744420df9e23cdd76957)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-20 23:12:27 +00:00
Paul Eggleton
309b2e6c57 recipetool: create: support creation of additional files by plugins
Allow plugins to create additional files to go alongside the recipe. The
plugins don't know what the output filename is going to be, so they need
to put the files in a temporary location and add them to an "extrafiles"
dict within extravalues where the destination filename is the key and
the temporary path is the value.

devtool add was also extended to ensure these files get moved in and
preserved upon reset if they've been edited by the user.

(From OE-Core rev: 334b9451111b7e3efbb43b3a4eecebcab8ec6f0e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 17:00:29 +00:00
Paul Eggleton
25d9c4e02a devtool: add build-sdk subcommand
Add a build-sdk command which is only available within the extensible
SDK that builds a derivative extensible SDK. The idea is recipes in the
workspace become a part of the new SDK - for example, this allows taking
a vendor provided SDK, adding a few libs and then producing a new SDK
with those included.

When normally building the extensible SDK, the workspace is excluded;
here we need to copy into the new SDK (renaming it in the process); the
recipes' task signatures become locked and thus the sources are no
longer needed, so they are removed along with the workspace bbappends
which would interfere with the locked signatures. Additionally we need
to just copy the configuration files (i.e. local.conf and auto.conf)
rather than filtering and appending to them since that work has already
been done when constructing the original SDK. The extra sstate artifacts
from workspace recipes are also determined and copied into the new SDK
in minimal mode (on the assumption that you won't set up a new sstate
mirror).

This reuses some code from build-image, so that needed to be
generalised to allow that.

Implements [YOCTO #8892].

(From OE-Core rev: 59e207ff6dd4b50a8905e14bc9292cf2794f4e7a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 00:11:38 +00:00
Paul Eggleton
41eb36dc4c devtool: build-image: rename module
Hyphens aren't allowed in python identifiers, so you shouldn't use them
in module names or they are more difficult to import.

(From OE-Core rev: e8f6db9436dfc923e236f2cbc08f357e3b24fd5d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 00:11:37 +00:00
Juro Bystricky
14dd07caac sdk.py: Fix undefined variable
"sdk_update" uses a variable newsdk_path, which was never declared.
This would cause the command:

    devtool sdk-update <poky-sdk-latest>

to fail with an error:
NameError: global name 'newsdk_path' is not defined

The remedy is to declare newsdk_path as it was no doubt intended,
corresponding to the argument specifying <poky-sdk-latest>.

[YOCTO#9042]

(From OE-Core rev: 6ac589a4a9290665f8c5295e2e2a03a2b187b957)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-03 10:49:00 +00:00
Paul Eggleton
10290f2245 devtool: upgrade: print new recipe name
Make this consistent with "devtool add" so that the user knows where to
find the new recipe.

(From OE-Core rev: 28fea4a104ba85aded2ecfe891e9bf997d72a103)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:01 +00:00
Paul Eggleton
5cd3be38f2 devtool: upgrade: drop PR on upgrade
The PR value should be reset to the default when upgrading, so we need
to drop it from the newly created file.

(From OE-Core rev: 4acf8a6912106dd3da5ceee9e30ccce86211d04b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:00 +00:00
Paul Eggleton
e6f684b10c devtool: upgrade: eliminate unnecessary datastore copy
We aren't modifying the datastore copy here, so we don't need a copy at
all.

(From OE-Core rev: 9f05bf98d4dbcb991b684a2ab480c6638c6be292)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:00 +00:00
Paul Eggleton
860574edb2 devtool: upgrade: fix several issues with extraction of new source
Fix several issues when extracting the new version source over the top
of the old one (when the recipe is not fetching from a git repo):

* Delete the old source first so we ensure files deleted in the new
  version are deleted. This also has the side-effect of fixing any
  issues where files aren't marked writeable in the old source and thus
  overwriting them failed (harfbuzz 1.1.3 contains such files).
* Fix incorrect variable name in abspath statement that made it a no-op

(From OE-Core rev: 4f7595a514661f0e2418393286849a1cb6c46f0d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:00 +00:00
Paul Eggleton
66a781c7e1 devtool: upgrade: fix constructing new branch from tarball releases
When we do an upgrade from one tarball version to another we want to:

1) Check out the old version as a new branch
2) Record the changes between the old and new versions as a commit
3) Check out the old version with patches applied
4) Rebase that onto the new branch

Where we went wrong was step #1 where instead we checked out the old
version with patches applied as the new branch, which meant the rebase
didn't do anything and any changes made by the patches to files still in
the new version were wiped out.

(From OE-Core rev: a4f3e6d5f4c549e36cce1e18801a565a3a3c9b0a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:00 +00:00
Paul Eggleton
d30cc76039 devtool: upgrade: fix renaming of recipe if PV is not in name
If the actual value of PV isn't in the name of the recipe (for example,
a git or svn recipe) there's no point trying to rename it. Additionally,
we already have the original filename, there's no need to guess it -
just pass it in.

(From OE-Core rev: e144db3650b4a7c0a59066621905f34a5400303e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:00 +00:00
Paul Eggleton
75eeeabe3c devtool: upgrade: fix moving version-specific files directory
We were trying to move this from the current directory instead of the
path. Let's just use shutil.move() instead of shelling out to mv.

(From OE-Core rev: 60454a0ba154d6c777e0c2b05b887b4e4fcde986)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:00 +00:00
Paul Eggleton
81ebb0b61e devtool: upgrade: fix version argument checking
For recipes that specify SRCREV, the code here wasn't quite doing the
right thing. If the recipe has a SRCREV then that needs changing on
upgrade, so ensure that the user specifies it. If it doesn't, then it'll
be "INVALID" not None since the former is the actual default, so handle
that properly as well. Additionally an unset variable was being
erroneously passed when raising the error about the version being the
same leading to a traceback, so fix that as well.

(From OE-Core rev: 1d0f821371d1cb93e30fad86f0c20e38cb93b54b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:00 +00:00
Paul Eggleton
e953b5711f devtool: upgrade: drop superfluous call to validate_pn
The recipename argument to devtool upgrade specifies an existing recipe,
so by definition the name will be valid (or it won't exist) - we don't
need to validate it ourselves, that's only needed for situations like in
devtool add where we're creating a new recipe.

(From OE-Core rev: 1e2bc7d861555a04350a87f19047efdc717046be)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:00 +00:00
Paul Eggleton
492b1eb1d2 devtool: upgrade: make source tree path optional
Make devtool upgrade consistent with devtool add/modify in defaulting to
sources/<recipename> under the workspace if no source tree path is
specified.

(From OE-Core rev: 8a952407b192313515e91570632446b6dff01665)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:00 +00:00
Paul Eggleton
942ae25279 devtool: modify: fix source tree default name when mapping virtuals
If you for example ran devtool modify virtual/libusb0 without specifying
a source tree path, the default was <workspace>/sources/virtual/libusb0
which isn't correct - it should be using the mapped name i.e.
libusb-compat (in the default OE-Core configuration). Reorder some of
the code to ensure that the mapped name is used.

(From OE-Core rev: c51736df17da8e6e561dd5b7ce59cb08254da870)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:33:00 +00:00
Paul Eggleton
e2334e126f devtool: add: tweak auto-determining name failure message
As suggested by Khem Raj.

(From OE-Core rev: 36cc6b81d0281348a0f241a80ddd427745a6a678)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:32:59 +00:00
Markus Lehtonen
73bf792c85 devtool: update-recipe: create config fragment
Create config fragment if the user makes modifications to kernel config.
User may change .config e.g. by directly editing it or by running the
'do_menuconfig' bitbake task. Devtool generates one monolithic fragment
by simply doing a diff between .config and .config.baseline files in the
source directory.  If either of these files is missing, the config
fragment is not gerenrated or updated. The output is a file,
'devtool-fragment.cfg' that gets added to SRC_URI in the recipe (as well
as copied into the 'oe-local-files' directory if that is present in the
source tree).

${S}/.config will be a symlink to ${B}/.config. We need to do this as
devtool is not able to access ${B} because ${B} is set in a .bbappend in
the workspace layer which is not parsed by devtool itself.

[YOCTO #8999]

(From OE-Core rev: 524da136e5b837a60682516ac08f3092c635e934)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:32:58 +00:00
Markus Lehtonen
2fbd1d77bd devtool: sync: update kernel config
Copy kernel config is copied to the source directory at a later phase in
_extract_source() so that it gets copied when devtool sync is done, too.

(From OE-Core rev: ff895be7a46c4b3b1b791e5387490d90bb34fce2)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-28 11:32:58 +00:00
Paul Eggleton
b2d44729e9 devtool: modify: tweak help description for behaviour change
I should have adjusted this in OE-Core commit
80a44e52609a89d9ffe816181ae193af491c06ac where the behaviour changed.

(From OE-Core rev: 13409a2b899bb74b8060c840b8c7ef8759d099cb)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:43 +00:00
Paul Eggleton
a8e0e5ecd3 devtool: deploy-target: preserve existing files
If files would be overwritten by the deployment, preserve them in a
separate location on the target so that they can be restored if you
later run devtool undeploy-target.

At the same time, also check for sufficient space before starting the
operation so that we avoid potentially failing part way through.

Fixes [YOCTO #8978].

(From OE-Core rev: a2da55712691bcf1066c53d813107d75213d6b10)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:43 +00:00
Paul Eggleton
2059a344b6 devtool: undeploy-target: support undeploying all recipes
If you want to put the target device back to exactly how it was before
devtool deploy-target started poking things into it, then it would make
things easier if you didn't have to figure out which recipes were
deployed. Now that we have the list stored on the target we can
determine this reliably, so add a -a/--all option to undeploy-target to
undeploy everything that has been deployed.

One of the side-effects of this is that the dry-run functionality for
undeploy-target had to be reimplemented to actually run the script on
the target, since we have no way of knowing what's been deployed from
the host side. We don't need to do the same for deploy-target though
since we know exactly which files will be deployed without referring to
the target.

(From OE-Core rev: 41fed83060f0041e14e455d1446397bda277d953)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:42 +00:00
Paul Eggleton
b95c72c4dc devtool: deploy-target: write deployed files list to target
When running devtool deploy-target, we save a list of deployed files,
and this list is used by devtool undeploy-target (or the next time
deploy-target is run if the list is present, in case any files have been
renamed or deleted since the first time). We were writing this file to
the host, but it makes more sense to write the list to the target
instead, so that if we for example swap in a different board, or switch
hosts, things will work as expected.

In order to do this properly we have to construct a shell script and
ship it over to the target so we can run it. The manifest is written out
to a hidden directory in the root (/.devtool).

Fixes [YOCTO #7908].

(From OE-Core rev: a16a0c9334b785e2df896266c8911a2c7a1806b8)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:42 +00:00
Paul Eggleton
62989efe6d devtool: sdk-update: tweak command-line handling of updateserver
Get the default value for updateserver from the configuration file and
show it in the help; also only make the parameter optional if it's
specified. This means we can also drop the check in the function as
argparse will then ensure it's specified if there's no config setting.

(From OE-Core rev: 82497bde58fc8bdae8d8acfbf025a1a90b14cc3e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:42 +00:00
Paul Eggleton
cada5a83c7 devtool: (un)deploy-target: add help descriptions
Add a long description used when running --help on the specific command.

(From OE-Core rev: eb7787d1652fd84a149fd394969f4f1099406051)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:42 +00:00
Paul Eggleton
32ef523898 devtool: categorise and order subcommands in help output
The listing of subcommands in the --help output for devtool was starting
to get difficult to follow, with commands appearing in no particular
order (due to some being in separate modules and the order of those
modules being parsed). Logically grouping the subcommands as well as
being able to exercise some control over the order of the subcommands
and groups would help, if we do so without losing the dynamic nature of
the list (i.e. that it comes from the plugins). Argparse provides no
built-in way to handle this and really, really makes it a pain to add,
but with some subclassing and hacking it's now possible, and can be
extended by any plugin as desired.

To put a subcommand into a group, all you need to do is specify a group=
parameter in the call to subparsers.add_parser(). you can also specify
an order= parameter to make the subcommand sort higher or lower in the
list (higher order numbers appear first, so use negative numbers to
force items to the end if that's what you want). To add a new group, use
subparsers.add_subparser_group(), supplying the name, description and
optionally an order number for the group itself (again, higher numbers
appear first).

(From OE-Core rev: e1b9d31e6ea3c254ecfe940fe795af44761e0e69)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:42 +00:00
Paul Eggleton
9f7df76eb4 devtool: update-recipe: don't show workspace recipe warning if no update
If we didn't make any changes to the file then there's no point warning
the user that we have done.

(From OE-Core rev: 391b9ba30d802ac420ddf382588e03e718861c01)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:42 +00:00
Paul Eggleton
51972edbb0 devtool: reset: fix preserving patches/other files next to recipes
If files had been created next to the recipe (for example devtool add,
edit the source and commit and then devtool update-recipe), running
devtool reset failed to preserve those files and gave an error due
to trying to rmdir the directory containing them which wasn't empty.
Fix the preservation of files in the "attic" directory properly so
we catch anything under the directory for the recipe, and replicate
the same structure in the attic directory rather than slightly
flattening it as we were before.

(From OE-Core rev: bbe63eb97ae7f78959f117d6066ef821c4da1c77)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:42 +00:00
Paul Eggleton
e54f9c159d devtool / recipetool: use common code for launching editor
Looking at Chris Larson's code for starting the user's editor for
"recipetool newappend" it was slightly better than what I wrote for
"devtool edit-recipe" in that it checks VISUAL as well as EDITOR and
defaults to vi if neither are set, so break this out to its own function
and call it from both places. The broken out version passes shell=True
however in case it's a more complicated command rather than just a name
of an executable.

(From OE-Core rev: 184a256931e8cdc7bea97a905c4e67a435964de0)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:42 +00:00
Paul Eggleton
dd35f69340 devtool: minor fix for error message
There is no -N/--name option for devtool, that's a recipetool option -
with devtool you just specify the name as a positional argument.

(From OE-Core rev: d2bc0cba5ca8a7220ffe1ef96acf856fe972ce7c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:42 +00:00
Paul Eggleton
86f3464b47 devtool: add: fix adding from a local source directory
Fix a regression introduced in in OE-Core revision
aedfc5a5db1c4b2b80a36147c9a13b31764d91dd where specifying a local source
tree without specifying a name resulted in a traceback.

Fixes [YOCTO #9086].

(From OE-Core rev: 67ea9d21f20a371a0ee443b46326018cb1527b62)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:33:03 +00:00
Paul Eggleton
fa50153bc2 devtool: modify: make -x the default behaviour
It's going to be more common for users not to have the prepared source
tree for a recipe already, so the default behaviour ought to be to
extract it for them from the recipe. Change the default to extract
(effectively making the -x option a no-op) and add a --no-extract/-n
option to disable it. Later we can look at trying to be smart and
reusing an existing source tree instead of erroring out if it exists;
for now this is just the default reversal.

(From OE-Core rev: 80a44e52609a89d9ffe816181ae193af491c06ac)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:33:03 +00:00
Paul Eggleton
ddfe74447b devtool: commit for extra tasks that modify source when extracting
When extracting source for a recipe, if there are additional custom
tasks run that make changes to the source, create a commit in the
generated git branch so they are contained. This is particularly
useful for tasks that come before do_patch since otherwise the changes
might get incorporated in the first applied patch, but otherwise it
helps avoid the tree being dirty at any point.

Fixes [YOCTO #7626].

(From OE-Core rev: 997a77d9b20af1778b804778e5d8c8a7424f7582)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:33:02 +00:00
Richard Purdie
32b498cba3 scripts/devtool: Add getVarFlag expand argument
(From OE-Core rev: afa9b1081848cf0dbc40bbaf565bcc2ee19e8f6e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-06 23:10:44 +00:00
Markus Lehtonen
aadb879e5b devtool: extract: use the correct datastore for builddir
(From OE-Core rev: 6f5bec4570a9237681fcee3922209af2a48f6c07)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26 22:31:58 +00:00
Paul Eggleton
89a7ed5b67 devtool: add configure-help subcommand
When you need to set EXTRA_OECONF for a recipe, you need to know what
options the configure script actually supports; the configure script
however is only accessible from within a devshell and (at least in the
case of autotooled software fetched from an SCM repository) may not
actually exist until do_configure has run. Thus, provide a "devtool
configure-help" subcommand that runs the configure script for a recipe
with --help and shows you the output through a pager (e.g. less),
prefaced by a header describing the current options being specified.

There is basic support for autotools, cmake and bare configure scripts.
The cmake support is a little hacky since cmake doesn't really have a
concise help option that lists user-defined knobs (without actually
running through the configure process), however that being a design
feature of cmake there's not much I can think of to do about that at
the moment.

(From OE-Core rev: 0e5d84d9705091b338000ef02720cfa090f76888)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26 18:05:14 +00:00
Paul Eggleton
84720c8ce9 devtool: properly handle bb.build.FuncFailed when extracting source
When we run the tasks required to extract the source for a recipe (e.g.
within "devtool modify" or "devtool extract") if one of those tasks
fails you get a bb.build.FuncFailed exception; handle this properly so
you don't see a traceback.

(From OE-Core rev: 95d8631b3bdf216001e57f48277535c65a4cc49e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26 18:05:13 +00:00
Paul Eggleton
c3f0f7be84 devtool: add: warn if modified recipe found in attic directory
If a recipe generated by "devtool add" has been modified since then when
you run "devtool reset", it will be moved into the "attic" subdirectory
of the workspace in case those modifications need to be preserved. It
seems natural that if those modifications were worth preserving we
should warn the user if such a file exists when they run "devtool add"
to create the same recipe again, so they can pick up where they left off
if they want to.

(From OE-Core rev: 0a39b907ff997c3a62c92ab22325c726b612de5b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26 18:05:13 +00:00
Paul Eggleton
e559b66289 devtool: build-image: allow specifying packages to add to image
Provide an option to devtool build-image to specify the list of packages
instead of taking the list of packages produced by recipes in the
workspace. Sometimes you don't want all of these packages; other times
you want to add more.

This is the most immediate fix for [YOCTO #8855], though it is a little
crude so I would like to provide better means of customising the image
contents later.

(From OE-Core rev: b3a44951a74fe58714b72e71a7a558b67a71e1e3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26 18:05:13 +00:00
Paul Eggleton
e00eac862e devtool: move edit-recipe to a separate module
standard.py is getting a bit large; move the "utility" commands to
another module.

(From OE-Core rev: 5089b93f5b341dc28c343f7afe15efda2081ed36)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-26 18:05:13 +00:00
Paul Eggleton
50e40fc91f devtool / recipetool: support specifying a subdirectory within the fetched source
Sometimes you don't want to build an entire project, just a subdirectory
of it; add a --src-subdir option to make that easier. (We still look for
a single subdirectory in what gets unpacked, e.g. what you might find
within a tarball, so whatever you specify with this option is added onto
the end of that.)

(From OE-Core rev: 59682d78f95732e014f78f13e0a05f843860d9bb)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:33 +00:00
Paul Eggleton
b435225e8a devtool: add sdk-install subcommand
Add the ability to install additional pre-built items (from shared
state) into the extensible SDK. This can already be done implicitly by
adding something to DEPENDS within a recipe you're working on and then
running "devtool build", but it's useful to be able to explicitly
install things particularly if you're using the extensible SDK as a
traditional toolchain.

Note that for this command to be useful you need to have SSTATE_MIRRORS
set in your SDK configuration, and that mirror needs to be populated
with sstate artifacts for recipes you wish to be able to install.

(From OE-Core rev: 3474a42954908d1688fd3a6cb600eed315b27833)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:32 +00:00
Paul Eggleton
44d1a2a45c devtool: sdk-update: improve SDK update process robustness
Make the following improvements to the SDK update process:

* Use a manifest file with sha256sums to track files other than sstate
  and metadata that we need to update - e.g. conf files. This allows us
  to handle where files such as auto.conf may or may not be present,
  as well as the configuration changing without affecting task signatures
  - we still want the config files copied in that case rather than it
  saying nothing needs to be done.
* Write the SSTATE_MIRRORS_append to site.conf rather than local.conf
  so that local.conf remains static (since we don't want to trigger an
  update every time). Also, If there is an SSTATE_MIRRORS value already
  set in the configuration we can skip this and assume it contains the
  needed packages.
* Allow the update process to be run in any directory, don't assume
  we're already at the base of the SDK
* Where practical, fetch remote files into a temporary location and
  then move them to the desired location at the end, to avoid a
  failed update leaving the SDK in a broken state.
* Update all installed do_populate_sysroot / do_packagedata tasks
  instead of using the SDK targets. This ensures any item installed
  through dependencies after installation (e.g. when running
  "devtool build") won't go stale.

(From OE-Core rev: 3d35631121f0e030bc8151f5c23d84008d06f44b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:32 +00:00
Paul Eggleton
3360baa96b devtool: sdk-update: improve temp directory handling
* Use tempfile.mkdtemp() instead of hardcoding temp dir
* Set a variable early for the temp locked sigs file and use that
  everywhere
* Delete the temp dir at the end

(From OE-Core rev: bad5d1a8c047a8118d30d9fa708b021d1599e0dc)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:32 +00:00
Paul Eggleton
d193531d5e devtool: build: ensure pkgdata is written out
When you run devtool build, you need to have the pkgdata written out at
the end, so that if what you're adding is a library and the next thing
you add is something that depends on that library, the necessary
information to map the dependency back to the recipe is present. In
practical terms all this means is we need do_packagedata to run in
addition to do_populate_sysroot.

This does mean that do_package needs to run which wasn't running before,
and that means that the few package QA tests that run within do_package
such as installed-vs-shipped will now be run. This may be a bit
bothersome, and prompted a fix for one of our oe-selftest tests as a
result, but I don't see an easy way around it. Ultimately if you care
about using the recipe in an image you'll need to fix any such errors
anyway.

Fixes [YOCTO #8887].

(From OE-Core rev: 6579c7120ee5a541427ff5b6b07f838d52f9fe7c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:32 +00:00
Paul Eggleton
d3a4f72896 classes/populate_sdk_ext: add option to bring in pkgdata for world
Add a variable SDK_INCLUDE_PKGDATA which you can set to "1" to include
pkgdata for all recipes in the world target. There are a couple of uses
for this:

1) If you use "devtool add" to add a recipe that builds something which
   depends on anything in world, the dependency can then be correctly
   mapped to the recipe providing it and that recipe can be added to
   DEPENDS, since we have the pkg-config and shared library dependency
   data within pkgdata.
2) You'll be able to search for these recipes and any files they
   package for the target with "devtool search" since that also uses
   pkgdata

This of course assumes you've tailored world through EXCLUDE_FROM_WORLD
to only include recipes you'd want built in your distro, but I think
that's a reasonable assumption; failing that there is a
WORLD_PKGDATA_EXCLUDE variable that you can set to exclude any recipes
you don't want.

Note that this patch relies on functionality implemented in a recent
BitBake patch and will not work without it.

Implements [YOCTO #8600].

(From OE-Core rev: 67149ea097d6fab7496b43e85a40853f40bd527e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:32 +00:00
Richard Purdie
0269219fbe populate_sdk_ext: Use new --setscene-only option to bitbake instead of workarounds
Rather than horrible workarounds, use the new --setscene-only option
of bitbake to pre-populate the SDK structures.

(From OE-Core rev: 38237b7ac53c416f85c4a70a61acafc3404c8b5f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 23:26:28 +00:00
Paul Eggleton
b2fe2a8fdc devtool: build: support using BBCLASSEXTENDed names
It's logical that you would want to build BBCLASSEXTENDed items
separately through devtool build, so simply allow that - we're just
passing the name verbatim to bitbake, so all it means is adjusting the
validation.

(From OE-Core rev: 25dc5ac42c9da53c01416e7fdcc819d729281133)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton
38ed039753 devtool: reset: support recipes with BBCLASSEXTEND
If the recipe file itself was created in the workspace, and it uses
BBCLASSEXTEND (e.g. through devtool add --also-native), then we need to
clean the other variants as well.

(From OE-Core rev: e1bf6a30679a83d4dbcf37276204f639541e95f9)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton
532f42985f devtool: refactor code for getting local recipe file
We're doing this in a couple of places, let's just find the recipe file
if it exists within the workspace (which it will if it's been added
through "devtool add") when we read in the workspace.

(From OE-Core rev: 81cf5580993c99050e3f4d6d891bc67534721487)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton
ec9016821d devtool: add: support adding a native variant
Sometimes you need to build a variant of a recipe for the build
host as well as for the target (i.e. BBCLASSEXTEND = "native"); add a
 --also-native command line option to "recipetool create" that enables
this and plumb it through from an identical option for "devtool add".

(We could conceivably do the same for nativesdk, but I felt it might be
confusing within the context of the extensible SDK, where nativesdk
isn't really relevant to the user.)

(From OE-Core rev: f3bea83db173cce921a9a30f04e88b7e3ed98854)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton
99e3872d89 devtool: reset: do clean for multiple recipes at once with -a
We need to run the clean for all recipes that are being reset before we
start deleting things from the workspace; if we don't, recipes providing
dependencies may be missing when we come to clean a recipe later (since
we don't and couldn't practically reset them in dependency order). This
also improves performance since we have the startup startup time for the
clean just once rather than for every recipe.

(From OE-Core rev: c10a2de75a99410eb5338dd6da0e0b0e32bae6f5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton
4deed25330 devtool: sdk-update: add option to skip preparation step
For debugging purposes it's useful to be able to skip the preparation
step so you can inspect what the state of the build system is first.

(From OE-Core rev: 0bba4b5afd2ce2c3a79445eee886979a77f1a4d8)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton
d586a11ef7 devtool: sdk-update: fix error checking
Running "raise" with no arguments here is invalid, we're not in
exception handling context. Rather than also adding code to catch the
exception I just moved the check out to the parent function from which
we can just exit.

(From OE-Core rev: 0164dc66467739b357ab22bf9b8c0845f3eff4a4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton
c1b7d83d77 devtool: sdk-update: fix metadata update step
* Clone the correct path - we need .git on the end
* Pull from the specified path instead of expecting a remote to be set
* up in the repo already (it isn't by default)

(From OE-Core rev: 1a60ee8bd21e156022c928f12bb296ab5caaa766)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:45 +00:00
Paul Eggleton
efead10e6f devtool: sdk-update: fix not using updateserver config file option
We read the updateserver setting from the config file but we never
actually used that value - the code then went on to use only the value
supplied on the command line.

Fix courtesy of Dmitry Rozhkov <dmitry.rozhkov@intel.com>

(From OE-Core rev: 1c85237803038fba539d5b03bf4de39d99380684)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:45 +00:00
Ed Bartosh
23302ee03a devtool: use cp instead of shutil.copytree
Copied layers with 'cp -a' instead of calling shutil.copytree as
copytree fails to copy broken symlinks.

More pythonic fix would be to use copytree with 'ignore' parameter,
but this could slow down copying complex directory structures.

[YOCTO #8825]

(From OE-Core rev: e5b841420b9fdd33829f7665a62cd06a3017f7e6)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:19 +00:00
Paul Eggleton
498e4834a8 devtool: sync: tweak help / messages
Much of this was copy/pasted from the extract subcommand code; make it
specific to sync.

(From OE-Core rev: 08ca966dd48db881d8bfb7d7fa0aba49f1f7cffe)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:14 +00:00
Paul Eggleton
b272c5174f devtool: reset: print message about leaving source tree behind
We deliberately leave the source tree alone when resetting in case it
contains any work in progress belonging to the user; tell them that
we're doing this so they aren't surprised about it still existing later
on.

(From OE-Core rev: 1c616802ba2cfb37f8403f1dc6cf91ce3bd99ac2)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:14 +00:00
Paul Eggleton
95a234e7b3 devtool: status: list recipe file within workspace if one exists
If a recipe in the workspace actually exists as a file within the
workspace (e.g. after doing "devtool add" or "devtool upgrade") then
show the path to the recipe file on the status line for the recipe.

(From OE-Core rev: 29833828a90c8433af3b231b50e99cd97edf19ff)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:14 +00:00
Paul Eggleton
e11673960f devtool: modify: default source tree path
As per the changes to "devtool add", make the source tree path optional
and use the default path if none is specified.

(From OE-Core rev: 83707d1334fb094fd1877bcfd07a83866601048a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:14 +00:00