Commit Graph

263 Commits

Author SHA1 Message Date
Paul Eggleton
4c71afb378 recipetool: create: ensure URL parameters don't make it into the name
When auto-detecting the name for a recipe from the URL, strip off any
parameters (";name=value...") before parsing the URL, otherwise this
just ends up in the recipe name which is undesirable.

(From OE-Core rev: d3c46b5d0abd56bcadd4f2f1ef985f13d67f605b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:33:03 +00:00
Paul Eggleton
f767757b3e recipetool: create: determine name/version from github/bitbucket URLs
Both BitBucket and GitHub provide "release" downloads that unfortunately
don't include the actual project name. For recipetool this means we have
to have special handling for these URLs or else the current code assumes
the name of the project is something like "v0.9.1". Borrow and adapt
some code from autospec to do this, and put it in its own function for
clarity.

(From OE-Core rev: e8435fde7f82c39cc0efe1d4bf043b29f63898e9)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:33:03 +00:00
Paul Eggleton
d94c7e367c recipetool: create: support cmake find_library directive
CMake supports a find_library() directive to find named libraries, so
detect dependencies from this.

(From OE-Core rev: d0265bc67f797ee4b7760cf37335994133809abf)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:33:03 +00:00
Christopher Larson
6fab5fc1bb recipetool.newappend: add -e/--edit argument
(From OE-Core rev: 2bd518fe3a8cb0022a816a7d1ac8d9a3dedee6d9)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:27:49 +00:00
Paul Eggleton
c1f4e929c9 recipetool: create: better fix for fetch error handling
I was a little bit hasty in OE-Core revision
c2cc5abe34169eae92067d97ce1e747e7c1413f5 - it turns out BitBake's
fetcher code is not consistent in whether it logs something useful or
not; when fetching from an http URL it does but with a git repository
it doesn't. In advance of any major reworking of fetch error handling in
BitBake, let's just print the text of the exception and then we know we
have shown something to the user.

Additionally, we were only catching FetchException here but there are
several other classes of exception that the fetcher can raise (e.g.
MalformedUrl); catch the parent BBFetchException class instead so we
avoid tracebacks for those other classes as well.

(From OE-Core rev: 578d3873a6415c9203c185c21cff472f7d2dab02)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:33 +00:00
Paul Eggleton
10c8d14d1a recipetool: create: fix extraction of name from URLs ending in /
If the URL ends in a / then we want to strip that off the path we split
out of the URL before calling os.path.basename() on it.

(From OE-Core rev: 308189beda8a31541481d09e3d5e86187e843d8d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:33 +00:00
Paul Eggleton
b307e0a604 recipetool: create: extract SRC_URI from local git repositories
If you specify a local directory which happens to be a git repository
with an origin remote (and it is in fact remote), we can use that for
SRC_URI rather than leaving it blank in the recipe.

(From OE-Core rev: b143d414846854dc8b3e0a47358daf5646eded38)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:33 +00:00
Paul Eggleton
50e40fc91f devtool / recipetool: support specifying a subdirectory within the fetched source
Sometimes you don't want to build an entire project, just a subdirectory
of it; add a --src-subdir option to make that easier. (We still look for
a single subdirectory in what gets unpacked, e.g. what you might find
within a tarball, so whatever you specify with this option is added onto
the end of that.)

(From OE-Core rev: 59682d78f95732e014f78f13e0a05f843860d9bb)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:33 +00:00
Paul Eggleton
7e1691de0a recipetool: create: strip quotes from values extracted from CMakeLists.txt
Quoting is optional in CMakeLists.txt and is occasionally used, so strip
out quotes if they are present.

(From OE-Core rev: 4ffe2e1ec9df05b92a2ad5746fb0ca6d218fd77e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:33 +00:00
Paul Eggleton
c1492c4e4f recipetool: create: add a couple more license checksums
I found these when I was looking at libftdi and they seem to be
generic enough to show up in at least a couple of other packages so I
figure I'll add them.

(From OE-Core rev: 9fa3ff44e05930d4dfa153db777077e747ecbf45)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:53 +00:00
Paul Eggleton
2c8c9fe3b4 recipetool: create: add basic support for extracting dependencies from cmake
Add support for extracting dependencies from CMakeLists.txt. There's
still a bunch of things missing that are outside the scope of OE-Core
and we still lack a proper extension mechanism, but this is a good
start.

This also adds an oe-selftest test to exercise the new code a bit.

Implements [YOCTO #7635].

(From OE-Core rev: 77e73e6930381fdbd6e78d3913d6467572e16568)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:53 +00:00
Paul Eggleton
3eb397fab6 recipetool: create: force GL libraries to virtual/*
We want to specify dependencies on virtual/* rather than whatever
library is selected in the current configuration.

(From OE-Core rev: e1ac0c45b27ded9962edaf34597f827d0b41ba82)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:52 +00:00
Paul Eggleton
726dbda396 recipetool: create: move dependency mapping code to RecipeHandler
Some refactoring to allow access to the library/header/pkg-config
mappings and the DEPENDS / unmapped dependency output code from other
classes than AutotoolsRecipeHandler.

(From OE-Core rev: 40c10d998b90dd59c6d36c28f8ba11ec598bfa0f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:51 +00:00
Paul Eggleton
788e4bb23e recipetool: create: fix overzealous mapping of git URLs
The regex for detecting git URLs was unanchored, leading to it matching
where it shouldn't have. An example of where this went wrong was
http://taglib.github.io/releases/taglib-1.9.1.tar.gz.

(From OE-Core rev: bacff751c88b680fbfb07843b18c59c8bc80a9ea)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:51 +00:00
Paul Eggleton
ece0a2e1a0 recipetool: create: support additional autoconf macros from autoconf-archive
Support a number of macros from autoconf-archive when reading
configure.ac to extract dependencies.

(From OE-Core rev: ee977a62c58ded361c2abd78654bd25637fe9ea1)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:50 +00:00
Paul Eggleton
903d471355 recipetool: create: detect flex/bison dependency
There are a few different macros that can be used to pick up these
tools, add support for them all.

(From OE-Core rev: 7dfff4b7f05653aea230294ff1a7c023730deff9)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:50 +00:00
Paul Eggleton
a66f4ac086 recipetool: create: pick up boost macros in configure.ac
The presence of BOOST_REQUIRE or AX_BOOST.* indicates that boost is a
dependency.

(From OE-Core rev: 02570b1fc31c7f4e9643aea8365806089622c0e7)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:49 +00:00
Paul Eggleton
dbe91a3d6a recipetool: create: improve extraction of pkg-config / lib deps
* The regexes for PKG_CHECK_MODULES / AC_CHECK_LIB were a bit too strict
  and thus we were skipping some macros.
* Add support for PKG_CHECK_EXISTS
* Avoid duplicates in warning on missing pkg-config dependencies
* Ignore dependency on musl (since this may come up if it's the selected
  C library)

(From OE-Core rev: c58669fb0977f7f0cb79f252484d5c5ef0dfb7e4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:48 +00:00
Paul Eggleton
ec9016821d devtool: add: support adding a native variant
Sometimes you need to build a variant of a recipe for the build
host as well as for the target (i.e. BBCLASSEXTEND = "native"); add a
 --also-native command line option to "recipetool create" that enables
this and plumb it through from an identical option for "devtool add".

(We could conceivably do the same for nativesdk, but I felt it might be
confusing within the context of the extensible SDK, where nativesdk
isn't really relevant to the user.)

(From OE-Core rev: f3bea83db173cce921a9a30f04e88b7e3ed98854)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton
5ef716c280 recipetool: create: support creating standalone native/nativesdk recipes
If the recipe name ends with -native then we should inherit native;
likewise if it starts with nativesdk- then inherit nativesdk.

(Note that the recipe name must actually be specified by the user in
order to trigger this - we won't do it based on any name auto-detected
from e.g. the tarball name.)

Since we're doing this based on the name, "devtool add" will also gain
this functionality automatically.

(From OE-Core rev: a216fb1f5953327790dce3d2c1a9af616c0b410d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton
1e503c0a51 recipetool: create: lower case name when determining from filename
As a matter of general convention we expect recipe names to be lower
case; in fact some of the packaging backends insist upon it. Since in
this part of the code we're auto-determining the name, we should convert
the name to lowercase if it's not already so that we're following
convention and avoiding any validation issues later on.

(From OE-Core rev: 64986decbc11afa0d1e11251f5f7adcba1860d19)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton
891673101e recipetool: create: fix error when extracting source to a specified directory
Having fetched the source and unpacked it to a temporary directory, we
then move part of it to the destination directory, or if the source is at
the top level we move the whole temporary directory, but in the latter
case we were later attempting to delete the temporary directory which no
longer existed. Clear out the variable so that doesn't happen.

(From OE-Core rev: 91714a52e91cddba5a16c73cf5765d1f47f7856c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:14 +00:00
Paul Eggleton
fe28c251b1 recipetool: create: improve autotools support
* tar and binutils we can assume are there
* libsocket is only relevant on BSD systems, so we can ignore it.
* Detect more things implying gettext/intltool is needed
* Detect glib-2.0 requirement.

(From OE-Core rev: 2c4c78a6a9970533f3352f1067b2263f45098493)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:14 +00:00
Paul Eggleton
ebe5f0b872 recipetool: create: basic extraction of name/version from filename
Often the filename (e.g. source tarball) contains the name and version
of the software it contains.

(This isn't intended to be exhaustive, just to catch the common case.)

(From OE-Core rev: 944eacfb849ee69b41e12c9de4f264406281ac6a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton
db5f9645ad recipetool: create: support extracting name and version from build scripts
Some build systems (notably autotools) support declaring the name and
version of the program being built; since we need those for the recipe
we can attempt to extract them. It's a little fuzzy as they are often
omitted or may not be appropriately formatted for our purposes, but it
does work on a reasonable number of software packages to be useful.

(From OE-Core rev: 3b3fd33190d89c09e62126eea0e45aa84fe5442e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton
6a7661b800 recipetool: create: set up priority system for recipe handlers
Sometimes we want to force one handler to run before another; if the two
handlers are in different plugins that's difficult without some kind of
priority number, so add one and sort by it.

(From OE-Core rev: 0219d4fb9cefcee635387b46fc1d215f82753d92)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton
38803e38d6 recipetool: create: detect when specified URL returns a web page
If the user specifies a URL that just returns a web page, then it's
probably incorrect (or broken); attempt to detect this and show an error
if it's the case.

(From OE-Core rev: 83b1245b2638eb5d314fe663d33cd52a776a34a7)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton
e61645b504 recipetool: create: minor fix for potential issue in python handling
If SRC_URI happened not to be in the pre-generated lines then this code
would error out. This is unlikely to happen with the way the create code
is structured at the moment, but handle it just in case.

(From OE-Core rev: 95d33e90f2d5d9dd5ccc950856b8a939fefb831e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:12 +00:00
Paul Eggleton
ae2141b84f recipetool: create: fix do_install handling for makefile-only software
In my testing here it appears make -qn returns an error (exit code 2)
whereas make -n doesn't; I can't immediately tell why based on the
documentation. We don't actually care for it to be quiet since we're
capturing the output, so let's just leave -q off and have this work
properly as a result.

(From OE-Core rev: 30c4cd9efdac400d713dff645f23f2627277d75a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:12 +00:00
Paul Eggleton
c2f17428b8 recipetool: create: avoid traceback on fetch error
If a fetch error occurs, the fetcher already prints a reasonable error -
we don't need the traceback as well, so catch that and exit if it
occurs.

(From OE-Core rev: c2cc5abe34169eae92067d97ce1e747e7c1413f5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:12 +00:00
Paul Eggleton
470f20bc02 recipetool: create: handle https://....git URLs
When you grab a URL for a github repository you'll almost certainly find
it in https://github.com/path/to/repository.git format; but bitbake's
fetcher can't handle that because it'll see https:// at the start and
assume it should use wget to fetch it. If the URL starts with http:// or
https:// and the path part ends with .git then assume it's a git
repository and adjust it accordingly.

(From OE-Core rev: bdbc4cf41d30eddb8a9ed882dedcc1670ce8fdd6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:12 +00:00
Paul Eggleton
0cc3810299 recipetool: make plugin registration function name consistent with devtool
This should have been register_commands rather than register_command;
I used register_commands in devtool so lets change this here to be
consistent with that. (Since this is extensible through layers though we
need to remain compatible with the old name, so fall back to that if the
new function name isn't there.)

(From OE-Core rev: 1047f6592ac81643cd847f104da766dc4a4c81ea)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:05 +00:00
Paul Eggleton
b381f804a5 recipetool: add setvar subcommand
Add a recipetool subcommand "setvar" to set a variable in a file. This
uses our existing logic such that it doesn't matter if the variable is
already set in the recipe, if it's set in the recipe or some inc file,
and if the variable is not currently set that the line setting the
variable gets inserted in the right place in the file.

Implements [YOCTO #7676].

(From OE-Core rev: 7c33ef77fa165182d24f0a9ae769e9e630e6bd47)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:05 +00:00
Paul Eggleton
3f46af2b19 devtool: drop unused plugin_init() functions
This function is no longer required to be defined for a plugin, so drop
it where it's a no-op.

(From OE-Core rev: 49716a5e27e704598321bd6c82275b16afc444aa)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:04 +00:00
Christopher Larson
4e2c5e171c recipetool.append: don't choke on a trailing ; in a url
Apparently bb.fetch.URI can't handle this at the moment.

(From OE-Core rev: d3e0a300810251f34932f46daf5263a23846fedd)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-25 07:50:30 +00:00
Paul Eggleton
8cacd228e6 recipetool: create: fix change in path structure if --extract-to path exists
If the directory specified by --extract-to exists, because we were using
shutil.move() to move the temporary extracted directory to the specified
path, a subdirectory was being created under that directory instead of
moving the contents, which was a different result than if the directory
didn't previously exist. We could try to always move the contents but
that's complicated when any symlinks are involved; the simplest thing is
just to remove the directory (which should be empty anyway) before
moving the temporary directory across in its place.

(From OE-Core rev: 2880bd23b471c1966661b9f05726faf60f9c0e7e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-07 00:09:10 +01:00
Paul Eggleton
c4181c6e7e devtool / recipetool: add handling for binary-only packages
Add a means of creating recipes for package files or archives that
contain a directory structure to be installed verbatim, for example an
rpm file. (We mostly just re-use bin_package here and skip some of the
normal build system checks.) This support is available in "recipetool
create" and "devtool add" which wraps the former.

(From OE-Core rev: 9e0a6b2e6f16185f8032d36b77d40802bc388987)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23 09:53:20 +01:00
Paul Eggleton
210d959023 recipetool: create: fix handling of URIs containing #
The # character in a URI denotes a fragment; we don't care about this
since it is never supposed to be sent to the server, so remove it from
the URI before actually trying to fetch it or use it in SRC_URI within
the recipe.

(This has come up because download links on pypi.python.org seem to have
a fragment containing the md5sum of the download; without stripping this
off the fetcher will choke on it.)

(From OE-Core rev: 58dc726808817210764eab963a4d453dc4ff49aa)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23 09:53:16 +01:00
Paul Eggleton
a35ad723ce recipetool: create: fix creating empty shell functions
The shell considers empty functions to be a syntax error, so for
template shell functions that contain only comments (or no lines at all)
then add a : to act as a no-op which avoids the syntax error.

(From OE-Core rev: ff14d9e5b935b99b2efde479515e54c02ba58f6e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23 09:53:16 +01:00
Christopher Larson
d18612a8a9 recipetool: add 'newappend' sub-command
This sub-command creates a bbappend for the specified target and prints the
path to the bbappend. The -w argument, as with some of the other recipetool
commands, will make a version-independent bbappend.

Example usage: recipetool newappend meta-mylayer virtual/kernel

[YOCTO #7964]

(From OE-Core rev: ac053163c7823e482ca1af2962342e64a54bfb52)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-18 09:05:30 +01:00
Ed Bartosh
9485888656 devtool: make plugin_init optional
So far all devtool and recipetool plugins were expected
to have plugin_init function. This function is empty in
most of plugins as they don't require initialisation.

Making plugin_init optional would allow not having empty
plugin_init in every plugin.

(From OE-Core rev: b99640c89f067866b264f4ee4030fae4c2f338c0)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-12 22:48:30 +01:00
Paul Eggleton
ee0c11d988 scriptutils: split out simple fetching function from recipetool
This will now also be used by "devtool upgrade".

(From OE-Core rev: 0d0b8425eaf74a6d7f3d9f6471e6edca1a273c06)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-09 14:27:50 +01:00
Christopher Larson
89878f5906 recipetool.append: add extralines arg to appendsrc
This makes the function more reusable for other sub-commands.

(From OE-Core rev: c6feb6e97c9929560b1e3cec280c1d7976ecc030)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27 23:29:12 +01:00
Christopher Larson
c56cb31565 recipetool: appendsrcfile: handle S == STAGING_KERNEL_DIR
When determining the path from WORKDIR to the extracted sources, we're using
S, but if S is in work-shared, that's problematic and won't give us good
results, so assume 'git' for that case, warning when appropriate.

(From OE-Core rev: dc70cf470f18c97b6762cbf7aca723769b47bc7d)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-23 08:48:41 +01:00
Christopher Larson
5c080f71bf recipetool: appendsrcfile: use -D, not -d for destdir
-d is already taken for --debug.

(From OE-Core rev: fc174debb073c99a0cabf4f00e66ab36c7e93989)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-23 08:48:41 +01:00
Christopher Larson
023e9b2685 recipetool: appendsrcfile: fix duplicate SRC_URI check
(From OE-Core rev: e9986529d15220a8482f5a7a2fdbe86110ee35b5)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-23 08:48:41 +01:00
Richard Purdie
ee69463f4e recipetool/append: Fix selftest failure with multilib
If you have multilib enabled, it finds lib32-base-files instead of
base-files for test_recipetool_appendfile_basic causing a test
failure. Add a fix for this.

(From OE-Core rev: c9821a56da9c6e341408ea21e0d8a4cc5291dba6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:55:38 +01:00
Christopher Larson
b428f88bc1 recipetool: add appendsrcfile(s) sub-commands
This adds the `appendsrcfile` and `appendsrcfiles` sub-commands, which let the
user add or replace one or more files in the recipe sources, either in a path
relative to `S` or `WORKDIR`. Each file gets added to `SRC_URI` as a file://
URI, using the subdir= parameter to specify the destination directory.

Examples:

    # Adds our defconfig as file://defconfig. If it's already in SRC_URI, it
    # won't be duplicated.
    recipetool appendsrcfile --workdir meta-mylayer linux-mel defconfig
    recipetool appendsrcfiles --workdir meta-mylayer linux-mel defconfig

    # Does the same, handling the different local filename
    recipetool appendsrcfile --workdir meta-mylayer linux-mel defconfig.mine defconfig

    # Adds our device tree files to the source tree
    recipetool appendsrcfiles --destdir arch/arm/boot/dts meta-mylayer linux-mel *.dts

Of course, for the latter example to be of use, the new dts files would need
to be added to `KERNEL_DEVICETREE` as well, and depending on the kernel,
`DEFCONFIG` or `KERNEL_DEFCONFIG` may need to be set.

Cc: Paul Eggleton <paul.eggleton@linux.intel.com>
(From OE-Core rev: 8227172d05fb394d79ff83d3c36f5208ebebf92c)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27 22:42:55 +01:00
Christopher Larson
49a089f8e1 recipetool.append: use argparse types for validation
This validates the arguments early, when argparse is parsing the arguments, in
a consistent way.

Cc: Paul Eggleton <paul.eggleton@linux.intel.com>
(From OE-Core rev: bec92899f5324a4423b4ee70365eaa5dfb6891a6)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27 22:42:55 +01:00
Paul Eggleton
32ed5f5769 recipetool: appendfile: fix file command error handling
* It turns out that not all versions of the file command support the -E
  option - the version in Ubuntu 14.04 doesn't support it for example.
  This option is supposed to force file to return an error if the file
  can't be opened - since we can't rely upon it then fall back to
  looking at the output instead. (The results of this issue were simply
  that we didn't notice if the file was executable and give a warning,
  which tripped an oe-selftest failure - so it was minor.)
* If we receive an error there's not much point looking at the output to
  see what type was returned because there wasn't one.

(From OE-Core rev: 7bf9dccef1aa626adc9c45addcd066fed69cace9)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-18 09:14:06 +01:00
Paul Eggleton
44e036904c recipetool: appendfile: clarify help text
Add a long description and tweak some of the argument descriptions so
that it's clearer what the appendfile subcommand does and how it works.

(From OE-Core rev: a0a595307d28d0350c3752293ab8ebf499ee416f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-18 09:14:06 +01:00
Paul Eggleton
fbfc06a969 recipetool: add appendfile subcommand
Locating which recipe provides a file in an image that you want to
modify and then figuring out how to bbappend the recipe in order to
replace it can be a tedious process. Thus, add a new appendfile
subcommand to recipetool, providing the ability to create a bbappend
file to add/replace any file in the target system. Without the -r
option, it will search for the recipe packaging the specified file
(using pkgdata from previously built recipes). The bbappend will be
created at the appropriate path within the specified layer directory
(which may or may not be in your bblayers.conf) or if one already exists
it will be updated appropriately.

Fairly extensive oe-selftest tests are also provided.

Implements [YOCTO #6447].

(From OE-Core rev: dd2aa93b3c13d2c6464ef0fda59620c7dba450bb)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-20 21:41:04 +01:00
Paul Eggleton
fa61a9468b recipetool: allow specifying SRCREV via rev= in URI
Provide a means to set SRCREV immediately by using rev= in the URI; if
it is specified then it is removed before setting SRC_URI and SRCREV is
set to the same value instead.

(From OE-Core rev: c9304fcb0a2b81700d0ed5c13b4d976bd4230ce3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-01 17:08:50 +01:00
Paul Eggleton
4502daefb8 devtool: add: use the appropriate file naming and versioning for SCM recipes
* Recipes that fetch from git, svn or hg by OpenEmbedded convention
  should normally be named with this as a suffix, since PV is meant to
  be set appropriately within the recipe, so follow this. In order to
  make this work we need to be able to have the version independent from
  the file name, so add a -V option to recipetool create to allow this
  to be specified.
* If -V is specified on the devtool add command line, ensure at PV gets
  set to include this version.

(From OE-Core rev: 2b997c41c6476a13bf516586d56a9051aceb93ec)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-01 17:08:50 +01:00
Paul Eggleton
a4fca1d523 recipetool: ensure git clone is standalone when extracting
If -x is specified and the specified URI was a git repository, we need
to ensure that the resulting clone is a sandalone and not one that has
pointers into the temporary fetch location or DL_DIR (since the git
fetcher does a local clone with -s). Split out the code from devtool
that already does this for "devtool modify -x" and reuse that.

(From OE-Core rev: fc47e8652ef32e7399f57c80593df90dc52d8b84)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-01 17:08:50 +01:00
Paul Eggleton
24e5ad268c recipetool: avoid second-level subdir when extracting
When -x was specified, we were getting the normal unpack subdirectory
which we don't really want - if there's only one subdirectory unpacked
then we should effectively copy just it to the extraction path, not as a
subdirectory under it.

(From OE-Core rev: 0eeecce96a0aa757c2c4c4ac4d82e3bcbf0f982c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-01 17:08:49 +01:00
Paul Eggleton
0a45ccb253 recipetool: fix duplicate licenses being picked up
If a license file matched more than one of the specifications (e.g.
COPYING.GPL) then it was being added to LIC_FILES_CHKSUM more than once.

(From OE-Core rev: 58316a2890782d206e9b9472ba483367f7560109)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20 11:27:49 +00:00
Paul Eggleton
5eb9ffe1e0 recipetool: fix regression caused by previous commit
Option was renamed in the setup code but not in the code that used it.

(From OE-Core rev: 4b4f2d4f2869d6d5d564cc1b2d733f41ab5a3b9b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-23 17:35:27 +00:00
Paul Eggleton
a3dfb70b62 recipetool: improve command-line help
Based on feedback from Scott Rifenbark <scott.m.rifenbark@intel.com>

(From OE-Core rev: 2c59b2b20c32577085645056e4cbf4f9c259e4d7)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-08 08:00:27 +00:00
Paul Eggleton
3f739a86a9 recipetool: replace version in S value
If a versioned recipe filename is specified, replace the version in the
value of S with ${PV} just as we do with SRC_URI to make future upgrades
of the recipe easier.

(From OE-Core rev: 0b3c81ed5bcce9b608c4f804496d769288fe8c04)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-03 14:53:50 +00:00
Christopher Larson
1b7b58ac97 recipetool: add python dependency scanning support
This uses a standalone python script named `pythondeps` which now lives in
scripts. It supports scanning for provided packages and imported
modules/packages, the latter via the python ast. It's not perfect, and
obviously conditional imports and try/except import blocks are handled
naively, listing all the imports even if they aren't all used at once, but it
gives the user a solid starting point for the recipe.

Currently `python_dir` from setup.py isn't being handled in an ideal way. This
is easily seen when testing the python-async package. There, the root of the
project is the async package, so the root has __init__.py and friends, and the
python provides scanning currently just assumes the basedir of that dir is the
package name in this case, which is not correct. Forthcoming patches will
resolve this.

(From OE-Core rev: cb093aca3b78f130dc7da820a8710342a12d1231)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-23 11:36:28 +00:00
Christopher Larson
e490d79fb7 recipetool: add python buildsystem support
- Handles distutils & setuptools.
- Supports pulling metadata from PKG-INFO, .egg-info, & setup.py (via two
  different mechanisms).
- Doesn't handle python 3 yet.

(From OE-Core rev: 6ff693c71d97b4bcfde198c84cf9fac7185cccfd)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-23 11:36:28 +00:00
Paul Eggleton
5638ca2b94 scripts/recipetool: Add a recipe auto-creation script
Add a more maintainable and flexible script for creating at least the
skeleton of a recipe based on an examination of the source tree.
Commands can be added and the creation process can be extended through
plugins.

[YOCTO #6406]

(From OE-Core rev: fa07ada1cd0750f9aa6bcc31f8236205edf6b4ed)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-23 10:18:15 +00:00