Commit Graph

35 Commits

Author SHA1 Message Date
Paul Eggleton
ae2141b84f recipetool: create: fix do_install handling for makefile-only software
In my testing here it appears make -qn returns an error (exit code 2)
whereas make -n doesn't; I can't immediately tell why based on the
documentation. We don't actually care for it to be quiet since we're
capturing the output, so let's just leave -q off and have this work
properly as a result.

(From OE-Core rev: 30c4cd9efdac400d713dff645f23f2627277d75a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:12 +00:00
Paul Eggleton
c2f17428b8 recipetool: create: avoid traceback on fetch error
If a fetch error occurs, the fetcher already prints a reasonable error -
we don't need the traceback as well, so catch that and exit if it
occurs.

(From OE-Core rev: c2cc5abe34169eae92067d97ce1e747e7c1413f5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:12 +00:00
Paul Eggleton
470f20bc02 recipetool: create: handle https://....git URLs
When you grab a URL for a github repository you'll almost certainly find
it in https://github.com/path/to/repository.git format; but bitbake's
fetcher can't handle that because it'll see https:// at the start and
assume it should use wget to fetch it. If the URL starts with http:// or
https:// and the path part ends with .git then assume it's a git
repository and adjust it accordingly.

(From OE-Core rev: bdbc4cf41d30eddb8a9ed882dedcc1670ce8fdd6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:12 +00:00
Paul Eggleton
0cc3810299 recipetool: make plugin registration function name consistent with devtool
This should have been register_commands rather than register_command;
I used register_commands in devtool so lets change this here to be
consistent with that. (Since this is extensible through layers though we
need to remain compatible with the old name, so fall back to that if the
new function name isn't there.)

(From OE-Core rev: 1047f6592ac81643cd847f104da766dc4a4c81ea)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:05 +00:00
Paul Eggleton
b381f804a5 recipetool: add setvar subcommand
Add a recipetool subcommand "setvar" to set a variable in a file. This
uses our existing logic such that it doesn't matter if the variable is
already set in the recipe, if it's set in the recipe or some inc file,
and if the variable is not currently set that the line setting the
variable gets inserted in the right place in the file.

Implements [YOCTO #7676].

(From OE-Core rev: 7c33ef77fa165182d24f0a9ae769e9e630e6bd47)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:05 +00:00
Paul Eggleton
3f46af2b19 devtool: drop unused plugin_init() functions
This function is no longer required to be defined for a plugin, so drop
it where it's a no-op.

(From OE-Core rev: 49716a5e27e704598321bd6c82275b16afc444aa)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:04 +00:00
Christopher Larson
4e2c5e171c recipetool.append: don't choke on a trailing ; in a url
Apparently bb.fetch.URI can't handle this at the moment.

(From OE-Core rev: d3e0a300810251f34932f46daf5263a23846fedd)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-25 07:50:30 +00:00
Paul Eggleton
8cacd228e6 recipetool: create: fix change in path structure if --extract-to path exists
If the directory specified by --extract-to exists, because we were using
shutil.move() to move the temporary extracted directory to the specified
path, a subdirectory was being created under that directory instead of
moving the contents, which was a different result than if the directory
didn't previously exist. We could try to always move the contents but
that's complicated when any symlinks are involved; the simplest thing is
just to remove the directory (which should be empty anyway) before
moving the temporary directory across in its place.

(From OE-Core rev: 2880bd23b471c1966661b9f05726faf60f9c0e7e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-07 00:09:10 +01:00
Paul Eggleton
c4181c6e7e devtool / recipetool: add handling for binary-only packages
Add a means of creating recipes for package files or archives that
contain a directory structure to be installed verbatim, for example an
rpm file. (We mostly just re-use bin_package here and skip some of the
normal build system checks.) This support is available in "recipetool
create" and "devtool add" which wraps the former.

(From OE-Core rev: 9e0a6b2e6f16185f8032d36b77d40802bc388987)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23 09:53:20 +01:00
Paul Eggleton
210d959023 recipetool: create: fix handling of URIs containing #
The # character in a URI denotes a fragment; we don't care about this
since it is never supposed to be sent to the server, so remove it from
the URI before actually trying to fetch it or use it in SRC_URI within
the recipe.

(This has come up because download links on pypi.python.org seem to have
a fragment containing the md5sum of the download; without stripping this
off the fetcher will choke on it.)

(From OE-Core rev: 58dc726808817210764eab963a4d453dc4ff49aa)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23 09:53:16 +01:00
Paul Eggleton
a35ad723ce recipetool: create: fix creating empty shell functions
The shell considers empty functions to be a syntax error, so for
template shell functions that contain only comments (or no lines at all)
then add a : to act as a no-op which avoids the syntax error.

(From OE-Core rev: ff14d9e5b935b99b2efde479515e54c02ba58f6e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23 09:53:16 +01:00
Christopher Larson
d18612a8a9 recipetool: add 'newappend' sub-command
This sub-command creates a bbappend for the specified target and prints the
path to the bbappend. The -w argument, as with some of the other recipetool
commands, will make a version-independent bbappend.

Example usage: recipetool newappend meta-mylayer virtual/kernel

[YOCTO #7964]

(From OE-Core rev: ac053163c7823e482ca1af2962342e64a54bfb52)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-18 09:05:30 +01:00
Ed Bartosh
9485888656 devtool: make plugin_init optional
So far all devtool and recipetool plugins were expected
to have plugin_init function. This function is empty in
most of plugins as they don't require initialisation.

Making plugin_init optional would allow not having empty
plugin_init in every plugin.

(From OE-Core rev: b99640c89f067866b264f4ee4030fae4c2f338c0)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-12 22:48:30 +01:00
Paul Eggleton
ee0c11d988 scriptutils: split out simple fetching function from recipetool
This will now also be used by "devtool upgrade".

(From OE-Core rev: 0d0b8425eaf74a6d7f3d9f6471e6edca1a273c06)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-09 14:27:50 +01:00
Christopher Larson
89878f5906 recipetool.append: add extralines arg to appendsrc
This makes the function more reusable for other sub-commands.

(From OE-Core rev: c6feb6e97c9929560b1e3cec280c1d7976ecc030)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27 23:29:12 +01:00
Christopher Larson
c56cb31565 recipetool: appendsrcfile: handle S == STAGING_KERNEL_DIR
When determining the path from WORKDIR to the extracted sources, we're using
S, but if S is in work-shared, that's problematic and won't give us good
results, so assume 'git' for that case, warning when appropriate.

(From OE-Core rev: dc70cf470f18c97b6762cbf7aca723769b47bc7d)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-23 08:48:41 +01:00
Christopher Larson
5c080f71bf recipetool: appendsrcfile: use -D, not -d for destdir
-d is already taken for --debug.

(From OE-Core rev: fc174debb073c99a0cabf4f00e66ab36c7e93989)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-23 08:48:41 +01:00
Christopher Larson
023e9b2685 recipetool: appendsrcfile: fix duplicate SRC_URI check
(From OE-Core rev: e9986529d15220a8482f5a7a2fdbe86110ee35b5)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-23 08:48:41 +01:00
Richard Purdie
ee69463f4e recipetool/append: Fix selftest failure with multilib
If you have multilib enabled, it finds lib32-base-files instead of
base-files for test_recipetool_appendfile_basic causing a test
failure. Add a fix for this.

(From OE-Core rev: c9821a56da9c6e341408ea21e0d8a4cc5291dba6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12 22:55:38 +01:00
Christopher Larson
b428f88bc1 recipetool: add appendsrcfile(s) sub-commands
This adds the `appendsrcfile` and `appendsrcfiles` sub-commands, which let the
user add or replace one or more files in the recipe sources, either in a path
relative to `S` or `WORKDIR`. Each file gets added to `SRC_URI` as a file://
URI, using the subdir= parameter to specify the destination directory.

Examples:

    # Adds our defconfig as file://defconfig. If it's already in SRC_URI, it
    # won't be duplicated.
    recipetool appendsrcfile --workdir meta-mylayer linux-mel defconfig
    recipetool appendsrcfiles --workdir meta-mylayer linux-mel defconfig

    # Does the same, handling the different local filename
    recipetool appendsrcfile --workdir meta-mylayer linux-mel defconfig.mine defconfig

    # Adds our device tree files to the source tree
    recipetool appendsrcfiles --destdir arch/arm/boot/dts meta-mylayer linux-mel *.dts

Of course, for the latter example to be of use, the new dts files would need
to be added to `KERNEL_DEVICETREE` as well, and depending on the kernel,
`DEFCONFIG` or `KERNEL_DEFCONFIG` may need to be set.

Cc: Paul Eggleton <paul.eggleton@linux.intel.com>
(From OE-Core rev: 8227172d05fb394d79ff83d3c36f5208ebebf92c)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27 22:42:55 +01:00
Christopher Larson
49a089f8e1 recipetool.append: use argparse types for validation
This validates the arguments early, when argparse is parsing the arguments, in
a consistent way.

Cc: Paul Eggleton <paul.eggleton@linux.intel.com>
(From OE-Core rev: bec92899f5324a4423b4ee70365eaa5dfb6891a6)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-27 22:42:55 +01:00
Paul Eggleton
32ed5f5769 recipetool: appendfile: fix file command error handling
* It turns out that not all versions of the file command support the -E
  option - the version in Ubuntu 14.04 doesn't support it for example.
  This option is supposed to force file to return an error if the file
  can't be opened - since we can't rely upon it then fall back to
  looking at the output instead. (The results of this issue were simply
  that we didn't notice if the file was executable and give a warning,
  which tripped an oe-selftest failure - so it was minor.)
* If we receive an error there's not much point looking at the output to
  see what type was returned because there wasn't one.

(From OE-Core rev: 7bf9dccef1aa626adc9c45addcd066fed69cace9)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-18 09:14:06 +01:00
Paul Eggleton
44e036904c recipetool: appendfile: clarify help text
Add a long description and tweak some of the argument descriptions so
that it's clearer what the appendfile subcommand does and how it works.

(From OE-Core rev: a0a595307d28d0350c3752293ab8ebf499ee416f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-18 09:14:06 +01:00
Paul Eggleton
fbfc06a969 recipetool: add appendfile subcommand
Locating which recipe provides a file in an image that you want to
modify and then figuring out how to bbappend the recipe in order to
replace it can be a tedious process. Thus, add a new appendfile
subcommand to recipetool, providing the ability to create a bbappend
file to add/replace any file in the target system. Without the -r
option, it will search for the recipe packaging the specified file
(using pkgdata from previously built recipes). The bbappend will be
created at the appropriate path within the specified layer directory
(which may or may not be in your bblayers.conf) or if one already exists
it will be updated appropriately.

Fairly extensive oe-selftest tests are also provided.

Implements [YOCTO #6447].

(From OE-Core rev: dd2aa93b3c13d2c6464ef0fda59620c7dba450bb)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-20 21:41:04 +01:00
Paul Eggleton
fa61a9468b recipetool: allow specifying SRCREV via rev= in URI
Provide a means to set SRCREV immediately by using rev= in the URI; if
it is specified then it is removed before setting SRC_URI and SRCREV is
set to the same value instead.

(From OE-Core rev: c9304fcb0a2b81700d0ed5c13b4d976bd4230ce3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-01 17:08:50 +01:00
Paul Eggleton
4502daefb8 devtool: add: use the appropriate file naming and versioning for SCM recipes
* Recipes that fetch from git, svn or hg by OpenEmbedded convention
  should normally be named with this as a suffix, since PV is meant to
  be set appropriately within the recipe, so follow this. In order to
  make this work we need to be able to have the version independent from
  the file name, so add a -V option to recipetool create to allow this
  to be specified.
* If -V is specified on the devtool add command line, ensure at PV gets
  set to include this version.

(From OE-Core rev: 2b997c41c6476a13bf516586d56a9051aceb93ec)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-01 17:08:50 +01:00
Paul Eggleton
a4fca1d523 recipetool: ensure git clone is standalone when extracting
If -x is specified and the specified URI was a git repository, we need
to ensure that the resulting clone is a sandalone and not one that has
pointers into the temporary fetch location or DL_DIR (since the git
fetcher does a local clone with -s). Split out the code from devtool
that already does this for "devtool modify -x" and reuse that.

(From OE-Core rev: fc47e8652ef32e7399f57c80593df90dc52d8b84)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-01 17:08:50 +01:00
Paul Eggleton
24e5ad268c recipetool: avoid second-level subdir when extracting
When -x was specified, we were getting the normal unpack subdirectory
which we don't really want - if there's only one subdirectory unpacked
then we should effectively copy just it to the extraction path, not as a
subdirectory under it.

(From OE-Core rev: 0eeecce96a0aa757c2c4c4ac4d82e3bcbf0f982c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-01 17:08:49 +01:00
Paul Eggleton
0a45ccb253 recipetool: fix duplicate licenses being picked up
If a license file matched more than one of the specifications (e.g.
COPYING.GPL) then it was being added to LIC_FILES_CHKSUM more than once.

(From OE-Core rev: 58316a2890782d206e9b9472ba483367f7560109)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20 11:27:49 +00:00
Paul Eggleton
5eb9ffe1e0 recipetool: fix regression caused by previous commit
Option was renamed in the setup code but not in the code that used it.

(From OE-Core rev: 4b4f2d4f2869d6d5d564cc1b2d733f41ab5a3b9b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-23 17:35:27 +00:00
Paul Eggleton
a3dfb70b62 recipetool: improve command-line help
Based on feedback from Scott Rifenbark <scott.m.rifenbark@intel.com>

(From OE-Core rev: 2c59b2b20c32577085645056e4cbf4f9c259e4d7)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-08 08:00:27 +00:00
Paul Eggleton
3f739a86a9 recipetool: replace version in S value
If a versioned recipe filename is specified, replace the version in the
value of S with ${PV} just as we do with SRC_URI to make future upgrades
of the recipe easier.

(From OE-Core rev: 0b3c81ed5bcce9b608c4f804496d769288fe8c04)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-03 14:53:50 +00:00
Christopher Larson
1b7b58ac97 recipetool: add python dependency scanning support
This uses a standalone python script named `pythondeps` which now lives in
scripts. It supports scanning for provided packages and imported
modules/packages, the latter via the python ast. It's not perfect, and
obviously conditional imports and try/except import blocks are handled
naively, listing all the imports even if they aren't all used at once, but it
gives the user a solid starting point for the recipe.

Currently `python_dir` from setup.py isn't being handled in an ideal way. This
is easily seen when testing the python-async package. There, the root of the
project is the async package, so the root has __init__.py and friends, and the
python provides scanning currently just assumes the basedir of that dir is the
package name in this case, which is not correct. Forthcoming patches will
resolve this.

(From OE-Core rev: cb093aca3b78f130dc7da820a8710342a12d1231)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-23 11:36:28 +00:00
Christopher Larson
e490d79fb7 recipetool: add python buildsystem support
- Handles distutils & setuptools.
- Supports pulling metadata from PKG-INFO, .egg-info, & setup.py (via two
  different mechanisms).
- Doesn't handle python 3 yet.

(From OE-Core rev: 6ff693c71d97b4bcfde198c84cf9fac7185cccfd)

Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-23 11:36:28 +00:00
Paul Eggleton
5638ca2b94 scripts/recipetool: Add a recipe auto-creation script
Add a more maintainable and flexible script for creating at least the
skeleton of a recipe based on an examination of the source tree.
Commands can be added and the creation process can be extended through
plugins.

[YOCTO #6406]

(From OE-Core rev: fa07ada1cd0750f9aa6bcc31f8236205edf6b4ed)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-23 10:18:15 +00:00