Commit Graph

60 Commits

Author SHA1 Message Date
Paul Eggleton
3ec9a621d0 recipetool: record unknown license files
Add a comment to the recipe listing license files that were found but
not able to be identified, so that the user can find and examine them
by hand fairly easily.

Fixes [YOCTO #9882].

(From OE-Core rev: 4b7d1bf8172533e9ac91a49ade152a05e2ee4146)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26 08:56:32 +01:00
Paul Eggleton
f465039737 recipetool: create: support specifying a file as the local source
It is currently possible to specify a file (e.g. a tarball) on the local
disk as the source, but you have to know to put file:// in front of it.
There's really no need to force users to jump through that hoop if they
really want to do this so check if the specified source is a file and
prefix it with file:// if that's the case.

Also ensure the same works for "devtool add" at the same time.

(From OE-Core rev: 71350003790c38e84b0e525a71a2fe5d24e3d083)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-12 23:10:15 +01:00
Paul Eggleton
54486a1ac3 recipetool: create: fix handling of github URLs
For a while now, Github hasn't been advertising a specific repository
URL since cloning the web URL with git works. Armed with this knowledge
and fully expecting people to just paste the github URL, we need to
handle this situation specially. If it looks like a github URL to the
root of a repository then treat it as a git repository instead of a
normal https URL to be fetched by the wget fetcher.

(From OE-Core rev: 7998dc3597657229507e5c140fceef1e485ac402)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-12 23:10:15 +01:00
Paul Eggleton
1063622cc6 recipetool: create: drop unused convert_pkginfo() function
Code cleanup, no functional changes - this code was never used.

(From OE-Core rev: 397b76c7f26e38e761b94b1f7987aafd55048e10)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:24 +01:00
Paul Eggleton
75f1a0ed42 recipetool: create: avoid decoding errors with Python 3
We're opening source files with the default encoding (utf-8) but we
can't necessarily be sure that they are UTF-8 clean - for example,
recipetool create ftp://mama.indstate.edu/linux/tree/tree-1.7.0.tgz
prior to this patch resulted in a UnicodeDecodeError. Use the
"surrogateescape" mode to avoid this.

Fixes [YOCTO #9822].

(From OE-Core rev: 50fcd9d1b9a20d49bc873467a82a071f2f2f8b5a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:24 +01:00
Ed Bartosh
0d7db8e732 scripts: Fix urlparse imports for python3
Used urllib.parse instead of urlparse to make code
working in python 3.

(From OE-Core rev: 0a064f2216895db0181ee033a785328e704ddc0b)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:01 +01:00
Ed Bartosh
3ee70cb725 scripts: Fix encoding errors for python3
Moved call of decode('utf-8') as close as possible to
call of subprocess API to avoid calling it in a lot of
other places.

Decoded binary data to utf-8 where appropriate to fix devtool
and recipetool tests in python 3 environment.

(From OE-Core rev: 30d02e2aa2d42fdf76271234b2dc9f37bc46b250)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:01 +01:00
Ed Bartosh
ed7abe6b9a scripts: Replace basestring -> str for python3
Python 3 doesn't have basestring type as all string
are unicode strings.

(From OE-Core rev: e8cfab060f4ff3c4c16387871354d407910e87aa)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:01 +01:00
Ed Bartosh
7eab022d4b scripts: Fix deprecated dict methods for python3
Replaced iteritems -> items, itervalues -> values,
iterkeys -> keys or 'in'

(From OE-Core rev: 25d4d8274bac696a484f83d7f3ada778cf95f4d0)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:01 +01:00
Paul Eggleton
49557a5e9d recipetool / devtool: set a fixed SRCREV by default when fetching from git
If fetching source from a git repository, typically within OpenEmbedded
we encourage setting SRCREV to a fixed revision, so change to do that by
default and add a  -a/--autorev option to use "${AUTOREV}" instead.

(From OE-Core rev: 000480c42797dd2f03ebc3bc6d1dabfc6a7b75f5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-01 12:38:41 +01:00
Paul Eggleton
0a7c699b33 recipetool: create: use ${BP} for subdir for binary packages
If we use ${BP} for the subdirectory, the default value of S will work
rather than having to have an ugly value derived from the package
file name in both places. This does mean that we have to assume the
default though (we can't just let the normal logic work because the
value of BP is the default until later on, so the replacement doesn't
work).

(From OE-Core rev: 13bc2438d61c345a8f229b9d83bf36a14d08916f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-01 12:38:41 +01:00
Paul Eggleton
eb09735737 recipetool: create: extract variable values from .deb/.ipk/.rpm
Extract the metadata from package files and use it to set variable
values in the recipe (including recipe name and version, LICENSE,
SUMMARY, DESCRIPTION, SECTION and HOMEPAGE). For LICENSE we take care
not to step on any value determined by our license file scan; if there
is one we simply add a comment above the LICENSE setting so the user can
resolve it.

(From OE-Core rev: 19e6b661d38c5ae9b19d6340762c289830baba59)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-01 12:38:41 +01:00
Paul Eggleton
2b510f5e01 recipetool: create: support extracting SUMMARY and HOMEPAGE
Allow plugins to set any variable value through the extravalues dict,
and use this to support extracting SUMMARY and HOMEPAGE values from spec
files included with the source; additionally translate "License:" to a
comment next to the LICENSE field (we have our own logic for setting
LICENSE, but it will often be useful to see what the spec file says if
one is present).

Also use the same mechanism for setting the same variables for node.js
modules; this was already supported but wasn't inserting the settings in
the appropriate place in the file which this will now do.

(From OE-Core rev: 91fc35ff5e89aa6d4c4ad945e45406fb4f71018e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-01 12:38:41 +01:00
Paul Eggleton
8f275ff0d8 recipetool: create: fix falling back to declared license for npm packages
Fix two problems falling back to the "license" field from package.json
when no license file is present:
1) The function that was supposed to return the license field value was
   always explicitly returning None, and this was never noticed (because
   the test cases never exercised the fallback as they provided license
   files for each module).
2) Fix the main package not falling back because it had a default of an
   empty list, which evaluates to '' instead of 'Unknown'.

(From OE-Core rev: 59381a9450949ce6b4b03adb717e950b999830f3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-06 10:31:14 +01:00
Paul Eggleton
600b7007f6 recipetool: create: don't create extra files directory unconditionally
The extra directory next to the recipe should only be created if there
are files to put into it; currently only the npm plugin does this. I
didn't notice the issue earlier because the test was actually able to
succeed under these circumstances if the recipe file came first in the
directory listing, which was a fault in my original oe-selftest test;
apparently on some YP autobuilder machines the order came out reversed.

With this change we can put the oe-selftest test that highlighted the
issue back to the way it was, with an extra check to reinforce that only
a single file should be created.

(From OE-Core rev: b8b778345eb0997c2cd952a1f61fdd2050b6b894)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-20 23:12:25 +00:00
Paul Eggleton
309b2e6c57 recipetool: create: support creation of additional files by plugins
Allow plugins to create additional files to go alongside the recipe. The
plugins don't know what the output filename is going to be, so they need
to put the files in a temporary location and add them to an "extrafiles"
dict within extravalues where the destination filename is the key and
the temporary path is the value.

devtool add was also extended to ensure these files get moved in and
preserved upon reset if they've been edited by the user.

(From OE-Core rev: 334b9451111b7e3efbb43b3a4eecebcab8ec6f0e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 17:00:29 +00:00
Paul Eggleton
2279eb2a4c recipetool: create: check if npm available if npm:// URL specified
If the user specifies an npm:// URL then the fetcher needs npm to be
available to run, so check if it's available early rather than failing
later.

(From OE-Core rev: a08d12ad867c292f7474731a0fe5e51e712446d6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 17:00:29 +00:00
Paul Eggleton
91455005b6 recipetool: create: split npm module dependencies into packages
Rather than rolling all of an npm module's dependencies into the same
package, split them into one module per package, setting the SUMMARY and
PKGV values from the package.json file for each package. Additionally,
mark each package with the appropriate license using the license
scanning we already do, falling back to the license stated in the
package.json file for the module if unknown. All of this is mostly in
aid of ensuring all modules and their licenses now show up in the
manifests for the image.

Additionally we set the main LICENSE value more concretely once we've
calculated the per-package licenses, since we have more information at
that point.

(From OE-Core rev: 8226805f83d21e7c1d2ba21969f3e8ee4b137496)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 17:00:29 +00:00
Paul Eggleton
d46827cfd3 recipetool: create: add license file crunching
Matching license texts directly to md5sums only goes so far. Some
licenses make the copyright statement an intrinsic part of the license
statement (e.g. MIT) which of course varies between projects. Also,
people often seem to take standard license texts such as GPLv2 and
reformat them cosmetically - re-wrapping lines at a different width or
changing quoting styles are seemingly popular examples. In order to
match license files to their actual licenses more effectively, "crunch"
out these elements before comparing to an md5sum. (The existing plain
md5sum matching has been left in since it's a shortcut, and our list of
crunched md5sums isn't a complete replacement for it.)

As always, this code isn't providing any guarantees (legal or otherwise)
that it will always get the license correct - as indicated by the
accompanying comments the LICENSE values it writes out to the recipe are
indicative and you should verify them yourself by looking at the
documentation supplied from upstream for the software being built if you
have any concerns.

(From OE-Core rev: 553bb4ea5d51be5179e7d8c019740cf61ece76ea)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 17:00:29 +00:00
Paul Eggleton
3fd244b94f recipetool: create: match *LICENSE* as a license file
For example, this picks up a file named MIT-LICENSE.txt.

(From OE-Core rev: 103b4d26b340cbdf70bf43906e293f3497671fdc)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 17:00:29 +00:00
Paul Eggleton
2b6a35212d recipetool: create: improve mapping for autotools program macros
Make the following improvements to mapping items specified in
AC_CHECK_PROG, AC_PATH_PROG and AX_WITH_PROG to recipes/classes:

* Produce a map of native recipe -> binary for all binaries currently in
  STAGING_BINDIR_NATIVE and use this when mapping items
* Add some more entries to the class map
* Ignore autotools binaries since they are covered by the inherit of
  autotools
* Ignore coreutils-native since that would almost always be a bogus
  dependency

(From OE-Core rev: 5614c5ae6a004d4367eccc34dd3cc7ee61fb7e57)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 17:00:29 +00:00
Paul Eggleton
524ee088b3 recipetool: create: improve CMake package mapping
* Package names are actually case sensitive near as I can tell, so we
  shouldn't be lowercasing them everywhere.
* Look for CMake packages in pkgdata and map those back to recipes,
  so we aren't dependent on the hardcoded mappings (though those are
  still preserved).
* Avoid duplicates in the unmapped package list

(From OE-Core rev: 2ddad52ccca07245eea43d9b844c6c7d4b667ca3)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-21 09:32:43 +00:00
Paul Eggleton
a5095d146d recipetool: create: set S when we set SRC_URI from local git repo
If you specify a local directory which happens to be a git repository
with an origin remote (and it is in fact remote), we can use that for
SRC_URI as implemented by OE-Core revision
b143d414846854dc8b3e0a47358daf5646eded38, however we also need to set S
if the recipe is going to be of any use fetching from that SRC_URI
later.

(From OE-Core rev: 8bbbd2d63f1bc752f9a30054a089dc2caf5fd84c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:33:03 +00:00
Paul Eggleton
ca5a36cbd1 recipetool: create: convert http git URLs that don't end in .git but contain /git/
When recipetool create is given a URL that starts with http(s):// and
contains /git/, such as the URLs at git.yoctoproject.org, it's fairly safe
to assume it's a git repository and not something that should be fetched
with wget, so rewrite the URL.

(From OE-Core rev: 3ca04757a670e8b6f78799cc0454d75691809ac4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:33:03 +00:00
Paul Eggleton
4c71afb378 recipetool: create: ensure URL parameters don't make it into the name
When auto-detecting the name for a recipe from the URL, strip off any
parameters (";name=value...") before parsing the URL, otherwise this
just ends up in the recipe name which is undesirable.

(From OE-Core rev: d3c46b5d0abd56bcadd4f2f1ef985f13d67f605b)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:33:03 +00:00
Paul Eggleton
f767757b3e recipetool: create: determine name/version from github/bitbucket URLs
Both BitBucket and GitHub provide "release" downloads that unfortunately
don't include the actual project name. For recipetool this means we have
to have special handling for these URLs or else the current code assumes
the name of the project is something like "v0.9.1". Borrow and adapt
some code from autospec to do this, and put it in its own function for
clarity.

(From OE-Core rev: e8435fde7f82c39cc0efe1d4bf043b29f63898e9)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 12:33:03 +00:00
Paul Eggleton
c1f4e929c9 recipetool: create: better fix for fetch error handling
I was a little bit hasty in OE-Core revision
c2cc5abe34169eae92067d97ce1e747e7c1413f5 - it turns out BitBake's
fetcher code is not consistent in whether it logs something useful or
not; when fetching from an http URL it does but with a git repository
it doesn't. In advance of any major reworking of fetch error handling in
BitBake, let's just print the text of the exception and then we know we
have shown something to the user.

Additionally, we were only catching FetchException here but there are
several other classes of exception that the fetcher can raise (e.g.
MalformedUrl); catch the parent BBFetchException class instead so we
avoid tracebacks for those other classes as well.

(From OE-Core rev: 578d3873a6415c9203c185c21cff472f7d2dab02)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:33 +00:00
Paul Eggleton
10c8d14d1a recipetool: create: fix extraction of name from URLs ending in /
If the URL ends in a / then we want to strip that off the path we split
out of the URL before calling os.path.basename() on it.

(From OE-Core rev: 308189beda8a31541481d09e3d5e86187e843d8d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:33 +00:00
Paul Eggleton
b307e0a604 recipetool: create: extract SRC_URI from local git repositories
If you specify a local directory which happens to be a git repository
with an origin remote (and it is in fact remote), we can use that for
SRC_URI rather than leaving it blank in the recipe.

(From OE-Core rev: b143d414846854dc8b3e0a47358daf5646eded38)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:33 +00:00
Paul Eggleton
50e40fc91f devtool / recipetool: support specifying a subdirectory within the fetched source
Sometimes you don't want to build an entire project, just a subdirectory
of it; add a --src-subdir option to make that easier. (We still look for
a single subdirectory in what gets unpacked, e.g. what you might find
within a tarball, so whatever you specify with this option is added onto
the end of that.)

(From OE-Core rev: 59682d78f95732e014f78f13e0a05f843860d9bb)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-24 09:40:33 +00:00
Paul Eggleton
c1492c4e4f recipetool: create: add a couple more license checksums
I found these when I was looking at libftdi and they seem to be
generic enough to show up in at least a couple of other packages so I
figure I'll add them.

(From OE-Core rev: 9fa3ff44e05930d4dfa153db777077e747ecbf45)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:53 +00:00
Paul Eggleton
2c8c9fe3b4 recipetool: create: add basic support for extracting dependencies from cmake
Add support for extracting dependencies from CMakeLists.txt. There's
still a bunch of things missing that are outside the scope of OE-Core
and we still lack a proper extension mechanism, but this is a good
start.

This also adds an oe-selftest test to exercise the new code a bit.

Implements [YOCTO #7635].

(From OE-Core rev: 77e73e6930381fdbd6e78d3913d6467572e16568)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:53 +00:00
Paul Eggleton
3eb397fab6 recipetool: create: force GL libraries to virtual/*
We want to specify dependencies on virtual/* rather than whatever
library is selected in the current configuration.

(From OE-Core rev: e1ac0c45b27ded9962edaf34597f827d0b41ba82)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:52 +00:00
Paul Eggleton
726dbda396 recipetool: create: move dependency mapping code to RecipeHandler
Some refactoring to allow access to the library/header/pkg-config
mappings and the DEPENDS / unmapped dependency output code from other
classes than AutotoolsRecipeHandler.

(From OE-Core rev: 40c10d998b90dd59c6d36c28f8ba11ec598bfa0f)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:51 +00:00
Paul Eggleton
788e4bb23e recipetool: create: fix overzealous mapping of git URLs
The regex for detecting git URLs was unanchored, leading to it matching
where it shouldn't have. An example of where this went wrong was
http://taglib.github.io/releases/taglib-1.9.1.tar.gz.

(From OE-Core rev: bacff751c88b680fbfb07843b18c59c8bc80a9ea)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-19 17:24:51 +00:00
Paul Eggleton
ec9016821d devtool: add: support adding a native variant
Sometimes you need to build a variant of a recipe for the build
host as well as for the target (i.e. BBCLASSEXTEND = "native"); add a
 --also-native command line option to "recipetool create" that enables
this and plumb it through from an identical option for "devtool add".

(We could conceivably do the same for nativesdk, but I felt it might be
confusing within the context of the extensible SDK, where nativesdk
isn't really relevant to the user.)

(From OE-Core rev: f3bea83db173cce921a9a30f04e88b7e3ed98854)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton
5ef716c280 recipetool: create: support creating standalone native/nativesdk recipes
If the recipe name ends with -native then we should inherit native;
likewise if it starts with nativesdk- then inherit nativesdk.

(Note that the recipe name must actually be specified by the user in
order to trigger this - we won't do it based on any name auto-detected
from e.g. the tarball name.)

Since we're doing this based on the name, "devtool add" will also gain
this functionality automatically.

(From OE-Core rev: a216fb1f5953327790dce3d2c1a9af616c0b410d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton
1e503c0a51 recipetool: create: lower case name when determining from filename
As a matter of general convention we expect recipe names to be lower
case; in fact some of the packaging backends insist upon it. Since in
this part of the code we're auto-determining the name, we should convert
the name to lowercase if it's not already so that we're following
convention and avoiding any validation issues later on.

(From OE-Core rev: 64986decbc11afa0d1e11251f5f7adcba1860d19)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-11 15:41:46 +00:00
Paul Eggleton
891673101e recipetool: create: fix error when extracting source to a specified directory
Having fetched the source and unpacked it to a temporary directory, we
then move part of it to the destination directory, or if the source is at
the top level we move the whole temporary directory, but in the latter
case we were later attempting to delete the temporary directory which no
longer existed. Clear out the variable so that doesn't happen.

(From OE-Core rev: 91714a52e91cddba5a16c73cf5765d1f47f7856c)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:14 +00:00
Paul Eggleton
ebe5f0b872 recipetool: create: basic extraction of name/version from filename
Often the filename (e.g. source tarball) contains the name and version
of the software it contains.

(This isn't intended to be exhaustive, just to catch the common case.)

(From OE-Core rev: 944eacfb849ee69b41e12c9de4f264406281ac6a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton
db5f9645ad recipetool: create: support extracting name and version from build scripts
Some build systems (notably autotools) support declaring the name and
version of the program being built; since we need those for the recipe
we can attempt to extract them. It's a little fuzzy as they are often
omitted or may not be appropriately formatted for our purposes, but it
does work on a reasonable number of software packages to be useful.

(From OE-Core rev: 3b3fd33190d89c09e62126eea0e45aa84fe5442e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton
6a7661b800 recipetool: create: set up priority system for recipe handlers
Sometimes we want to force one handler to run before another; if the two
handlers are in different plugins that's difficult without some kind of
priority number, so add one and sort by it.

(From OE-Core rev: 0219d4fb9cefcee635387b46fc1d215f82753d92)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton
38803e38d6 recipetool: create: detect when specified URL returns a web page
If the user specifies a URL that just returns a web page, then it's
probably incorrect (or broken); attempt to detect this and show an error
if it's the case.

(From OE-Core rev: 83b1245b2638eb5d314fe663d33cd52a776a34a7)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:13 +00:00
Paul Eggleton
c2f17428b8 recipetool: create: avoid traceback on fetch error
If a fetch error occurs, the fetcher already prints a reasonable error -
we don't need the traceback as well, so catch that and exit if it
occurs.

(From OE-Core rev: c2cc5abe34169eae92067d97ce1e747e7c1413f5)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:12 +00:00
Paul Eggleton
470f20bc02 recipetool: create: handle https://....git URLs
When you grab a URL for a github repository you'll almost certainly find
it in https://github.com/path/to/repository.git format; but bitbake's
fetcher can't handle that because it'll see https:// at the start and
assume it should use wget to fetch it. If the URL starts with http:// or
https:// and the path part ends with .git then assume it's a git
repository and adjust it accordingly.

(From OE-Core rev: bdbc4cf41d30eddb8a9ed882dedcc1670ce8fdd6)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-28 09:25:12 +00:00
Paul Eggleton
0cc3810299 recipetool: make plugin registration function name consistent with devtool
This should have been register_commands rather than register_command;
I used register_commands in devtool so lets change this here to be
consistent with that. (Since this is extensible through layers though we
need to remain compatible with the old name, so fall back to that if the
new function name isn't there.)

(From OE-Core rev: 1047f6592ac81643cd847f104da766dc4a4c81ea)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:05 +00:00
Paul Eggleton
8cacd228e6 recipetool: create: fix change in path structure if --extract-to path exists
If the directory specified by --extract-to exists, because we were using
shutil.move() to move the temporary extracted directory to the specified
path, a subdirectory was being created under that directory instead of
moving the contents, which was a different result than if the directory
didn't previously exist. We could try to always move the contents but
that's complicated when any symlinks are involved; the simplest thing is
just to remove the directory (which should be empty anyway) before
moving the temporary directory across in its place.

(From OE-Core rev: 2880bd23b471c1966661b9f05726faf60f9c0e7e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-07 00:09:10 +01:00
Paul Eggleton
c4181c6e7e devtool / recipetool: add handling for binary-only packages
Add a means of creating recipes for package files or archives that
contain a directory structure to be installed verbatim, for example an
rpm file. (We mostly just re-use bin_package here and skip some of the
normal build system checks.) This support is available in "recipetool
create" and "devtool add" which wraps the former.

(From OE-Core rev: 9e0a6b2e6f16185f8032d36b77d40802bc388987)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23 09:53:20 +01:00
Paul Eggleton
210d959023 recipetool: create: fix handling of URIs containing #
The # character in a URI denotes a fragment; we don't care about this
since it is never supposed to be sent to the server, so remove it from
the URI before actually trying to fetch it or use it in SRC_URI within
the recipe.

(This has come up because download links on pypi.python.org seem to have
a fragment containing the md5sum of the download; without stripping this
off the fetcher will choke on it.)

(From OE-Core rev: 58dc726808817210764eab963a4d453dc4ff49aa)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23 09:53:16 +01:00
Paul Eggleton
a35ad723ce recipetool: create: fix creating empty shell functions
The shell considers empty functions to be a syntax error, so for
template shell functions that contain only comments (or no lines at all)
then add a : to act as a no-op which avoids the syntax error.

(From OE-Core rev: ff14d9e5b935b99b2efde479515e54c02ba58f6e)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23 09:53:16 +01:00