A flaw was found when I run:
$ recipetool create "ssh://git@xxx.xxx:7999/xxx.git"
the url turned out to be: "git://git@xxx.xxx/7999/xxx.git;protocol=ssh"
after parsing, the port number was parsed as part of the path, this is
definitely wrong and lead to fetching failures.
This issue could be fixed in reformat_git_uri, by filtering out port
numbers when formatting ":".
(From OE-Core rev: 4290e04b69360b5e1da9f37166015e30f66cb335)
Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you specify 'tag=' for a git URL and passed to recipetool create, you
will get into Bitbake expansion error shown below:
----- snip -----
$ devtool add --version 2.4.2 mbedtls "git://github.com/ARMmbed/mbedtls;tag=mbedtls-2.4.2"
...
bb.data_smart.ExpansionError: Failure expanding variable SRCPV, expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception FetchError: Fetcher failure: Conflicting revisions (abeccb9dbd7e19ae91ac50e1edd3803111c5f9b6 from SRCREV and mbedtls-2.4.2 from the url) found, please specify one valid value
----- snip -----
Assuming the tag is valid, we should get the tag commit hash and
drop the usage of 'tag=' from SRC_URI. By using a commit hash
corresponding to the tag will prevent bitbake from accessing
remote repository in order to expand SRCPV.
(From OE-Core rev: 53f8effa3eb07dc7035ff9933e7918318f242579)
Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
github archive URLs are not guaranteed to be stable [1] and thus we
should show a warning if a user specifies one to recipetool create (or
devtool add).
[1] http://lists.openembedded.org/pipermail/openembedded-core/2017-September/142519.html
(From OE-Core rev: 7e84a777aa924a237b4e604120ebf8a4b3ba53b2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the source tree happens to contain a kernel module as an example, a
test or under a "contrib" directory then we shouldn't be picking it up
and making the determination that the entire thing is a kernel module.
An example that triggered this is zstd, which ships a kernel module
under contrib/linux-kernel:
https://github.com/facebook/zstd
(From OE-Core rev: c2b3154158d4bb0855daa56477393341139d4cf9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We shouldn't be passing a relative path to the plugins if that's what's
been specified on the recipetool command line.
(From OE-Core rev: 949067384c5166058ebc76f931cc492dad1db645)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A recipe added with "devtool add" requires to be able to take precedence on recipes
previously defined with PREFERRED_PROVIDER.
By adding the parameter "--provides" to "devtool add" it is possible to specify
an element to be provided by the recipe. A devtool recipe can override a previous
PREFERRED_PROVIDER using the layer configuration file in the workspace.
E.g.
devtool add my-libgl git@git://my-libgl-repository --provides virtual/libgl
[YOCTO #10415]
(From OE-Core rev: adeea2fe6895898a5e6006e798898f0f5dabd890)
Signed-off-by: Juan M Cruz Alcaraz <juan.m.cruz.alcaraz@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since OE-Core revision 9a47a6690052ef943c0d4760630ee630fb012153 the
mechanism we were using to suppress the warnings about
NPM_LOCKDOWN and NPM_SHRINKWRAP not being set on the first fetch of the
source is no longer available since we are using the normal fetch/unpack
tasks to do the job. Use the newly added noverify parameter to suppress
the warnings again.
(From OE-Core rev: cb083b6f5f6e909b7c85548bcb1a92ca34d0c18a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you're fetching from an SCM other than git (for example subversion or
mercurial) then we need to use a different prefix for the SRCPV in PV
instead of +git.
(From OE-Core rev: ad1200c8729f21b325d347649f9dd5e5598de93e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The branch and tag handling code that was recently added in OE-Core revs
ecca596b75cfda2f798a0bdde75f4f774e23a95b and
3afdcbdc9a3e65bc925ec61717784ffec67d529d is specific to git, so only
apply it when we're fetching from a git URL.
(From OE-Core rev: 5d4bfe6cf788ce971a2e9419bc13492153023681)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
At the moment when fetching source from a git repository you have to
know that you can specify the revision and branch in the URL with
';rev=' and ';branch=' respectively, and you can also get thrown off by
the shell splitting on the ; character if you forget to surround the URL
in quotes. Add explicit -S/--srcrev and -B/--srcbranch options
(consistent with devtool upgrade) to make this easier for the user to
discover and use. (The rev and branch URL parameters will continue to
work, however.)
(From OE-Core rev: 2d86cac853d6daa496c0315a5cb0662ebf1165b0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were being a bit prescriptive in setting LICENSE and
LIC_FILES_CHKSUM. We can't always trust what's in the metadata
accompanying some source which plugins will almost always be pulling
from, however we do want to allow plugins to set the LICENSE and
LIC_FILES_CHKSUM values. Merge what we find in our license file scan
with what the plugin sends back.
Additionally, plugins can now add a "license" item to the handled list
in order to inhibit the normal LICENSE / LIC_FILES_CHKSUM handling if
they have already taken care of it completely.
Thanks to Mark Horn <mark.d.horn@intel.com> for prompting, testing and
fixing this patch.
(From OE-Core rev: 1df60b09f7a60427795ec828c9c7180e4e52f98c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously if we were able to auto-determine the name from the URL, that
took precedence over any name that might be set in extravalues by a
plugin. Some plugins might be able to get a better idea of the name and
thus we should move defaulting of the name further down after the
plugins have had a chance to set it.
(From OE-Core rev: 3bb979c13463705c4db6c59034661c4cd8100756)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
During recipe creation, it seems that the automation for replacing
${PV} at the SRCURI for tag, (e.g mbed-tls-${PV}) is causing some
issue due to PV assuming it's a git source. A fix is implemented in
this patch to resolve this issue.
(From OE-Core rev: 9d3ec76c1b7dd75d904f5ff47297de0fb65b21c2)
Signed-off-by: Stanley Phoong <stanley.cheong.kwan.phoong@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a git URL is passed to recipetool create with a tag=, recipetool
should handle it assuming that the tag is valid.
[YOCTO #11393]
(From OE-Core rev: 3afdcbdc9a3e65bc925ec61717784ffec67d529d)
Signed-off-by: Stanley Phoong <stanley.cheong.kwan.phoong@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This change is to improve the buildability of the recipe created by
recipetool and devtool.
When recipetool create is run on a git URL and a revision specified
that is not on master, and "branch=" isn't already in the URL, then
we should get the correct branch and append the branch to the URL.
If the revision was found on multiple branches and 'master' is not
in the list, we will display error to inform user to provide a
correct branch and exit.
[YOCTO #11389]
(From OE-Core rev: ecca596b75cfda2f798a0bdde75f4f774e23a95b)
Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When creating new recipes, we are almost certainly fetching a new
source rather that something that has already been fetched. I have
disable PREMIRRORS and MIRRORS settings in the recipe that created
by devtool while leaving an option for users to enable them manually
if needed. Since devtool already has this options, we need to ensure
that recipetool is able to handle the options passed from devtool.
(From OE-Core rev: 091cee2bdc2378a3425a4ef8558d03e6f9c021ff)
Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have two variables here, srcuri and fetchuri. srcuri is what
eventually ends up in the recipe, whereas fetchuri is what we actually
pass to the fetcher when we fetch the source within recipetool -
sometimes these need to be different particularly for an upcoming patch
to handle automatically setting the branch parameter. In OE-Core
revision 9a47a6690052ef943c0d4760630ee630fb012153 I erroneously changed
the call to scriptutils.fetch_url() to pass srcuri instead of fetchuri -
this likely didn't have any ill effect, but change it back to passing
fetchuri to match the original intent.
(From OE-Core rev: b66b73bcf5ee7e4488970576fdc31dfa25b35f5e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Across devtool and recipetool we had an ugly set of code for ensuring
that we can call an npm binary, and much of that ugliness was a result
of not being able to run build tasks when tinfoil was active - if
recipetool found that npm was required and we didn't know beforehand
(e.g. we're fetching from a plain git repository as opposed to an npm://
URL where it's obvious) then it had to exit and return a special result
code, so that devtool knew it needed to build nodejs-native and then
call recipetool again. Now that we are using real build tasks to fetch
and unpack, we can drop most of this and move the code to the one place
where it's still needed (i.e. create_npm where we potentially have to
deal with node.js code in a plain source repository).
(From OE-Core rev: 8450de16ddb02d863204b411a94c6d84e0f88817)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that we have the ability to run the tasks in a more standard context
through tinfoil, change recipetool's fetching code to use that to fetch
files using it. This has the major advantage that any dependencies of
do_fetch and do_unpack (e.g. for subversion or npm) will be handled
automatically. This also has the beneficial side-effect of fixing a
recent regression that prevented this fetch operation from working with
memory resident bitbake.
Also fix devtool's usage of fetch_uri() at the same time so that we can
completely replace it.
Fixes [YOCTO #11710].
(From OE-Core rev: 9a47a6690052ef943c0d4760630ee630fb012153)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When dealing with package files (.rpm, .ipk etc.) we need to unpack them
ourselves to get the metadata, which is thrown away when the fetcher
unpacks them. However, since we've already fetched the file once, I'm
not sure as to why I thought I needed to fetch it again - we can just
get the local path and then unpack it directly.
(From OE-Core rev: be45e9b17e9dbc8c2594d3a939be377ab0720a7c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you pointed recipetool at a URL that should be a tarball e.g.
https://tls.mbed.org/download/start/mbedtls-2.4.2-apache.tgz but instead
it returns an HTML page, we try to unpack it, gzip complains but the
operation doesn't seem to fail - instead we just get back an empty
source tree. Change the checks to account for this - if the source tree
is empty, check if the downloaded file in DL_DIR looks like an HTML file
and error accordingly if it is. If it's not, error out anyway because
no source was unpacked and it should have been (otherwise we just
blindly set up EXTERNALSRC for this which is pointless).
Fixes an aspect of [YOCTO #11407].
(From OE-Core rev: 8496113b63d5a5d1f99056610c0fdb972a6200d4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
recipetool seems to be mangling and stripping out the parameters for git
URI. This will fix this issue as well as resolve the conflict of
protocol parameter added by user. If a user adds their own protocol as
an argument, it'll be honored.
[YOCTO #11390]
[YOCTO #11391]
(From OE-Core rev: 0cd2fc8ca278ebaa76de95545eef26a07b350c8e)
Signed-off-by: Stanley Cheong Kwan, Phoong <stanley.cheong.kwan.phoong@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For git repositories in the absence of any other indicator, it's not an
unreasonable assumption that the name of the repository is the name of
the software package it contains, so use that as PN if we don't have
anything else.
(From OE-Core rev: ef73fa70f0955912b0da140922465a3c817424e9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If devtool is called with a URL to a source repository containing a
node.js module, we don't know that until recipetool has fetched it, and
due to the structure of the code we have to exit with a special code in
order to let devtool know it needs to build nodejs-native. We also want
to suppress the error message that recipetool would normally print under
these circumstances; there is already a mechanism for this but it wasn't
operative in the case where we're pointed to a source repository rather
than an npm:// URL, so create some plumbing so that we know to hide the
message.
(From OE-Core rev: 0c2d0fbb1c6c5b82183799eb7ef80074f86bcfc4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The change over to recipe specific sysroots means that we can no longer
get a known location simply from configuration for the npm binary - we
need to get the recipe sysroot for nodejs-native, look there for npm if
we need to check it's present, and add that to PATH when calling out to
npm. Unfortunately this means anywhere we need to get that path we have
to have parsed all recipes, otherwise we have no reliable way of
resolving nodejs-native. Thus we have to change recipetool create to
always parse all recipes (the structure of the code does not allow us to
do this conditionally).
In the worst case, if npm hasn't already been added to its own sysroot
and we are fetching from a source repository rather than an npm
registry, this gets a bit ugly because we end up parsing recipes three
times:
1) recipetool startup, which then fetches the code and determines it's
a node.js module, finds that npm isn't available and then exits with
a specific error to tell devtool it needs to build npm
2) when we invoke bitbake -c addto_recipe_sysroot nodejs-native
3) when we re-invoke recipetool
This code is badly in need of refactoring, but now is unfortunately not
the time to do that, so we're going to have to live with this ugliness
for now.
Fixes [YOCTO #10992].
(From OE-Core rev: acfdbd796c99882b8586023c8c6b848716105c8d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Web applications built using e.g. angular2, usually requires that the
packages in devDependencies are available.
Thus, add an option '--fetch-dev' to both devtool add and recipetool, to
add npm packages in devDependencies to DEPENDS.
(From OE-Core rev: f246f820d53b459596fde6758a09f7a0d7db7c4c)
Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that the datastore works dynamically we don't need the update_data calls
so we can just remove them. They're not actually done anything at all for
a while.
(From OE-Core rev: 8de0c5d3bd01919e2bf0394f9c485936d6098cec)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Numbers within SCM (e.g. git) URLs are extremely unlikely to be valid
version numbers - more likely they are just part of the name, thus don't
try to extract them and use them as the version - doing so causes pretty
bad behaviour within devtool:
--------- snip ---------
$ devtool add https://github.com/inhedron/libtr50
NOTE: Fetching git://github.com/inhedron/libtr50;protocol=https...
...
NOTE: Using default source tree path .../build/workspace/sources/libtr
...
RecursionError: maximum recursion depth exceeded while calling a Python object
--------- snip ---------
(This was because ${PV} was being substituted into the URL, but PV's
value was being set to include ${SRCPV}, so there was a circular
reference.)
(From OE-Core rev: 3427508b6ce865654f8bf01a6fc04b83c70315d3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.
Search made with the following regex: getVar ?\(( ?[^,()]*), True\)
(From OE-Core rev: 0a36bd96e6b29fd99a296efc358ca3e9fb5af735)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The .deb import feature did not import postinst, postrm, preinst, or
prerm functions. This change checks to see if those files exist, and
if so, adds the appropriate functions.
[ YOCTO #10421 ]
(From OE-Core rev: ebb73aa6ad920bfd6a23f8c20105d6bcf07dd3d5)
Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
recipetool sets the LICENSE value based on licenses detected from the
source tree. If there are multiple licenses then they were being
separated by spaces, but this isn't actually legal formatting and if
you're using "devtool add" you get a warning printed when devtool
parses the recipe internally.
Earlier I had made a conscious decision to do it this way since it's up
to the user to figure out whether the multiple licenses should all apply
(in which case they'd be separated with &) or if there is a choice of
license (in which case | is the correct separator). However, I've come
to the conclusion that we can just default to & and then the ugly
warning goes away, and it's the safest alternative of the two (and most
likely to be correct, since it's more common to have a codebase which is
made up of code with different licenses, i.e. all of them apply to the
combined work).
I've tweaked the comment that we add to the recipe to explicitly state
that we've used & and that the user needs to change that if that's not
accurate.
Fixes [YOCTO #10413].
(From OE-Core rev: ecac6aee8cf3313350b58c21012bcd67cfb915e4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running devtool add, instead of hiding the recipetool create
output, change it so that it's appropriate to show in the devtool
context and show it in real-time. This means that you get status output
such as when a URL is being fetched (though currently no progress
information.) recipetool create now has a hidden --devtool option to
enable this display mode.
(From OE-Core rev: 219aec8803de4ef04c514c87ecfb15359c9424a6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the user runs devtool add on an npm:// URL (or source tree that uses
node.js), and npm is not available, just build nodejs-native instead of
telling the user they need to do it; if that fails because there isn't
any such recipe (which would be the default, since it's not in OE-Core)
then produce a slightly more readable error message hinting at what the
user needs to do.
Note that this forces the use of nodejs-native rather than npm on the
host - this makes sense for two reasons: (1) we need it to be compatible
with nodejs for the target, and (2) we have to have a recipe for that
anyway, so allowing you to avoid having a recipe for the native version
isn't really beneficial.
There's a bit of a hack in here in order to allow this - for node.js
sources that aren't fetched via npm we don't know that they are that
until we've fetched and unpacked them, by which time we're inside
recipetool and have an active tinfoil instance that will prevent bitbake
being run. To avoid this being an issue, we allow recipetool to get to
the point where we know we need npm and then exit with a specific exit
code, at which point devtool can try to build it and then if that
succeeds, it will re-execute recipetool. This is definitely not ideal,
but it can't really be refactored and done properly until we do the
tinfoil2 refactoring; in the mean time though we still want to be
helpful to the user.
Fixes [YOCTO #10337].
(From OE-Core rev: f40662bde5aab158c4e4c3c3ff5e68665a4194a5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In keeping with making recipetool create / devtool add as easy to use as
possible, users shouldn't have to know how to reformat git short form ssh
URLs for consumption by BitBake's fetcher (for example
user@git.example.com:repo.git should be expressed as
git://user@git.example.com/repo.git;protocol=ssh ) - instead we should
just take care of that automatically. Add some logic in the appropriate
places to do that.
(From OE-Core rev: 78c672a72f49c4b6cfd8c247efcc676b0ba1681a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Filter out a plain "Licensed under the XXXX license" statement, as seen
in the capnproto project (and no doubt others).
(From OE-Core rev: ba4aa319fd49ee02ce2e30c2db0f3988c0e8833c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When creating a recipe for an existing local git clone, we attempt to
use the fetcher to determine if it supports the SRCREV variable.
Unfortunately running this code does a network check to get the latest
revision as a direct result of us using '${AUTOREV}' as a default value.
If you don't have a network connection this will of course fail. Rather
than have this block creating the recipe, catch the exception and just
guess from the URL.
Ultimately this should probably be fixed in the fetcher but for now this
will at least resolve the issue on this end.
(From OE-Core rev: f7e43f931d7d6019a3b2509b2b2635978fbbae36)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I ran into an example where recipetool was getting the name/version
completely wrong:
https://bitbucket.org/sortsmill/libunicodenames/downloads/libunicodenames-1.1.0_beta1.tar.xz
>From this it would create a libunicodenames-1.1.0-beta1_1.1.0-beta1.bb
file (likely because it couldn't split the file name and therefore took
all of it, then got the version from one of the files inside the
tarball). When this happens it's just irritating because you then have
to delete the recipe / run devtool reset and then run recipetool create
/ devtool add again and specify the version manually.
This patch is the result of systematically running the
determine_from_filename() function over the files on the Yocto Project
source mirror and my local downloads directory and fixing as many of the
generic issues as reasonably practical - it now gets the name and
version correct much more often. There are still cases where it won't,
but they are now in the minority.
(From OE-Core rev: 7b018b1d493a8d10fd02b8cc220990b191c87fe5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we output extra blank lines (because of some automated editing) then
it makes the output recipe look a bit untidy. You could argue that we
should simply have the editing code not do that, but sometimes we don't
have enough context there for that to be practical. It's simple enough
to just filter out the extra blank lines when writing the file, so just
do it that way.
(From OE-Core rev: cbebc9a2edf7d7a422ee5c71219e79e3b349de3b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you make adjustments to the source tree (as create_npm.py will be)
then you will need to re-run the license variable handling code at the
end so that we get all of the files that should go into
LIC_FILES_CHKSUM if nothing else. Split out the license variable
handling to a separate function in order to allow this.
(From OE-Core rev: f0d6f4b7e87ea781ac0dffcc8d0310570975811b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For debugging it's useful to be able to tell recipetool to keep the
temporary directory.
(From OE-Core rev: 480a6b745a85b2881e5cc1a0bbb572e3235ca008)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure we fetch submodules and set SRC_URI correctly when pointing to a
git repository that contains submodules.
(From OE-Core rev: 65d5cc62d4ecfc78ce4b37b3886a7fe5aa05a75e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The regex here needs to be anchored to the end or it'll match longer
URLs, which was exactly what I was trying to avoid. This regression was
introduced in OE-Core revision 7998dc3597657229507e5c140fceef1e485ac402.
Fixes [YOCTO #10023].
(From OE-Core rev: 9291c5d3c257d5ada7605dfe46ababda08f6d3c1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a comment to the recipe listing license files that were found but
not able to be identified, so that the user can find and examine them
by hand fairly easily.
Fixes [YOCTO #9882].
(From OE-Core rev: 4b7d1bf8172533e9ac91a49ade152a05e2ee4146)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is currently possible to specify a file (e.g. a tarball) on the local
disk as the source, but you have to know to put file:// in front of it.
There's really no need to force users to jump through that hoop if they
really want to do this so check if the specified source is a file and
prefix it with file:// if that's the case.
Also ensure the same works for "devtool add" at the same time.
(From OE-Core rev: 71350003790c38e84b0e525a71a2fe5d24e3d083)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For a while now, Github hasn't been advertising a specific repository
URL since cloning the web URL with git works. Armed with this knowledge
and fully expecting people to just paste the github URL, we need to
handle this situation specially. If it looks like a github URL to the
root of a repository then treat it as a git repository instead of a
normal https URL to be fetched by the wget fetcher.
(From OE-Core rev: 7998dc3597657229507e5c140fceef1e485ac402)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Code cleanup, no functional changes - this code was never used.
(From OE-Core rev: 397b76c7f26e38e761b94b1f7987aafd55048e10)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're opening source files with the default encoding (utf-8) but we
can't necessarily be sure that they are UTF-8 clean - for example,
recipetool create ftp://mama.indstate.edu/linux/tree/tree-1.7.0.tgz
prior to this patch resulted in a UnicodeDecodeError. Use the
"surrogateescape" mode to avoid this.
Fixes [YOCTO #9822].
(From OE-Core rev: 50fcd9d1b9a20d49bc873467a82a071f2f2f8b5a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used urllib.parse instead of urlparse to make code
working in python 3.
(From OE-Core rev: 0a064f2216895db0181ee033a785328e704ddc0b)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moved call of decode('utf-8') as close as possible to
call of subprocess API to avoid calling it in a lot of
other places.
Decoded binary data to utf-8 where appropriate to fix devtool
and recipetool tests in python 3 environment.
(From OE-Core rev: 30d02e2aa2d42fdf76271234b2dc9f37bc46b250)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python 3 doesn't have basestring type as all string
are unicode strings.
(From OE-Core rev: e8cfab060f4ff3c4c16387871354d407910e87aa)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If fetching source from a git repository, typically within OpenEmbedded
we encourage setting SRCREV to a fixed revision, so change to do that by
default and add a -a/--autorev option to use "${AUTOREV}" instead.
(From OE-Core rev: 000480c42797dd2f03ebc3bc6d1dabfc6a7b75f5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we use ${BP} for the subdirectory, the default value of S will work
rather than having to have an ugly value derived from the package
file name in both places. This does mean that we have to assume the
default though (we can't just let the normal logic work because the
value of BP is the default until later on, so the replacement doesn't
work).
(From OE-Core rev: 13bc2438d61c345a8f229b9d83bf36a14d08916f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extract the metadata from package files and use it to set variable
values in the recipe (including recipe name and version, LICENSE,
SUMMARY, DESCRIPTION, SECTION and HOMEPAGE). For LICENSE we take care
not to step on any value determined by our license file scan; if there
is one we simply add a comment above the LICENSE setting so the user can
resolve it.
(From OE-Core rev: 19e6b661d38c5ae9b19d6340762c289830baba59)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow plugins to set any variable value through the extravalues dict,
and use this to support extracting SUMMARY and HOMEPAGE values from spec
files included with the source; additionally translate "License:" to a
comment next to the LICENSE field (we have our own logic for setting
LICENSE, but it will often be useful to see what the spec file says if
one is present).
Also use the same mechanism for setting the same variables for node.js
modules; this was already supported but wasn't inserting the settings in
the appropriate place in the file which this will now do.
(From OE-Core rev: 91fc35ff5e89aa6d4c4ad945e45406fb4f71018e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix two problems falling back to the "license" field from package.json
when no license file is present:
1) The function that was supposed to return the license field value was
always explicitly returning None, and this was never noticed (because
the test cases never exercised the fallback as they provided license
files for each module).
2) Fix the main package not falling back because it had a default of an
empty list, which evaluates to '' instead of 'Unknown'.
(From OE-Core rev: 59381a9450949ce6b4b03adb717e950b999830f3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The extra directory next to the recipe should only be created if there
are files to put into it; currently only the npm plugin does this. I
didn't notice the issue earlier because the test was actually able to
succeed under these circumstances if the recipe file came first in the
directory listing, which was a fault in my original oe-selftest test;
apparently on some YP autobuilder machines the order came out reversed.
With this change we can put the oe-selftest test that highlighted the
issue back to the way it was, with an extra check to reinforce that only
a single file should be created.
(From OE-Core rev: b8b778345eb0997c2cd952a1f61fdd2050b6b894)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow plugins to create additional files to go alongside the recipe. The
plugins don't know what the output filename is going to be, so they need
to put the files in a temporary location and add them to an "extrafiles"
dict within extravalues where the destination filename is the key and
the temporary path is the value.
devtool add was also extended to ensure these files get moved in and
preserved upon reset if they've been edited by the user.
(From OE-Core rev: 334b9451111b7e3efbb43b3a4eecebcab8ec6f0e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the user specifies an npm:// URL then the fetcher needs npm to be
available to run, so check if it's available early rather than failing
later.
(From OE-Core rev: a08d12ad867c292f7474731a0fe5e51e712446d6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than rolling all of an npm module's dependencies into the same
package, split them into one module per package, setting the SUMMARY and
PKGV values from the package.json file for each package. Additionally,
mark each package with the appropriate license using the license
scanning we already do, falling back to the license stated in the
package.json file for the module if unknown. All of this is mostly in
aid of ensuring all modules and their licenses now show up in the
manifests for the image.
Additionally we set the main LICENSE value more concretely once we've
calculated the per-package licenses, since we have more information at
that point.
(From OE-Core rev: 8226805f83d21e7c1d2ba21969f3e8ee4b137496)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Matching license texts directly to md5sums only goes so far. Some
licenses make the copyright statement an intrinsic part of the license
statement (e.g. MIT) which of course varies between projects. Also,
people often seem to take standard license texts such as GPLv2 and
reformat them cosmetically - re-wrapping lines at a different width or
changing quoting styles are seemingly popular examples. In order to
match license files to their actual licenses more effectively, "crunch"
out these elements before comparing to an md5sum. (The existing plain
md5sum matching has been left in since it's a shortcut, and our list of
crunched md5sums isn't a complete replacement for it.)
As always, this code isn't providing any guarantees (legal or otherwise)
that it will always get the license correct - as indicated by the
accompanying comments the LICENSE values it writes out to the recipe are
indicative and you should verify them yourself by looking at the
documentation supplied from upstream for the software being built if you
have any concerns.
(From OE-Core rev: 553bb4ea5d51be5179e7d8c019740cf61ece76ea)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For example, this picks up a file named MIT-LICENSE.txt.
(From OE-Core rev: 103b4d26b340cbdf70bf43906e293f3497671fdc)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make the following improvements to mapping items specified in
AC_CHECK_PROG, AC_PATH_PROG and AX_WITH_PROG to recipes/classes:
* Produce a map of native recipe -> binary for all binaries currently in
STAGING_BINDIR_NATIVE and use this when mapping items
* Add some more entries to the class map
* Ignore autotools binaries since they are covered by the inherit of
autotools
* Ignore coreutils-native since that would almost always be a bogus
dependency
(From OE-Core rev: 5614c5ae6a004d4367eccc34dd3cc7ee61fb7e57)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Package names are actually case sensitive near as I can tell, so we
shouldn't be lowercasing them everywhere.
* Look for CMake packages in pkgdata and map those back to recipes,
so we aren't dependent on the hardcoded mappings (though those are
still preserved).
* Avoid duplicates in the unmapped package list
(From OE-Core rev: 2ddad52ccca07245eea43d9b844c6c7d4b667ca3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you specify a local directory which happens to be a git repository
with an origin remote (and it is in fact remote), we can use that for
SRC_URI as implemented by OE-Core revision
b143d414846854dc8b3e0a47358daf5646eded38, however we also need to set S
if the recipe is going to be of any use fetching from that SRC_URI
later.
(From OE-Core rev: 8bbbd2d63f1bc752f9a30054a089dc2caf5fd84c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When recipetool create is given a URL that starts with http(s):// and
contains /git/, such as the URLs at git.yoctoproject.org, it's fairly safe
to assume it's a git repository and not something that should be fetched
with wget, so rewrite the URL.
(From OE-Core rev: 3ca04757a670e8b6f78799cc0454d75691809ac4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When auto-detecting the name for a recipe from the URL, strip off any
parameters (";name=value...") before parsing the URL, otherwise this
just ends up in the recipe name which is undesirable.
(From OE-Core rev: d3c46b5d0abd56bcadd4f2f1ef985f13d67f605b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Both BitBucket and GitHub provide "release" downloads that unfortunately
don't include the actual project name. For recipetool this means we have
to have special handling for these URLs or else the current code assumes
the name of the project is something like "v0.9.1". Borrow and adapt
some code from autospec to do this, and put it in its own function for
clarity.
(From OE-Core rev: e8435fde7f82c39cc0efe1d4bf043b29f63898e9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I was a little bit hasty in OE-Core revision
c2cc5abe34169eae92067d97ce1e747e7c1413f5 - it turns out BitBake's
fetcher code is not consistent in whether it logs something useful or
not; when fetching from an http URL it does but with a git repository
it doesn't. In advance of any major reworking of fetch error handling in
BitBake, let's just print the text of the exception and then we know we
have shown something to the user.
Additionally, we were only catching FetchException here but there are
several other classes of exception that the fetcher can raise (e.g.
MalformedUrl); catch the parent BBFetchException class instead so we
avoid tracebacks for those other classes as well.
(From OE-Core rev: 578d3873a6415c9203c185c21cff472f7d2dab02)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the URL ends in a / then we want to strip that off the path we split
out of the URL before calling os.path.basename() on it.
(From OE-Core rev: 308189beda8a31541481d09e3d5e86187e843d8d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you specify a local directory which happens to be a git repository
with an origin remote (and it is in fact remote), we can use that for
SRC_URI rather than leaving it blank in the recipe.
(From OE-Core rev: b143d414846854dc8b3e0a47358daf5646eded38)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes you don't want to build an entire project, just a subdirectory
of it; add a --src-subdir option to make that easier. (We still look for
a single subdirectory in what gets unpacked, e.g. what you might find
within a tarball, so whatever you specify with this option is added onto
the end of that.)
(From OE-Core rev: 59682d78f95732e014f78f13e0a05f843860d9bb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I found these when I was looking at libftdi and they seem to be
generic enough to show up in at least a couple of other packages so I
figure I'll add them.
(From OE-Core rev: 9fa3ff44e05930d4dfa153db777077e747ecbf45)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add support for extracting dependencies from CMakeLists.txt. There's
still a bunch of things missing that are outside the scope of OE-Core
and we still lack a proper extension mechanism, but this is a good
start.
This also adds an oe-selftest test to exercise the new code a bit.
Implements [YOCTO #7635].
(From OE-Core rev: 77e73e6930381fdbd6e78d3913d6467572e16568)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We want to specify dependencies on virtual/* rather than whatever
library is selected in the current configuration.
(From OE-Core rev: e1ac0c45b27ded9962edaf34597f827d0b41ba82)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some refactoring to allow access to the library/header/pkg-config
mappings and the DEPENDS / unmapped dependency output code from other
classes than AutotoolsRecipeHandler.
(From OE-Core rev: 40c10d998b90dd59c6d36c28f8ba11ec598bfa0f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The regex for detecting git URLs was unanchored, leading to it matching
where it shouldn't have. An example of where this went wrong was
http://taglib.github.io/releases/taglib-1.9.1.tar.gz.
(From OE-Core rev: bacff751c88b680fbfb07843b18c59c8bc80a9ea)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes you need to build a variant of a recipe for the build
host as well as for the target (i.e. BBCLASSEXTEND = "native"); add a
--also-native command line option to "recipetool create" that enables
this and plumb it through from an identical option for "devtool add".
(We could conceivably do the same for nativesdk, but I felt it might be
confusing within the context of the extensible SDK, where nativesdk
isn't really relevant to the user.)
(From OE-Core rev: f3bea83db173cce921a9a30f04e88b7e3ed98854)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the recipe name ends with -native then we should inherit native;
likewise if it starts with nativesdk- then inherit nativesdk.
(Note that the recipe name must actually be specified by the user in
order to trigger this - we won't do it based on any name auto-detected
from e.g. the tarball name.)
Since we're doing this based on the name, "devtool add" will also gain
this functionality automatically.
(From OE-Core rev: a216fb1f5953327790dce3d2c1a9af616c0b410d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As a matter of general convention we expect recipe names to be lower
case; in fact some of the packaging backends insist upon it. Since in
this part of the code we're auto-determining the name, we should convert
the name to lowercase if it's not already so that we're following
convention and avoiding any validation issues later on.
(From OE-Core rev: 64986decbc11afa0d1e11251f5f7adcba1860d19)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having fetched the source and unpacked it to a temporary directory, we
then move part of it to the destination directory, or if the source is at
the top level we move the whole temporary directory, but in the latter
case we were later attempting to delete the temporary directory which no
longer existed. Clear out the variable so that doesn't happen.
(From OE-Core rev: 91714a52e91cddba5a16c73cf5765d1f47f7856c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Often the filename (e.g. source tarball) contains the name and version
of the software it contains.
(This isn't intended to be exhaustive, just to catch the common case.)
(From OE-Core rev: 944eacfb849ee69b41e12c9de4f264406281ac6a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some build systems (notably autotools) support declaring the name and
version of the program being built; since we need those for the recipe
we can attempt to extract them. It's a little fuzzy as they are often
omitted or may not be appropriately formatted for our purposes, but it
does work on a reasonable number of software packages to be useful.
(From OE-Core rev: 3b3fd33190d89c09e62126eea0e45aa84fe5442e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes we want to force one handler to run before another; if the two
handlers are in different plugins that's difficult without some kind of
priority number, so add one and sort by it.
(From OE-Core rev: 0219d4fb9cefcee635387b46fc1d215f82753d92)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the user specifies a URL that just returns a web page, then it's
probably incorrect (or broken); attempt to detect this and show an error
if it's the case.
(From OE-Core rev: 83b1245b2638eb5d314fe663d33cd52a776a34a7)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a fetch error occurs, the fetcher already prints a reasonable error -
we don't need the traceback as well, so catch that and exit if it
occurs.
(From OE-Core rev: c2cc5abe34169eae92067d97ce1e747e7c1413f5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When you grab a URL for a github repository you'll almost certainly find
it in https://github.com/path/to/repository.git format; but bitbake's
fetcher can't handle that because it'll see https:// at the start and
assume it should use wget to fetch it. If the URL starts with http:// or
https:// and the path part ends with .git then assume it's a git
repository and adjust it accordingly.
(From OE-Core rev: bdbc4cf41d30eddb8a9ed882dedcc1670ce8fdd6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This should have been register_commands rather than register_command;
I used register_commands in devtool so lets change this here to be
consistent with that. (Since this is extensible through layers though we
need to remain compatible with the old name, so fall back to that if the
new function name isn't there.)
(From OE-Core rev: 1047f6592ac81643cd847f104da766dc4a4c81ea)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the directory specified by --extract-to exists, because we were using
shutil.move() to move the temporary extracted directory to the specified
path, a subdirectory was being created under that directory instead of
moving the contents, which was a different result than if the directory
didn't previously exist. We could try to always move the contents but
that's complicated when any symlinks are involved; the simplest thing is
just to remove the directory (which should be empty anyway) before
moving the temporary directory across in its place.
(From OE-Core rev: 2880bd23b471c1966661b9f05726faf60f9c0e7e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a means of creating recipes for package files or archives that
contain a directory structure to be installed verbatim, for example an
rpm file. (We mostly just re-use bin_package here and skip some of the
normal build system checks.) This support is available in "recipetool
create" and "devtool add" which wraps the former.
(From OE-Core rev: 9e0a6b2e6f16185f8032d36b77d40802bc388987)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The # character in a URI denotes a fragment; we don't care about this
since it is never supposed to be sent to the server, so remove it from
the URI before actually trying to fetch it or use it in SRC_URI within
the recipe.
(This has come up because download links on pypi.python.org seem to have
a fragment containing the md5sum of the download; without stripping this
off the fetcher will choke on it.)
(From OE-Core rev: 58dc726808817210764eab963a4d453dc4ff49aa)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The shell considers empty functions to be a syntax error, so for
template shell functions that contain only comments (or no lines at all)
then add a : to act as a no-op which avoids the syntax error.
(From OE-Core rev: ff14d9e5b935b99b2efde479515e54c02ba58f6e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will now also be used by "devtool upgrade".
(From OE-Core rev: 0d0b8425eaf74a6d7f3d9f6471e6edca1a273c06)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Provide a means to set SRCREV immediately by using rev= in the URI; if
it is specified then it is removed before setting SRC_URI and SRCREV is
set to the same value instead.
(From OE-Core rev: c9304fcb0a2b81700d0ed5c13b4d976bd4230ce3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Recipes that fetch from git, svn or hg by OpenEmbedded convention
should normally be named with this as a suffix, since PV is meant to
be set appropriately within the recipe, so follow this. In order to
make this work we need to be able to have the version independent from
the file name, so add a -V option to recipetool create to allow this
to be specified.
* If -V is specified on the devtool add command line, ensure at PV gets
set to include this version.
(From OE-Core rev: 2b997c41c6476a13bf516586d56a9051aceb93ec)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If -x is specified and the specified URI was a git repository, we need
to ensure that the resulting clone is a sandalone and not one that has
pointers into the temporary fetch location or DL_DIR (since the git
fetcher does a local clone with -s). Split out the code from devtool
that already does this for "devtool modify -x" and reuse that.
(From OE-Core rev: fc47e8652ef32e7399f57c80593df90dc52d8b84)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When -x was specified, we were getting the normal unpack subdirectory
which we don't really want - if there's only one subdirectory unpacked
then we should effectively copy just it to the extraction path, not as a
subdirectory under it.
(From OE-Core rev: 0eeecce96a0aa757c2c4c4ac4d82e3bcbf0f982c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a license file matched more than one of the specifications (e.g.
COPYING.GPL) then it was being added to LIC_FILES_CHKSUM more than once.
(From OE-Core rev: 58316a2890782d206e9b9472ba483367f7560109)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Option was renamed in the setup code but not in the code that used it.
(From OE-Core rev: 4b4f2d4f2869d6d5d564cc1b2d733f41ab5a3b9b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
