By default the sdk-install subcommand expects to restore the requested
items from sstate and fails if it can't. If the user is OK with building
from source, add a -s/--allow-build option to allow them to do that. In
the process, ensure we show the status output while we're installing.
Also add the missing header to the top of the file.
(From OE-Core rev: a86b426cdd465ec5cb08bb5fa7729e4e673d94bb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Otherwise (if the symlink is named .config) kernel build considers
source tree as dirty and fails.
[YOCTO #9270]
(From OE-Core rev: d36699b28c661880957d744420df9e23cdd76957)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow plugins to create additional files to go alongside the recipe. The
plugins don't know what the output filename is going to be, so they need
to put the files in a temporary location and add them to an "extrafiles"
dict within extravalues where the destination filename is the key and
the temporary path is the value.
devtool add was also extended to ensure these files get moved in and
preserved upon reset if they've been edited by the user.
(From OE-Core rev: 334b9451111b7e3efbb43b3a4eecebcab8ec6f0e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a build-sdk command which is only available within the extensible
SDK that builds a derivative extensible SDK. The idea is recipes in the
workspace become a part of the new SDK - for example, this allows taking
a vendor provided SDK, adding a few libs and then producing a new SDK
with those included.
When normally building the extensible SDK, the workspace is excluded;
here we need to copy into the new SDK (renaming it in the process); the
recipes' task signatures become locked and thus the sources are no
longer needed, so they are removed along with the workspace bbappends
which would interfere with the locked signatures. Additionally we need
to just copy the configuration files (i.e. local.conf and auto.conf)
rather than filtering and appending to them since that work has already
been done when constructing the original SDK. The extra sstate artifacts
from workspace recipes are also determined and copied into the new SDK
in minimal mode (on the assumption that you won't set up a new sstate
mirror).
This reuses some code from build-image, so that needed to be
generalised to allow that.
Implements [YOCTO #8892].
(From OE-Core rev: 59e207ff6dd4b50a8905e14bc9292cf2794f4e7a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Hyphens aren't allowed in python identifiers, so you shouldn't use them
in module names or they are more difficult to import.
(From OE-Core rev: e8f6db9436dfc923e236f2cbc08f357e3b24fd5d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"sdk_update" uses a variable newsdk_path, which was never declared.
This would cause the command:
devtool sdk-update <poky-sdk-latest>
to fail with an error:
NameError: global name 'newsdk_path' is not defined
The remedy is to declare newsdk_path as it was no doubt intended,
corresponding to the argument specifying <poky-sdk-latest>.
[YOCTO#9042]
(From OE-Core rev: 6ac589a4a9290665f8c5295e2e2a03a2b187b957)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make this consistent with "devtool add" so that the user knows where to
find the new recipe.
(From OE-Core rev: 28fea4a104ba85aded2ecfe891e9bf997d72a103)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The PR value should be reset to the default when upgrading, so we need
to drop it from the newly created file.
(From OE-Core rev: 4acf8a6912106dd3da5ceee9e30ccce86211d04b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We aren't modifying the datastore copy here, so we don't need a copy at
all.
(From OE-Core rev: 9f05bf98d4dbcb991b684a2ab480c6638c6be292)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix several issues when extracting the new version source over the top
of the old one (when the recipe is not fetching from a git repo):
* Delete the old source first so we ensure files deleted in the new
version are deleted. This also has the side-effect of fixing any
issues where files aren't marked writeable in the old source and thus
overwriting them failed (harfbuzz 1.1.3 contains such files).
* Fix incorrect variable name in abspath statement that made it a no-op
(From OE-Core rev: 4f7595a514661f0e2418393286849a1cb6c46f0d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we do an upgrade from one tarball version to another we want to:
1) Check out the old version as a new branch
2) Record the changes between the old and new versions as a commit
3) Check out the old version with patches applied
4) Rebase that onto the new branch
Where we went wrong was step #1 where instead we checked out the old
version with patches applied as the new branch, which meant the rebase
didn't do anything and any changes made by the patches to files still in
the new version were wiped out.
(From OE-Core rev: a4f3e6d5f4c549e36cce1e18801a565a3a3c9b0a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the actual value of PV isn't in the name of the recipe (for example,
a git or svn recipe) there's no point trying to rename it. Additionally,
we already have the original filename, there's no need to guess it -
just pass it in.
(From OE-Core rev: e144db3650b4a7c0a59066621905f34a5400303e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were trying to move this from the current directory instead of the
path. Let's just use shutil.move() instead of shelling out to mv.
(From OE-Core rev: 60454a0ba154d6c777e0c2b05b887b4e4fcde986)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For recipes that specify SRCREV, the code here wasn't quite doing the
right thing. If the recipe has a SRCREV then that needs changing on
upgrade, so ensure that the user specifies it. If it doesn't, then it'll
be "INVALID" not None since the former is the actual default, so handle
that properly as well. Additionally an unset variable was being
erroneously passed when raising the error about the version being the
same leading to a traceback, so fix that as well.
(From OE-Core rev: 1d0f821371d1cb93e30fad86f0c20e38cb93b54b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recipename argument to devtool upgrade specifies an existing recipe,
so by definition the name will be valid (or it won't exist) - we don't
need to validate it ourselves, that's only needed for situations like in
devtool add where we're creating a new recipe.
(From OE-Core rev: 1e2bc7d861555a04350a87f19047efdc717046be)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make devtool upgrade consistent with devtool add/modify in defaulting to
sources/<recipename> under the workspace if no source tree path is
specified.
(From OE-Core rev: 8a952407b192313515e91570632446b6dff01665)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you for example ran devtool modify virtual/libusb0 without specifying
a source tree path, the default was <workspace>/sources/virtual/libusb0
which isn't correct - it should be using the mapped name i.e.
libusb-compat (in the default OE-Core configuration). Reorder some of
the code to ensure that the mapped name is used.
(From OE-Core rev: c51736df17da8e6e561dd5b7ce59cb08254da870)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As suggested by Khem Raj.
(From OE-Core rev: 36cc6b81d0281348a0f241a80ddd427745a6a678)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create config fragment if the user makes modifications to kernel config.
User may change .config e.g. by directly editing it or by running the
'do_menuconfig' bitbake task. Devtool generates one monolithic fragment
by simply doing a diff between .config and .config.baseline files in the
source directory. If either of these files is missing, the config
fragment is not gerenrated or updated. The output is a file,
'devtool-fragment.cfg' that gets added to SRC_URI in the recipe (as well
as copied into the 'oe-local-files' directory if that is present in the
source tree).
${S}/.config will be a symlink to ${B}/.config. We need to do this as
devtool is not able to access ${B} because ${B} is set in a .bbappend in
the workspace layer which is not parsed by devtool itself.
[YOCTO #8999]
(From OE-Core rev: 524da136e5b837a60682516ac08f3092c635e934)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Copy kernel config is copied to the source directory at a later phase in
_extract_source() so that it gets copied when devtool sync is done, too.
(From OE-Core rev: ff895be7a46c4b3b1b791e5387490d90bb34fce2)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I should have adjusted this in OE-Core commit
80a44e52609a89d9ffe816181ae193af491c06ac where the behaviour changed.
(From OE-Core rev: 13409a2b899bb74b8060c840b8c7ef8759d099cb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If files would be overwritten by the deployment, preserve them in a
separate location on the target so that they can be restored if you
later run devtool undeploy-target.
At the same time, also check for sufficient space before starting the
operation so that we avoid potentially failing part way through.
Fixes [YOCTO #8978].
(From OE-Core rev: a2da55712691bcf1066c53d813107d75213d6b10)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you want to put the target device back to exactly how it was before
devtool deploy-target started poking things into it, then it would make
things easier if you didn't have to figure out which recipes were
deployed. Now that we have the list stored on the target we can
determine this reliably, so add a -a/--all option to undeploy-target to
undeploy everything that has been deployed.
One of the side-effects of this is that the dry-run functionality for
undeploy-target had to be reimplemented to actually run the script on
the target, since we have no way of knowing what's been deployed from
the host side. We don't need to do the same for deploy-target though
since we know exactly which files will be deployed without referring to
the target.
(From OE-Core rev: 41fed83060f0041e14e455d1446397bda277d953)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running devtool deploy-target, we save a list of deployed files,
and this list is used by devtool undeploy-target (or the next time
deploy-target is run if the list is present, in case any files have been
renamed or deleted since the first time). We were writing this file to
the host, but it makes more sense to write the list to the target
instead, so that if we for example swap in a different board, or switch
hosts, things will work as expected.
In order to do this properly we have to construct a shell script and
ship it over to the target so we can run it. The manifest is written out
to a hidden directory in the root (/.devtool).
Fixes [YOCTO #7908].
(From OE-Core rev: a16a0c9334b785e2df896266c8911a2c7a1806b8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Get the default value for updateserver from the configuration file and
show it in the help; also only make the parameter optional if it's
specified. This means we can also drop the check in the function as
argparse will then ensure it's specified if there's no config setting.
(From OE-Core rev: 82497bde58fc8bdae8d8acfbf025a1a90b14cc3e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a long description used when running --help on the specific command.
(From OE-Core rev: eb7787d1652fd84a149fd394969f4f1099406051)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The listing of subcommands in the --help output for devtool was starting
to get difficult to follow, with commands appearing in no particular
order (due to some being in separate modules and the order of those
modules being parsed). Logically grouping the subcommands as well as
being able to exercise some control over the order of the subcommands
and groups would help, if we do so without losing the dynamic nature of
the list (i.e. that it comes from the plugins). Argparse provides no
built-in way to handle this and really, really makes it a pain to add,
but with some subclassing and hacking it's now possible, and can be
extended by any plugin as desired.
To put a subcommand into a group, all you need to do is specify a group=
parameter in the call to subparsers.add_parser(). you can also specify
an order= parameter to make the subcommand sort higher or lower in the
list (higher order numbers appear first, so use negative numbers to
force items to the end if that's what you want). To add a new group, use
subparsers.add_subparser_group(), supplying the name, description and
optionally an order number for the group itself (again, higher numbers
appear first).
(From OE-Core rev: e1b9d31e6ea3c254ecfe940fe795af44761e0e69)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we didn't make any changes to the file then there's no point warning
the user that we have done.
(From OE-Core rev: 391b9ba30d802ac420ddf382588e03e718861c01)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If files had been created next to the recipe (for example devtool add,
edit the source and commit and then devtool update-recipe), running
devtool reset failed to preserve those files and gave an error due
to trying to rmdir the directory containing them which wasn't empty.
Fix the preservation of files in the "attic" directory properly so
we catch anything under the directory for the recipe, and replicate
the same structure in the attic directory rather than slightly
flattening it as we were before.
(From OE-Core rev: bbe63eb97ae7f78959f117d6066ef821c4da1c77)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Looking at Chris Larson's code for starting the user's editor for
"recipetool newappend" it was slightly better than what I wrote for
"devtool edit-recipe" in that it checks VISUAL as well as EDITOR and
defaults to vi if neither are set, so break this out to its own function
and call it from both places. The broken out version passes shell=True
however in case it's a more complicated command rather than just a name
of an executable.
(From OE-Core rev: 184a256931e8cdc7bea97a905c4e67a435964de0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no -N/--name option for devtool, that's a recipetool option -
with devtool you just specify the name as a positional argument.
(From OE-Core rev: d2bc0cba5ca8a7220ffe1ef96acf856fe972ce7c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix a regression introduced in in OE-Core revision
aedfc5a5db1c4b2b80a36147c9a13b31764d91dd where specifying a local source
tree without specifying a name resulted in a traceback.
Fixes [YOCTO #9086].
(From OE-Core rev: 67ea9d21f20a371a0ee443b46326018cb1527b62)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's going to be more common for users not to have the prepared source
tree for a recipe already, so the default behaviour ought to be to
extract it for them from the recipe. Change the default to extract
(effectively making the -x option a no-op) and add a --no-extract/-n
option to disable it. Later we can look at trying to be smart and
reusing an existing source tree instead of erroring out if it exists;
for now this is just the default reversal.
(From OE-Core rev: 80a44e52609a89d9ffe816181ae193af491c06ac)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When extracting source for a recipe, if there are additional custom
tasks run that make changes to the source, create a commit in the
generated git branch so they are contained. This is particularly
useful for tasks that come before do_patch since otherwise the changes
might get incorporated in the first applied patch, but otherwise it
helps avoid the tree being dirty at any point.
Fixes [YOCTO #7626].
(From OE-Core rev: 997a77d9b20af1778b804778e5d8c8a7424f7582)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When you need to set EXTRA_OECONF for a recipe, you need to know what
options the configure script actually supports; the configure script
however is only accessible from within a devshell and (at least in the
case of autotooled software fetched from an SCM repository) may not
actually exist until do_configure has run. Thus, provide a "devtool
configure-help" subcommand that runs the configure script for a recipe
with --help and shows you the output through a pager (e.g. less),
prefaced by a header describing the current options being specified.
There is basic support for autotools, cmake and bare configure scripts.
The cmake support is a little hacky since cmake doesn't really have a
concise help option that lists user-defined knobs (without actually
running through the configure process), however that being a design
feature of cmake there's not much I can think of to do about that at
the moment.
(From OE-Core rev: 0e5d84d9705091b338000ef02720cfa090f76888)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we run the tasks required to extract the source for a recipe (e.g.
within "devtool modify" or "devtool extract") if one of those tasks
fails you get a bb.build.FuncFailed exception; handle this properly so
you don't see a traceback.
(From OE-Core rev: 95d8631b3bdf216001e57f48277535c65a4cc49e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a recipe generated by "devtool add" has been modified since then when
you run "devtool reset", it will be moved into the "attic" subdirectory
of the workspace in case those modifications need to be preserved. It
seems natural that if those modifications were worth preserving we
should warn the user if such a file exists when they run "devtool add"
to create the same recipe again, so they can pick up where they left off
if they want to.
(From OE-Core rev: 0a39b907ff997c3a62c92ab22325c726b612de5b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Provide an option to devtool build-image to specify the list of packages
instead of taking the list of packages produced by recipes in the
workspace. Sometimes you don't want all of these packages; other times
you want to add more.
This is the most immediate fix for [YOCTO #8855], though it is a little
crude so I would like to provide better means of customising the image
contents later.
(From OE-Core rev: b3a44951a74fe58714b72e71a7a558b67a71e1e3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
standard.py is getting a bit large; move the "utility" commands to
another module.
(From OE-Core rev: 5089b93f5b341dc28c343f7afe15efda2081ed36)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes you don't want to build an entire project, just a subdirectory
of it; add a --src-subdir option to make that easier. (We still look for
a single subdirectory in what gets unpacked, e.g. what you might find
within a tarball, so whatever you specify with this option is added onto
the end of that.)
(From OE-Core rev: 59682d78f95732e014f78f13e0a05f843860d9bb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the ability to install additional pre-built items (from shared
state) into the extensible SDK. This can already be done implicitly by
adding something to DEPENDS within a recipe you're working on and then
running "devtool build", but it's useful to be able to explicitly
install things particularly if you're using the extensible SDK as a
traditional toolchain.
Note that for this command to be useful you need to have SSTATE_MIRRORS
set in your SDK configuration, and that mirror needs to be populated
with sstate artifacts for recipes you wish to be able to install.
(From OE-Core rev: 3474a42954908d1688fd3a6cb600eed315b27833)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make the following improvements to the SDK update process:
* Use a manifest file with sha256sums to track files other than sstate
and metadata that we need to update - e.g. conf files. This allows us
to handle where files such as auto.conf may or may not be present,
as well as the configuration changing without affecting task signatures
- we still want the config files copied in that case rather than it
saying nothing needs to be done.
* Write the SSTATE_MIRRORS_append to site.conf rather than local.conf
so that local.conf remains static (since we don't want to trigger an
update every time). Also, If there is an SSTATE_MIRRORS value already
set in the configuration we can skip this and assume it contains the
needed packages.
* Allow the update process to be run in any directory, don't assume
we're already at the base of the SDK
* Where practical, fetch remote files into a temporary location and
then move them to the desired location at the end, to avoid a
failed update leaving the SDK in a broken state.
* Update all installed do_populate_sysroot / do_packagedata tasks
instead of using the SDK targets. This ensures any item installed
through dependencies after installation (e.g. when running
"devtool build") won't go stale.
(From OE-Core rev: 3d35631121f0e030bc8151f5c23d84008d06f44b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Use tempfile.mkdtemp() instead of hardcoding temp dir
* Set a variable early for the temp locked sigs file and use that
everywhere
* Delete the temp dir at the end
(From OE-Core rev: bad5d1a8c047a8118d30d9fa708b021d1599e0dc)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When you run devtool build, you need to have the pkgdata written out at
the end, so that if what you're adding is a library and the next thing
you add is something that depends on that library, the necessary
information to map the dependency back to the recipe is present. In
practical terms all this means is we need do_packagedata to run in
addition to do_populate_sysroot.
This does mean that do_package needs to run which wasn't running before,
and that means that the few package QA tests that run within do_package
such as installed-vs-shipped will now be run. This may be a bit
bothersome, and prompted a fix for one of our oe-selftest tests as a
result, but I don't see an easy way around it. Ultimately if you care
about using the recipe in an image you'll need to fix any such errors
anyway.
Fixes [YOCTO #8887].
(From OE-Core rev: 6579c7120ee5a541427ff5b6b07f838d52f9fe7c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a variable SDK_INCLUDE_PKGDATA which you can set to "1" to include
pkgdata for all recipes in the world target. There are a couple of uses
for this:
1) If you use "devtool add" to add a recipe that builds something which
depends on anything in world, the dependency can then be correctly
mapped to the recipe providing it and that recipe can be added to
DEPENDS, since we have the pkg-config and shared library dependency
data within pkgdata.
2) You'll be able to search for these recipes and any files they
package for the target with "devtool search" since that also uses
pkgdata
This of course assumes you've tailored world through EXCLUDE_FROM_WORLD
to only include recipes you'd want built in your distro, but I think
that's a reasonable assumption; failing that there is a
WORLD_PKGDATA_EXCLUDE variable that you can set to exclude any recipes
you don't want.
Note that this patch relies on functionality implemented in a recent
BitBake patch and will not work without it.
Implements [YOCTO #8600].
(From OE-Core rev: 67149ea097d6fab7496b43e85a40853f40bd527e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than horrible workarounds, use the new --setscene-only option
of bitbake to pre-populate the SDK structures.
(From OE-Core rev: 38237b7ac53c416f85c4a70a61acafc3404c8b5f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's logical that you would want to build BBCLASSEXTENDed items
separately through devtool build, so simply allow that - we're just
passing the name verbatim to bitbake, so all it means is adjusting the
validation.
(From OE-Core rev: 25dc5ac42c9da53c01416e7fdcc819d729281133)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the recipe file itself was created in the workspace, and it uses
BBCLASSEXTEND (e.g. through devtool add --also-native), then we need to
clean the other variants as well.
(From OE-Core rev: e1bf6a30679a83d4dbcf37276204f639541e95f9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're doing this in a couple of places, let's just find the recipe file
if it exists within the workspace (which it will if it's been added
through "devtool add") when we read in the workspace.
(From OE-Core rev: 81cf5580993c99050e3f4d6d891bc67534721487)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes you need to build a variant of a recipe for the build
host as well as for the target (i.e. BBCLASSEXTEND = "native"); add a
--also-native command line option to "recipetool create" that enables
this and plumb it through from an identical option for "devtool add".
(We could conceivably do the same for nativesdk, but I felt it might be
confusing within the context of the extensible SDK, where nativesdk
isn't really relevant to the user.)
(From OE-Core rev: f3bea83db173cce921a9a30f04e88b7e3ed98854)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to run the clean for all recipes that are being reset before we
start deleting things from the workspace; if we don't, recipes providing
dependencies may be missing when we come to clean a recipe later (since
we don't and couldn't practically reset them in dependency order). This
also improves performance since we have the startup startup time for the
clean just once rather than for every recipe.
(From OE-Core rev: c10a2de75a99410eb5338dd6da0e0b0e32bae6f5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For debugging purposes it's useful to be able to skip the preparation
step so you can inspect what the state of the build system is first.
(From OE-Core rev: 0bba4b5afd2ce2c3a79445eee886979a77f1a4d8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Running "raise" with no arguments here is invalid, we're not in
exception handling context. Rather than also adding code to catch the
exception I just moved the check out to the parent function from which
we can just exit.
(From OE-Core rev: 0164dc66467739b357ab22bf9b8c0845f3eff4a4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Clone the correct path - we need .git on the end
* Pull from the specified path instead of expecting a remote to be set
* up in the repo already (it isn't by default)
(From OE-Core rev: 1a60ee8bd21e156022c928f12bb296ab5caaa766)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We read the updateserver setting from the config file but we never
actually used that value - the code then went on to use only the value
supplied on the command line.
Fix courtesy of Dmitry Rozhkov <dmitry.rozhkov@intel.com>
(From OE-Core rev: 1c85237803038fba539d5b03bf4de39d99380684)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Copied layers with 'cp -a' instead of calling shutil.copytree as
copytree fails to copy broken symlinks.
More pythonic fix would be to use copytree with 'ignore' parameter,
but this could slow down copying complex directory structures.
[YOCTO #8825]
(From OE-Core rev: e5b841420b9fdd33829f7665a62cd06a3017f7e6)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Much of this was copy/pasted from the extract subcommand code; make it
specific to sync.
(From OE-Core rev: 08ca966dd48db881d8bfb7d7fa0aba49f1f7cffe)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We deliberately leave the source tree alone when resetting in case it
contains any work in progress belonging to the user; tell them that
we're doing this so they aren't surprised about it still existing later
on.
(From OE-Core rev: 1c616802ba2cfb37f8403f1dc6cf91ce3bd99ac2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a recipe in the workspace actually exists as a file within the
workspace (e.g. after doing "devtool add" or "devtool upgrade") then
show the path to the recipe file on the status line for the recipe.
(From OE-Core rev: 29833828a90c8433af3b231b50e99cd97edf19ff)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As per the changes to "devtool add", make the source tree path optional
and use the default path if none is specified.
(From OE-Core rev: 83707d1334fb094fd1877bcfd07a83866601048a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having to specify -f is a little bit ugly when a URI is distinctive
enough to recognise amongst the other positional parameters, so take it
as an optional positional parameter. -f/--fetch is still supported, but
deprecated.
(From OE-Core rev: aedfc5a5db1c4b2b80a36147c9a13b31764d91dd)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
recipetool create now has all the logic in it for auto-detecting the
name and version, and using those in the file name - so we can make the
name an optional parameter for devtool add and we pick up the file name
that recipetool has used after the fact.
(From OE-Core rev: 70ab08146e930f1fc55fdf5726a87303e20bd60f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Assuming we're fetching source remotely (from a URI) we can default the
source tree that will be extracted from it to a "sources" directory
under the workspace in order to save the user specifying it if they
don't have a preferred location.
(From OE-Core rev: ffdad964c7271972e4b067e4898bf7c338c25b68)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The bbappend already exists at this point, so we know what its path is -
there's no need to figure it out from scratch here.
(From OE-Core rev: c0754d672966901f22dff1bcd40bbd08d1219c7a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're repeating this in a couple of places, so we might as well have a
function to do it.
(From OE-Core rev: 67a28109a1ee1383d1b17a8dafa4fe510948238b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a few clarifying words.
(From OE-Core rev: 2103fa9dc7faf2189c8b426b87fb9d421a9983ac)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add an "edit-recipe" subcommand that runs your default editor (as
specified by the EDITOR environment variable) on the specified recipe in
the workspace. Note that by default the recipe file itself must be in
the workspace - i.e. as a result of "devtool add" or "devtool upgrade";
however there is a -a/--any-recipe option to override this.
(From OE-Core rev: dbfe8fa2e86c2bb50bef47c389017cdf93543321)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the SDK update server hasn't been set in the config (when building
the extensible SDK this would be set via SDK_UPDATE_URL) and it wasn't
specified on the command line then we were failing with a traceback
because we didn't pass the default value properly - None is interpreted
as no default, meaning raise an exception if no such option exists.
Additionally we don't need the try...except anymore either because with
a proper default value, NoSectionError is caught as well.
(From OE-Core rev: 9763c1b83362f8445ed6dff2804dd7d282861f79)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add 'do_kernel_configme' and 'do_kernel_configcheck' to
SRCTREECOVEREDTASKS of kernel packages. These tasks should not be run
because kernel meta in the srctree is not necessarily up-to-date or
even present which causes build failures and/or invalid kernel config.
Especially so because 'do_patch' which is a dependency of
'do_kernel_configme' is not being run.
We now store .config in the srctree and 'do_configure' task is able to
run successfully.
(From OE-Core rev: 7ce4c18a4ba1ebcb9f46e652a881ace1f21d2292)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes the correct kernel config to be used when building kernel
from srctree (extrernalsrc). If no kernel config is present in the
builddir 'do_configure' task copies .config from the srctree.
(From OE-Core rev: 3b516332e038a587685f6e0c14a7f04990bdd6cc)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use bb.utils.edit_metadata() to replace some of the logic in this
function; this avoids us effectively having two implementations of the
same thing. In the process fix the following issues:
* Insert values before any leading comments for the next variable
instead of after them
* Insert overridden variables (e.g. RDEPENDS_${PN}) in the correct place
* Properly handle replacing varflag settings (e.g. SRC_URI[md5sum])
(From OE-Core rev: 0f81b83fc5fd908efa7f6b837137830ca65f6ed6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Make some minor clarifications to help text
* Drop ArgumentDefaultsHelpFormatter and just put the defaults in the
text itself where needed (because otherwise you get defaults shown for
store_true options which is somewhat confusing).
(From OE-Core rev: a90ffea30c4578fd6acda2c5945b816ad33b13f5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enable access to the configuration object in register_commands() so that
we can read configuration values there; this allows us to show the
task that will be run in the command line help for the build subcommand.
(From OE-Core rev: 142d006de3235a034839ef7bbe147c56fc7af04a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use a bbappend file to set PARALLEL_MAKE instead of a postfile; this
is a bit neater and only affects the specified recipe.
(From OE-Core rev: b5bafc845892ac39f85f3642b120fb7b785a3d58)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is just belt-and-braces but we ought to use try..finally in this
kind of situation, so just do it.
(From OE-Core rev: a30b407474d4eb6620f1ec549b54187ebbaff008)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Print the SUMMARY value for each matched item assuming it's not the
default.
(From OE-Core rev: 596dee8882ebddb45a6cce9f12aa919107106156)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This function is no longer required to be defined for a plugin, so drop
it where it's a no-op.
(From OE-Core rev: 49716a5e27e704598321bd6c82275b16afc444aa)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than reconstructing the output path for packages, use the proper
variable.
(From OE-Core rev: 06d47777ed1650c479fadf98388f9a3a1f1f9eda)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For subcommands that don't actually involve the workspace, don't
auto-create the workspace.
(From OE-Core rev: 90cba7992bc1d227e242666cd486414bd4a45f7e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you're upgrading a git recipe to a revision on a release branch
that's different to the branch for the current revision, then you'll
need to update the branch parameter in SRC_URI, so add a --srcbranch/-B
command-line parameter to let you do that easily. It handles both when
the branch is stated verbatim in the recipe, and when a reference to
another variable is used (a common convention is to use a SRCBRANCH
variable for this, though the code doesn't care what variable is used
if any).
(From OE-Core rev: e49a66fd898dd44e54c77a838ebef3d983ed2a03)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we're upgrading a recipe that fetches from git, and we've simply
fetched a tarball of the repo instead of directly from the upstream repo
(this can happen if you have PREMIRRORS set up as in poky with a core recipe,
e.g. kernelshark) then we won't have any new revisions, and the checkout
will fail with "fatal: reference is not a tree: <hash>". To avoid this,
do a "git fetch" before checking out the new revision.
(From OE-Core rev: c4daebf3fe797a8063dcbc2ab229be2fbedc8134)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we're upgrading a git recipe the recipe file usually won't need
renaming; for some unknown reason we were throwing an error here which
isn't correct.
(From OE-Core rev: 656348dff9bc9dd1cafc8fff11e5e374e3667f0f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This code was clearly never tested. Fix the following issues:
* Actually set SRCREV if it's been specified
* Enable history tracking and reparse so that we handle if variables are
set in an inc file next to the recipe
* Use a more accurate check for PV being in the recipe which will work
if it's in an inc file next to the recipe
(From OE-Core rev: 8b8f04226ebf464fa61c05ca7af7c6cbda392339)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you did a "devtool add" followed by "devtool upgrade" and then did
a "devtool reset" on the recipe you upgraded, the first recipe would
also be deleted from the workspace - this was because we were
erroneously adding the entire "recipes" subdirectory and its contents to
be tracked for removal on reset. Remove the unnecessary call to
os.path.dirname() that caused this.
(From OE-Core rev: 65354e066f87df7d3138adceb22d6a05d1685904)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The external source of kernel has been patched during the
construction of git repository. Include the do_patch task in the
SRCTREECOVEREDTASKS.
(From OE-Core rev: 0731c5a9e98f7b7f6e5ada9bbb99acb3f5884516)
Signed-off-by: Tzu-Jung Lee <roylee17@currantlabs.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The sync command is similar to the extract command, except it
fetches the sync'ed and patched branch to an existing git repository.
This enables users to keep track the upstream development while
maintaining their own local git repository at the same time.
(From OE-Core rev: e2fe4c99f1020a20b579832c4feafcd0e9bc2f75)
Signed-off-by: Tzu-Jung Lee <roylee17@currantlabs.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For modify / extract / upgrade, if the specified "recipe" is not
actually a recipe but a virtual target such as virtual/kernel, map it
correctly to the actual recipe and make sure we use that name within the
workspace. Thanks to Chris Larson for reminding me this was still broken
and for a hint on how to fix it.
(From OE-Core rev: 1f7752282ffb47d2621030ddb2fa42a5e491d6d2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If recipe parsing fails we need to exit out instead of attempting to use
the data object that's set to None, which resulted in a traceback.
(From OE-Core rev: a46fb1a06be2df00a0bceb4e60ebdecb55b5974c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use "git cherry" against the original tag that we made when we extracted
the source in order to find the revisions that are definitely new. This
allows you to modify a commit in the middle of the series and then run
devtool update-recipe and not have the subsequent patches unnecessarily
modified.
Fixes [YOCTO #8388].
(From OE-Core rev: 7baf57ad896112cf2258b3e2c2a1f8b756fb39bc)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The name of the directory for local source files under srctree is
'oe-local-files', not 'local-files'. Fixes a bug that slipped through
in b7ab82485e4514e07ab8a76e554da27ddc92e6c0.
(From OE-Core rev: 68cbe684fe6d8fe3a44e47ed85837d09077df6d5)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enable variable history tracking so that the variables are updated in
the correct file - i.e. in the file they are already defined.
[YOCTO #7715]
(From OE-Core rev: b54796d013c562972e962126400503085281b425)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This change makes it possible to have local files (non-remote SRC_URI
files, i.e. files that are located in the "recipe space") under the
srctree even if S!=WORKDIR. The files must be placed under the
'local-files' subdirectory.
Complements the previous patch that imports local files into srctree.
[YOCTO #7602]
(From OE-Core rev: b7ab82485e4514e07ab8a76e554da27ddc92e6c0)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* extract: Copy all local source files (i.e. non-compressed/non-arcived
SRC_URI files that have file:// URI prefix) - excluding patches - to
the srctree repository. The files will be placed in a subdirectory
called 'oe-local-files'. The oe-local-files directory is not committed
to the Git repository, but, marked to be ignored by a .gitignore file.
The developer can manually add and commit the files to Git if the
changes to them need to be tracked.
Before this patch, local source files (were copied (and committed) to
the srctree repository only in some special cases (basically when
S=WORKDIR) when doing devtool-extract. For most of the packages local
files were not copied at all.
* update-recipe: This patch causes the local files to be 'synced' from
the srctree (i.e. from the 'oe-local-files' subdirectory) to the
layer. Being 'synced' means that in addition to copying modified
files over the original sources, devtool will also handle removing and
adding local source files and updating the recipe accordingly. We
don't want to create patches against the local source files but rather
update them directly. Thus, 'oe-local-file' directory is ignored in
patch generation when doing update-recipe, even if committed to Git.
This functionality is only enabled if the 'oe-local-files' directory
is present in srctree.
[YOCTO #7602]
(From OE-Core rev: a3bb5bd25b72bd1bcc156dabd0ffa2d9184bb160)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Helper function for replacing a pattern like:
target_dir = os.path.dirname(target)
bb.utils.mkdirhier(target_dir)
shutil.move(source, target)
(From OE-Core rev: c09e5b11225a673534594c3642ceead3eb5653a3)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implement new function that handles patch file generation. The new
function also does the discovery of new, updated and deleted patches.
(From OE-Core rev: 183812e60e1dce09c77b41a5a927ab7fb6fdcf9d)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When adding multiple new patches append them to SRC_URI in correct order
so that they apply correctly.
(From OE-Core rev: 819680092c1b49c16f4ab01d135d44311a9dacca)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We only need the base configuration to get the variable values we want
to get here, there's no need to parse recipes / load the cache.
(From OE-Core rev: 84172ff7b325f081dba6430fd27f12e3745838b5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Do not change change current working directory permanently, but, only
for the duration of tinfoil initialization instead. The previous fix
caused very unintuitive behavior where using relative paths were solved
with respect to the builddir instead of the current working directory.
E.g. calling "devtool extract zlib ./zlib" would always create create
srctree in ${TOPDIR}/zlib, independent of the users cwd.
(From OE-Core rev: 4c7f159b0e17a0475a4a4e9dc4dd012e3d2e6a1f)
(From OE-Core rev: 05060699e63cd25d089e83e9aa56c11d5baa8fd8)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds a subcommand to search to find the target recipe name providing
some file or capability. This is implemented by searching on recipe
name, package name, description, package contents (file names), and
runtime file provides. For example:
$ devtool search libGL
mesa
$ devtool search X11
xextproto
libxxf86vm
xf86driproto
xf86vidmodeproto
libxfixes
xproto
libx11
...
$ devtool search /bin/sed
busybox
sed
This is particularly useful within the extensible SDK but is also made
available in devtool alongside the build system.
Note of course that because this searches pkgdata, useful results depend
upon do_packagedata(_setscene) having executed for the recipe being
searched for.
(From OE-Core rev: 48cbde0ea77ed20126eceba5feb37c42a9229500)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We ship the runqemu script and if we build QEMU itself within the
extensible SDK, then it would be nice to be able to run it. This is a
very thin wrapper around runqemu, supplying the machine and image name
so the user doesn't need to. (This subcommand is only available within
the extensible SDK since it only really makes sense there where it is
otherwise hard to run runqemu directly.)
Implements [YOCTO #6657].
(From OE-Core rev: abca7a0cac7068ffe6a6b873d0842f804388b621)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a means of creating recipes for package files or archives that
contain a directory structure to be installed verbatim, for example an
rpm file. (We mostly just re-use bin_package here and skip some of the
normal build system checks.) This support is available in "recipetool
create" and "devtool add" which wraps the former.
(From OE-Core rev: 9e0a6b2e6f16185f8032d36b77d40802bc388987)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Upon further reflection, it seems to me that this bbappend ought to just
be deleted at the end of the build. This keeps things simple; you never
have to remember to delete any files to get back to where you were
before with the image. This means we can also drop the slightly awkward
message reminding the user how to do that. I've also updated the test to
look at the image manifest to determine if the command has worked
instead of looking for the (now deleted) bbappend.
(From OE-Core rev: f6b90bceaedf9bad3d111e6ca1fa79e59f472c73)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Make image optional for the extensible SDK (auto-determine it based on
the targets the SDK was built for)
* Check that specified recipe is in fact an image
(From OE-Core rev: 8884875aacfedc69cc72898684e391e69fea00ba)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the user is running "devtool build-image" within the extensible SDK
then they probably won't know where to find the resulting output files,
so we should tell them explicitly.
(From OE-Core rev: 9baf9fd7a53142a98c7f1cd49c7b001760af51f9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We build recipes and include packages into the image, adjust the
terminology used in code and messages accordingly. Also fix a few typos.
(From OE-Core rev: 866f6e2de20b7022803e53c4de3ff341521b4db5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we end up printing a message about the build directory being the same
as the source, we should print that first and then print the message
about the recipe file possibly needing to be edited to the end so that
it has slightly more impact.
(From OE-Core rev: 97398d14c444fe2408dd6101ef46a0a406924bb5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the fetched source isn't already a git repository, initialise it as
one and then branch and tag, just as we do with "devtool modify". This
makes it easier to make changes, commit them and then use the
"devtool update-recipe" command to turn those commits into patches
on the recipe.
(From OE-Core rev: 2dd865086c37c9eff63c6d0bbfa9f2e909f9fffe)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some recipes don't extract any source (for example, opkg-keyrings). We
were producing a traceback in this case because we weren't checking if
the directory existed.
(From OE-Core rev: 087ca0c644aad19e6e7d3921f99f3056471deb65)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Recipes that use work-shared (such as libgcc) are capable of unpacking
the source, but it doesn't necessarily unpack to ${WORKDIR}/${BP}. Use
the last part of the actual S value instead which is more likely to
work.
(From OE-Core rev: 0bab8086fdef93b88d5ec6b88b07ccc921dd786f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you try to use "devtool modify -x" or "devtool extract" on a recipe
where do_unpack has been set as noexec (e.g. glibc-locale), then we get
an error because the expected source wasn't ever unpacked. Do a check up
front for noexec being set on do_unpack and error out with a reasonable
message if that's the case.
(From OE-Core rev: 39a93c85e798e72edd6ab3c05f8adbb1a97e893e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we were adding a recipe for software that would typically be built
in the same directory as the source, we were always using a separate
build directory unless the user explicitly specified not to, leading to
errors for software that doesn't expect to be built that way (such as
Python modules using distutils). Split out the code that makes this
determination automatically from the "devtool modify" and "devtool
upgrade" code and re-use that here so the behaviour is consistent.
(From OE-Core rev: 320585b7ff6340df0b0dbc63f95ed3ca8fc3a993)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These functions ostensibly allowed parsing a recipe without bbappends
but this clearly hadn't been tested because a variable was unassigned in
both of them in that case.
(From OE-Core rev: d2bb9f08303bb120e811c03af2f5339e8f262cfa)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes, particularly if you extracted the source to /tmp which is on
tmpfs, the external source tree that is being pointed to may no longer
exist when you come to run "devtool build" or "devtool update-recipe"
etc. Make all of the commands that need to check for a recipe being in
the workspace call a single function and have that function additionally
check the source tree still exists where appropriate.
(From OE-Core rev: 0c3f289576a2ab35b1d1d8854d6763553cc3bf09)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is not actually used for anything - I thought that we would need to
use it within devtool to set global configuration, but we're able to do
everything we need within the bbappends it creates, which also saves on
parse time. If we're not going to use work-config.inc let's just drop it
completely.
(From OE-Core rev: 84a1f82acd3b6ebb3c073aae6b996d2203dad2ce)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use already existing bb.fetch.decodeurl() for getting the parameters for
a URI. This is more fault tolerant and maintainable.
(From OE-Core rev: cd201664b827e37fcabca272262016b171e997d9)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
So far all devtool and recipetool plugins were expected
to have plugin_init function. This function is empty in
most of plugins as they don't require initialisation.
Making plugin_init optional would allow not having empty
plugin_init in every plugin.
(From OE-Core rev: b99640c89f067866b264f4ee4030fae4c2f338c0)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enables creating packages using devtool within the extensible SDK. (This
is only enabled within the extensible SDK because it provides no
advantage over just running bitbake directly there).
(From OE-Core rev: 6dc0269bca3e874582d61b40dbf0d495331fb96a)
Signed-off-by: Brendan Le Foll <brendan.le.foll@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a recipe has been added to the workspace via "devtool add"
or "devtool upgrade" then the recipe file itself will be in the
workspace layer; if you run "devtool update-recipe" particularly in the
upgrade case you might think it would apply the upgrade to the original
recipe, but it will not - in order to remain consistent it has to update
the recipe that's in use which is the one in the workspace. Warn the
user in this situation so that they know what they need to do when they
are finished.
(From OE-Core rev: 4801b64243e57e554a593f0857dd53621d8f52e5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enhance the logic behind the 'auto' mode a bit by only updating the
SRCREV if the changes are already found upstream. The logic is simple:
update SRCREV only if the current local HEAD commit is found in the
remote branch (i.e. 'origin/<branch_name>'). Otherwise resort to
patching.
This affects a couple of the oe-selftest tests so update those as well.
[YOCTO #7907]
(From OE-Core rev: 9b9733b7d74032aef4979bec553019421e77da14)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Through -s/--disable-parallel-make, the user can turn off parallelism
on the make tool. This can be useful when debuging race condition issues.
In order to set PARALLEL_MAKE = "" a post-config file created and then
passed into the build.
[YOCTO #7589]
(From OE-Core rev: 0bf2e4b3edfc43e7a2e8d3387a9370d110533b7c)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The intention is to have a single file for each devtool feature
so devtool can grow in a modular way. In this direction, this patch creates
build.py, moving all related build features from standard.py to build.py.
(From OE-Core rev: 61bb1759f7ecb8b404f7d97573c61aef31f2f109)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Upgrades a recipe to a particular version and downloads the source code
into a folder. User can avoid patching the source code.
These are the general steps of the upgrade function:
- Extract current recipe source code into srctree and create a branch
- Extract upgrade recipe source code into srctree and rebase with
previous branch. In case the rebase is not correctly applied, source
code will not be deleted, so user correct the patches
- Creates the new recipe under the workspace
[YOCTO #7642]
(From OE-Core rev: 4020f5d91b3e4d011150d5081d36215f8eab732e)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow the _add_md5() function to be called with a directory in order to
recursively add the files under it. Additionally, we need to skip
preserving empty directories (since directories aren't listed in the md5
file).
(From OE-Core rev: 9383af78adc854a6f6de8b1520edf3cea0c477a6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Do not put 'do_shared_workdir' into SRCTREECOVEREDTASKS when creating
bbappend for kernel packages. This will allow building packages that
depend on the shared build artifacts of kernel.
[YOCTO #6658]
(From OE-Core rev: 2355ccc627c0003a14693d1a023a003b7b44ea53)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enable updating the installed extensible SDK from a local or remote
server, avoiding the need to install it again from scratch when
updating. (This assumes that the updated SDK has been built and then
published somewhere using the oe-publish-sdk script beforehand.)
This plugin is only enabled when devtool is used within the extensible
SDK since it doesn't make sense to use it next to a normal install of
the build system.
E.g.
devtool sdk-update /mnt/sdk-repo/
devtool sdk-update http://mysdkhost/sdk
(From OE-Core rev: 32cbd4c57fc8ca097a18929fc404c07322ef36dd)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The kernel package needs "kern-tools-native" in order for it's
do_kernel_metadata. Thus, devtool extract for kernel in a pristine
environment fails. With the current bb.tinfoil implementation it is not
possible to run arbitrary bitbake commands - e.g. run
"bitbake kern-tools-native -c populate_sysroot" in our case. This patch
implements an ugly workaround for that problem, basically by hardcoding
this dependency and running the required bitbake task(s) before tinfoil
is initialized.
[YOCTO #6658]
(From OE-Core rev: a9eb80de164fb8a464e29bf1cc6c7cf397f36662)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Change handling of kernel packages so that the "initial rev" is parsed
correctly. Also, the devtool-specific git tags (devtool-base and
devtoo-patched) are now generated for kernel packages as well.
[YOCTO #6658]
(From OE-Core rev: 65fe3e45b10c5578446c758f66840891c62cd774)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Set SRCTREECOVEREDTASKS appropriately in the workspace .bbappend file
for kernel recipes. This tries to ensure that all needed tasks (esp.
configure and patch) are run when building the kernel - tasks which
would normally be disabled by externalsrc.bbclass.
[YOCTO #6658]
(From OE-Core rev: f0a3a38a653c643468452eba43a12795136720c9)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added couple of hopefully useful comments to the code.
(From OE-Core rev: 72dfe5b58c637d74971e025aef3ce0a64dc8172c)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Removed <image>.bbappend before generating it again as
it may cause tinfoil to fail due to its wrong content.
It's safe to do as <image>.bbappend is regenerated anyway.
(From OE-Core rev: 82c6452ca953eb32e2919d9f9e64497a15212be5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added logger calls to show if image is modified by the
plugin or not.
(From OE-Core rev: f719e956a6263784963b6ae9514030a1a1dc2aeb)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added notification callback to <image>.bbapend to notify
user that image is modified by build-image plugin.
(From OE-Core rev: 53cb00b8af58c326e2a045cd6f8a04a24dc4c1b5)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Filtered out non-target recipes and recipes with
recipe name != package name in build-image plugin.
Isolated all logic of getting recipes in _get_recipes
function.
(From OE-Core rev: efe685711ae6f4beec06ba591c74140ce56b96af)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Made parser help message and description more clear in
build-image plugin.
(From OE-Core rev: 39714557dde70c4b1ce8d08c7e1d21fd39a1d1a6)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Used logger variable name instead of LOG as it is used the rest
of the devtool code.
Pylint complains about 'logger' being invalid constant name,
but it's better to be consistent in naming.
(From OE-Core rev: a1c811df93e831a198464008564ce33ec98a3049)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This reverts commit 69c63728da.
Moved add_md5 back to standard.py as it's not used in
any plugin anymore.
(From OE-Core rev: 3823b7abf8c1e82e0448d7283de95422fd262135)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moved standard.py:_parse_recipe -> __init__.py:parse_recipe and
standard.py:_get_recipe_file -> __init__.py:get_recipe_file
to be able to call them from other modules.
(From OE-Core rev: f0e61a0d5597017c5f5d2dafb41118b79f505d9b)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It doesn't make sense to use it as image recipe is not
in workspace. It means that we can't do 'devtool reset'
for the recipe, which is a main point of using add_md5.
(From OE-Core rev: f69613ed9d56c6e6ba322d8c9db07b7ed802042a)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implemented new plugin to build image from workspace packages.
Plugin creates <image>.bbappend file, adds
all workspace packages to the image using IMAGE_INSTALL_append
variable in bbappend file. After that it runs 'bitbake <image>'.
(From OE-Core rev: 00bc43868da3ea2a4532215d3abef8e150c7b2e5)
(From OE-Core rev: fc35c10fed382e385f00b76abcee94a0148b4aee)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moved _add_md5 function from standard.py to __init__.py to
be able to call it from other modules.
(From OE-Core rev: ee38bb20dc7ba21dac782d8d13383f81dfedef55)
(From OE-Core rev: b07da7d38bcefe8efdd6b22bb9251bef599ef040)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Let's have the caller do this and then the function is a bit more
flexible (e.g. we can choose to parse with bbappends or not); fix up
calls to this function appropriately (of which there are only two, both
within devtool).
(From OE-Core rev: 6ba969f1ac5a1a0e277a21287fc5ae1622a6b14e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a recipe is BBCLASSEXTENDed (e.g. to -native), its PN value and the
name of the bbappend will be different; we were assuming them to be the
same when reading in the workspace, leading to us seeing the base recipe
name everywhere afterwards.
Also add a test so we ensure this doesn't regress in future.
Fixes [YOCTO #8157].
(From OE-Core rev: b63fca00c2e24ad0c8b8b3c492d93ee4372fa92d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure that the "patches" subdirectory is removed from the right
location when S == WORKDIR (e.g. devtool extract makedevs).
(From OE-Core rev: 2062c88726400e09599aff51af95799a866b90c9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In case the user has set up externalsrc outside of devtool, force
EXTERNALSRC to blank for the recipe when extracting so that the original
source URI is still in SRC_URI and we're still able to extract it. (This
isn't a problem with devtool itself because the bbappends within the
workspace layer that apply externalsrc are explicitly filtered out when
devtool parses a recipe).
(From OE-Core rev: 5be16d639d1b78d114755bfd552ac901d0fdf4a9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes it easier to extend, as a layer can add its own sub-commands.
Argument parsing is also separated into two steps, the same way it's done in
recipetool, as we need access to the global command-line arguments early,
before plugins are loaded, both for debugging arguments and for the bitbake
path (we need to load the bitbake module to get tinfoil, which is now needed
to load the plugins).
Rather than constructing tinfoil once and passing it through into sub-commands
for their use, we have to construct it for configuration metadata, use it, and
then shut it down, as some sub-commands call out to recipetool, which needs
its own tinfoil instance, and therefore needs to acquire the bitbake lock. If
we're still holding the lock at that point, that's clearly a problem.
[YOCTO #7625]
(From OE-Core rev: f9bc3b27244a141ec7273445d3ea139a047e0ddf)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If patches fail to apply with git, quilt it used as a fallback. If that
happens, the code in this class is meant to handle cleanup of these patch
files. In the case where ${S} is a subdir of the git tree, the code doesn't
correctly set the patches directory.
This change correctly sets the patches directory (which is different to the
location of the git repository).
[YOCTO #7911]
(From OE-Core rev: de6e0f3af5e858960676ea291036e59105fd806f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It turns out that scp can't be used to copy symlinks because it follows
them instead of copying them, and this is by design (since it emulates
rcp which also behaved this way); the unfortunate result is that
symlinks that point to valid files on the host translate into the host
file being copied to the target (yuck). The simplest alternative that
does not have this undesirable behaviour is to use tar and pipe it over
ssh.
At the same time, it would be even better if we properly reflect file
permissions and ownership on the target that have been established
within the pseudo environment. We can do this by executing the copy
process under pseudo, which turns out to be quite easy with access to
the pseudo environment set up by the build system.
Fixes [YOCTO #7868].
(From OE-Core rev: 69adaed0e982d627ebfa57b360b0ee049ea7a276)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use DevtoolError exception more widely for handling error cases. This
exception is now caught in the main script and raising it can be used to
exit with an error. This hopefully simplifies error handling. The
change also makes exit codes more consistent, always returning '1' when
an error occurs.
(From OE-Core rev: 2e4f1dcade7ccb581c7a390c32163ea3deeac6d5)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Continue refactoring of update_recipe() by splitting out the 'patch'
mode into a separate function.
(From OE-Core rev: cdcfedec5489a5d8d0df56bbe100e5fc2cca03af)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Refactor update_recipe() (i.e. the implementation of the update-recipe
command) by splitting out the 'srcrev' into a distinct function.
(From OE-Core rev: 5da26bfd8b34af9075b9b900d353df555d8f2ef0)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Just refactor the code.
(From OE-Core rev: 06f6b20f040d2e4eee577bb2111351523ee97af2)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A slight simplification of the code.
(From OE-Core rev: aff88bcebe335b0277df660ac22eeed28d65da44)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Split out the logic of determining "initial rev" and "update rev" into a
separate function.
(From OE-Core rev: 17206934822aab31d93318bffea8099bf9965112)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Quite often what you want to do having made customisations to a piece of
software is to apply those customisations in your own layer rather than
in the original recipe. Thus, add a -a/--append option to the
update-recipe subcommand which allows you to specify the layer to write
a bbappend into. The bbappend will be created at the appropriate path
within the specified layer directory (which may or may not be in your
bblayers.conf) or if one already exists it will be updated
appropriately.
(This re-uses code written for recipetool appendfile.)
Implements [YOCTO #7587].
(From OE-Core rev: 87d487ea4fdfb6cd30e3b3fad47732db12e86f23)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Most of the time when bb.note() gets called we want to see the output,
so ensure the level is set appropriately depending on the command line
options instead of being fixed at warning. (We don't want to see the
notes for fetch/unpack/patch though as they are too verbose).
(From OE-Core rev: 69f426a2d966a2228cbdc708b9ddab31005c6d96)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using tinfoil here is quicker and tidier than shelling out to
bitbake -e and interpreting its output.
(From OE-Core rev: 986ad99aee98dd5b7f30d59098dd9275097b8276)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we execute an external command, we ought to prepare for the
possibility that it can fail and handle the failure appropriately. We
can especially expect this to happen when running bitbake in this
scenario. Ensure we return the appropriate exit code to the calling
process.
Fixes [YOCTO #7757].
(From OE-Core rev: 98a716d79bfc5434a5b42d3ca683eab3eea30a41)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Before this change, all files from the recipe (SRC_URI), including
patches, were added to to srctree repository when S==WORKDIR. The patch
files are useless as they are automatically applied on top of the
srctree by devtool.
This change causes devtool extract to not commit these unnecessary (and
possibly confusing) patch file(s) into srctree repository.
[YOCTO #7602]
(From OE-Core rev: 3e0ffff619e49b1f0c13e5f6a663455be3ed26af)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After OE-Core commit 67db7182faf6742b0d971d61d8c5ba34f69d2e12, PV is
appended to the end of the gcc-source PN, thus we need to handle that in
devtool and the corresponding test.
Part of the fix for [YOCTO #7729].
(From OE-Core rev: 440029dc229a566b9bead1481215d5e5760c5fe6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Recipes that fetch from git, svn or hg by OpenEmbedded convention
should normally be named with this as a suffix, since PV is meant to
be set appropriately within the recipe, so follow this. In order to
make this work we need to be able to have the version independent from
the file name, so add a -V option to recipetool create to allow this
to be specified.
* If -V is specified on the devtool add command line, ensure at PV gets
set to include this version.
(From OE-Core rev: 2b997c41c6476a13bf516586d56a9051aceb93ec)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a -f/--fetch option to fetch a remote URI (into the already
specified source tree path) and set this as SRC_URI within the recipe.
This simply wraps around the existing functionality in recipetool.
Implements [YOCTO #7644].
(From OE-Core rev: f22fd77e735fc5f4a3434e3d1f567a9d7d191cf4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If -x is specified and the specified URI was a git repository, we need
to ensure that the resulting clone is a sandalone and not one that has
pointers into the temporary fetch location or DL_DIR (since the git
fetcher does a local clone with -s). Split out the code from devtool
that already does this for "devtool modify -x" and reuse that.
(From OE-Core rev: fc47e8652ef32e7399f57c80593df90dc52d8b84)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you've done "devtool add" (or "devtool modify" without -x) then it's
possible that the external source tree is not a git repository, so we
should handle that case here instead of printing a traceback.
(From OE-Core rev: eb2147aa8facd4ef33a0749e9ae660ec686dad48)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the proper bbappend file name, don't just assume it will have a
version suffix (because it won't if the original recipe doesn't).
Fixes [YOCTO #7651].
(From OE-Core rev: 3332d68ef7b2a300ce8dcf5021497d98e5b17baa)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you manually delete files in the workspace layer (which you really
shouldn't) it was possible to get yourself into the situation where you
couldn't reset because we were attempting to check if the file had been
modified and erroring out if it couldn't be opened. If the file's not
there anymore there's not much point checking if it needs to be
preserved, just skip it.
(From OE-Core rev: d2d352f7b747a97a54df9d20eb1455d166aa1ee3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We weren't adding the version into the bbappend file name when -V was
specified which meant that building or resetting failed.
Also adjust one of the tests so that we're testing devtool add both with
and without this option.
Fixes [YOCTO #7647].
(From OE-Core rev: bdbeff0cd342e31053d7203d78fc5dda611052b1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to get correct metadata, SRCREV for example.
Fixes [YOCTO #7648].
(From OE-Core rev: 8b1794559dd7fd956716179d628e61cffdce1686)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This option can be used to have a separate build directory, in order to
keep the srctree directory clean for packages that do not need to be
built in the source directory.
(From OE-Core rev: 5cc348c0a3016e4ccc1e80d099e2e197b3b38bd2)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Makes the build succeed for packages which do not support separate build
directory, e.g. zlib. The same outcome could be achieved with the
--same-dir option, but, it's generally hard to tell if a random package
would need that option. The negative side effect of this patch is that
dev srctree (of some packages that build fine without this modification)
gets dirtied by build artefacts.
(From OE-Core rev: 6ac5692426956e276ba2119f917b0f30791e6cd6)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Acked-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ubuntu's default dash shell causes oe-init-build-env to behave a bit
differently - (a) it can't pick up the OE root directory and (b) it
can't see any build directory specified as a command-line argument
(since dash doesn't pass through any arguments specified to sourced
scripts). We could work around these but doing so requires some internal
knowledge of the script; a much simpler fix is just to force running the
command under bash since it's expected to be installed on every distro.
Thanks to Chen Qi <Qi.Chen@windriver.com> for this fix.
Fixes [YOCTO #7614].
(From OE-Core rev: 27942f546e6b08cdf9f2dbda2e24d237cde7f5f5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Give the correct data object ("config data" instead of "recipe data") as
an argument to oe.recipeutils.patch_recipe()
Fixes [YOCTO #7595]
(From OE-Core rev: 7d2b918019b0afe215a6489dad697afac34f73e9)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a dry-run option to the deploy-target and undeploy-target
subcommands so you can see the list of files to be deployed or
un-deployed before actually carrying out the operation.
(From OE-Core rev: b9625581f3fe72fc402632be2d87cf889301c6a2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The -q option to scp does stop the progress being shown, which is mostly
superfluous, however it also stops errors from ssh being shown - if
there's a problem, you'll just get "lost connection" which really isn't
that helpful. As a compromise, add a -s/--show-status option and
advertise this when the command fails.
(From OE-Core rev: 5cbb026212b4c8f5206a07d70b94f57edeee0839)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you haven't built the recipe yet or if the output directory (${D}) is
empty, then we should tell the user rather than have scp error out.
(From OE-Core rev: 3fd9ac277393bef5c646f16b9d8f30c277881d70)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you're testing with multiple images/devices that have the same IP
address / hostname then it can be annoying to deal with host key
mismatches all of the time. As a MITM attack is unlikely in the local
test environment, provide a command line option to pass the appropriate
options to scp/ssh to disable the host key checking.
Note: if you wish to apply this permanently, the best way is to do it
through your ssh configuration e.g. by adding the following to your
~/.ssh/config:
Host 192.168.7.2
UserKnownHostsFile=/dev/null
StrictHostKeyChecking no
(From OE-Core rev: 81dd1319112a99bc38b7a7ced0663918ac5b09a4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Consolidate code for checking compatible recipes and consider meta and
packagegroup recipes as well as package-index and gcc-source to be
incompatible.
(From OE-Core rev: 4be9bf637583b341a89af1b9924752abc7d49c94)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you point devtool modify to a source tree previously created by
devtool modify or devtool extract, then we need to try to pick up the
correct initial revision so that devtool update-recipe knows where to
start looking for commits that match up with patches in the recipe.
(From OE-Core rev: c20e10543e268ebb43074a3f8d6e7ed991e54ec8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a -a/--all option to allow you to quickly reset all recipes in your
workspace.
(From OE-Core rev: 0c83788b111a761f6f500b86780cc51aed255402)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
No functional changes, just use a unique name for each parser.
(From OE-Core rev: 5fabc59d6221c3fe7137b70e31ec2761a4276a6c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Pass correct arguments to undeploy() function
* If an error occurs during undeploy(), exit instead of continuing
(From OE-Core rev: c938dee4b28af7e6296c86347dfa533f85605033)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Because we move the workdir when extracting source, then move the source
and delete the temporary workdir, you lose the indirection symlink
pointed to by the alternates file (which is created when the fetcher
clones it from DL_DIR with -s) and the resulting repository is broken.
In any case, for a source repo that the user may put their own changes
into, we can't really rely on a clone made with -s in case the
original goes away - because of cleanall, DL_DIR disappearing, etc. So
repack the repository so that it is a complete, non-shared clone after
unpacking.
(While I'm at it, add a test for devtool modify with a git recipe which
verifies that this works.)
(From OE-Core rev: dccdde4321c48a571677995a4e1dfc583e899cbe)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enable source extraction used by devtool extract / devtool modify -x for
recipes that use a shared workdir (e.g. the kernel and gcc).
(From OE-Core rev: 73a5bdc7921791b954ca717293e76c7e3091a19e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If S is outside of WORKDIR then we shouldn't try to get the relative
path in order to work out where the source subdirectory is as we do by
default.
(From OE-Core rev: 7392ecd559ef71241405c704a65da171ee216ca1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There wouldn't be any point to using these with an image recipe, so
disallow it.
(From OE-Core rev: 611c27182a6e52e159a1ce1b5bac054405c611d2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When updating git-based recipes, in a lot of cases what you want is to
push the changes to the repository and update SRCREV rather than to
apply patches within the recipe. Updating SRCREV is now the default
behaviour for recipes that fetch from git, but this can be overridden
in both directions using a new -m/--mode option.
(From OE-Core rev: 654792bb87610ee3569d02a85fa9ec071bf8ab6d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you want to be able to make use of libraries in conjunction with
devtool then we need to install them into the sysroot for other recipes
to use. Make it a configuration option in case it needs to be changed at
runtime.
(From OE-Core rev: 94f517ad8e55edfbe6f06afd963bcfeb849626ff)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you've added a new recipe, you want the output cleaned when you do
devtool reset, otherwise cruft from building the recipe may remain which
could interfere with future builds.
(From OE-Core rev: 664d1a7fe8f8288fabc582d00f6e36ab29496ec5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The default behaviour is to build in a separate directory to the source,
however some projects can't be built this way, so add an option to do
that (or override the automatic behaviour in the case of modify).
(From OE-Core rev: cf094ed2f616a581eb07d78db4366c83a441fde5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Based on feedback from Scott Rifenbark <scott.m.rifenbark@intel.com>
(From OE-Core rev: ec3378f3a7013e289daa0f5c52329488b861f99c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Deploy recipe output files to live target machine using scp
Store the files list and target machine info in localhost if deployment
is done
Undeploy recipe output files in target machine using the previous
deployment info
[YOCTO #6654]
(From OE-Core rev: cc10751aa3121218b21ec2058358f3ea02a4500a)
Signed-off-by: Junchun Guan <junchunx.guan@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Provides an easy means to work on developing applications and system
components with the build system.
For example to "modify" the source for an existing recipe:
$ devtool modify -x pango /home/projects/pango
Parsing recipes..done.
NOTE: Fetching pango...
NOTE: Unpacking...
NOTE: Patching...
NOTE: Source tree extracted to /home/projects/pango
NOTE: Recipe pango now set up to build from /home/paul/projects/pango
The pango source is now extracted to /home/paul/projects/pango, managed
in git, with each patch as a commit, and a bbappend is created in the
workspace layer to use the source in /home/paul/projects/pango when
building.
Additionally, you can add a new piece of software:
$ devtool add pv /home/projects/pv
NOTE: Recipe /path/to/workspace/recipes/pv/pv.bb has been
automatically created; further editing may be required to make it
fully functional
The latter uses recipetool to create a skeleton recipe and again sets up
a bbappend to use the source in /home/projects/pv when building.
Having done a "devtool modify", can also write any changes to the
external git repository back as patches next to the recipe:
$ devtool update-recipe mdadm
Parsing recipes..done.
NOTE: Removing patch mdadm-3.2.2_fix_for_x32.patch
NOTE: Removing patch gcc-4.9.patch
NOTE: Updating recipe mdadm_3.3.1.bb
[YOCTO #6561]
[YOCTO #6653]
[YOCTO #6656]
(From OE-Core rev: 716d9b1f304a12bab61b15e3ce526977c055f074)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
