Specifically, 'devtool upgrade' will now do these things:
1) determine if any of the license checksums need updating; if so,
write the new checksums into the LIC_FILES_CHKSUM value in the recipe
that is written to the workspace;
2) print a notice to the standard output:
NOTE: New recipe is /home/ak/development/poky/build/workspace/recipes/puzzles/puzzles_git.bb
NOTE: License checksums have been updated in the new recipe; please refer to it for the difference between the old and the new license texts.
3) and the cool part: devtool will create a diff of the old and new licenses,
and write the diff into the workspace recipe as a comment, like this:
======
FIXME: the LIC_FILES_CHKSUM values have been updated by 'devtool upgrade'.
The following is the difference between the old and the new license text.
Please update the LICENSE value if needed, and summarize the changes in
the commit message via 'License-checksum-change:' tag.
(example: 'License-checksum-change: copyright years updated.')
The changes:
--- LICENCE
+++ LICENCE
@@ -1,8 +1,9 @@
This software is copyright (c) 2004-2014 Simon Tatham.
Portions copyright Richard Boulton, James Harvey, Mike Pinna, Jonas
-Klker, Dariusz Olszewski, Michael Schierl, Lambros Lambrou, Bernd
-Schmidt, Steffen Bauer, Lennard Sprong and Rogier Goossens.
+Kölker, Dariusz Olszewski, Michael Schierl, Lambros Lambrou, Bernd
+Schmidt, Steffen Bauer, Lennard Sprong, Rogier Goossens and Michael
+Quevillon.
Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation files
======
(From OE-Core rev: ccb37f9e81eb78ed0eb2a238d7c3e196db9b2f72)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Specifically, 'devtool upgrade' will use the latest upstream release if available
or latest commit if upstream never makes releases.
(From OE-Core rev: 45b4242b105ad36e94ae15a96d588a58b917b8e8)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This command queries the upstream server for what the latest release is and prints
the output; it is a much neater way to find out these things than fumbling with distrodata,
'bitbake -c checkpkg' and awkward to read csv output in a file.
Examples:
python3 (tarballs):
NOTE: Current version: 3.5.3
NOTE: Latest version: 3.6.3
rpm (git):
NOTE: Current version: 4.13.90
NOTE: Latest version: 4.14.0
NOTE: Latest version's commit: da3720f62e57648fb1dc2a632744d38866139971
puzzles (git without version tags):
NOTE: Latest commit: ee8ea9b9785964694cb2b3ad77c3fb2460f49510
(From OE-Core rev: e8f5b5cc25ce7a9882f21473cefc47edcebf77d4)
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you have a recipe that uses overrides to conditionally extend
SRC_URI to add additional patches, then you will often need to update
those patches if you're making other changes to the source tree (for
example if you're upgrading the underlying source). Make this possible
with devtool by creating devtool-override-* branches for each override
that conditionally appends/prepends SRC_URI, and have devtool
update-recipe / finish check each branch out in turn and update the
corresponding patches.
A current example of a recipe that does this is the quota recipe - it
applies an additional patch if musl is the selected C library (i.e.
libc-musl is in OVERRIDES).
Note that use of this functionality does require some care - in
particular, updates to patches that appear on the main branch (named
"devtool" by default) should be made there and not only on one of the
specific devtool-override-* branches that are created for each override.
The recommended procedure is to make the changes you want to make to the
main branch first, then check out and rebase each devtool-override-*
branch, testing each one by activating the corresponding configuration,
and then finally run devtool finish.
Fixes [YOCTO #11516].
(From OE-Core rev: aa87603d1ffd695027847f4df75c0406cf4e14d8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you're not sure what changes devtool finish is going to make, or
you're not sure you're finished with your modifications, it is useful to
be able to see what devtool finish is going to do beforehand, so add
a -N/--dry-run option to make that possible.
(It's also very useful for debugging devtool finish itself.)
(From OE-Core rev: 05f2d5d2ce00c53825ccea5cd9c2262f9d27a638)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a file is going to be effectively removed from the destination by
devtool finish, we should report that rather than just reporting that
we're removing files from the workspace. This is a little tricky because
the way we actually operate when finishing is to:
(1) remove all original files (as recorded by devtool upgrade, if that
was used)
(2) as part of updating the recipe file, remove the files from next to
the new recipe (i.e. in the workspace for an upgrade, real recipe
otherwise) corresponding to commits not in the git tree
(3) copy over remaining files from the workspace to the destination
To report the files removed with respect to what was originally there,
we need to swap steps 1 and 2 so we can see what no longer exists after
the deletion, and suppress the reporting currently done in step 2 -
however, we still want to report removal in step 2 for the non-upgrade
case, so the latter is conditional.
(From OE-Core rev: db1d663507509cac9d97d7c96ac8590478767ba2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the files that the devtool-source class is supposed to create in the
source tree aren't found in the temporary directory then we know that
the class hasn't worked properly - say that explicitly.
(From OE-Core rev: 4621152509c037532b133e5e6d5b73bda7ddb602)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Only log one warning message instead of one per line
* Be a bit more verbose
* "if list" is more pythonic than "if len(list)"
(From OE-Core rev: 2d11e9e6e73648c1cb514c0c10111c7886acae78)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the directory where the source code extracts to changes (for
example, when upgrading iucode-tool from 1.5 to 2.1.1, the subdirectory
in the tarball changed from "iucode_tool-${PV}" to "iucode-tool-${PV}")
then handle this automatically. Also handle when it changes to match the
default S value (i.e. "${WORKDIR}/${BP}") in which case we just drop
setting S in the recipe.
Fixes [YOCTO #10939].
(From OE-Core rev: d29881a652bf03627d257a1eac5f02ec17315b8b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Give the user a little more insight into what's being done.
(From OE-Core rev: 9cf2089bd22b9fc4eb0eec8d4924e44519412dad)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
devtool upgrade did not properly handle setting SRC_URI checksums for
recipes that use named SRC_URI entries and also use those names in the
SRC_URI checksums. A further complication was where the name contained
an expression that changed with the version e.g. ${PV} (probably quite
rare, but the dnsmasq recipe in meta-networking is currently one such
recipe.) All of these are now handled properly.
Additionally, drop the _get_checksums() function that wasn't being
called from anywhere in the code.
Note that this now turns nowrap_vars in recipeutils.py to be a list of
regexes, hence things such as [ and ] need to be appropriately escaped.
(From OE-Core rev: c914a5e1ad6d96e316746222e5d42f2ba9110060)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
devtool finish will check if the destination layer is part of
bblayers.conf so that we avoid the user getting confused about the
recipe vanishing from their configuration if it isn't. devtool finish
also accepts a path underneath a layer so that you have a bit
more control over where it ends up. However if you used a path
underneath a layer then it wasn't converting this to the base of the
layer before checking it against BBLAYERS, thus the warning was being
shown erroneously in that case.
(From OE-Core rev: ab1b8d55e551fea3e8656aab7a786d1bfec62d0f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the git repository for a recipe in the workspace has uncommitted
changes in it then it's possible that the user has forgotten to commit
something, so check and exit if there are any. Provide a -f/--force
option to continue in the case where the uncommitted changes aren't
needed.
Separately, if the repository is in the middle of a rebase or git am /
apply then error out (without the opportunity to force) since the user
really needs to sort this out before finishing.
(From OE-Core rev: bfebd18982c0c82ef2da63ec8f22175c93b2e308)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If S points to a subdirectory of the source rather than the "base" of
the source tree then print that rather than the subdirectory path when
telling the user they need to remove the source tree, since that is the
directory that they will need to remove.
(From OE-Core rev: 9e8808099046478e98c6cf1903dc6787d69132fc)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After some reconsideration I think it's a bit annoying for users to be
forced to use an option to work with recipes where the file isn't in the
workspace, so let's just have these commands check the workspace first
for the recipe, and if it isn't there then load the cache and get it
that way.
(From OE-Core rev: 46683c61069a386658676a79d797062404bf1140)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Show a warning in devtool upgrade if the version is less than the
current version suggesting that the user may need to bump PE in the
recipe
* Show a warning in devtool add and devtool upgrade if the version looks
like a pre-release version suggesting using a version number that
won't mess up the progression when you come to upgrade to the final
release version.
(From OE-Core rev: 92c4d9be9ed365c449ea1ac5a18ddb15e4a36b2d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If S points to a subdirectory of the source rather than the "base" of
the source tree then we weren't handling the oe-local-files directory
properly - it got extracted to the base of the tree but devtool
update-recipe and devtool finish assumed it would be under S which would
be the subdirectory, thus it would be missing and devtool would assume
the files had been deleted and remove them from the recipe. Record the
base of the source tree in the bbappend and read it into the in-memory
workspace so we can use that to find out where oe-local-files should be
found.
(From OE-Core rev: 30d2ea67b2c4727e23d06a35745b1afa64b130cc)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When devtool upgrade is upgrading to a new version where the source is
fetched as an archive (e.g. a tarball), we create a single commit in the
git repository that is the upgrade from the old version to the new. We
do this by extracting the old source, committing it, deleting all files,
copying in the new files, running git add on each new/changed/deleted
file, and then committing the result. When a lot of files have changed
in an upgrade (such as QEMU 2.8.1.1 -> 2.10.0) the penultimate step of
running git add it can take quite a long time; in order to reduce this
and show some feedback to the user, run git add with batches of 100
files at once and also show a progress bar. In a local test with the
aforementioned QEMU upgrade it took the time down from over 7 minutes
down to about 13 seconds.
Fixes [YOCTO #11948].
(From OE-Core rev: 8b184f6c874b60324ee107af53853687173d3434)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With versions of git older than 2.0, "git add" on a deleted file (i.e.
in this case a file that was removed between versions) will not add the
delete to be committed by default, with the result that the rebase of
patches on top of the new branch will fail. We need to use the -A
option in order to force that for older git versions.
(From OE-Core rev: c7f4c9f050c11c0de7fcf5badcc19a8fbc6428cf)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It appears that when fixing the signature unlocking in OE-Core commit
4e9a0be32fc30fb87d65da7cd1a4015c99533aff I swapped the parameters here
and did not test it within the eSDK (it does nothing outside of the
eSDK) resulting in a TypeError when devtool upgrade was used in the
eSDK. Swap the parameters around to the correct ordering.
Fixes [YOCTO #12285].
(From OE-Core rev: 05e2c4ada7083f40866846a21fe76c852f1dfefe)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If SRCREV contains a variable reference, any devtool command that
would try to update it would fail. E.g., if SRCREV = "R${PV}", then
devtool finish without having committed any changes would fail with:
oe.patch.CmdError: Command Error: 'sh -c 'git format-patch R${PV} -o
/tmp/oepatchb_doareb -- .'' exited with 0 Output:
fatal: bad revision 'R'
(From OE-Core rev: 094499c819722ad698ccb64ec65dd439b211c31c)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Search made with the following regex: getVar ?\((.*), True\).
(From OE-Core rev: b848c3cb495905605283c57c79f2ed8ca17758db)
Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Alongside reworking the way devtool extracts source, we now need to
ensure that within the extensible SDK where task signatures are locked,
the signatures of the tasks for the recipes being worked on get unlocked
at the right time or otherwise we'll now get taskhash mismatches when
running devtool modify on a recipe that was included in the eSDK such as
the kernel (due to a separate bug). The existing mechanism for
auto-unlocking recipes was a little weak and was happening too late, so
I've reimplemented it so that:
(a) it gets triggered immediately when the recipe/append is created
(b) we avoid writing to the unlocked signatures file unnecessarily
(since it's a global configuration file) and
(c) within the eSDK configuration we whitelist SIGGEN_UNLOCKED_RECIPES
to avoid unnecessary reparses every time we perform one of the
devtool operations that does need to change this list.
Fixes [YOCTO #11883] (not the underlying cause, but this manifestation
of the issue).
(From OE-Core rev: 4e9a0be32fc30fb87d65da7cd1a4015c99533aff)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since it was first implemented, devtool's source extraction (as used by
the devtool modify, extract and upgrade subcommands) ignored other recipe
dependencies - so for example if you ran devtool modify on a recipe that
fetches from svn or is compressed using xz then it would fail if those
dependencies hadn't been built first. Now that we can execute tasks in
the normal way (i.e. tinfoil.build_targets()) then we can rework it to
use that. This is slightly tricky in that the source extraction needs to
insert some logic in between tasks; luckily we can use a helper class
that conditionally adds prefuncs to make that possible.
Some side-effects / aspects of this change worth noting:
* Operations are a little slower because we have to go through the task
dependency graph generation and other startup processing. There's not
really any way to avoid this though.
* devtool extract didn't used to require a workspace, now it does
because it needs to create a temporary bbappend for the recipe. (As
with other commands the workspace be created on the fly if it doesn't
already exist.)
* I want any existing sysroot files and stamps to be left alone during
extraction since we are running the tasks off to the side, and
especially devtool extract should be able to be used without touching
these. However, this was hampered by the automatic removal process in
sstate.bbclass triggered by bb.event.ReachableStamps when the task
signatures change, thus I had to introduce a way to disable this
removal on a per-recipe basis (we still want it to function for any
dependencies that we aren't working on). To implement this I elected
to use a file written to tmp/sstate-control which gets deleted
automatically after reading so that there's less chance of stale files
affecting future sessions. I could have used a variable but this would
have needed to be whitelisted and I'd have to have poked its value in
using the setVariable command.
Fixes [YOCTO #11198].
(From OE-Core rev: 830dbd66992cbb9e731b48d56fddf8f220349666)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A recipe added with "devtool add" requires to be able to take precedence on recipes
previously defined with PREFERRED_PROVIDER.
By adding the parameter "--provides" to "devtool add" it is possible to specify
an element to be provided by the recipe. A devtool recipe can override a previous
PREFERRED_PROVIDER using the layer configuration file in the workspace.
E.g.
devtool add my-libgl git@git://my-libgl-repository --provides virtual/libgl
[YOCTO #10415]
(From OE-Core rev: adeea2fe6895898a5e6006e798898f0f5dabd890)
Signed-off-by: Juan M Cruz Alcaraz <juan.m.cruz.alcaraz@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If user.name or user.email haven't been set then git rebase can't really
work properly. Check that the user has set these and error out if not.
(Elsewhere we are relying on OE's git patch functionality which forces
a dummy OE value - that's OK there as it's completely under OE's control
and therefore it's OK for a dummy OE user to be the committer, but here
the rebase may require intervention so it's reasonable to have the
user's actual name and email on the operation.)
Fixes [YOCTO #11947].
(From OE-Core rev: 129a3be07e272013be2db17552c13b4d8cc2cf6e)
(From OE-Core rev: 802829f1c38d8c5eee11ba1d9ddd37cf02597f6e)
Signed-off-by: paul <paul@peggleto-mobl.ger.corp.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sorted entries are easier to read.
(From OE-Core rev: d0a123ec564f6d36977e472f8bc63f9c050ee616)
Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
At the moment when fetching source from a git repository you have to
know that you can specify the revision and branch in the URL with
';rev=' and ';branch=' respectively, and you can also get thrown off by
the shell splitting on the ; character if you forget to surround the URL
in quotes. Add explicit -S/--srcrev and -B/--srcbranch options
(consistent with devtool upgrade) to make this easier for the user to
discover and use. (The rev and branch URL parameters will continue to
work, however.)
(From OE-Core rev: 2d86cac853d6daa496c0315a5cb0662ebf1165b0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
OE-Core commit 5a16b3c804c5eca331a1c08a7ce31a54909af105 attempted to use
the same function to get the path to a recipe as the new "find-recipe"
command it implemented, except that cannot work because (a) it didn't
return anything and (b) event if it had tried, a command function can
only return an exit code and we don't want that for find-recipe if it
succeeded. Split out a separate reusable function for both commands.
(From OE-Core rev: d5191840212adbf480961ba6fc68e1ab17e5a77a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we're upgrading a recipe that appends additional patches for, say,
class-native, and we're just upgrading the target variant, then when we
copied the recipe into the workspace we skipped copying the additional patches
for the native variant. This caused warnings because the workspace
recipe is preferred. Look at SRC_URI for all variants when copying files
to work around this.
More work is needed to make it easier to work with recipes that use
BBCLASSEXTEND where you need to build more than one variant at once, but
this at least fixes the immediate ugliness.
(From OE-Core rev: 56bf5e93358187e31160d7893f57906bb3dc7ad7)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If your BBLAYERS has non-absolute paths in it (e.g.
"${COREBASE}/../something") then none of the paths matched in
copy_recipe_files() with the result that no files got copied and you
ended up with an error later on because the recipe file couldn't be
found at the destination. Fix this as well as adding an explicit check
to see if no files got copied - error out earlier if so.
Fixes [YOCTO #10981].
(From OE-Core rev: 3861486ad06f90c8644ebab119bbc5ddb9e693ca)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Patches that we identify as having been "deleted" (i.e. patches in
SRC_URI that no longer appear in the git tree) need to be dropped even
if we're updating in srcrev mode. This fixes the case where HEAD of the
git tree is valid upstream (i.e. no extra commits), but there are
patches left over in the recipe, e.g. when we do devtool upgrade and
then all of the commits rebased on top of the new branch get skipped.
Fixes [YOCTO #11972].
(From OE-Core rev: 350f83dc1e317aeb93539f13966caca6d894f569)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
New devtool deploy-target option --strip which enables deploying
stripped binaries, saving some space on target.
* Copies the files of ${D} into a new directory and strips them in place
* Used oe.package.strip_execs for stripping directory
* Added devtool.conf option "strip" for changing default behavior
Config example:
[Deploy]
strip = true
[YOCTO #11227]
(From OE-Core rev: 7f10c5118793da6ded59ae6e60e796152dbd7ca3)
Signed-off-by: Tobias Hagelborn <tobiasha@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Takes a tar archive created by 'devtool export' and imports (untars) it
into the workspace. Currently the whole tar archive is imported, there
is no way to limit what is imported.
https://bugzilla.yoctoproject.org/show_bug.cgi?id=10510
[YOCTO #10510]
(From OE-Core rev: 2de8ba89ef10fefcc97246dfeb4b8d1e48ee8232)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In case the proposed md5sum to be appended to the .devtool_md5 file
is already present, do not append it.
(From OE-Core rev: f958c5cba3b0d24ca696b2b707857009c9a7b5b8)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default, exports the whole workspace (all recipes) including the source code.
User can also limit what is exported with --included/--excluded flags. As
a result of this operation, a tar archive containing only workspace metadata
and its corresponding source code is created, which can be properly imported
with 'devtool import'.
https://bugzilla.yoctoproject.org/show_bug.cgi?id=10510
[YOCTO #10510]
(From OE-Core rev: f9bc3b5101b554a72298266519dbdd1497f262a6)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When devtool upgrade is run on a recipe with revision specified
that is not on master branch, and branch isn't set by --srcbranch or -B,
then we should get the correct branch and append the branch to the URL.
If the revision was found on multiple branches, we will display error
to inform user to provide a correct branch and exit.
[YOCTO #11484]
(From OE-Core rev: 29ced7387a92aed17b7fe93b1654790a981734c1)
Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit 4657bc9d165e51981e034e73e7b92552e873eef7 replaced the git pull logic with
the git fetch + git reset --hard combo, but resetting to HEAD does not really
pull in new commits from remote... Replace with resetting to the upstream branch
instead.
(From OE-Core rev: 0dcdb146f59a184419bffd4f24cdf8343a43c0ea)
Signed-off-by: Andrea Galbusera <gizero@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* If an error is logged while executing a task, we need to ensure we
exit instead of assuming everything went OK.
* If we receive CookerExit, the server is shutting down and we need to
stop waiting for events and probably exit (knotty does this). This
will occur if an exception or bb.fatal() happens during an event
handler.
This fixes a couple of issues highlighted when using devtool upgrade or
modify on a non-supported recipe with intel-iot-refkit together with
bitbake master, but I'd be very surprised if it were hard to reproduce
in other scenarios.
(From OE-Core rev: 65e644368fc9c294af96906528ee0cf30305e0a6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since we have provide an option to manually enable PREMIRRORS and MIRRORS
in recipetool, we need to make sure devtool is having the same options
as devtool uses recipetool in creating new recipes.
(From OE-Core rev: 198bddeb928a318c3ad168bcb4f83b5cd9a604fb)
Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
devtool find-recipe will prints out the path to the recipe
in a workspace.
This subcommand can also help to find recipe outside of
current workspace using "-a" or "--any-recipe" option.
This enhancement helps developer to get the recipe path
when working with devtool.
[YOCTO #11434]
(From OE-Core rev: 5a16b3c804c5eca331a1c08a7ce31a54909af105)
Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
logger was not defined in scriptutils.py based on the
observation in python traceback.
Traceback (most recent call last):
File "/workdir/poky/scripts/devtool", line 351, in <module>
ret = main()
File "/workdir/poky/scripts/devtool", line 338, in main
ret = args.func(args, config, basepath, workspace)
File "/workdir/poky/scripts/lib/devtool/utilcmds.py", line 55, in
edit_recipe
return scriptutils.run_editor(find_recipe(args, config, basepath,
workspace))
File "/workdir/poky/scripts/lib/scriptutils.py", line 141, in
run_editor
logger.error("Execution of '%s' failed: %s" % (editor, exc))
NameError: name 'logger' is not defined
We pass in logger as parameter to run_editor() from where it has
been called (devtool/utilcmds.py and recipetool/newappend.py),
which both modules already has logger setup.
(From OE-Core rev: 21f04b61973dd9029f0e6bff5445e31cd762bf32)
Signed-off-by: Chang Rebecca Swee Fun <rebecca.swee.fun.chang@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This avoids test failures like:
INFO - ======================================================================
INFO - FAIL [1.755s]: test_devtool_layer_plugins (devtool.DevtoolTests)
INFO - ----------------------------------------------------------------------
INFO - Traceback (most recent call last):
File "/media/build1/poky/meta/lib/oeqa/core/decorator/__init__.py", line 32, in wrapped_f
return func(*args, **kwargs)
File "/media/build1/poky/meta/lib/oeqa/selftest/cases/devtool.py", line 1354, in test_devtool_layer_plugins
self.assertEqual(result.output, s[::-1])
AssertionError: "NOTE: Starting bitbake server...\noY senu[36 chars]rciM" != "oY senuZ s'enoynA morF tiforP oN edaM tfosorciM"
- NOTE: Starting bitbake server...
oY senuZ s'enoynA morF tiforP oN edaM tfosorciM
INFO - ----------------------------------------------------------------------
since there is corruption in the output. Setting the logging up before
calling tinfoil.prepare() resolves this.
(From OE-Core rev: 3c479fb17ae4d3e7e5f0889af0f68257ef66475c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Across devtool and recipetool we had an ugly set of code for ensuring
that we can call an npm binary, and much of that ugliness was a result
of not being able to run build tasks when tinfoil was active - if
recipetool found that npm was required and we didn't know beforehand
(e.g. we're fetching from a plain git repository as opposed to an npm://
URL where it's obvious) then it had to exit and return a special result
code, so that devtool knew it needed to build nodejs-native and then
call recipetool again. Now that we are using real build tasks to fetch
and unpack, we can drop most of this and move the code to the one place
where it's still needed (i.e. create_npm where we potentially have to
deal with node.js code in a plain source repository).
(From OE-Core rev: 8450de16ddb02d863204b411a94c6d84e0f88817)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that we have the ability to run the tasks in a more standard context
through tinfoil, change recipetool's fetching code to use that to fetch
files using it. This has the major advantage that any dependencies of
do_fetch and do_unpack (e.g. for subversion or npm) will be handled
automatically. This also has the beneficial side-effect of fixing a
recent regression that prevented this fetch operation from working with
memory resident bitbake.
Also fix devtool's usage of fetch_uri() at the same time so that we can
completely replace it.
Fixes [YOCTO #11710].
(From OE-Core rev: 9a47a6690052ef943c0d4760630ee630fb012153)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When extracting linux-yocto kernel source, we don't need to dance around
shutting down and starting up tinfoil anymore, we can just execute the
tasks as needed when needed using tinfoil's new build_targets()
function. This allows us to tidy up the code structure a bit.
(From OE-Core rev: 5c7f5031023fb74b5f2f26d6b3c829981f2f54d2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If for any reason the parse_recipe fail in extract command
the process gets locked because Cooker is expecting the
finish event by tinfoil.
For example:
$ devtool extract remake /tmp/remake
ERROR: remake is unavailable:
remake was skipped: PREFERRED_PROVIDER_virtual/make set to make, not remake
(From OE-Core rev: 2c0062b59178fa668b26487b6d2f1e81a0d868e0)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The remote script is run with `set -e`, so doing rm without -f on a
$file that is already gone will exit the whole script, failing any
redeployment. Assume a use case where packages sometimes produces
certain test binaries stored on volatile media (tmpfs), and where the
system is occasionally rebooted.
(From OE-Core rev: db54c9a22a9b66c673df8e836de5e47fc9edda0b)
Signed-off-by: Daniel Lublin <daniel@lublin.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Either both or none of the paths must be passed through
os.path.abspath or things like 'A//B', 'A/./B/', and 'A/B/' in S will
cause unintentional mismatches even when B = "${S}".
Using os.path.abspath for both seems more likely to be correct as that
will also handle the case where ${B} != ${S} but the abspaths are
equal.
(From OE-Core rev: 061f2aab40fecbfe0dcb928baa95d6b3a6b45eed)
Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
find may be provided by busybox, which might be compiled without support
for -exec.
(From OE-Core rev: 404e8e3661469175e1ea087ebfaf3a7867bf4df2)
Signed-off-by: Daniel Lublin <daniel@lublin.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Perf is a tool build from the kernel source, which is normally available
in /work-shared/..., but when devtool is used to modify the kernel
source code, perf is not buildable since it gets an error about being unable
to add a depends to a non-exisit task do_patch.
This patch removes do_patch from the SRCTREECOVEREDTASKS and creates an empty
do_patch task to enable the VarFlags code to have someplace to attach depends
information to.
[YOCT #11120]
(From OE-Core rev: 86c793595e560e7bc52e3cd2a2752746e6adcb76)
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a non-existent event - we already have the actual
bb.build.TaskSucceeded further down in the list hence why it wasn't
noticed earlier.
(From OE-Core rev: 4e059a5ceb6f44401154e89e37f56de1d664a7cb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a task such as do_fetch fails when we're extracting source for a
recipe (within devtool modify / upgrade / extract / sync) then we should
naturally stop processing instead of blundering on; in order to do that
we need to be listening for the TaskFailed event. Thanks to Richard
Purdie for noticing and fixing this.
(From OE-Core rev: 9174b845bf6a6be7753bf6b921959b1f3f2dcbc0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Most of the other extract-based commands have this option but oddly I
left it out for modify - I guess because if I was debugging an issue here
I just used devtool extract to do so, but there's no reason why we can't
have it here and it is useful.
(From OE-Core rev: 98fbc46e1a51237213bd7825a922389d3ab2ad9b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If recipetool returns with exit code 14 this means devtool needs to
build nodejs-native and then call it again. If recipetool returns exit
code 14 again then clearly something has gone wrong and we should just
quit with an error.
(From OE-Core rev: 8d7cced6e06d7c2037f5ab75ac859f501129532e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The change over to recipe specific sysroots means that we can no longer
get a known location simply from configuration for the npm binary - we
need to get the recipe sysroot for nodejs-native, look there for npm if
we need to check it's present, and add that to PATH when calling out to
npm. Unfortunately this means anywhere we need to get that path we have
to have parsed all recipes, otherwise we have no reliable way of
resolving nodejs-native. Thus we have to change recipetool create to
always parse all recipes (the structure of the code does not allow us to
do this conditionally).
In the worst case, if npm hasn't already been added to its own sysroot
and we are fetching from a source repository rather than an npm
registry, this gets a bit ugly because we end up parsing recipes three
times:
1) recipetool startup, which then fetches the code and determines it's
a node.js module, finds that npm isn't available and then exits with
a specific error to tell devtool it needs to build npm
2) when we invoke bitbake -c addto_recipe_sysroot nodejs-native
3) when we re-invoke recipetool
This code is badly in need of refactoring, but now is unfortunately not
the time to do that, so we're going to have to live with this ugliness
for now.
Fixes [YOCTO #10992].
(From OE-Core rev: acfdbd796c99882b8586023c8c6b848716105c8d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
devtool/runqemu.py was relying on STAGING_BINDIR_NATIVE to find the host
tools it needed like qemu-system-<arch>. In the post RSS world, this no
longer exists. This patch points it to
{STAGING_DIR}/{BUILD_ARCH}/{bindir_native}.
[YOCTO #11223]
(From OE-Core rev: 1910f9e9336bfedc8278a3bc02e7e7f934a4fc86)
Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When devtool writes to the kconfig fragment, it writes the output of
the diff command returned from pipe.communicate(). This function
returns binary objects. We should open the kconfig fragment file in
binary mode if we expect to write binary objects to it.
[YOCTO #11171]
(From OE-Core rev: 72bec63ab0e78753fb6ed1794d11beef9485c014)
Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since the tinfoil2 refactoring, if an error occurred during parsing, we
were showing a traceback and not correctly exiting (since we weren't
calling shutdown()). Fix both of these issues.
(From OE-Core rev: 18304036e1b513fd12c049dbf549ba75c503ed84)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Web applications built using e.g. angular2, usually requires that the
packages in devDependencies are available.
Thus, add an option '--fetch-dev' to both devtool add and recipetool, to
add npm packages in devDependencies to DEPENDS.
(From OE-Core rev: f246f820d53b459596fde6758a09f7a0d7db7c4c)
Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After running sdk-install we need to ensure that the standalone sysroots are
updated as done when the eSDK is originally built. Add such a call so this
happens automatically and the envrionment scripts in the SDK work correctly
after updates.
(From OE-Core rev: 4f422071d6f9a074986f399d9e648977bd2e0a68)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Enable using, e.g. host port 2222 for connection to qemu target.
Defaults to 22 for standard ssh/scp port.
[YOCTO #11079]
(From OE-Core rev: a2bfa2cc9ee19f617f7d3b6447896e45eb855d2e)
Signed-off-by: Tim Orling <timothy.t.orling@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
upgrade.py imports oe.recipeutils in meta/lib/ but path to oe.recipeutils
is not provided. This fails populate_sdk_ext.
(From OE-Core rev: 5f140359f859fea9cfe8c8d9c9584bceec875adb)
Signed-off-by: Luck Hoang <huyht1205@gmail.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the move to tinfoil2, the behaviour when parsing failed has changed
a bit - exceptions are now raised, so handle these appropriately.
Specifically when if parsing the recipe created when running devtool add
fails, rename it to .bb.parsefailed so that the user can run bitbake
afterwards without parsing being interrupted.
(From OE-Core rev: b9592bd3c1ab2155b9f62d18ce593b9c88eff279)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When extracting source for a recipe within devtool (for extract, modify
or upgrade) We need to redirect WORKDIR, STAMPS_DIR etc. under a
temporary directory so that:
(a) we pick up all files that get unpacked to the WORKDIR, and
(b) we don't disturb the existing build
However, with recipe-specific sysroots the sysroots for the recipe will
be prepared under WORKDIR, and if we used the system temporary directory
i.e. usually /tmp) as used by mkdtemp by default, then our attempts to
hardlink files into the recipe-specific sysroots will fail on systems
where /tmp is a different filesystem, and we'd have to fall back to
copying the files which is a waste of time. Put the temp directory under
the WORKDIR to prevent that from being a problem.
(From OE-Core rev: e10a973cd9390eacb13bdb99693a0622bd3695f5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
[RP: Add needed mkdirhier call]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There were a few straggling expansion parameter removals left for
getVar/getVarFlag where the odd whitespace meant they were missed
on previous passes. There were also some plain broken ussages such
as:
d.getVar('ALTERNATIVE_TARGET', old_name, True)
path = d.getVar('PATH', d, True)
d.getVar('IMAGE_ROOTFS', 'True')
which I've corrected (they happend to work by luck).
(From OE-Core rev: 688f7a64917a5ce5cbe12f8e5da4d47e265d240f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using devtool modify on the kernel, we have to do a bit of a dance
with tinfoil instances because we only find out that we're working on a
kernel recipe after tinfoil is initialised, but then we need to build
kern-tools-native which we're doing just by running bitbake directly.
With the tinfoil2 changes, a datastore for the recipe that we were
keeping around across the opening and closing of tinfoil is no longer
able to be used. Re-parse the recipe to avoid this problem.
(In future this whole thing will be able to be done in the same tinfoil
instance thanks to tinfoil2, but that refactoring is yet to be done.)
(From OE-Core rev: 06127d0115ba449bf04e2579cd1010065e0ed6e3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.
Search made with the following regex: getVar ?\(( ?[^,()]*), True\)
(From OE-Core rev: 0a36bd96e6b29fd99a296efc358ca3e9fb5af735)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using the setVariable commands here followed by buildFile will result in
"basehash mismatch" errors, and that's expected since we are deviating
*at runtime* from what was previously seen by changing these variable
values. Set BB_HASH_IGNORE_MISMATCH to turn off the errors.
(From OE-Core rev: b0169796f294bbec0397b7eae86454a46b68cdc5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extracting the source for a recipe (as used by devtool's extract, modify
and upgrade subcommands) requires us to run do_fetch, do_unpack,
do_patch and any tasks that the recipe has inserted inbetween, and do so
with a modified datastore primarily so that we can redirect WORKDIR and
STAMPS_DIR in order to have the files written out to a place of our
choosing and avoid stamping the tasks as having executed in a real build
context respectively. However, this all gets much more difficult when in
memres mode since we can't call internal functions such as
bb.build.exec_func() directly - instead we need to execute the tasks on
the server. To do this we use the buildFile command which already exists
for the purpose of supporting bitbake -b, and setVariable commands to
set up the appropriate datastore.
(I did look at passing the modified datastore to the buildFile command
instead of using setVar() on the main datastore, however its use of
databuilder makes that very difficult, and we'd also need a different
method of getting the changes in the datastore over to the worker as
well.)
(From OE-Core rev: eb63b5339014fc72ba4829714e0a96a98e135ee2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If PATCHTOOL is "git", and PATCH_COMMIT_FUNCTIONS is set to "1", for
additional tasks between do_unpack and do_patch, make a git commit. This
logic was previously implemented in devtool itself, but it makes more
sense for it to be implemented in the patch class since that's where the
rest of the logic is for this (or in lib/oe/patch.py). It also makes
it possible for this to work with tinfoil2.
(From OE-Core rev: f24f59ea1d8bc335ea8576f6a346d0935f4a3548)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use Tinfoil.parse_recipe_file() and Tinfoil.parse_recipe() instead of
the recipeutils equivalents, and replace any local duplicate
implementations. This not only tidies up the code but also allows these
calls to work in memres mode.
(From OE-Core rev: f13b56266ee96dfab65a3a7db50e8051aa9f071a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
setup_tinfoil() already calls prepare(), we don't need to call it again
ourselves and doing so with tinfoil2 results in "ERROR: Only one copy of
bitbake should be run against a build directory". Calling prepare()
twice should probably still be allowed, so that ought to be fixed
separately, but in the mean time this code is still wrong so fix it
here.
(From OE-Core rev: 38b8a7d4aff096ea0a62f2ddf3fe2de1df591bf5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In previous implementation, a UnicodeDecodeError exception will be
raised if multi-byte encoded characters are printed by the subprocess.
As an example, the following command will fail in an en_US.UTF-8
environment because wget quotes its saving destination with '‘'(0xE2
0x80 0x98), while just the first byte is provided for decoding:
devtool add recipe http://example.com/source.tar.xz
The patch fixes the issue by avoiding such kind of incomplete decoding.
(From OE-Core rev: 1875ea92546d23abcab1b40b562477a0016f712d)
Signed-off-by: Jiajie Hu <jiajie.hu@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The hello-mod recipe is unusual in that it has only local files in
SRC_URI and builds these out of ${WORKDIR}. When you use devtool modify
on it, devtool puts all of those files in an "oe-local-files"
subdirectory of the source tree, which is not ${S} (or ${B}) any more
and thus building the recipe afterwards fails. It's a bit of a hack, but
symlink the files in oe-local-files into the source tree (and commit the
symlinks with an ignored commit so that the repo is clean) to work
around the problem. We only do this at time of extraction, so any files
added to or removed from oe-local-files after that won't be handled, but
I think there's a limit to how far we should go to support these kinds
of recipes - ultimately they are anomalies.
I initially tried a hacky workaround where I set effectively set B =
"${WORKDIR}" and that allowed it to build, but other things such as the
LIC_FILES_CHKSUM checks still broke because they expected to find files
in ${S}. Another hack where I set the sourcetree to point to the
oe-local-files subdirectory works for hello-mod but not for makedevs
since whilst that is similar, unlike hello-mod it does in fact have
files in the source tree (since it has a patch that adds COPYING) and
thus the same issue occurred.
Also tweak one of the tests that tries devtool modify / update-recipe on
the makedevs recipe to try building it since that would have caught this
issue.
Fixes [YOCTO #10616].
(From OE-Core rev: 857c06d6a1d161bf5a01311d07758bd4241929a3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you have a patch remotely fetched in a recipe (e.g. from an http
server) that needs updating then add a local version and substitute the
entry in SRC_URI to point to it.
One can argue about how desirable it is to be modifying patches fetched
in this way, but then one can argue about how desirable it is to have
such patches in the recipe in the first place - and in any case if
devtool update-recipe is to correctly transfer changes to such patches
made in the git repository within the source tree to the recipe then
there isn't much choice but to do it this way.
(From OE-Core rev: a19c26cc78a181f9dd2706dd42e7e450d7ad4082)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is possible to use gzip or bzip2 to compress patches and still refer
to them in compressed form in the SRC_URI value within a recipe. If you
run "devtool modify" on such a recipe, make changes to the commit for
the patch and then run devtool update-recipe, we need to correctly
associate the commit back to the compressed patch file and re-compress
the patch, neither of which we were doing previously.
Additionally, add an oe-selftest test to ensure this doesn't regress in
future.
Fixes [YOCTO #8278].
(From OE-Core rev: e47d21624dfec6f71742b837e91da553f18a28c5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As of the move to Python 3 and the fixes we applied at that time,
bb.process.run() will return a byte array of length 0 rather than an
empty string if the output is empty. That may be a bug that we should
fix, but for now it's easiest to just check the result here before
treating it as a string. This fixes running "devtool update-recipe" or
"devtool finish" on a recipe which has no source tree, for example
initramfs-framework.
Fixes [YOCTO #10563].
(From OE-Core rev: 66bf6978fc807ecc422fb6b6328f68bc3406cf15)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When you run devtool add on a source tree we attempt to figure out the
correct name and version for the recipe. However, despite our best
efforts, sometimes the name and/or version we come up with isn't
correct, and the only way to remedy that up until now was to reset the
recipe, delete the source tree and start again, specifying the name this
time. To avoid this slightly painful procedure, add a "rename"
subcommand that lets you rename the recipe and/or change the version.
(From OE-Core rev: 9303d8055c45a0f6af295d70a6f6a8b9d8d8a7c9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you run devtool finish to move a recipe created in the workspace by
devtool add or devtool upgrade to a layer, and that layer is not
currently included in bblayers.conf (perhaps unintentionally), then the
recipe will no longer be visible to bitbake. In this scenario, show a
warning so that the user isn't surprised by the recipe "going missing".
(From OE-Core rev: 4da8a58e2997db4f24ae0cac0ba27259d7857a05)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If devtool finish is run on a recipe where the recipe file itself is in
the workspace (e.g. where devtool add / devtool upgrade has been used)
and the specified destination layer is not in bblayers.conf, then we
need to avoid running bitbake -c clean at the end because the recipe has
been moved, but the bbappend is still present in the workspace layer at
that point and so if we do it will fail due to the dangling bbappend.
It's difficult to do the clean at the point we'd want to because tinfoil
is holding bitbake.lock for most of the time, but in any case cleaning
the recipe is less important than it used to be since we started
managing the sysroot contents more strictly, so just disable cleaning
under these circumstances to avoid the problem.
Fixes [YOCTO #10484].
(From OE-Core rev: c6980307d43632f4172e79d9607004203af4e9c8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running devtool add, instead of hiding the recipetool create
output, change it so that it's appropriate to show in the devtool
context and show it in real-time. This means that you get status output
such as when a URL is being fetched (though currently no progress
information.) recipetool create now has a hidden --devtool option to
enable this display mode.
(From OE-Core rev: 219aec8803de4ef04c514c87ecfb15359c9424a6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The new runqemu script assumes that if OECORE_NATIVE_SYSROOT is set then
it shouldn't try to run bitbake to find out the values of various
variables such as DEPLOY_DIR_IMAGE; this assumption is incorrect for the
extensible SDK. To work around this, clear OECORE_NATIVE_SYSROOT in the
environment when running runqemu.
Fixes [YOCTO #10447].
(From OE-Core rev: abff69a48bf3076ce8e21356accdc8d85d2c8dbf)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With recent changes to recipeutils, the list of local files returned
by get_recipe_local_files could possibly include source files. This
only happens when the recipe contains a SRC_URI using subdir= to put
files in the source tree. These files should be ignored when
populating the list of local files for oe-local-files directory.
[YOCTO #10326]
introduced in
OE-Core revision 9069fef5dad5a873c8a8f720f7bcbc7625556309
(From OE-Core rev: 31f1bbad248c36a8c86dde4ff57ce42efc664082)
Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Relying on that awk is installed on the target just to extract the
fourth column (i.e., the free volume size) from `df -P` is an
unnecessary dependency for devtool deploy-target. As it is already
using sed to mangle the output from `df -P`, this can easily be
modified to only extract the free volume size.
(From OE-Core rev: 7bab454b0bf0075fbb2a5de06286a9da1df2adc6)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the user runs devtool add on an npm:// URL (or source tree that uses
node.js), and npm is not available, just build nodejs-native instead of
telling the user they need to do it; if that fails because there isn't
any such recipe (which would be the default, since it's not in OE-Core)
then produce a slightly more readable error message hinting at what the
user needs to do.
Note that this forces the use of nodejs-native rather than npm on the
host - this makes sense for two reasons: (1) we need it to be compatible
with nodejs for the target, and (2) we have to have a recipe for that
anyway, so allowing you to avoid having a recipe for the native version
isn't really beneficial.
There's a bit of a hack in here in order to allow this - for node.js
sources that aren't fetched via npm we don't know that they are that
until we've fetched and unpacked them, by which time we're inside
recipetool and have an active tinfoil instance that will prevent bitbake
being run. To avoid this being an issue, we allow recipetool to get to
the point where we know we need npm and then exit with a specific exit
code, at which point devtool can try to build it and then if that
succeeds, it will re-execute recipetool. This is definitely not ideal,
but it can't really be refactored and done properly until we do the
tinfoil2 refactoring; in the mean time though we still want to be
helpful to the user.
Fixes [YOCTO #10337].
(From OE-Core rev: f40662bde5aab158c4e4c3c3ff5e68665a4194a5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We want to remove the -f/--fetch option at some point (as you can now
specify a URL as a positional argument instead) so display a warning
that it's deprecated if it is used.
(From OE-Core rev: 43476d77a91d50454ca26e016a3413b24e9f3aec)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were supposed to be printing out the specified recipe name here but I
forgot to specify a parameter for the string.
(From OE-Core rev: 87f844e533adfc229a5d26857a82cc6b125216c8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now that recipeutils.validate_pn() properly validates characters used in
the name, we can drop this bit checking for '/' since that's not
permitted by validate_pn(). (The FIXME comment here - that I myself
apparently wrote - is questionable since that function was clearly never
intended to allow '/', perhaps I was misled because it was broken and
did so).
(From OE-Core rev: e010d9be3709cf3c607ffc03c3188abe4e1e9eb4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In keeping with making recipetool create / devtool add as easy to use as
possible, users shouldn't have to know how to reformat git short form ssh
URLs for consumption by BitBake's fetcher (for example
user@git.example.com:repo.git should be expressed as
git://user@git.example.com/repo.git;protocol=ssh ) - instead we should
just take care of that automatically. Add some logic in the appropriate
places to do that.
(From OE-Core rev: 78c672a72f49c4b6cfd8c247efcc676b0ba1681a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's rare but there are recipes that have individual files (as opposed
to archives) in SRC_URI using subdir= to put them under the source tree,
the examples in OE-Core being bzip2 and openssl. This broke devtool
update-recipe (and devtool finish) because the file wasn't unpacked into
the oe-local-files directory and thus when it came time to update the
recipe, the file was assumed to have been deleted by the user and thus
the file was erroneously removed. Add logic to handle these properly so
that this doesn't happen.
(We still have another potential problem in that these files become part
of the initial commit from upstream, which could be confusing because
they didn't come from there - but that's a separate issue and not one
that is trivially solved.)
(From OE-Core rev: 9069fef5dad5a873c8a8f720f7bcbc7625556309)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We should always shut down tinfoil when we're finished with it, either
by explicitly calling the shutdown() method or by using it as a
context manager ("with ...").
(From OE-Core rev: 5ec6d9ef309b841cdcbf1d14ac678d106d5d888a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using PATCHTOOL = "git", the user of the system is not really the
committer - it's the build system itself. Thus, specify "dummy" values
for username and email instead of using the user's configured values.
Various parts of the devtool code that need to make commits have also
been updated to use the same logic.
This allows PATCHTOOL = "git" and devtool to be used on systems where
git user.name / user.email has not been set (on versions of git where
it doesn't default a value under this circumstance).
If you want to return to the old behaviour where the externally
configured user name / email are used, set the following in your
local.conf:
PATCH_GIT_USER_NAME = ""
PATCH_GIT_USER_EMAIL = ""
Fixes [YOCTO #8703].
(From OE-Core rev: 765a9017eaf77ea3204fb10afb8181629680bd82)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The missing split() causes dev and dbg packages to match.
(From OE-Core rev: bf83e0f0a3d52958c4380599f1afc4b8e058afd7)
Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The --wilcard-version flag was only used in the srcrev variant of the
update-recipe command.
(From OE-Core rev: d3057cba0b01484712fcee3c52373c143608a436)
Signed-off-by: Ola x Nilsson <ola.x.nilsson@axis.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Unfortunately to implenent multiconfig support in bitbake some APIs
had to change. This updates code in OE to match the changes in bitbake.
Its mostly periperhal changes around devtool/recipetool
[Will need a bitbake version requirement bump which I'll make when merging]
(From OE-Core rev: 041212fa37bb83acac5ce4ceb9b7b77ad172c5c3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a subcommand which will "finish" the work on a recipe. This is
effectively the same as update-recipe followed by reset, except that the
destination layer is required and it will do the right thing depending
on the situation - if the recipe file itself is in the workspace (e.g.
as a result of devtool add), the recipe file and any associated files
will be moved to the destination layer; or if the destination layer is
the one containing the original recipe, the recipe will be overwritten;
otherwise a bbappend will be created to apply the changes. In all cases
the layer path can be loosely specified - it could be a layer name, or
a partial path into a recipe. In the case of upgrades, devtool finish
will also take care of deleting the old recipe.
This avoids the user having to figure out the correct actions when
they're done - they just do "devtool finish recipename layername" and
it saves their work and then removes the recipe from the workspace.
Addresses [YOCTO #8594].
(From OE-Core rev: fa550fcb9333d59b28fc0e4aebde888831410f5c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This provides us with the information we need to remove the original
version recipe and associated files when running "devtool finish" after
"devtool upgrade".
(From OE-Core rev: 92eb42c347af919cd9f8739515fdf806c12b5ba8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will be called by "devtool finish" to allow it to update the recipe
or create the bbappend depending on the destination.
(From OE-Core rev: 5067cdc73483b53d46d9bf584723e41957c7ec54)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This will be called by "devtool finish" to allow it to reset the recipe
at the end.
(From OE-Core rev: b8d398516556eaf97679e28ad58448f570984b52)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If there are files in the oe-local-files directory which are identical
to the original version, then we shouldn't be copying them to the
destination layer. This is particularly important when using the -a
option to create a bbappend.
(From OE-Core rev: 9230bfcc839eb35630949f0a8ed058ca1fa944b1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
devtool update-recipe was defaulting to the ${BPN} named directory when
adding patches next to a recipe, but that meant if you already had files
in a ${BP} named directory (i.e. name and version) or "files" then you'd
end up with two directories next to the recipe, which is usually not
what you want. To avoid this, look through FILESPATH and take the first
one that's the same level or one level down from the recipe and already
exists, if any.
(From OE-Core rev: c7a8190cf8bdf86ba850b6780b8e951e90232c06)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is currently possible to specify a file (e.g. a tarball) on the local
disk as the source, but you have to know to put file:// in front of it.
There's really no need to force users to jump through that hoop if they
really want to do this so check if the specified source is a file and
prefix it with file:// if that's the case.
Also ensure the same works for "devtool add" at the same time.
(From OE-Core rev: 71350003790c38e84b0e525a71a2fe5d24e3d083)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Certain recipes cannot be used with devtool extract / modify / upgrade -
usually because they don't provide any source. Return a specific exit
code (4) so that scripts such as scripts/contrib/devtool-stress.py know
the difference between this and a genuine failure.
(From OE-Core rev: ffd295fed4ab81fc0bd00bb145ef4d72c49584bf)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were attempting to open the recipe file unconditionally here - we
need to account for the possibility that the recipe file has been
deleted or moved away by the user.
(From OE-Core rev: 47822a2aff56fd338c16b5ad756feda9f395a8a1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In OE-Core revision 7baf57ad896112cf2258b3e2c2a1f8b756fb39bc I changed
the default update-recipe behaviour to only update patches for commits
that were changed; unfortunately I failed to handle the --initial-rev
option which was broken after that point. Rework how the initial
revision is passed in so that it now operates correctly.
(From OE-Core rev: b2ca2523cc9e51a4759b4420b07b0b67b3f5ac43)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow specifying more than one recipe on the devtool reset command line.
Also tweak the help text slightly.
(From OE-Core rev: ad92ed8e4f7f48a3d212962531d596b36f6b284f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Converted return value of items() keys() and values() to
lists when dictionary is modified in the loop and when
the result is added to the list.
(From OE-Core rev: 874a269eb1d70060c2f3b3f8b70800e2aea789f4)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Moved call of decode('utf-8') as close as possible to
call of subprocess API to avoid calling it in a lot of
other places.
Decoded binary data to utf-8 where appropriate to fix devtool
and recipetool tests in python 3 environment.
(From OE-Core rev: 30d02e2aa2d42fdf76271234b2dc9f37bc46b250)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python 3 doesn't have basestring type as all string
are unicode strings.
(From OE-Core rev: e8cfab060f4ff3c4c16387871354d407910e87aa)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
xrange() no longer exists in python 3, use range()
(From OE-Core rev: d022b4335100612d6596cc4c4956cb98ed5873cc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Iterators now return views, not lists in python3. Where we need
lists, handle this explicitly.
(From OE-Core rev: caebd862bac7eed725e0f0321bf50793671b5312)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If fetching source from a git repository, typically within OpenEmbedded
we encourage setting SRCREV to a fixed revision, so change to do that by
default and add a -a/--autorev option to use "${AUTOREV}" instead.
(From OE-Core rev: 000480c42797dd2f03ebc3bc6d1dabfc6a7b75f5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The -S / --srcrev option must be specified if fetching from a git
repository, so spell that out in the help text.
(From OE-Core rev: 1465f205c235a1688a85844ebf5259e8971038ae)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make a couple of changes to the rebase operation:
1) Only wrap the actual rebase command in try...except since a failure
in any of the other commands should be an error, not a warning
2) If it's a conflict (which unfortunately we can only tell by checking
for the keyword "conflict" since git doesn't return error codes based
on the type of error) then print a message clarifying that the user
needs to resolve the issue themselves to finish the upgrade.
(From OE-Core rev: 2a76be958432a35a0de30e9a5433089a54a06cad)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The gdb recipe in OE-Core has an inc file with the version in it;
since the inc file is pulled in with a "require ${PV}.inc", when
upgrading the recipe we need to also rename the inc file it will fail to
parse and the upgrade itself will fail.
Fixes [YOCTO #9574].
(From OE-Core rev: 3c623aac9333d20a62475279c72b6b6ec3d7dd6b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having two code paths here makes maintenance difficult, and it doesn't
seem likely that you would use the local case in real usage anyway, so
drop the local support entirely.
This should allow us to resolve [YOCTO #9301].
(From OE-Core rev: 7a4c9c96fee4fb514c2b69b52e811c4f896a16f1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If an image with the filename foo.bb could be built using the name "bar"
instead, then build-sdk would fail to create the derivative sdk.
This was because the code assumed that the file name matched the target,
which is not necessarily the case.
(From OE-Core rev: d58a326b6960be14b8a049253559aec9582b7d0d)
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If UNINATIVE_CHECKSUM changes over an SDK update, bitbake within the
extensible SDK will be broken because it will see that the matching
uninative tarball doesn't exist and if there is a default value of
UNINATIVE_URL it will attempt to download the file and will then fail
because the checksums don't match up; alternatively if no UNINATIVE_URL
is set then it'll also fail with an error about misconfiguration. To fix
this, add some logic to devtool sdk-update to download the matching
uninative tarball(s) for the checksum(s) in the newly fetched SDK
configuration.
Fixes [YOCTO #9301].
(From OE-Core rev: 14ff58ad98a5afac08db77068d80f152d8875766)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Replace git pull with fetch and reset to avoid the merge logic in the
event that the layers repo in the published SDK we're updating to isn't
fast-forward merge from the local repo.
Also add gitignore and committer info during publish to avoid errors and
to be sure that the first commit has a dummy user in it.
[ YOCTO #9368 ]
(From OE-Core rev: 4657bc9d165e51981e034e73e7b92552e873eef7)
Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is unusual but not impossible to find recipes whose first entry is
not the main source URL but instead some patch or other local file, for
example python-cryptography in meta-python (which sets SRC_URI before
inheriting pypi). There's nothing inherently wrong with this, and we
shouldn't assume that the first entry is the main source URL, so just
take the first non-local entry instead.
(From OE-Core rev: c868198c1f6006789707b497c2ae34d7cc5e706f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Strictly speaking we ought to explicitly shut down a tinfoil instance
when we're done with it. This doesn't affect modify's operation but is
important if you want to be able to call into modify() from another
plugin (though anyone doing so should be advised that the function is
by no means a stable API and is subject to change in future releases).
(From OE-Core rev: 626dbadf22b57a22a8f8b9d1957937120f4ba4d5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we add from a fetched URL we are supposed to turn the resulting
source tree into a git repository (if it isn't already one). However, we
were using the older deprecated option name here instead of the
positional argument, so "devtool add -f <url>" resulted in the repo
being created but "devtool add <url>" didn't, which was wrong.
Also update the oe-selftest tests to check that this worked.
(From OE-Core rev: a7b6b1f8cc1c096724f794ac9dee312b0f771f66)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The npm class just installs whatever is in ${S}; if you're using
externalsrc in conjunction with it the symlinks (oe-workdir and oe-logs
by default) plus singletask.lock will end up in the final package, which
isn't really right. Introduce a variable so we know the path the files
will be installed into within npm.bbclass, and append to do_install
within the workspace bbappend to delete the files.
(From OE-Core rev: 766845e06db9d7d595e836ea1364c16fa132a413)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code here for running do_configure if it hadn't already been run was
using the wrong string substitution parameters; fix it and test it.
(From OE-Core rev: b2677a4448dbc42e523c731b953b44006749252c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default the sdk-install subcommand expects to restore the requested
items from sstate and fails if it can't. If the user is OK with building
from source, add a -s/--allow-build option to allow them to do that. In
the process, ensure we show the status output while we're installing.
Also add the missing header to the top of the file.
(From OE-Core rev: a86b426cdd465ec5cb08bb5fa7729e4e673d94bb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Otherwise (if the symlink is named .config) kernel build considers
source tree as dirty and fails.
[YOCTO #9270]
(From OE-Core rev: d36699b28c661880957d744420df9e23cdd76957)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow plugins to create additional files to go alongside the recipe. The
plugins don't know what the output filename is going to be, so they need
to put the files in a temporary location and add them to an "extrafiles"
dict within extravalues where the destination filename is the key and
the temporary path is the value.
devtool add was also extended to ensure these files get moved in and
preserved upon reset if they've been edited by the user.
(From OE-Core rev: 334b9451111b7e3efbb43b3a4eecebcab8ec6f0e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a build-sdk command which is only available within the extensible
SDK that builds a derivative extensible SDK. The idea is recipes in the
workspace become a part of the new SDK - for example, this allows taking
a vendor provided SDK, adding a few libs and then producing a new SDK
with those included.
When normally building the extensible SDK, the workspace is excluded;
here we need to copy into the new SDK (renaming it in the process); the
recipes' task signatures become locked and thus the sources are no
longer needed, so they are removed along with the workspace bbappends
which would interfere with the locked signatures. Additionally we need
to just copy the configuration files (i.e. local.conf and auto.conf)
rather than filtering and appending to them since that work has already
been done when constructing the original SDK. The extra sstate artifacts
from workspace recipes are also determined and copied into the new SDK
in minimal mode (on the assumption that you won't set up a new sstate
mirror).
This reuses some code from build-image, so that needed to be
generalised to allow that.
Implements [YOCTO #8892].
(From OE-Core rev: 59e207ff6dd4b50a8905e14bc9292cf2794f4e7a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Hyphens aren't allowed in python identifiers, so you shouldn't use them
in module names or they are more difficult to import.
(From OE-Core rev: e8f6db9436dfc923e236f2cbc08f357e3b24fd5d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"sdk_update" uses a variable newsdk_path, which was never declared.
This would cause the command:
devtool sdk-update <poky-sdk-latest>
to fail with an error:
NameError: global name 'newsdk_path' is not defined
The remedy is to declare newsdk_path as it was no doubt intended,
corresponding to the argument specifying <poky-sdk-latest>.
[YOCTO#9042]
(From OE-Core rev: 6ac589a4a9290665f8c5295e2e2a03a2b187b957)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make this consistent with "devtool add" so that the user knows where to
find the new recipe.
(From OE-Core rev: 28fea4a104ba85aded2ecfe891e9bf997d72a103)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The PR value should be reset to the default when upgrading, so we need
to drop it from the newly created file.
(From OE-Core rev: 4acf8a6912106dd3da5ceee9e30ccce86211d04b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We aren't modifying the datastore copy here, so we don't need a copy at
all.
(From OE-Core rev: 9f05bf98d4dbcb991b684a2ab480c6638c6be292)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix several issues when extracting the new version source over the top
of the old one (when the recipe is not fetching from a git repo):
* Delete the old source first so we ensure files deleted in the new
version are deleted. This also has the side-effect of fixing any
issues where files aren't marked writeable in the old source and thus
overwriting them failed (harfbuzz 1.1.3 contains such files).
* Fix incorrect variable name in abspath statement that made it a no-op
(From OE-Core rev: 4f7595a514661f0e2418393286849a1cb6c46f0d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we do an upgrade from one tarball version to another we want to:
1) Check out the old version as a new branch
2) Record the changes between the old and new versions as a commit
3) Check out the old version with patches applied
4) Rebase that onto the new branch
Where we went wrong was step #1 where instead we checked out the old
version with patches applied as the new branch, which meant the rebase
didn't do anything and any changes made by the patches to files still in
the new version were wiped out.
(From OE-Core rev: a4f3e6d5f4c549e36cce1e18801a565a3a3c9b0a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the actual value of PV isn't in the name of the recipe (for example,
a git or svn recipe) there's no point trying to rename it. Additionally,
we already have the original filename, there's no need to guess it -
just pass it in.
(From OE-Core rev: e144db3650b4a7c0a59066621905f34a5400303e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We were trying to move this from the current directory instead of the
path. Let's just use shutil.move() instead of shelling out to mv.
(From OE-Core rev: 60454a0ba154d6c777e0c2b05b887b4e4fcde986)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For recipes that specify SRCREV, the code here wasn't quite doing the
right thing. If the recipe has a SRCREV then that needs changing on
upgrade, so ensure that the user specifies it. If it doesn't, then it'll
be "INVALID" not None since the former is the actual default, so handle
that properly as well. Additionally an unset variable was being
erroneously passed when raising the error about the version being the
same leading to a traceback, so fix that as well.
(From OE-Core rev: 1d0f821371d1cb93e30fad86f0c20e38cb93b54b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recipename argument to devtool upgrade specifies an existing recipe,
so by definition the name will be valid (or it won't exist) - we don't
need to validate it ourselves, that's only needed for situations like in
devtool add where we're creating a new recipe.
(From OE-Core rev: 1e2bc7d861555a04350a87f19047efdc717046be)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make devtool upgrade consistent with devtool add/modify in defaulting to
sources/<recipename> under the workspace if no source tree path is
specified.
(From OE-Core rev: 8a952407b192313515e91570632446b6dff01665)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you for example ran devtool modify virtual/libusb0 without specifying
a source tree path, the default was <workspace>/sources/virtual/libusb0
which isn't correct - it should be using the mapped name i.e.
libusb-compat (in the default OE-Core configuration). Reorder some of
the code to ensure that the mapped name is used.
(From OE-Core rev: c51736df17da8e6e561dd5b7ce59cb08254da870)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As suggested by Khem Raj.
(From OE-Core rev: 36cc6b81d0281348a0f241a80ddd427745a6a678)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create config fragment if the user makes modifications to kernel config.
User may change .config e.g. by directly editing it or by running the
'do_menuconfig' bitbake task. Devtool generates one monolithic fragment
by simply doing a diff between .config and .config.baseline files in the
source directory. If either of these files is missing, the config
fragment is not gerenrated or updated. The output is a file,
'devtool-fragment.cfg' that gets added to SRC_URI in the recipe (as well
as copied into the 'oe-local-files' directory if that is present in the
source tree).
${S}/.config will be a symlink to ${B}/.config. We need to do this as
devtool is not able to access ${B} because ${B} is set in a .bbappend in
the workspace layer which is not parsed by devtool itself.
[YOCTO #8999]
(From OE-Core rev: 524da136e5b837a60682516ac08f3092c635e934)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Copy kernel config is copied to the source directory at a later phase in
_extract_source() so that it gets copied when devtool sync is done, too.
(From OE-Core rev: ff895be7a46c4b3b1b791e5387490d90bb34fce2)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I should have adjusted this in OE-Core commit
80a44e52609a89d9ffe816181ae193af491c06ac where the behaviour changed.
(From OE-Core rev: 13409a2b899bb74b8060c840b8c7ef8759d099cb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If files would be overwritten by the deployment, preserve them in a
separate location on the target so that they can be restored if you
later run devtool undeploy-target.
At the same time, also check for sufficient space before starting the
operation so that we avoid potentially failing part way through.
Fixes [YOCTO #8978].
(From OE-Core rev: a2da55712691bcf1066c53d813107d75213d6b10)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you want to put the target device back to exactly how it was before
devtool deploy-target started poking things into it, then it would make
things easier if you didn't have to figure out which recipes were
deployed. Now that we have the list stored on the target we can
determine this reliably, so add a -a/--all option to undeploy-target to
undeploy everything that has been deployed.
One of the side-effects of this is that the dry-run functionality for
undeploy-target had to be reimplemented to actually run the script on
the target, since we have no way of knowing what's been deployed from
the host side. We don't need to do the same for deploy-target though
since we know exactly which files will be deployed without referring to
the target.
(From OE-Core rev: 41fed83060f0041e14e455d1446397bda277d953)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When running devtool deploy-target, we save a list of deployed files,
and this list is used by devtool undeploy-target (or the next time
deploy-target is run if the list is present, in case any files have been
renamed or deleted since the first time). We were writing this file to
the host, but it makes more sense to write the list to the target
instead, so that if we for example swap in a different board, or switch
hosts, things will work as expected.
In order to do this properly we have to construct a shell script and
ship it over to the target so we can run it. The manifest is written out
to a hidden directory in the root (/.devtool).
Fixes [YOCTO #7908].
(From OE-Core rev: a16a0c9334b785e2df896266c8911a2c7a1806b8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Get the default value for updateserver from the configuration file and
show it in the help; also only make the parameter optional if it's
specified. This means we can also drop the check in the function as
argparse will then ensure it's specified if there's no config setting.
(From OE-Core rev: 82497bde58fc8bdae8d8acfbf025a1a90b14cc3e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a long description used when running --help on the specific command.
(From OE-Core rev: eb7787d1652fd84a149fd394969f4f1099406051)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The listing of subcommands in the --help output for devtool was starting
to get difficult to follow, with commands appearing in no particular
order (due to some being in separate modules and the order of those
modules being parsed). Logically grouping the subcommands as well as
being able to exercise some control over the order of the subcommands
and groups would help, if we do so without losing the dynamic nature of
the list (i.e. that it comes from the plugins). Argparse provides no
built-in way to handle this and really, really makes it a pain to add,
but with some subclassing and hacking it's now possible, and can be
extended by any plugin as desired.
To put a subcommand into a group, all you need to do is specify a group=
parameter in the call to subparsers.add_parser(). you can also specify
an order= parameter to make the subcommand sort higher or lower in the
list (higher order numbers appear first, so use negative numbers to
force items to the end if that's what you want). To add a new group, use
subparsers.add_subparser_group(), supplying the name, description and
optionally an order number for the group itself (again, higher numbers
appear first).
(From OE-Core rev: e1b9d31e6ea3c254ecfe940fe795af44761e0e69)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If we didn't make any changes to the file then there's no point warning
the user that we have done.
(From OE-Core rev: 391b9ba30d802ac420ddf382588e03e718861c01)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If files had been created next to the recipe (for example devtool add,
edit the source and commit and then devtool update-recipe), running
devtool reset failed to preserve those files and gave an error due
to trying to rmdir the directory containing them which wasn't empty.
Fix the preservation of files in the "attic" directory properly so
we catch anything under the directory for the recipe, and replicate
the same structure in the attic directory rather than slightly
flattening it as we were before.
(From OE-Core rev: bbe63eb97ae7f78959f117d6066ef821c4da1c77)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Looking at Chris Larson's code for starting the user's editor for
"recipetool newappend" it was slightly better than what I wrote for
"devtool edit-recipe" in that it checks VISUAL as well as EDITOR and
defaults to vi if neither are set, so break this out to its own function
and call it from both places. The broken out version passes shell=True
however in case it's a more complicated command rather than just a name
of an executable.
(From OE-Core rev: 184a256931e8cdc7bea97a905c4e67a435964de0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no -N/--name option for devtool, that's a recipetool option -
with devtool you just specify the name as a positional argument.
(From OE-Core rev: d2bc0cba5ca8a7220ffe1ef96acf856fe972ce7c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fix a regression introduced in in OE-Core revision
aedfc5a5db1c4b2b80a36147c9a13b31764d91dd where specifying a local source
tree without specifying a name resulted in a traceback.
Fixes [YOCTO #9086].
(From OE-Core rev: 67ea9d21f20a371a0ee443b46326018cb1527b62)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's going to be more common for users not to have the prepared source
tree for a recipe already, so the default behaviour ought to be to
extract it for them from the recipe. Change the default to extract
(effectively making the -x option a no-op) and add a --no-extract/-n
option to disable it. Later we can look at trying to be smart and
reusing an existing source tree instead of erroring out if it exists;
for now this is just the default reversal.
(From OE-Core rev: 80a44e52609a89d9ffe816181ae193af491c06ac)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When extracting source for a recipe, if there are additional custom
tasks run that make changes to the source, create a commit in the
generated git branch so they are contained. This is particularly
useful for tasks that come before do_patch since otherwise the changes
might get incorporated in the first applied patch, but otherwise it
helps avoid the tree being dirty at any point.
Fixes [YOCTO #7626].
(From OE-Core rev: 997a77d9b20af1778b804778e5d8c8a7424f7582)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When you need to set EXTRA_OECONF for a recipe, you need to know what
options the configure script actually supports; the configure script
however is only accessible from within a devshell and (at least in the
case of autotooled software fetched from an SCM repository) may not
actually exist until do_configure has run. Thus, provide a "devtool
configure-help" subcommand that runs the configure script for a recipe
with --help and shows you the output through a pager (e.g. less),
prefaced by a header describing the current options being specified.
There is basic support for autotools, cmake and bare configure scripts.
The cmake support is a little hacky since cmake doesn't really have a
concise help option that lists user-defined knobs (without actually
running through the configure process), however that being a design
feature of cmake there's not much I can think of to do about that at
the moment.
(From OE-Core rev: 0e5d84d9705091b338000ef02720cfa090f76888)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we run the tasks required to extract the source for a recipe (e.g.
within "devtool modify" or "devtool extract") if one of those tasks
fails you get a bb.build.FuncFailed exception; handle this properly so
you don't see a traceback.
(From OE-Core rev: 95d8631b3bdf216001e57f48277535c65a4cc49e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a recipe generated by "devtool add" has been modified since then when
you run "devtool reset", it will be moved into the "attic" subdirectory
of the workspace in case those modifications need to be preserved. It
seems natural that if those modifications were worth preserving we
should warn the user if such a file exists when they run "devtool add"
to create the same recipe again, so they can pick up where they left off
if they want to.
(From OE-Core rev: 0a39b907ff997c3a62c92ab22325c726b612de5b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Provide an option to devtool build-image to specify the list of packages
instead of taking the list of packages produced by recipes in the
workspace. Sometimes you don't want all of these packages; other times
you want to add more.
This is the most immediate fix for [YOCTO #8855], though it is a little
crude so I would like to provide better means of customising the image
contents later.
(From OE-Core rev: b3a44951a74fe58714b72e71a7a558b67a71e1e3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
standard.py is getting a bit large; move the "utility" commands to
another module.
(From OE-Core rev: 5089b93f5b341dc28c343f7afe15efda2081ed36)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes you don't want to build an entire project, just a subdirectory
of it; add a --src-subdir option to make that easier. (We still look for
a single subdirectory in what gets unpacked, e.g. what you might find
within a tarball, so whatever you specify with this option is added onto
the end of that.)
(From OE-Core rev: 59682d78f95732e014f78f13e0a05f843860d9bb)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add the ability to install additional pre-built items (from shared
state) into the extensible SDK. This can already be done implicitly by
adding something to DEPENDS within a recipe you're working on and then
running "devtool build", but it's useful to be able to explicitly
install things particularly if you're using the extensible SDK as a
traditional toolchain.
Note that for this command to be useful you need to have SSTATE_MIRRORS
set in your SDK configuration, and that mirror needs to be populated
with sstate artifacts for recipes you wish to be able to install.
(From OE-Core rev: 3474a42954908d1688fd3a6cb600eed315b27833)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make the following improvements to the SDK update process:
* Use a manifest file with sha256sums to track files other than sstate
and metadata that we need to update - e.g. conf files. This allows us
to handle where files such as auto.conf may or may not be present,
as well as the configuration changing without affecting task signatures
- we still want the config files copied in that case rather than it
saying nothing needs to be done.
* Write the SSTATE_MIRRORS_append to site.conf rather than local.conf
so that local.conf remains static (since we don't want to trigger an
update every time). Also, If there is an SSTATE_MIRRORS value already
set in the configuration we can skip this and assume it contains the
needed packages.
* Allow the update process to be run in any directory, don't assume
we're already at the base of the SDK
* Where practical, fetch remote files into a temporary location and
then move them to the desired location at the end, to avoid a
failed update leaving the SDK in a broken state.
* Update all installed do_populate_sysroot / do_packagedata tasks
instead of using the SDK targets. This ensures any item installed
through dependencies after installation (e.g. when running
"devtool build") won't go stale.
(From OE-Core rev: 3d35631121f0e030bc8151f5c23d84008d06f44b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Use tempfile.mkdtemp() instead of hardcoding temp dir
* Set a variable early for the temp locked sigs file and use that
everywhere
* Delete the temp dir at the end
(From OE-Core rev: bad5d1a8c047a8118d30d9fa708b021d1599e0dc)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When you run devtool build, you need to have the pkgdata written out at
the end, so that if what you're adding is a library and the next thing
you add is something that depends on that library, the necessary
information to map the dependency back to the recipe is present. In
practical terms all this means is we need do_packagedata to run in
addition to do_populate_sysroot.
This does mean that do_package needs to run which wasn't running before,
and that means that the few package QA tests that run within do_package
such as installed-vs-shipped will now be run. This may be a bit
bothersome, and prompted a fix for one of our oe-selftest tests as a
result, but I don't see an easy way around it. Ultimately if you care
about using the recipe in an image you'll need to fix any such errors
anyway.
Fixes [YOCTO #8887].
(From OE-Core rev: 6579c7120ee5a541427ff5b6b07f838d52f9fe7c)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a variable SDK_INCLUDE_PKGDATA which you can set to "1" to include
pkgdata for all recipes in the world target. There are a couple of uses
for this:
1) If you use "devtool add" to add a recipe that builds something which
depends on anything in world, the dependency can then be correctly
mapped to the recipe providing it and that recipe can be added to
DEPENDS, since we have the pkg-config and shared library dependency
data within pkgdata.
2) You'll be able to search for these recipes and any files they
package for the target with "devtool search" since that also uses
pkgdata
This of course assumes you've tailored world through EXCLUDE_FROM_WORLD
to only include recipes you'd want built in your distro, but I think
that's a reasonable assumption; failing that there is a
WORLD_PKGDATA_EXCLUDE variable that you can set to exclude any recipes
you don't want.
Note that this patch relies on functionality implemented in a recent
BitBake patch and will not work without it.
Implements [YOCTO #8600].
(From OE-Core rev: 67149ea097d6fab7496b43e85a40853f40bd527e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than horrible workarounds, use the new --setscene-only option
of bitbake to pre-populate the SDK structures.
(From OE-Core rev: 38237b7ac53c416f85c4a70a61acafc3404c8b5f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's logical that you would want to build BBCLASSEXTENDed items
separately through devtool build, so simply allow that - we're just
passing the name verbatim to bitbake, so all it means is adjusting the
validation.
(From OE-Core rev: 25dc5ac42c9da53c01416e7fdcc819d729281133)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the recipe file itself was created in the workspace, and it uses
BBCLASSEXTEND (e.g. through devtool add --also-native), then we need to
clean the other variants as well.
(From OE-Core rev: e1bf6a30679a83d4dbcf37276204f639541e95f9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're doing this in a couple of places, let's just find the recipe file
if it exists within the workspace (which it will if it's been added
through "devtool add") when we read in the workspace.
(From OE-Core rev: 81cf5580993c99050e3f4d6d891bc67534721487)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes you need to build a variant of a recipe for the build
host as well as for the target (i.e. BBCLASSEXTEND = "native"); add a
--also-native command line option to "recipetool create" that enables
this and plumb it through from an identical option for "devtool add".
(We could conceivably do the same for nativesdk, but I felt it might be
confusing within the context of the extensible SDK, where nativesdk
isn't really relevant to the user.)
(From OE-Core rev: f3bea83db173cce921a9a30f04e88b7e3ed98854)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to run the clean for all recipes that are being reset before we
start deleting things from the workspace; if we don't, recipes providing
dependencies may be missing when we come to clean a recipe later (since
we don't and couldn't practically reset them in dependency order). This
also improves performance since we have the startup startup time for the
clean just once rather than for every recipe.
(From OE-Core rev: c10a2de75a99410eb5338dd6da0e0b0e32bae6f5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For debugging purposes it's useful to be able to skip the preparation
step so you can inspect what the state of the build system is first.
(From OE-Core rev: 0bba4b5afd2ce2c3a79445eee886979a77f1a4d8)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Running "raise" with no arguments here is invalid, we're not in
exception handling context. Rather than also adding code to catch the
exception I just moved the check out to the parent function from which
we can just exit.
(From OE-Core rev: 0164dc66467739b357ab22bf9b8c0845f3eff4a4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Clone the correct path - we need .git on the end
* Pull from the specified path instead of expecting a remote to be set
* up in the repo already (it isn't by default)
(From OE-Core rev: 1a60ee8bd21e156022c928f12bb296ab5caaa766)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We read the updateserver setting from the config file but we never
actually used that value - the code then went on to use only the value
supplied on the command line.
Fix courtesy of Dmitry Rozhkov <dmitry.rozhkov@intel.com>
(From OE-Core rev: 1c85237803038fba539d5b03bf4de39d99380684)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Copied layers with 'cp -a' instead of calling shutil.copytree as
copytree fails to copy broken symlinks.
More pythonic fix would be to use copytree with 'ignore' parameter,
but this could slow down copying complex directory structures.
[YOCTO #8825]
(From OE-Core rev: e5b841420b9fdd33829f7665a62cd06a3017f7e6)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Much of this was copy/pasted from the extract subcommand code; make it
specific to sync.
(From OE-Core rev: 08ca966dd48db881d8bfb7d7fa0aba49f1f7cffe)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We deliberately leave the source tree alone when resetting in case it
contains any work in progress belonging to the user; tell them that
we're doing this so they aren't surprised about it still existing later
on.
(From OE-Core rev: 1c616802ba2cfb37f8403f1dc6cf91ce3bd99ac2)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a recipe in the workspace actually exists as a file within the
workspace (e.g. after doing "devtool add" or "devtool upgrade") then
show the path to the recipe file on the status line for the recipe.
(From OE-Core rev: 29833828a90c8433af3b231b50e99cd97edf19ff)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As per the changes to "devtool add", make the source tree path optional
and use the default path if none is specified.
(From OE-Core rev: 83707d1334fb094fd1877bcfd07a83866601048a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having to specify -f is a little bit ugly when a URI is distinctive
enough to recognise amongst the other positional parameters, so take it
as an optional positional parameter. -f/--fetch is still supported, but
deprecated.
(From OE-Core rev: aedfc5a5db1c4b2b80a36147c9a13b31764d91dd)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
recipetool create now has all the logic in it for auto-detecting the
name and version, and using those in the file name - so we can make the
name an optional parameter for devtool add and we pick up the file name
that recipetool has used after the fact.
(From OE-Core rev: 70ab08146e930f1fc55fdf5726a87303e20bd60f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Assuming we're fetching source remotely (from a URI) we can default the
source tree that will be extracted from it to a "sources" directory
under the workspace in order to save the user specifying it if they
don't have a preferred location.
(From OE-Core rev: ffdad964c7271972e4b067e4898bf7c338c25b68)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The bbappend already exists at this point, so we know what its path is -
there's no need to figure it out from scratch here.
(From OE-Core rev: c0754d672966901f22dff1bcd40bbd08d1219c7a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're repeating this in a couple of places, so we might as well have a
function to do it.
(From OE-Core rev: 67a28109a1ee1383d1b17a8dafa4fe510948238b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a few clarifying words.
(From OE-Core rev: 2103fa9dc7faf2189c8b426b87fb9d421a9983ac)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add an "edit-recipe" subcommand that runs your default editor (as
specified by the EDITOR environment variable) on the specified recipe in
the workspace. Note that by default the recipe file itself must be in
the workspace - i.e. as a result of "devtool add" or "devtool upgrade";
however there is a -a/--any-recipe option to override this.
(From OE-Core rev: dbfe8fa2e86c2bb50bef47c389017cdf93543321)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the SDK update server hasn't been set in the config (when building
the extensible SDK this would be set via SDK_UPDATE_URL) and it wasn't
specified on the command line then we were failing with a traceback
because we didn't pass the default value properly - None is interpreted
as no default, meaning raise an exception if no such option exists.
Additionally we don't need the try...except anymore either because with
a proper default value, NoSectionError is caught as well.
(From OE-Core rev: 9763c1b83362f8445ed6dff2804dd7d282861f79)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add 'do_kernel_configme' and 'do_kernel_configcheck' to
SRCTREECOVEREDTASKS of kernel packages. These tasks should not be run
because kernel meta in the srctree is not necessarily up-to-date or
even present which causes build failures and/or invalid kernel config.
Especially so because 'do_patch' which is a dependency of
'do_kernel_configme' is not being run.
We now store .config in the srctree and 'do_configure' task is able to
run successfully.
(From OE-Core rev: 7ce4c18a4ba1ebcb9f46e652a881ace1f21d2292)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This makes the correct kernel config to be used when building kernel
from srctree (extrernalsrc). If no kernel config is present in the
builddir 'do_configure' task copies .config from the srctree.
(From OE-Core rev: 3b516332e038a587685f6e0c14a7f04990bdd6cc)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use bb.utils.edit_metadata() to replace some of the logic in this
function; this avoids us effectively having two implementations of the
same thing. In the process fix the following issues:
* Insert values before any leading comments for the next variable
instead of after them
* Insert overridden variables (e.g. RDEPENDS_${PN}) in the correct place
* Properly handle replacing varflag settings (e.g. SRC_URI[md5sum])
(From OE-Core rev: 0f81b83fc5fd908efa7f6b837137830ca65f6ed6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
