Commit Graph

245 Commits

Author SHA1 Message Date
Ross Burton
c472b4ce59 recipetool: don't dump stack traces if a toml parser can't be found
If we can't find tomllib or tomli then we can just tell the user politely
that we can't parse the pyproject.toml file, there's no need to dump
exception stack traces.

Move the parser exception handler to catch the actual parse, as otherwise
it will never be used.

Whilst here, also add some debug statements to make it clear what of the
handlers is being called.

(From OE-Core rev: 1a3ba4c312844d80ae382912b319e60ad8b30737)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-10 14:13:24 +00:00
Adrian Freihofer
233d2d8c0d recipetool: cleanup imports
- os is missing
- glob is not used

(From OE-Core rev: f560bc8972e97d2e497d7b976586e4a79ffc74ec)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-08 23:14:51 +00:00
Vyacheslav Yurkov
5eb10c53d8 recipetool: Disregard version in URL for replaced modules
Major module version is a part of name, but not necessary part of the
actual URL (See https://go.dev/ref/mod#module-path).

Nevertheless, name detection function can't handle that suffix, so get
rid of it to determine component name.

For replaced modules that name might be different that the actual module
name defined in go.mod file.

(From OE-Core rev: 0cccfa1041d48f0ae3a2dc89a129cf7884fc08f0)

Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-19 12:21:22 +00:00
Vyacheslav Yurkov
0576bd1edd recipetool: Proceed even with a missing license file
Whenever the recipe uses a CLOSED license, the list is going to be
empty. It's a discouraged practice not to have a license, but proceed
anyway to finish recipe generation.

(From OE-Core rev: 5ca920284d0946346f5b06f5e443c80d9d8b85ce)

Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-19 12:21:22 +00:00
Vyacheslav Yurkov
0161d08ac9 recipetool: Don't fail on local go modules
Local modules are usually referenced with a 'replace' directive in
go.mod file. If that's the case, remove them from populating SRC_URI.

(From OE-Core rev: 9f220f61e3e44a650a46ee997b47f1d87b7c4ef0)

Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-19 12:21:22 +00:00
Tim Orling
9707dc5549 recipetool: add python_maturin support
Add the new python_maturin PEP-517 backend

Add selftest for 'pydantic-core' pypi package.

(From OE-Core rev: 69b679380616a94a631681caa05d9bf7610f9372)

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-17 19:07:21 +00:00
Tim Orling
78ef0313ee recipetool: pypi: do not clobber SRC_URI checksums
The pypi change:
"85a2a6f68af recipetool: create_buildsys_python: add pypi support"
deleted all the SRC_URI variables, including the SRC_URI checksums.
These are not generated by the pypi.bbclass (how could they be trusted?)

Without the checksum(s), we are vulnerable to a man-in-the-middle attack
and zero checks on the validity of the downloaded tarball from pypi.org.

Fix by only setting S and SRC_URI to None.

(From OE-Core rev: 560181a52111569f7bc57b09139b42510e0d0325)

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-09 19:17:11 +00:00
Peter Kjellerstedt
45d2f8d4bc recipetool: create: Only include the expected SRC_URI checksums
Rather than including all SRC_URI checksums, include the ones that are
expected. These are the same as are output if no checksums are included
when building the recipe.

(From OE-Core rev: c2af83eb5e8573480179b6c0bcce50606b547099)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-08 16:58:34 +00:00
Peter Kjellerstedt
decf6e66df recipetool: create: Improve identification of licenses
Rather than having a static list of crunched MD5 checksums for some of
the most common licenses, calculate it for all common licenses. This
should improve the identification of license text variantions.

(From OE-Core rev: 377f9513dc56e9b8e5f5813c1535be0206756949)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-08 16:58:34 +00:00
Julien Stephan
bc9291c5fd recipetool: appendsrcfile(s): add a mode to update the recipe itself
Add a new option -u/--update-recipe to update the recipe itself instead
of creating/updating a bbappend file

(From OE-Core rev: c9a80f55ed0bf9ce249205d6a9ba670fbb7f59ba)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:50 +00:00
Julien Stephan
29dc0d7315 recipetool: appendsrcfile(s): use params instead of extraline
appendsrc function relies on oe.recipeutils.bbappend_recipe to
copy files and add the corresponding entries in SRC_URI.

Currently, appendsrc function build itself the new SRC_URI entry to add the
correct subdir param, and gives it using the extralines parameter.
This has 2 drawbacks:
- oe.recipeutils.bbappend_recipe can already do this if we specify the
  correct params, so we have duplicate code
- the duplicated code is not fully functional: for example, it doesn't
  take into account the -m/--machine parameter

So fix this by not using extralines but give correctly formatted params.

Also remove the check for already existing entries as
oe.recipeutils.bbappend_recipe already implement it

The new bbappend file now have the SRC_URI entry after the
FILESEXTRAPATHS so fix the selftest.

Update test_recipetool_appendsrcfile_existing_in_src_uri_diff_params
test because recipetool appendsrcfiles used to not add new src_uri entry
if the entry already exist even with different parameters while
oe.recipeutils.bbappend_recipe adds it if parameters are different (and
remove the old entry)

(From OE-Core rev: cd5de8d53849a6f3bb6f82e45fb301e39892c789)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
de5ab8e863 recipetool: appendsrcfile(s): add dry-run mode
Add dry-run mode for recipetool appendsrcfile and appendsrcfiles, with
if necessary, a diff of changes

(From OE-Core rev: 456c726e6b52f4dc57ff605e1cf1687097537002)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
85a2a6f68a recipetool: create_buildsys_python: add pypi support
Today, we can use devtool/recipetool to create recipes for python projects
using the github url or the direct release tarball of the project, but the
create_buildsys_python plugin doesn't support the pypi class, since we cannot
know from the extracted source if the package is available on pypi or not.

By implementing the new optional process_url callback, we can detect
that the url is a pypi one (i.e 'https://pypi.org/project/<package>')
and retrieve the release tarball location.
Also detect if the url points to a release tarball hosted on
"files.pythonhosted.iorg" (i.e https://files.pythonhosted.org/packages/...)

In both cases, adds the pypi class, remove 'S' and 'SRC_URIxxx'
variables from the created recipe as they will be handled by the pypi class
and add the PYPI_PACKAGE variable

This helps to produce cleaner recipes when package is hosted on pypi.

If the url points to a github url or a release tarball not coming from
"files.pythonhosted.org", the created recipe is the same as before.
One can also use the newly added "--no-pypi" switch to NOT inherit
from pypi class on matching url, to keep legacy behaviour.

To create a recipe for a pypi package, one can now use one of the
new following syntax (using recipetool create / devtool add):

* recipetool create https://pypi.org/project/<package>
* recipetool create https://pypi.org/project/<package>/<version>
* recipetool create https://pypi.org/project/<package> --version <version>

or the old syntax:
* recipetool create https://files.pythonhosted.org/packages/<...>

(From OE-Core rev: 097a43846cd99a7d74d004efc57f583ce78970a4)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
6c06fb0a43 recipetool: create: add new optional process_url callback for plugins
Add a new process_url callback that plugins can optionally implement if
they which to handle url.

Plugins can implement this callback for example, to:
* transform the url
* add special variables using extravalues
* add extra classes
* ...

If a plugin handles the url, it must append 'url' to the handled
list and must return the fetchuri

No functional changes expected for plugins non implementing this
optional callback

(From OE-Core rev: 3e5ce351e7dbe283562bf1db1f2ce3b121c49b53)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
86b9510bd3 recipetool: create: add trailing newlines
create_recipe() function relies on oe.recipeutils.patch_recipe_lines()
which relies on bb.utils.edit_metadata(). edit_metada expect lines to
have trailing newlines, so add it to each lines before calling
patch_recipe_lines, otherwise edit_metadata will not be able to squash
blank line if there are two consecutive blanks after a removal

(From OE-Core rev: 31b27cc76a1b669e2b126c332608fd772c124d69)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
344e10a21b recipetool: create_buildsys_python.py: initialize metadata
In the case pyproject.toml doesn't contains metadatas, the metadata
variable is not initialized and the plugin throws an error and falls back
to another plugin, which is not the desired behaviour. So just ignore
metadata if we don't have them

(From OE-Core rev: 88d15877ba18309c521740d7a9649e14d77189bc)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
89f1662484 devtool: add support for git submodules
Adding the support of submodules required a lot of changes on the
internal data structures:
* initial_rev/startcommit used as a starting point for looking at new
  / updated commits was replaced by a dictionary where the keys are the
  submodule name ("." for main repo) and the values are the
  initial_rev/startcommit

* the extractPatches function now extracts patch for the main repo and
  for all submodules and stores them in a hierarchical way describing the
    submodule path

* store initial_rev/commit also for all submodules inside the recipe
  bbappend file

* _export_patches now returns dictionaries that contains the 'patchdir'
  parameter (if any). This parameter is used to add the correct
  'patchdir=' parameter on the recipe

Also, recipe can extract a secondary git tree inside the workdir.

By default, at the end of the do_patch function, there is a hook in
devtool that commits everything that was modified to have a clean
repository. It uses the command: "git add .; git commit ..."

The issue here is that, it adds the secondary git tree as a submodule
but in a wrong way. Doing "git add <git dir>" declares a submodule but do
not adds a url associated to it, and all following "git submodule foreach"
commands will fail.

So detect that a git tree was extracted inside S and correctly add it
using "git submodule add <url> <path>", so that it will be considered as a
regular git submodule

(From OE-Core rev: 900129cbdf25297a42ab5dbd02d1adbea405c935)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-01 11:48:25 +00:00
Lukas Funke
7de36b25d1 recipetool: Add handler to create go recipes
[RP: Replace python 3.9 use of removeprefix()]
(From OE-Core rev: 049afb933f42851d56d2e6522150a32bcccf2a6f)

Signed-off-by: Lukas Funke <lukas.funke@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-06 16:41:11 +00:00
Lukas Funke
f7f304fa3e recipetool: Ignore *.go files while scanning for licenses
(From OE-Core rev: 1765acc8bdc08e392daa8b77add8b4ce1a9e70b6)

Signed-off-by: Lukas Funke <lukas.funke@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-06 16:41:11 +00:00
Tim Orling
9ffac3fc3d recipetool: add python_hatchling support
One of the newer PEP-517 backends to be added was python_hatchling.bbclass
but it was not included in the recent improvements.

Add selftest for 'jsonschema' pypi package.

(From OE-Core rev: d99b4883b4fee82bc588fd235ba90fedf1550cb8)

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-30 08:50:46 +00:00
Julien Stephan
d4debbf5b7 recipetool/create_buildsys_python: add PEP517 support
Add basic support for PEP517 [1] for the 3 following backends that are
supported by bitbake:
  - setuptools.build_meta
  - poetry.core.masonry.api
  - flit_core.buildapi

If a pyproject.toml file is found, use it to create the recipe,
otherwise fallback to the old setup.py method.

Some projects can declare a minimal pyproject.toml file, and put all
the metadata in setup.py/setup.cfg/requirements.txt .. theses cases
are not handled. If a pyproject.toml file is found, assumes it has all
necessary metadata.

As for the old setup.py method, version numbers for dependencies are not
handled.

Some features may be missing, such as the extra dependencies.

[YOCTO #14737]

[1]: https://peps.python.org/pep-0517/

(From OE-Core rev: c7d8d15b2d0a9ecd210bd247fa0df31d9f458873)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 08:28:38 +01:00
Julien Stephan
e64e92f2de recipetool/create_buildsys_python: refactor code for futur PEP517 addition
In order to prepare the support for pyproject.toml (PEP517 [1]) enabled
projects, refactor the code and move setup.py specific code into a
specific class in order to allow sharing the PythonRecipeHandler class

No functionnal changes expected

[1]: https://peps.python.org/pep-0517/#source-tree

(From OE-Core rev: 2281e93347da4129062cfb40710df03c87c63168)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 08:28:38 +01:00
Julien Stephan
be129bd0bc recipetool/create_buildsys_python: prefix created recipes with python3-
By convention, all python recipes start with "python3-" so update
create_buildsys_python to do this

This rule doesn't apply for packages already starting with "python"

Update recipetool's selftest accordingly

(From OE-Core rev: b0d87440e610b80f763d09784d4a90a148bb3e7b)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 08:28:38 +01:00
Julien Stephan
0021e3573a recipetool/create_buildsys_python: fix license note
License field of setup is not always standardized, so we usually use the
classifier to determine the correct license format to use in the recipe.

A warning note is added above the LICENSE field of the create recipe
in case a license is provided in setup. But when the plugin is called,
"LICENSE =" is not yet present so we can never display this note.
Replace the "LICENSE =" condition with "##LICENSE_PLACEHOLDER##"
to actually be able to display the note message

(From OE-Core rev: b7c26ca2028aa60f740464de85a11a01a531f32e)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 08:28:38 +01:00
Chris Laplante
893e17f669 recipetool/create_buildsys_python: use importlib instead of imp
'imp' was deprecated in Python 3.4 and removed in 3.12. The
piece of importlib we use has been around since 3.3.

(From OE-Core rev: 457f0dad87b4e45a53865b5ad2c150215bd74019)

Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-06 11:51:11 +01:00
Richard Purdie
d4f2f8269c recipetool/devtool: Ensure server knows about changed files
Bitbake has changed to require notification when metadata changes in the middle of tinfoil
sessions. Add the required function calls at the places metadata is changed.

(From OE-Core rev: e5574163ab49a8f51b2b34fd37acfd1cad9b7595)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-09-18 11:35:38 +01:00
Richard Purdie
65318019cd recipes/classes/scripts: Drop SRCPV usage in OE-Core
Now that SRCPV isn't needed we can simplify things in a few places...

(From OE-Core rev: 843f82a246a535c353e08072f252d1dc78217872)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-08-24 16:50:24 +01:00
BELOUARGA Mohamed
2dacac93bc recipetool: create: npm: Add support to handle peer dependencies
NPM changed its manner to handle peer dependencies over its versions.
Before NPM 3: NPM installs automatically peer dependencies
between NPM 3 and 7: NPM shows a warning about peer dependencies
After NPM 3: NPM reworked its manner how to handle peer dependencies

The shrinkwrap doesn't have the parameters of the peer dependencies, so we cannot
fetch them. in the same time peer dependencies are not direct dependencies, they should
be installed as run time dependencies.

(From OE-Core rev: f36021a749974ef3d4a6abe4d5429544a815071a)

Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-29 11:10:39 +01:00
BELOUARGA Mohamed
dbc1da2fb8 recipetool: create: npm: Add support for the new format of the shrinkwrap file
The shrinkwrap file changed its format, but npm does not version this file. So we can use it properly.
The actual changes make the script check if the npm package has dependencies in the actual shrinkwrap format.

(From OE-Core rev: 488d17c2af0c927ec66f0eee124bf6fc5b7f7c95)

Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-29 11:10:39 +01:00
BELOUARGA Mohamed
2a3888069f recipetool: create: npm: Remove duplicate function to not have future conflicts
Npm packages do not have yocto friendly names. fore instance we can have names like
"@example/npmPackage"

npm fetcher has a function that convert these names to yocto friendly names.
But in recipe tool we have an other function (duplicate).

(From OE-Core rev: 18e5438de5389b58c8b6a548d4474128d510a28d)

Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-29 11:10:39 +01:00
Yoann Congal
356d2369c7 recipetool: Fix inherit in created -native* recipes
native and nativesdk classes are special and must be inherited last :
put them at the end of the gathered classes to inherit.

(From OE-Core rev: a6614fd800cbe791264aeb102d379ba79bd145c2)

Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-28 23:02:50 +01:00
Richard Purdie
dc850a1066 recipetool: Update for class changes
(From OE-Core rev: 0ea8afd4f9599469d1a23824c451c62eabb76660)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-12 15:27:17 +01:00
Thomas Roos
603652a38e recipetool/devtool: Fix python egg whitespace issues in PACKAGECONFIG
Substitute expressions or whitespace from python egg requires.txt when
generating PACKAGECONFIG

Pysetuptools sees the uvicorn.egg-info/requires.txt as extra requirements.
Recipetool parses this information to generate the PACKAGECONFIG.
These extra requirements contain expressions and whitespace, which are not allowed in PACKGAGECONFIG.
This patch substitute them by hyphens to make PACKAGECONFIG parsable and readable.
Also adding an oe-selftest for this.

[YOCTO #14446]

(From OE-Core rev: a854d95a79e64f3f82abfa4cc1daec750abf4249)

Signed-off-by: Thomas Roos <throos@amazon.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-01 11:31:42 +01:00
Stefan Herbrechtsmeier
f72889eb00 recipetool: Do not use mutable default arguments in Python
Remove mutable default arguments in Python because they can lead to all
sorts of nasty and horrible bugs.

https://florimond.dev/en/posts/2018/08/python-mutable-defaults-are-the-source-of-all-evil/

Revert `recipetool: Change default paramter fallback_licenses of
function split_pkg_licenses from None to []` and instead check
fallback_licenses before use.

(From OE-Core rev: 99dee60b8db557f54783bf0f61098587badc683c)

Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-19 14:14:12 +01:00
Richard Purdie
8a1de49b56 recipetool/devtool: Further SPDX identifier cleanups
Some of these are hard to know what to do with since the original source
files for the checksum aren't present. The safe option is to use "-only"
as often the main license is ambiguous and the source files themselves
determine the "or-later" possibility. The "-only" option therefore is
realistically what we need to use in this code.

(From OE-Core rev: 2b0cbafc7854de0308a624b17b8aaba704b031d5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-02 18:43:24 +00:00
Peter Kjellerstedt
68d1a56e98 recipetool: Use SPDX license identifiers
There are still a couple of cases where the license may be set as, e.g.,
"GPL" or "GPL-2.0" since there is not enough information to decide the
actual SPDX license. It is then assumed that the developer will have to
correct the information.

(From OE-Core rev: e7df51f8d2361e9fe2d67669d2e17f0a5d01004e)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-01 23:44:59 +00:00
Peter Kjellerstedt
949aa76210 recipetool/create_buildsys_python: Add support for more known licenses
Add all OSI approved licenses from https://pypi.org/classifiers/. Also
add support for Other/Proprietary (Proprietary) and Public Domain (PD).

(From OE-Core rev: 99ef134d1019e5b98b845cf71f3eb39871218f9d)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-01 23:44:59 +00:00
Saul Wold
254fbcf614 recipetool: Fix circular reference in SRC_URI
When creating a new recipe.bb file for a binary, don't use BP which
includes the version information, instead use BPN which is just the
name base Package Name.

Since PB is not specified, it takes the default:
PV = "1.0+git${SRCPV}"

But SRCPV is defined in terms of the SRC_URI, which leads to infinite
recursion (traceback below). Here are the pertinent variables which
cause the recursion:

SRC_URI = "git://github.com/lvc/abi-dumper;protocol=https;subdir=${BP}"
BP = "${BPN}-${PV}"
PV = "1.0+git${SRCPV}"
SRCPV = "${@bb.fetch2.get_srcrev(d)}"

def get_srcrev(d, method_name='sortable_revision'):
    # ... trimmed
    scms = []
    fetcher = Fetch(d.getVar('SRC_URI').split(), d)
    # ... trimmed

[YOCTO #14040]

(From OE-Core rev: 3b8d43fc53ee13d39abc3b2a1f706a97fcf752aa)

Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-05 17:46:05 +00:00
Tim Orling
fe86a14649 recipetool/create_buildsys_python.py: less distutils
'distutils' is deprecated in Python 3.10 with removal in Python 3.12
(~October 2023). Replace 'distutils.command.build_py' with
'setuptools.command.build_py'.

To avoid an AttributeError, we call super().__init__() which provides
the missing 'distribution' attribute. However, for some reason, __init__()
in 'setuptools.command.build_py.build_py' class requires a 'dist' positional
argument which must be a 'Distribution' instance. It is not clear why
'distutils.command.build_py.build_py' class does not require this.

There is still a check which decides to inherit setuptools3 vs distutils3
that will need to be refactored when we add pyproject.toml and setup.cfg
support for more modern PEP 517 packaging.

Once distutils3.bbclass is dropped, any recipe inheriting distutils3
will throw a parsing error. The plan is to move distutils*.bbclasses to
meta-python. However if meta-python is not in bblayers, the parsing
error would still occur.

[YOCTO #14610]

(From OE-Core rev: 619a3eb1266459daf16e10386113e9201fbf9cf5)

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-01-12 21:09:02 +00:00
Stefan Herbrechtsmeier
32f56de278 recipetool: Separate licenses with & operator
Separate licenses with & operator since it should be satisfied most use
cases and it is a reasonable assumption that all the licenses apply.
Furthermore flat, split and sort the licenses to minimize license string
changes.

Separate package licenses with & operator:
-LICENSE:${PN} = "MIT ISC"
+LICENSE:${PN} = "ISC & MIT"

Respect | and brackets in LICENSE:
-LICENSE = "BSD-3-Clause & (ISC & | & MIT)"
+LICENSE = "BSD-3-Clause & (ISC | MIT)"

Sort licenses:
-LICENSE = "MIT & BSD-3-Clause & ISC"
+LICENSE = "BSD-3-Clause & ISC & MIT"

Remove duplicates:
-LICENSE = "MIT & ISC & MIT"
+LICENSE = "ISC & MIT"

(From OE-Core rev: 60a84ecc53d20118c5e7f86dd3e3cafbfed1cf0a)

Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-20 15:29:01 +00:00
Stefan Herbrechtsmeier
2be34ea6ed recipetool: Sort output of guess_license function to be deterministic
Sort the output of guess_license function by license file to be
deterministic and to support self-testing.

(From OE-Core rev: ff2a7520813b3f775f96879d1905222e08b8c83f)

Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-20 15:29:01 +00:00
Stefan Herbrechtsmeier
392f2933b6 recipetool: Set master branch only as fallback
The commit 'meta/scripts: Manual git url branch additions (dc53fe75cc)'
sets the branch= parameter too early to master and thereby breaks the
-B/--srcbranch option.

ERROR: branch= parameter and -B/--srcbranch option cannot both be specified - use one or the other

(From OE-Core rev: 34ece8030e7a6a100b5e3e7b94e6c786c0e199a6)

Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-09 10:33:25 +00:00
Ross Burton
3cf97f84c6 recipetool: extend curl detection when creating recipes
If a configure.ac uses LIBCURL_CHECK_CONFIG it wants curl.

(From OE-Core rev: 16e830ca1352cee61e4ae4b98b1203f849bf71f3)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-03 23:37:16 +00:00
Ross Burton
ad10932343 recipetool: handle GitLab URLs like we do GitHub
GitHub URLs are automatically transformed to git: fetches, so handle
GitLab URLs too.

(From OE-Core rev: 651fb951819840fe122458ddbd852ee6c7ec0455)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-03 23:37:16 +00:00
Richard Purdie
dc53fe75cc meta/scripts: Manual git url branch additions
Following the scripted conversion adding branches to git://
SRC_URI entries, add the remaining references, mainly in the selftests
and recipetool.

(From OE-Core rev: 5340c0d688036c1be6c938f05d8a8c1e3b49ec38)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-30 18:56:47 +01:00
Stefan Herbrechtsmeier
7ab5afa819 recipetool: Simplify common source files skip in guess_license
(From OE-Core rev: 189c0297632968900715d7a4a5edcdf3a56e25f5)

Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-23 17:42:28 +01:00
Stefan Herbrechtsmeier
0e973d814b recipetool: Rework crunch_license to recognize more variants
(From OE-Core rev: 6c70f5bd2acd036c21be5b03df89d3a8f381a822)

Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14 11:48:45 +01:00
Stefan Herbrechtsmeier
36965547e3 recipetool: npm: Use README as license fallback
Use the README as license fallback if a license file is missing. Use the
linenumbers parameter of get_license_md5sums function to determine the
license text inside the README.

(From OE-Core rev: eff85c86f36673a1cb5a5dc8c66598e0dc457374)

Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14 11:48:45 +01:00
Stefan Herbrechtsmeier
4ca5b3fcb0 recipetool: npm: Do not add package.json files to LIC_FILES_CHKSUM
The package.json files doesn't contain any licenses. The name of the
license doesn't comply the license requirements of most liceneses.

(From OE-Core rev: 194df9c240378b6befeb10183889093ec7bb4d5f)

Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14 11:48:45 +01:00
Stefan Herbrechtsmeier
aee9854734 recipetool: Add support for linenumbers to licenses.csv
Add support for linenumbers (begin and end lines) to licenses.csv. Add
an optional linenumbers parameter to get_license_md5sums to support
different use cases.

(From OE-Core rev: d5c4979669f125e73c24dcc73fa3c4f3787bbb62)

Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14 11:48:45 +01:00