Commit Graph

3781 Commits

Author SHA1 Message Date
Enguerrand de Ribaucourt
01d17cd5d4 recipetool: create_npm: resolve licenses defined in package.json
Some npm packages do not copy the LICENSE or COPY file into their
git repository. They'll instead simply use SPDX identifiers in their
package.json. A fallback for those repositories attempted to match
the README file to a license file instead, which had a very low
probability of success.

This commit replaces this fallback with parsing the package.json and
looking for the license in COMMON_LICENSE_DIR. If the license is not
found, "Unknown" will still be produced.

This also generates "Unknown" for packages which had no README file,
which could silently not appear in the generated recipe. The user was
more likely to miss them.

Co-authored-by: Tanguy Raufflet <tanguy.raufflet@savoirfairelinux.com>
(From OE-Core rev: 445604cfc4a5813ea635f18053cd1f673bf0b830)

Signed-off-by: Tanguy Raufflet <tanguy.raufflet@savoirfairelinux.com>
Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-23 09:51:36 +01:00
Trevor Gamblin
6c1e2d7971 scripts/patchtest.README: cleanup, add selftest notes
Make some minor fixes to grammar and layout, and add a short new section
describing how to setup and use the patchtest selftests properly.

(From OE-Core rev: 264185d2cfa31b1cf2534a8de21e4b17bb5bbf79)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-05 14:31:57 +01:00
Alexander Kanavin
18e4194539 devtool/upgrade: rename RECIPE_UPDATE_EXTRA_TASKS -> RECIPE_UPGRADE_EXTRA_TASKS
'UPDATE' as a name is somewhat unfortunate as the variable is intended only for
the 'devtool upgrade' operation and devtool also has an 'update-recipe' operation.

(From OE-Core rev: 4467aa0661e233f44c4ce029428c67d88fccfc07)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-03 07:56:10 +01:00
Marcus Folkesson
d045fed31a bootimg-partition: break out code to a common library.
Break out the code that parse IMAGE_BOOT_FILES to a common library.

(From OE-Core rev: 1e07fe51bdb24070308c85e83df0b80ab9f83cea)

Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>
Reviewed-by: Quentin Schulz <quentin.schulz@cherry.de>
Reviewed-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-23 11:17:11 +01:00
Adrian Freihofer
a1b494e651 pybootchartgui.py: python 3.12+ regexes
$ ./scripts/pybootchartgui/pybootchartgui.py
./scripts/pybootchartgui/pybootchartgui/parsing.py:460: SyntaxWarning: invalid escape sequence '\d'
  disk_regex_re = re.compile ('^([hsv]d.|mtdblock\d|mmcblk\d|cciss/c\d+d\d+.*)$')
./scripts/pybootchartgui/pybootchartgui/parsing.py:597: SyntaxWarning: invalid escape sequence '\['
  timestamp_re = re.compile ("^\[\s*(\d+\.\d+)\s*]\s+(.*)$")
./scripts/pybootchartgui/pybootchartgui/parsing.py:598: SyntaxWarning: invalid escape sequence '\S'
  split_re = re.compile ("^(\S+)\s+([\S\+_-]+) (.*)$")
./scripts/pybootchartgui/pybootchartgui/parsing.py:643: SyntaxWarning: invalid escape sequence '\@'
  p = re.match ("\@ (\d+)", rest)
./scripts/pybootchartgui/pybootchartgui/draw.py:799: SyntaxWarning: invalid escape sequence '\s'
  ('system.cpu', 'CPU', lambda s: re.sub('model name\s*:\s*', '', s, 1)),

(From OE-Core rev: 0d94c22dd8d6c5655c2237ae740e8d9bb2adc751)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-22 16:53:06 +01:00
Alexander Kanavin
15e5193e9f devtool/upgrade: use PN instead of BPN for naming newly created upgraded recipes
BPN isn't correct, as it is set to 'cmake' when 'cmake-native' is being upgraded
(or libva for libva-initial etc.)

(From OE-Core rev: e634316547f86ce5662fa4899440a5af1047d494)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-22 16:53:06 +01:00
Alexander Kanavin
3d98aafc43 recipeutils/get_recipe_upgrade_status: group recipes when they need to be upgraded together
This will allow 'lockstep upgrades' of such recipes, improving success
rates in automated version updating process.

devtool check-upgrade-status now prints:

These recipes need to be upgraded together {
glib-2.0                  2.80.2          2.80.4          Anuj Mittal <anuj.mittal@intel.com>
glib-2.0-initial          2.80.2          2.80.4          Anuj Mittal <anuj.mittal@intel.com>
}
These recipes need to be upgraded together {
util-linux                2.39.3          2.40.2          Chen Qi <Qi.Chen@windriver.com>
util-linux-libuuid        2.39.3          2.40.2          Chen Qi <Qi.Chen@windriver.com>
}
These recipes need to be upgraded together {
cmake                     3.29.3          3.30.0          Unassigned <unassigned@yoctoproject.org>
cmake-native              3.29.3          3.30.0          Unassigned <unassigned@yoctoproject.org>
}

etc.

(From OE-Core rev: 7874aea5c62be3e8dbd19e04fce5389c5ed7aab6)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-22 16:53:06 +01:00
Alexander Kanavin
597b87a468 lib/oe/recipeutils: return a dict in get_recipe_upgrade_status() instead of a tuple
Putting various things in a tuple is an anti-pattern of sorts, as the consumers
have to unpack it into local variables for readability, or access items directly
with indexes, which makes code pretty much unreadable.

(From OE-Core rev: e86aa26d209eb9809198f6dd40cd058366318e3d)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-22 16:53:06 +01:00
Joshua Watt
cc191968a7 scripts/pull-spdx-licenses.py: Add script
Adds a script to pull the SPDX license data and update the license list
JSON data, as well as update the license directory.

(From OE-Core rev: cc2eb66fc0389664501baf57808e23f4cb4ee479)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-02 22:29:13 +01:00
Adithya Balakumar
c79c8ce71c wic/partition.py: Set hash_seed for empty ext partition
Although setting hash_seed is handled for the rootfs plugin case, but
this is missed when deploying an empty ext partition.

(From OE-Core rev: 0202fb594fb05098cb8d8b6088e63beb40b5906e)

Signed-off-by: Adithya Balakumar <adithya.balakumar@toshiba-tsip.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-18 08:44:41 +01:00
Ross Burton
4574919d30 scripts/makefile-getvar: add script to get values from Makefiles
There is often a need to extract a value from a Makefile, and standard
GNU Make doesn't provide a way to do this.  This script lets you access
values from Makefiles directly:

$ makefile-getvar curl/tests/server/Makefile noinst_PROGRAMS
getpart resolve rtspd sockfilt sws tftpd fake_ntlm socksd disabled mqttd

(From OE-Core rev: 881aa40d12d9dde73a932277093e5ceca8eb5c68)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-13 09:11:17 +01:00
Tim Orling
82d07928fe devtool upgrade: enable RECIPE_UPDATE_EXTRA_TASKS
For some recipes, such as those that inherit cargo-update-recipe-crates,
we need to run additional tasks once the new sources have been unpacked.

Introduce a new variable RECIPE_UPDATE_EXTRA_TASKS which is a space-
delimited list of tasks to run after the new sources have been
unpacked in scripts/lib/devtool/upgrade.py ugrade() method.

(From OE-Core rev: 59894f3b5b0bc257837d7ce4ea684f1d8c382cec)

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-13 09:11:17 +01:00
Aleksandar Nikolic
a43f15565b install-buildtools: update base-url, release and installer version
Update the following default values:
- DEFAULT_BASE_URL (https instead of http)
- DEFAULT_RELEASE (5.0.1)
- DEFAULT_INSTALLER_VERSION (5.0.1)

(From OE-Core rev: b2ebb965a5aa128b05ed7ddc9216171463777732)

Signed-off-by: Aleksandar Nikolic <aleksandar.nikolic@zeiss.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-12 16:08:32 +01:00
Aleksandar Nikolic
062c289efa install-buildtools: fix "test installation" step
The "Test installation" step fails with some harmless error messages
(see [1]). This can however make a user think that the buildtools
have not been installed correctly.

Two reasons for the error messages:
- some envvars in the environment-setup-<arch>-pokysdk-linux file
  start and end with double quotes (e.g., PATH) and are as such
  written into python os.environ. This leads that their usage is
  not valid later when testing the installation. This patch removes
  the double quotes before writing, if they are present.
- if installation directory (install_dir), given through the option
  --directory, is given as a relative path, checking if the path to
  a tool (e.g., gcc) in buildtools starts it will always fail. This
  patch converts the install_dir variable to an absolute path.

[1]
ERROR: Something went wrong: tar not found in ./build-tools
ERROR: Something went wrong: installation failed

(From OE-Core rev: e4eb0b14ecf9bd2fba13260441c9d86eb348f41e)

Signed-off-by: Aleksandar Nikolic <aleksandar.nikolic@zeiss.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-12 16:08:32 +01:00
Aleksandar Nikolic
cedb7197cc install-buildtools: remove md5 checksum validation
No need to validate with the md5 checksum, as the file is not even
uploaded to the Yocto release webpage (the download never failed due
to a wrong indentation of an else statement). For validation purposes,
use the sha256 checksum only.

(From OE-Core rev: b740d2f9d40aef1e18c022d1e82b4fb2c5c1fc22)

Signed-off-by: Aleksandar Nikolic <aleksandar.nikolic@zeiss.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-12 16:08:32 +01:00
Enrico Jörns
1d5b4d19c7 wic: bootimg-efi: fix error handling
If we check for empty variables, it does not make sense to print their
content in case they are empty.

Additionally, the error message in the 'kernel' check attempted to print
the wrong variable ('target') which was not even defined, yet.

Also, raising WicError doesn't require an extra newline.

(From OE-Core rev: f31cf475d0235f42d73aeec07694f79b9937fd76)

Signed-off-by: Enrico Jörns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-06 11:56:04 +01:00
Enrico Jörns
4f600110ee wic: engine.py: use raw string for escape sequence
Fixes:

| poky/scripts/lib/wic/engine.py:362: SyntaxWarning: invalid escape sequence '\/'

(From OE-Core rev: e33d7241f6c2897e930aff41e18b154891197ab9)

Signed-off-by: Enrico Jörns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-06 11:56:04 +01:00
Richard Purdie
6b9eb4e9f0 siteconfig: Drop siteconfig class/code/support
The siteconfig code was only used for 5 cache values. The complexity added to sstate
to support this code was considerable and the runtime much more significant than
any benefit the cache files would have added. Drop the support for this which
was only used minimally for ncurses and zlib.

(From OE-Core rev: f3766dc038f7ba9780ddaf5eb8d27385ea31d7d0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-31 16:58:36 +01:00
Ninette Adhikari
f66f8fa9f6 oe-build-perf-report: Add commit hash link to chart tooltip
Update tooltip to include the commit hash link to the poky repository

(From OE-Core rev: 1187a460c2608369df8de08fe52a5542e4ab6ecd)

Signed-off-by: Ninette Adhikari <ninette@thehoodiefirm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-24 16:59:31 +01:00
Richard Purdie
569f8e09f0 recipetool/devtool: Update to work correctly with UNPACKDIR
Tweak recipetool and devtool to correctly use UNPACKDIR. This allows some
simplification of the code. This patch makes things basically work but there
are likely deeper improvements that can be made now that WORKDIR != UNPACKDIR.

(From OE-Core rev: d2eeaa88b27a2875c419591d1d91bcc85d7b129c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-22 22:26:30 +01:00
Richard Purdie
6d77296d83 devtool: Drop oe-local-files and simplify
The only real reason for oe-local-files was to support S = WORKDIR. With changes to
drop support for that, it makes sense to simplify devtool and to try and make both
the code and the processes/workflows simpler.

This patch drops support for S = WORKDIR, removes oe-local-files and then updates
the test cases to match this new situation.

At the code level, we assume we can always now track code changes using git and
that things committed into git are handled as patches (as before) but delta against
HEAD is saved as specific file level changes to the recipe.

One test is disabled as it is no longer approproate. It is being keped until we can
make WORKDIR != UNPACKDIR at which point it should be revisited.

(From OE-Core rev: ce8190c519052fed10b5233697b69a75868db45a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-21 12:08:04 +01:00
Anton Almqvist
9a894dac19 devtool: modify: Catch git submodule error for go code
One of the git submodule commands failed for source extracted for
recipes using go.bbclass.  The root cause is probably the path set up
by go_do_unpack which makes S and gitroot not match.

This patch does not fix the root problem, but at least it is no worse
than before the git submodule support.

The extracted source will still have two .git folders, one in S
created by devtool and one in the go path which will contain the tru
git history.

[ YOCTO #15483 ]

(From OE-Core rev: fe242408af40dd1f6e47d9b2b232bdc76756c80a)

Signed-off-by: Anton Almqvist <antonal@axis.com>
Signed-off-by: Ola x Nilsson <olani@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-15 16:39:19 +01:00
Ninette Adhikari
cf0c866b51 oe-build-perf-report: Add dark mode
Update css to add dark mode when window prefers-color-scheme is dark.

(From OE-Core rev: ed02a235d42202279ad5e4e3153247f9e5e2bba8)

Signed-off-by: Ninette Adhikari <ninette@thehoodiefirm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-15 16:39:19 +01:00
Ninette Adhikari
20fafa0854 oe-build-perf-report: Update chart tooltip and chart type
- Update chart tooltip format to show value as size in MB for 'rootfs size'
and timestamp for 'tmpdir size'
- Add commit number to tooltip
- Update chart type to 'step chart' instead of 'line chart'

(From OE-Core rev: fb41cbbe6008e442b6eac77308eadeae327eed7d)

Signed-off-by: Ninette Adhikari <ninette@thehoodiefirm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-15 16:39:19 +01:00
Ninette Adhikari
314a220280 oe-build-perf-report: Improve report styling and add descriptions
Styling updates are added including page margin, labels for x and y axis, tooltip, and section descriptions.

(From OE-Core rev: 2a2568fcf1a3f8e467bd814f4fb13dffae8ec61d)

Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-15 16:39:19 +01:00
Ninette Adhikari
99861c17e7 oe-build-perf-report: Display more than 300 commits and date instead of commit number
- This commit updates measurement statistics data to include start_time so that time can be displayed instead of commit numbers on the chart.
- It also updates default commit history length to 300.

(From OE-Core rev: 64275a41c37130aaaacc5e592f94a1afe057119b)

Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-15 16:39:19 +01:00
Ninette Adhikari
836b4b57d6 oe-build-perf-report: Add apache echarts to make report interactive
- Add Apache echarts (https://echarts.apache.org/en/index.html) library to create build performance charts.
- Restructure data to time and value array format so that it can be used by echarts.
- This commit also converts test duration to minutes to map against the values axis.
- Zoom is added to the line charts.

(From OE-Core rev: 63c9321832aae79d20a4ddd199a4a1385f81de53)

Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-15 16:39:19 +01:00
Antonin Godard
12c447b38a devtool: ide-sdk: correct help typo
(From OE-Core rev: 50e043387a2f0f9a5c2f7a5f914c465c830d329b)

Signed-off-by: Antonin Godard <antoningodard@pm.me>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-09 09:48:58 +01:00
Richard Purdie
808f700efc classes/lib/scripts: Initial WORKDIR -> UNPACKDIR updates
Work through the initial issues I found where we need to change WORKDIR
to UNPACKDIR.

(From OE-Core rev: 86fec41b1e809d1a2fa2feadc26d29020df53d39)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-02 15:15:51 +01:00
Alexander Kanavin
ad6470a4c2 scripts/oe-setup-build: write a build environment initialization one-liner into the build directory
With this, users no longer have to know where oe-init-build-env is relative to the
build directory; that information is contained in the one liner and then
it's possible to simply use that:

. /path/to/build/init-build-env

This will particularly help with initializing builds in unpacked
build bundles, as users won't have to know where oe-init-build-env
is in the bundle directory tree - similar to esdk initialization.

(From OE-Core rev: 1cabdf287c2739accdab3a766df060f1bc802b63)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-05-02 11:47:51 +01:00
Joe Slater
a46e374d43 oe-debuginfod: add option for data storage
Storing the data files under $HOME can be unreliable if debuginfod
is used for several projects, especially if $HOME is shared
between machines.  We provide an option to save files under the
project directory.  The default behavior is unchanged.

(From OE-Core rev: e1e0cf82f559077e2a51447baf137086202c0c4a)

Signed-off-by: Joe Slater <joe.slater@windriver.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-04-30 22:20:16 +01:00
John Ripple
5756ac9936 rootfs.py: Fix logger error message format
--_000_PH7PR17MB61307CB6690EC00DEB4ED9B39C0F2PH7PR17MB6130namp_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

This patch adds a missing "%s" format in a logger.error call. Without
this addition the logger itself would error out and not print a useful
message.

(From OE-Core rev: 2c892e5dd9ba72a51c0a8fb851599cc2dc3a8b5c)

Signed-off-by: John Ripple <john.ripple@keysight.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-04-25 10:19:28 +01:00
Julien Stephan
28516320d8 devtool: standard: update-recipe/finish: fix update localfile in another layer
When trying to use devtool update-recipe/finish on another layer, with modified
local file we have the following error:

  Traceback (most recent call last):
    File "<..>/poky/scripts/devtool", line 350, in <module>
      ret = main()
            ^^^^^^
    File "<..>/poky/scripts/devtool", line 337, in main
      ret = args.func(args, config, basepath, workspace)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "<..>/poky/scripts/lib/devtool/standard.py", line 1968, in update_recipe
      updated, _, _ = _update_recipe(args.recipename, workspace, rd, args.mode, args.append, args.wildcard_version, args.no_remove, args.initial_rev, dry_run_outdir=dry_run_outdir, no_overrides=args.no_overrides, force_patch_refresh=args.force_patch_refresh)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "<..>/poky/scripts/lib/devtool/standard.py", line 1930, in _update_recipe
      updated, appendf, removed = _update_recipe_patch(recipename, workspace, srctree, crd, appendlayerdir, wildcard_version, no_remove, no_report_remove, initial_rev, dry_run_outdir, force_patch_refresh)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "<..>/poky/scripts/lib/devtool/standard.py", line 1747, in _update_recipe_patch
      patchdir = param.get('patchdir', ".")
                 ^^^^^^^^^
  AttributeError: 'str' object has no attribute 'get'

This was introduced when adding support for git submodules.
No selftest case exists to catch this, so a selftest will be
added in another commit.

(From OE-Core rev: de7ca9f800e15e10271502da7e51e3ae08e0c85b)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-04-25 10:10:14 +01:00
Vincent Kriek
629063944c devtool: sync: Fix Execution error
When executing devtool sync on a recipe that was extract with devtool
extract earlier the following error occured:

    Traceback (most recent call last):
    [...]
    bb.process.ExecutionError: Execution of 'git fetch file:///home/vin/projects/poky/build/tmp/work/all-poky-linux/netbase/6.4/devtooltmp-figt1jmr/workdir/netbase devtool:devtool' failed with exit code 128:
    fatal: refusing to fetch into branch 'refs/heads/devtool' checked out at '/home/vin/projects/poky/build/netbase-src'

Fix this by adding --update-head-ok and --force to git fetch so it will
override the current head even if it is checked out and has changes.
Possible existing changes in the devtool branch can be retrieved by
checking out the devtool.bak branch

(From OE-Core rev: 4c89b061003a0f4143c6d3838e39fc2643edc8d6)

Signed-off-by: Vincent Kriek <vincent@coelebs.dev>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-04-25 10:10:14 +01:00
Sven Schwermer
8219eefdd8 recipetool: Handle several go-import tags in go resolver
When dynamically resolving go modules, the HTML page may contain several
go-import meta tags. We must handle all and pick the correct one based
on the module name. An example for such a behaviour is
gonum.org/v1/gonum:

<meta name="go-import" content="gonum.org/v1/exp git https://github.com/gonum/exp">
<meta name="go-import" content="gonum.org/v1/gonum git https://github.com/gonum/gonum">
<meta name="go-import" content="gonum.org/v1/hdf5 git https://github.com/gonum/hdf5">
<meta name="go-import" content="gonum.org/v1/netlib git https://github.com/gonum/netlib">
<meta name="go-import" content="gonum.org/v1/plot git https://github.com/gonum/plot">
<meta name="go-import" content="gonum.org/v1/tools git https://github.com/gonum/tools">

(From OE-Core rev: 9c36a61e29359067165bddc7f2accdf2c4c8a761)

Signed-off-by: Sven Schwermer <sven.schwermer@disruptive-technologies.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-04-23 13:40:24 +01:00
Sven Schwermer
e4c3483ecf recipetool: Handle unclean response in go resolver
It appears that some go modules repond with a 404 error when trying to
resolve them dynamically. The response body may still contain the
go-import meta tag. An example for such behaviour is gonum.org/v1/gonum.

(From OE-Core rev: 8f2e14ab6562a9a68819a960c66a258ea9dbe246)

Signed-off-by: Sven Schwermer <sven.schwermer@disruptive-technologies.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-04-23 13:40:24 +01:00
Julien Stephan
834e91ce53 devtool: standard: throws appropriate error if source is in detached HEAD
If source is in detached HEAD, we get the following error when using
detvool finish command:

    [...]
    File "<...>/poky/scripts/lib/devtool/standard.py", line 1938, in _update_recipe
      bb.process.run('git checkout %s' % startbranch, cwd=srctree)
    File "<...>/poky/bitbake/lib/bb/process.py", line 189, in run
      raise ExecutionError(cmd, pipe.returncode, stdout, stderr)
  bb.process.ExecutionError: Execution of 'git checkout (HEAD detached at 9bbf87e)' failed with exit code 2:
  /bin/sh: -c: line 1: syntax error near unexpected token `('
  /bin/sh: -c: line 1: `git checkout (HEAD detached at 9bbf87e)'

Check this and throws an appropriate error in this case

(From OE-Core rev: d9c686b5ff9f591ec6b928ed539084c02df4c8a5)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-04-12 17:27:53 +01:00
Anibal Limon
7c3f956a0a wic: bootimg-partition allow to set var to get boot files
This is a generic functionality that takes a set of files and create a
boot parition based on IMAGE_BOOT_FILES var, allow to sepcify the
variable name so a BSP plugin can extend this and re-use the
functionality.

(From OE-Core rev: 3a2339455be29a6ff65c8c5f6865a4cd35dd7f63)

Signed-off-by: Anibal Limon <anibal@limonsoftware.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-26 17:10:51 +00:00
Trevor Woerner
f15676592b bmaptool: now part of Yocto Project
The bmaptool (previously: bmap-tools, bmap-tool, bmaptool) has been moved
to be under the Yocto Project umbrella and is now hosted at:

	github.com/yoctoproject/bmaptool

[RP: Added a couple of missing renames]
(From OE-Core rev: 7a036b1a1ec7dcd27dbe18d4c2e703bd2a8af182)

Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-03-05 12:24:50 +00:00
Adrian Freihofer
c82c57621b devtool: ide-sdk prefer sources from workspace
Improve the previous commit:
- log an error if some assumptions are not true
- Use TARGET_DBGSRC_DIR variable
- Do the same for ide none

Why the additional source mapping is required:

For example the cmake-example recipe refers to sources like this:
./recipe-sysroot-native/usr/bin/x86_64-poky-linux/x86_64-poky-linux-readelf \
  -wi image/usr/bin/cmake-example | grep -B1 DW_AT_comp_dir
    ...
    <560>   DW_AT_name        : (indirect line string, offset: 0x1da):
    /usr/src/debug/cmake-example/1.0/oe-local-files/cpp-example.cpp
    ...

Another example is powertop:
./recipe-sysroot-native/usr/bin/x86_64-poky-linux/x86_64-poky-linux-readelf \
  -wi image/usr/sbin/powertop | grep -B1 DW_AT_comp_dir
    ...
    <561>   DW_AT_name        : (indirect line string, offset: 0x1da):
    /usr/src/debug/powertop/2.15/src/devlist.cpp
    ...

For recipes with local files this works. The oe-local-files folder is
not available in the rootfs-dbg and therefore the sources are first
found in the workspace folder. GDB searches for source files in various
places:
https://sourceware.org/gdb/current/onlinedocs/gdb.html/Source-Path.html
However, for the powertop example the sources opened in the editor are
from the rootfs-dbg instead of from the workspace.

Bitbake calls the compiler with
  -fmacro-prefix-map=${S}=${TARGET_DBGSRC_DIR}
where TARGET_DBGSRC_DIR defaults to "/usr/src/debug/${PN}/${PV}".

A source map which maps the recipe specific path from TARGET_DBGSRC_DIR
to the workspace fixes this.
The already existing source map for /usr/src/debug applies for all other
recipes. It finds the sources (read only) in the rootfs-dbg folder.

(From OE-Core rev: 06601632c1879cb80276f9b36de91fb7808311a5)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-27 11:38:46 +00:00
Enguerrand de Ribaucourt
a5b75e29ff devtool: ide-sdk source mapping for vscode
When launching the debug configuration, the source files from the debug
rootfs were openened in the editor instead of the local workspace files.
We add an exception to properly map them to the file being developed and
compiled by the IDE integration. This also more closely matches what the
user would expect compared to native development.

This is also true for the devtool fallback mode.

(From OE-Core rev: 24db2b8d0d7104960c1cdb2c7ee5216c830a6754)

Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-27 11:38:46 +00:00
Adrian Freihofer
4c6f3bbc77 devtool: ide-sdk launch.json per recipe only
If multiple recipes are processed at once, the launch.json and the
tasks.json of the second recipe contains also the configurations for the
binaries of the first recipe.

Example:
  devtool ide-sdk powertop cmake-example oe-selftest-image
generated a launch and a tasks configuration for the cmake-example
recipe which also offers debugging the powertop binary.

(From OE-Core rev: 63986b2c40d90fe96cdc6a46aa649efcf17f6ac2)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-27 11:38:46 +00:00
Enrico Jörns
54c27d84b6 wic: 'empty' plugin: fix typo in comment
(From OE-Core rev: 1a9013c3767d174a970d31e80748bc8d73af0dba)

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-27 11:35:43 +00:00
Simone Weiß
8a3f025b3a patchtest: provide further guidance for failed testcases
Cross-reference the wiki page on patchtest now that it is updated and contains
more information how to address failed testcases. Adding it in patchtest only
is enough as patchtest-send-result already points to the wikipage for failures.

(From OE-Core rev: 51267f3c5d647fc6483ce6b597ed9e25c14bd425)

Signed-off-by: Simone Weiß <simone.p.weiss@posteo.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-24 16:10:23 +00:00
Adrian Freihofer
4e3b6c24d5 devtool: ide-sdk python 3.12 escaping
scripts/lib/devtool/ide_sdk.py:709: SyntaxWarning: invalid escape sequence '\.'
  re_so = re.compile('.*\.so[.0-9]*$')

scripts/lib/devtool/ide_plugins/__init__.py:87: SyntaxWarning: invalid escape sequence '\$'
  gdbserver_cmd_start += "test -f \$TEMP_DIR/pid && exit 0; "

scripts/lib/devtool/ide_plugins/__init__.py:88: SyntaxWarning: invalid escape sequence '\$'
  gdbserver_cmd_start += "mkdir -p \$TEMP_DIR; "

scripts/lib/devtool/ide_plugins/__init__.py:89: SyntaxWarning: invalid escape sequence '\$'
  gdbserver_cmd_start += "%s --multi :%s > \$TEMP_DIR/log 2>&1 & " % (

scripts/lib/devtool/ide_plugins/__init__.py:91: SyntaxWarning: invalid escape sequence '\$'
  gdbserver_cmd_start += "echo \$! > \$TEMP_DIR/pid;"

scripts/lib/devtool/ide_plugins/__init__.py:94: SyntaxWarning: invalid escape sequence '\$'
  gdbserver_cmd_stop += "test -f \$TEMP_DIR/pid && kill \$(cat \$TEMP_DIR/pid); "

scripts/lib/devtool/ide_plugins/__init__.py:95: SyntaxWarning: invalid escape sequence '\$'
  gdbserver_cmd_stop += "rm -rf \$TEMP_DIR; "

(From OE-Core rev: e8c64921de7206bf617fc42433286867ae3c931d)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-24 16:10:23 +00:00
Enguerrand de Ribaucourt
e2f8ed72dd devtool: ide: define compilerPath for meson projects
The compile_commands.json file output by meson uses the compiler as if
present in the $PATH. However, when using an IDE, the $PATH used by
bitbake is not there.

The vscode-cpptools now allows to define the compilerPath in addition
to replace the one from compile_commands.json.

(From OE-Core rev: d9f5c27c8beee07c7cbbed11f5d45058e7315846)

Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-23 14:29:03 +00:00
Enguerrand de Ribaucourt
c0d340c52e devtool: ide: vscode: Configure read-only files
When debugging or browsing files, the user may fall into external
sources from other packages in the sysroot or dbg-rootfs. Modifying them
will only lead to confusion since they will be overwritten by Yocto. The
user should open them in a separate devtool modify session if they want
to make changes. Meanwhile, we should prevent write access to them.

(From OE-Core rev: 8a95ad9aecb81732c865e00a987bb8bd3d6cb91d)

Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-20 12:58:41 +00:00
Enguerrand de Ribaucourt
8e84fbba72 devtool: ide_sdk: Use bitbake's python3 for generated scripts
The generated scripts use the sys.path configuration found inside
bitbake. It can be a different python version than the one used on the
host through the IDE.

For instance, when running the generated script
deploy_target_cmake-example-core2-64 from an eSDK generated on another
machine, I got the following exception:
    AssertionError: SRE module mismatch

We need to match the sys.executable to the sys.path.

(From OE-Core rev: 45704319661570b45ef69fddd6b4b4fa22ca80d1)

Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-20 12:58:41 +00:00
Tim Orling
9f7475b4e5 recipetool; add support for python_mesonpy class
* Add support to detect the "mesonpy" build-backend for recipetool create.
* Add oe-selftest case for creating a recipe for "siphash24" from pypi.
  https://pypi.org/project/siphash24/

This is by far the simplest recipe using the mesonpy build backend.

Upstream does not provide LICENSE file(s) and we do not detect the
LICENSE so don't check for that result in the test. Likewise, upstream
does not define HOMEPAGE, so skip that result.

(From OE-Core rev: 256749322671d2f4ea994db671d73c4de10e1723)

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-20 11:39:45 +00:00
Peter Kjellerstedt
4cfd0f7e4e lib/oe/patch: Use git notes to store the filenames for the patches
The old way of keeping track of the filenames for the patches that
correspond to the commits was to add a special comment line to the end
of the commit message, e.g., "%% original patch: <filename>", using a
temporary git hook. This method had some drawbacks, e.g.:

* It caused problems if one wanted to push the commits upstream as the
  comment line had to be manually removed.
* The comment line would end up in patches if someone used git
  format-path rather than devtool finish to generate the patches.
* The comment line could interfere with global Git hooks used to
  validate the format of the Git commit message.
* When regenerating patches with `devtool finish --force-patch-refresh`,
  the process typically resulted in adding empty lines to the end of the
  commit messages in the updated patches.

A better way of keeping track of the patch filenames is to use Git
notes. This way the commit messages remain unaffected, but the
information is still shown when, e.g., doing `git log`. A special Git
notes space, refs/notes/devtool, is used to not intefere with the
default Git notes. It is configured to be shown in, e.g., `git log` and
to survive rewrites (i.e., `git commit --amend` and `git rebase`).

Since there is no longer any need for a temporary Git hook, the code
that manipulated the .git/hooks directory has also been removed. To
avoid potential problems due to global Git hooks, --no-verify was added
to the `git commit` command.

To not cause troubles for those who have done `devtool modify` for a
recipe with the old solution and then do `devtool finish` with the new
solution, the code will fall back to look for the old strings in the
commit message if no Git note can be found.

While not technically motivated like above, the way to keep track of
ignored commits is also changed to use Git notes to avoid having
different methods to store similar information.

(From OE-Core rev: f5e6183b9557477bef74024a587de0bfcc2b7c0d)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-19 16:03:22 +00:00