Commit Graph

3818 Commits

Author SHA1 Message Date
Ross Burton
4336f06475 scripts/checklayer: check for SECURITY.md
Add a check for a SECURITY.md file (or similar) to yocto-check-layer, as
knowing where to report security issues is important.

(From OE-Core rev: c7cb22ac4ceed60f88452e83c907a4c4f33660e4)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-11-22 16:53:37 +00:00
Nicolas Dechesne
79ef781499 scripts: patchreview: fix failure when running from a different folder
When running patchreview with --blame, the scripts runs a git log
command on the analyzed patch. When trying to analyse a layer which is
not in poky tree, we might be running the git log command from outside
the git workspace where the file is located, which results in such
failures:

Missing Signed-off-by tag ([truncated]/meta-qcom-hwe/recipes-devtools/partition-utils/qcom-ptool/0001-ptool.py-Generate-zero-files-in-output-folder-when-s.patch)
fatal: not a git repository (or any parent up to mount point /local/mnt)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

Fix this situation by setting the current work dir inside the git
workspace of the patch when running git log.

(From OE-Core rev: 8cc1c900b91d60e633f62bfe16a2ffc2d61c3f55)

Signed-off-by: Nicolas Dechesne <nicolas.dechesne@oss.qualcomm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-11-21 12:16:28 +00:00
Richard Purdie
f0d4814d4d resulttool: Allow store to filter to specific revisions
We have a challenge on the autobuilder where test results from both OE-Core
and poky are being mixed together during result storage which is confusing the
data. Add a way to filter to specific revisions as the least worst way to fix
the various issues this is causing.

(From OE-Core rev: 3f276a0dc65341668788853be2cf27ab6aa12b13)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-11-14 11:45:22 +00:00
Martin Jansa
6eb0a35580 patchreview: use check_upstream_status() from oe.qa
* the idea was to reuse the same function as I've noticed that the
  QA check which was added to insane.bbclass in:
  https://git.openembedded.org/openembedded-core/commit/?id=76a685bfcf927593eac67157762a53259089ea8a
  is in some cases more strcit than scripts/contrib/patchreview.py

  To be honest I wasn't aware of scripts/contrib/patchreview.py
  existence when I've asked about moving check_upstream_status()
  to oe.qa in order to write standalone script just like
  patchreview.py

* I've sent this long time ago:
  https://lists.openembedded.org/g/openembedded-core/message/177207
  but didn't like the sys.path.append to find oe.qa much or the
  duplicated path to .patch file in the output, then I've forgot about
  it until today in https://github.com/OE4T/meta-tegra/pull/1749
  where checklayer found one more issue, which I haven't noticed
  with patchreview.py before (because I've accidentally used a version
  without this change). It's not perfect, but at least it will be
  consistent with checklayer and patch-status QA check.

(From OE-Core rev: f291c08ea6a95638c3ad1f70434678bd5e374195)

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-11-12 11:19:44 +00:00
Vince Chang
6d90d0ba44 wic: add WIC_SECTOR_SIZE variable
Currently WIC is unable to generate images that requires a sector
size different of 512. Add WIC_SECTOR_SIZE variable to handle
the sector size of 4096 for UFS.

For "wic ls" command modify get_partitions() to support WIC_SECTOR_SIZE.

(From OE-Core rev: 2255f28b579bc5db4138bcacbb829661ae0ee721)

Signed-off-by: Vince Chang <vince_chang@aspeedtech.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-29 11:19:57 +00:00
Peter Marko
3b551fc466 cve-check: add support for cvss v4.0
https://nvd.nist.gov/general/news/cvss-v4-0-official-support

CVSS v4.0 was released in November 2023
NVD announced support for it in June 2024

Current stats are:
* cvss v4 provided, but also v3, so cve-check showed a value
sqlite> select count(*) from nvd where scorev4 != 0.0 and scorev3 != 0.0;
2069
* only cvss v4 provided, so cve-check did not show any
sqlite> select count(*) from nvd where scorev4 != 0.0 and scorev3 = 0.0;
260

(From OE-Core rev: 358dbfcd80ae1fa414d294c865dd293670c287f0)

Signed-off-by: Peter Marko <peter.marko@siemens.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-25 15:25:33 +01:00
Ross Burton
9dd8d5d6d2 buildstats-summary: look for buildstats if not specified
If the user hasn't specified a buildstats directory, use the latest
entry under $BUILDDIR.

(From OE-Core rev: aeb69fbe130dca37b39d4065ec983441e0052803)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-25 15:25:32 +01:00
Mikko Rapeli
5fe59b3144 oeqa selftest uki.py: add tests for uki.bbclass
Tests builds and boots qemu into uki binary with systemd and sysvinit.
Due to depedency to x86 specific ovmf UEFI firmware, tests
are specific to x86 curently. UEFI firmware for ARM can be generated
via qemuarm64-secureboot machine in meta-arm and similar tests
on qemu will pass.

(From OE-Core rev: 8a3cb17876dbcaf07696a4bcd454e2f9a444fb1b)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-25 15:12:22 +01:00
Mikko Rapeli
2f0ef8cd0e wic bootimg-efi.py: change UKI support from wic plugin to uki.bbclass
Remove custom wic plugin implementation and use systemd ukify reference
implementation when generating UKI images. Fail if users still have
create-unified-kernel-image in wic image config. uki.bbclass use is
detected from IMAGE_CLASSES variable ("inherit uki" in image
recipe) so export that to wic plugins.

If UKI is used, then only generate a minimal loader config for
systemd-boot which basically just sets a timeout. Also set 5 second
timeout by default instead of failing if wic bootloader config is
missing. Boot menu is generated at runtime based on UKI binaries
found from ESP partition.

(From OE-Core rev: 725fed6ea40c7443b5e0e69dc1dd9c38ac814c56)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-25 15:12:22 +01:00
Mikko Rapeli
c50552a2a5 wic bootimg-efi.py: keep timestamps and add debug prints
Keep timestamps etc to help build reproducibility.

Add prints to see what is being copied to ESP partition.

(From OE-Core rev: cedcd25c5e3cd002dd34651c182193731d7c964b)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-25 15:12:22 +01:00
Antonin Godard
bc67fcf621 devtool: ide-sdk: fix help typo
Change "paramter" -> "parameter".

(From OE-Core rev: e6e548170a5dee957b34d2a25161632fa37bb567)

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
Signed-off-by: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-22 12:28:18 +01:00
Changqing Li
a8f7f1b87e resulttool/regression.py: skip checking status for ptestresult.rawlogs/ptestresult.sections
ptestresult.rawlogs/ptestresult.sections don't have status is expected,
so skip them to avoid following error when running "resulttool regression
base target":

ERROR: Failed to retrieved base test case status: ptestresult.rawlogs
ERROR: Failed to retrieved base test case status: ptestresult.sections

(From OE-Core rev: c83a535d1b32f7fd292cd9caea1ec962bc3c735b)

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-11 15:57:45 +01:00
Changqing Li
3315ffac47 resulttool/regression.py: fix AttributeError
Fix following AttributeError when running "resulttool regression base target":
  File "/yocto/poky/scripts/lib/resulttool/regression.py", line 322, in regression_common
    res, resstr = compare_result(logger, c, b, base_results[a][c], target_results[a][b], args.limit)
AttributeError: 'Namespace' object has no attribute 'limit'

(From OE-Core rev: d773edde8db2019550916d2552171c45fe31ac2a)

Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-11 15:57:45 +01:00
Yi Zhao
8028c989cc wic/rootfs.py: allow --exclude-path option to exclude symlinks
Currently, if we specify a symbolic link in --exclude-path option,
we will get the following error in do_image_wic:

ERROR: --exclude-path: Must point inside the rootfs: usr/bin/hello.link

This is because it uses os.path.realpath to eliminate symbolic links. To
exclude symbolic links, use os.path.abspath instead of os.path.realpath.

(From OE-Core rev: 42e829ac1e9d74646b6dfb327b18b15f6b0df60b)

Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-11 15:57:45 +01:00
Richard Purdie
d074707199 efi-bootdisk.wks: Increase overhead-factor to avoid test failures
After commit d74bfb2d5c9e6247e4c0a3c2fdba0cc4a7585395:
"linux-yocto: Enable l2tp drivers when ptest featuee is on"
was merged, oe-selftest efibootpartition.GenericEFITest.test_boot_efi
breaks due to space issues.

Increase the disk space available to avoid this and allow functional
automated testing again.

(From OE-Core rev: eb76c15de881a56ead0a18f6428c5564291249c9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-10 11:58:02 +01:00
Richard Purdie
35c3b9132d migrate_localcount: Drop long obsolete code
If BB_URI_LOCALCOUNT isn't set, the code does nothing. That code was removed in 2012:

https://git.yoctoproject.org/poky/commit/?id=d0f35207f9e19b440393a79ebf621649c495738d

Therefore drop the rest of it!

(From OE-Core rev: fca25fc4d7721f85f64c942307ebe7ba9f2fad3e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-08 13:26:39 +01:00
Jon Mason
9377886a7d default-distrovars: Have KERNEL_CONSOLE reference SERIAL_CONSOLES
Currently, KERNEL_CONSOLE has a default value of "ttyS0".  However, Arm
machines and those using virtio serial prefer to use "ttyAMA0" or "hvc0"
(or something else).  These are usually defined by the machine config
file as SERIAL_CONSOLES, which has one or more entries.  Take the first
one of those instead of ttyS0, but default back to ttyS0 if nothing is
set.

Also, use this variable in the efi wic file instead of "ttyS0".
Of note, this changes the default speed of the default kernel console
from undefined (9600) to 115200.  This allows for users of the
mkefidisk.wks to work as before but any users of this variable could see
changed behavior and would now need to define this as:
KERNEL_CONSOLE ?= "ttyS0,9600"

This includes revisions suggested by Quentin Schulz and Ross Burton.

(From OE-Core rev: da42fc9ad55d1d60a04e38ff94c965f711f60cd6)

Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-10-08 13:26:39 +01:00
Joshua Watt
dfa892cfa6 Add script to make SPDX bindings
Adds a script to generate the SPDX code bindings

(From OE-Core rev: f0a5fdf54f975f9bc30758aec1f6f27e2d8149de)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-30 17:07:18 +01:00
Trevor Gamblin
1d813b0d53 scripts/patchtest: split out result messages
Add a minor refactor for the result output.

(From OE-Core rev: 5553edc0f8d58dc6e3a4f703af9bdb04fda1e12f)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-30 17:00:50 +01:00
Trevor Gamblin
4c378fc895 patchtest: simplify, rename modules
- simplify base.py, data.py
- move some leftover regex patterns to patterns.py
- remove pyparsing path logic, since this is no longer needed
- rename PatchTestInput class to PatchtestParser
- data.py: rename to patchtest_parser.py
- patterns.py: rename to patchtest_patterns.py
- move PatchTestDataStore to test_metadata.py since that's the only
  place it's used
- remove unused logger code

(From OE-Core rev: 1e971b05b036b0b1eb0bdbd9b26b54d06e74294c)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-30 17:00:50 +01:00
Trevor Gamblin
d6ede9c73b patchtest: mbox.py: new data implementation
Consolidate and improve some objects:

- absorb utils.py functionality
- repo.py: use mbox.py
- repo.py: remove some cruft
- utils.py: replace with logs.py
- utils.py: delete
- patch.py: delete
- scripts/patchtest: use logging directly
- general cleanup

(From OE-Core rev: d4fbdb1d15f281b236137d63710c73bca8911a36)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-30 17:00:50 +01:00
Adrian Freihofer
90827ca3cf devtool: drop useless variables
Drop some unused code.

The actual intention was to look for remnants of S = WORKDIR, which
required the extra complicated oe-local-files directory. The remaining
code dealing with oe-local-files still seems to be useful.

(From OE-Core rev: 02b52a2de3bf5766bc05531138a2e23acb00a276)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-30 17:00:50 +01:00
Adrian Freihofer
bac51c7151 devtool: menuconfig remove useless code
When I tried to understand whether oe-local-files is actually still
needed here, I found some useless lines that can be dropped.

(From OE-Core rev: 43f0c6beaa2a99301a565837944250bb2e56c98a)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-30 17:00:50 +01:00
Adrian Freihofer
21917a63ab devtool: drop S = WORKDIR workaround
Dropping support for S = WORKDIR allows to drop this ugly workaround.

With S = WORKDIR it was possible to refer to a file via oe-local-files
symlink or via direct file path. Ensuring the pseudo database is
consistent for both paths was extra complicated and required this bad
function. Really nice to drop it now!

(From OE-Core rev: 2b799fdf267f44c26797593984d9828c4fd0fd31)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-30 17:00:50 +01:00
Richard Purdie
7c5d68f27a scripts/install-buildtools: Update to 5.0.3
Update to the 5.0.3 release of the 5.0 series for buildtools.

(From OE-Core rev: c922ca720a0c3b7b4d3d3187539e7cf77d93d457)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-17 12:25:21 +01:00
Ryan Eatmon
40052a34c6 oe-setup-build: Change how we get the SHELL value
There are times that a user might not have SHELL set for some reason.
We should default back to a known shell in the event that SHELL is not
set.

(From OE-Core rev: 9f3099b7eddccf2b7328c0fdd6423269d17138ce)

Signed-off-by: Ryan Eatmon <reatmon@ti.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-12 16:15:09 +01:00
Konrad Weihmann
eb016e9758 runqemu: remove unused uid variable
(From OE-Core rev: 2521f67f19cb0eea0a2d6c5e8c39fa5edd60330a)

Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-10 13:05:00 +01:00
Konrad Weihmann
c9a9999400 runqemu: keep generating tap devices
in case there is no tap device the script tries to
generate a new one.
The new device is then unguarded for a moment, so
the newly generated device could be acquired
by a different instance or user, before it is locked to
the instance with acquire_taplock.
To fix that keep generating new tap devices in case
the lock can't be acquired up to 5 times.
If no tap device can be locked it fails in the existing
error handling

(From OE-Core rev: 23876576d054ebbab9b02c0012782aa56feda123)

Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-10 13:05:00 +01:00
Ryan Eatmon
f0e84fa257 oe-setup-build: Fix typo
A variable was mistyped in an error message resulting in this error:

NameError: name 'tempalte_name' is not defined. Did you mean: 'template_name'?

(From OE-Core rev: 275a3c015d37729c3b0c9cc4395d50ea2f210f02)

Signed-off-by: Ryan Eatmon <reatmon@ti.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-05 21:47:22 +01:00
Adrian Freihofer
6e0dc47ce4 devtool: remove obsolete SRCTREECOVEREDTASKS handling
devtool modify generates a bbappend for kernel recipes which contains:
  SRCTREECOVEREDTASKS="\
    do_fetch \
    do_kernel_checkout \
    do_kernel_configcheck \
    do_unpack \
    do_validate_branches \
  "
  do_patch[noexec] = "1"

If the linux-yocto kernel is used, this is redundant. The
linux-yocto.bbclass already does the same:

linux-yocto sets SRCTREECOVEREDTASKS to
  SRCTREECOVEREDTASKS="\
    do_fetch \
    do_kernel_checkout \
    do_kernel_configcheck \
    do_patch \
    do_unpack \
    do_validate_branches \
  "

Also the do_patch[noexec] is redundant because the purpose of
SRCTREECOVEREDTASKS is to delete these tasks if the externalsrc.bbclass
is used.

The default value of SRCTREECOVEREDTASKS initialized in
externalsrc.bbclass is:
  SRCTREECOVEREDTASKS ?= "do_patch do_unpack do_fetch"
This is fine for kernels which do not inherit the linux-yocto.bbclass.

The code in devtool modify is redundant and therefore removed.

(From OE-Core rev: 94ff1be36a1eeef7ddceb4fcf20425a03cd052de)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-04 12:38:44 +01:00
Adrian Freihofer
b161ca0008 devtool: modify kernel adds append twice
Drop the redundant generation of the do_configure:append section for the
kernel. The same append is generated twice:

if bb.data.inherits_class('kernel', rd):
    f.write('\ndo_configure:append() {\n'
    '    cp ${B}/.config ${S}/.config.baseline\n'
    '    ln -sfT ${B}/.config ${S}/.config.new\n'
    '}\n')

KCONFIG_CONFIG_ENABLE_MENUCONFIG ??= "true"
KCONFIG_CONFIG_ROOTDIR ??= "${B}"
if rd.getVarFlag('do_menuconfig', 'task'):
    f.write('\ndo_configure:append() {\n'
    '    if [ ${@oe.types.boolean(d.getVar("KCONFIG_CONFIG_ENABLE_MENUCONFIG"))} = True ]; then\n'
    '        cp ${KCONFIG_CONFIG_ROOTDIR}/.config ${S}/.config.baseline\n'
    '        ln -sfT ${KCONFIG_CONFIG_ROOTDIR}/.config ${S}/.config.new\n'
    '    fi\n'
    '}\n')

In contradiction to the first code block the second code block considers
the variables which is correct.

(From OE-Core rev: b0733c440e861ed7bf70efdd9b7a73afb4701218)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-09-04 12:38:44 +01:00
Alessandro Pecugi
35cb7bc646 runqemu: add sd card device
runqemu currently only supports scsi, ide and virtio drive types.
Implement QB_DRIVE_TYPE=/dev/mmcblk which adds an sdhci-pci device and
mounts the rootfs file as an sd card.

(From OE-Core rev: 0740896afbe13f37058e990a0a13b2b51bba70fd)

Signed-off-by: Alessandro Pecugi <alessandro.pecugi@seco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-30 21:59:38 +01:00
Jörg Sommer
4b60e8690c runqemu: Fix detection of -serial parameter
The pattern `-serial` matches also `-device usb-serial` and `virtio-serial`
which are not the desired parameter. This causes the serial console ttyS1 is
missing and Systemd's getty@ttyS1 fails constantly.

(From OE-Core rev: b6d035894120b45b42f146ab5b3110522c58d178)

Signed-off-by: Jörg Sommer <joerg.sommer@navimatix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-30 21:59:38 +01:00
Clara Kowalsky
5aeabd3217 resulttool: Add support to create test report in JUnit XML format
This adds the functionality to convert the results of the
testresults.json file to a unit test report in JUnit XML format. The
unit test report can be used in the CI/CD pipeline to display the test
results.

To use the resulttool scripts, first source oe environment, then run the
entry point script to look for help.
	$ resulttool

To generate the unit test report, execute the below
	$ resulttool junit <json_file>

By default the unit test report is stored as
<build_dir>/tmp/log/oeqa/junit.xml.

(From OE-Core rev: 3f9be03946243feaa09b908d7010899769091fe6)

Signed-off-by: Clara Kowalsky <clara.kowalsky@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-29 21:58:19 +01:00
Marta Rybczynska
38cfb6525e cve-json-to-text: add script
Add a script that converts the cve-check result from the JSON format
to the TEXT format.

(From OE-Core rev: da4e01dfa538578936e565413871a7496b8752df)

Signed-off-by: Marta Rybczynska <marta.rybczynska@ygreky.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-25 15:36:30 +01:00
Enguerrand de Ribaucourt
0a0caacfa5 recipetool: create_npm: reuse license utils
create_npm.py duplicated the logic for matching licenses from files and
also finding them. This patch refactors the code to reuse the license
utils. This will make the code more maintainable and also align both
behaviors. For instance, some licenses weren't matched properly because
the duplicate logic did not support the difference in format in the md5
tables for COMMON_LICENSE_DIR and licenses.csv.

This is also faster since the license files were being read twice.
The result is slightly more accurate since the utils have better
implementations, and I was able to reuse the logic for the root PN
package, as well as the base LICENSE variable.

I chose to extract generate_common_licenses_chksums into create.py
since it can be considered a general utility function to allow
other recipetool creators to refer to COMMON_LICENSE_DIR files.

I updated the wording in the code when appropriate.

v3:
 - added commit
 - this replaces the commit that added all the COMMON_LICENSE_DIR md5
   to licenses.csv

(From OE-Core rev: 7bc18bed63b94689890bcde63402d7cc1cedffa9)

Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-23 09:51:36 +01:00
Enguerrand de Ribaucourt
69bf37a3dd recipetool: create: split guess_license function
The npm recipetool handler redefines the license code the could be
unified. In order to do this refactoring, extract the bits we'll
need into separate functions.

guess_license() is renamed to find_licenses() and is split into
find_license_files() and match_licenses().

(From OE-Core rev: f1ec28feaea8ea6a2df894dd4ddba561c8a04ed2)

Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-23 09:51:36 +01:00
Enguerrand de Ribaucourt
01d17cd5d4 recipetool: create_npm: resolve licenses defined in package.json
Some npm packages do not copy the LICENSE or COPY file into their
git repository. They'll instead simply use SPDX identifiers in their
package.json. A fallback for those repositories attempted to match
the README file to a license file instead, which had a very low
probability of success.

This commit replaces this fallback with parsing the package.json and
looking for the license in COMMON_LICENSE_DIR. If the license is not
found, "Unknown" will still be produced.

This also generates "Unknown" for packages which had no README file,
which could silently not appear in the generated recipe. The user was
more likely to miss them.

Co-authored-by: Tanguy Raufflet <tanguy.raufflet@savoirfairelinux.com>
(From OE-Core rev: 445604cfc4a5813ea635f18053cd1f673bf0b830)

Signed-off-by: Tanguy Raufflet <tanguy.raufflet@savoirfairelinux.com>
Signed-off-by: Enguerrand de Ribaucourt <enguerrand.de-ribaucourt@savoirfairelinux.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-23 09:51:36 +01:00
Trevor Gamblin
6c1e2d7971 scripts/patchtest.README: cleanup, add selftest notes
Make some minor fixes to grammar and layout, and add a short new section
describing how to setup and use the patchtest selftests properly.

(From OE-Core rev: 264185d2cfa31b1cf2534a8de21e4b17bb5bbf79)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-05 14:31:57 +01:00
Alexander Kanavin
18e4194539 devtool/upgrade: rename RECIPE_UPDATE_EXTRA_TASKS -> RECIPE_UPGRADE_EXTRA_TASKS
'UPDATE' as a name is somewhat unfortunate as the variable is intended only for
the 'devtool upgrade' operation and devtool also has an 'update-recipe' operation.

(From OE-Core rev: 4467aa0661e233f44c4ce029428c67d88fccfc07)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-08-03 07:56:10 +01:00
Marcus Folkesson
d045fed31a bootimg-partition: break out code to a common library.
Break out the code that parse IMAGE_BOOT_FILES to a common library.

(From OE-Core rev: 1e07fe51bdb24070308c85e83df0b80ab9f83cea)

Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>
Reviewed-by: Quentin Schulz <quentin.schulz@cherry.de>
Reviewed-by: Konrad Weihmann <kweihmann@outlook.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-23 11:17:11 +01:00
Adrian Freihofer
a1b494e651 pybootchartgui.py: python 3.12+ regexes
$ ./scripts/pybootchartgui/pybootchartgui.py
./scripts/pybootchartgui/pybootchartgui/parsing.py:460: SyntaxWarning: invalid escape sequence '\d'
  disk_regex_re = re.compile ('^([hsv]d.|mtdblock\d|mmcblk\d|cciss/c\d+d\d+.*)$')
./scripts/pybootchartgui/pybootchartgui/parsing.py:597: SyntaxWarning: invalid escape sequence '\['
  timestamp_re = re.compile ("^\[\s*(\d+\.\d+)\s*]\s+(.*)$")
./scripts/pybootchartgui/pybootchartgui/parsing.py:598: SyntaxWarning: invalid escape sequence '\S'
  split_re = re.compile ("^(\S+)\s+([\S\+_-]+) (.*)$")
./scripts/pybootchartgui/pybootchartgui/parsing.py:643: SyntaxWarning: invalid escape sequence '\@'
  p = re.match ("\@ (\d+)", rest)
./scripts/pybootchartgui/pybootchartgui/draw.py:799: SyntaxWarning: invalid escape sequence '\s'
  ('system.cpu', 'CPU', lambda s: re.sub('model name\s*:\s*', '', s, 1)),

(From OE-Core rev: 0d94c22dd8d6c5655c2237ae740e8d9bb2adc751)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-22 16:53:06 +01:00
Alexander Kanavin
15e5193e9f devtool/upgrade: use PN instead of BPN for naming newly created upgraded recipes
BPN isn't correct, as it is set to 'cmake' when 'cmake-native' is being upgraded
(or libva for libva-initial etc.)

(From OE-Core rev: e634316547f86ce5662fa4899440a5af1047d494)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-22 16:53:06 +01:00
Alexander Kanavin
3d98aafc43 recipeutils/get_recipe_upgrade_status: group recipes when they need to be upgraded together
This will allow 'lockstep upgrades' of such recipes, improving success
rates in automated version updating process.

devtool check-upgrade-status now prints:

These recipes need to be upgraded together {
glib-2.0                  2.80.2          2.80.4          Anuj Mittal <anuj.mittal@intel.com>
glib-2.0-initial          2.80.2          2.80.4          Anuj Mittal <anuj.mittal@intel.com>
}
These recipes need to be upgraded together {
util-linux                2.39.3          2.40.2          Chen Qi <Qi.Chen@windriver.com>
util-linux-libuuid        2.39.3          2.40.2          Chen Qi <Qi.Chen@windriver.com>
}
These recipes need to be upgraded together {
cmake                     3.29.3          3.30.0          Unassigned <unassigned@yoctoproject.org>
cmake-native              3.29.3          3.30.0          Unassigned <unassigned@yoctoproject.org>
}

etc.

(From OE-Core rev: 7874aea5c62be3e8dbd19e04fce5389c5ed7aab6)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-22 16:53:06 +01:00
Alexander Kanavin
597b87a468 lib/oe/recipeutils: return a dict in get_recipe_upgrade_status() instead of a tuple
Putting various things in a tuple is an anti-pattern of sorts, as the consumers
have to unpack it into local variables for readability, or access items directly
with indexes, which makes code pretty much unreadable.

(From OE-Core rev: e86aa26d209eb9809198f6dd40cd058366318e3d)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-22 16:53:06 +01:00
Joshua Watt
cc191968a7 scripts/pull-spdx-licenses.py: Add script
Adds a script to pull the SPDX license data and update the license list
JSON data, as well as update the license directory.

(From OE-Core rev: cc2eb66fc0389664501baf57808e23f4cb4ee479)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-07-02 22:29:13 +01:00
Adithya Balakumar
c79c8ce71c wic/partition.py: Set hash_seed for empty ext partition
Although setting hash_seed is handled for the rootfs plugin case, but
this is missed when deploying an empty ext partition.

(From OE-Core rev: 0202fb594fb05098cb8d8b6088e63beb40b5906e)

Signed-off-by: Adithya Balakumar <adithya.balakumar@toshiba-tsip.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-18 08:44:41 +01:00
Ross Burton
4574919d30 scripts/makefile-getvar: add script to get values from Makefiles
There is often a need to extract a value from a Makefile, and standard
GNU Make doesn't provide a way to do this.  This script lets you access
values from Makefiles directly:

$ makefile-getvar curl/tests/server/Makefile noinst_PROGRAMS
getpart resolve rtspd sockfilt sws tftpd fake_ntlm socksd disabled mqttd

(From OE-Core rev: 881aa40d12d9dde73a932277093e5ceca8eb5c68)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-13 09:11:17 +01:00
Tim Orling
82d07928fe devtool upgrade: enable RECIPE_UPDATE_EXTRA_TASKS
For some recipes, such as those that inherit cargo-update-recipe-crates,
we need to run additional tasks once the new sources have been unpacked.

Introduce a new variable RECIPE_UPDATE_EXTRA_TASKS which is a space-
delimited list of tasks to run after the new sources have been
unpacked in scripts/lib/devtool/upgrade.py ugrade() method.

(From OE-Core rev: 59894f3b5b0bc257837d7ce4ea684f1d8c382cec)

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-13 09:11:17 +01:00
Aleksandar Nikolic
a43f15565b install-buildtools: update base-url, release and installer version
Update the following default values:
- DEFAULT_BASE_URL (https instead of http)
- DEFAULT_RELEASE (5.0.1)
- DEFAULT_INSTALLER_VERSION (5.0.1)

(From OE-Core rev: b2ebb965a5aa128b05ed7ddc9216171463777732)

Signed-off-by: Aleksandar Nikolic <aleksandar.nikolic@zeiss.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-06-12 16:08:32 +01:00