Commit Graph

1634 Commits

Author SHA1 Message Date
BELOUARGA Mohamed
2dacac93bc recipetool: create: npm: Add support to handle peer dependencies
NPM changed its manner to handle peer dependencies over its versions.
Before NPM 3: NPM installs automatically peer dependencies
between NPM 3 and 7: NPM shows a warning about peer dependencies
After NPM 3: NPM reworked its manner how to handle peer dependencies

The shrinkwrap doesn't have the parameters of the peer dependencies, so we cannot
fetch them. in the same time peer dependencies are not direct dependencies, they should
be installed as run time dependencies.

(From OE-Core rev: f36021a749974ef3d4a6abe4d5429544a815071a)

Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-29 11:10:39 +01:00
BELOUARGA Mohamed
dbc1da2fb8 recipetool: create: npm: Add support for the new format of the shrinkwrap file
The shrinkwrap file changed its format, but npm does not version this file. So we can use it properly.
The actual changes make the script check if the npm package has dependencies in the actual shrinkwrap format.

(From OE-Core rev: 488d17c2af0c927ec66f0eee124bf6fc5b7f7c95)

Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-29 11:10:39 +01:00
BELOUARGA Mohamed
2a3888069f recipetool: create: npm: Remove duplicate function to not have future conflicts
Npm packages do not have yocto friendly names. fore instance we can have names like
"@example/npmPackage"

npm fetcher has a function that convert these names to yocto friendly names.
But in recipe tool we have an other function (duplicate).

(From OE-Core rev: 18e5438de5389b58c8b6a548d4474128d510a28d)

Signed-off-by: BELOUARGA Mohamed <m.belouarga@technologyandstrategy.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-29 11:10:39 +01:00
Yoann Congal
356d2369c7 recipetool: Fix inherit in created -native* recipes
native and nativesdk classes are special and must be inherited last :
put them at the end of the gathered classes to inherit.

(From OE-Core rev: a6614fd800cbe791264aeb102d379ba79bd145c2)

Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-28 23:02:50 +01:00
Charlie Wu
3c73bbf966 devtool: Fix the wrong variable in srcuri_entry
devtool crashes when running "update-recipe" and append changes on the recipe.
"$ devtool update-recipe -a <layer> <recipe>"
Traceback (most recent call last):
...
File "/ovss/ovss_quanta/poky/scripts/lib/devtool/standard.py", line 1636, in srcuri_entry
    return 'file://%s%s' % (basepath, paramstr)
                            ^^^^^^^^
NameError: cannot access free variable 'basepath' where it is not associated with a value in enclosing scope

The input variable 'fname' should have the same meaning as the variable 'basepath'.
Modify the 'fname' to 'basepath' and solve the issue.

(From OE-Core rev: c3231756bbc2cb5641204414ad3670d7f8607ed3)

Signed-off-by: Charlie Wu <chiachiwu@google.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-15 09:53:38 +01:00
Ming Liu
0241bdb49c meta: introduce KCONFIG_CONFIG_ENABLE_MENUCONFIG
Currently, uboot do_menuconfig task is breaking when UBOOT_CONFIG is
chosen rather than UBOOT_MACHINE, it simply fails with the following
errors:
| make: *** No rule to make target 'menuconfig'. Stio.
| Command failed.
| Press any key to continue...

this is due to the work directory of do_menuconfig is set to ${B} but
not ${B}/$config.

We should distinguish two situations:
1) When there is only one config item in UBOOT_CONFIG, do_menuconfig
   should work just like how it works for UBOOT_MACHINE.
2) When there are multiple config items in UBOOT_CONFIG, do_menuconfig
   should print out some information saying it's not supported other
   than just failing.

This patch mainly aims to fix that by introducing a extra variable
KCONFIG_CONFIG_ENABLE_MENUCONFIG, it would be set to 'false' for
situation 2), and when it's set to 'true', then set
KCONFIG_CONFIG_ROOTDIR correctly in uboot-config.bbclass to let
do_menuconfig task work.

DEVTOOL_DISABLE_MENUCONFIG could be replaced by this new variable
KCONFIG_CONFIG_ENABLE_MENUCONFIG.

(From OE-Core rev: f9e834e317880cf47dbb4f8285bc36d743beae5e)

Signed-off-by: Ming Liu <liu.ming50@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-09 13:55:21 +01:00
Frieder Paape
f7035ce464 image_types: Fix reproducible builds for initramfs and UKI img
I've encountered issues reproducing initramfs and UKI image builds,
which will be fixed with this patch.

1. initramfs
There's a symbolic link to /sbin/init, which is appended to the cpio archive after creation.
The links timestamp needs to be static and the cpio append command needs the '--reproducible' flag to produce deterministic outcomes.

2. Unified Kernel Image
'--preserve-dates' is required for a static 'Time/Date' entry.
I've added '--enable-deterministic-archives' although in my case this
didn't change anything.

(From OE-Core rev: fd027729bafb4e085ba0949e38e724f3a8cad102)

Signed-off-by: Frieder Paape <frieder@konvera.io>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-06-06 23:41:13 +01:00
Leon Anavi
08d26fb375 wic: Add argument --hidden to default imager
Add argument --hidden to avoid MS Windows prompting to format
partition after flashing to a USB stick, SD card on another media.
Set Bit 0 (RequiredPartition) to mark that the partition is
required for the platform to function on GUID Partition Table
(GPT).

The new argument simplifies setting RequiredPartition on GPT
through a WKS file and the default imager plugin. Otherwise,
without this feature, to achieve the same result a new imager
plugin has to be implemented and set in WIC_CREATE_EXTRA_ARGS.

(From OE-Core rev: 7a111ff58d7390b79e2e63c8059f6c25f40f8977)

Signed-off-by: Leon Anavi <leon.anavi@konsulko.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-05-25 10:29:08 +01:00
Sergei Zhmylev
6722f40600 wic: add support for proper kernel name to bootimg-pcbios
Use appropriate kernel image name instead of hard-coded vmlinuz for bootimg-pcbios plugin.

(From OE-Core rev: 8d9f00ba456fe76e0f4ef0e68ec64fc538c90d89)

Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-05-04 12:37:00 +01:00
Randolph Sapp
ef577b03d7 wic/bootimg-efi: if fixed-size is set then use that for mkdosfs
This is a bit of a compatibility issue more than anything. Some devices
get upset if the FAT file system contains less blocks than the
partition.

The fixed-size argument is currently respected by the partition creation
step but not by the file system creation step. Let's make it so the file
system respects this value as well.

(From OE-Core rev: 38e1a235f5eceade7c871f96dc97f6c384384c7b)

Signed-off-by: Randolph Sapp <rs@ti.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-27 14:41:31 +01:00
Dit Kozmaj
18b04f9c3f wic: use part_name when defined
So far part.label has been used to define GPT partition label even if
part.part_name was defined.
Fix the code to use part.part_name whenever available, as it makes sense
to have a GPT partition label which is different from the contained
filesystem label.

(From OE-Core rev: 7704d5fc36eb065224792bf4d5543814eaa5fed3)

Signed-off-by: Dit Kozmaj <dit.kozmaj@kynetics.com>
Signed-off-by: Diego Rondini <diego.rondini@kynetics.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-13 11:56:07 +01:00
Andrew Geissler
e9e5a7910e filemap.py: enforce maximum of 4kb block size
The logic in this script validates that the length of data sections are
evenly divisible by the block size. On most systems the block size is
4KB and all is good. Some systems though, such as ppc64le, have a block
size larger then 4KB. For example on a POWER9 based ppc64le system, the
block size is 64KB.

This results in this script failing with errors like this when building
wic images:
|440, in _do_get_mapped_ranges
|     assert extent_len % self.block_size == 0
|            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| AssertionError

In this case the data section size was 268KB and the block size was
64KB, resulting in the above assert failure.

Resolves https://bugzilla.yoctoproject.org/show_bug.cgi?id=15075

(From OE-Core rev: 1e23b803af6991fc20e4a4e88a0ef0541399e722)

Signed-off-by: Andrew Geissler <geissonator@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-27 15:44:02 +01:00
Ross Burton
41de1ecadd scripts/lib/buildstats: handle top-level build_stats not being complete
If we try to parse a buildstats directory which was either aborted or
is still being built then the top-level build_stats file doesn't
contain an elapsed value which causes an exception:

UnboundLocalError: local variable 'elapsed' referenced before assignment

Default both start and elapsed to 0 so that the parse succeeds.

(From OE-Core rev: 701d985aa8f2e9c2b9c0736fa25b424f3701889e)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-25 09:39:28 +00:00
Richard Purdie
bedf823032 resulttool: Improve overlapping ptest result reporting
With the move to a ptest per image, the multiple ptest log scenario
is much more likely. Tweak the handling to only warn if there are
overlapping files.

(From OE-Core rev: 8391f1668fcbe932ba846299b0ec22de09e06dd4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-16 22:52:15 +00:00
Ross Burton
9db9f8c575 resulttool: add log --list-ptest
Add a convenience argument to the log subcommand to list all of the
ptest logs in a testresults file.

(From OE-Core rev: cd2d7adf02005d46f19c93a40db1e10ce01ac261)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-14 17:13:11 +00:00
Ross Burton
37ab2ecbb7 lib/resulttool: fix typo breaking resulttool log --ptest
ptestresult_get_log() looked for a key called 'ptestresuls.sections',
which should be 'ptestresult.sections'

(From OE-Core rev: 7c8c9f7283e54bf8b1521fbaad7dceb66a8fcdbb)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-14 17:13:11 +00:00
Alexander Kanavin
31bdac666f devtool/upgrade: do not delete the workspace/recipes directory
If it exists, there is no need to delete it, and if it does not,
devtool prints an ugly traceback.

(From OE-Core rev: af82e59e8f08369aabd5fa6eb43022982d4e59a7)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-14 17:10:00 +00:00
Alexis Lothoré
1eae53e277 scripts/resulttool: do not count newly passing tests as regressions
resulttool regression module simply compare a base test status to a target test
result status. This approach raises many false positives since all XXX -> PASS
transitions (XXX being any status different from PASS) are flagged as
regression.

- Do not list XXX -> PASS transitions in regression  report, instead count them
  and print a summary of "newly passing tests"
- If an inspected pair has only "newly passing tests", do not print detailed
  list and print it as "Improvement" instead of "Regression"

Updated output example looks like the following:
[...]
Improvement: oeselftest_fedora-37_qemux86-64_20230127010225
             oeselftest_ubuntu-22.04_qemux86-64_20230226120516
             (+1 test(s) passing)
[...]
Match:       oeselftest_almalinux-8.7_qemuarm64_20230127015830
             oeselftest_almalinux-8.7_qemuarm64_20230227015258
[...]
Regression:  oeselftest_almalinux-9.1_qemumips_20230127000217
             oeselftest_opensuseleap-15.4_qemumips_20230226130046
    ptestresult.glibc-user.debug/tst-read-chk-cancel: PASS -> None
    ptestresult.glibc-user.nptl/tst-mutexpi4: PASS -> FAIL
    ptestresult.glibc-user.nptl/tst-mutexpi5a: PASS -> FAIL
    Additionally, 44 previously failing test(s) is/are now passing

(From OE-Core rev: c335f96f687c73fde443ac330ca3e17113794d9e)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-12 23:39:13 +00:00
Alexis Lothoré
1feffaf41a scripts/resulttool: fix ptests results containing a non reproducible path
Some ptests results may be wrongly sampled, resulting in some part of the error
being propagated in test name. For example:

"ptestresult.binutils-ld.in testcase /home/pokybuild/yocto-worker/qemumips/build/build-st-1666126/tmp/work/mips32r2-poky-linux/binutils-cross-testsuite/2.40-r0/git/ld/testsuite/ld-ctf/ctf.exp"
"ptestresult.gcc.Couldn't create remote directory /tmp/runtest.455781 on ssh"
"ptestresult.gcc-libstdc++-v3.Couldn't create remote directory /tmp/runtest.3814266 on target"

While the root errors must be fixed in corresponding ptests packages for those
tests, the test results history must still be usable even with this issue

Add new filters to detect if temporary test directories (build-st-*,
/tmp/runtime.*) are present in name. If so, truncate test name. As a side
effect, it will aggregate multiple failing errors into one, but the regression
will still be raised

(From OE-Core rev: 601eecfddd26bfe2954835a73ed1116bb520235f)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-12 23:39:13 +00:00
Alexis Lothoré
e8048a5b66 scripts/resulttool: call fixup_ptest_names in regression_common
ptests names not only need to be fixed for regression based on git testresults
but also for testsresults provided "manually"

Move ptests naming fixup in regression_common to share the fixup between both
regression use cases

(From OE-Core rev: f772ccd108dc3d618db9d479d672c0f3edd203ca)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-12 23:39:12 +00:00
Richard Purdie
c45d58f003 resulttool/regression: Improve matching of poor ptest test names
Some test case naming is poor and contains random strings, particularly
lttng/babeltrace but also curl. Truncating the test names works since they
contain file and line number identifiers which allows us to match them
without the random components, or in the case or curl, test IDs.

Going forward we may be able to improve the test names but this
tweak allows historical test results to work in reports.

(From OE-Core rev: 541a2e2683531355e678fd93524a0c4a8c43a8ff)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-26 11:59:52 +00:00
Richard Purdie
401d022a26 resulttool/regression: Ensure LTP results are only compared against other LTP runs
If a test result contains LTP test results, it should only be compared with
other runs containing LTP test results.

(From OE-Core rev: 4dbbf2f4a85620a08dc2fa65095dc17fe6c530f8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-26 11:59:52 +00:00
Alexis Lothoré
383cd86595 scripts/resulttool/regression: add metadata filtering for oeselftest
When generating regression reports, many false positive can be observed since
some tests results are compared while the corresponding tests sets are not the
same, as it can be seen for example for oeselftest tests (oeselftest is run
multiple time but with different parameters, resulting in different tests sets)

Add a filtering mechanism in resulttool regression module to enable a better
matching between tests. The METADATA_MATCH_TABLE defines that when the TEST_TYPE
is "oeselftest", then resulttool should filter pairs based on
OESELFTEST_METADATA appended to test configuration. If metadata is absent from
test results, in order to keep compatibility with older results, add a
"guessing" mechanism to generate the missing OESELFTEST_METADATA. The guessed
data is tightly coupled to the autobuilder configuration, where all oe-selftest
executions are described

(From OE-Core rev: 94ab7c2b892bf292dd86619ca9c63ddd7bf53f3c)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-26 11:59:52 +00:00
Alexis Lothoré
88345ecf47 scripts/resulttool/regression: remove unused import
(From OE-Core rev: 4d6a74fcee1479562bf66360f20d528107c70887)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-26 11:59:52 +00:00
Pavel Zhukov
4add8776ab wic: Fix populating of IMAGE_EFI_BOOT_FILES with uefi-kernel
wic plugin crashed if IMAGE_EFI_BOOT_FILES is not empty and uefi-kernel
loader specified because of preliminary return from the function.
[Yocto #15033]
Fixes:
|   File "/mnt/builds/yocto/sources/scripts/lib/wic/plugins/source/bootimg-efi.py", line 371, in do_prepare_partition
|     for src_path, dst_path in cls.install_task:
|                               ^^^^^^^^^^^^^^^^
| AttributeError: type object 'BootimgEFIPlugin' has no attribute 'install_task'. Did you mean: 'do_install_disk'?

(From OE-Core rev: a6372f70f31b39ce9867b705d02205621d7a8685)

Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-26 11:49:00 +00:00
Chee Yang Lee
80a2028813 checklayer: check for patch file upstream status
yocto-check-layer to check all .patch file in layer for
Upstream-status and list down all .patch file without Upstream-Status.

set this test as expected failure for now, so this wont fail the check
layer while still able to capture any patches with malformed or missing
Upstream-Status in report.

[YOCTO #14642]

(From OE-Core rev: 237c1b66e5014123c1e5c3e78f9ab0357bcd62dc)

Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-24 17:05:24 +00:00
Alexander Kanavin
ca7306f78a devtool: ignore patch-fuzz errors when extracting source
So that patch fuzz issues can actually be fixed,
as extracting source with 'devtool modify' is the first step
for that.

(From OE-Core rev: 7067abd31d9dd4b98ec70c1c7effbe2904797cd1)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-24 13:31:45 +00:00
Ross Burton
163aa6f149 lib/buildstats: handle tasks that never finished
If a task is aborted the buildstats file isn't complete, so calculate
when the build finished and use that as a end time.

(From OE-Core rev: 23ebaec476dc46aebe5997f025661137f3e341bd)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-17 18:01:01 +00:00
Pavel Zhukov
7f6a17a19c wic: Fix usage of fstype=none in wic
This allows to specify partition with fstype=none in the wks file
to have partition created but without following mkfs. The none fstype
is in the list already but the usage is not documented.

Example;
part /data --ondisk mmcblk0 --fstype=none  --align 4096 --fixed-size 512

will create a partition, filesystem may be created manualy on the host
or target and data will be preserved if the device is reflashed using
same wks. Works with bmaptool and probably does not work with dd.
Use case is persistent filesystem/data between reflashing of the image.

(From OE-Core rev: 351cb64da37aa43113e5192605d04436652aa3b8)

Signed-off-by: Pavel Zhukov <pavel@zhukoff.net>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-15 10:21:34 +00:00
Xiaotian Wu
324b03e15f base: add support for loongarch64
(From OE-Core rev: 759baaceb4dd623d5da12ba0d01540fa080154ba)

Signed-off-by: Xiaotian Wu <wuxiaotian@loongson.cn>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-04 17:02:10 +00:00
Martin Jansa
94149c4f37 meta: remove True option to getVar and getVarFlag calls (again)
* True is default since 2016 and most layers were already updated
  not to pass this parameter where not necessary, e.g. oe-core was
  updated couple times, first in:
  https://git.openembedded.org/openembedded-core/commit/?id=7c552996597faaee2fbee185b250c0ee30ea3b5f

  Updated with the same regexp as later oe-core update:
  https://git.openembedded.org/openembedded-core/commit/?id=9f551d588693328e4d99d33be94f26684eafcaba

  with small modification to replace not only d.getVar, but also data.getVar as in e.g.:
  e.data.getVar('ERR_REPORT_USERNAME', True)

  and for getVarFlag:
  sed -e 's|\(d\.getVarFlag \?\)( \?\([^,()]*, \?[^,()]*\), \?True)|\1(\2)|g' \
      -i $(git grep -E 'getVarFlag ?\( ?([^,()]*), ?([^,()]*), ?True\)' \
          | cut -d':' -f1 \
          | sort -u)

(From OE-Core rev: 26c74fd10614582e177437608908eb43688ab510)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-02 09:50:02 +00:00
Richard Purdie
c0981c9622 wic/efi-bootdisk.wks: Fix for 6.1 kernel versions
oe-selftest efibootpartition.GenericEFITest.test_boot_efi was failing for 6.1 kernels with:

| ERROR: _exec_cmd: export PATH=[...] mcopy -i [...]/rootfs_boot.1.vfat -s [...]/rootfs1/* ::/ returned '1' instead of 0
| output: Disk full

I believe we hit a file boundary size and having "0" overhead in the image meant
the files couldn't be installed. Allow a small amount of overhead to avoid
the error.

(From OE-Core rev: 16e0b8a8fc36f5525b1801888851958f0dbe84c2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-01-26 21:57:48 +00:00
Alexander Kanavin
c9c2b6b613 devtool: process local files only for the main branch
devtool modify/upgrade are not currently equipped to handle conditional local files
in SRC_URI, and provide only the main no-override set in a workspace under
source/component/oe-local-files/ (this is done via meta/classes/devtool-source.bbclass).

On the other hand, updating the changes from workspace into a recipe
is run iteratively against all overrides; this works for patches (as they
all are directed into their own override branches in the workspace
git source tree), but breaks down when trying to match local files
in a workspace against local files in overridden SRC_URI lists, resulting in
bad recipe breakage.

(there's an additional twist here: existing code has a guard against this
but the guard relies on metadata in workspace .bbappend that is only there
in modify operations, but not upgrades. This commit replaces the guard
with a general check that will work everywhere).

Implementing multiple sets of local files is significant work; let's for now
simply not touch local files in recipes except when on the no-override variant.

Also, adjust the selftest cases to include conditional local files in sample
recipes, so the situation is covered by the tests.

(From OE-Core rev: 3a8654b860fa98f94e80c3c3fff359ffed14bbe7)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-31 17:10:21 +00:00
Alexander Kanavin
04d2a394ea devtool/upgrade: correctly handle recipes where S is a subdir of upstream tree
'devtool modify' writes additional settings to workspace .bbappend so that this
can be handled correctly, but 'devtool upgrade' does not. This adds the missing
settings.

In particular, local files should not anymore mysteriously disappear from
SRC_URIs on upgrades.

(From OE-Core rev: 0817aa5537a8d7cc9591c53dfaa1d225f4c327f7)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-18 19:48:00 +00:00
Richard Purdie
7458ad340f yocto-check-layer: Allow OE-Core to be tested
For unknown reasons we've never seemingly run the check layer script
against OE-Core itself. This isn't entirely straightforward as the core
layer is a bit of a special case, we can't for example compare signatures
against ourselve and we can't remove core from bblayers.conf.

Core does have distro, machine and software components too, in the case
of distro, our fallback default settings. Whilst the qemu machines could
be split into a seperate layer directory, core wouldn't then parse at all
standalone due to the lack of any machine so it seems a bit pointless to
do that.

These changes tweak the script to handle core's special cases, specifically
to allow distro and machine directories and to account for the README placed
a directory level higher than other layers.

(From OE-Core rev: ba312ed228507d05f280aeb96819d671b01400b8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-09 13:18:41 +00:00
Richard Purdie
9773050910 scripts/checklayer: Update to match bitbake changes
Bitbake additions for the addpylib API mean we need to update the parsing
function call to be clear we're parsing in configuration context.

(From OE-Core rev: ef7677dc90fac089f8b9f6da301cca022ed7284c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-08 11:54:39 +00:00
Ross Burton
d9b5b8cb21 lib/buildstats: fix parsing of trees with reduced_proc_pressure directories
The /proc/pressure support in buildstats is creating directories in the
buildstats tree called reduced_proc_pressure, which confuses the parsing
logic as that cannot be parsed as a name-epoc-version-revision tuple.

Explicitly skip this directory to solve the problem.

(From OE-Core rev: 24f0331f0b7e51161b1fa43d4592b491d2037fe9)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-08 10:48:36 +00:00
Sergei Zhmylev
be91408e10 wic: make ext2/3/4 images reproducible
Ext2/3/4 FS contains not only mtime, but also ctime, atime and crtime.
Currently, all the files are being added into the rootfs image using
mkfs -d functionality which affects all the timestamps excluding mtime.
This patch ensures these timestamps inside the FS image equal to
the SOURCE_DATE_EPOCH if it is set.

(From OE-Core rev: 75d2dd0ea7790db2e8ee921784ca373abff2df65)

Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-07 21:57:21 +00:00
ciarancourtney
d2a483338f wic: swap partitions are not added to fstab
- Regression in 7aa678ce804c21dc1dc51b9be442671bc33c4041

(From OE-Core rev: f1243572ad6b6303fe562e4eb7a9826fd51ea3c3)

Signed-off-by: Ciaran Courtney <ciaran.courtney@activeenergy.ie>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-29 16:28:36 +01:00
Sergei Zhmylev
0ee936da2e wic: honor the SOURCE_DATE_EPOCH in case of updated fstab
In case user requested to build a binary repeatable package,
it's required to honor the SOURCE_DATE_EPOCH environment
variable. So forcefully set mtime inside all the routines
which modify fstab in case it is updated.

(From OE-Core rev: 99719a3712a88dce8450994d995803e126e49115)

Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-26 12:28:39 +01:00
Sergei Zhmylev
36b9f4f3d0 wic: implement binary repeatable disk identifiers
When SOURCE_DATE_EPOCH variable is set, binary repeatable build
is expected. This commit implements reproducable disk identifiers
in such a case using its value as a Random seed.

(From OE-Core rev: 2c0c54e12169e76f16fb3398904bc897a9190397)

Signed-off-by: Sergei Zhmylev <s.zhmylev@yadro.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-26 12:28:39 +01:00
Maxim Uvarov
2cd51cfe99 wic: bootimg-efi: implement --include-path
--include-path can be used for placing files on ESP,
like dtbs.

(From OE-Core rev: 46ca73f504b0792766dc4d4bbad38beb04ef4ed0)

Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-26 12:28:37 +01:00
Maxim Uvarov
5b0de8447e wic: add UEFI kernel as UEFI stub
Linux kernel can be compiled as UEFI stub and loaded directly
with UEFI firmware without grub or other UEFI shell.

Tested with wic file:
bootloader  --ptable gpt --timeout=0  --append="rootwait"
part /boot --source bootimg-efi --sourceparams="loader=uefi-kernel" \
 --ondisk sda  --fstype=vfat --label bootfs \
  --active --align 1024 --use-uuid
part / --source rootfs --fstype=ext4 --label rootfs \
  --align 1024 --exclude-path boot/ --use-label

(From OE-Core rev: b18025fc8a2dad963b6f4b697e24455f2511b279)

Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-26 12:28:37 +01:00
Ross Burton
e7619f7650 Revert "wic/plugins/images/direct: Allow changes in fstab on rootfs"
Commit 20d43a2 allows changes in fstab for the / mount. However, if the
fstab contains / already then this results in two entries for / in the
fstab, which is confusing and results in systemd producing errors on
boot:

  systemd-fstab-generator[11101]: Failed to create unit file
  /run/systemd/generator/-.mount, as it already exists. Duplicate entry
  in /etc/fstab?

Wic should be extended to support merging mount points in fstab, but as
we're about to release revert the patch that introduced this change as
it hasn't been in a release yet, and the issue which prompted the patch
in the first place can be revisited post-release.

[ YOCTO #14865 ]

(From OE-Core rev: 4a7b3ef0a69b7e08030519b712a6fe4dd199c571)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-28 08:01:11 +01:00
Alexander Kanavin
f2fb3c54a3 devtool: do not leave behind source trees in workspace/sources
These are typically auto-extracted with modify/upgrade from recipes
and can be easily recreated. On the rare occasions where they need
to be reused, they are still available under workspace/attic (which
is already used for old recipes and appends), so nothing gets lost.

This avoids the annoyance of devtool refusing to proceed because
there is a previous source tree in workspace/sources.

For independent source trees behave as before: do nothing.

Adjust the test that previously deleted those trees by hand.

(From OE-Core rev: 9bfb95d070d68d5ab5adfe0ea096f5fbf9cad8b0)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-21 22:51:42 +01:00
JeongBong Seo
0b85162b9c wic: add 'none' fstype for custom image
It's not possible to set the label (of gpt entry) normally
when I want to use non-listed fstype as a rawcopy.

Example)
part ? --source rawcopy --ondisk mmcblk0 --label mypart --sourceparams file=mypart.raw

To resolve this problem, this patch addes a 'none' fstype
and ignore do_image_label on rawcopy (that actually set the partition label.)

(From OE-Core rev: 949cf797eb54f3d22a093910087e937633ce9b23)

Signed-off-by: JeongBong Seo <jb.seo@lge.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-12 19:22:04 +01:00
Richard Purdie
dc850a1066 recipetool: Update for class changes
(From OE-Core rev: 0ea8afd4f9599469d1a23824c451c62eabb76660)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-12 15:27:17 +01:00
Richard Purdie
7350e82ce1 scripts: Add copyright statements to files without one
Where there isn't a copyright statement, add one to make it explicit.
Also drop editor config lines where they were present and add license
identifiers as MIT if there isn't one.

(From OE-Core rev: deb3ccec53e0bd63bc4235cf2b0d3fc781687361)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-12 11:58:01 +01:00
Alexander Kanavin
a782a46c37 devtool/upgrade: catch bb.fetch2.decodeurl errors
Otherwise, workspace cleanup (removing bogus recipe and source tree)
will not happen, leaving breakage behind.

(From OE-Core rev: 74774f9b67580a8c56f605dfd4cc7b856bbeeae8)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-04 16:27:25 +01:00
Alexander Kanavin
b8c5b2bfd5 devtool/upgrade: correctly clean up when recipe filename isn't yet known
There is a coding error in the second invocation of _upgrade_error:
rf is passed into it before it is initialized in the try: block. And so
bogus recipes are left behind in the workspace, causing breakage.

Instead, rewrite the functions to take the recipe directory name in the
workspace layer, which can be calculated in advance.

(From OE-Core rev: e653996369c1d2b5ac8367ad85f4816d679b6c98)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-04 16:27:25 +01:00