Commit Graph

3497 Commits

Author SHA1 Message Date
Martin Jansa
a0b0b25a82 create-pull-request: don't switch the git remote protocol to git://
Many git repos prefer https:// nowadays and many removed support
for git://.

This breaks the script when using github.com even when selected remote
is ssh (git@github.com:openembedded/...), it will re-write it to git://
before calling git pull-request causing:

openembedded-core $ scripts/create-pull-request -u github -b jansa/artifacts -o pull-kernel
NOTE: Assuming local branch HEAD, use -l to override.
fatal: unable to connect to github.com:
github.com[0: 140.82.121.3]: errno=Connection timed out

warn: No match for commit ea003bd026aa24bb4c8b7562f44ed6512e921259 found at git://github.com/shr-distribution/oe-core
warn: Are you sure you pushed 'jansa/artifacts' there?
ERROR: git request-pull reported an error

(From OE-Core rev: 5aac691040502e6f578316dfc090643d341dd3cb)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 64c466920b808c35d1ac87b47cf438bc79becea7)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-10-11 21:56:13 +01:00
Alexandre Belloni
a5eeb959bf runqemu: display host uptime when starting
In order to be able to debug issues on the host that have an impact on the
guest, it is useful to get the uptime of the host while starting so we can
match with the events in dmesg. Also include the uptime when cleaning up.

(From OE-Core rev: 08406e03abddc7290c0c2296aa179725a58155d3)

Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2d96499823f7de6e16a461426491e015ba63c1ec)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-16 17:53:23 +01:00
Ulrich Ölmann
2a82c3d93a scripts/runqemu.README: fix typos and trailing whitespaces
(From OE-Core rev: b765d234c9d9d1e5e6bb02b074f42cea8fb48df2)

Signed-off-by: Ulrich Ölmann <u.oelmann@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 217b00d378f359689613ca4c0666bb2eed040f69)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-09-12 08:41:48 +01:00
Ross Burton
cd88865fd2 wic/bootimg-efi: use cross objcopy when building unified kernel image
We can't rely on the host objcopy knowing how to process target binaries,
so use the cross objcopy in the sysroot instead.

Also construct the command argument-by-argument as the format expression
was getting unwieldy.

(From OE-Core rev: 0264aeedbf21e9e7a104243c11b3b57f00e38bda)

(From OE-Core rev: 4e8b803d7efa13c950353bb00ab65be22eb61736)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-31 16:54:17 +01:00
Ross Burton
8191b79123 wic: add target tools to PATH when executing native commands
We might want to run a cross tool, such as objcopy, in wic.  These are
in a TARGET_SYS/ subdirectory under /usr/bin, so add that directory to
the search path too.

(From OE-Core rev: c523549141e5c31edc75281f581d97867b7d251d)

(From OE-Core rev: f8e0512503410ca5137fcf114fbffb52aa98be07)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-31 16:54:17 +01:00
Paul Eggleton
84eb7f3b98 relocate_sdk.py: ensure interpreter size error causes relocation to fail
If there is insufficent space to change the interpreter, we were
printing an error here but the overall script did not return an error
code, and thus the SDK installation appeared to succeed - but some of
the binaries will not be in a working state. Allow the relocation to
proceed (so we still get a full list of the failures) but error out at
the end so that the installation is halted.

(From OE-Core rev: 3f258378bcaebb9c42c0011c7c424c920ef71d22)

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit c5a9a448e462d3e5457e8403c5a1a54148ecd224)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-23 15:22:52 +01:00
Alexander Kanavin
ac8186042d scripts/oe-setup-builddir: make it known where configurations come from
"with some default values" isn't clear; if the user wants to change or
inspect the default values and their history, we should help them find
where they are. This becomes especially important when using template
configs other than poky's.

(From OE-Core rev: b60f4c7f3205b354469c1aa8b56ceaacc11d486b)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit ec34783ffc34eb9e9697f1b192c5a0043f1ca2c6)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-23 15:22:52 +01:00
Mark Hatle
cb87867c47 runqemu: Add missing space on default display option
(From OE-Core rev: fcb55a198eddf4110fd4baf67614a7598441d952)

Signed-off-by: Mark Hatle <mark.hatle@kernel.crashing.org>
Signed-off-by: Mark Hatle <mark.hatle@amd.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ee9428611fc38bc711b5b3e12cf0d3257b1b5680)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-23 15:22:52 +01:00
Alexander Kanavin
0ff2b0bbf8 devtool/upgrade: catch bb.fetch2.decodeurl errors
Otherwise, workspace cleanup (removing bogus recipe and source tree)
will not happen, leaving breakage behind.

(From OE-Core rev: 72bfdca08029c031cedc9dbbf366663632c1c8db)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit 74774f9b67580a8c56f605dfd4cc7b856bbeeae8)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-23 15:22:52 +01:00
Alexander Kanavin
53019423e4 devtool/upgrade: correctly clean up when recipe filename isn't yet known
There is a coding error in the second invocation of _upgrade_error:
rf is passed into it before it is initialized in the try: block. And so
bogus recipes are left behind in the workspace, causing breakage.

Instead, rewrite the functions to take the recipe directory name in the
workspace layer, which can be calculated in advance.

(From OE-Core rev: a7d406dce577192f98d76dbae411c72a8b7f07d4)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
(cherry picked from commit e653996369c1d2b5ac8367ad85f4816d679b6c98)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-23 15:22:52 +01:00
Roland Hieber
58d3e1b897 devtool: error out when workspace is using old override syntax
When the workspace bbappends are still using the old override syntax
with EXTERNALSRC_pn-*, externalsrc_re will not match, and pn will never
be assigned, leading to a nondescript UnboundLocalError being raised on
the user's terminal. Try to detect that situation and give the user a
hint how to solve it.

(From OE-Core rev: 3aee3425956020166d99ec085e35e21b3daf625f)

Signed-off-by: Roland Hieber <rhi@pengutronix.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit d42ea8e849cf2df3708406418b961168268b316a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-23 15:22:52 +01:00
Mihai Lindner
f327cf1077 wic/plugins/rootfs: Fix NameError for 'orig_path'
Fix "NameError: name 'orig_path' is not defined".
It's a typo from when this error was handled outside this function.

(From OE-Core rev: 15015dda2cfccb9d2894f8d9d8f04043604b7a5e)

Signed-off-by: Mihai Lindner <mihai.lindner@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 2124ec0d9f9de2da476f0024a0ccf70da987420f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-08-04 16:29:15 +01:00
Paul Eggleton
ee60960da3 devtool: finish: handle patching when S points to subdir of a git repo
If devtool finish needs to create a patch and have it applied to the
sources for a recipe where S points to a subdirectory of the sources,
then the patch needs to be applied at the root of the repo i.e. we need
to add a patchdir= parameter to the SRC_URI entry.

(From OE-Core rev: 7cf53810b1dc3d14c4838a610b3d53170f552c19)

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ad3736d9ca14cac14a7da22c1cfdeda219665e6f)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-25 15:11:46 +01:00
Paul Eggleton
de0d04dfa3 devtool: ignore pn- overrides when determining SRC_URI overrides
If (perhaps foolishly) at your configuration level you have e.g.

  SRC_URI_append_pn-recipename = " file://patchname.patch"

and then run devtool modify on a different recipe, an error occurs:

  INFO: SRC_URI contains some conditional appends/prepends - will create branches to represent these
  ...
  ERROR: [Errno 2] No such file or directory: '/path/to/downloads/patchname.patch'

pn- overrides would not constitute an alternative configuration that we
should handle in this context, so just ignore them to avoid the issue.

(From OE-Core rev: 9beb3472624050593fc30d5a00d3d13fec4441df)

Signed-off-by: Paul Eggleton <paul.eggleton@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3f2a812ade42ece0bb59b2d303125a91b29936dd)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-25 15:11:46 +01:00
Thomas Roos
0b047cb6e7 recipetool/devtool: Fix python egg whitespace issues in PACKAGECONFIG
Substitute expressions or whitespace from python egg requires.txt when
generating PACKAGECONFIG

Pysetuptools sees the uvicorn.egg-info/requires.txt as extra requirements.
Recipetool parses this information to generate the PACKAGECONFIG.
These extra requirements contain expressions and whitespace, which are not allowed in PACKGAGECONFIG.
This patch substitute them by hyphens to make PACKAGECONFIG parsable and readable.
Also adding an oe-selftest for this.

[YOCTO #14446]

(From OE-Core rev: 5a1fd88439c28c473a1723a040d780f100d6295e)

Signed-off-by: Thomas Roos <throos@amazon.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit a854d95a79e64f3f82abfa4cc1daec750abf4249)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-08 08:27:16 +01:00
Martin Jansa
2fd4a20088 wic: fix WicError message
* add missing % to print the values instead of:
  | INFO: Build artifacts not found, exiting.
  | INFO:   (Please check that the build artifacts for the machine
  | INFO:    selected in local.conf actually exist and that they
  | INFO:    are the correct artifacts for the image (.wks file)).
  |
  | ERROR: ("The artifact that couldn't be found was %s:\n  %s", 'kernel-dir', '/OE/build/deploy/images/qemux86-64')

(From OE-Core rev: 772c9f66633e85c5059670d328e1b5fad407457f)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e104c2b1273d8c5bd97893f318bf2a2699ef7f2d)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-07-08 08:27:16 +01:00
Xiaobing Luo
38f2a77176 devtool: Fix _copy_file() TypeError
when devtool finish, the _copy_file() failed.
--------------------------------------------
TypeError: _copy_file() got an unexpected keyword argument
'base_outdir'
--------------------------------------------

Fixes: 05f2d5d2ce00 ("devtool: finish: add dry-run option")

(From OE-Core rev: 5e6f4d0d3d314897b8ab2f45b3a78b0da9df99ab)

Signed-off-by: Xiaobing Luo <luoxiaobing0926@gmail.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit a45d9dc089fb2719ca69b92870917f8c0925f632)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-06-22 23:46:28 +01:00
Felix Moessbauer
a8a094e511 wic/plugins/rootfs: Fix permissions when splitting rootfs folders across partitions
This patches makes locating the file database containing the file and
folder usernames and permissions more reliable.
In addition to locating it relative to the partition directory, we also
try to locate it relative to the IMAGE_ROOTFS.

Prior to this patch, the database was not found when using
--rootfs-dir=${IMAGE_ROOTFS}/<x> in the WIC script, leading to erronous file
permissions and ownership.

(From OE-Core rev: a5406116ef647ff0cabc6b9466aa0e34e6a20750)

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 09e18ee246da8b56f446c4db548fb9c7e895142b)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-20 10:08:06 +01:00
Richard Purdie
7868ccd57a scripts/git: Ensure we don't have circular references
This is horrible but I'm running out of better ideas. We hit circular reference
issues which we were trying to avoid in the core HOSTTOOLS code. When building
the eSDK, there can be two copies of the script.

Therefore assume git will never be in a directory called scripts. This
fixes eSDK build failures.

(From OE-Core rev: b9dcaa76b3274ced1e4b9e2ca33f778e8cd50032)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 27de610ac30d4c81352efc794df7e9b1060f7a68)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-12 16:44:06 +01:00
Richard Purdie
db10b2623d scripts: Make git intercept global
The previous minimially invasive git intercept simply isn't enough. For example,
meson used in the igt-gpu-tools recipe hardcodes the path to git in the configure
step so at install time, changing PATH has no effect.

There are lots of interesting things we could do to try and avoid problems but
making the git intercept and dropping fakeroot privs for git global is probably
the least worst solution at this point. It will add slight overhead to git calls
but we don't make many so the overall impact is likely minimal.

(From OE-Core rev: 07f282fb94a5a7c0a3fad451c844e5b26074c744)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit af27c81eaf68ee681dcd9456a74cca6a9ab40bf6)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-12 16:44:06 +01:00
Steve Sakoman
4280a7dd25 scripts/contrib/oe-build-perf-report-email.py: remove obsolete check for phantomjs and optipng
Use of those tools was removed in b5c131006e3fad0a15e6cdf81f71dc1e96647028
perf-build-test/report: Drop phantomjs and html email reports support

(From OE-Core rev: 1063525be9b040ece8636c03ac8bab13952eb561)

Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 33df447affa7a3a360b1da028e6b12fbcd388db6)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-12 16:44:05 +01:00
Paul Gortmaker
f6eb720890 install/devshell: Introduce git intercept script due to fakeroot issues
In a devshell, recent versions of git will complain if the repo is owned
by someone other than the current UID - consider this example:

 ------
  bitbake -c devshell linux-yocto

  [...]

  kernel-source#git branch
  fatal: unsafe repository ('/home/paul/poky/build-qemuarm64/tmp/work-shared/qemuarm64/kernel-source' is owned by someone else)
  To add an exception for this directory, call:

        git config --global --add safe.directory /home/paul/poky/build-qemuarm64/tmp/work-shared/qemuarm64/kernel-source
  kernel-source#
 ------

Of course the devshell has UID zero and the "real" UID is for "paul" in
this case.  And so recent git versions complain.

As the whole purpose of the devshell is to invoke a shell where development
can take place, having a non-functional git is clearly unacceptable.

Richard suggested we could use PSEUDO_UNLOAD=1 to evade this issue, and I
suggested we probably will see other similar instances like this and should
make use of PATH to intercept via devshell wrappers - conveniently we already
have examples of this.

Here, we copy the existing "ar" example and tune it to the needs of git to
combine Richard's suggestion and mine.

As such we now also can store commit logs and use send-email with our user
specific settings, instead of "root", so in additon to fixing basic
commands like "git branch" it should also increase general usefulness.

RP: Tweaked the patch so the PATH change only applies to the devshell task
and is a generic git intercept rather than devshell specific.

RP: Also apply the PATH change to do_install tasks since that also runs under
fakeroot and several software projects inject "git describe" output into
their binaries (systemd, iputils, llvm, ipt-gpu-tools at least) causing
reproducibility issues from systems with different git versions.

(From OE-Core rev: 3a320c1555bf39b2d3c218ffc36827d9dda60fe1)

Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3266c327dfa186791e0f1e2ad63c6f5d39714814)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-04 13:07:34 +01:00
Henning Schild
c5ca6110ce wic: do not use PARTLABEL for msdos partition tables
When using "msdos" partition tables and "--label" but not "--use-uuid"
one can generate images which will not find their root, because
PARTLABEL does not work for "msdos".

Fix that by simply not going the PARTLABEL path in case of "msdos".

Fixes: 2fb247c5ecf0 ("wic: support rootdev identified by partition label")
(From OE-Core rev: 354ef6b723f50b5f0b46a2bf5797e5b982c6ea73)

Signed-off-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 9ea1a838b946020e026edc032039552b723fcaa4)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-05-04 13:07:33 +01:00
Kai Kang
09d2ca20cb update_udev_hwdb: fix multilib issue with systemd
It duplicates udevadm in systemd recipe to make it could run
update_udev_hwdb with multilib enabled. Since systemd last update,
it deploys a shared library libsystemd-shared-250.so in /lib/systemd/.
The library will be overwritten when multilib enabled. Then if both
udev-hwdb and lib32-udev-hwdb are installed, it fails to run the
multilib version postinstall intercept update_udev_hwdb:

| /path/to/build/tmp-glibc/work/intel_x86_64-wrs-linux/wrlinux-image-small/1.0-r1/rootfs/usr/libexec/lib32-udevadm:
    error while loading shared libraries: libsystemd-shared-250.so: wrong ELF class: ELFCLASS64

`udevadm hwdb --update` just concatenates .hwdb files in dirs
/etc/udev/hwdb.d and /lib/udev/hwdb.d. The output file hwdb.bin is
identical with the one created by lib32-udevadm. So do NOT duplicate
lib32-udevadm in systemd and eudev. And update intercept script
update_udev_hwdb that re-run udevadm with same arch qemuwrapper if run
${binprefix}qemuwrapper failed.

(From OE-Core rev: 74fe1b5af064f644a7d555b61527bb7d02cc30b8)

Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 3dba872a42c2be7d0865a30118984ab013850292)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-28 17:07:18 +01:00
Scott Murray
d3f246413b runqemu: Do not auto detect graphics if publicvnc is specified
The graphics option auto detection logic added in 7f78bb7a was not
checking if the publicvnc option had been specified, meaning that
it would be ignored and the auto detection result used instead.
Add setting a flag variable in the argument parsing and check
it along with the ones for the other graphics backend options.

(From OE-Core rev: 4b73b55c2d258768cda2bf7262ebb36bcb7fed5b)

Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-15 17:30:34 +01:00
Richard Purdie
0be2abfe7c runqemu: Allow auto detection of the correct graphics options
Running "runqemu qemux86 kvm" when qemu is configured for sdl and/or gtk
display output currently leads to a poor user experience with no cursor
and corrupted fonts in the gtk case. This is due to no options being
passed to qemu which leads to the loss of the font envirornment variable
and the show-cursor option.

If the user hasn't specified a display type, grep the output of
"qemu-system-xxx --help" for the display types and pick the "best"
which ensures our config is passed in. That resolves the gtk font issue
and the cursor issue with both sdl and gtk.

(From OE-Core rev: 7f78bb7a7baf67b9226fb460ca9e12fde6ef40c8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-14 09:47:00 +01:00
Abongwa Amahnui Bonalais
13eda43533 oe-init-build-env: add quotes around variables to prevent word splitting
Used shellcheck to add quotes to the variables. This is to make sure that
directories with names that have space between, such as
"Desktop/projects/test repo/poky" will not be considered as 2 separate
words. With this modification, running the command "source
oe-init-build-env" will not give the error "bash: oe-init-build-env: No
such file or directory"

(From OE-Core rev: b07a70fbf78f2beba639580e37dffbc0a73bc99f)

Signed-off-by: Abongwa Bonalais Amahnui <abongwabonalais@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-05 22:23:40 +01:00
Sundeep KOKKONDA
601befb6c7 meta: scripts - relocation script adapted to support big-endian machines
relocate_sdk.py was developed for little-endian architures and when tries
to install SDK for big-endian machines errors like below will be shown.
Error: struct.error: unpack requires a string argument of length 32. SDK could
not be set up. Relocate script failed. Abort!
Error: IOError: [Errno 22] Invalid argument. SDK could not be set up. Relocate
script failed. Abort!

To fix this, script is modified to support big-endian architecture.

(From OE-Core rev: 7d6f4b1373e4dfafc63702ef2426cd45100f18a3)

Signed-off-by: Sundeep KOKKONDA <sundeep.kokkonda@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-04-05 22:23:40 +01:00
Simon Kuhnle
5b5a7567bc convert-variable-renames: Fix typo in description
(From OE-Core rev: 8e3aa9638691709e136bf2005541bdfd4bb1a6f7)

Signed-off-by: Simon Kuhnle <simon.kuhnle@methodpark.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-31 17:52:58 +01:00
Alexandre Belloni
d68b2090e9 scripts/patchreview: handle Inactive-Upstream status
(From OE-Core rev: 44afce53725f59fefb0ca5df6babe2b8bec6a68b)

Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-18 23:32:46 +00:00
Richard Purdie
10d9a8ba7a perf-build-test/report: Drop phantomjs and html email reports support
phantomjs isn't reliable and we've moved to sharing the reports via a webserver.
Update the scripts to more match those being used in the autobuilder helper
where the html email support was removed.

(From OE-Core rev: b5c131006e3fad0a15e6cdf81f71dc1e96647028)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-16 10:31:41 +00:00
Richard Purdie
d38868cfff scripts/runqemu: Fix memory limits for qemux86-64
When setting memory to 4GB, qemu is only running with 2GB for x86_64.
Avoid this by removing the mem= option to the kernel and letting the
qemu configuration handle it for x86 in a similar way to mips.

(From OE-Core rev: 2fd53417eba354c31c058c4bb066bb882e098add)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-15 08:40:09 +00:00
Peter Kjellerstedt
6ad018f213 oe-pkgdata-util: Adapt to the new variable override syntax
(From OE-Core rev: 2bf6a0ca9fdf639418646700b20b65c9960efdbe)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-11 06:56:02 +00:00
Saul Wold
d9e500f83d meta/scripts: Improve internal variable naming
Update internal variable names to improve the terms used.

(From OE-Core rev: f408068e5d7998ae165f3002e51bc54b380b8099)

Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-10 08:00:28 +00:00
Bill Pittman
bf707d4431 wic: Use custom kernel path if provided
If the custom kernel path is provided in options, then
use that path instead of the default path.

(From OE-Core rev: 1068102216a894c467f71f6046fdb37d5577545c)

Signed-off-by: Bill Pittman <bill.pittman@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-08 09:44:28 +00:00
Saul Wold
1507286c7d convert-variable-renames: Fix output string
(From OE-Core rev: a8d6882144e76f384022fe7d2b4ee13ad876317a)

Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-04 17:14:15 +00:00
Richard Purdie
8a1de49b56 recipetool/devtool: Further SPDX identifier cleanups
Some of these are hard to know what to do with since the original source
files for the checksum aren't present. The safe option is to use "-only"
as often the main license is ambiguous and the source files themselves
determine the "or-later" possibility. The "-only" option therefore is
realistically what we need to use in this code.

(From OE-Core rev: 2b0cbafc7854de0308a624b17b8aaba704b031d5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-02 18:43:24 +00:00
Robert P. J. Day
eec3bbdd27 scripts: Various typo/grammar/punctuation fixes
Among other things, fix misspellings of:

  - absolute
  - deprecated
  - suitable
  - handle

and a bunch of other things.

(From OE-Core rev: c3773cd6c44dfe82be9ecd248120e7d6c753f891)

Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-02 18:43:24 +00:00
Peter Kjellerstedt
68d1a56e98 recipetool: Use SPDX license identifiers
There are still a couple of cases where the license may be set as, e.g.,
"GPL" or "GPL-2.0" since there is not enough information to decide the
actual SPDX license. It is then assumed that the developer will have to
correct the information.

(From OE-Core rev: e7df51f8d2361e9fe2d67669d2e17f0a5d01004e)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-01 23:44:59 +00:00
Peter Kjellerstedt
949aa76210 recipetool/create_buildsys_python: Add support for more known licenses
Add all OSI approved licenses from https://pypi.org/classifiers/. Also
add support for Other/Proprietary (Proprietary) and Public Domain (PD).

(From OE-Core rev: 99ef134d1019e5b98b845cf71f3eb39871218f9d)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-03-01 23:44:59 +00:00
Khem Raj
20f23b5f65 scripts/documentation-audit: Use renamed LICENSE_FLAGS_ACCEPTED variable
(From OE-Core rev: c8f8fe5a4f57febb1fb9b54f53d2a0b95f01179b)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-25 08:38:46 +00:00
Scott Murray
3c7c2fa055 scripts: fix file writing in convert-spdx-licenses
The convert-spdx-licenses.py script needs the same file closing
fix as was made to convert-variable-renames to ensure modified
file contents get flushed out.

(From OE-Core rev: 46135c87345c7189053dafbed92c754f9f328c32)

Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-22 15:24:51 +00:00
Scott Murray
ccbc29bbc4 scripts: fix file writing in convert-variable-renames
In my test environments (Fedora 35 and Debian 10.10 on AMD 2970WX),
running the convert-variable-renames.py rename script was resulting
in empty files instead of updated ones.  From inspection, the new
files are never flushed/closed before moving them into place, which
seems inherently racy.  Adding an explicit close to flush the modified
contents out before moving into place fixes the issue for me.

(From OE-Core rev: 187ac1ea0a701a5ba9ec92f6aa32f2a67600a584)

Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-22 08:23:50 +00:00
Saul Wold
0b46552625 meta: Rename LICENSE_FLAGS variable
(From OE-Core rev: 5c5b3bc563059ba728dc9724656cc69669f8e25f)

Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21 23:37:27 +00:00
Richard Purdie
3be8a7a6b1 icecc: Improve variables/terminology
The SYSTEM and USER seperation between variables seems odd and not necessary,
drop it. Avoid the use of whitelist/blacklist and also change "packages" to
"recipes" since that misuse causes confusion.

(From OE-Core rev: 0df0eb6401a02139b9110bc95e21d97a67125ec5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21 23:37:27 +00:00
Saul Wold
5a3d6c7bda scripts: Add convert-variable-renames script for inclusive language variable renaming
This script searches for a list of variable that have been renamed
and converts them to their more descriptive names. It also searches
for a list of variables that have been removed or deprecated and
prints a message.

It will print a message to inform the user that there are terms that
need to be updated in their files. Many of these changes are context
sensitive and may not be modified as they might be existing calls to
other libraries. This message is informational only.

I have tested this on poky and meta-openembedded so far.

(From OE-Core rev: 50fe7ba8dba05a9681c9095506f798796cfc2750)

(From OE-Core rev: 75f319c105484d0b312a858cc0bd8148728c8622)

Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21 23:37:27 +00:00
Richard Purdie
492214104a meta/scripts: Change BB_ENV_EXTRA_WHITE -> BB_ENV_PASSTHROUGH_ADDITIONS
After the change to bitbake, update the references in OE-Core to match the updates.

(From OE-Core rev: 193affb9f28b0116c3fd619834f145326fee08c5)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21 23:37:26 +00:00
Denys Dmytriyenko
bb4f2bfcec yocto-check-layer: add ability to perform tests from a global bbclass
This is useful when needing to test layer's recipes, where this special
bbclass can define a global python function that gets called on each
recipe parsing during "bitbake -S none world" signature dump and be able
to fail layer's check accordingly.

First test being added is to detect recipes skipping "installed-vs-shipped"
QA check. As "installed-vs-shipped" is a packaging QA check, it happens very
late in the build process and failing it could mean some potential issues
with packaging, especially when recipe uses BBCLASSEXTEND="nativesdk" and
resulting package is used in an SDK.

In OE-Core failing this QA check leads to an error, but other layers can
suppress it or change it to a warning. Detecting weird packaging problems
with SDKs is quite difficult and time consuming. Also, waiting for the
actual "installed-vs-shipped" packaging QA check to fail means that all
recipes in the layer under test have to run through all standard tasks in
the build chain, equivalent to a multi-hour world-build.

Hence yocto-check-layer takes a shortcut and only detects a mere attempt
at skipping "installed-vs-shipped" QA check in the INSANE_SKIP list during
initial parsing when dumping the signature information for the layer.

(From OE-Core rev: e8baa75535fc888f1d768b23a0140475e832c910)

Signed-off-by: Denys Dmytriyenko <denis@denix.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21 23:37:26 +00:00
Alexander Kanavin
8f4b84c98c devtool: explicitly set main or master branches in upgrades when available
In particular this resolves devtool's inability to pick a branch
when the same tagged commit is avaiable in main and in a release maintenance
branch.

Thanks to Peter Kjellerstedt for the suggestion.

(From OE-Core rev: bcb21ee2760a2c76039412a56c6cda43fbf96fd0)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-21 21:53:52 +00:00
Richard Purdie
301d24952e scripts: Add a conversion script to use SPDX license names
We're standardising on SPDX license names so it is overdue to change
the ones in the LICENSE fileds to the SPDX values. Add a conversion
script which makes this straightforward on the most part.

Ultimately this allows the core code to be more efficient and not need
to consult the mappings.

(From OE-Core rev: 512cd4ca91bc5107d68b7e721257a8f62f878994)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-20 16:45:25 +00:00
Stefan Herbrechtsmeier
0c20edddfc wic: rawcopy: Add support for packed images
Add support for packed images to wic rawcopy handler do minimize disk
usage in deploy directory and reuse of packed images between wic and
swupdate. Add `unpack` to sourceparams to unpack an bz2, gz and xz
archives.

Example:
part / --source rawcopy --sourceparams="file=core-image-minimal-qemu.ext4.gz,unpack"

(From OE-Core rev: 4c97d25791389ece041565981ba3207ce9949a1a)

Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-16 09:46:29 +00:00
Stefan Herbrechtsmeier
5d16ba9078 wic: partition: Support valueless keys in sourceparams
Accept valueless keys in sourceparams without equals sign (=) to match
the comment and support Boolean entries.

(From OE-Core rev: 052a9b0f7d2ee5c284cf1b4f1f45caed0489221a)

Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-16 09:46:29 +00:00
Richard Purdie
8fcf25bcda scriptutils: Fix handling of srcuri urls
Bitbake is now requiring there be a call to get_srcrev() when urls contain
floating revisions. This breaks some recipetool workflows since it is running
these queries to determine those revisions. Add a PV entry which triggers
that call to avoid the errors.

(From OE-Core rev: bdaf0c798da21f9c15d9cb30ab4258a3c75c8bbf)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-12 17:05:35 +00:00
Alexander Kanavin
0f82f85dc9 runqemu: preload uninative libraries when host gl drivers are in use
Some of the host distributions build the drivers in a way (RPATH/RUNPATH)
that tricks uninative loader into loading pieces of the host libc, if
the same pieces haven't been previously loaded by native binaries. Mixing
the two libc versions leads to failures.

This change ensures that the correct (uninative) versions are always in use.

(From OE-Core rev: 39c10816d5ec9d9c7952d786d7a3f942d25d0c27)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-07 10:08:59 +00:00
Alexander Kanavin
276975e4ea qemu: replace a gtk wrapper with directly setting environment from runqemu
The wrapper is executed by host bash, and host bash refuses to run
when pseudo libc is preloaded via LD_PRELOAD (which is the case
when gl is enabled).

Only the fontconfig setting is carried over as local testing showed that
only that is necessary for the gui to look ok nowadays; adjust further
if necessary.

(From OE-Core rev: 34f152de14f803fcfe5c92c515bf585838bba10a)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-07 10:08:59 +00:00
Florian Amstutz
555f9166bf devtool: deploy-target: Remove stripped binaries in pseudo context
deploy-target may fail the second time with "pseudo abort" because
devtool-deploy-target-stripped is deleted outside of pseudo's fakeroot
context.

(From OE-Core rev: 2338a33b690b0bbe279cde3f73764911b239cb50)

Signed-off-by: Florian Amstutz <florian.amstutz@scs.ch>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-07 10:08:59 +00:00
Ross Burton
8463a37d70 yocto-check-layer: check for duplicate layers when finding layers
detect_layers() is very greedy and if it recurses into poky or bitbake
it will find the test suite layers, such as
bitbake/lib/layerindexlib/tests/testdata/layer4. This is a dummy layer
which claims to be openembedded-layer, so if the real openembedded-layer
is a dependency then layer4 may be used instead, which will cause
errors: initially because it's only compatible with Sumo, but later
because it doesn't contain any recipes.

Add a check that the set of layers we've found doesn't contain any
duplicate collection names with different patterns, and abort if that is
the case as the test will be non-deterministic.

(From OE-Core rev: 0df4bae4ec67d38442620fa08c839528b425e2a8)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-05 17:46:05 +00:00
Daniel Müller
2b3ae4d1ab scripts/runqemu-ifdown: Don't treat the last iptables command as special
The runqemu-ifup script performs a bunch of setup steps that
runqemu-ifdown attempts to undo later on. While a bunch of said setup
operations are considered fatal should they fail, the iptables based NAT
setup notably is not.
The tear down procedure in runqemu-ifdown, however, has the iptables
based tear down as the last operation, with the status of it determining
the overall status of the script. Hence, if this step fails, the script
is considered a failure overall. That is arguably inconsistent: If the
NAT setup did not succeed, the tear down cannot succeed either.
To ensure similarity of the two paths, let's not treat the last iptables
tear down operation any special and allow it to fail the runqemu-ifdown
script, but just ignore failures.

Background: we have seen a NAT related setup problem on the ifup path
(which didn't cause script failure), but then saw an issue bubbled up
when this operation was meant to be undone on the ifdown path.

(From OE-Core rev: 0ebcfb034bcad81efef5f746f0aa0b69772901a0)

Signed-off-by: Daniel Müller <muellerd@fb.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-05 17:46:05 +00:00
Saul Wold
254fbcf614 recipetool: Fix circular reference in SRC_URI
When creating a new recipe.bb file for a binary, don't use BP which
includes the version information, instead use BPN which is just the
name base Package Name.

Since PB is not specified, it takes the default:
PV = "1.0+git${SRCPV}"

But SRCPV is defined in terms of the SRC_URI, which leads to infinite
recursion (traceback below). Here are the pertinent variables which
cause the recursion:

SRC_URI = "git://github.com/lvc/abi-dumper;protocol=https;subdir=${BP}"
BP = "${BPN}-${PV}"
PV = "1.0+git${SRCPV}"
SRCPV = "${@bb.fetch2.get_srcrev(d)}"

def get_srcrev(d, method_name='sortable_revision'):
    # ... trimmed
    scms = []
    fetcher = Fetch(d.getVar('SRC_URI').split(), d)
    # ... trimmed

[YOCTO #14040]

(From OE-Core rev: 3b8d43fc53ee13d39abc3b2a1f706a97fcf752aa)

Signed-off-by: Saul Wold <saul.wold@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-05 17:46:05 +00:00
Richard Purdie
a242274d98 prelink: Drop support for it
Prelink is being dropped by glibc in 2.36. It already causes issues with
binary corruption, has a number of open bugs and is of questionable benefit
without disabling load address randomization and PIE executables.

We disabled it by default a while back but left people able to use it.
We would be unable to maintain it alone without glibc support so remove
the remaining pieces.

(From OE-Core rev: 23c0be78106f1d1e2bb9c724174a1bb8c56c2469)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-05 12:26:36 +00:00
Peter Kjellerstedt
00bf15c118 devtool: sdk-update: Remove an unnecessary \n from SSTATE_MIRRORS
Since commit 044fb04d in bitbake (fetch2: Allow whitespace only mirror
entries) there is no need to separate the entries in SSTATE_MIRRORS
with "\n".

(From OE-Core rev: 6e66ec3cd201d32d3d97df2222f41bb8c475fb69)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-03 09:05:13 +00:00
Ross Burton
f2666fbc7f yocto-check-layer: add debug output for the layers that were found
When debugging weird yocto-check-layer output it is useful to know
what the tool found when looking for layers.

(From OE-Core rev: 711e2d4d7baf36f8497741c14268d7f72d0db016)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-02-01 07:31:17 +00:00
Tim Orling
fe86a14649 recipetool/create_buildsys_python.py: less distutils
'distutils' is deprecated in Python 3.10 with removal in Python 3.12
(~October 2023). Replace 'distutils.command.build_py' with
'setuptools.command.build_py'.

To avoid an AttributeError, we call super().__init__() which provides
the missing 'distribution' attribute. However, for some reason, __init__()
in 'setuptools.command.build_py.build_py' class requires a 'dist' positional
argument which must be a 'Distribution' instance. It is not clear why
'distutils.command.build_py.build_py' class does not require this.

There is still a check which decides to inherit setuptools3 vs distutils3
that will need to be refactored when we add pyproject.toml and setup.cfg
support for more modern PEP 517 packaging.

Once distutils3.bbclass is dropped, any recipe inheriting distutils3
will throw a parsing error. The plan is to move distutils*.bbclasses to
meta-python. However if meta-python is not in bblayers, the parsing
error would still occur.

[YOCTO #14610]

(From OE-Core rev: 619a3eb1266459daf16e10386113e9201fbf9cf5)

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-01-12 21:09:02 +00:00
Richard Purdie
f799d9a3b0 buildtools-installer: Update to use 3.4
This updates buildtools to use the tarball from the 3.4 release which
contains some bug fixes and is what the autobuilder currently uses for
testing on older distros.

(From OE-Core rev: 7479861c60a1c205b9502c1a811ac3a9dc51cd07)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-01-12 21:09:01 +00:00
Alexander Kanavin
2a3e849326 qemuboot/runqemu: fully form the ip= kernel parameter
New systemd is actually parsing this in systemd-network-generator
and fails if it is not fully formed. 'off' means 'static ip, do nothing':
https://www.kernel.org/doc/Documentation/filesystems/nfs/nfsroot.txt

(From OE-Core rev: 2cf12c8dde0f05917797f8b4a80883dc0647b95d)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-01-11 10:53:44 +00:00
Richard Purdie
fc2bc42eec scripts: Update to use exec_module() instead of load_module()
This is deprecated in python 3.12 and Fedora 35 is throwing warnings so
move to the new functions.

(From OE-Core rev: 655cd3f614d736416eab0d708b7c49674bf5c977)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-01-05 17:18:16 +00:00
Tim Orling
48e7c7cc4d scripts/buildhistory-diff: drop use of distutils
The use of distutils.version.LooseVersion to check for GitPython > 0.3.1
is not really needed anymore since any supported distribution has at least
1.0.0 (centos-7 via epel7, debian-9, ubuntu-16.04)

If we want to reinstate this check, alternatives would be to require
python3-packaging on all hosts and use packaging.version.Version or
use an imported LooseVersion in bb.version.

[YOCTO #14610]

(From OE-Core rev: bc90dcae9f53ddc246942f4d9b8ae8943e3b9754)

Signed-off-by: Tim Orling <timothy.t.orling@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-01-04 23:14:05 +00:00
Stefan Herbrechtsmeier
32f56de278 recipetool: Separate licenses with & operator
Separate licenses with & operator since it should be satisfied most use
cases and it is a reasonable assumption that all the licenses apply.
Furthermore flat, split and sort the licenses to minimize license string
changes.

Separate package licenses with & operator:
-LICENSE:${PN} = "MIT ISC"
+LICENSE:${PN} = "ISC & MIT"

Respect | and brackets in LICENSE:
-LICENSE = "BSD-3-Clause & (ISC & | & MIT)"
+LICENSE = "BSD-3-Clause & (ISC | MIT)"

Sort licenses:
-LICENSE = "MIT & BSD-3-Clause & ISC"
+LICENSE = "BSD-3-Clause & ISC & MIT"

Remove duplicates:
-LICENSE = "MIT & ISC & MIT"
+LICENSE = "ISC & MIT"

(From OE-Core rev: 60a84ecc53d20118c5e7f86dd3e3cafbfed1cf0a)

Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-20 15:29:01 +00:00
Stefan Herbrechtsmeier
2be34ea6ed recipetool: Sort output of guess_license function to be deterministic
Sort the output of guess_license function by license file to be
deterministic and to support self-testing.

(From OE-Core rev: ff2a7520813b3f775f96879d1905222e08b8c83f)

Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-20 15:29:01 +00:00
Alexander Kanavin
6616b85af4 runqemu: additional setting to force software rendering with sdl 2.0.18
(From OE-Core rev: 282e28bec1419d4e34076234027f34af3a49e7b5)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-17 09:56:15 +00:00
Stefan Herbrechtsmeier
392f2933b6 recipetool: Set master branch only as fallback
The commit 'meta/scripts: Manual git url branch additions (dc53fe75cc)'
sets the branch= parameter too early to master and thereby breaks the
-B/--srcbranch option.

ERROR: branch= parameter and -B/--srcbranch option cannot both be specified - use one or the other

(From OE-Core rev: 34ece8030e7a6a100b5e3e7b94e6c786c0e199a6)

Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-09 10:33:25 +00:00
Ross Burton
67700e174d runqemu: check the qemu PID has been set before kill()ing it
If runqemu is killed, check that we have a valid PID for the qemu before
sending a kill() to it.

[ YOCTO #14651 ]

(From OE-Core rev: 0f3afbd3a6a6bef668612f818517df7543c0a683)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-09 10:33:25 +00:00
Joshua Watt
467c21abe4 yocto-check-layer: Relax README case checks
Relaxes the case requirements for checks in the README file so that word
like "Maintainer" and "Patch" are allowed

(From OE-Core rev: fdedf94d406ad3da85cb45d43ef87d3fdc8c14d4)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-04 23:39:52 +00:00
Jon Mason
17d1c837a4 yocto-check-layer: disregard checks if referencing another README file
It is permissible to have a layer's README reference another README.  If
this is the case, avoid other checks.  Do this by checking for the word
README (case insensitive) in the README file.  This might be too
permissive, but follows the pattern of looking for exact words (like
"patch" or "maintainer") in READMEs.

(From OE-Core rev: 443c73d638519d6a7ea44d1c0e80d76306687ddc)

Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-04 23:39:52 +00:00
Dhruva Gole
aec629185e yocto-check-layer: Add additional README checks
This adds:
- Test case to review if the README file specifies the maintainer.
- Test case to review if the README file specifies the word patch.
- Test case to review if the README file contains an email address.

[YOCTO #11131]

(From OE-Core rev: 5d4937bce191b7e22b807c0595d4845c88ecc560)

Signed-off-by: Dhruva Gole <goledhruva@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-03 23:37:16 +00:00
Ross Burton
3cf97f84c6 recipetool: extend curl detection when creating recipes
If a configure.ac uses LIBCURL_CHECK_CONFIG it wants curl.

(From OE-Core rev: 16e830ca1352cee61e4ae4b98b1203f849bf71f3)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-03 23:37:16 +00:00
Ross Burton
ad10932343 recipetool: handle GitLab URLs like we do GitHub
GitHub URLs are automatically transformed to git: fetches, so handle
GitLab URLs too.

(From OE-Core rev: 651fb951819840fe122458ddbd852ee6c7ec0455)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-03 23:37:16 +00:00
Dhruva Gole
95de7e2b62 scripts/checklayer/common.py: Fixed a minor grammatical error
(From OE-Core rev: 8ea17456ae5318ed7a3b4c8f75c8441456d8b979)

Signed-off-by: Dhruva Gole <goledhruva@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-12-01 16:23:45 +00:00
Jacob Kroon
096c9037bc native/cross: Add ar wrapper for determinism
Add a wrapper around ar calls for native/cross recipes. This wrapper adds
the -D option so that deterministic archives are built for native/cross
output. This improves the changes of hash equivalence matches and hence
build artefact reuse.

We don't need this in the target case since we compile binutils-cross
with an option making this the default. We need a wrapper since we need
to remove the "u" option and replace it with "D" but also allow things like
"--version" to continue to work too.

(From OE-Core rev: 59922c95fcb20c66634c5677012d490be2246b0b)

Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-26 17:01:08 +00:00
Richard Purdie
8dd0a45c85 nativesdk: Handle chown/chgrp calls in nativesdk do_install tasks
We disable the useradd code for nativesdk targets since we don't support
postinstalls or multiple users in those cases. This means any usage
of chown/chgrp inside do_install tasks won't work and would have to be
conditional. Rather than require all recipes to do that, add intercepts
of the calls and map those to root/root user/groups. We can't just ignore
them as some calls are used to remove host contamination from the host
user ID so they need to be made, just as root.

(From OE-Core rev: e1f40670c438e33cae87678425de72ca03566888)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-26 17:01:08 +00:00
Schmidt, Adriaan
9d1697e51d wic: support rootdev identified by partition label
We already support specifying the rootfs by PARTUUID. This adds general
support for letting the kernel find the rootfs by PARTLABEL.

(From OE-Core rev: 2fb247c5ecf057bb96649a3c0234794b4991c050)

Signed-off-by: Adriaan Schmidt <adriaan.schmidt@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-24 09:55:54 +00:00
Mingli Yu
a3b12619ba wic: use shutil.which
Use shutil.which to find the executable instead to silence the below warning:
 $ cat tmp/work/intel_x86_64-poky-linux/core-image-base/1.0-r5/temp/log.do_image_wic
 [snip]
 DEBUG: Executing shell function do_image_wic
 /path/layers/oe-core/scripts/wic:27: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
  from distutils import spawn
 INFO: Creating image(s)...
 [snip]

[RP: Added conversion for missed function reference]
(From OE-Core rev: 3966cbf5c8a2dbc3a4f0f3eefdbeeb83f522bf87)

Signed-off-by: Mingli Yu <mingli.yu@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-21 11:05:02 +00:00
Ross Burton
7320b2b830 scripts/lnr: remove
lnr is a script that implements the same behaviour as 'ln --relative
--symlink', as at the time of creation --relative was only available in
coreutils 8.16 onwards which was too new for the older supported distros.

Now, however, everyone has a new enough coreutils, so we can remove this
script.

All users of lnr should be replaced with ln --relative --symbolic.

(From OE-Core rev: 723b6e40f5943426364bffce7c58ade65c4abbba)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-21 11:05:01 +00:00
Adrian Freihofer
9c3e6fd9b6 runqemu: support rootfs mounted ro
Optionally allow to set QB_KERNEL_ROOT to e.g. "/dev/vda ro" to mount the
rootfs reay-only in Qemu.

(From OE-Core rev: 448eb1fd21287ba16b17e9402ce040b86ae3638c)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-13 16:11:28 +00:00
Jon Mason
045bc7682c scripts/lib/wic/help.py: Update Fedora Kickstart URLs
The URLs describing Kickstart are no longer valid and do not redirect to
the correct location.  Update them with the correct location.

(From OE-Core rev: e5ac75f93c8128b0761af5fee99e8603ddd1657d)

Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-11 22:26:46 +00:00
Richard Purdie
50b85e4855 scripts/oe-package-browser: Handle no packages being built
Give the user a proper error message if there aren't packages built,
rather than a less friendly traceback.

[YOCTO #14619]

(From OE-Core rev: b14c176b7dd74b7d63ca0f72e6e00fbf209f5a0b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-11 17:09:17 +00:00
Richard Purdie
e21c967bd4 scripts/oe-package-browser: Fix after overrides change
After the overrides change, the format of pkgdata changed and this
usage of configparser no longer works. This change is a bandaid to make
things work but the pkgdata format isn't very similar to ini files
so this may need to be reimplmented in a better way in the long run.

[YOCTO #14619]

(From OE-Core rev: 25a8ec6e2891b71bc280aacaf5f62ecc4b0bd1d1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-11 17:09:17 +00:00
Kai Kang
0a0052e456 convert-srcuri.py: use regex to check space in SRC_URI
There may be none, one or more spaces including tab before backslash in
SRC_URI. Use regex to check and update. It helps to avoid malformed uri
such as recipe open-iscsi-user in meta-openstack:

SRC_URI = "git://github.com/open-iscsi/open-iscsi.git;protocol=https  ;branch=master \

And help to check more recipes such as concurrent-ruby in the same
layer:

SRC_URI = "git://github.com/ruby-concurrency/concurrent-ruby.git;protocol=https;tag=v1.1.6\

(From OE-Core rev: a69a53573b1987ee5834a6fc27763f9bbf5fe5a4)

Signed-off-by: Kai Kang <kai.kang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-11 17:09:17 +00:00
Khem Raj
c44131cf61 wic: Use os.rename instead of bb.utils.rename
This is not invoked with bitbake context as a result bb.utils is not
visible when this function is called during image creation and builds
fail e.g.

NameError: name 'bb' is not defined

(From OE-Core rev: df9dca9fe4dd1abfe5f3986389a8e8ff524da5d7)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Cc: Joshua Watt <JPEWhacker@gmail.com>
Cc: Devendra Tewari <devendra.tewari@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-10 19:27:28 +00:00
Jon Mason
859776220f runqemu: work without SERIAL_CONSOLES being defined
Not all machine definitions need to have SERIAL_CONSOLES defined, but
runqemu currently will fail with the following script error if not
present:

Traceback (most recent call last):
  File "/builder/meta-arm/poky/scripts/runqemu", line 1604, in main
    config.setup_final()
  File "/builder/meta-arm/poky/scripts/runqemu", line 1446, in setup_final
    self.setup_serial()
  File "/builder/meta-arm/poky/scripts/runqemu", line 1381, in setup_serial
    self.kernel_cmdline_script += ' console=%s' %entry.split(';')[1]
IndexError: list index out of range

To get around this issue, add a sanity check to runqemu to avoid the
parsing of SERIAL_CONSOLES if empty.

(From OE-Core rev: e8359000e765f876c041b6ccabe63f069f83efec)

Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-03 10:12:42 +00:00
Richard Purdie
e177e03320 scripts/convert-srcuri: Update SRC_URI conversion script to handle github url changes
Github are dropping support for git:// protocol fetching. Update the script
to learn about corner cases found in the previous conversion and
support remapping the github urls as needed too.

(From OE-Core rev: e59fe8279b209f67ff79b9d6dbb69389a64db236)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-11-03 10:12:42 +00:00
Richard Purdie
dc53fe75cc meta/scripts: Manual git url branch additions
Following the scripted conversion adding branches to git://
SRC_URI entries, add the remaining references, mainly in the selftests
and recipetool.

(From OE-Core rev: 5340c0d688036c1be6c938f05d8a8c1e3b49ec38)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-30 18:56:47 +01:00
Richard Purdie
ddcf16d1f7 meta: Add explict branch to git SRC_URIs
There is uncertainty about the default branch name in git going forward.
To try and cover the different possible outcomes, add branch names to all
git:// and gitsm:// SRC_URI entries.

This update was made with the script added to contrib in this patch which
aims to help others convert other layers.

(From OE-Core rev: b51c405faf6f8c0365f7533bfaf470d79152a463)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-30 18:56:47 +01:00
Alexander Kanavin
05ce40b2ce runqemu: unbreak non-gl displays
Correct two issues:
1. Looking for dri.pc is only needed when gl is enabled.
2. virtio-vga-gl works only when gl is enabled via -display,
otherwise virtio-vga needs to be selected.

(From OE-Core rev: d6d5fe44510d2087b735758ac85e804533a1778c)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-30 18:56:47 +01:00
Stefan Herbrechtsmeier
7ab5afa819 recipetool: Simplify common source files skip in guess_license
(From OE-Core rev: 189c0297632968900715d7a4a5edcdf3a56e25f5)

Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-23 17:42:28 +01:00
Richard Purdie
b85fd33e7d sstate: Fixes for eSDK generation after zstd switch
(From OE-Core rev: f5a90df08631117cd66bec68cb1851db96822bb8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-15 12:58:00 +01:00
Michael Halstead
9a7bc68135 scripts/autobuilder-worker-prereq-tests: jinja2 check
Ensure the jinja2 module is available during bringup.

(From OE-Core rev: c46a6ec91bd40a458cb0ef5ec84bc0cc274d9cef)

Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14 22:39:50 +01:00
Henry Kleynhans
3579782a44 sstate: Switch to ZStandard compressor support
This patch switches the compressor from Gzip to ZStandard for ssate cache
files.

Zstandard compression provides a significant improvement in
decompression speed as well as improvement in compression speed and disk
usage over the 'tgz' format in use.  Furthermore, its configurable
compression level offers a trade-off between time spent compressing
sstate cache files and disk space used by those files.  The reduced disk
usage also contributes to saving network traffic for those sharing their
sstate cache with others.

Zstandard should therefore be a good choice when:
* disk space is at a premium
* network speed / resources are limited
* the CI server can sstate packages can be created at high compression
* less CPU on the build server should be used for sstate decompression

(From OE-Core rev: 0710e98f40ec7394567ede1f88f7c7bae306b925)

Signed-off-by: Henry Kleynhans <hkleynhans@fb.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14 22:37:43 +01:00
Jon Mason
47d033595a runqemu: add DEPLOY_DIR_IMAGE replacement in QB_OPT_APPEND
Add the ability to replace DEPLOY_DIR_IMAGE with that path in
QB_OPT_APPEND.  This allows for anything present in DEPLOY_DIR_IMAGE to
be passed into the qemu parameters.  This is especially useful if you
want to run multiple flash images (as -bios only allows for one).

(From OE-Core rev: 1f0bec0421617e8aa9645c385195a755f0d44e75)

Signed-off-by: Jon Mason <jdmason@kudzu.us>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14 22:32:50 +01:00
Stefan Herbrechtsmeier
0e973d814b recipetool: Rework crunch_license to recognize more variants
(From OE-Core rev: 6c70f5bd2acd036c21be5b03df89d3a8f381a822)

Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-14 11:48:45 +01:00