Commit Graph

3675 Commits

Author SHA1 Message Date
Julien Stephan
86b9510bd3 recipetool: create: add trailing newlines
create_recipe() function relies on oe.recipeutils.patch_recipe_lines()
which relies on bb.utils.edit_metadata(). edit_metada expect lines to
have trailing newlines, so add it to each lines before calling
patch_recipe_lines, otherwise edit_metadata will not be able to squash
blank line if there are two consecutive blanks after a removal

(From OE-Core rev: 31b27cc76a1b669e2b126c332608fd772c124d69)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Julien Stephan
344e10a21b recipetool: create_buildsys_python.py: initialize metadata
In the case pyproject.toml doesn't contains metadatas, the metadata
variable is not initialized and the plugin throws an error and falls back
to another plugin, which is not the desired behaviour. So just ignore
metadata if we don't have them

(From OE-Core rev: 88d15877ba18309c521740d7a9649e14d77189bc)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-06 22:55:49 +00:00
Kareem Zarka
0a39f1aa09 wic: bootimg-efi: Make kernel image installation configurable
The issue with installing the kernel image to both rootfs
and boot partition is that some systems rely on the kernel image in
rootfs and not in the boot partition.
This leads to duplication of the kernel image, which can cause
unnecessary storage usage.
This patch provides a solution to the problem by adding a new
parameter "install-kernel-into-boot-dir" to the wic kickstart file.
If this parameter is set to 'true', the plugin will install the
kernel image to the boot partition. If the parameter is set to
'false', the plugin will skip installing the kernel image, avoiding
duplication.

(From OE-Core rev: d3599afe5f604ea5afd9411e114934dcb52b2d48)

Signed-off-by: Kareem Zarka <kareem.zarka@huawei.com>
Signed-off-by: Christian Taedcke <christian.taedcke@weidmueller.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-04 11:23:37 +00:00
Julien Stephan
89f1662484 devtool: add support for git submodules
Adding the support of submodules required a lot of changes on the
internal data structures:
* initial_rev/startcommit used as a starting point for looking at new
  / updated commits was replaced by a dictionary where the keys are the
  submodule name ("." for main repo) and the values are the
  initial_rev/startcommit

* the extractPatches function now extracts patch for the main repo and
  for all submodules and stores them in a hierarchical way describing the
    submodule path

* store initial_rev/commit also for all submodules inside the recipe
  bbappend file

* _export_patches now returns dictionaries that contains the 'patchdir'
  parameter (if any). This parameter is used to add the correct
  'patchdir=' parameter on the recipe

Also, recipe can extract a secondary git tree inside the workdir.

By default, at the end of the do_patch function, there is a hook in
devtool that commits everything that was modified to have a clean
repository. It uses the command: "git add .; git commit ..."

The issue here is that, it adds the secondary git tree as a submodule
but in a wrong way. Doing "git add <git dir>" declares a submodule but do
not adds a url associated to it, and all following "git submodule foreach"
commands will fail.

So detect that a git tree was extracted inside S and correctly add it
using "git submodule add <url> <path>", so that it will be considered as a
regular git submodule

(From OE-Core rev: 900129cbdf25297a42ab5dbd02d1adbea405c935)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-01 11:48:25 +00:00
Julien Stephan
17427db136 devtool: tag all submodules
In the case of a repository with submodules, we need to add the
"devtool-base" and "devtool-patched" tag on all submodules in order to
properly detect the added/removed/modified patches

(From OE-Core rev: 241da68805d177d4ec4b302c8a997645cc645286)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-01 11:48:25 +00:00
Julien Stephan
1ae4cc7a11 devtool: finish/update-recipe: restrict mode srcrev to recipes fetched from SCM
When specifying --mode / -m srcrev with devtool finish/update-recipe on
recipes that are not fetched from a SCM repository we get the following
error:

  Traceback (most recent call last):
  [..]
    File "<...>/poky/meta/lib/oe/patch.py", line 49, in runcmd
    raise CmdError(cmd, exitstatus >> 8, "stdout: %s\nstderr: %s" % (stdout, stderr))
oe.patch.CmdError: Command Error: 'sh -c 'git format-patch --no-signature --no-numbered INVALID -o /tmp/oepatchbj7pfmzj -- .'' exited with 0  Output:
stdout:
stderr: fatal: bad revision 'INVALID'

Fix this by adding a check and abort with a proper error message.

(From OE-Core rev: 5b77f60e85d07921ae5e808daa1fd7d8c3dc40ea)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-01 11:48:25 +00:00
Julien Stephan
fcff067407 devtool: fix update-recipe dry-run mode
When running devtool update-recipe with --mode=srcrev AND --append switch
in dry-run, we get the following error:

  Traceback (most recent call last):
  [...]
  Exception: destpath should be set here

Fix this by removing a misplaced else statement in _update_recipe_srcrev

(From OE-Core rev: 85ba125703d5b442133fd7c470b915460ee68ac9)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-30 08:43:04 +00:00
Malte Schmidt
5d12c0a3dc wic: rawcopy: add support for zstd decompression
Add support for zstd decompression in rawcopy plugin. zstd claims
to reach higher, uniform decompression rates.

(From OE-Core rev: 707471b71a417187f1e581d0c540e058825f7242)

Signed-off-by: Malte Schmidt <malte.schmidt@weidmueller.com>
Signed-off-by: Lukas Funke <lukas.funke@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-27 22:54:59 +00:00
Lukas Funke
7de36b25d1 recipetool: Add handler to create go recipes
[RP: Replace python 3.9 use of removeprefix()]
(From OE-Core rev: 049afb933f42851d56d2e6522150a32bcccf2a6f)

Signed-off-by: Lukas Funke <lukas.funke@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-06 16:41:11 +00:00
Lukas Funke
f7f304fa3e recipetool: Ignore *.go files while scanning for licenses
(From OE-Core rev: 1765acc8bdc08e392daa8b77add8b4ce1a9e70b6)

Signed-off-by: Lukas Funke <lukas.funke@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-06 16:41:11 +00:00
Alexis Lothoré
838b8e78dc scripts/resulttool: group all regressions in regression report
Commit c304fcbe05 introduced a grouping when
listing regressions. This grouping has been added only for ptests. It has
been observed that any other kind of tests could benefit from it. For
example, current regression reports can show the following:

1 regression(s) for oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash
    oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash: PASSED -> SKIPPED
1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_help
    oescripts.OEPybootchartguyTests.test_pybootchartguy_help: PASSED -> SKIPPED
1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_pdf_output
    oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_pdf_output: PASSED -> SKIPPED
1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_png_output
    oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_png_output: PASSED -> SKIPPED
1 regression(s) for oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_svg_output
    oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_svg_output: PASSED -> SKIPPED
[...]

This output is not so useful in its current state and should be grouped per
test type too.
Enable grouping for all kind of tests, to make it llok like the following
in reports:
5 regression(s) for oescripts
    oescripts.OEGitproxyTests.test_oegitproxy_proxy_dash: PASSED -> SKIPPED
    oescripts.OEPybootchartguyTests.test_pybootchartguy_help: PASSED -> SKIPPED
    oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_pdf_output: PASSED -> SKIPPED
    oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_png_output: PASSED -> SKIPPED
    oescripts.OEPybootchartguyTests.test_pybootchartguy_to_generate_build_svg_output: PASSED -> SKIPPED

(From OE-Core rev: 982798ef96e3a32bf15341bdd3bb7c4356709412)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-05 08:44:06 +00:00
Alexander Kanavin
81a400a8f5 scripts/esdk-tools: use a dedicated, static directory for esdk tools
This allows easier replication of esdk environment (which provides
a curated, limited set of tools that for example does not include bitbake)
in a standard yocto build. Switchover between various sets can be achieved
via PATH manipulation.

(From OE-Core rev: 20c548f2edca3888152adb63de7b23d84e3848e7)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-05 08:44:06 +00:00
Trevor Gamblin
b53d1287d2 patchtest-send-results: fix sender parsing
Not all mbox 'from' fields will contain angle brackets, so the
re.findall invocation used for getting a reply_address may fail. Use a
simpler reference to the field to get the sender's email address.

(From OE-Core rev: 86e9afe09a346586114133f5a7470304d2ed733f)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-02 21:48:30 +00:00
Trevor Gamblin
2bddb6bb97 patchtest-send-results: add In-Reply-To
Rework the script for sending results to use send_raw_email and specify
the 'In-Reply-To' field so that patchtest replies to the emails, rather
than sending them standalone to the submitter and mailing list.

(From OE-Core rev: 0c45c92e7f26aea4edf2cfa577b7ba51384e59d3)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-11-01 19:19:40 +00:00
Trevor Gamblin
a88b6d2452 patchtest-send-results: send results to submitter
Modify patchtest-send-results so that it extracts the submitter's email
address and responds to them with the patch testresults. Also make a
minor adjustment to the suggestions provided with each email and include
a link to the Patchtest wiki page for additional clarification on
specific failures.

(From OE-Core rev: 64ed88e32cf9e04772319ff6e66c602d1cff4fd7)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-30 16:38:15 +00:00
Ross Burton
28324600cb scripts/contrib/patchreview: fix commit identification
git show-ref looks at the _remote_ ref called HEAD, which is fine when it
matches the local HEAD but problematic when you're iterating a series of
commits.

Use rev-parse to resolve the local name to a proper hash.

(From OE-Core rev: 3c04747b681cf6090ba9c77752f6c2f304dbbe17)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-30 11:53:20 +00:00
Tim Orling
9ffac3fc3d recipetool: add python_hatchling support
One of the newer PEP-517 backends to be added was python_hatchling.bbclass
but it was not included in the recent improvements.

Add selftest for 'jsonschema' pypi package.

(From OE-Core rev: d99b4883b4fee82bc588fd235ba90fedf1550cb8)

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-30 08:50:46 +00:00
Ross Burton
2cbbe428db scripts/contrib/patchreview: consolidate imports
Move most imports to the top of the file.

(From OE-Core rev: d2c287db0739b249604cd1beaa03ec38512ba718)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 17:48:11 +01:00
Ross Burton
62c80e3a79 scripts/contrib/patchreview: add commit and recipe count fields to JSON
The autobuilder scripts post-process the generated JSON to inject recipe
and commit counts into the data.  We can do this easily in patchreview
instead.

(From OE-Core rev: 77c96e43090cbf485aec612cc2315b85e5635dda)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 17:48:11 +01:00
Ross Burton
116c044212 scripts/patchreview: rework patch detection
A previous patch[1] added the ability to allow the search pattern for
patches to be changed, so that patchreview can be used across the entire
meta-oe repository by changing the patterns.

However, this means the caller needs to write long patterns when calling
patchreview.

Instead, we can see if the specified directory contains a layer by
checking if conf/layer.conf exists.  If it does, then search for patches
inside this directory.  If it doesn't, assume that the specified
directory is a repository that contains sublayers (such as
meta-openembedded) and look through each of the directories that match
the pattern meta-*.

This means patchreview can both scan either a single layer (eg
.../poky/meta) or a repository of sublayers (eg .../meta-openembedded).

[1] oe-core 599046ea9302af0cf856d3fcd827f6a2be75b7e1

(From OE-Core rev: a3a868519beab1b9cac94fefd7dbeffb09d047e9)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 17:48:11 +01:00
Alexander Kanavin
2cf13f2513 lib/oe/sstatesig.py: dump locked.sigs.inc only when explicitly asked via -S lockedsigs
This was writing out locked-sigs.inc into cwd with every
'bitbake -S' invocation. When the intent is only to to get task
stamps (-S none), or print the difference between them (-S printdiff),
the file is unnecessary clutter.

A couple of selftests/scripts were however relying on this, so they're
adjusted to explicitly request the file.

eSDK code calls dump_lockedsigs() separately via
oe.copy_buildsystem.generate_locked_sigs() and so isn't affected.

(From OE-Core rev: ad57c3cac2a8d3e60222e3cca0685f582dcea135)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 10:53:43 +01:00
Julien Stephan
d4debbf5b7 recipetool/create_buildsys_python: add PEP517 support
Add basic support for PEP517 [1] for the 3 following backends that are
supported by bitbake:
  - setuptools.build_meta
  - poetry.core.masonry.api
  - flit_core.buildapi

If a pyproject.toml file is found, use it to create the recipe,
otherwise fallback to the old setup.py method.

Some projects can declare a minimal pyproject.toml file, and put all
the metadata in setup.py/setup.cfg/requirements.txt .. theses cases
are not handled. If a pyproject.toml file is found, assumes it has all
necessary metadata.

As for the old setup.py method, version numbers for dependencies are not
handled.

Some features may be missing, such as the extra dependencies.

[YOCTO #14737]

[1]: https://peps.python.org/pep-0517/

(From OE-Core rev: c7d8d15b2d0a9ecd210bd247fa0df31d9f458873)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 08:28:38 +01:00
Julien Stephan
e64e92f2de recipetool/create_buildsys_python: refactor code for futur PEP517 addition
In order to prepare the support for pyproject.toml (PEP517 [1]) enabled
projects, refactor the code and move setup.py specific code into a
specific class in order to allow sharing the PythonRecipeHandler class

No functionnal changes expected

[1]: https://peps.python.org/pep-0517/#source-tree

(From OE-Core rev: 2281e93347da4129062cfb40710df03c87c63168)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 08:28:38 +01:00
Julien Stephan
be129bd0bc recipetool/create_buildsys_python: prefix created recipes with python3-
By convention, all python recipes start with "python3-" so update
create_buildsys_python to do this

This rule doesn't apply for packages already starting with "python"

Update recipetool's selftest accordingly

(From OE-Core rev: b0d87440e610b80f763d09784d4a90a148bb3e7b)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 08:28:38 +01:00
Julien Stephan
0021e3573a recipetool/create_buildsys_python: fix license note
License field of setup is not always standardized, so we usually use the
classifier to determine the correct license format to use in the recipe.

A warning note is added above the LICENSE field of the create recipe
in case a license is provided in setup. But when the plugin is called,
"LICENSE =" is not yet present so we can never display this note.
Replace the "LICENSE =" condition with "##LICENSE_PLACEHOLDER##"
to actually be able to display the note message

(From OE-Core rev: b7c26ca2028aa60f740464de85a11a01a531f32e)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-27 08:28:38 +01:00
Trevor Gamblin
d1c21e9eea patchtest-send-results: check max line length, simplify responses
Check that the maximum line length of the testresult file is less than
220 characters, to help guard against malicious changes being sent in
email responses. If any line exceeds this length, replace the normal
testresults used in the response with a line stating that tests failed,
but the results could not be processed. Also clean up the respone
substrings slightly to go along with the change.

(From OE-Core rev: b0d53cf587dc9afb97f00c1089e45b758e96dd7c)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-25 15:52:13 +01:00
Trevor Gamblin
e66dcd89b4 patchtest-send-results: improve subject line
Pull the actual email's subject line from the .mbox file and use that in
patchtest's test results response, so that it's clearer which patch it
is replying to.

(From OE-Core rev: 98ca0b151517b3544454fd5c1656a2de631c4897)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-24 14:21:37 +01:00
Alexis Lothoré
ea09071364 scripts/yocto_testresults_query: add option to change display limit
Add a "-l"/"--limit" option to allow changing the display limit in
resulttool.
- If no value is passed, resulttool uses its default value.
- If 0 is passed, the display limit is removed and every regression will be
  displayed
- If a custom value is passed, this value overrides the vlaue configured in
  resulttool

(From OE-Core rev: d3f536b3fc3f7027f6f5cf8bdaf5d7c050c7974b)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-23 10:49:19 +01:00
Alexis Lothoré
198110b1b9 scripts/resulttool: make additional info more compact
Since "matched" and "improved" tests are not as important as regressions,
reduce the place they take in the regression report:

- merge "matched" and "improved" tests, while removing the label
- add a single line of additional info per pair

Those changes make the "Matches and improvements" look like the following
sample:

oeselftest_almalinux-9.2_qemux86-64_20230910083156
oeselftest_almalinux-8.8_qemux86-64_20231018010951
    -> +7 test(s) present

oeselftest_almalinux-9.2_qemux86-64_20230911010538
oeselftest_debian-11_qemux86-64_20231017150459

oeselftest_debian-11_qemux86-64_20230910012927
oeselftest_debian-11_qemux86-64_20231017151319
    -> +7 test(s) present
[...]

(From OE-Core rev: 6de4426d9a7da67deed7d3a3918892fb56238ff3)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-23 10:49:19 +01:00
Alexis Lothoré
faef63eab6 scripts/resulttool: rearrange regressions report order
Regressions reports currently reports matching pairs and improved pairs
first, then regressions.

Change order to print regressions first, which is the most valuable info in
the report, and then print improvements and matches at the bottom.

(From OE-Core rev: 599267467430e70fa4dc8ba6b2a8b126bf6da359)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-23 10:49:19 +01:00
Alexis Lothoré
c304fcbe05 scripts/resulttool: limit the number of changes displayed per test
Most of the changes list generated in regression reports fall in one
of the two following categories:
- there is only a few (<10) changes listed and the info is
  valuable/relevant
- the list is huge (> 100 ? 1000 ?) and basically tells us that the whole
  tests category suffers the same status (test missing, test failing, test
  skipped, etc)

Prevent those huge, worthless lists by limiting the output for each test
group:
- current default limit is arbitrarily set to 50
- limit can still be overriden with a new "-l"/"--limit" flag, either with
  custom value, or with 0 to print the whole lists of changes
- limit is applied per test family: currently it distinguishes only types
  of ptests, but it can be adapted to other kind of tests

(From OE-Core rev: cec118406f3ad81cb4709f6e6ae1cef65799658e)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-23 10:49:19 +01:00
Logan Gunthorpe
91acd29313 runqemu: Add squashfs filesystem types
When using a squashfs filesystem type, runqemu requires specifying the
full path to the image because it doesn't list squashfs types in its
fstypes variable. Add them to provide the same support as other
filesystem types.

(From OE-Core rev: c9c9a077e85b56f495f09187483548149f142a8d)

Signed-off-by: Logan Gunthorpe <logang@deltatee.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-23 10:49:19 +01:00
Peter Kjellerstedt
cec84a072d bb-matrix-plot.sh: Show underscores correctly in labels
Underscores previously caused the next character in the label to be
printed using subscript due to the enhanced string support in gnuplot.

(From OE-Core rev: 282b48f90f77e0766993018d22fe03dd303febdc)

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-23 10:49:19 +01:00
Ross Burton
3fbd82e194 patchtest: sort when reading patches from a directory
When reading patches from a directory it's important to sort the output
of os.listdir(), as that returns the files in an effectively random
order.  We can't test the patches apply if they're applied in the wrong
order, and typically patch filenames are prefixed with a counter to
ensure the order is correct.

(From OE-Core rev: 4d6b586d37ab4528ed6dae6779cd730af9ef09c2)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-19 14:53:45 +01:00
Ross Burton
4cea51a9ad patchtest: remove unused imports
(From OE-Core rev: be8429d986335aae65c2426862b97836ba46e42a)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-19 14:53:45 +01:00
Trevor Gamblin
4ba54e29ff patchtest: check for untracked changes
[YOCTO #15243]

Avoid overwriting local changes when running patchtest by checking for
anything unstaged or uncommitted in the target repo, and logging an
error if something is found. This will provide the user helpful feedback
if (for example) they forgot to commit a change for their patch under
test, and will leave the target repository in a reasonable state (rather
than a temporary branch created by patchtest).

(From OE-Core rev: 2cc2dda6c3d5a97173b3fc434cc16ff2a930f43f)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-19 14:53:45 +01:00
Alexander Kanavin
15b08b8234 scripts/bitbake-whatchanged: remove
This is not documented or tested, and indeed hasn't been producing useful reports
for some time.

The script works by redirecting STAMPS_DIR into a separate location,
then running bitbake -S none, then comparing the two sets of stamp
filenames with regexes:

 Match the stamp's filename
 group(1): PE_PV (may no PE)
 group(2): PR
 group(3): TASK
 group(4): HASH
stamp_re = re.compile("(?P<pv>.*)-(?P<pr>r\d+)\.(?P<task>do_\w+)\.(?P<hash>[^\.]*)")

Then there's some code that finds out what changed in the above between the two sets.

Messing about with STAMPS_DIR like that isn't supported, and will either do nothing,
or remove the original stamps. Also stamp filenames aren't really a 'public API'.

For finding out the changes between two builds, 'bitbake -s printdiff' is a supported
and tested option. It may be a bit too verbose, but that can be more easily fixed than
rewriting bitbake-whatchanged into a working state.

(From OE-Core rev: f8193978eb0944e693e6a5cfbf9035e104e489f0)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-19 13:26:05 +01:00
Trevor Gamblin
b5a87b5f98 patchtest: update SPDX identifiers
Replace full license headers with SPDX identifiers and adjust all
patchtest-related code to use GPL-2.0-only.

(From OE-Core rev: 9bea6b39074296bb8d8719a3300636e316f19d1b)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-17 22:53:30 +01:00
Trevor Gamblin
26e3d58e71 patchtest: set default repo and testdir targets
Since patchtest is in oe-core, the Python os module's methods can be
used to retrieve the repository path and tests directory by default.
This reduces the number of mandatory arguments for invocation of
patchtest unless the user wants to use a custom test suite or test
patches against a different repo. The REPO and TESTDIR arguments are
likewise adjusted so that they are optional. Also, make it more obvious
what the --startdir flag is meant for on the command line by renaming it
to --testdir, and update the scripts/patchtest.README file to be
consistent with the new usage.

(From OE-Core rev: bae7421ece4806f5148f164293810b9fe75e0756)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-17 11:41:34 +01:00
Trevor Gamblin
6e53a778f1 patchtest: add scripts to oe-core
Add the following from the patchtest repo:

- patchtest: core patch testing tool
- patchtest-get-branch: determine the target branch of a patch
- patchtest-get-series: pull patch series from Patchwork
- patchtest-send-results: send test results to selected mailing list
- patchtest-setup-sharedir: create sharedir for use with patchtest guest
  mode
- patchtest.README: instructions for using patchtest based on the README
  in the original repository

Note that the patchtest script was modified slightly from the repo
version to retain compatibility with the oe-core changes.
patchtest-send-results and patchtest-setup-sharedir are also primarily
intended for automated testing in guest mode, but are added for
consistency.

(From OE-Core rev: cf318c3c05fc050b8c838c04f28797325c569c5c)

Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-17 11:41:34 +01:00
Mickael RAMILISON
9c7148318f scripts/patchreview: Add a custom pattern for finding recipe patches
This introduces support for specifying a search pattern with the -p/--pattern
option in the patchreview.py script. This is designed to accommodate
the directory structure of meta-openembedded.

(From OE-Core rev: 599046ea9302af0cf856d3fcd827f6a2be75b7e1)

Signed-off-by: Mickael RAMILISON <mickael.ramilison@smile.fr>
Reviewed-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-15 09:11:47 +01:00
André Draszik
c2e094ddfd wic: fix a typo in help output
(From OE-Core rev: 1c35beb935d42e200370b54f9aa3848b940966dd)

Signed-off-by: André Draszik <git@andred.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-10 12:56:39 +01:00
Richard Purdie
44b4c74cad runqemu/qemurunner: Use nodelay with tcp serial connections
This disables Nagle's algorithm for our tcp serial connections which may
be causing data transfer issues.

(From OE-Core rev: f8eff4c427881a98333fdf7c42f66ed6603e4f03)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-10 09:53:32 +01:00
Julien Stephan
91db19fc36 scripts/bblock: add a script to lock/unlock recipes
bblock script allows to lock/unlock recipes to latest task signatures.
The idea is to prevent some recipes to be rebuilt during development.
For example when working on rust recipe, one may not want rust-native to be
rebuilt.

This tool can be used, with proper environment set up, using the following
command:

bblock <recipe_name>

See help for more details

if a <recipe_name>'s task signature change, this task will not be built again and
sstate cache will be used.

[YOCTO #13425]

(From OE-Core rev: 2d9ab0cfd7f3cacc347954676f1323342a6b286f)

Signed-off-by: Julien Stephan <jstephan@baylibre.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-09 15:58:47 +01:00
Marcus Flyckt
561c63e947 devtool/upgrade: check all git config locations
Fixes [YOCTO #15222]

In some situations its more practical to keep git configuration
at `/etc/gitconfig` instead of `$HOME/.gitconfig` (e.g., when mounting
git configuration into a docker container).

This change makes `devtool upgrade` consider any available
git configuration instead of only checking `--global`.

(From OE-Core rev: 30a9f7de45050c8bac49d4b37419cc2e067a75fa)

Signed-off-by: Marcus Flyckt <marcus.flyckt@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-08 13:05:25 +01:00
Chris Laplante
893e17f669 recipetool/create_buildsys_python: use importlib instead of imp
'imp' was deprecated in Python 3.4 and removed in 3.12. The
piece of importlib we use has been around since 3.3.

(From OE-Core rev: 457f0dad87b4e45a53865b5ad2c150215bd74019)

Signed-off-by: Chris Laplante <chris.laplante@agilent.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-06 11:51:11 +01:00
Jermain Horsman
02b8c36efe scripts/oe-setup-layers: Update how to determine if directory is git repo
Previously _is_repo_git_repo() would return a result containing b'true\n' or
b'false\n' if 'git rev-parse' ran successfully, instead of True of False.

While this can be solved using e.g. result.strip().decode("utf-8") == "true",
there are some other cases to consider.
First, .git can be a file and not a directory when using a worktree.
Second, an emtpy .git directory in 'repodir' for which some parent of
'repodir' is an actual git repo will still return True in this case.

To account for these cases as well, use 'git rev-parse --show-toplevel'
and compare the result against 'repodir' instead of
using 'git rev-parse --is-inside-git-dir'.

(From OE-Core rev: 0830f53900dd7fd669a7d6492325559ad3225302)

Signed-off-by: Jermain Horsman <jermain.horsman@nedap.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-06 11:51:11 +01:00
Richard Purdie
141707c232 wic: Add console parameters to qemux86 canned-wks
Currently our wic test images boot up without kernel output on the consoles
which means we have no way to debug if anything goes wrong. Add the console
parameters runqemu would have added if the kernel wasn't built into an image
to improve our chances of debugging.

(From OE-Core rev: 63ea1efdaaf1173ef4f2f69b5a3403afef4b556a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-04 23:05:54 +01:00
Richard Purdie
8c4bb95c1c scritps/runqemu: Ensure we only have two serial ports
I have a theory that some of the console boot issues we're seeing are due to
starting images with three serial ports yet only starting gettys on two of them.

This means that occasionally, depending on the port numbering we may not get
a login prompt on the console we expect it on.

To fix this, change the runqemu code so that if serial ports are passed in on
the commandline (as is the case in automated testing), we don't add any other
GUI serial consoles.

We do need to make sure we do have at least two serial ports since we don't want
getty timeout warnings.

(From OE-Core rev: 1b0348535dce3b776efbcf26406b94730a51eb85)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-04 13:00:33 +01:00
Daniel Semkowicz
578a627081 wic: bootimg-partition: Fix file name in debug message
Debug message about using custom configuration file includes file name
with incorrect extension. Correct file name to "extlinux.conf".

(From OE-Core rev: dd63e1520454b2d53a48b72eaae126059af9809b)

Signed-off-by: Daniel Semkowicz <dse@thaumatec.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-10-01 08:12:59 +01:00