Commit Graph

7325 Commits

Author SHA1 Message Date
David Reyna
f62e4d81ec bitbake: taskexp_ncurses: ncurses version of taskexp.py
* Create an ncurses version of the GTK app "taskexp.py".
* Add these additional features:
  - Sort tasks in recipes by their dependency order
  - Print individual and/or recipe-wide dependencies to a file
  - Add a wild card filter
  - Show the target recipes on BOLD
* Provide a GUI self test
* Provide a non-ncurses self test for ptest

(Bitbake rev: f49bec66ad51c8cddeceafbbe2445c46e396ee8b)

Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-18 17:38:47 +00:00
Richard Purdie
cdb1cb2ed8 bitbake: runqueue: Improve setcene performance when encoutering many 'hard' dependencies
"bitbake world -n --setscene-only" shows poor performance as the numbers of tasks
increases. Analysys shows this is due to the "deferred" hard dependencies being
repeatedly processed which doesn't allow much forward progress in overall task
execution.

To avoid this, mark when it has been done and don't reprocess until dependencies
are updated. We have to be careful as we've seen bugs where these dependencies
aren't processed at the right time. They also have to be reproceseed during task
migration/rehashing so the code has to "self heal" the data structures.

(Bitbake rev: e5609bac06c17dabcf6286b47b1a3f19f5a1160f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-13 13:52:10 +00:00
Richard Purdie
71f55957f0 bitbake: runqueue: Optimise taskname lookups in next_buildable_task
A quick profile of bitbake world showed 147 million calls to taskname_from_tid().
The next_buildable_task function is performance senstive so move the call
inside the if block to reduce the number of calls and speed the code up.

(Bitbake rev: 8b332c16a7b6b85c5cbe1919dd8cae45fda6adf9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-13 13:52:10 +00:00
Richard Purdie
a7bbb105d3 bitbake: runqueue: Improve performance for executing tasks
Now that runqueue performance profiling works again we can see a lot
of time is lost in build_taskdepdata. Whilst we can't compute that
in advance, we can compute the individual entries.

Therefore put a cache in place to compute those and save overhead in
starting up tasks.

(Bitbake rev: c4519b542702ba25023e53d77b275a6fa571ec50)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-13 13:52:10 +00:00
Adrian Freihofer
1a3f01cad4 bitbake: bitbake/lib/bs4/tests/test_tree.py: python 3.12 regex
Python 3 interprets string literals as Unicode strings, and therefore
\s is treated as an escaped Unicode character which is not correct.
Declaring the RegEx pattern as a raw string instead of unicode is
required for Python 3.

(Bitbake rev: a5f1bc69ef2e456bd163303a07cfb73825b01576)

Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-13 13:52:10 +00:00
Richard Purdie
babd5fea4a bitbake: process/server: Fix typo
Ensure the message matches the filenames the code actually uses.

(Bitbake rev: deb7db2e2b125c6a6732db4f185f4de5926494fd)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-10 15:25:22 +00:00
Michael Opdenacker
0ac07ddf71 bitbake: doc: README: simpler link to contributor guide
(Bitbake rev: 57d2f54e00374fe7452e123ec3c6e7ac27afb024)

Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-10 14:13:51 +00:00
Thomas Perrot
987ab2a446 bitbake: wget.py: always use the custom user agent
Add the "--user-agent" paramater in the wget base command to
perform all wget commands with this parameter, because a few
HTTP servers block requests with the default wget user agent.

For example, "hg.openjdk.org" never send a response to requests
have been sent with wget:
wget https://hg.openjdk.org/jdk8u/jdk8u/archive/jdk8u272-ga.tar.bz2
https://hg.openjdk.org/jdk8u/jdk8u/archive/jdk8u272-ga.tar.bz2
Resolving hg.openjdk.org (hg.openjdk.org)... 23.54.129.73
Connecting to hg.openjdk.org (hg.openjdk.org)|23.54.129.73|:443... connected.
HTTP request sent, awaiting response...

(Bitbake rev: d6fa261a9603677f0b3abbd309c1ca6073b63f4c)

Signed-off-by: Thomas Perrot <thomas.perrot@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-10 14:13:51 +00:00
Richard Purdie
0f50f21151 bitbake: process: Add profile logging for main loop
When the idle/main loop was added, we didn't include profiling information
for it. There is a performance issue in there, add logging for it.

(Bitbake rev: d8d5cd43a60560f67e86f4f625113b0f73b944c0)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-10 14:13:51 +00:00
Toni Lammi
e834e8ee91 bitbake: support temporary AWS credentials
Support AWS_SESSION_TOKEN which is used in temporary
AWS credentials.

Fixes [YOCTO #15384].

(Bitbake rev: ae1e4c90bbc2002cb2728c64649c095c00220ceb)

Signed-off-by: Toni Lammi <toni.lammi@kone.com>
Reported-by: Toni Lammi <toni.lammi@tl-software.fi>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-09 14:06:16 +00:00
Martin Jansa
5276b7f548 bitbake: tests: fetch.py: use real subversion repository
* github no longer provides support for subversion clients:
  https://docs.github.com/en/enterprise-server@3.11/get-started/working-with-subversion-on-github/support-for-subversion-clients
  it was shut down on 2024-01-08:
  https://github.blog/2023-01-20-sunsetting-subversion-support/

  and this test was now failing with:

======================================================================
ERROR: test_external_svn (bb.tests.fetch.SVNTest.test_external_svn)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/OE/layers/bitbake/lib/bb/tests/fetch.py", line 1287, in test_external_svn
    fetcher.download()
  File "/OE/layers/bitbake/lib/bb/fetch2/__init__.py", line 1896, in download
    raise FetchError("Unable to fetch URL from any source.", u)
bb.fetch2.FetchError: Fetcher failure for URL: 'svn:///tmp/bitbake-fetch-zdvedqt_/svnfetch_localrepo_wup8mgn6/project;module=trunk;protocol=file;externals=allowed;rev=2'. Unable to fetch URL from any source.

Stdout:
Fetch svn:///tmp/bitbake-fetch-zdvedqt_/svnfetch_localrepo_wup8mgn6/project;module=trunk;protocol=file;externals=allowed;rev=2
Failed to fetch URL svn:///tmp/bitbake-fetch-zdvedqt_/svnfetch_localrepo_wup8mgn6/project;module=trunk;protocol=file;externals=allowed;rev=2, attempting MIRRORS if available
Fetcher failure: Fetch command export PSEUDO_DISABLED=1; /usr/bin/env svn --non-interactive --trust-server-cert co --no-auth-cache -r 2 file:///tmp/bitbake-fetch-zdvedqt_/svnfetch_localrepo_wup8mgn6/project/trunk@2 trunk failed with exit code 1, output:
A    trunk/README.md
 U   trunk
Checked out revision 2.

svn: warning: W205011: Error handling externals definition for 'trunk/bitbake':
svn: warning: W170013: Unable to connect to a repository at URL 'https://github.com/PhilipHazel/pcre2.git'
svn: E205011: Failure occurred processing one or more externals definitions

  in the rare cases where subversion was still installed on the host
  running bitbake-selftest :).

  to avoid this use still alive repository from https://svn.apache.org/
  and pick something rather small and only the trunk subdirectory which
  is fast to fetch:
  svn co https://svn.apache.org/repos/asf/serf/trunk
  takes just 2 sec here

  adjust expected dir/file to use "protocols/fcgi_buckets.h" instead of
  "trunk/README"

(Bitbake rev: a735898abcf056f897c9350bb128a5637e6b4617)

Signed-off-by: Martin Jansa <martin.jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-08 10:59:32 +00:00
Martin Jansa
58155ea1a3 bitbake: bitbake-diffsigs: fix walking the task dependencies and show better error
* when comparing 2 tmp/stamps/*do_configure.sigdata* I got
  TypeError("filename must be a str or bytes object, or a file")
  but both files I was comparing were
  Zstandard compressed data (v0.8+), Dictionary ID: None
  according to "file"

  with TypeError catched to show which file it failed to open I got better
  error which shows it was trying to read "do_prepare_recipe_sysroot.sigdata"
  file now and after a while you might notice that it's not just the expected
  file, but a dict with 'path', 'sstate', 'time'.

  Fix that in bitbake-diffsigs but keep the TypeError and add OSError
  in case it will eventually walk on file which isn't zstd compressed
  pipecompress throws CompressionError.

ERROR: Failed to open {'path': '5.15.do_prepare_recipe_sysroot.sigdata.99b12a401341a0df7c3553cb00c87a7674295496bd5c25ed71764ee0d0fb8eb8', 'sstate': False, 'time': 1707136354.991718}: filename must be a str or bytes object, or a file
Traceback (most recent call last):
  File "bitbake/bin/bitbake-diffsigs", line 192, in <module>
    output = bb.siggen.compare_sigfiles(options.sigdatafile1, options.sigdatafile2, recursecb, color=color)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "bitbake/lib/bb/siggen.py", line 1039, in compare_sigfiles
    recout = recursecb(dep, a[dep], b[dep])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "bitbake/bin/bitbake-diffsigs", line 102, in recursecb
    out2 = bb.siggen.compare_sigfiles(hashfiles[hash1], hashfiles[hash2], recursecb, color=color)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "bitbake/lib/bb/siggen.py", line 857, in compare_sigfiles
    raise err
  File "bitbake/lib/bb/siggen.py", line 853, in compare_sigfiles
    with bb.compress.zstd.open(a, "rt", encoding="utf-8", num_threads=1) as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "bitbake/lib/bb/compress/zstd.py", line 12, in open
    return bb.compress._pipecompress.open_wrap(ZstdFile, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "bitbake/lib/bb/compress/_pipecompress.py", line 59, in open_wrap
    raise TypeError("filename must be a str or bytes object, or a file")
TypeError: filename must be a str or bytes object, or a file

* if I replace zstd file with just plaintext it fails with:
  $ echo foo > foo
  $ echo foo > bar
  $ bitbake-diffsigs foo bar
  zstd: /*stdin*\: unsupported format
  ERROR: Process died with 1
  sys:1: ResourceWarning: unclosed file <_io.BufferedReader name='foo'>

  with this change it shows the name of the file which it failed to uncompress:

  $ bitbake-diffsigs foo bar
  zstd: /*stdin*\: unsupported format
  ERROR: Failed to open sigdata file 'foo': Process died with 1
  ERROR: Process died with 1
  sys:1: ResourceWarning: unclosed file <_io.BufferedReader name='foo'>

(Bitbake rev: f3f843a4fd6e0a9e8f6edef5dd3cf1fce29c50ba)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-02-08 10:59:32 +00:00
Ross Burton
ff0bc2a284 bitbake: bitbake: Version bump for inherit_defer addition
We've added a new statement, inherit_defer, so bump the version so this
can be checked.

(Bitbake rev: 191e6eb2bceb467c97e315301f1f64722cf0e976)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-26 16:25:39 +00:00
Richard Purdie
c56fce9e56 bitbake: ast/BBHandler: Add inherit_defer support
Add support for an inherit_defer statement which works as
inherit does but is only evaulated at the end of parsing.

This allows conditional expressions to be evaulated 'late' meaning changes
to PACKAGECONFIG from bbappends can be applied for example. This addresses
a usability/confusion issue users have with the current conditional
inherit mechanism since they don't realise the condition has to be
expanded immediately with the current model.

There is a commented out warning we could enable in future which
warns about the use of a conditional inherit that isn't deferred.

There is a behaviour difference in the placement of the inherit,
particularly around variables set using ?= which means wen swapping
from one to the other, caution must be used as values can change.

(Bitbake rev: 5c2e840eafeba1f0f754c226b87bfb674f7bea29)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-18 10:15:58 +00:00
Richard Purdie
61182659c2 bitbake: runqueue: Fix runall all bug
Where chains of RDEPENDS are multiple levels deep, the runall code was not
accounting for this and recursing deeply enough to gather all dependencies.

Fix this by iterating over the result until no more dependencies are found.

Tested-by: Jonas Gorski <jonas.gorski@bisdn.de>
Reported-by: Jonas Gorski <jonas.gorski@bisdn.de>
(Bitbake rev: 966f25dfc23a6d17b2b6d3e0100e9ae264f99025)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-12 11:59:36 +00:00
Alexander Kanavin
67e3f24333 bitbake: bitbake/runqueue: rework 'bitbake -S printdiff' logic
Previously printdiff code would iterate over tasks that were reported as invalid or absent,
trying to follow dependency chains that would reach the most basic invalid items in the tree.

While this works in tightly controlled local builds, it can lead to bizarre reports
against industrial-sized sstate caches, as the code would not consider whether the
overall target can be fulfilled from valid sstate objects, and instead report
missing sstate signature files that perhaps were never even created due to hash
equivalency providing shortcuts in builds.

This commit reworks the logic in two ways:

- start the iteration over final targets rather than missing objects
and try to recursively arrive at the root of the invalid object dependency.

A previous version of this patch relied relies on finding the most 'recent'
signature in stamps or sstate in a different function later, and recursively
comparing that to the current signature, which is unreliable on real world caches.

- if a given object can be fulfilled from sstate, recurse only into
its setscene dependencies; bitbake wouldn't care if dependencies
for the actual task are absent, and neither should printdiff

I wrote a recursive function for following dependencies, as
doing recursive algorithms non-recursively can result in write-only
code, as was the case here.

[YOCTO #15289]

(Bitbake rev: aadeca63da5d96160ce4d6d71da556e2e033f9b7)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-10 16:32:13 +00:00
Mark Asselstine
98c5c96dd3 bitbake: ui/knotty: properly handle exceptions when calling runCommand()
In runCommand() the send() and recv() can fail and raise
BrokenPipeError and EOFError exceptions when the bitbake-server is
unexpectedly terminated. In these cases a python traceback is
currently dumped. Similarly updateFromServer() which calls
runCommand() can also raise these and other exceptions, and currently
lacks proper exception handling resulting in python traceback.

We wrap calls to runCommand() and updateFromServer() in a try/except
block as well as improve the exception handling for updateToServer().

This along with the earlier commit which added text to the
BrokenPipeError and EOFError exceptions in runCommand() to indicate a
bitbake-server termination may have occurred, should improve the user's
ability to understand and handle these errors.

An easy way to trigger each of the runCommand() exceptions is to
'kill -9' bitbake-server before (causes EOFError) or after
(causes BrokenPipeError) the "Loading Cache" stage.

(Bitbake rev: 804d366ee3ddc0f37f0a6c712c8d42db45b119bc)

Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-10 14:02:38 +00:00
Mark Asselstine
bc22d82c2f bitbake: server/process: catch and expand multiprocessing connection exceptions
Doing builds on systems with limited resources, or with high demand
package builds such as chromium it isn't uncommon for the OOM Killer
to be triggered and for bitbake-server to be selected as the process
to be killed. When the bitbake-server does terminate unexpectedly due
to the OOM Killer or otherwise, this currently results in a generic
python traceback with little indication as to what has failed.

Here we trap and raise the exceptions while extending the exception
text in runCommand() to make it clear that this is most likely caused
by the bitbake-server unexpectedly terminating.

Callers of runCommand() should be updated to properly handle the
BrokenPipeError and EOFError exceptions to avoid printing a python
traceback, but even if they don't, the added text in the exceptions
should provide some hints as to what might have caused the failure.

(Bitbake rev: 5ff62b802f79acc86bbd6a99484f08501ff5dc2d)

Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-10 14:02:38 +00:00
Richard Purdie
c665a2c933 bitbake: ast: Fix EXPORT_FUNCTIONS bug
If you have two classes, both of which set EXPORT_FUNCTIONS for the same funciton
and a standard funciton definition for the function that is exported, the export
function can sometimes overwrite the standard one.

The issue is that the internal flag the code uses isn't ovweritten if the variable
is giving a new value. Fix the issue by using a comment in the code that is injected
so that we know if it is ours or not.

Also add some testing for EXPORT_FUNCTIONS, not perfect but a start.

(Bitbake rev: 66306d5151acb0a26a171c338d8f60eb9eb16c6b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-10 13:55:33 +00:00
Alexander Kanavin
56b1af37dc bitbake: fetch/wget/checkstatus(): include the URL in debugging output about status check failure
Previously the output wasn't useful for finding out what was the actual
URL that failed, particularly in heavily multi-threaded invocations:

DEBUG: checkstatus() urlopen failed: HTTP Error 404: Not Found

With this change, the problem is described specifically:

DEBUG: checkstatus() urlopen failed for http://cdn.jsdelivr.net/yocto/sstate/all/universal/4f/91/sstate:gettext-minimal-native:x86_64-linux:0.22.4:r0:x86_64:11:4f91b650ebd7be601cbd0e3a37a8cc6385a3f4ee616f931969b50709ed8bf044_create_spdx.tar.zst: HTTP Error 404: Not Found

This will help with CDN cache tests in particular. When some object
isn't available, we need to know why: 4xx error, 5xx error, timeout
error or any other issue.

(Bitbake rev: ecd9b92815563509f55264ed6e7498aee797cedd)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-10 13:55:33 +00:00
Alexander Kanavin
4890c6a7ca bitbake: fetch/checkstatus(): do not print the URI twice in FetchError exception
Previously, there was duplicate clutter in the output, particularly if the
URI points to sstate cache items:

bb.fetch2.FetchError: Fetcher failure for URL: {uri}. URL {uri} doesn't work

(Bitbake rev: 61537b8a98b963e4af265e046d41407b32fa5935)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-10 13:55:33 +00:00
Robert Yang
4e2fcdb383 bitbake: bitbake: tests/event: Add test_lineno_in_eventhandler
Add test_lineno_in_eventhandler to test lineno in eventhandler.

(Bitbake rev: 4e5de537bebb68180c5755858c81b095eb9ae2f6)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-10 13:55:33 +00:00
Robert Yang
3f41592920 bitbake: bitbake: event: Inject empty lines to make code match lineno in filename
So that we can get the correct error messages.

* In python 3.10.9, the error message was:
  ERROR: Unable to register event handler 'defaultbase_eventhandler':
    File "/path/to/poky/meta/classes-global/base.bbclass", line 4
      # SPDX-License-Identifier: MIT
             ^^^^^
  SyntaxError: invalid syntax

  This is hard to debug since the error line number 4 is incorrect, but nothing
  is wrong with the code in line 4.

* Now the error message and lineno is correct:
  ERROR: Unable to register event handler 'defaultbase_eventhandler':
    File "/path/to/poky/meta/classes-global/base.bbclass", line 256
      an error line
         ^^^^^
  SyntaxError: invalid syntax

And no impact on parsing time:
* Before:
$ rm -fr cache tmp; time bitbake -p
real    0m27.254s

* Now:
$ rm -fr cache tmp; time bitbake -p
real    0m27.200s

(Bitbake rev: c212933d9c786806852c87f188250a4f0a14c048)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-10 13:55:33 +00:00
Alassane Yattara
36dc42bde7 bitbake: toaster/tests: Bug-fix ToasterTable show_rows testcases
Test if some rows are visible in table instead of compare row to row_to_show,
because sometime full avaiblable content did not display

Failed: https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/147/steps/12/logs/stdio

(Bitbake rev: 5b0a48265aafa62259c575707c3afa6dd56f8008)

Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-10 13:55:33 +00:00
Alassane Yattara
0dcf84f830 bitbake: toaster/tests: Bug-fix "element not interactable" in TestLayerDetailsPage::test_edit_layerdetails
Failed: https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/143/steps/12/logs/stdio

(Bitbake rev: 187e96eb7393632f28a195f280fa133439bdc0fa)

Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-10 13:55:33 +00:00
Alassane Yattara
c0ade693af bitbake: toaster/tests: bug-fix "#hint-error-project-name" should be visible
Failed: https://autobuilder.yoctoproject.org/typhoon/#/builders/161/builds/142

(Bitbake rev: 0ee5f4e06476b0ec2f5ea8c9f05d299ddda6312b)

Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-10 13:55:33 +00:00
Alassane Yattara
49ccf2f5e0 bitbake: toaster/tests: Setup delay after driver action self.get(url)
Recurring test failures result from insufficient delays in driver actions.

(Bitbake rev: b0de2a61d14fbf30e338751b285b3bab80192275)

Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-10 13:55:33 +00:00
Alassane Yattara
070aa22705 bitbake: toaster/tests: Delay driver first action on create new project page
Wait for element visible on create new project page

(Bitbake rev: 664de3f6d3484b94f5d82ec634b512b825553aa9)

Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-07 22:59:55 +00:00
Alassane Yattara
1cd56989a0 bitbake: toaster/tests: Bug-fix element click intercepted
Fix "element click intercepted" on TestProjectConfigTab::test_project_config_tab_right_section

(Bitbake rev: c8685c762aa1fab687ff3a0943487675ef720755)

Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-07 22:59:55 +00:00
Alassane Yattara
74c234bf0d bitbake: toaster/tests: Bug-fix on TestProjectConfigTab::test_image_recipe_show_rows
Check some rows are visible in table instead of compare table row to
row_to_show, because recipe image table sometime doesn't display full avaiblable
images

(Bitbake rev: 1e2e5927ef7a8adfd3d0a3be1c75b4aa410d9908)

Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-07 22:59:55 +00:00
Richard Purdie
71c0d311ba bitbake: bitbake: Version bump for find_siginfo chanages
Bump the version to 2.7.1 for the find_siginfo changes.

(Bitbake rev: 03995e16bf7186f5368f772f617d563f4d280641)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-05 11:59:08 +00:00
Alexander Kanavin
7d400f94fe bitbake: bitbake/runqueue: prioritize local stamps over sstate signatures in printdiff
Even with the reworked printdiff code, sstate which is heavily used in parallel
can throw races at the tests: if a new matching, but otherwise unrelated
sstate signature appears between writing out local stamps and listing
matching sstate files, then that signature will be deemed 'the latest'
and the actual local stamp will be discarded. This change ensures
the scenario does not happen.

It also makes use of the reworked find_siginfo(), particularly the 'sstate'
entry in returned results.

(Bitbake rev: c8574b796dabb69699c70540dd95a44d8f7388ab)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-05 11:59:08 +00:00
Richard Purdie
859786a83f bitbake: siggen: Ensure version of siggen is verified
Since we need to change the form of the siggen function, we need to add versioning
and some verison checks. This means if a newer bitbake is used with older metadata
we can detect it.

(Bitbake rev: 721556568413508213d22c29985e305a45a8d68a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-05 11:59:08 +00:00
Alexander Kanavin
cc85c8eb9d bitbake: bitbake-diffsigs/runqueue: adapt to reworked find_siginfo()
In particular having 'time' explicitly used as a sorting key should make it
more clear how the entries are being sorted.

(Bitbake rev: 5439aca056c84ab4410aaf24bdb68e896191d8e1)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-05 11:59:08 +00:00
Richard Purdie
a41b54ccbd bitbake: bitbake: Post release version bump to 2.7.0
Bump to a development version post release.

(Bitbake rev: 28364c08f36c778a5cb2e3f20ceb052370ef153c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-05 11:39:33 +00:00
Alexander Kanavin
b0c47ae555 bitbake: bitbake/runqueue: add debugging for find_siginfo() calls
(Bitbake rev: 52f5503e8cf048331134233fb681db6dc736ae38)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-05 11:39:33 +00:00
Alassane Yattara
831b3d5b22 bitbake: toaster/toastergui: Bug-fix verify given layer path only if import/add local layer
(Bitbake rev: 94e88efa9dbefd37f1d48459ade19797b6034b84)

Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-04 23:48:29 +00:00
Alexander Kanavin
3958182710 bitbake: bitbake/codeparser.py: address ast module deprecations in py 3.12
Specifically:
/srv/work/alex/poky/bitbake/lib/bb/codeparser.py:279: DeprecationWarning: ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
  if isinstance(node.args[0], ast.Str):
/srv/work/alex/poky/bitbake/lib/bb/codeparser.py:280: DeprecationWarning: Attribute s is deprecated and will be removed in Python 3.14; use value instead
  value = node.args[0].s

(Bitbake rev: de8ba2770d9a1a94af3d084f9540da7e2fae6022)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-02 14:41:06 +00:00
Alassane Yattara
280917c2e7 bitbake: toaster/tests: Bug-fix "#project-created-notification" should be visible
Added more delay between click on create project but and when notification is displayed

(Bitbake rev: 5382cc0699eebc1e91675a2a147f8fe7dab23c14)

Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-01 23:13:50 +00:00
Alassane Yattara
350300f836 bitbake: toaster/tests: Skip to show more then 100 item in ToasterTable
(Bitbake rev: 860c931381e0694a854fd90775fb18dadb7d76c6)

Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-01 23:13:50 +00:00
Alassane Yattara
3e6d3817da bitbake: toaster/tests: bug-fix An element matching "#lastest_builds" should be on the page
(Bitbake rev: 5bcba4596cd9f4f54c7ae7ebd9322897c2f829cd)

Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-01 23:13:50 +00:00
Alassane Yattara
6a67f24773 bitbake: toaster/tests: bug-fix An element matching "#projectstable" should be visible
(Bitbake rev: 1edb97f741a48481b1b9f26c5cb31acd9059f07f)

Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2024-01-01 23:13:50 +00:00
Richard Purdie
fe5d2f0b66 bitbake: lib/bb: Add workaround for libgcc issues with python 3.8 and 3.9
With python 3.8 and 3.9, we see intermittent errors of:

libgcc_s.so.1 must be installed for pthread_cancel to work
Aborted (core dumped)

which seem related to:

https://stackoverflow.com/questions/64797838/libgcc-s-so-1-must-be-installed-for-pthread-cancel-to-work
https://bugs.ams1.psf.io/issue42888

These tend to occur on debian 11 and ubuntu 20.04.

Workaround this by ensuring libgcc is preloaded in all cases.

(Bitbake rev: c6666f6cfafd55e1c980239a7c5ff908f1a69196)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-30 11:03:25 +00:00
Joshua Watt
34bafb3cf2 bitbake: contrib/vim: Syntax improvements
Makes a few improvments to the vim Bitbake syntax plugin:
 1) Highlight python expansion expressions "${@...}" in
    inherit/include/require
 2) Highlight variables "${..}" and python expressions "${@...}" in
    addtask/deltask/addhandler
 3) Correctly handle multi-line sequences in addtask/deltask/addhanlder

(Bitbake rev: 39691d5d0f44a266f917a13884707283f83543de)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-23 16:20:25 +00:00
Marta Rybczynska
1720dae4ed bitbake: toastergui: verify that an existing layer path is given
Verify that an existing layer path was given when adding a new
layer.

Manually using the shell for globbing is unnecessary, use the glob
function instead for cleaner code.

(Bitbake rev: fe0881615896de844141393b21a121f7c3fa9d16)

Signed-off-by: Marta Rybczynska <marta.rybczynska@syslinbit.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-20 22:08:31 +00:00
Alassane Yattara
98214f1bc2 bitbake: toaster/tests: Bug-fix test_functional_basic, delay driver actions
The errors causing faileds on functional_basic are dû to the delay between actions,
increase between driver actions.

(Bitbake rev: e8f8f6203b63c46249673e80872fea40475f6875)

Signed-off-by: Alassane Yattara <alassane.yattara@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-20 12:02:07 +00:00
Joshua Watt
490b94c3b5 bitbake: bitbake-hashserv: Add description of permissions
Adds a text description of the possible permissions in the hash server
help text

(Bitbake rev: 8295ac1b6672c25bee595cff6e000b2af817f904)

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-18 09:18:01 +00:00
Richard Purdie
69a4180c88 bitbake: runqueue: Remove tie between rqexe and starts_worker
We've been moving to try and separate several pieces of runqueue. Allow
start_worker to operate separately to rqexe since they don't need to be
tied. This allows rqexe to be available to print_diff for future
improvements.

(Bitbake rev: 834e452243ff2eea6e8e2e7f4935b5233ffb4b00)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-17 19:07:21 +00:00
Richard Purdie
e706249f7b bitbake: utils: Fix mkdir with PosixPath
Avoid:
    Exception: AttributeError: 'PosixPath' object has no attribute 'find'

(Bitbake rev: 0b37fe89ba12549109905b6d0e6d07d342162436)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-17 19:07:21 +00:00
Marlon Rodriguez Garcia
ff7aba28f7 bitbake: toaster: Added validation to stop import if there is a build in progress
Added validation to prevent simultaneous imports from running because the database fails at runtime.
The option to create a queue was taken into consideration. However, it will require the use of Celery https://pypi.org/project/celery/ or Background Task https://pypi.org/project/django-background-tasks/ which require the use of external services and multiple dependencies.
If required we could explore the alternative in the future.

(Bitbake rev: eb417e27be5717a259f27e98dbd73255b1a42fc9)

Signed-off-by: Marlon Rodriguez Garcia <marlon.rodriguez-garcia@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-12-16 13:06:11 +00:00