poky/bitbake
Pedro Ferreira 921c46de4f bitbake: sigen: Avoid bitbake abort with EOFerror from an incomplete hashserv communication
The issue itself is sporadic but aborts the build with an uncaught exception.

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_func_python() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:sstate_report_unihash(d)
     0003:
File: '(...)/poky/meta/classes/sstate.bbclass', lineno: 882, function: sstate_report_unihash
     0878:    report_unihash = getattr(bb.parse.siggen, 'report_unihash', None)
     0879:
     0880:    if report_unihash:
     0881:        ss = sstate_state_fromvars(d)
 *** 0882:        report_unihash(os.getcwd(), ss['task'], d)
     0883:}
     0884:
     0885:#
     0886:# Shell function to decompress and prepare a package for installation
File: '(...)/poky/lib/bb/siggen.py', lineno: 651, function: report_unihash
     0647:                method = self.method
     0648:                if tid in self.extramethod:
     0649:                    method = method + self.extramethod[tid]
     0650:
 *** 0651:                data = self.client().report_unihash(taskhash, method, outhash, unihash, extra_data)
     0652:                new_unihash = data['unihash']
     0653:
     0654:                if new_unihash != unihash:
     0655:                    hashequiv_logger.debug('Task %s unihash changed %s -> %s by server %s' % (taskhash, unihash, new_unihash, self.server))
File: '(...)/poky/lib/bb/asyncrpc/client.py', lineno: 139, function: wrapper
     0135:        pass
     0136:
     0137:    def _get_downcall_wrapper(self, downcall):
     0138:        def wrapper(*args, **kwargs):
 *** 0139:            return self.loop.run_until_complete(downcall(*args, **kwargs))
     0140:
     0141:        return wrapper
     0142:
     0143:    def _add_methods(self, *methods):
File: '/usr/lib/python3.9/asyncio/base_events.py', lineno: 642, function: run_until_complete
     0638:            future.remove_done_callback(_run_until_complete_cb)
     0639:        if not future.done():
     0640:            raise RuntimeError('Event loop stopped before Future completed.')
     0641:
 *** 0642:        return future.result()
     0643:
     0644:    def stop(self):
     0645:        """Stop running the event loop.
     0646:
File: '(...)/poky/lib/hashserv/client.py', lineno: 70, function: report_unihash
     0066:        m["taskhash"] = taskhash
     0067:        m["method"] = method
     0068:        m["outhash"] = outhash
     0069:        m["unihash"] = unihash
 *** 0070:        return await self.invoke({"report": m})
     0071:
     0072:    async def report_unihash_equiv(self, taskhash, method, unihash, extra={}):
     0073:        await self._set_mode(self.MODE_NORMAL)
     0074:        m = extra.copy()
File: '(...)/poky/lib/bb/asyncrpc/client.py', lineno: 104, function: invoke
     0100:        async def proc():
     0101:            await self.socket.send_message(msg)
     0102:            return await self.socket.recv_message()
     0103:
 *** 0104:        return await self._send_wrapper(proc)
     0105:
     0106:    async def ping(self):
     0107:        return await self.invoke({"ping": {}})
     0108:
File: '(...)/poky/lib/bb/asyncrpc/client.py', lineno: 82, function: _send_wrapper
     0078:    async def _send_wrapper(self, proc):
     0079:        count = 0
     0080:        while True:
     0081:            try:
 *** 0082:                await self.connect()
     0083:                return await proc()
     0084:            except (
     0085:                OSError,
     0086:                ConnectionError,
File: '(...)/poky/lib/bb/asyncrpc/client.py', lineno: 70, function: connect
     0066:        await self.socket.send("")
     0067:
     0068:    async def connect(self):
     0069:        if self.socket is None:
 *** 0070:            self.socket = await self._connect_sock()
     0071:            await self.setup_connection()
     0072:
     0073:    async def close(self):
     0074:        if self.socket is not None:
File: '(...)/poky/lib/bb/asyncrpc/client.py', lineno: 55, function: connect_sock
     0051:        import websockets
     0052:
     0053:        async def connect_sock():
     0054:            try:
 *** 0055:                websocket = await websockets.connect(uri, ping_interval=None)
     0056:            except (OSError, asyncio.exceptions.TimeoutError, websockets.InvalidHandshake, websockets.InvalidURI) as exc:
     0057:                raise ConnectionError("Could not connect to websocket: %s" % exc) from exc
     0058:            return WebsocketConnection(websocket, self.timeout)
     0059:
File: '/usr/local/lib/python3.9/dist-packages/websockets/asyncio/client.py', lineno: 444, function: __await_impl__
     0440:            async with asyncio_timeout(self.open_timeout):
     0441:                for _ in range(MAX_REDIRECTS):
     0442:                    self.connection = await self.create_connection()
     0443:                    try:
 *** 0444:                        await self.connection.handshake(*self.handshake_args)
     0445:                    except asyncio.CancelledError:
     0446:                        self.connection.close_transport()
     0447:                        raise
     0448:                    except Exception as exc:
File: '/usr/local/lib/python3.9/dist-packages/websockets/asyncio/client.py', lineno: 104, function: handshake
     0100:        # receiving a response, when the response cannot be parsed, or when the
     0101:        # response fails the handshake.
     0102:
     0103:        if self.protocol.handshake_exc is not None:
 *** 0104:            raise self.protocol.handshake_exc
     0105:
     0106:    def process_event(self, event: Event) -> None:
     0107:        """
     0108:        Process one incoming event.
File: '/usr/local/lib/python3.9/dist-packages/websockets/client.py', lineno: 315, function: parse
     0311:
     0312:    def parse(self) -> Generator[None]:
     0313:        if self.state is CONNECTING:
     0314:            try:
 *** 0315:                response = yield from Response.parse(
     0316:                    self.reader.read_line,
     0317:                    self.reader.read_exact,
     0318:                    self.reader.read_to_eof,
     0319:                )
File: '/usr/local/lib/python3.9/dist-packages/websockets/http11.py', lineno: 238, function: parse
     0234:
     0235:        try:
     0236:            status_line = yield from parse_line(read_line)
     0237:        except EOFError as exc:
 *** 0238:            raise EOFError("connection closed while reading HTTP status line") from exc
     0239:
     0240:        try:
     0241:            protocol, raw_status_code, raw_reason = status_line.split(b" ", 2)
     0242:        except ValueError:  # not enough values to unpack (expected 3, got 1-2)
Exception: EOFError: connection closed while reading HTTP status line

(Bitbake rev: 5ba7c2f0797a72536a81f57276d4e5c75f23011c)

Signed-off-by: Pedro Ferreira <Pedro.Silva.Ferreira@criticaltechworks.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2025-07-23 11:34:31 +01:00
..
bin bitbake: main: Add an option to specify what to profile 2025-07-17 10:45:57 +01:00
contrib bitbake: contrib: vim: ftdetect: don't conflict with other filetypes 2025-03-03 21:38:57 +00:00
doc bitbake: doc/bitbake-user-manual-fetching: update the Git fetcher tag description 2025-07-22 14:19:25 +01:00
lib bitbake: sigen: Avoid bitbake abort with EOFerror from an incomplete hashserv communication 2025-07-23 11:34:31 +01:00
.b4-config bitbake: b4-config: Add basic support for b4 contribution workflow 2025-02-06 10:40:10 +00:00
.gitattributes bitbake: .gitattributes: Add to improve git diff for minified css/js files 2019-03-07 12:18:48 +00:00
AUTHORS misc: Update the email address to a working one. 2011-01-04 14:36:54 +00:00
ChangeLog *: Fix typo in documentation 2010-08-04 16:12:39 +01:00
LICENSE bitbake: bitbake: Add initial pass of SPDX license headers to source code 2019-05-04 10:44:04 +01:00
LICENSE.GPL-2.0-only bitbake: bitbake: Add initial pass of SPDX license headers to source code 2019-05-04 10:44:04 +01:00
LICENSE.MIT bitbake: bitbake: Add initial pass of SPDX license headers to source code 2019-05-04 10:44:04 +01:00
README bitbake: doc: README: simpler link to contributor guide 2024-02-10 14:13:51 +00:00
SECURITY.md bitbake: SECURITY.md: add file 2023-10-24 12:49:56 +01:00
toaster-requirements.txt bitbake: Update toaster-requirements to add django-log-viewer==1.1.7 2023-10-15 09:12:05 +01:00

Bitbake

BitBake is a generic task execution engine that allows shell and Python tasks to be run efficiently and in parallel while working within complex inter-task dependency constraints. One of BitBake's main users, OpenEmbedded, takes this core and builds embedded Linux software stacks using a task-oriented approach.

For information about Bitbake, see the OpenEmbedded website: https://www.openembedded.org/

Bitbake plain documentation can be found under the doc directory or its integrated html version at the Yocto Project website: https://docs.yoctoproject.org

Bitbake requires Python version 3.8 or newer.

Contributing

Please refer to our contributor guide here: https://docs.yoctoproject.org/contributor-guide/ for full details on how to submit changes.

As a quick guide, patches should be sent to bitbake-devel@lists.openembedded.org The git command to do that would be:

git send-email -M -1 --to bitbake-devel@lists.openembedded.org

If you're sending a patch related to the BitBake manual, make sure you copy the Yocto Project documentation mailing list:

git send-email -M -1 --to bitbake-devel@lists.openembedded.org --cc docs@lists.yoctoproject.org

Mailing list:

https://lists.openembedded.org/g/bitbake-devel

Source code:

https://git.openembedded.org/bitbake/

Testing

Bitbake has a testsuite located in lib/bb/tests/ whichs aim to try and prevent regressions. You can run this with "bitbake-selftest". In particular the fetcher is well covered since it has so many corner cases. The datastore has many tests too. Testing with the testsuite is recommended before submitting patches, particularly to the fetcher and datastore. We also appreciate new test cases and may require them for more obscure issues.

To run the tests "zstd" and "git" must be installed.

The assumption is made that this testsuite is run from an initialized OpenEmbedded build environment (i.e. source oe-init-build-env is used). If this is not the case, run the testsuite as follows:

export PATH=$(pwd)/bin:$PATH
bin/bitbake-selftest

The testsuite can alternatively be executed using pytest, e.g. obtained from PyPI (in this case, the PATH is configured automatically):

pytest