poky/meta/lib/oeqa/sdk/cases/python.py
Hongxu Jia d40767ac83 testsdk: fix skipped testcase output "UNKNOWN" status while multiprocess execution
Usually skipped testcase output "SKIPPED"
[snip serial execution]
|RESULTS - buildgalculator.GalculatorTest.test_galculator - Testcase -1: SKIPPED (0.01s)
|RESULTS - python.PythonTest.test_python3 - Testcase -1: SKIPPED (0.01s)
[snip serial execution]

But if enable multiprocess execution, skipped testcase output "UNKNOWN" status
[snip enable multiprocess execution]
|RESULTS - buildgalculator.GalculatorTest.test_galculator - Testcase -1: UNKNOWN
|RESULTS - python.PythonTest.test_python3 - Testcase -1: UNKNOWN
[snip enable multiprocess execution]

Here is my investigation:

There is a class pairs TestProtocolClient and TestProtocolServer
provided by python3-subunit. The TestProtocolClient generates a
subunit stream of TestResult from a test run, and TestProtocolServer
parses the stream of subunit TestResult.

The class ProtocolTestCase is a unittest.TestCase adapter and it
uses TestProtocolServer to parse the stream of subunit TestResult.

In Yocto testsdk, it forks multiple processes to execute testcases
and use TestProtocolClient to generate TestResult stream; and then
it creates multiple threads to use ProtocolTestCase to parse stream
of subunit TestResult through pipe; finally it passes multiple
ProtocolTestCase as TestCase instance to main process and output
status result.

The problem point is TestProtocolServer parses `skip:' directive
after reading a `test:' directive. Without `test:' directive,
`skip:' directive will be ignored. All above requires SkipTest should
be raised inside a test method rather than setUpClass method.

Throwing SkipTest inside setUp works correctly

(From OE-Core rev: 4828a88556d59e4d06933164c2ebeb9361b7450e)

Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-11-14 11:14:39 +00:00

17 lines
676 B
Python

import subprocess, unittest
from oeqa.sdk.case import OESDKTestCase
class PythonTest(OESDKTestCase):
def setUp(self):
if not (self.tc.hasHostPackage("nativesdk-python3") or
self.tc.hasHostPackage("python3-native")):
raise unittest.SkipTest("No python package in the SDK")
def test_python3(self):
try:
cmd = "python3 -c \"import codecs; print(codecs.encode('Uryyb, jbeyq', 'rot13'))\""
output = self._run(cmd)
self.assertEqual(output, "Hello, world\n")
except subprocess.CalledProcessError as e:
self.fail("Unexpected exit %d (output %s)" % (e.returncode, e.output))