There are cases where the 'while loop' waiting for login prompt fails
and the bootlog variable does not get populated, thus use the the new
qemurunner member (self.msg) which stores all output coming from the qemu
process.
[YOCTO #12113]
(From OE-Core rev: 39ffa0f3779305c5e8ef86fe4572e961c5912021)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need QEMU PID in order to access "/proc/<qemupid>/cmdline"
Having a valid QEMU PID does not mean we can access the proc entry
immediately, we need to wait for the /proc/<qemupid> to appear
before we can access it.
(From OE-Core rev: d2d069fa9910d1c7a94c898355a63fca03ec5ad8)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ask QEMU to store the QEMU process PID in a file, this way we don't
have to parse running processes and analyze descendents.
This is done via QEMU command line argument "-pidfile".
[YOCTO #12001]
(From OE-Core rev: 67612dcd2a8a1aa1d683dddb0bd2f592886ff020)
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python unittest intercepts stdout and stderr however qemurunner sets
up a streamhandler before that interception occurs, hence the messages
spam the unittest output.
By moving the logging init to the class init time, we use the unittest
stdout/stderr and this means unittest can only show the log output upon
failure. This cleans up the selftest and testimage output whilst still
showing logging upon failure.
(From OE-Core rev: 9099cecc727fe0ae5f1559582426d30ba7a9f4d3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're seeing timeouts in the autobuilder testing code. Increase these
timeouts to 120, print the length of time we're having to wait, change
the error messages to really be errors and don't print empty logs, its
not helpful, print a message about the empty log instead.
(From OE-Core rev: b87a33d9c86cc1d2ea196259020e1d3dc712ccef)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The section might be None or '', so use "if section" for it.
Fixed:
File "/buildarea/lyang1/poky/meta/lib/oeqa/utils/logparser.py", line 113, in log_as_files
prefix += section
TypeError: Can't convert 'NoneType' object to str implicitly
[YOCTO #11547]
(From OE-Core rev: 320ea01f9eb33df462534bf08488ff6ada9bbe7b)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some test cases maybe skipped, let's parse it.
[YOCTO #11547]
(From OE-Core rev: f23c0f6a5aa93d38a5ea9d450f0accff0a5cc869)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tests that use bitbake("my-test-image") can run for a long time
without any indication to the user of oe-selftest about what's going
on. The test author has to log the bitbake output explicitly,
otherwise it is lost in case of test failures.
Now it is possible to use bitbake("my-test-image",
output_log=self.logger) to get more output both on the console and in
the XML output (when xmlrunner is installed). Example output:
2017-06-23 12:23:14,144 - oe-selftest - INFO - Running tests...
2017-06-23 12:23:14,145 - oe-selftest - INFO - ----------------------------------------------------------------------
2017-06-23 12:23:14,151 - oe-selftest - INFO - Running: bitbake my-test-image
2017-06-23 12:23:16,363 - oe-selftest - INFO - Loading cache...done.
2017-06-23 12:23:17,575 - oe-selftest - INFO - Loaded 3529 entries from dependency cache.
2017-06-23 12:23:18,811 - oe-selftest - INFO - Parsing recipes...done.
2017-06-23 12:23:19,659 - oe-selftest - INFO - Parsing of 2617 .bb files complete (2612 cached, 5 parsed). 3533 targets, 460 skipped, 0 masked, 0 errors.
2017-06-23 12:23:19,659 - oe-selftest - INFO - NOTE: Resolving any missing task queue dependencies
Because the implementation was already using threading, the same is
done to decouple reading and writing the different pipes instead of
trying to multiplex IO in a single thread. Previously the helper
thread waited for command completion, now that is done in the main
thread.
The most common case (no input data, joined stdout/stderr) still uses
one extra thread and a single read(), so performance should be roughly
the same as before.
Probably unintentionally, result.error was left as byte string when
migrating to Python3. OE-core doesn't seem to use runCmd() with split
output at the moment, so changing result.error to be treated the same
as result.output (i.e. decoded to a normal strings) seems like a
relatively safe API change (or rather, implementation fix).
(From OE-Core rev: 00b8c7ff17cd8f1920728fdc2653068e63d71724)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
merge: wait()
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python function subprocess.call() returns the return value of the
executed process. If return values are not checked, errors may
go unnoticed and bad things can happen.
Change all callers of subprocess.call() which do not check for
the return value to use subprocess.check_call() which raises
CalledProcessError if the subprocess returns with non-zero value.
https://docs.python.org/2/library/subprocess.html#using-the-subprocess-module
All users of the function were found with:
$ git grep "subprocess\.call" | \
egrep -v 'if.*subprocess\.call|=\ +subprocess\.call|return.*subprocess\.call'
Tested similar patch on top of yocto jethro. Only compile tested
core-image-minimal on poky master branch.
(From OE-Core rev: 578c8205fd14c48c6d30ef2889d86f1b4aee060a)
Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to maintain compatibility with oe-selftest, the
load_test_components needs to be re-used, so the script
executor needs to pass to only load components supported
by certain script (oe-test, oe-selftest).
(From OE-Core rev: d6b78ae711b93b4059690320cb8d821aaadd1684)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We use rev-parse for turning git object names into SHA-1 and checking
their existence. Using --verify option makes sure git-rev-parse does
what we expect.
(From OE-Core rev: f5b420c7f07a008c11d492b055c7cc8869f644d3)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
That a utility function permanently changes the process environment is
bad style and leads to subtle, hard to debug problems.
For example, we had one oe-selftest which used runqemu() with an
override for DEPLOY_DIR_IMAGE. Another test then just called runCmd()
and ended up passing the wrong DEPLOY_DIR_IMAGE set earlier in
os.environ.
The approach used here is to pass the desired environment dict to the
launch() method as a new, optional parameter, which then gets passed
on to subproject.Popen(). The modified env variables do not get
logged, as before.
[YOCTO #11443]
(From OE-Core rev: cab20f3b2fe668a63c58b44f2ad797fed74226fe)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move get_os_release() from oeqa.utils.metadata to oe.lsb, merging the
code with release_dict_osr() from oe.lsb. This removes some code
duplication and makes get_os_release() more robust.
(From OE-Core rev: 56b883f7765f6bd72e83dec26a5db8c7108c835d)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Don't hard-code /tmp as the tmpdir, instead use WORKDIR as the tmpdir if the
instantiater doesn't specify a value.
(From OE-Core rev: c43c966e0ed4ed836bdf90b1d4c3f2f45426f1ec)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than hardcoding /tmp as the default tmpdir make a more unique tmpdir
with tempfile.mkdtemp() when the caller doesn't specify a tmpdir value.
(From OE-Core rev: 9425c2658fea0b45468a04574cd77bffc6668a8d)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This change allows quering for variables with the format A_B, i.e.
PREFERRED_PROVIDER_virtual/kernel instead of just A.
(From OE-Core rev: 3810738eff6bdcf27c7e291dbeaedc699ab14bfc)
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
By default, QemuRunner avoids modifying the image files that it boots
into by enabling the qemu snapshot mode. However, some tests may want
to test changes that must persists across reboots, so this mode
should be optional.
This can be combined by copying the image file to a temporary location
first and then booting with that copy. It's also useful when testing
with additional drives attached to a virtual machine.
QemuTinyRunner doesn't use the snapshot parameter and therefore ignores
the new parameter.
Long term, a better way of passing these various configuration
parameters should be used, and perhaps QemuRunner and QemuTinyRunner
can be merged into one again to avoid code duplication. But for now
the patch follows the exiting style.
Also beware that QemuTarget.start() now acts in two different modes
(with or without explicit launch command), and depending on that mode
parameters like discard_writes must be ignored, i.e. not get passed to
launch().
(From OE-Core rev: 969d079a33a57f5a8f7af86d7bab04d35ab07584)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
runqemu() takes all parameters for the virtual machine from the
variables of the given recipe. By allowing the caller to provide a
hash with variables that get applied locally, the caller gets more
control.
Here's the intended usage:
<prepare internal-image in self.resultdir>
overrides = {
'DEPLOY_DIR_IMAGE': self.resultdir,
'IMAGE_LINK_NAME': 'internal-image-%s' % self.image_arch,
}
with runqemu('refkit-installer-image', ssh=False,
overrides=overrides) as qemu:
....
This can be used to replace the image completely with something else
or to copy it before allowing runqemu() to write into it.
(From OE-Core rev: 4c6ba32abd6b9de33f4b8a0b87e8a56432ed7825)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes it is useful to reconfigure the qemu virtual machine
directly. runqemu has the "qemuparams" parameter for that, and the
underlying start() methods also supported modifying that via their
"params" parameter. Only the runqemu() wrapper function lacked
a way to specify additional parameters.
One potential usage is to attach additional disks.
(From OE-Core rev: b2a1cf09ea65ed6f561886b1b4241f9adf3e5709)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sometimes, the OEQA utility code aborts with:
...
File ".../meta/lib/oeqa/utils/qemurunner.py", line 131, in start
return self.launch(launch_cmd, qemuparams=qemuparams, get_ip=get_ip, extra_bootparams=extra_bootparams)
File ".../meta/lib/oeqa/utils/qemurunner.py", line 259, in launch
sread, swrite, serror = select.select(socklist, [], [], 5)
InterruptedError: [Errno 4] Interrupted system call
strace shows that this is because of a SIGWINCH:
Connection from 127.0.0.1:52668
select(21, [20], [], [], {5, 0}) = ? ERESTARTNOHAND (To be restarted if no handler)
--- SIGWINCH {si_signo=SIGWINCH, si_code=SI_KERNEL} ---
This is related to some special conditions:
* whether qemu opens a graphical console window (enabled in Poky by default)
* where that window gets opened
* whether the window manager changes the size of the shell window (mine
is a tiling window manager and reorders and resizes windows automatically)
Ignoring the interrupted system calls avoids the problem. Code elsewhere (for example,
run() in ssh.py) already does the same thing.
(From OE-Core rev: 44fe106baf5fd5aebe26c5f28004e2b18d839b7c)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make it possible to limit the length of output lines shown in runCmd
exceptions. E.g when running bitbake we easily get thousands of lines of
log output, where only the last few (tens) are interesting or relevant
when an error occurs.
(From OE-Core rev: 403dd205828002d6ef4e8b474aedb6082289e22f)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The launch() doesn't need runqemuparams, we need handle it in start().
(From OE-Core rev: 16400f2b8bffc4cae0263bb597522071299b46ca)
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Modify the following files to test runqemu:
targetcontrol.py
utils/commands.py
utils/qemurunner.py
We need simulate how "runqemu" works in command line, so when test
"runqemu", the targetcontrol.py, utils/commands.py and
utils/qemurunner.py don't have to find the rootfs or set env vars.
[YOCTO #10249]
(From OE-Core rev: 9305d816bdf8837ea3a407091cb7f24a9a3ae8dc)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some commands might need to run longer than the default timeout of
five seconds. If that occurred, run_serial() returned with a status
code of zero (sic!) and no other indication of what went wrong.
Now the timeout is configurable (with five still the default) and
an explicit warning ("<<< run_serial(): command timed out after 5 seconds without output >>>")
gets appended at the end of the data returned to the caller.
While at it, the logic for checking for the timeout was updated a bit
because both implementations could overshoot the timeout when entering
select() right before the final deadline.
(From OE-Core rev: accf0362f964cc9d6330b6e52e83d748d890521f)
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Configured guest network interface through serial connection
when kernel is not run by qemu.
This should make it possible to test wic images with testimage.
[YOCTO #10833]
(From OE-Core rev: 2032d9be26b539bf867622c0090fb4696209eba9)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Parsed runqemu output to get guest network configuration
if it's not present in runqemu command line.
[YOCTO #10833]
(From OE-Core rev: d4d7ed48c1cff1351ddc2f60bcfa153c373a8ab8)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
qemu output can contain control characters. This cause qemurunner
API to crash when decoding the output to utf-8:
Traceback (most recent call last):
File "/usr/lib64/python3.4/threading.py", line 911, in _bootstrap_inner
self.run()
File "meta/lib/oeqa/utils/qemurunner.py", line 472, in run
threading.Thread.run(self)
File "/usr/lib64/python3.4/threading.py", line 859, in run
self._target(*self._args, **self._kwargs)
File "meta/lib/oeqa/utils/qemurunner.py", line 465, in threadtarget
self.eventloop()
File "meta/lib/oeqa/utils/qemurunner.py", line 526, in eventloop
self.logfunc(data)
File "meta/lib/oeqa/utils/qemurunner.py", line 77, in log
msg = msg.decode("utf-8")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xda in position 0:
unexpected end of data
Added errors='ignore' to decode call to fix this.
(From OE-Core rev: 4a46dd5190d97fdcb6297a0c1d8c824d425c4c51)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
qemu runner picks up first fsimage type from the hard-coded
list of supported types. This makes it impossible to test
particular image type unless it's not ext4(first type in
the hardcoded list of types).
Added image_fstypes argument to commands.runqemu and QemuTarget
__init__ to specify type of the image to run qemu with.
This will be used to pass wic image type to test efi wic images.
(From OE-Core rev: f1f224a2d4d3f2a760632c2254e91a8f94c8814f)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added possibility to pass additional runqemu parameters
down the stack of APIs:
commands.runqemu -> QemuTarget.start -> QemuRunner.start
This will be used to pass ovmf parameter in testing of
efi wic images under qemu.
(From OE-Core rev: 5aa4b5a10fb8191cd3453d09701c8beeff9a952f)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Out of the box subprocess.CalledProcessError.__str__() just displays the command
and exit code, which isn't very useful for debugging.
Add a function to oeqa.utils.subprocesstweak to monkey-patch __str__() so that
it can also display the value of stdout and stderr.
(From OE-Core rev: c55401ba1646202fa36e4973b05dbacaa146cb16)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The word 'uninstall' was spelled incorrectly in various parts of
oeqa files.
(From OE-Core rev: af365e025030436f83b233fa51ecc9c58c58ce3c)
(From OE-Core rev: 9d25188e873645b849584b51a77b86588a51d4ba)
Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
because of the way that the test cases are looked in the results,
the file logger is incompatible with extended implementations of the
unittest runner.
as the xml runner extends the unittest runner, it shares the id()
method which returns the full name of the test, not only the test
function name. With that, a single check of the full name reviews at
the same time the class name as well as the function name.
[YOCTO#11012]
(From OE-Core rev: 19b025333846018fd3e4ee4ca5cc18d375fa6213)
Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The runtime oeqa tests were failing in the on-target case, only for
aarch64. This was because an old config.sub was being used.
Similar to the SDK testing, call gnu-configize in the on-target
case too to resolve the failing tests.
(From OE-Core rev: 7918fe08d8e48f46cf2402afa66d35063eb72545)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you use subprocess.check_output() the traceback will contain the output
when the command fails which is very useful for debugging. There is no
good reason not to use this everywhere.
(From OE-Core rev: ad750dd1cc9d789abe723daddd098ce41d8547f5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows to use the package install feature with
the new OEQA framework.
[YOCTO #10234]
(From OE-Core rev: 077dc19445574457769eb4f231de97e8059cb75e)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
testimage support the installation of packages without a package
manager in the target. This adds support for package extraction
required to support the installation feature.
[YOCTO #10234]
(From OE-Core rev: 8c7335290cb00ed0683241249297ca573ebd353a)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Don't use bitbake references inside utils modules, in order todo
that changes getVar calls for arguments in the __init__ method like
dl_dir for all the classes and testlogdir, builddatetime in
SDKBUildProject.
Also don't export proxies inside _download_archive method, a good
practice is to setup the proxies at init of the process instead of
do it in this helper module.
[YOCTO #10231]
[YOCTO #10599]
(From OE-Core rev: 581c34d1efe9839f50ef322761269b4e4d8a56a6)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The new buildproject module will contain only BuildProject class
a helper class for build source code.
The remaining classes TargetBuildProject and SDKBuildProject was
move to runtime and sdk respectively.
[YOCTO #10599]
(From OE-Core rev: 525fd2a5cda00890e921b63f7f608a10bc024d73)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The bitbake logger changes the way debug is logged and adds
different levels within debug, this is passed as argument
to the function and breaks compatibility with vanilla loggers.
This implements a way to handle this adding a new function for
debug, that will dispatch the correct logging method signature.
Also overrides info method to use logging.INFO + 1 in order to
see plain data.
Also this commit fix the issue of not showing the test summary
and results when running from bitbake.
[YOCTO #10686]
(From OE-Core rev: 619c9ab308fbef9e3563dc661e432603e764b562)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When runqemu fails, qemu-system process would keep running
and won't be killed, setpgrp() was used when runqemu was
a shell script but it seems it doesn't work always with python.
This would kill qemu-system explicity and to avoid leaving
it behind.
(From OE-Core rev: 9d2b1aa1bcfb2f1933a8eeb9470b4174d5da2f0d)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This adds the missing sys module used by the child process
to exit. It seems the exception was cached in testimage and
selftest. It seems nobody noticed this because the module
is only used for sys.exit().
(From OE-Core rev: 66f66d1d763ff7bbaab9e8fcdf7fc882f2dfbb13)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Inlude values of BB_NUMBER_THREADS and PARALLEL_MAKE in the metadata.
[YOCTO #10590]
(From OE-Core rev: 97c07a55815c2fc3915705317b6f30d212fa1d45)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Make it possible to store any bitbake config variables in the metadata.
Config values will be stored under a new config element in the xml report:
<config>
<variable name="MACHINE">qemux86</variable>
</config>
The value of MACHINE is moved there instead of having a dedicated
<machine> element.
[YOCTO #10590]
(From OE-Core rev: 6e7e6e37664b0a86111272f5f6f4a4e1d0f23302)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Have the layer name as an attribute instead of of the name of the
element itself. That is, have <layer name="layer_name"/> instead of
<layer_name/>. A bit better XML design.
[YOCTO #10590]
(From OE-Core rev: 50ea44c19005b536a2791113f8b536fd10548ead)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Makes it easier to put the commits into a timeline.
[YOCTO #10590]
(From OE-Core rev: 7757970bced4ecd6503991c0cf11f4d9158f650c)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Revision is a bit vague and could point to a tag, for example. Git
commit objects are unambiguous and persistent so be explicit that the
element should contain git commit hash.
[YOCTO #10590]
(From OE-Core rev: a6521b89e86ddba6bd646dd3974d7b0390323b56)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Always return a valid branch name, or, '(nobranch)' if the current HEAD
is detached. Also, always return the hash of the commit object that HEAD
is pointing to. Previous code returned an incorrect branch name (or
crashed) e.g. in the case of detached HEAD.
[YOCTO #10590]
(From OE-Core rev: 02d3ba17a8090bd088beb973980651d664f713bb)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It's better just to not have the xml elements than to have elements with
faux data. One could have git branch named 'unknown', for example.
[YOCTO #10590]
(From OE-Core rev: fce531c21f5e56d0f416b3405a0b0fc5ba567679)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use the same format, based on /etc/os-release, as for host distro
information.
[YOCTO #10590]
(From OE-Core rev: 0156ef46ccf5334ee72f0202f1089249c62af37b)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Put all host distro data under one <host_distro> element. In addition
take the data directly from /etc/os-release instead of the "lsb API".
The /etc/os-release file is virtually ubiquitous, now, and using its
field names and values provides a more standardized and extensible
format.
[YOCTO #10590]
(From OE-Core rev: 98cad0b4063772dad94fea96edce1a5422256c32)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tests may need to run a native tool that is not available on the host
filesystem, but can be built using one of the *-native recipes. In such case,
the tool will be available in native sysroot, and running in from that location
will require adjustments to PATH.
runCmd() can now take a path to native sysroot as one of its arguments and
setup PATH accordingly.
(From OE-Core rev: f2a04631949db72d4261d1c142c5044fad3741f9)
Signed-off-by: Maciej Borzecki <maciej.borzecki@rndity.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
git module is not included into standard Python
library and therefore causes import errors on the systems
where PythonGit is not installed.
As git module only used in the code implementing --repository
functionality it's better to import git only in the scope
that requires it.
[YOCTO #10821]
(From OE-Core rev: 66be32c1a075201d6ee0e9b9e10b84e6a2ace745)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.
Search made with the following regex: getVar ?\(( ?[^,()]*), True\)
(From OE-Core rev: 7c552996597faaee2fbee185b250c0ee30ea3b5f)
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Commit 9d55e9d489 broke calling get_bb_vars()
when called without arguments. This fix this issue.
(From OE-Core rev: 91f856426c7523e1ebdf6d6f93f5fa7e509d6e49)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Use tinfoil.parse_recipe() in order to allow oe-selftest to be used in
memres mode.
(From OE-Core rev: 499ee9006271112f22cfe08fa5ba5c21be95380b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We don't need to see the parsing/cache loading message in the
oe-selftest output, so use the newly added quiet option to disable it.
(From OE-Core rev: c023bc55ff000d1de891d1a8e2a163e94bf63de6)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adds functions to get metadata from the host running the tests.
[YOCTO #9954]
(From OE-Core rev: 10b05794254886e55c76f29f7778d783c550befa)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The function get_bb_vars will remove items for the list passed
as the function argument, this will leave the caller with an
empty list and the function never says it will consume the items.
This hasn't been found before because only get_bb_var uses this
function.
(From OE-Core rev: 22b7fa24fefcc3974806d1b282c93b8c5880f6a4)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Deal with an interrupted system call gracefully:
| File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-qa-systemd/build/meta/lib/oeqa/utils/sshcontrol.py", line 55, in _run
| if select.select([self.process.stdout], [], [], 5)[0] != []:
| InterruptedError: [Errno 4] Interrupted system call
(From OE-Core rev: 556125e4004fb7ac5169b59f51dc151f18c1806a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
StreamHandler was added due missing log information on the console in
oe-selftest with Qemu Runner
(From OE-Core rev: a4e2df151af781edbcb6b0e17b51b5ed226bf77f)
Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This logs the launch command that was used for runqemu while running -c
testimage. This way, if I'd like to easily launch qemu manually in
order to debug a failed test, I know what commmand was run to create
the qemu instance.
(From OE-Core rev: 34aa20c6f323bbf7ad53beb643126e4e03634708)
Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to avoid race condition when test if exists the linkfile
use bb.utils.lock, the best solution is to create a unique name
for the link file.
There is no way to create a unique linkfile name at this decorator
because is needed the machine and image variables, those variables
can't be passed easily in this code.
To avoid broke test export functionality use a try/except because bb
isn't available when use test export
[YOCTO #10225]
(From OE-Core rev: 059d475b6bce1e5414170a4fe2e7989f6b0eacd6)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Rather than copying images, use the snapshot option to qemu. This fixes a regression
caused by the recent runqemu changes where the wrong images were being testes since
the image is copied without the qemuboot.conf file. This means the latest image is
found by runqemu rather than the specified one, leading to various confused testing
results.
It could be fixed by copying more files but use snapshot mode instead.
(From OE-Core rev: eab91997d415b0e690b3482749a32087e6a8b00a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no runqemu-internal any more.
(From OE-Core rev: 14bacf7203ab7a638b67eb143225d8c75bbb703d)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a new 'is_topdir' argument to the GitRepo init method which
validates that the given path is the top directory of a Git repository.
Without this argument GitRepo also accepts subdirectories of a Git
repository (in which case GitRepo will point to the parent directory
that is the top directory of this repository) which may have undesired
in some cases.
(From OE-Core rev: 044c81bd916fbe7140d184eb103f74786cfef604)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Extend GitRepo.run_cmd so that the caller may redefine and/or define
additional environment variables that will be used when the git command
is run.
(From OE-Core rev: 9b3c7c47f5d0fa473fe1db81b59b26531414781c)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Unfortunately to implenent multiconfig support in bitbake some APIs
had to change. This updates code in OE to match the changes in bitbake.
Its mostly periperhal changes around devtool/recipetool
[Will need a bitbake version requirement bump which I'll make when merging]
(From OE-Core rev: 041212fa37bb83acac5ce4ceb9b7b77ad172c5c3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using kvm can provide significant speedups when running qemux86* machines
on an x86* host. Enabled by using the new QEMU_USE_KVM variable.
[YOCTO #9298]
(From OE-Core rev: ebac2c8d1fcd09ebce0659a4abb445e4f1c18571)
Signed-off-by: Bill Randle <william.c.randle@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now Runner's support extra_bootargs for the kernel so add
extra_bootparams to the start() methods to avoid exception.
(From OE-Core rev: 4c28c03a2322fbcb9a5c268b08eaeb71d940ee04)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently when copying a symlink to the target it will fail
throwing an exception. This will recreate symlinks from the
system performing the tests to the device under tests.
[YOCTO #9932]
(From OE-Core rev: 5705b7a55bc300e14c34b0530f4d49df101edd3c)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When run a command sometimes the output isn't provided so validate
before trying to encode to utf-8, also some output like BIOS/EFI
contains characters that can't be codified into utf-8 for this reason
set errors='replace'.
[YOCTO #10019]
(From OE-Core rev: f2a04faf3c5d0a3cc562061b22e1c4873e1ca769)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When checking if a link exists before creating it, use os.path.lexists() as
otherwise os.path.exists() on a broken link will return False.
(From OE-Core rev: ec24b6de2b8686e1f779fef3a963e66f70eeba74)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
updateEnv() can be used in other places so move the
function to utils/commands.py
(From OE-Core rev: 380e5d80898cac4ffc9715b3f597d0b62a0643ff)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add ability to specify extra_bootargs (kernel cmdline) in order to enable systemd
debug log in images that enables systemd init.
[YOCTO #9299]
(From OE-Core rev: 09d62551c289b5607341a4f9c46eecd6390ad774)
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A new helper module for easier interaction with Git repositories.
Provides GitRepo class that represents one local Git repository clone.
The GitRepo class currently only has one method, run_cmd(), for running
arbitrary git commands in the repository. More specialized methods for
commonly used git operations can be added later.
(From OE-Core rev: 4eaf434f885afbda03fe67ab6e9ff291c7a9c77e)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Useful if one wants to separate stdout and stderr.
(From OE-Core rev: de9744c91a997a5ab0e7a19dbe13d8def8d62800)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Get rid of duplicate code.
(From OE-Core rev: cdd6b7386afd460337705d8117a4328d4993ecef)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A new function for getting values of multiple bitbake variables at the
same time.
(From OE-Core rev: fe3039322e2f846b336ac5af5177e9da27d79695)
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some upstreams need more than just gnu-configize ran before ./configure works,
such as ./autogen.sh or autoreconf. Add extra_args (defaulting to
gnu-configize) so that this can be done in test cases.
(From OE-Core rev: 7096f2889f1623ce97a6696f6f4c7217f0efb972)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The re.sub() used to transform a tarball into a best guess folder name wasn't
right, as there isn't enough escaping and tar.xz was missing.
(From OE-Core rev: ac99135b009a1066486ed2afb2f298d0a5a3a854)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since the stream can contain invalid binary characters (e.g. from
ppc's bootloader) use surrogateescape decoding to ensure we do process
the character stream, else it can hang/timeout.
(From OE-Core rev: 28a0030430d4cfcaf5dfc3e71bda07cdbfbbf4a7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch contains all the other misc pieces of the transition to
python3 which didn't make sense to be broken into individual patches.
(From OE-Core rev: fcd6b38bab8517d83e1ed48eef1bca9a9a190f57)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Converted str to bytes before sending to socket.
Converted bytes to str after receiving from socket.
This should fix TypeError: 'str' does not support the buffer interface
for qemurunner.run_serial method.
(From OE-Core rev: 210e290c9251839dc74e3aabdcea3655dd707a50)
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Iterators now return views, not lists in python3. Where we need
lists, handle this explicitly.
(From OE-Core rev: caebd862bac7eed725e0f0321bf50793671b5312)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
python3 standardises its use of iteration operations. Update
the code to match the for python3 requires.
(From OE-Core rev: 2476bdcbef591e951d11d57d53f1315848758571)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In python3, strings are unicode by default. We need to encode/decode
from command pipelines and other places where we interface with the
real world using the correct locales. This patch updates various
call sites to use the correct encoding/decodings.
(From OE-Core rev: bb4685af1bffe17b3aa92a6d21398f38a44ea874)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This new file just contain one function to return an OE package
manager, this depends in the current packaging method.
[YOCTO #8694]
(From OE-Core rev: 9b4b150ccbcd2f7a0b27d083eb406578584af3fa)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Update older exception syntax to modern one required by python 3.
Compatible with python 2.7.
(From OE-Core rev: d13f0ac614f1d1e2ef2c8ddc71cbfcf76a8dc3f2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Found some more syntax cleanups needed for python3.
(From OE-Core rev: 1181d86e8707c5b8e8d43d5e785d7d9cf01fa491)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
