Commit Graph

279 Commits

Author SHA1 Message Date
Patrick Ohly
206d532180 oeqa/runtime/systemd.py: skip instead of failing without avahi
The SystemdServiceTests assume that avahi-daemon is installed,
which is not necessarily the case depending on the image being
tested.

Better check this dependency before starting the tests and skip
them if the service is not installed. This has to be done for
each test instead of for the entire module, because other
tests in the module can run without avavi.

(From OE-Core rev: ca23337d517cfdb7119e5fd8bd9a9a663ae135de)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-04-10 18:10:25 +01:00
Lucian Musat
b0b8f47e99 oeqa/runtime: Added a new auto rpm test
The test tries to query the rpm list with a non-root user

(From OE-Core rev: 9d5deedfdae28d0257e84e1980c2cb7fd4e38c45)

Signed-off-by: Lucian Musat <george.l.musat@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-22 08:49:57 +00:00
Lucian Musat
b7a9a06b2d oeqa/runtime: Fixed a problem in logparser where it failed to whitelist some common errors.
And some minor tweaks like moving some errors from qemu to common.

(From OE-Core rev: c25474be02a562585ded66a6d538aee21056f628)

Signed-off-by: Lucian Musat <george.l.musat@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-20 11:03:46 +00:00
Randy Witt
cd5e168a8f lib/oeqa/runtime/systemd: Add a basic runtime test for the journal
The test_systemd_journal() test will fail if journalctl exits with a
non-zero exit status. The exit status is non-zero even if there are no
journal files, which is what drove the creation of this test.

[Yocto #7388]

(From OE-Core rev: 27baa182d44872f84bee3da36b95997ce0597d89)

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-16 17:41:51 +00:00
Richard Purdie
7f30749fe0 oeqa/parselogs: Skip hda opcode errors
These occur when running images under virtualisation on machines with
high load which sometimes trigger timeouts in the kernel DMA code.
They're harmless to ignore in these cases.

Adding this since "failing" the build due to this is more annoying
thank useful.

[YOCTO #7387]

(From OE-Core rev: fb29441216435b9bae47ca9cd42db5a6b1fe77d8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-12 12:50:24 +00:00
Richard Purdie
b3e92f4b46 oeqa/parselogs: Add exception for new mips error message
The 3.19 kernel introduces this error, ignore it for now.

(From OE-Core rev: 2a0a14275ca00b2d3ca867c82548a41e3bb10986)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-21 22:05:34 +00:00
Lucian Musat
26be755cb5 oeqa/ptest: Fixed complementary package install detection and added ptest-runner exit code check
(From OE-Core rev: b214c7b901fe0fad081d7a97f9e91cc218c66bd6)

Signed-off-by: Lucian Musat <george.l.musat@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-19 07:51:38 +00:00
Lucian Musat
4c00256dd9 oeqa/ptest: Removed buildhistory as requirement for ptest
(From OE-Core rev: 990c0048fc7fe363b679943fe0ed0434645711d9)

Signed-off-by: Lucian Musat <george.l.musat@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-15 21:58:28 +00:00
Lucian Musat
6cac3ad6fa oeqa/smart: Added some new test cases
(From OE-Core rev: 945cdab87ed247524059183e1376923a8655069c)

Signed-off-by: Lucian Musat <george.l.musat@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-15 21:58:28 +00:00
Paul Eggleton
9f03969994 oe-pkgdata-util: improve command-line usage
* Use argparse instead of optparse for standardised help output, options
  and a much cleaner code structure
* Look up pkgdata directory automatically so the user doesn't have to
  specify it
* Use standard logging

NOTE: this does mean a slight change in syntax - if you do want to
specify the pkgdata directory (usually only necessary if you're calling
it from within the build process) you need to use the parameter -p (or
 --pkgdata-dir) and specify this before the command, not after it.

Examples:

oe-pkgdata-util find-path /sbin/mke2fs
oe-pkgdata-util lookup-recipe libelf1
oe-pkgdata-util read-value PKGSIZE libc6
oe-pkgdata-util -p /home/user/oe/build/tmp/sysroots/qemux86-64/pkgdata read-value PKGSIZE libc6

(From OE-Core rev: 04dc571ac7c26f0dcf1a1fcd466482e22519998d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-02-14 08:41:01 +00:00
Lucian Musat
19399d5d29 oeqa/parselogs: Added a check in case the folder location does not contain any log files
(From OE-Core rev: affa3a126ba214f4d9b9a770e51323a6a5863bf2)

Signed-off-by: Lucian Musat <george.l.musat@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-08 09:24:49 +00:00
Richard Purdie
4477069b64 oeqa/parselogs: Add harmless failure warning introduced by new qemu version on qemuppc
(From OE-Core rev: d9f49e0e6b377c402e01de631edca73bb6537ab9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-31 17:04:43 +00:00
Lucian Musat
16e8b40b50 oeqa/runtime: Added some new filters for the log parser Including custom filters for edgerouter, minnow and jasperforest.
(From OE-Core rev: 20310e68a2930dd1ddc6d0d9fc577fffa5a2777f)

Signed-off-by: Lucian Musat <george.l.musat@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-11-25 13:03:29 +00:00
Richard Purdie
54a755bbb5 oeqa/dmesg: Whitelist usbhid failures
We fixed these in parselog but not here. This test really can just be
deleted now really.

(From OE-Core rev: c598dbdbaebf95cc26e95138b4c3fcb15af67a88)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-10-02 10:58:24 +01:00
Richard Purdie
399192d156 oeqa/parselogs: Ignore qemu usbhid errors
These are harmless from the USB pointer device we install, ignore them.

(From OE-Core rev: f5bdf41e78ff378fe23d8ba1543917bc64def62f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-10-02 00:42:44 +01:00
Lucian Musat
7c16a538bc oeqa/runtime: Automatic test for ptest
For images without ptest the packages are automatically installed alongside ptest-runner. Log results are saved in ./results folder.
No cleanup is done for packages after the test is finished.

(From OE-Core rev: f8e99fa8baa020c6414da19428b73c1fd30c9523)

Signed-off-by: Lucian Musat <georgex.l.musat@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-10 15:38:56 +01:00
Richard Purdie
91375aff99 oeqa: xorg log test is being replaced by parselogs so remove
Now we have the parselogs test, this one can be removed.

(From OE-Core rev: 7977a3c28677d9a248059b0be230f345227e798a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-29 23:44:37 +01:00
Richard Purdie
09296cd292 oeqa/pasrselogs: Improve the machine/string whitelist
Currently the whitelist is imcomplete, inaccurate and suffers duplication.

These changes:
 * Add common groups of errors
 * Change to make the default whitelist clear
 * Correctly (or at least better) escape the regexp expressions
 * Add in missing machines to allow builds on the autobuilder to suceed

(From OE-Core rev: 620aa5f9022335a9166b4d47bdcdce611ff5466a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-29 23:44:37 +01:00
Lucian Musat
c730c94a21 oeqa/runtime: Automatic test for parsing the logs on a machine and search for certain error keywords.
This adds a common new qa test for general processing of log files. One
significant improvement is machine dependent ignore filters.

This can be used to replace several weaker individual QA tests
that are currently used.

(From OE-Core rev: a14d076a401397b6773d5d1b99e49126261f1eb4)

Signed-off-by: Lucian Musat <georgex.l.musat@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-29 23:44:37 +01:00
Corneliu Stoicescu
fe5b333e1a oeqa/runtime: add new cpp test and file
This tests out the limits header which we've noticed does have problems in
some SDK builds.

(From OE-Core rev: 63cbed337241191f33fe951662a39ce59dce6774)

Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-11 12:30:52 +01:00
Richard Purdie
14a7cde49f lib/oeqa: Fix accidental revert of code
(From OE-Core rev: 188545ba82119d75f80dde322a73712ce1f0f762)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-25 17:29:23 +01:00
Lucian Musat
8e26ab7d52 oeqa/rutime: Added testcase decorators for automated runtime tests. Also added LogResults decorator for oeTest class in oetest.py
(From OE-Core rev: 95b83084487d0712362ade8ac487999c3274bb96)

Signed-off-by: Lucian Musat <georgex.l.musat@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-25 16:54:42 +01:00
Lucian Musat
984f03457a oeqa/runtime: Added skipModule import for test modules that use it.
The modules that use skipModule should import it themselves and not rely on somebody else to import it.

(From OE-Core rev: 6a14db407d471e717f41342ac0700e6a383c32c3)

Signed-off-by: Lucian Musat <georgex.l.musat@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-25 15:34:00 +01:00
Chen Qi
14735be703 oeqa: fix return status in pam.py to match shadow-4.2.1
The return statuses of commands like `su --help' or `passwd --help'
in shadow 4.2.1 version are different from those in shadow 4.1.4.3
version.

Now that we've upgraded shadow to 4.2.1, we need to fix these statuses
in the pam.py to make things work as expected.

(From OE-Core rev: 6bc53438735690866358194dd9e88fa1d7435e2c)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-17 12:36:19 +01:00
Stefan Stanacar
3972259964 lib/oeqa: add a test target controller for EFI targets
The purpose of this module is to deploy a test image on a EFI-enabled hardware
and run our runtime tests. A bit of background:
 - testimage.bbclass uses the concept of TEST_TARGET which is a class name
that is responsible for target deploying. A layer can provide
it's own TEST_TARGET. Right now has OE-core has a QemuTarget and a SimpleRemoteTarget
(ssh into an already up and running machine and run tests), the default one being qemu.
 - basically testimage does something like:
    target.deploy()
    try:
	target.start()
        runTests()
    finally:
	target.stop()

This module assumes a running EFI machine with gummiboot as bootloader and
core-image-testmaster installed (or similar). Also your hardware under test has
to be in a DHCP-enabled network that gives it the same IP for each reboot.

One time setup (master image):
 - build core-image-testmaster with EFI_PROVIDER = "gummiboot"
 - install the image on the target

Test image setup:
 - build your test image, e.g core-image-sato as you usually do, but with these in local.conf:
    IMAGE_FSTYPES += "tar.gz"
 - Now run the tests:
    INHERIT += "testimage"
    TEST_TARGET = "GummibootTarget"
    TEST_TARGET_IP = "192.168.2.3"
    bitbake core-image-sato -c testimage

Other notes:
 - TEST_POWERCONTROL_CMD (togheter with TEST_POWERCONTROL_EXTRA_ARGS) can be a command that runs on the host and does power cycling.
The test code passes one argument to that command: off, on or cycle (off then on). In my case I use something like
TEST_POWERCONTROL_CMD="powercontrol.exp test 10.11.12.1 nuc1" in local.conf.
Basically my expect script does: 'ssh test@10.11.12.1 "pyctl nuc1 <arg>" and runs a python script there that controls power for a label called nuc1'.
The reason why my expect script has to ssh into another machine is because of network topology, and that machine is the one actually connected
to the test rack and the power strip. That's why TEST_POWERCONTROL_CMD and _ARGS need to be customized for one's setup, the only requirement being
that it accepts: on/off/cycle as the last argument.
 - if no command is defined it would use classic reboot. This is fine as long as the machine
actually reboots (as in the ssh test hasn't failed), but it's useful for "simple-setup-with-one-board-on-the-desk" scenario, where
some manual interaction is okay from time to time.

[YOCTO #5614]

(From OE-Core rev: e00f888a88d0851b088c232dec66418e575a2e90)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-31 22:53:46 +01:00
Richard Purdie
2163893b77 oeqa/runtime/dmesg: Ignore DMA timeout errors
We see these from qemu images on servers under load. They're not interesting
and clutter the build output so lets ignore them.

(From OE-Core rev: 053eba4885ab4016f7efc7899ec9e02df3384d11)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-02 17:25:28 +00:00
Stefan Stanacar
3d0dccd026 oeqa/runtime: smart: don't run in auto mode when rpm is not the pm
TEST_SUITES = "auto" would run smart tests even for non-rpm images,
which is pointless.

(From OE-Core rev: 1913f38d79a56f7d21999b6839835f967ca28abd)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-13 11:41:21 +00:00
Stefan Stanacar
6d0f1d30d4 oeqa/utils: targetbuild: take proxy into account
A previous commit broke downloads
when proxies are involved, let's fix it.

(From OE-Core rev: 97e263b99cbe8184a74f80738fd471cfdef29e0c)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-11 11:56:56 +00:00
Ross Burton
c8962ab4db oeqa/runtime/systemd: remove race in settle()
The settle() function had a race where services could still be activating at two
minutes but then when the final log is output, they've activated.

Remove this race and generally clean up the code.

(From OE-Core rev: 8d107e0a828868702cfe035104c1f0b51da4291e)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-04 12:43:07 +00:00
Stefan Stanacar
a1dff02d03 oeqa/runtime: syslog: update --help test
busybox 1.22 now returns exitcode 0 instead of 1 for
--help options, so this test needs to be updated when
busybox gets upgraded to 1.22.

https://bugs.busybox.net/show_bug.cgi?id=5612
http://git.busybox.net/busybox/commit/?id=efd0698f74caab0a0c8a51228b923ee142e8e278

(From OE-Core rev: 1e560d234b6d3040a7a9f0eb023f1e4a654be1ea)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-02 11:22:12 +00:00
Stefan Stanacar
a2cefa01f5 oeqa/runtime: smart: fix adding channels
Without this it won't add core2-64 and similar channels, as the
directory name in deploy_dir/rpm uses _ not - as the package arch.

(From OE-Core rev: 14e3a8f58e309121760fec70619633281dd9d88c)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-29 17:38:49 +00:00
Stefan Stanacar
db1a2a6178 oeqa/runtime: drop restartTarget method
Restart is a property of a specific target, not
of a test class, should a test really need to restart
the target the direct method should be called.
Also some tests used this to enforce more ram, which
makes sense only for qemu targets only (and the inital
reason this was needed isn't valid anymore, qemu machines had
the default ram size bumped a while ago).

(From OE-Core rev: 333a4326082e500bdbcd323af37e183e74adf617)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-29 17:38:49 +00:00
Stefan Stanacar
82893ffced oeqa/runtime: rpm: fix finding the test rpm file
Translate TUNE_PKGARCH to find the right file (this
used to work because tune for qemux86-64 used to be
x86_64 now it's core2-64)
Also, while using packagedata was nice, it's harder to make
the test exportable and runnable outside of the
build system. (where oe.packagedata isn't available)

(From OE-Core rev: fd0e9ad4d295ca11b33c3e3e11069421dee834e8)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-28 18:01:47 +00:00
Ross Burton
cc8170f118 oeqa/runtime/systemd: wait for services to start/fail
When checking that no services have failed to start, actually wait for services
to finish starting by waiting for there not be no units in the "activating"
state.

(From OE-Core rev: 4d6422a84eba005a6fd788ce18c9dd42b079e2a8)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-14 11:33:50 +00:00
Ting Wang
bcacd18175 lib/oeqa/runtime: add test for python
Run a python script on the target
1)checks the output.
2)Call os.system method create a testfile

(From OE-Core rev: 4465c9368b0c37a3a2c41b68f65de08690a8179b)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-06 11:14:01 +00:00
Stefan Stanacar
b24f2172a5 lib/oeqa/runtime: rewrite the systemd test module
They are basically the same tests but:
- they look cleaner, using one single method / assert
- output from unittest will be cleaner (and includes a verbose status when needed)
- they are better grouped and use a real, active, enabled service
  (machineid will be dropped and hostnamed was a static service)

(From OE-Core rev: 0aa5b97f9e0d8941ce342e9f162dd3b19d4b49bb)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-20 12:26:29 +00:00
Ross Burton
599eed2f3b lib/oeqa/runtime: systemd failure case improvements
If the hostnamed service can't be started or stopped, show the output from
systemctl status to assist debugging.

(From OE-Core rev: 024cc1cee3a9954272ecbbff1ba7153725c56dce)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-14 09:11:19 +00:00
Ross Burton
8dee33872b lib/oeqa/runtime: output more logging from systemd when services have failed
If some services have failed to start, get the status of them and some of their
log to help debug the problem.

(From OE-Core rev: e371765431be25b81dbdb385233e3db5851e59d0)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-05 14:25:30 +00:00
Ross Burton
5c58ba9fca lib/oeqa/runtime: print connman status if connman failed to start
If connman isn't running and we're running under systemd, use systemctl to get
the state according to systemd and the end of the connman log.

(From OE-Core rev: e77bcc8e4baf11701b105d071a979a21a2a303d5)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-05 14:25:30 +00:00
Stefan Stanacar
b4d9b4208b testimage: use the new targetcontrol.py module for running tests
This patch makes the necessary changes for using the targetcontrol.py module
so that one can run the same tests on a qemu instance or a remote machine
based on the value of TEST_TARGET variable: "qemu" or "simpleremote".
The default value is "qemu" which starts a qemu instance and it's the
with what we currently have.

With "simpleremote", the remote machine must be up with network and ssh
and you need to set TEST_TARGET_IP with the IP address of the remote machine
(it can still be a qemu instance that was manually started).

Basically testimage.bbclass now does something along the lines of:
 - load tests -> deploy (prepare) / start target -> run tests.
There were a couple of changes necessary for tests and
also some cleanups/renames that were needed to adjust this change. (use
ip everywhere when refering to target and server_ip when refering to host/build machine)
Also two unnecessary and unsed methods were dropped from sshcontrol.

[ YOCTO #5554 ]

(From OE-Core rev: a7820350fa3271d78ed7476e02f4aef593be1125)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:51 +00:00
Ross Burton
afe82ef3c4 oeqa/connman: disable unique test
The connman unique test starts another instance of connmand and then does a ps to
verify that there's only one of these running, on the assumption that the new
one has quit because there's already one running (started by init).

However, connmand is forking into the background straight away so there's a race
between running ps and the second connmand discovering the first and exiting.
This race can be seen because the test displays the output of ps, and by the
time that second ps has been executed the new connmand has exited.

This is a classic race condition and on a heavily loaded autobuilder inserting
an arbitrary sleep isn't wise.  In the scheme of things this test isn't very
useful, so delete it.

(From OE-Core rev: 80ef721140c79e29430d0a5692a5c176db0061e6)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-22 12:03:02 +00:00
Stefan Stanacar
d3cb369365 lib/oeqa/runtime: build kernel module on target test
Builds a simple Hello World module on target.
Added to the defaults for core-image-sato-sdk.

(From OE-Core rev: c61c3dee162aa1f5bf31b2a09d8b916dc1712056)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-18 15:58:16 +01:00
Stefan Stanacar
41f88ee1db lib/oeqa/runtime: ping: wait for 5 echo replies
Instead of considering that ping test passed after 1 reply,
wait for at least 5 consecutive replies in 60 seconds (which should
be enough time for connman to reconfigure the interface in systemd
images and help with the fake ssh/tests fails.)

(From OE-Core rev: 6fd19a9df0ad25b2822f12e2c3a97f1b71068d4e)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-26 16:37:55 +01:00
Stefan Stanacar
ad77d10386 lib/oeqa/runtime: cleanup and improve output readability for some tests
- move everything in the same test. setUp/tearDown aren't quite
the right thing here, everything it's part of the same test. (and
it get's confusing when ssh fails)

ldd:
   - change test name and add output to error message

vnc:
   - remove unnecessary check as there is no point in doing
both ps and netstat. Also improve error output a bit.

(From OE-Core rev: 2f91bb438cfcdd0a40daed5902b6e98fc0aee67f)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:33 +01:00
Saul Wold
14fee32867 oeqa/runtime/smart: Increase timeout to 1500 test
(From OE-Core rev: a99edb5552839fd50326dd3aa4ee2f36f6026882)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-17 19:34:55 +01:00
Stefan Stanacar
214bb6828e lib/oeqa/runtime: timeout increases
Increase the timeout for smart commands as under load for qemumips
it's still to small. Also give ping more time fixing a potential
timeout for sato systemd.

(From OE-Core rev: daa3ad5807f6fc0d15b9310937d07a16edac6d22)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-14 08:21:00 +01:00
Stefan Stanacar
01135ee6f2 lib/oeqa/runtime: smart: limit channel add to useful ones
Don't add inappropiate channels on the target.
This happens when building two different machines in the
same dir and then running the tests for each machine.

(From OE-Core rev: 9e4e475b598f6b9bd2b653c72a1c1d5bd5c0b8c9)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-11 23:31:00 +01:00
Stefan Stanacar
41d1c2d630 lib/oeqa/runtime: add basic scanelf test
This uses scanelf from the pax-utils package and scans the binaries in PATH
for TEXTREL and RPATH information. For a sato image with pax-utils installed
it shows no output (which is good).

(From OE-Core rev: 629099ad66f5fa2814e5f7908b426149e8978e43)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-06 23:04:50 +01:00
Mihai Lindner
faed110e66 oeqa/runtime/scp: replace dd call
Use a file object to generate a our test file instead of calling `dd`;
removes dd's output from testimage.log, keeps unittest output clean.
Also remove unused imports.

(From OE-Core rev: 6ac48ffbab29a37b0eada533191878aeae3c91f0)

Signed-off-by: Mihai Lindner <mihaix.lindner@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-06 23:03:23 +01:00
Mihai Lindner
f6fed84380 oeqa/oetest: oeRuntimeTest: enable long messages
Set longMessage to True for all tests derived from oeRuntimeTest, in
order to have somewhat info on assertions with cryptic or no messages.

(From OE-Core rev: e5196ebd6c652431a3b80cdc9f60b752401314f4)

Signed-off-by: Mihai Lindner <mihaix.lindner@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-06 23:03:23 +01:00
Stefan Stanacar
e3bc93c2eb lib/oeqa/runtime: smart: serve repo on host ip only and increase timeout
Don't start the http server on 0.0.0.0, listen on host ip (end of tap interface) only.
Also use the timeout option (default is 300s for ssh commands) for all the commands
run in this module (mostly because smart update timeouts on mips).

(From OE-Core rev: 8c272641ef3e8410f331ca4133d28dea8f36e4f4)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-03 19:57:38 +01:00
Stefan Stanacar
efff0d5504 lib/oeqa/runtime: ping: fix ping false fail
We run the ping test as soon as we reach the login prompt.
But sometimes (seen in sato systemd) we end up with link down/link up stuff like:

    qemux86-64 login: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
    e1000: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
    IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

The logic behind ping -w 30 -c 1 was to wait at most 30 seconds
for at least one reply,  but there is a catch: reply doesn't seems
to be echo reply but any reply (non-reply means loss not network error)
ping's man page:
    -w deadline
              Specify  a  timeout, in seconds, before ping exits regardless of
              how many packets have been sent or received. In this  case  ping
              does  not  stop after count packet are sent, it waits either for
              deadline expire or until count probes are answered or  for  some
              error notification from network.

Just when the link up/link down happens ping returns:
    From 192.168.7.1 icmp_seq=1 Destination Host Unreachable
    --- 192.168.7.2 ping statistics ---
    1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms

and exits sooner than the 30 seconds timeout.

This patch should do what was originally intended (wait at most
30 seconds for at least one reply).

(From OE-Core rev: 56d144fd22d37189e49cdf3032afb00f0be469c6)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-30 18:08:05 +01:00
Stefan Stanacar
8f32af94c0 lib/oeqa/runtime: syslog: fix test for logger in case of systemd
Recently syslog behaviour changed for systemd images (log
it's in a buffer not in /var/log/messages), account
for the new stuff.

(From OE-Core rev: 32576c4cc1621fa3013eac66c7caaa1e1fd14995)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-30 16:23:47 +01:00
Stefan Stanacar
c52981c9bc lib/oeqa: move skeletoninit.py where it belongs
OE-core commit fcc59cbcdb1550489d372edf9f465efa7165245f /
poky commit 748ddc39e5 added a new test, but
in the wrong location.
I took the patch from Alex's branch but renamed it from meta/lib/oeqa/runtime/skeleton.py to meta/lib/oeqa/skeletoninit.py before sending. This was
unintentional, it should have been under meta/lib/oeqa/runtime.

(From OE-Core rev: f12c346ef48cb44be2e356e4cf4f28d015c3f507)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-30 16:23:44 +01:00
Stefan Stanacar
5fba9d8c6c classes/testimage: add support for finding tests in other layers
A layer can add tests in lib/oeqa/runtime (provided it extends BBPATH as
normal) and enable them with TEST_SUITES_append = " testname". Test
module names shouldn't collide though.

(From OE-Core rev: e1e347a2d509303e1c566450b0f2b485d3d6629f)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-28 23:36:08 +01:00
Stefan Stanacar
4e62e9cb76 lib/oeqa/runtime: remove some unnecessary checks from setUpModule
These checks are unnecessary.
setUpModule is run when a module is loaded and we
shouldn't run commands on the target here, (plus if
ssh doesn't work we error out in setup multiple times, instead
of skipping the real test, which might depend on test_ssh).

(From OE-Core rev: 188acd0a75e188fd7c0d2979acaf13fd18b12106)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-28 23:36:08 +01:00
Stefan Stanacar
44c3f72684 lib/oeqa/runtime: smart: add checks for smart output
Sometimes smart throws:

    Committing transaction...
    Preparing...                    ######################################## [  0%]
       1:Removing psplash-default   ######################################## [100%]
    error: Couldn't fork %postun: Cannot allocate memory

and returns a 0 exit code (it thinks it succesfully removed the package,
when in reality it didn't), so we need to catch those specifically.

Also, sometimes output from download command is:
    Saving cache...http://192.168.7.1:49456/rpm/x86_64_x32/psplash-default-0.1+git0+afd4e228c6-r15.x86_64_x32.rpm
and that tricks our smart download test, so use a regex there.

(From OE-Core rev: 2ac7783e04f5e8e6005f967e1a6dd65d2fc6a19a)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 16:29:18 +01:00
Stefan Stanacar
a855180fe6 lib/oeqa/runtime: multilib: fix typo
The check was obviously wrong and it surfaced
with the recent change in behaviour for skipping tests.

(From OE-Core rev: 4a14535cd493cb2bdd46b2a5f2a1cd2b38161f0a)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 16:29:18 +01:00
Mihai Prica
d7077bc8e5 lib/oeqa/runtime: add iptables, cvs and sudoku projects build tests on target
Downloads iptables/cvs/sudoku-savant sources and builds them on target.

(From OE-Core rev: df4568205c3a7e0b20c6299e29f96bd30560146b)

Signed-off-by: Mihai Prica <mihai.prica@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 11:47:21 +01:00
Mihai Lindner
ecb21fd7f8 lib/oeqa/runtime: smart: add new smart tests
Add class to be inherited by smart tests, along with more basic tests and tests
using a rpm repository.

(From OE-Core rev: f6186b4204dcc421b4e616774315c8a2a77fb5c5)

Signed-off-by: Mihai Lindner <mihaix.lindner@linux.intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 11:47:21 +01:00
Stefan Stanacar
db7e4849c3 lib/oeqa/runtime: rpm: add install and erase tests
Copies to target rpm-doc file from deploy_dir
and tries to install and then remove that package.
rpm-doc was chosen because it's small, it only adds
a few files to target, and it's almost always found in
deploy_dir for images with package-management/rpm.

(From OE-Core rev: a2d2f2b7b111863d3c50dedded37aab813d9634f)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 11:47:20 +01:00
Mihai Prica
3d30fd2eb0 lib/oeqa/runtime: add vncserver for target test
(From OE-Core rev: 56bc5d717b34563ed36c0618305e4ec5080c3a27)

Signed-off-by: Mihai Prica <mihai.prica@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 11:47:20 +01:00
Mihai Prica
5e1b0cb2e4 lib/oeqa/runtime: add adjust date and time test
(From OE-Core rev: ede300f44f1770b1e3e5c59f65cf5079379a2bc1)

Signed-off-by: Mihai Prica <mihai.prica@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 11:47:20 +01:00
Alexandru Palalau
0ecc510b92 lib/oeqa/runtime: add new scp test
Copies a 5MB to target using scp, more of an
network test than a scp one.

(From OE-Core rev: 2ec4a0686b9a91e56dfba3fa2e574c0c531508ff)

Signed-off-by: Alexandru Palalau <alexandrux.palalau@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 11:47:20 +01:00
Alexandru Palalau
4ea9e94f55 lib/oeqa/runtime: add new PAM support test
New test which verifies some usual commands functionality with PAM support

(From OE-Core rev: 06c7914e3354ff4e430a6b664f40e5a71e212761)

Signed-off-by: Alexandru Palalau <alexandrux.palalau@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 11:47:20 +01:00
Alexandru Palalau
b91969e4f7 lib/oeqa/runtime: add new logrotate test
New logrotate test which verifies the log directory change in logrotate.conf.
Needs an image with logrotate installed.

(From OE-Core rev: 186d79a603b5cbf5a93e6f5dbba5f62ed8d4d8d8)

Signed-off-by: Alexandru Palalau <alexandrux.palalau@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 11:47:20 +01:00
Cornel Stoicescu
3b47a0a4bc lib/oeqa/runtime: add a test for ldd
This test checks that at least one path in RTLDLIST exists.

(From OE-Core rev: d53ffbf21a6b17f0f0537929a95a502958d82a09)

Signed-off-by: Cornel Stoicescu <corneliux.stoicescu@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 11:47:20 +01:00
Cornel Stoicescu
dfd627d7ac lib/oeqa/runtime: add test for perl
This test runs a perl script on the target and checks the output.

(From OE-Core rev: 95abdbf2d715c9e7e3368f5193167feb5efc6261)

Signed-off-by: Cornel Stoicescu <corneliux.stoicescu@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 11:47:20 +01:00
Alexandru Georgescu
ec8aea9c01 lib/oeqa/runtime: add basic test for x32 images
Checks that an x86-64-x32 image has the right binaries.

(From OE-Core rev: da42e38b2bc4a606b717ac302cfc77aefb1a5795)

Signed-off-by: Alexandru Georgescu <alexandru.c.georgescu@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 11:47:19 +01:00
Alexandru Palalau
dbee00c9e9 lib/oeqa/runtime: add new systemd tests
New systemd runtime tests for enable/disable service, start/stop service and list services.

(From OE-Core rev: 6386dc718f85210c9b6b9f69878ec9a7847b78de)

Signed-off-by: Alexandru Palalau <alexandrux.palalau@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-13 23:06:00 +01:00
Stefan Stanacar
1990a6c507 lib/oeqa/runtime: rework syslog test
Add separate tests for restarting syslog and using logger, and
skip the configuration test for systemd images which always fail
because syslog's systemd service doesn't read a config by default
(see YB#4860).

(From OE-Core rev: c75f3e2385dde44ee96e33f4e5d064894dfb7d52)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-13 23:06:00 +01:00
Stefan Stanacar
b8962edd6b oeqa/runtime/xorg: use the right ps command for xorg test
Use our determined ps command, otherwise test fails for images with procps.

(From OE-Core rev: 18e4d014bea3a5748f42e6dac648fa9b02230344)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-03 10:33:03 +01:00
Stefan Stanacar
3dee534f1e lib/oeqa: fix dependecy check
Adds missing skip for smart test and fix the check (which I somehow broke
a while ago).

(From OE-Core rev: cf1790d992f067be8d5f9894458f55f6f1bdc61f)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-07-16 10:04:17 +01:00
Alexandru Palalau
2079a30b1f lib/oeqa/runtime: add tests for syslog and df
Add tests for free space and syslog.

Changed in v2:
    - limit df's output to /
    - syslog: fix restart in case of systemd

(From OE-Core rev: 1b39d57e7b5c9b69d565cf4d188ebc2f14e66ae6)

Signed-off-by: Alexandru Palalau <alexandru.palalau@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-07-16 10:04:16 +01:00
Stefan Stanacar
7f57d3c524 lib/oeqa/oetest.py: provide a ps command for all tests
Many tests will use 'ps' but we need to know if it's busybox
or standard ps.
Drop the existing check from the connman test.

(From OE-Core rev: 1515d33d2c5b7275a3ac20e07c1db1d8273de796)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-07-16 10:04:16 +01:00
Stefan Stanacar
b69f91f786 lib/oeqa/runtime/multilib.py: use readelf instead of file
file was pulled by rpm in sato images, readelf comes with binutils
so it's in ipk/deb too.
Also the test was looking at connmand instead of connman-applet
(which is what the testcase says and the AB nightly-multilib target installs)

(From OE-Core rev: c2c47a002996b8dbcf9b311dff677202b1f11a64)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-07-16 10:04:15 +01:00
Stefan Stanacar
46d3bb502f lib/oeqa/runtime/dmesg.py: whitelist dmesg message for systemd net interface name
Messages such as:
"systemd-udevd[79]: error changing net interface name eth0 to enp0s3: Device or resource busy"
are harmless as systemd can't rename interfaces in qemu (the interface is already active).
Alternatively, passing net.ifnames=0 to the kernel will stop systemd renaming the interfaces.

(From OE-Core rev: c072fed6531f2ce3c687f8342a97f593ebf37653)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-07-16 10:04:15 +01:00
Stefan Stanacar
4285e856aa lib/oeqa/runtime: add gcc test
gcc compile test and support files.

(From OE-Core rev: ccbce75335971abf5098a492755e92ca60cf67bd)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-07-09 10:53:45 +01:00
Stefan Stanacar
06d6f9d52e lib/oeqa/runtime: image sanity tests
These are basic sanity tests. A test can be force run by setting
TEST_SUITES = "ping ssh <module-name>" in local.conf.
By default there are suites for minimal, sato and sato-sdk images.

(From OE-Core rev: dd3dc2804395f050df74fa936e65ce5e911442eb)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Radu Moisan <radu.moisan@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-07-09 10:53:45 +01:00