Commit Graph

279 Commits

Author SHA1 Message Date
Anuj Mittal
ba598c7db3 oeqa/parselogs: remove references to BSPs that no longer exist
These platform specifc BSPs were removed from meta-intel and superseded by
intel-core* BSPs.

(From OE-Core rev: a6fd20fc57da3e1910e775f5994bd71bf79934ea)

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-18 10:18:41 +01:00
Chen Qi
946a48be9e oeqa/runtime/cases/parselog.py: ignore a message from weston
The following error message when starting core-image-weston is not
critical as long as the image could start up correctly. So extend
the common_errors list for parselog.py test case to ignore this
message.

  logind: cannot setup systemd-logind helper (-61), using legacy fallback

[YOCTO #12835]

(From OE-Core rev: edc244b2f346ad2eb3c90664ec17c904b356b992)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-05 11:39:06 +01:00
Hongzhi.Song
ec2bb22965 oeqa/runtime: Add testcases for kernel sample
We are going to let runtime test support kernel tests. Now we just add
kernel self-contained sample tests. And we plan to add overall kernel
tests in the future.

This patch is just add kernel samples test which contains about 13 tests
enabled by kernel-sample.scc. So it needs statement,
KERNEL_FEATURES_append += " features/kernel-sample/kernel-sample.scc" in
local.conf.

(From OE-Core rev: be6ef728fd36049ea3822a54909a995bdecf6dd0)

Signed-off-by: Hongzhi.Song <hongzhi.song@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-07-02 21:37:47 +01:00
Chen Qi
c7fbb0251b oeqa/runtime/cases/rpm.py: change to use base-passwd-doc
The test cases assume that rpm-doc is built out, as it says it its
comment. This is not always true. And it sometimes results in
following error.

  |     cls.tc.target.copyTo(test_file, dst)
  | UnboundLocalError: local variable 'test_file' referenced before assignment

Change to use base-passwd-doc, as this package is more likely to be
built out than rpm-doc.

(From OE-Core rev: 2e23543b48921182307065c1fa9e8b9d7fbb3cdc)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-18 11:07:57 +01:00
Chen Qi
a77b860335 oeqa/runtime/cases/parselogs.py: ignore a message from watchdog
The following message from watchdog is not an error.

  "error retry time-out = 60 seconds"

Checking watchdog's source codes, we have:

  src/watchdog.c: log_message(LOG_INFO, "error retry time-out = %d seconds", retry_timeout);

It's clear this is an info message, so parselogs should ignore it.

(From OE-Core rev: c90dff78310ab1517555c4df39b3e2d2f59ff06d)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-18 11:07:57 +01:00
Chen Qi
d3f4363223 oeqa/runtime/cases/parselogs.py: extend common_errors list
Add the following line to common_errors list.

  Failed to read /var/lib/nfs/statd/state: Success

This message is not harmful, it does not result in rpc.statd starting
failure.

(From OE-Core rev: 0f2c39149941d95fa979ec675f9dc25655886e2c)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-18 11:07:57 +01:00
Richard Purdie
c1d72abe5e oeqa/runtime/dnf: Fix test error when static libs are enabled
The test works by excluding curl-dev which curl-staticdev depends upon.
When static libraries aren't disabled, this leads to an odd looking test
failure.

Simply exclude curl-staticdev as well in case its enabled to make sure
the test always works.

(From OE-Core rev: 0a77f4b680332a692abf0231e397fab44c59bde3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-07 08:52:55 +01:00
Alexander Kanavin
ec840e6638 oeqa: replace Alex Kanavin's @intel email address with a personal one
As I will be leaving Intel, this address will no longer be valid,
so swap it for my personal one for now.

(From OE-Core rev: ea58ff101e28dfda3410de66d775df3d8a1e5a96)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-07 08:52:55 +01:00
Chen Qi
7937e471b6 oeqa/runtime/cases/multilib.py: fix test_file_connman skipping logic
The test_file_connman should be executed only when 'lib32-connman' is
installed and 'connman' is not installed.

When lib32-connman and connman are both installed, the /usr/sbin/connmand
could be from connman or lib32-connman, depending on the installation
order. What we want to check is the connmand command from lib32-connman,
so we need to make sure that connman is not there to cause chaos.

(From OE-Core rev: bc6839394c06bb695b92b2183337e7381da1e86c)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-04 15:15:00 +01:00
Chen Qi
ce57973fd9 oeqa/runtime/cases/multilib.py: skip if needed packages are not available
1) The test cases use 'readelf' command to do the check. This command
   is from binutils. So skip the test if the needed binutils package is
   not installed.

   The related error message in log.do_testimage is like below.

     Output:  sh: readelf: not found

2) The test case tests /lib/libc.so.6 from lib32-libc6. So skip the test
   if lib32-libc6 is not installed.

   The related error message in log.do_testimage is like below.

     Output:   readelf: Error: 'lib/libc.so.6': No such file

(From OE-Core rev: eae929a5c224f5c3468d6a0466d1bbb3f678a5a1)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-04 15:15:00 +01:00
Chen Qi
b60954ca5f oeqa/runtime/cases/rpm.py: skip if rpm not available
This test case should only run when rpm package is installed.
So skip it if rpm package is not installed. This fixes:

  RESULTS - rpm.RpmBasicTest.test_rpm_help - Testcase 1059: FAILED

(From OE-Core rev: bb909a60c04248d015d988e4454f0a11b1c287da)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-06-04 15:15:00 +01:00
Jose Perez Carranza
cb68e9a2fe runtime/dnf: Add new dnf test cases
Add test cases to test “exclude” and “installroot“ options, also modify
the logic of filtering packages on the feed to have all the packages
needed by the tests.

[YOCTO #10744]

(From OE-Core rev: 1121806603c6f621d084b692216f3f616a0768dc)

Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-15 10:56:48 +01:00
Alexander Kanavin
1ef125ded6 oeqa/runtime/cases/python: use python 3 rather than python 2
For example, core-image-sato skipped the test alltogether, as it
no longer pulls in Python 2.x at all.

(From OE-Core rev: 5ad0fe9ac6b6362011a17afaa7bee8e788093915)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-04 13:28:02 +01:00
Yi Zhao
602c7d65bb logrotate: update to 3.14.0
Since the wtmp and btmp definitions had been moved from logrotate.conf
to logrotate.d in this release, we also need to install them to
/etc/logrotate.d/.

Also update oeqa runtime logrotate test case.

(From OE-Core rev: 5b4aedd6b18b6ba6ca1bcd460a0b51ced41656cd)

Signed-off-by: Yi Zhao <yi.zhao@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-05-04 13:28:02 +01:00
Victor Kamensky
7e03441c9f oeqa/runtime/stap.py: add runtime test for systemtap
Add runtime test for stap to test basic SystemTap
operations: can compile very basic module and run on
target device.

Note we disable (-DSTP_NO_VERREL_CHECK) SystemTap
additional kernel release check since during OE testing
mismatching kernel-devsrc and kernels are used.

(From OE-Core rev: 659d19fcddb7edaca8f5221148d479e73304b430)

Signed-off-by: Victor Kamensky <kamensky@cisco.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-04-18 18:57:06 +01:00
Richard Purdie
0f85563eac make-mod-scripts/kernel-devsrc: Fix objtool issues
Kernels which use tools/objtool can now fail when building external modules
due to objtool being missing, the generated files can also cause problems
for kernel-devsrc.

Ensure objtool is generated in make-mod-scripts by also calling
"make prepare".

For devsrc, delete the generated binaries since they'd be native
binaries and unsuitable for the target.

The oeqa kernel module tests also need to have the additional "make prepare"
step added.

(From OE-Core rev: 52fd2993784b4218f5df4f343e7da45d964df305)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-30 00:31:18 +01:00
Yeoh Ee Peng
2bce61ee41 oeqa/runtime/opkg.py: add runtime test for opkg
Add runtime test for opkg to test that it can install ipk
package from remote source.

[YOCTO# 11488]

(From OE-Core rev: 9dd4af2b70f58540b2799823957aff3413068126)

Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-04 11:35:42 +00:00
Yeoh Ee Peng
5a351c2aaa oeqa/runtime/apt.py: add runtime test for apt
Add runtime test for apt to test that it can install deb
package from remote source.

[YOCTO# 11488]

(From OE-Core rev: f380fa77d69051212fdf7dff97da611e884d05d2)

Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-04 11:35:42 +00:00
Ross Burton
f05dc3f817 oeqa/runtime: add gobject-introspection test
(From OE-Core rev: 497a95b576e19e39e20ac280d0db24f51b7c9679)

(From OE-Core rev: 7fe336e34bbdc16a28f95ce6ec043943f8033002)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-29 10:31:58 +00:00
Richard Purdie
3b0245936e oeqa/runtime/buildcpio: Use our own mirror for source
We see occasional network glitches which break this test. Use our own
mirror (which has a .gz instead of .bz2) to avoid the errors, we're not
trying to test network connectivity.

(From OE-Core rev: 038d7270aef0cccf87d50a117160c58261beb9b9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-06 10:13:37 +00:00
Alexander Kanavin
82cb765392 runtime/cases/ptest.py: fail when ptests fail on target
That's the whole point isn't it? Previously this testcase succeeded
even if some of the underlying on-target tests failed; the only way
to find out if anything was wrong was to manually inspect the logs.

(From OE-Core rev: 21b27d1e9d54d4aab412facff22cd5d3d77827a8)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-05 11:55:35 +00:00
Alexander Kanavin
8288a832e1 runtime/cases/ptest.py: do not require ptest-pkgs in IMAGE_FEATURES; run only when ptest-runner is availalble;
Previously the test would execute only when all available ptests
for packages in the image were installed; some of those tests may
be broken, never finish, take a very long time or simply irrelevant
to the user who wants to check ptests of only a few specific packages,
and does so by listing them explicitly via IMAGE_INSTALL_append or similar.

Presence of ptest-runner means there is at least one ptest package installed
as they pull it in via a class dependency; ptest-runner is not generally
installed otherwise.

(From OE-Core rev: e07a2b9c2b08a465baeaaca86461e07817f84a52)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-01-05 11:55:35 +00:00
Robert Yang
20704967f6 runtime/cases/_ptest.py: rename it to ptest.py
It works now.

[YOCTO #11547]

(From OE-Core rev: 836393b467c3c0d53e4ace7611f991b0a5649182)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-23 14:16:01 +01:00
Robert Yang
86e062d612 runtime/cases/_ptest.py: add skip status
The packages' test cases maybe skipped, check and save them.

[YOCTO #11547]

(From OE-Core rev: 5c89060e96919c9865034a0c64fde382763da71d)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-23 14:16:01 +01:00
Robert Yang
dccf0efb41 runtime/cases/_ptest.py: revive it
* Make it work with current oeqa
* Skip the test if ptest is not in DISTRO_FEATURES
* Skip the test if ptest-pkgs is not in IMAGE_FEATURES
* The logs are saved to:
  testimage/ptest_log -> testimage/ptest_log.<datetime>
* This provides data that could be used to detect regressions in ptest results

[YOCTO #11547]

(From OE-Core rev: f1dfb59495db9e79441c8aa623ede7ef20045a20)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-23 14:16:01 +01:00
Richard Purdie
eb9264cf31 oeqa/parselogs: Ignore device node warnings on mips
THe newner kernels have ope firmware support enabled which leads to warnings
during boot on mips. These aren't interesting and we should ignore them.

(From OE-Core rev: 4ba9a6ffb4e7f6c7eee47aa13252fd981cfe5618)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-23 12:06:52 +01:00
Jose Perez Carranza
37e0cd0e91 oeqa/runtime: Replace buildiptables for buildlzip on runtime tests
Buildiptables test cases are conflicting with images built with “musl”
as standard C library, in order to avoid those issues lzip package was
selected to be used on the tests as this does not have any "musl"
dependency.

This patch is applicable for testimage tests

[YOCTO # 11713]

(From OE-Core rev: 41683e0ab316049e28b1f4ceaf39f0fe17722d92)

Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-08 13:34:46 +01:00
Aníbal Limón
d95700ff6e oeqa/runtime/context.py: Add support to specify port in target_ip
(From OE-Core rev: 7fe4c074611eb21abdf811d8f216d4df7ab6a3ea)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-06 19:02:44 +01:00
Aníbal Limón
9153e6dd16 oeqa/runtime/case: Don't use OEQA framework internal methods
The OEQA framework has internal methods for provide functionality
in decorators so Test components aren't expected to override it.

Use the base unittest methods for setUp and tearDown.

(From OE-Core rev: 21df9f0f6d6272adc6131cdc113000a5e6ac9d46)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-02 13:36:14 +01:00
California Sullivan
0cff8d03ff parselogs: whitelist failed to enable keyboard error on intel-corei7-64
This error is seen on the Braswell RVP platform we have been using for
testing. The error is caused by atkbd.c keyboard driver, which is only
for AT and PS/2 keyboards. I tested a PS/2 keyboard with the board,
which worked fine, and the board does not have a separate AT connector,
so this error won't cause any functional issues.

[YOCTO #10110].

(From OE-Core rev: c9f83639242313ca04ec36b49602a8464e10dae8)

Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-11 16:59:19 +01:00
California Sullivan
0100bfa01c parselogs: whitelist bluetooth firmware load error for intel-corei7-64
The NUC6 has issues bringing up Bluetooth early in the boot sequence. We
see:

[    4.091790] Bluetooth: hci0: Minimum firmware build 1 week 10 2014
[    4.097326] Bluetooth: hci0: Found device firmware: intel/ibt-11-5.sfi
[    4.145317] Bluetooth: hci0: Failed to send firmware data (-38)

Followed by this later on:

[   11.509870] Bluetooth: hci0: Minimum firmware build 1 week 10 2014
[   11.509988] Bluetooth: hci0: Found device firmware: intel/ibt-11-5.sfi
[   13.090308] Bluetooth: hci0: Waiting for firmware download to complete
[   13.090829] Bluetooth: hci0: Firmware loaded in 1549114 usecs
[   13.090987] Bluetooth: hci0: Waiting for device to boot
[   13.101958] Bluetooth: hci0: Device booted in 10818 usecs

Bluetooth does successfully come up and the firmware is loaded. This
behavior is consistent across all kernels I've tested.

[YOCTO #10628].

(From OE-Core rev: 045ee89342ea8ce16e78fea9f1c73d978d66a337)

Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-05-11 16:59:19 +01:00
Joshua Lock
633ad6c9f4 oeqa/runtime/utils/targetbuildproject: use parent classes defaults tmpdir
Rather than hard-coding the tmpdir for TargetBuildProject to /tmp allow the
parent's default handling to define an appropriate tmpdir.

(From OE-Core rev: 901659a51cd53625a93f57a9c5865e90a07ec09d)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-06 10:13:39 +01:00
Jose Perez Carranza
163bb0e21b runtime/dnf : Add ID to the dnf test cases
Add the corresponding Testopia ID to the test cases for dnf

(From OE-Core rev: 434b48566f2febcc5bd5d6cd9c04788f008ffce1)

Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-05 23:22:12 +01:00
Mariano Lopez
739130370f oeqa/runtime/cases: Migrate underscore cases
There were two missing cases to be migrated to the new framework: _qemutiny and
_ptest.

qemutiny was straightforward.

ptest on the other hand wasn't working even in previous releases; it has been
migrated from smart to dnf, and how ptest packages are gathered to be
installed, adapted to use unicode, and removed a lot of code that wasn't needed
anymore.

(From OE-Core rev: ee7c19546b686e852d01df25143504d9798d10d6)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-04-01 08:14:57 +01:00
Alexander Kanavin
0a435c7e6a oeqa: use WORKDIR/oe-testimage-repo to look for RPM packages
Using RPM deploy dir was causing errors when pre-built images were
used with these steps:
https://wiki.yoctoproject.org/wiki/Quality_Assurance_yocto_project

[YOCTO #11173]

(From OE-Core rev: f633abed51ed19522340b1d923ffc12ed7e291d6)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-23 13:19:49 +00:00
brian avery
c0970e0af9 meta/lib/oeqa: change sdk test from cvs to cpio
We currently fetch,configure,build, and install cvs as our test for the
sdk.  cvs unfortunately, requires a default editor in order to run.  The
change in 94790a8254 that checks to see if you have something like vi
installed is fragile since you may have a different default editor. This
patch switches from using cvs as a test to using cpio. cpio also uses
autotools so the functionality tested is equivalent.

(From OE-Core rev: 233d36b0382a8b2e430c3377e50885d1a0c3ba21)

Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22 11:35:21 +00:00
Alexander Kanavin
aa66e8782e dnf: add /usr/bin/dnf symlink that points to /usr/bin/dnf-2
All documentation refers to dnf binary as 'dnf' yet make install
does not create one - it's done by Fedora's spec file when building
the rpm. Let's replicate this behavior.

(From OE-Core rev: 456c4a8ffc9a292d7a3e036d92baf4a8f14d1f45)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-17 16:53:04 +00:00
Alexander Kanavin
ea4bac7e45 testimage.bbclass: fix runtime test for rpm, port smart tests to dnf
(From OE-Core rev: 749a496d273f9fd378588e309cf976294584ca5f)

Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-14 14:42:17 +00:00
Mariano Lopez
cf0b92f893 oeqa/runtime/context.py: Fix use of getTarget() with testexport
The idea on getTarget is to use kwargs to send custom variables
to different targets, instead of this, a new variable was added
(just used for custom targets) and this broke testexport. So
in order to fix it, just add the custom variable to kwargs.

This fixes the use of getTarget() in testexport class that was
introduced in 1dc8010afd71fe46fb28bb86fb7c07a5fbd3d7cf.

(From OE-Core rev: cf138029a1f18f991fc7a28c81d85884942e9d56)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-23 12:49:48 -08:00
Juro Bystricky
b1f99066bd testimage: Implement custom target loading
This patch implements custom target loading for testimage, currently
missing due to major changes to the test framework.
Custom targets can be defined in various meta-layers, so we
need an extra path information in order to find them.
Any other additional info is retrieved as usual via the variables
TEST_TARGET and TEST_SUITES

(From OE-Core rev: 1dc8010afd71fe46fb28bb86fb7c07a5fbd3d7cf)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-15 09:29:55 -08:00
Richard Purdie
6840618f1a oeqa/runtime: Improve failure log output
Printing a message which says "configure failed" without the log output
is effectively useless. If a command fails, print the output by default
and simplify the calling code which makes debugging any of these failures
much easier.

(From OE-Core rev: b6352ff001c29f0bff10c18879b92c5618ec645c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-02-05 09:22:17 +00:00
Aníbal Limón
90b4e075f4 oeqa/runtime/cases: Rename syslog module to oe_syslog
Debian based distros has a builtin syslog module so when
try to load tests using unittest it references the builtin
module instead of runtime/cases.

[YOCTO #10964]

(From OE-Core rev: 9923e3cdb58c2b3c54ec5fe99b2cec4cdc9fff92)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 23:27:47 +00:00
Mariano Lopez
5582b67f6d runtime/cases: Fix case numbers, missing cases and unused classes
gcc: Removed unneded lines.

multilib: fixed case number.

syslog: added a missing test.

[YOCTO #10964]

(From OE-Core rev: 3f1c57191802309e58a7b3b42e8ec812e83bd8ed)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 23:27:47 +00:00
Mariano Lopez
e8935e20d6 runtime/cases/smart.py: Check for IMAGE_PKGTYPE instead of PACKAGE_CLASSES
smart test requires to build the image using rpm packages, this check was
included, but it checked for PACKAGE_CLASSES=='package_rpm', and this is
not true when building packages for rpm and deb/ipk. So this would check
IMAGE_PKGTYPE instead.

[YOCTO #10964]

(From OE-Core rev: 570f50b30072a10c5eadfbbfea2647339ee85176)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:43:01 +00:00
Mariano Lopez
45ef4ef2c4 runtime/cases/connman.py: Stop using oeRuntimeTest
oeRuntimeTest class is not used anymore as part of runtime
migration, this particular case was missed, so fix it.

[YOCTO #10964]

(From OE-Core rev: d928e5166d8812e7e20bc95c0a8b5cb5439ec497)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-31 14:43:01 +00:00
Mariano Lopez
1cc2bac7ec oeqa/runtime/context.py: Add defaults for runtime context
This adds default values to OERuntimeTestContextExecutor class in
order to make easier the execution of exported test that were
generated with testexport class.

[YOCTO #10686]

(From OE-Core rev: c78aeaac3b75610bada62b138c9670815a07ee80)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:21 +00:00
Mariano Lopez
58789be270 testimage.bbclass: Add package install feature
This allows to use the package install feature with
the new OEQA framework.

[YOCTO #10234]

(From OE-Core rev: 077dc19445574457769eb4f231de97e8059cb75e)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez
41e4db0eea runtime/cases/smart.py: Migrate smart tests
This migrates the smart test from the old framework to
the new one. This has its own commit because smart
test was using bb and oe libraries that are available
when exporting the test cases to run in a different host.

Because of the removal of bb and oe libraries index and
packages feeds creation will be managed in testimage bbclass.

[YOCTO #10234]

(From OE-Core rev: 8d64ac4208e8dcb8a6fde6ea2959c9b3edfe2172)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez
b569aa0e00 oeqa/runtime/cases: Migrate runtime tests.
This migrates current runtime test suite to be used with the new framework.

[YOCTO #10234]

(From OE-Core rev: b39c61f2d442c79d03b73e8ffd104996fcb2177e)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez
71e456add8 oeqa/runtime/context.py: Add logger to getTarget
Current targets (ssh and qemu) require a logger in their
constructors, so in order to get a new target we need
to provide the logger.

[YOCTO #10686]

(From OE-Core rev: 26bae3c255bc1e1cc8d81db0cffc809de0182a43)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Aníbal Limón
0e8d0a03f0 oeqa/runtime/context.py: Prepare for qemu
This commit will prepare context to use qemu and not just
simpleremote.

(From OE-Core rev: 7485a08c967916fb6edff4cc573d9314ec577031)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Aníbal Limón
2d34b08533 oeqa/utils/dump: Move get_host_dumper to OERuntimeTestContextExecutor class
To avoid getVar calls inside a utils module, also moves
get_host_dumper import inside testexport isn't needed.

[YOCTO #10231]

(From OE-Core rev: f8beaf94e943a8b20d146be47a756af312ef107c)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Aníbal Limón
1247118870 oeqa/runtime/context: Move helper functions for process args to executor
(From OE-Core rev: 07ee2fa0008a50da87fd840aea5e8bb6051c68fa)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Aníbal Limón
6c3ca00884 oeqa/runtime/context: Add runtime option group and options for target type and server ip.
(From OE-Core rev: d3cc6761a13fd1da673829797b45323aed7997d0)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez
c35b42cdbb oeqa/runtime/utils/targetbuildproject.py: Don't use more than 80 characters per line
(From OE-Core rev: 95a55e0736fc59ecdcb88d43f08b447ffa5a43ed)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez
5f57fd0b08 oeqa/runtime/files: Move runtime files from old directory
As part of the refactor we require to move the files used
in runtime testing to the new directory. This also adds
the path to the runtime test context.

[YOCTO #10234]

(From OE-Core rev: f80c21b46f69c42ff0853bf8abd01cf0c082c346)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez
9ee0816ca9 oeqa/runtime: Add OEHasPackage decorator
This new decorator will be used to skip the test
if the image under test doesn't have the required
packages installed.

[YOCTO #10234]

(From OE-Core rev: 021449938ff0b4d182d7f02930a80693f109c8ba)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez
4cd982566b oeqa/runtime: Add case, context and loader classes for runtime testing
This adds OERuntimeTestCase, OERuntimeTestContext, and OERuntimeTestLoader
to be used for runtime testing.

As expected there are some changes in runtime context:

- Adds the target to be used for runtime testing, the default
is a SSH connection to the device under test running a OE image.
- Runtime context requires image manifest because several
tests are skipped if a package is missing or installed.
- Several tests require the output of the ps command and it changes
its output and arguments if busybox o procps is installed, so the
case must use the correct ps command.

[YOCTO #10234]

(From OE-Core rev: f995f178de79d6d11422cd879d06371811f50651)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Aníbal Limón
b61326efb1 oeqa/runtime: Move to runtime_cases
The new oeqa core framework will modify the structure of the runtime
folder the new runtime folder will have python code inside to support
runtime test cases.

(From OE-Core rev: 637b712096e9d230e15b1a432a561e4118db34c8)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Aníbal Limón
ba1aec3407 oeqa: Fix files handling on runtime tests.
Common files was move to oeqa/files from oeqa/runtime/files
because the same files are used across Runtime,SDK,eSDK tests.

(From OE-Core rev: f099302efe8f222c3e4ae3604429f5ede4fd8c67)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Aníbal Limón
e5f05409d0 oeqa/runtime: Fix TargetBuildProject instances
TargetBuildProject was refactored to avoid bitbake dependency so
the instance don't allow to pass data store anymore.

classes/testimage: Export proxies before run tests

The TargetBuildProject based tests download archives from network.

(From OE-Core rev: e275f29de500a338a02402ecc570405309963b35)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Aníbal Limón
51be188063 oeqa/utils: {Target,SDK,}BuildProject remove dependency of bb
Don't use bitbake references inside utils modules, in order todo
that changes getVar calls for arguments in the __init__ method like
dl_dir for all the classes and testlogdir, builddatetime in
SDKBUildProject.

Also don't export proxies inside _download_archive method, a good
practice is to setup the proxies at init of the process instead of
do it in this helper module.

[YOCTO #10231]
[YOCTO #10599]

(From OE-Core rev: 581c34d1efe9839f50ef322761269b4e4d8a56a6)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Aníbal Limón
72e9ae377d oeqa/utils: Move targetbuild to buildproject module
The new buildproject module will contain only BuildProject class
a helper class for build source code.

The remaining classes TargetBuildProject and SDKBuildProject was
move to runtime and sdk respectively.

[YOCTO #10599]

(From OE-Core rev: 525fd2a5cda00890e921b63f7f608a10bc024d73)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Aníbal Limón
9a23e0f703 oeqa/{runtime,sdk}/files: Move testsdkmakefile from runtime to sdk module
It doesn't make sense to have files related to sdk module into runtime
module.

[YOCTO #10599]

(From OE-Core rev: 9a3be58b17afbe4ef00030b0e6ad8b20b03adc49)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Aníbal Limón
8d3640b90d oeqa: Move common files to oeqa/files instead of runtime only
Those files are used by runtime and sdk test cases, so move to
base directory of oeqa module.

[YOCTO #10599]

(From OE-Core rev: ec73e8a3d3149f3866b7bfc06f169c6e05e2d338)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:19 +00:00
Jose Perez Carranza
970b76a728 runtime: Add cleanup for logrotate tests
Delete logrotate dir to avoid errors
when test are executed more than 1
time on the same target.

(From OE-Core rev: 02c98c848163a6837692fe74a91754701a44a6c0)

Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-16 18:05:12 +00:00
Richard Purdie
022a8b58c8 meta/scripts: Various getVar/getVarFlag expansion parameter fixes
There were a few straggling expansion parameter removals left for
getVar/getVarFlag where the odd whitespace meant they were missed
on previous passes. There were also some plain broken ussages such
as:

d.getVar('ALTERNATIVE_TARGET', old_name, True)
path = d.getVar('PATH', d, True)
d.getVar('IMAGE_ROOTFS', 'True')

which I've corrected (they happend to work by luck).

(From OE-Core rev: 688f7a64917a5ce5cbe12f8e5da4d47e265d240f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-09 13:39:11 +00:00
Joshua Lock
c4e2c59088 meta: remove True option to getVar calls
getVar() now defaults to expanding by default, thus remove the True
option from getVar() calls with a regex search and replace.

Search made with the following regex: getVar ?\(( ?[^,()]*), True\)

(From OE-Core rev: 7c552996597faaee2fbee185b250c0ee30ea3b5f)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-16 10:23:23 +00:00
California Sullivan
840ea755f2 parselogs.py: Don't clog QA with Joule errors
The Joule is very new hardware and there is ongoing kernel and firmware
work to fix these issues, which will be available in future kernel and
firmware releases. In the meantime, don't clog QA reports.

[YOCTO #10611]

(From OE-Core rev: facf9fa905100945738c13f9f79e938ed4a81030)

Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 22:55:21 +00:00
Jair Gonzalez
5956492c20 parselogs: Whitelist GPT warnings as the device is fully functional
The warning occurs when the GPT image is not the same size than the
media into which it's being flashed, causing the backup GPT table
not being at the end of the disk. However, this is expected as the
image is created before having the information about the destination
media. The error is harmless, so it will be whitelisted.

Fixes [YOCTO 10481].

(From OE-Core rev: 5cc5cdc788308a79f8f0706e6d794c602ef427ed)

Signed-off-by: Jair Gonzalez <jair.de.jesus.gonzalez.plascencia@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
California Sullivan
23d2a7d156 parselogs.py: Whitelist iwlwifi firmware load error messages
The iwlwifi module of any given kernel has a minimum and maximum
supported firmware version. The kernel begins by attempting to load the
maximum version, and decrements until it is successful. The 4.8 kernel's
maximum supported firmware version is 24, but thus far only 22 has been
released, meaning we get errors for 24 and 23.

Filter out iwlwifi firmware load error messages, as they are not
necessarily indicative of real problems.

(From OE-Core rev: 7df570c2310efac8f9898da15deaac2b7df16655)

Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07 10:37:59 +00:00
Jose Perez Carranza
b81477d329 runtime: Update test cases numbers for runtime tests
Update test case numbers on runtime tests to do match
with templates defined on Testopia for 2.3 release

(From OE-Core rev: d9df762b4c62b74f6d3a1521642ea86c26793a22)

Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-30 15:48:08 +00:00
California Sullivan
f3c9e66e8d parselogs: Whitelist NUC6 firmware load error message on genericx86-64
This was already whitelisted, but the 4.8 kernel changed the error
message, causing it to get caught by parselogs again.

Fixes [YOCTO #10494].

(From OE-Core rev: e1bad14231115f3b1a2bf844ef5b2022c648b55d)

Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-11-23 11:10:14 +00:00
California Sullivan
ed12917d59 parselogs.py: Add disabling eDP error to x86_common whitelist
The NUC6 firmware tells the kernel to try and initialize an embedded
DisplayPort it does not have, causing this warning. Its harmless, so
just whitelist it.

Fixes [YOCTO #9434].

(From OE-Core rev: 4c3fb7f63aad4a5d1b9720c76091cd0646859c2a)

Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-05 10:10:11 +01:00
Jianxun Zhang
e0e8cd039c parselogs: Update uvesafb errors in qemu whitelist
This change only whitelists the timeout message in infinite
wait case in uvesafb driver.

With the latest timeout patch introducing infinite wait in
uvesafb driver, we whitelist the timeout message since it works
as a warning for issues related to timeout not to be fixed in
build servers.

We remove other errors for bug-discovering purposes in some cases
where these lines are still worthy to be caught (not whitelisted):

The removed errors show up again in the infinite wait case. It
indicates a different root cause or the timeout patch doesn't work
correctly.

Timeout happens when developers explicitly set a non-negative timeout
of a limited period to wait for task completion in uvesafb driver.

Timeout or/and errors occur when kernel doesn't have the latest
timeout patch in driver.

Note: The latest timeout patch is tracked by:
a2966330bc

[YOCTO #8245]

(From OE-Core rev: 2e15b478343c6703c37b9a45e61c9de200d98027)

Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-30 16:51:15 +01:00
California Sullivan
bb744b82c7 parselogs.py: Add amd_nb error to x86_common whitelist
This has always silently failed on hardware without AMD Northbridge,
and a recent kernel patch made it not silent. It would be ideal to only
whitelist the error for genericx86 MACHINEs and disable the CONFIG
option that enables it in intel-* MACHINEs, but in order to disable
this configuration option we would have to enable EXPERT and
DEBUG_KERNEL, which we don't want. Instead just whitelist it on all
x86 MACHINEs.

Fixes [YOCTO #10261].

(From OE-Core rev: 9c432dae1045a087f8eb2de7c9bd3a9cbd46c459)

Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-28 10:16:03 +01:00
Aníbal Limón
a8cf594bce oeqa/runtime/parselogs.py: Add ignore of tsc calibration fail in x86
We are experimenting failures to calibrate CPU's using TSC in x86
VM's due to usage of nested KVM [1], this is a known issue [2][3]
in virtualization environments, for detail explnation see [4].

Also we already have an ignore for 'TSC Fast calibration fail'.

[1] http://errors.yoctoproject.org/Errors/Details/83684/
[2] https://bugzilla.redhat.com/show_bug.cgi?id=814231
[3] https://lists.nongnu.org/archive/html/qemu-devel/2010-09/msg01547.html
[4] https://www.kernel.org/doc/Documentation/virtual/kvm/timekeeping.txt

(From OE-Core rev: 2271f59a0f506f89f9fea6777701c4b40790ddd9)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-20 15:11:09 +01:00
Richard Purdie
56a4900273 Revert "oeqa.runtime.smart: work around smart race issues"
We have a proper fix now so we can remove the workaround.

This reverts commit 4d268abc2fc892c5d34449f78c8e9f2b1a9d6bac.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-14 22:22:13 +01:00
Edwin Plauchu
2c7849fdcb oeqa: Remove linux user utilized for rpm test.
When trying to re-test smart rpm tests. A fail arises
due to a linux user previously created upon the image.
We've added a few lines to delete such user and his home dir
when finishing test.

[YOCTO #9204]

(From OE-Core rev: d1a80ac434bb798634bbb792ff7df59148ec26be)

Signed-off-by: Edwin Plauchu <edwin.plauchu.camacho@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-14 22:22:07 +01:00
Richard Purdie
996aaf93ff oeqa/parselogs: Don't use cwd for file transfers
If you run:

MACHINE=A bitbake <image> -c testimage
MACHINE=B bitbake <image> -c testimage

and A has errors in parselogs, machine B can pick these up and cause
immense confusion. This is because the test transfers the log files
to cwd which is usually TOPDIR. This is clearly bad and this patch
uses a subdir of WORKDIR to ensure machines don't contaminate each
other.

Also ensure any previous logs are cleaned up from any existing
transfer directory.

(From OE-Core rev: ac8f1e58ca3a0945795087cad9443be3e3e6ead8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-14 22:22:06 +01:00
Markus Lehtonen
68e35c8e37 oeqa.runtime.smart: work around smart race issues
Yucku hack around test failures which ultimately are caused by a race in
smartpm itself. Issuing smartpm commands in quick succession causes
races in package cache of smartpm on some systems. This patch mitigates
the problem by sleeping for 1 second after each smartpm command that
modifies the system.

[YOCTO #10244]

(From OE-Core rev: 4d268abc2fc892c5d34449f78c8e9f2b1a9d6bac)

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 12:12:23 +01:00
Richard Purdie
d3991342ed oeqa/runtime/smart: Prune feeds to save memory
Full package feed indexes overload a 256MB image so reduce the number of rpms
the feed. Filter to p* since we use the psplash packages and this leaves some
allarch and machine arch packages too.

[YOCTO #8771]

(From OE-Core rev: f352c3b71cbf50846c7de31046202296b38713cc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 12:12:23 +01:00
Robert Yang
af7ec7afc0 parselogs: Whitelist qemux86 error message with qemu 2.7.0
qemu 2.7.0 introduces kernel errors:

[    2.310768] pci 0000:00:00.0: [11ab:4620] type 00 class 0x060000
[    2.311338] pci 0000:00:00.0: [Firmware Bug]: reg 0x14: invalid BAR (can't size)
[    2.311604] pci 0000:00:00.0: [Firmware Bug]: reg 0x18: invalid BAR (can't size)
[    2.311835] pci 0000:00:00.0: [Firmware Bug]: reg 0x1c: invalid BAR (can't size)
[    2.312063] pci 0000:00:00.0: [Firmware Bug]: reg 0x20: invalid BAR (can't size)
[    2.312323] pci 0000:00:00.0: [Firmware Bug]: reg 0x24: invalid BAR (can't size)
[    2.314320] pci 0000:00:0a.0: [8086:7110] type 00 class 0x060100
[    2.315363] pci 0000:00:0a.1: [8086:7111] type 00 class 0x010180

Whitelist this for now since this is preferable to the random failures
we're seeing from qemuppc with 2.6.0.

(From OE-Core rev: 4d542cdc86c34f0f4a3dde8b0aab059bca76a9fb)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 11:53:37 +01:00
Richard Purdie
e399ca1dd8 parselogs: Whitelist qemuppc error message with qemu 2.7.0
qemu 2.7.0 introduces kernel errors:

[0.474981] pci 0000:00:0f.0: reg 0x10: [io  0x0400-0x041f]
[0.483796] pci 0000:00:0f.0: reg 0x20: can't handle BAR above 4GB (bus address 0xffffffff82800000)
[0.484204] pci 0000:00:0f.0: reg 0x20: [mem size 0x00800000 64bit pref]
[0.488077] pci 0000:00:0f.0: reg 0x30: [mem 0x83000000-0x8303ffff pref]
[0.488903] pci 0000:00:10.0: [1af4:1005] type 00 class 0x00ff00
[0.490485] pci 0000:00:10.0: reg 0x10: [io  0x0480-0x049f]
[0.496512] pci 0000:00:10.0: reg 0x20: can't handle BAR above 4GB (bus address 0xffffffff83800000)
[0.496783] pci 0000:00:10.0: reg 0x20: [mem size 0x00800000 64bit pref]
[0.500345] pci 0000:00:11.0: [1af4:1001] type 00 class 0x010000
[0.501790] pci 0000:00:11.0: reg 0x10: [io  0x0500-0x053f]
[0.507362] pci 0000:00:11.0: reg 0x20: can't handle BAR above 4GB (bus address 0xffffffff84000000)
[0.507677] pci 0000:00:11.0: reg 0x20: [mem size 0x00800000 64bit pref]
[0.513905] pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to 00
[0.516493] PCI 0000:00 Cannot reserve Legacy IO [io  0x0000-0x0fff]
[0.517512] pci 0000:00:0f.0: BAR 4: assigned [mem 0x80800000-0x80ffffff 64bit pref]
[0.518877] pci 0000:00:10.0: BAR 4: assigned [mem 0x82800000-0x82ffffff 64bit pref]
[0.519890] pci 0000:00:11.0: BAR 4: assigned [mem 0x83800000-0x83ffffff 64bit pref]

Whitelist this for now since this is preferable to the random failures
we're seeing from qemuppc with 2.6.0.

(From OE-Core rev: 2472ed5393ab01ad79c011ea19c73224ed5125de)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 11:53:37 +01:00
Richard Purdie
42e2b97441 oeqa/parselogs: Add qemuarm64 warning from 4.8 kernel to whitelist
(From OE-Core rev: ae865fee26d2a32ae07236fc7aa1cf1b234a2156)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-05 11:56:01 +01:00
Bruce Ashfield
d9c2c02e96 oeqa/parselogs/qemuarm: Whitelist amba and jitter for 4.8+ kernels
With the update to the 4.8 kernel the versatile platform (and hence
qemuarm) has switched to a device tree boot.

We are using an ummodified mainline kernel versatilepb device tree,
which includes definitions of multiple amba devices. These devices
are not present in the qemu system emulation, hence throw warnings
during boot.

These warnings are not unique to oe-core, and rather than carry kernel
patches to the device tree (for now), we whitelist the known warnings
so qa testing will pass. We also can't turn amba off completely, since
it is providing valid devices (like the serial port) and AMBA is
force selected by other kconfig values.

We also have a jitterentropy warning that shows up on some hosts.
This warning is harmless, and like amba we can't turn it off in a
fragment since it is force selected by crypto (and we'd rather not
turn all crypto off). So we add it to the whitelist while investigations
continue into what is needed in the host to support this fully.

(From OE-Core rev: f5315b8c7998611da9984fd6bce2b48d6304ff6c)

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-05 11:56:01 +01:00
Ross Burton
8670979d22 oeqa/runtime/rpm: use su instead of sudo
This test works fine with su, which is more likely to be installed in images
than sudo.

(From OE-Core rev: 59d10be745a1f7d31c68e4d5da9e1c3461b7d390)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-25 23:03:45 +01:00
Richard Purdie
3e97690117 parselogs: Ignore uvesafb timeouts
We're periodically seeing uvesafb timeouts on the autobuilder. Whitelist these
errors as there is little it seems we can do about them and we therefore
choose to ignore them rather than fail the builds.

[YOCTO #8245]

There is a better solution proposed in the bug with a -1 timeout however
this avoids failed builds until such times as that is implemented.

(From OE-Core rev: 8097f2da79b7862733494d2321e3dfdb0880804d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-25 22:57:36 +01:00
Richard Purdie
94646f4828 oeqa/buildiptables: Switch from netfilter.org to yoctoproject.org mirror
We've had some upstream mirror instability so use our own mirror for the
iptables sources to ensure this doesn't affect the test results.

(From OE-Core rev: 25f6af8895d5f5c6dcedde0a21285d63522769c8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-23 17:59:43 +01:00
Richard Purdie
26b0657b2f parselogs: Ignore amb_nb warning messages under qemux86*
(From OE-Core rev: 857f4ca134e4575e71993b4fa255ebafec612d1e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-18 09:27:54 +01:00
California Sullivan
56e53db4d9 parselogs.py: Add failed to setup card detect gpio error on x86
This error has occurred on the MinnowBoard Max and Turbot since its
inception. It supposedly indicates a non-working SD card reader, but
ours works fine. Whitelist the error.

(From OE-Core rev: d577028a1d756b70da056dee73df657cf8000baf)

Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-18 09:27:54 +01:00
California Sullivan
a9b2c9d83a parselogs.py: Add dmi and ioremap errors to ignore list for core2
These errors have been occuring since the introduction of the 4.4
kernel with no apparent functionality loss. Whitelist for now.

(From OE-Core rev: 47b9058994f15507fc18ce0b08ac82a4c052966e)

Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-18 09:27:53 +01:00
California Sullivan
e290a83882 parselogs.py: Ignore Skylake graphics firmware load errors on genericx86-64
These errors can't be fixed without adding the firmware to the initramfs
and building it into the kernel, which we don't want to do for
genericx86-64. Since graphics still work acceptably without the firmware
blobs, just ignore the errors for that MACHINE.

(From OE-Core rev: d73a26a71b2b16be06cd9a80a6ba42ffae8412c4)

Signed-off-by: California Sullivan <california.l.sullivan@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-18 09:27:53 +01:00
Aníbal Limón
dde033ab4c oeqa/runtime/syslog.py: Improve test_syslog_logger on systemd
When an image uses systemd journald acts as a main syslog daemon using
/dev/log.

The test_syslog_logger try to log a predifined message into the syslog
using logger and then search using grep in /var/log/messages if this
fails for some reason (file rotated) now search the predifined message
into the journal.

(From OE-Core rev: 26d7e5060a35d20df6f2586b70ed8d2853cc0186)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-17 10:35:43 +01:00
Stephano Cetola
6b66e9317f Allow for simultaneous do_rootfs tasks with rpm
Give each rootfs its own RPM channel to use.  This puts the RPM metadata
in a private subdirectory of $WORKDIR, rather than living in DEPLOY_DIR
where other tasks may race with it.

This allows us to reduce the time that the rpm.lock is held to only the
time needed to hardlink the RPMs, allowing the majority of the rootfs
operation to run in parallel.

Also, this fixes the smart tests by generating an index for all packages
at the time of the test, rather than using the one provided by the
rootfs process.

Original credit for the enhancement should go to Steven Walter
stevenrwalter@gmail.com.

(From OE-Core rev: a92c196449c516fe51786d429078bbb1213bb029)

Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-12 15:25:22 +01:00
Aníbal Limón
1648504198 oeqa/runtime/syslog: test_syslog_logger Don't try to use logread when systemd is enabled
Busybox logread uses shmmem circular buffer to retrive [1] syslog messages
when systemd is enabled this shmem circular buffer isn't enabled because
systemd journald doesn't provide it.

[1] https://git.busybox.net/busybox/tree/sysklogd/logread.c?id=accd9eeb719916da974584b33b1aeced5f3bb346#n121

(From OE-Core rev: f49e4847ba00cdd072e5f072cb9ca69ef98af758)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-01 11:47:13 +01:00
Aníbal Limón
b4ba36a9d7 oeqa/runtime/syslog.py: Improve test_syslog_logger
Instead of make all the testing in a shell one liner, divide the
test into 3 operations to be able to know in what part is failing.

Parts,
	- Log message to syslog
	- Review if message exist in /var/log/messages
		- Review if message exist using logread

(From OE-Core rev: f3fe3590e887ee311f23723103eca41dcf58aa8b)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-01 11:47:13 +01:00
Ross Burton
38d2c59ab8 remove sukoku
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:44 +01:00
Ross Burton
c7147ceec8 oeqa/runtime/systemd: fix typo in skipUnlessPassed decorator
(From OE-Core rev: f0420d8c934154a94957cabcfdc993f0eae92cd7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:26 +01:00
Ross Burton
5c11e365e1 lib/oeqa: add Galculator to SDK and runtime tests
This is a GTK+3 application, so we don't need to ship GTK+ 2 in Sato just for
the SDK test suite.

[ YOCTO #9780 ]

(From OE-Core rev: 53fcfe4348a2ca727844f2b0bd3fca2902cbdda0)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-01 16:22:48 +01:00