Commit Graph

115 Commits

Author SHA1 Message Date
Mariano Lopez
32a3f4d6ab oeqa/core/context: Add option to select tests to run
This add the option to select what tests to run in the
<module>[.<class>[.<test>]] format.

Currently it just support modules

(From OE-Core rev: 0050565d71cee5a0cc22660b2398e4334d4b6719)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez
41b0b31a84 oeqa/core/decorator: Add skipIfNotDataVar and skipIfNotInDataVar
skipIfNotDataVar will skip a test if a variable doesn't have certain value.

skipIfNotInDataVar will skip a test if a value is not in a certain variable.

(From OE-Core rev: a81045f4e2b740173237f5ae4e80e2bc0b287faa)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez
bfe20fd23c oeqa/core/decorator/data.py: Add skipIfNotFeature decorator
This adds a new decorator to check if image under tests has
certain DISTRO_FEATURE or IMAGE_FEATURE.

[YOCTO #10234]

(From OE-Core rev: 8740803d0696a0e97b72210a56f4fbd3135826ed)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Mariano Lopez
6ad52a82ea oeqa/core/target Add OESSHTarget to sent commands to targets using SSH
With this commit now it is possible to add targets with SSH for testing.
Most of it was imported for existing code, with improvements in log
handling.

[YOCTO #10234]

(From OE-Core rev: 3bc13548df4adb85f09467d200530a9c9f60da04)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:20 +00:00
Aníbal Limón
388503c032 oeqa/core: Add README
The README has an introduction and explains how to run the test suite
and creates a new Test component.

(From OE-Core rev: 9d474172c47695be1a61538f5b87ca8d9db25fa7)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Aníbal Limón
9a5f285fe3 oeqa/core/cases: Add example test cases
Serves as an first input of how to the OEQA framework works.

(From OE-Core rev: 115f80adf1b230c5d0392e7833e9aeb274642bcb)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Aníbal Limón
94cb20f20d oeqa/core/context: Add support of OETestContextExecutor
The OETestContextExecutor class supports to use oe-test for run core
test component also is a base class for the other test components
(runtime, sdk, selftest).

Te principal functionality is to support cmdline parsing and execution
of OETestContext, the test components could extend the common options
to provide specific ones. The common options between test components
are test data file, output log and test cases path's to scan.

Also it initializes the logger to be passed to the whole OEQA framework.

[YOCTO #10230]

(From OE-Core rev: 039deafa5f2c8fab31b8373b39f8bc219377b893)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Mariano Lopez
7bf63b28f1 oeqa/core: Add tests for the OEQA framework
This test suite covers the current functionality for the OEQA
framework.

For run certain test suite,

$ cd meta/lib/oeqa/core/tests
$ ./test_data.py

(From OE-Core rev: 7d7d0dc3736fc12ae7848de2785f0066e6470cd1)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Aníbal Limón
4da4091cee oeqa/core/decorator: Add support for OETestDataDepends and skipIfDataVar
The OETestDataDepends decorator skips a test case if a variable
isn't into test data (d).

The skipIfDataVar decorator skips a test case if a variable
has certain value.

(From OE-Core rev: 7dc519d20e835ee7693c31903e164c4bc0e5e598)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Mariano Lopez
b685a7c1a8 oeqa/core/decorator: Add support for OETimeout decorator
The OETimeout provides support for specify certain timeout
in seconds for a test case, if the timeout is reach the SIGALRM
is sent and an exception is raised to notify the timeout.

[YOCTO #10235]

(From OE-Core rev: 1bf66a370361912e9950d7ff45e382c93622a169)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Mariano Lopez
b8cebdb96c oeqa/core/decorator: Add support for OETestID and OETestTag
These two decorators stores certain TAG or ID for the test case
also provides support for filtering in loading step.

[YOCTO #10236]

(From OE-Core rev: 047af4ce864bbf98e2617b348ae9ccb77ac52871)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Aníbal Limón
95a2ec6aab oeqa/core/decorator: Add support for OETestDepends
The OETestDepends decorator could be used over test cases to
define some dependency between them.

At loading time sorting the tests to grauntee that a test case
executes before also raise an exception if found a circular
dependency between test cases.

At before test case run reviews if the dependency if meet, in the
case of don't it skips the test case run.

(From OE-Core rev: 2385bd3c8a7c012fd1cad5465ec7d34675552c75)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Aníbal Limón
13c8c08b95 oeqa/core: Add loader, context and decorator modules
loader: Implements OETestLoader handling OETestDecorator
and filtering support when load tests. The OETestLoader is
responsible to set custom methods, attrs of the OEQA
frameowork.

[YOCTO #10231]
[YOCTO #10317]
[YOCTO #10353]

decorator: Add base class OETestDecorator to provide a common
way to define decorators to be used over OETestCase's, every
decorator has a method to be called when loading tests and
before test execution starts. Special decorators could be
implemented for filter tests on loading phase.

context: Provides HIGH level API for loadTests and runTests
of certain test component (i.e. runtime, sdk, selftest).

[YOCTO #10230]

(From OE-Core rev: 275ef03b77ef5f23b75cb01c55206d1ab0261342)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Mariano Lopez
abb55ab304 oeqa/core: Add utils module for OEQA framework
misc: Functions for transform object to other types.
path: Functions for path handling.
test: Functions for operations related to test cases and suites.

[YOCTO #10232]

(From OE-Core rev: 102d04ccca3ca89d41b76a8c44e0ca0f436b7004)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00
Aníbal Limón
08714d3b7e oeqa/core: Add base OEQA framework
case: Defines OETestCase base class that provides custom
    methods/attrs defined by the framework.
    Every OETestCase instance contains a reference to the test
    data (d), the test context (tc) and the logger.
    Also implements _oe{SetUp,TearDown}Class for make special
    handling of OEQA decorators and validations.

runner: Defines OETestRunner/OETestResult with support for RAW
    and XML result logs.

exception: Custom exceptions related to the OEQA framework based
    on class OEQAException.

[YOCTO #10230]
[YOCTO #10233]

(From OE-Core rev: c466086ccc4d4bb02d578a821cfb945945bfd529)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-23 12:05:18 +00:00