ref-manual/packages: move ptest section to the test-manual

[ YOCTO #15106 ]

It makes more sense to document ptests in the test-manual. Since ptests
are still related to packages, keep a link to ptests from packages.rst
to the test-manual.

Reported-by: Yoann Congal <yoann.congal@smile.fr>
(From yocto-docs rev: 8b6ada020d595d86c7bbe78a27b7a6301715b039)

Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
(cherry picked from commit b389c06b709e4791e1cce5e8a5b58f6b0cd03a14)
Signed-off-by: Antonin Godard <antonin.godard@bootlin.com>
Signed-off-by: Steve Sakoman <steve@sakoman.com>
This commit is contained in:
Antonin Godard 2024-12-26 16:20:16 +01:00 committed by Steve Sakoman
parent 1418534d22
commit 2497f29afd
10 changed files with 125 additions and 117 deletions

View File

@ -16,7 +16,7 @@ This section describes a few tasks that involve packages:
- :ref:`dev-manual/packages:generating and using signed packages`
- :ref:`Setting up and running package test
(ptest) <dev-manual/packages:testing packages with ptest>`
(ptest) <test-manual/ptest:testing packages with ptest>`
- :ref:`dev-manual/packages:creating node package manager (npm) packages`
@ -887,114 +887,8 @@ related to signed package feeds are available:
Testing Packages With ptest
===========================
A Package Test (ptest) runs tests against packages built by the
OpenEmbedded build system on the target machine. A ptest contains at
least two items: the actual test, and a shell script (``run-ptest``)
that starts the test. The shell script that starts the test must not
contain the actual test --- the script only starts the test. On the other
hand, the test can be anything from a simple shell script that runs a
binary and checks the output to an elaborate system of test binaries and
data files.
The test generates output in the format used by Automake::
result: testname
where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and
the testname can be any identifying string.
For a list of Yocto Project recipes that are already enabled with ptest,
see the :yocto_wiki:`Ptest </Ptest>` wiki page.
.. note::
A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest`
class.
Adding ptest to Your Build
--------------------------
To add package testing to your build, add the :term:`DISTRO_FEATURES` and
:term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which
is found in the :term:`Build Directory`::
DISTRO_FEATURES:append = " ptest"
EXTRA_IMAGE_FEATURES += "ptest-pkgs"
Once your build is complete, the ptest files are installed into the
``/usr/lib/package/ptest`` directory within the image, where ``package``
is the name of the package.
Running ptest
-------------
The ``ptest-runner`` package installs a shell script that loops through
all installed ptest test suites and runs them in sequence. Consequently,
you might want to add this package to your image.
Getting Your Package Ready
--------------------------
In order to enable a recipe to run installed ptests on target hardware,
you need to prepare the recipes that build the packages you want to
test. Here is what you have to do for each recipe:
- *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:*
Include the following line in each recipe::
inherit ptest
- *Create run-ptest:* This script starts your test. Locate the
script where you will refer to it using
:term:`SRC_URI`. Here is an
example that starts a test for ``dbus``::
#!/bin/sh
cd test
make -k runtest-TESTS
- *Ensure dependencies are met:* If the test adds build or runtime
dependencies that normally do not exist for the package (such as
requiring "make" to run the test suite), use the
:term:`DEPENDS` and
:term:`RDEPENDS` variables in
your recipe in order for the package to meet the dependencies. Here
is an example where the package has a runtime dependency on "make"::
RDEPENDS:${PN}-ptest += "make"
- *Add a function to build the test suite:* Not many packages support
cross-compilation of their test suites. Consequently, you usually
need to add a cross-compilation function to the package.
Many packages based on Automake compile and run the test suite by
using a single command such as ``make check``. However, the host
``make check`` builds and runs on the same computer, while
cross-compiling requires that the package is built on the host but
executed for the target architecture (though often, as in the case
for ptest, the execution occurs on the host). The built version of
Automake that ships with the Yocto Project includes a patch that
separates building and execution. Consequently, packages that use the
unaltered, patched version of ``make check`` automatically
cross-compiles.
Regardless, you still must add a ``do_compile_ptest`` function to
build the test suite. Add a function similar to the following to your
recipe::
do_compile_ptest() {
oe_runmake buildtest-TESTS
}
- *Ensure special configurations are set:* If the package requires
special configurations prior to compiling the test code, you must
insert a ``do_configure_ptest`` function into the recipe.
- *Install the test suite:* The :ref:`ref-classes-ptest` class
automatically copies the file ``run-ptest`` to the target and then runs make
``install-ptest`` to run the tests. If this is not enough, you need
to create a ``do_install_ptest`` function and make sure it gets
called after the "make install-ptest" completes.
See the :ref:`test-manual/ptest:Testing Packages With ptest` section of the
Yocto Project Test Environment Manual.
Creating Node Package Manager (NPM) Packages
============================================

View File

@ -220,7 +220,7 @@ Package Test (ptest)
Package Tests (ptest) are built but not installed by default. For
information on using Package Tests, see the
":ref:`dev-manual/packages:testing packages with ptest`"
":ref:`test-manual/ptest:testing packages with ptest`"
section in the Yocto Project Development Tasks Manual. For information on the
``ptest`` class, see the ":ref:`ref-classes-ptest`" section.

View File

@ -2478,7 +2478,7 @@ runtime tests for recipes that build software that provides these tests.
This class is intended to be inherited by individual recipes. However,
the class' functionality is largely disabled unless "ptest" appears in
:term:`DISTRO_FEATURES`. See the
":ref:`dev-manual/packages:testing packages with ptest`"
":ref:`test-manual/ptest:testing packages with ptest`"
section in the Yocto Project Development Tasks Manual for more information
on ptest.
@ -2491,7 +2491,7 @@ Enables package tests (ptests) specifically for GNOME packages, which
have tests intended to be executed with ``gnome-desktop-testing``.
For information on setting up and running ptests, see the
":ref:`dev-manual/packages:testing packages with ptest`"
":ref:`test-manual/ptest:testing packages with ptest`"
section in the Yocto Project Development Tasks Manual.
.. _ref-classes-python3-dir:

View File

@ -157,7 +157,7 @@ metadata:
- *ptest:* Enables building the package tests where supported by
individual recipes. For more information on package tests, see the
":ref:`dev-manual/packages:testing packages with ptest`" section
":ref:`test-manual/ptest:testing packages with ptest`" section
in the Yocto Project Development Tasks Manual.
- *smbfs:* Include SMB networks client support (for mounting

View File

@ -789,7 +789,6 @@ Errors and Warnings
use a relative path rather than an absolute one, or to pick up the path from
runtime configuration or environment variables.
Configuring and Disabling QA Checks
===================================

View File

@ -175,7 +175,7 @@ consists of the following pieces:
operation and functions. However, the test can also use the IP
address of a machine to test.
- :ref:`ptest <dev-manual/packages:testing packages with ptest>`:
- :ref:`ptest <test-manual/ptest:testing packages with ptest>`:
Runs tests against packages produced during the build for a given
piece of software. The test allows the packages to be run within a
target image.

View File

@ -6434,7 +6434,7 @@ system and gives an overview of their function and contents.
:term:`PTEST_ENABLED`
Specifies whether or not :ref:`Package
Test <dev-manual/packages:testing packages with ptest>` (ptest)
Test <test-manual/ptest:testing packages with ptest>` (ptest)
functionality is enabled when building a recipe. You should not set
this variable directly. Enabling and disabling building Package Tests
at build time should be done by adding "ptest" to (or removing it

View File

@ -12,6 +12,7 @@ Yocto Project Test Environment Manual
intro
test-process
ptest
understand-autobuilder
reproducible-builds
yocto-project-compatible

View File

@ -141,7 +141,7 @@ the following types of tests:
- *Package Testing:* A Package Test (ptest) runs tests against packages
built by the OpenEmbedded build system on the target machine. See the
:ref:`Testing Packages With
ptest <dev-manual/packages:Testing Packages With ptest>` section
ptest <test-manual/ptest:Testing Packages With ptest>` section
in the Yocto Project Development Tasks Manual and the
":yocto_wiki:`Ptest </Ptest>`" Wiki page for more
information on Ptest.

View File

@ -0,0 +1,114 @@
.. SPDX-License-Identifier: CC-BY-SA-2.0-UK
***************************
Testing Packages With ptest
***************************
A Package Test (ptest) runs tests against packages built by the
OpenEmbedded build system on the target machine. A ptest contains at
least two items: the actual test, and a shell script (``run-ptest``)
that starts the test. The shell script that starts the test must not
contain the actual test --- the script only starts the test. On the other
hand, the test can be anything from a simple shell script that runs a
binary and checks the output to an elaborate system of test binaries and
data files.
The test generates output in the format used by Automake::
result: testname
where the result can be ``PASS``, ``FAIL``, or ``SKIP``, and
the testname can be any identifying string.
For a list of Yocto Project recipes that are already enabled with ptest,
see the :yocto_wiki:`Ptest </Ptest>` wiki page.
.. note::
A recipe is "ptest-enabled" if it inherits the :ref:`ref-classes-ptest`
class.
Adding ptest to Your Build
==========================
To add package testing to your build, add the :term:`DISTRO_FEATURES` and
:term:`EXTRA_IMAGE_FEATURES` variables to your ``local.conf`` file, which
is found in the :term:`Build Directory`::
DISTRO_FEATURES:append = " ptest"
EXTRA_IMAGE_FEATURES += "ptest-pkgs"
Once your build is complete, the ptest files are installed into the
``/usr/lib/package/ptest`` directory within the image, where ``package``
is the name of the package.
Running ptest
=============
The ``ptest-runner`` package installs a shell script that loops through
all installed ptest test suites and runs them in sequence. Consequently,
you might want to add this package to your image.
Getting Your Package Ready
==========================
In order to enable a recipe to run installed ptests on target hardware,
you need to prepare the recipes that build the packages you want to
test. Here is what you have to do for each recipe:
- *Be sure the recipe inherits the* :ref:`ref-classes-ptest` *class:*
Include the following line in each recipe::
inherit ptest
- *Create run-ptest:* This script starts your test. Locate the
script where you will refer to it using
:term:`SRC_URI`. Here is an
example that starts a test for ``dbus``::
#!/bin/sh
cd test
make -k runtest-TESTS
- *Ensure dependencies are met:* If the test adds build or runtime
dependencies that normally do not exist for the package (such as
requiring "make" to run the test suite), use the
:term:`DEPENDS` and
:term:`RDEPENDS` variables in
your recipe in order for the package to meet the dependencies. Here
is an example where the package has a runtime dependency on "make"::
RDEPENDS:${PN}-ptest += "make"
- *Add a function to build the test suite:* Not many packages support
cross-compilation of their test suites. Consequently, you usually
need to add a cross-compilation function to the package.
Many packages based on Automake compile and run the test suite by
using a single command such as ``make check``. However, the host
``make check`` builds and runs on the same computer, while
cross-compiling requires that the package is built on the host but
executed for the target architecture (though often, as in the case
for ptest, the execution occurs on the host). The built version of
Automake that ships with the Yocto Project includes a patch that
separates building and execution. Consequently, packages that use the
unaltered, patched version of ``make check`` automatically
cross-compiles.
Regardless, you still must add a ``do_compile_ptest`` function to
build the test suite. Add a function similar to the following to your
recipe::
do_compile_ptest() {
oe_runmake buildtest-TESTS
}
- *Ensure special configurations are set:* If the package requires
special configurations prior to compiling the test code, you must
insert a ``do_configure_ptest`` function into the recipe.
- *Install the test suite:* The :ref:`ref-classes-ptest` class
automatically copies the file ``run-ptest`` to the target and then runs make
``install-ptest`` to run the tests. If this is not enough, you need
to create a ``do_install_ptest`` function and make sure it gets
called after the "make install-ptest" completes.