Add a selftest for netdev generic netlink. For now there is only a
single test that exercises the `queue-get` API.
The test works with netdevsim by default or with a real device by
setting NETIF.
Add a timeout param to cmd() since ethtool -L can take a long time on
real devices.
Signed-off-by: David Wei <dw@davidwei.uk>
Link: https://lore.kernel.org/r/20240507163228.2066817-3-dw@davidwei.uk
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Run tools/testing/selftest/net/csum.c as part of drv-net.
This binary covers multiple scenarios, based on arguments given,
for both IPv4 and IPv6:
- Accept UDP correct checksum
- Detect UDP invalid checksum
- Accept TCP correct checksum
- Detect TCP invalid checksum
- Transmit UDP: basic checksum offload
- Transmit UDP: zero checksum conversion
The test direction is reversed between receive and transmit tests, so
that the NIC under test is always the local machine.
In total this adds up to 12 testcases, with more to follow. For
conciseness, I replaced individual functions with a function factory.
Also detect hardware offload feature availability using Ethtool
netlink and skip tests when either feature is off. This need may be
common for offload feature tests and eventually deserving of a thin
wrapper in lib.py.
Missing are the PF_PACKET based send tests ('-P'). These use
virtio_net_hdr to program hardware checksum offload. Which requires
looking up the local MAC address and (harder) the MAC of the next hop.
I'll have to give it some though how to do that robustly and where
that code would belong.
Tested:
make -C tools/testing/selftests/ \
TARGETS="drivers/net drivers/net/hw" \
install INSTALL_PATH=/tmp/ksft
cd /tmp/ksft
sudo NETIF=ens4 REMOTE_TYPE=ssh \
REMOTE_ARGS="root@10.40.0.2" \
LOCAL_V4="10.40.0.1" \
REMOTE_V4="10.40.0.2" \
./run_kselftest.sh -t drivers/net/hw:csum.py
Signed-off-by: Willem de Bruijn <willemb@google.com>
Link: https://lore.kernel.org/r/20240507154216.501111-1-willemdebruijn.kernel@gmail.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
We're a bit too loose with error checking for background
processes. cmd() completely ignores the fail argument
passed to the constructor if background is True.
Default to checking for errors if process is not terminated
explicitly. Caller can override with True / False.
For bkg() the processing step is called magically by __exit__
so record the value passed in the constructor.
Reported-by: Willem de Bruijn <willemb@google.com>
Tested-by: Willem de Bruijn <willemb@google.com>
Link: https://lore.kernel.org/r/20240502025325.1924923-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Bugs in memory allocation failure paths are quite common.
Add a test exercising those paths based on qstat and page pool
failure hook.
Running on bnxt:
# ./drivers/net/hw/pp_alloc_fail.py
KTAP version 1
1..1
# ethtool -G change retval: success
ok 1 pp_alloc_fail.test_pp_alloc
# Totals: pass:1 fail:0 xfail:0 xpass:0 skip:0 error:0
I initially wrote this test to validate commit be43b7489a ("net/mlx5e:
RX, Fix page_pool allocation failure recovery for striding rq") but mlx5
still doesn't have qstat. So I run it on bnxt, and while bnxt survives
I found the problem fixed in commit 7301177307 ("eth: bnxt: fix counting
packets discarded due to OOM and netpoll").
Reviewed-by: Willem de Bruijn <willemb@google.com>
Link: https://lore.kernel.org/r/20240429144426.743476-7-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
When picking TCP ports to use, avoid all below 10k.
This should lower the chance of collision or running
afoul whatever random policies may be on the host.
Reviewed-by: Willem de Bruijn <willemb@google.com>
Link: https://lore.kernel.org/r/20240429144426.743476-5-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
The main use of the ip() wrapper over cmd() is that it can parse JSON.
cmd("ip -j link show") will return stdout as a string, and test has
to call json.loads(). With ip("link show", json=True) the return value
will be already parsed.
More tools (ethtool, bpftool etc.) support the --json switch.
To avoid having to wrap all of them individually create a tool()
helper.
Switch from -j to --json (for ethtool).
While at it consume the netns attribute at the ip() level.
Reviewed-by: Willem de Bruijn <willemb@google.com>
Link: https://lore.kernel.org/r/20240429144426.743476-4-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
More complex tests often have to spawn a background process,
like a server which will respond to requests or tcpdump.
Add support for creating such processes using the with keyword:
with bkg("my-daemon", ..):
# my-daemon is alive in this block
My initial thought was to add this support to cmd() directly
but it runs the command in the constructor, so by the time
we __enter__ it's too late to make sure we used "background=True".
Second useful helper transplanted from net_helper.sh is
wait_port_listen().
The test itself uses socat, which insists on v6 addresses
being wrapped in [], it's not the only command which requires
this format, so add the wrapped address to env. The hope
is to save test code from checking if address is v6.
Reviewed-by: Willem de Bruijn <willemb@google.com>
Link: https://lore.kernel.org/r/20240420025237.3309296-7-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
While writing tests with a lot more cases I got tired of having
to jump back and forth to add the name of the test to the ksft_run()
list. Most unittest frameworks do some name matching, e.g. assume
that functions with names starting with test_ are test cases.
Support similar flow in ksft_run(). Let the author list the desired
prefixes. globals() need to be passed explicitly, IDK how to work
around that.
Reviewed-by: Willem de Bruijn <willemb@google.com>
Link: https://lore.kernel.org/r/20240420025237.3309296-6-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Nothing surprising here, hopefully. Wrap the variables from
the environment into a class or spawn a netdevsim based env
and pass it to the tests.
Reviewed-by: Willem de Bruijn <willemb@google.com>
Link: https://lore.kernel.org/r/20240420025237.3309296-4-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Define the remote endpoint "model". To execute most meaningful device
driver tests we need to be able to communicate with a remote system,
and have it send traffic to the device under test.
Various test environments will have different requirements.
0) "Local" netdevsim-based testing can simply use net namespaces.
netdevsim supports connecting two devices now, to form a veth-like
construct.
1) Similarly on hosts with multiple NICs, the NICs may be connected
together with a loopback cable or internal device loopback.
One interface may be placed into separate netns, and tests
would proceed much like in the netdevsim case. Note that
the loopback config or the moving of one interface
into a netns is not expected to be part of selftest code.
2) Some systems may need to communicate with the remote endpoint
via SSH.
3) Last but not least environment may have its own custom communication
method.
Fundamentally we only need two operations:
- run a command remotely
- deploy a binary (if some tool we need is built as part of kselftests)
Wrap these two in a class. Use dynamic loading to load the Remote
class. This will allow very easy definition of other communication
methods without bothering upstream code base.
Stick to the "simple" / "no unnecessary abstractions" model for
referring to the remote endpoints. The host / remote object are
passed as an argument to the usual cmd() or ip() invocation.
For example:
ip("link show", json=True, host=remote)
Reviewed-by: Willem de Bruijn <willemb@google.com>
Link: https://lore.kernel.org/r/20240420025237.3309296-2-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add a test for dumping qstats device by device.
ksft framework grows a ksft_raises() helper, to be used
under with, which should be familiar to unittest users.
Link: https://lore.kernel.org/r/20240420023543.3300306-5-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Test cases need to exit with non-zero status if they failed,
we currently don't do that:
# KTAP version 1
# 1..3
# # At /root/ksft-net-drv/drivers/net/./ping.py line 18:
# # Check failed 1 != 2
# not ok 1 ping.test_v4
# ok 2 ping.test_v6
# ok 3 ping.test_tcp
# # Totals: pass:2 fail:1 xfail:0 xpass:0 skip:0 error:0
ok 1 selftests: drivers/net: ping.py
^^^^
It's a bit tempting to make the exit part of ksft_run(),
but that only works well for very trivial setups. We can
revisit this later, if people forget to call ksft_exit().
Link: https://lore.kernel.org/r/20240417231146.2435572-3-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Totals currently only pay attention to exceptions, if check fails
(say ksft_eq()) the test case will be counted as pass:
# At /ksft/drivers/net/./ping.py line 18:
# Check failed 1 != 2
not ok 1 ping.test_v4
ok 2 ping.test_v6
ok 3 ping.test_tcp
# Totals: pass:3 fail:0 xfail:0 xpass:0 skip:0 error:0
^^^^^^^^^^^^^
Pay attention to the result.
Fixes: b86761ff63 ("selftests: net: add scaffolding for Netlink tests in Python")
Link: https://lore.kernel.org/r/20240417231146.2435572-2-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
ping prints all the info to stdout. To make debug easier capture
stdout in the Exception raised when command unexpectedly fails.
Reviewed-by: Willem de Bruijn <willemb@google.com>
Link: https://lore.kernel.org/r/20240416004556.1618804-2-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add a Python test for the basic ops.
# ./net/nl_netdev.py
KTAP version 1
1..3
ok 1 nl_netdev.empty_check
ok 2 nl_netdev.lo_check
ok 3 nl_netdev.page_pool_check
# Totals: pass:3 fail:0 xfail:0 xpass:0 skip:0 error:0
Reviewed-by: Petr Machata <petrm@nvidia.com>
Link: https://lore.kernel.org/r/20240412141436.828666-7-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Using "with" on an entire driver test env is supported already,
but it's also useful to use "with" on an individual nsim.
Reviewed-by: Petr Machata <petrm@nvidia.com>
Link: https://lore.kernel.org/r/20240412141436.828666-6-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Instead of a summary line print the full exception.
This makes debugging Python tests much easier.
Reviewed-by: Petr Machata <petrm@nvidia.com>
Link: https://lore.kernel.org/r/20240412141436.828666-5-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Developing Python tests is a bit annoying because when test fails
we only print the fail message and no info about which exact check
led to it. Print the location (the first line of this example is new):
# At /root/ksft-net-drv/./net/nl_netdev.py line 38:
# Check failed 0 != 10
not ok 3 nl_netdev.page_pool_check
Reviewed-by: Petr Machata <petrm@nvidia.com>
Link: https://lore.kernel.org/r/20240412141436.828666-4-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
net/lib/py/nsim.py already contains the most useful parts
of the netdevsim wrapper classes. Reuse them.
Acked-by: Stanislav Fomichev <sdf@google.com>
Acked-by: Martin KaFai Lau <martin.lau@kernel.org>
Link: https://lore.kernel.org/r/20240409031549.3531084-5-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Add drivers/net as a target for mixed-use tests.
The setup is expected to work similarly to the forwarding tests.
Since we only need one interface (unlike forwarding tests)
read the target device name from NETIF. If not present we'll
try to run the test against netdevsim.
Reviewed-by: Petr Machata <petrm@nvidia.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
Add glue code for accessing the YNL library which lives under
tools/net and YAML spec files from under Documentation/.
Automatically figure out if tests are run in tree or not.
Since we'll want to use this library both from net and
drivers/net test targets make the library a target as well,
and automatically include it when net or drivers/net are
included. Making net/lib a target ensures that we end up
with only one copy of it, and saves us some path guessing.
Add a tiny bit of formatting support to be able to output KTAP
from the start.
Reviewed-by: Petr Machata <petrm@nvidia.com>
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: David S. Miller <davem@davemloft.net>
