The current jitter.sh script expects cpumask bits to fit into whatever
the awk interpreter uses for an integer, which clearly does not hold for
even medium-sized systems these days. This means that on a large system,
only the first 32 or 64 CPUs (depending) are subjected to jitter.sh
CPU-time perturbations. This commit therefore computes a given CPU's
cpumask using text manipulation rather than arithmetic shifts.
Reported-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
TREE03 tests RCU priority boosting, which is a real-time feature.
It would also be good if it tested something closer to what is
actually used by the real-time folks. This commit therefore adds
tree.use_softirq=0 to the TREE03 kernel boot parameters in TREE03.boot.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit uses the shiny new "all" and "N" cpumask options to decouple
the "nohz_full" and "rcu_nocbs" kernel boot parameters in the TREE04.boot
and TREE08.boot files from the CONFIG_NR_CPUS options in the TREE04 and
TREE08 files.
Reported-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The -lgcc command-line argument is placed poorly in the build options,
which can result in build failures, for exapmle, on ARM when uidiv()
is required. This commit therefore places the -lgcc argument after the
source files.
Fixes: b94ec36896 ("rcutorture: Make use of nolibc when available")
Tested-by: Valentin Schneider <valentin.schneider@arm.com>
Tested-by: Mark Rutland <mark.rutland@arm.com> [arm64]
Signed-off-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
RCU's rcutree.use_softirq=0 kernel boot parameter substitutes the per-CPU
rcuc kthreads for softirq, which is used in real-time installations.
However, none of the rcutorture scenarios test this parameter.
This commit therefore adds rcutree.use_softirq=0 to the RUDE01 and
TASKS01 rcutorture scenarios, both of which indirectly exercise RCU.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The sizes of vmlinux files built with KASAN enabled can approach a full
gigabyte, which can result in disk overflow sooner rather than later.
Fortunately, the xz command compresses them by almost an order of
magnitude. This commit therefore uses xz to compress vmlinux file built
by torture.sh with KASAN enabled.
However, xz is not the fastest thing in the world. In fact, it is way
slower than rotating-rust mass storage. This commit therefore also adds a
--compress-kasan-vmlinux argument to specify the degree of xz concurrency,
which defaults to using all available CPUs if there are that many files in
need of compression.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
In 2020, running KCSAN often requires careful choice of compiler.
This commit therefore adds a --kcsan-kmake-arg parameter to torture.sh
to allow specifying (for example) "CC=clang" to the kernel build process
to correctly build a KCSAN-enabled kernel.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds the command and arguments to the torture.sh log file, and
also outputs the results directory. This latter allows impatient users
to quickly find the results that are being generated by the current run.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds --configs-rcutorture, --configs-locktorture, and
--configs-scftorture arguments to torture.sh, allowing the desired
set of scenarios to be passed to each. The default for each has been
changed from a large-system-appropriate set to just CFLIST for each.
Users are encouraged to create scripts that provide appropriate settings
for their specific systems.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Now that kvm.sh puts all the relevant details in the "log" file,
there is no need for torture.sh to generate a separate "log.long"
file. This commit therefore drops this from torture.sh.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit causes torture.sh to check for zero-length runs and to take
the cowardly option of refusing to run them, logging its cowardice for
later inspection.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit causes torture.sh to use the torture.verbose_sleep_frequency
kernel boot parameter to throttle verbose refscale output on large systems.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit places "---" markers in the torture.sh script's allmodconfig
output, and uses "<<" to avoid overwriting earlier output from this
build test.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit saves a few lines of code by creating a doyesno helper bash
function for argument parsing.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
On large systems, the refscale printk() rate can overrun the file system's
ability to accept console log messages. This commit therefore uses the
new verbose_batched module parameter to rate-limit some of the higher-rate
printk() calls.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The .mod.c files created by allmodconfig builds interfers with the approach
torture.sh uses to enumerate types of rcuscale and refscale runs. This
commit therefore tightens the pattern matching to avoid this interference.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit uncomments the argument checking for the --duration argument
to torture.sh. While in the area, it also corrects the duration units
from seconds to minutes.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit improves torture.sh flexibility by autoscaling the number
of CPUs to be used in variable-CPUs torture tests, including scftorture,
refscale, rcuscale, and kvfree.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds the ability to do "make allmodconfig" to torture.sh,
given that normal rcutorture runs do not normally catch missing exports.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The bash "eval" command enables Bobby Tables attacks, which might not
be a concern in torture testing by themselves, but one could imagine
these combined with a cut-and-paste attack. This commit therefore gets
rid of them. This comes at a price in terms of bash quoting not working
nicely, so the "--bootargs" argument lists are now passed to torture_one
via a bash-variable side channel. This might be a bit ugly, but it will
also allow torture.sh to grow its own --bootargs parameter.
While in the area, add proper header comments for the bash functions.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit makes torture.sh use the new bash functions get_starttime()
and get_starttime_duration() created for kvm.sh.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Although tailoring a specific set of kvm.sh runs has served rcutorture
testing well over many years, it requires a relatively distraction-free
environment, which is not always available. This commit therefore
adds a prototype torture.sh script that by default tortures pretty much
everything the rcutorture scripting is designed to torture, and which
can be given command-line arguments to take a more focused approach.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds periodic toggling of 7 of 8 CPUs every second to TREE01
in order to test NOCB toggle code.
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: Joel Fernandes <joel@joelfernandes.org>
Cc: Neeraj Upadhyay <neeraju@codeaurora.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Inspired-by: Paul E. McKenney <paulmck@kernel.org>
Tested-by: Boqun Feng <boqun.feng@gmail.com>
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, if a scenario is repeated as in "--configs '4*TREE01'",
the Kconfig analysis is performed for each occurrance (four times in
this example) and each analysis places the exact same data into the
exact same files. This is not really an issue in this repetition-four
example, but it can needlessly consume tens of seconds of wallclock time
for something like "--config '128*TINY01'".
This commit therefore does Kconfig analysis only once per set of
repeats of a given scenario, courtesy of the "sort -u" command and an
automatically generated awk script.
While in the area, this commit also wordsmiths a comment.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Normally, kvm-recheck.sh is run from kvm.sh, which provides the
TORTURE_TRUST_MAKE environment variable that, if a non-empty string,
indicates that the --trust-make command-line parameter has been passed
to kvm.sh. If there was no --trust-make, kvm-recheck.sh insists
that the Make.out file contain at least one "CC" command. Thus, when
kvm-recheck.sh is run standalone to evaluate a prior --trust-make run,
it will incorrectly insist that a proper kernel build did not happen.
This commit therefore causes kvm-recheck.sh to also search the "log"
file in the top-level results directory for the string "--trust-make".
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Commit 757055ae8d ("init/console: Use ttynull as a fallback when
there is no console") results in the string "Warning: Failed to add
ttynull console. No stdin, stdout, and stderr for the init process!"
appearing on the console, which the rcutorture scripting interprets as
a warning, which causes every rcutorture run to be flagged. However,
the rcutorture init process never attempts to do any I/O, and thus does
not care that it has no stdin, stdout, or stderr.
This commit therefore causes the rcutorture scripting to ignore this
message.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit simplifies exit-code plumbing. It makes kvm-recheck.sh return
the value 1 for a build error and 2 for a runtime error. It also makes
kvm-find-errors.sh avoid checking runtime files for --build-only runs.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit changes the "STOP" file that is used to cleanly halt a running
rcutorture run to "STOP.1" because no scenario directory will ever end
with ".1". If there really was a scenario named "STOP", its directories
would instead be named "STOP", "STOP.2", "STOP.3", and so on. While in
the area, the commit also changes the kernel-run-time checks for this
file to look directly in the directory above $resdir, thus avoiding the
need to pass the TORTURE_STOPFILE environment variable to remote systems.
While in the area, move the STOP.1 file to the top-level directory
covering all of the scenarios.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
When all of the remote systems have the same number of CPUs, one
approach is to use one "--buildonly" run and one "--dryrun sched" run,
and then distributing the batches out one per remote system. However,
the output of "--dryrun sched" is not made for parsing, so this commit
adds a "--dryrun batches" that provides the same information in easily
parsed form.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
By default, the "panic" kernel parameter is zero, which causes the kernel
to loop indefinitely after a panic(). The rcutorture scripting will
eventually kill the corresponding qemu process, but only after waiting
for the full run duration plus a few minutes. This works, but delays
notifying the developer of the failure.
This commit therefore causes the rcutorture scripting to pass the
"panic=-1" kernel parameter, which caused the kernel to instead
unceremoniously shut down immediately. This in turn causes qemu to
terminate, so that if all of the runs in a given batch panic(), the
rcutorture scripting can immediately proceed to the next batch.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds the test summary to the end of the log in the top-level
directory containing the kvm.sh test artifacts. While in the area, it adds
the kvm.sh exit code to this test summary.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, passing something like "--kconfig CONFIG_NR_CPUS=2" to kvm.sh
has no effect on scenario scheduling. For scenarios that do not specify
the number of CPUs, this can result in kvm.sh wastefully scheduling only
one scenario at a time even when the --kconfig argument would allow
a number to be run concurrently. This commit therefore makes kvm.sh
consider the --kconfig arguments when scheduling scenarios across the
available CPUs.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The kvm.sh script uses kvm-find-errors.sh to evaluate whether or not
a build failed. Unfortunately, kvm-find-errors.sh returns success if
there are no failed runs (including when there are no runs at all) even if
there are build failures. This commit therefore makes kvm-find-errors.sh
return failure in response to build failures.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Given that kvm.sh in invoked from scripts, it is only natural for
different levels of scripting to provide their own Kconfig option values,
for example. Unfortunately, right now, the last such argument on the
command line wins.
This commit therefore makes the --bootargs, --configs, --kconfigs,
--kmake-args, and --qemu-args argument values accumulate. For example,
where "--configs TREE01 --configs TREE02" would previously have run only
scenario TREE02, now it will run both scenarios.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, the "date" command producing the output on the kvm.sh "Test
Summary" line is executed at the beginning of the test, which produces a
date that is less than helpful to someone wanting to know the duration
of the test. This commit therefore defers this command's execution to
the end of the test.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The rcutorture scripts' identify_qemu_vcpus() function expects `lscpu`
to have a "CPU: " line, for example:
CPU(s): 8
But different local language settings can give different results:
Processeur(s) : 8
As a result, identify_qemu_vcpus() may return an empty string, resulting
in the following warning (with the same local language settings):
kvm-test-1-run.sh: ligne 138 : test: : nombre entier attendu comme expression
This commit therefore changes identify_qemu_vcpus() to use getconf,
which produces local-language-independend output.
Cc: Josh Triplett <josh@joshtriplett.org>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Lai Jiangshan <jiangshanlai@gmail.com>
Cc: rcu@vger.kernel.org
Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds a config2csv.sh script that converts the specified
torture-test scenarios' Kconfig options and kernel-boot parameters to
.csv format. This allows easier comparison of scenarios when one fails
and another does not.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Distributed execution of rcutorture is eased if the qemu execution can
be split from the building of the kernel, as this allows target systems
to be used that are not set up to build kernels. It also avoids issues
with toolchain version skew across the cluster, aside of course from
qemu and KVM version skew.
This commit therefore records needed data as comments in the qemu-cmd file
and moves recording of the starting time to just before qemu is launched.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Scripts like kvm-check-branches.sh group runs under a single directory
in resdir in order to allow easier retrospective analysis. However, they
do this by letting kvm.sh create a directory as usual and then moving it
after the run. This can be very confusing when looking at the results
while kvm-check-branches.sh is running. This commit therefore enables
--datestamp to hand subdirectories to kvm.sh.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Knowing the number of builds that kvm.sh will split a run into allows
estimation of the duration of a test, give or take build duration.
This commit therefore adds a line of output to "--dryrun sched" that
gives the number of builds that will be run. This excludes "builds"
for repeated scenarios that reuse an earlier build.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Knowing the number of batches that kvm.sh will split a run into allows
estimation of the duration of a test, give or take the number of builds.
This commit therefore adds a line of output to "--dryrun sched" that
gives the number of batches that will be run.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The --kcsan argument to kvm.sh adds CONFIG_KCSAN_VERBOSE=y in order to
get more detail from the KCSAN reports. However, this Kconfig option
requires lockdep to be enabled. This commit therefore causes --kcsan
to also enable lockdep.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
For the rcutorture test summary log file console.log of virtual machines is
parsed. When a console.log contains "DEBUG", BUG counter is incremented
because regular expression does not handle to ignore DEBUG.
Signed-off-by: Anna-Maria Behnsen <anna-maria@linutronix.de>
Reviewed-by: Benedikt Spranger <b.spranger@linutronix.de>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently the kvm-check-branches.sh script calculates the number of CPUs
and passes this to the kvm.sh --cpus command-line argument. This works,
but this commit saves a line by instead using the new kvm.sh --allcpus
command-line argument.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit allows --build-only as a synonym for --buildonly, --kconfigs
for --kconfig, and --kmake-args for --kmake-arg.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The "--duration <minutes>" has worked well for a very long time, but
it can be inconvenient to compute the minutes for (say) a 28-hour run.
It can also be annoying to have to let a simple boot test run for a full
minute. This commit therefore permits an "s" suffix to specify seconds,
"m" to specify minutes (which remains the default), "h" suffix to specify
hours, and "d" to specify days.
With this change, "--duration 5" still specifies that each scenario
run for five minutes, but "--duration 30s" runs for only 30 seconds,
"--duration 8h" runs for eight hours, and "--duration 2d" runs for
two days.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Although the rcutorture scripting now deals correctly with full-up
security-induced pointer obfuscation, it is still counter-productive for
kernel hackers who are analyzing console output. This commit therefore
sets the debug_boot_weak_hash kernel boot parameter, which enables
printing of weak-hashed pointers for torture-test runs.
Please note that this change applies only to runs initiated by the
kvm.sh scripting. If you are instead using modprobe and rmmod, it is
your responsibility to build and boot the underlying kernel to your taste.
Please note further that this change does not result in a security hole
in normal use. The rcutorture testing runs with a negligible userspace,
no networking, and no user interaction. Besides which, there is no data
of value that can be extracted from an rcutorture guest OS that could
not also be extracted from the host that this guest is running on.
Suggested-by: Anna-Maria Gleixner <anna-maria@linutronix.de>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Even when the kernel panics and qemu dies, runs with jitter enabled will
continue uselessly until the jitter.sh processes terminate. This can
be annoying if a planned one-hour run instead dies during boot.
This commit therefore kills the jitter.sh processes when the run ends
more than one minute prior to the termination time specified by the
kvm.sh --duration argument or its default.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The SRCU-u scenario expects to enable lockdep but to also disable the
CONFIG_PREEMPT_COUNT kconfig option. This no longer works. This commit
therefore instead enables lockdep in SRCU-t, which then allows SRCU-u
to disable CONFIG_PREEMPT_COUNT.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The "NOHZ tick-stop error: Non-RCU local softirq work is pending"
warning happens frequently and appears to be irrelevant to the various
torture tests. This commit therefore filters it out.
If there proves to be a need to pay attention to it a later commit will
add an "advice" category to allow the user to immediately see that
although something happened, it was not an indictment of the system
being tortured.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The rcuscale test module does not use batches, so there is only
ever one batch. This commit therefore informs the kvm-recheck-rcuscale.sh
script of this fact of life.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds the ability to test performance and scalability of RCU
Tasks Trace updaters.
Reported-by: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The rcutorture scripting will do a "kill -9" on any guest OS that exceeds
its --duration by more than a few minutes, which is very valuable when
bugs result in hangs. However, this is a problem when the "hang" was due
to a --gdb debugging session.
This commit therefore refrains from killing the guest OS when a debugging
session is in progress. This means that the user must manually kill the
kvm.sh process group if a hang really does occur.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, the CONFIG_PREEMPT_NONE=y rcutorture TRACE01 rcutorture
scenario enables lockdep. This limits its ability to find bugs due to
non-preemptible sections of code being RCU readers, and pretty much all
code thus appearing to lockdep to be an RCU reader. This commit therefore
moves lockdep testing to the CONFIG_PREEMPT=y rcutorture TRACE02 scenario.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds a "--gdb" parameter to kvm.sh, which causes
"CONFIG_DEBUG_INFO=y" to be added to the Kconfig options, "nokaslr"
to be added to the boot parameters, and "-s -S" to be added to the qemu
arguments. Furthermore, the scripting prints messages telling the user
how to start up gdb for the run in question.
Because of the interactive nature of gdb sessions, only one "--configs"
scenario is permitted when "--gdb" is specified. For most torture types,
this means that a "--configs" argument is required, and that argument
must specify the single scenario of interest.
The usual cautions about breakpoints and timing apply, for example,
staring at your gdb prompt for too long will likely get you many
complaints, including RCU CPU stall warnings. Omar Sandoval further
suggests using gdb's "hbreak" command instead of the "break" command on
systems supporting hardware breakpoints, and further using the "commands"
option because the resulting non-interactive breakpoints are less likely
to get you RCU CPU stall warnings.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds a --help argument (along with its synonym -h) to display
the help text. While in the area, this commit also updates the help text.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, the CONFIG_PROVE_RCU_LIST=y case is untested. This commit
therefore adds CONFIG_PROVE_RCU_LIST=y to rcutorture's TREE05 scenario.
Cc: Madhuparna Bhowmik <madhuparnabhowmik10@gmail.com>
Cc: Joel Fernandes (Google) <joel@joelfernandes.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The rcu-test-image.txt documentation covers a very uncommon case where
a real userspace environment is required. However, someone reading this
document might reasonably conclude that this is in fact a prerequisite.
In addition, the initrd.txt file mentions dracut, which is no longer used.
This commit therefore provides the needed updates.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Rationale:
Reduces attack surface on kernel devs opening the links for MITM
as HTTPS traffic is much harder to manipulate.
Deterministic algorithm:
For each file:
If not .svg:
For each line:
If doesn't contain `\bxmlns\b`:
For each link, `\bhttp://[^# \t\r\n]*(?:\w|/)`:
If neither `\bgnu\.org/license`, nor `\bmozilla\.org/MPL\b`:
If both the HTTP and HTTPS versions
return 200 OK and serve the same content:
Replace HTTP with HTTPS.
Signed-off-by: Alexander A. Klimov <grandmaster@al2klimov.de>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit further avoids conflation of rcuperf with the kernel's perf
feature by renaming kernel/rcu/rcuperf.c to kernel/rcu/rcuscale.c, and
also by similarly renaming the functions and variables inside this file.
This has the side effect of changing the names of the kernel boot
parameters, so kernel-parameters.txt and ver_functions.sh are also
updated. The rcutorture --torture type was also updated from rcuperf
to rcuscale.
[ paulmck: Fix bugs located by Stephen Rothwell. ]
Reported-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, parse-torture.sh looks at the fifth field of torture-test
console output for the version number. This works fine for rcutorture,
but not for scftorture, which lacks the pointer field. This commit
therefore adjusts matching lines so that the parse-console.sh awk script
always sees the version number as the first field in the lines passed
to it.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
In the dim distant past, qemu commands needed to be run from the
rcutorture directory, but this is no longer the case. This commit
therefore removes the now-useless "cd $KVM" from the kvm-test-1-run.sh
script.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, the qemu command is constructed twice, once to dump it
to the qemu-cmd file and again to execute it. This is of course an
accident waiting to happen, but is done to ensure that the remainder
of the script has an accurate idea of the running qemu command's PID.
This commit therefore places both the qemu command and the PID capture
into a new temporary file and sources that temporary file. Thus the
single construction of the qemu command into the qemu-cmd file suffices
for both purposes.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds a script that transforms qemu-cmd files to allow them
and the corresponding kernels to be run in contexts other than the one
that they were created for, including on systems other than the one that
they were built on. For example, this allows the build products from a
--buildonly run to be transformed to allow distributed rcutorture testing.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Using --kcsan when the compiler does not support KCSAN results in this:
:CONFIG_KCSAN=y: improperly set
:CONFIG_KCSAN_REPORT_ONCE_IN_MS=100000: improperly set
:CONFIG_KCSAN_VERBOSE=y: improperly set
:CONFIG_KCSAN_INTERRUPT_WATCHER=y: improperly set
Clean KCSAN run in /home/git/linux-rcu/tools/testing/selftests/rcutorture/res/2020.06.16-09.53.16
This is a bit obtuse, so this commit adds checks resulting in this:
:CONFIG_KCSAN=y: improperly set
:CONFIG_KCSAN_REPORT_ONCE_IN_MS=100000: improperly set
:CONFIG_KCSAN_VERBOSE=y: improperly set
:CONFIG_KCSAN_INTERRUPT_WATCHER=y: improperly set
Compiler or architecture does not support KCSAN!
Did you forget to switch your compiler with --kmake-arg CC=<cc-that-supports-kcsan>?
Suggested-by: Marco Elver <elver@google.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Acked-by: Marco Elver <elver@google.com>
Currently, kvm-recheck.sh complains that qemu failed for --buildonly
runs, which is sort of true given that qemu can hardly succeed if not
invoked in the first place. Nevertheless, this commit swaps the order
of checks in kvm-recheck.sh so that --buildonly runs will be summarized
more straightforwardly.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
We need to pass the arguments provided to --kmake-arg to all make
invocations. In particular, the make invocations generating the configs
need to see the final make arguments, e.g. if config variables depend on
particular variables that are passed to make.
For example, when using '--kcsan --kmake-arg CC=clang-11', we would lose
CONFIG_KCSAN=y due to 'make oldconfig' not seeing that we want to use a
compiler that supports KCSAN.
Signed-off-by: Marco Elver <elver@google.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit pulls the simple pattern-based error detection from the
console log into a new console-badness.sh file. This will enable future
commits to end a run on the first error.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
When bisecting RCU issues, it is often the case that the first error in
an unsuccessful run will happen quickly, but that a successful run must
go on for some time in order to obtain a sufficiently low false-negative
error rate. In many cases, a bisection requires multiple concurrent
runs, in which case the first failure in any run indicates failure,
pure and simple. In such cases, it would speed things up greatly if
the first failure terminated all runs.
This commit therefore adds scripting that checks for a file named "STOP"
in the top-level results directory, terminating the run when it appears.
Note that in-progress builds will continue until completion, but future
builds and all runs will be cut short.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
One reason to do a --buildonly run is to use the build products elsewhere,
for example, to do the actual test on some other system. Part of doing
the test is the actual qemu command, which is not currently produced
by --buildonly runs. This commit therefore causes --buildonly runs to
create this file.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Leaving off the kvm.sh script's --cpus argument results in the script
testing the scenarios sequentially, which can be quite slow. However,
having to specify the actual number of CPUs can be error-prone.
This commit therefore adds a --allcpus argument that causes kvm.sh to
use all available CPUs.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The identify_qemu_vcpus bash function can return numbers including
whitespace characters, which can be a bit annoying in some bash
dollar-sign substitutions. This commit therefore strips all spaces and
tabs from the value that identify_qemu_vcpus outputs.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The current console parsing assumes that console lines containing "!!!"
are statistics lines from which it can parse the number of rcutorture
too-short grace-period failures. This prints confusing output for
other problems, including memory exhaustion. This commit therefore
differentiates between these cases and prints an appropriate error string.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The torture-test recheck logic fails to set the configfile variable to
the current scenario, so this commit properly initializes this variable.
This change isn't critical given that all errors for a given scenario
follow that scenario's heading, but it is easier on the eyes to repeat it.
And this repetition also prevents confusion as to whether a given message
goes with the previous heading or the next one.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds a kvm-check-branches.sh script that takes a list
of commits and commit ranges and runs a short rcutorture test on all
scenarios on each specified commit. A summary is printed at the end, and
the script returns success if all rcutorture runs completed without error.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
On some (probably misconfigured) systems, the torture-test scripting
will cause qemu to complain about missing EFI firmware, often because
qemu is trying to traverse broken symbolic links to find that firmware.
Which is a bit silly given that the default torture-test guest OS has
but a single binary for its userspace, and thus is unlikely to do much
in the way of networking in any case.
This commit therefore avoids such problems by specifying "-net none"
to qemu unless the TORTURE_QEMU_INTERACTIVE environment variable is set
(for example, by having specified "--interactive" to kvm.sh), in which
case "-net nic -net user" is specified to qemu instead. Either choice
may be overridden by specifying the "-net" argument of your choice to
the kvm.sh "--qemu-args" parameter.
Link: https://lore.kernel.org/lkml/20190701141403.GA246562@google.com
Reported-by: Joel Fernandes <joel@joelfernandes.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
This commit renames the rcutorture config/refperf to config/refscale to
further avoid conflation with the Linux kernel's perf feature.
Reported-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit further avoids conflation of refperf with the kernel's perf
feature by renaming kernel/rcu/refperf.c to kernel/rcu/refscale.c,
and also by similarly renaming the functions and variables inside
this file. This has the side effect of changing the names of the
kernel boot parameters, so kernel-parameters.txt and ver_functions.sh
are also updated.
The rcutorture --torture type remains refperf, and this will be
addressed in a separate commit.
Reported-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The old Kconfig option name is all too easy to conflate with the
unrelated "perf" feature, so this commit renames RCU_REF_PERF_TEST to
RCU_REF_SCALE_TEST.
Reported-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, it is necessary to manually edit the console output to see
anything more than statistics, and sometimes the statistics can indicate
outliers that need more investigation. This commit therefore dumps out
the per-experiment measurements, sorted in ascending order, just before
dumping out the statistics.
Cc: Joel Fernandes (Google) <joel@joelfernandes.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The experiment-number column is currently labeled "Threads", which is
misleading at best. This commit therefore relabels it as "Runs", and
adjusts the scripts accordingly.
Cc: Joel Fernandes (Google) <joel@joelfernandes.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit updates the rcutorture scripting to include the new refperf
torture-test module.
Cc: Joel Fernandes (Google) <joel@joelfernandes.org>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
fixes.2020.04.27a: Miscellaneous fixes.
kfree_rcu.2020.04.27a: Changes related to kfree_rcu().
rcu-tasks.2020.04.27a: Addition of new RCU-tasks flavors.
stall.2020.04.27a: RCU CPU stall-warning updates.
torture.2020.05.07a: Torture-test updates.
Make it a bit easier to apply KASAN to rcutorture runs with a new --kasan
argument, again leveraging the config_override_param() bash function.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit saves a few lines of code by also using the bash
config_override_param() to set the initial list of Kconfig options from
the CFcommon file. While in the area, it makes this function capable of
update-in-place on the file containing the cumulative Kconfig options,
thus avoiding annoying changes when adding another source of options.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit applies config_override_param() to allow scenario-specific
Kconfig options to override those in CFcommon. This in turn will allow
additional Kconfig options to be placed in CFcommon, for example, an
option common to all but a few scenario can be placed in CFcommon and
then overridden in those few scenarios. Plus this change saves one
whole line of code.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, attempting to override a --kcsan default with a --kconfig
option might or might not work. However, it would be good to allow the
user to adjust the --kcsan defaults, for example, to specify a different
time for CONFIG_KCSAN_REPORT_ONCE_IN_MS. This commit therefore uses the
new config_override_param() bash function to apply the --kcsan defaults
and then apply the --kconfig options, which allows this overriding
to occur.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit introduces a config_override_param() bash function that
folds in an additional set of Kconfig options. This is initially applied
to fold in the --kconfig kvm.sh parameter, but later commits will also
apply it to the Kconfig options added by the --kcsan kvm.sh parameter.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The #CHECK# directives that can be present in CFcommon and in the
rcutorture scenario Kconfig files are both copied to ConfigFragment
and grepped out of the two directive files and added to ConfigFragment.
This commit therefore removes the redundant "grep" commands and takes
advantage of the consequent opportunity to simplify redirection.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The KCSAN tool emits a great many warnings for current kernels, for
example, a one-hour run of the full set of rcutorture scenarios results
in no fewer than 3252 such warnings, many of which are duplicates
or are otherwise closely related. This commit therefore introduces
a kcsan-collapse.sh script that maps these warnings down to a set of
function pairs (22 of them given the 3252 individual warnings), placing
the resulting list in decreasing order of frequency of occurrence into
a kcsan.sum file. If any KCSAN warnings were produced, the pathname of
this file is emitted at the end of the summary of the rcutorture runs.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Although the existing --kconfig argument can be used to run KCSAN for
an rcutorture test, it is not as straightforward as one might like:
--kconfig "CONFIG_DEBUG_INFO=y CONFIG_KCSAN=y \
CONFIG_KCSAN_ASSUME_PLAIN_WRITES_ATOMIC=n \
CONFIG_KCSAN_REPORT_VALUE_CHANGE_ONLY=n \
CONFIG_KCSAN_REPORT_ONCE_IN_MS=100000 \
CONFIG_KCSAN_VERBOSE=y CONFIG_KCSAN_INTERRUPT_WATCHER=y"
This commit therefore adds a "--kcsan" argument that emulates the above
--kconfig command. Note that if you specify a Kconfig option using
-kconfig that conflicts with one that --kcsan adds, you get whatever
the script and the build system decide to give you.
Cc: Marco Elver <elver@google.com>
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The number of CPUs is tuned to allow "4*CFLIST TREE10" on a large system,
up from "3*CFLIST TREE10" previously.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
System hangs or killed rcutorture guest OSes can result in truncated
"Reader Pipe:" lines, which can in turn result in false-positive
reader-batch near-miss warnings. This commit therefore adjusts the
reader-batch checks to account for possible line truncation.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds a TRACE02 scenario which enables preemption and RCU
Tasks Trace IPIs, more specifically, disabling heavyweight readers.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Here are 3 SPDX patches for 5.7-rc1.
One fixes up the SPDX tag for a single driver, while the other two go
through the tree and add SPDX tags for all of the .gitignore files as
needed.
Nothing too complex, but you will get a merge conflict with your current
tree, that should be trivial to handle (one file modified by two things,
one file deleted.)
All 3 of these have been in linux-next for a while, with no reported
issues other than the merge conflict.
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
-----BEGIN PGP SIGNATURE-----
iG0EABECAC0WIQT0tgzFv3jCIUoxPcsxR9QN2y37KQUCXodg5A8cZ3JlZ0Brcm9h
aC5jb20ACgkQMUfUDdst+ykySQCgy9YDrkz7nWq6v3Gohl6+lW/L+rMAnRM4uTZm
m5AuCzO3Azt9KBi7NL+L
=2Lm5
-----END PGP SIGNATURE-----
Merge tag 'spdx-5.7-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/spdx
Pull SPDX updates from Greg KH:
"Here are three SPDX patches for 5.7-rc1.
One fixes up the SPDX tag for a single driver, while the other two go
through the tree and add SPDX tags for all of the .gitignore files as
needed.
Nothing too complex, but you will get a merge conflict with your
current tree, that should be trivial to handle (one file modified by
two things, one file deleted.)
All three of these have been in linux-next for a while, with no
reported issues other than the merge conflict"
* tag 'spdx-5.7-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/spdx:
ASoC: MT6660: make spdxcheck.py happy
.gitignore: add SPDX License Identifier
.gitignore: remove too obvious comments
This commit enables the KCSAN Kconfig options that (1) detect data
races between reads and writes even when the writes do not change the
variable's value and (2) detect data races involving plain C-language
writes. These changes only affect scripted rcutorture runs and can be
overridden using the kvm.sh --kconfig argument.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, kvm-find-errors.sh gives a usage prompt when given a bad
directory, but then soldiers on, giving a series of confusing error
messages. This commit therefore prints an error message and exits when
given a bad directory, hopefully reducing confusion.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
When running the default list of tests, the run summary of a successful
(that is, failed to find any errors) run fits easily on a 24-line screen.
But a run with something like "--configs '5*CFLIST'" will be 80 lines long,
and it is all too easy to miss a failure message when scrolling back.
This commit therefore prints out the number of runs with failing builds
or runtime failures, but only if there are any such failures.
For example, a run with a single build error and a single runtime error
would print two lines like this:
1 runs with build errors.
1 runs with runtime errors.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The small-system rcutorture configurations have served us well for a great
many years, but it is now time to add a larger one. This commit does
just that, but does not add it to the defaults in CFLIST. This allows
the kvm.sh argument '--configs "4*CFLIST TREE10" to run four instances
of each of the default configurations concurrently with one instance of
the large configuration.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The names of the per-test results directories are of the form
2019.11.29-20:42:19. This works, but the ":" characters make
tab-based shell name completion a bit onerous because the user must
remember to include a quote character somewhere before the first ":".
This commit therefore changes the ":" characters to periods, as in
2019.12.01-20.48.01", which allows tab-based completion to work more
naturally.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The bootparam_hotplug_cpu() bash function was checking for CPU-hotplug
kernel-boot parameters from --bootargs, but that check was specific to
rcutorture ("rcutorture\.onoff_"). This commit therefore makes this
check also work for locktorture ("torture\.onoff_").
Note that rcuperf does not do CPU-hotplug operations, so it is not
necessary to make a similar change for rcuperf.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
On some kernels, concurrent calls to the lscpu command result in severe
slowdowns. For example, on v4.16, a single lscpu invocation takes about
two milliseconds, four concurrent invocations more than two seconds,
and 16 concurrent invocations more than 20 seconds. Given that the only
goal is to learn the number of CPUs, invoking lscpu but once suffices.
This commit therefore invokes lscpu early in kvm.sh execution, setting
the initial value of the TORTURE_ALLOTED_CPUS environment variable.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
On a large system, it can be convenient to tell rcutorture to run
several instances of the default scenarios. Currently, this requires
explicitly listing them, for example, "--configs '2*SRCU-N 2*SRCU-P...'".
Although this works, it is rather inconvenient.
This commit therefore allows "CFLIST" to be specified to indicate the
default list of scenarios called out in the relevant CFLIST file, for
example, for RCU, tools/testing/selftests/rcutorture/configs/rcu/CFLIST.
In addition, multipliers may be used to run multiple instances of all
the scenarios. For example, on a 256-CPU system, "--configs '3*CFLIST'"
would run three instances of each scenario concurrently with one CPU
left over. Thus "--configs '3*CFLIST TINY01'" would exactly consume all
256 CPUs, which makes rcutorture's jitter feature more effective.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
This commit adds the worst-case results from any call_rcu()
forward-progress tests to the rcutorture test-summary output.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The rcutorture scripting uses the mpstat command to determine how much
the system is being used, and adjusts make's -j argument accordingly.
However, mpstat isn't installed by default, so it would be good if the
scripting does something useful when mpstat isn't present.
This commit therefore makes the scripts assumes that if mpstat is not
present, they are free to use all the CPUs.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Currently, jitter.sh assumes that the underlying hypervisor will be
configured with all CPUs hotpluggable, with the possible exception
of CPU 0. However, there are installations where the hypervisor
prohibits offlining, which breaks jitter.sh. This commit therefore
lists the CPUs that cannot be offlined up front, and checks for the
case where no CPU can be offlined in the loop.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
The dracut scripting does not work on all platforms, and there are no
known failures from the init binary based on the statically linked C
program. This commit therefore removes the dracut scripting so that the
statically linked C program is always used to create the init "script".
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
In many environments, gawk provides systime(), but awk doesn't.
This commit therefore changes awk scripts using systime() to instead be
gawk scripts.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
A number of mainstream CPU families are no longer capable of building
kernels having CONFIG_SMP=y and CONFIG_HOTPLUG_CPU=n, so this commit
removes this combination from the rcutorture scenarios having it.
People wishing to try out this combination may still do so using the
"--kconfig CONFIG_HOTPLUG_CPU=n CONFIG_SUSPEND=n CONFIG_HIBERNATION=n"
argument to the tools/testing/selftests/rcutorture/bin/kvm.sh script
that is used to run rcutorture.
Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
Since commit 05f415715c ("rcu: Speed up expedited GPs when interrupting
RCU reader") in v5.0 and through v5.1, booting with the threadirqs kernel
boot parameter caused self-deadlocks, which can be reproduced using the
following command on an 8-CPU system:
tools/testing/selftests/rcutorture/bin/kvm.sh --duration 5 --configs "TREE03" --bootargs "threadirqs"
This commit therefore adds the threadirqs kernel boot parameter to
the TREE03 rcutorture scenario in order to more quickly detect future
similar bugs.
Link: http://lkml.kernel.org/r/20190626135447.y24mvfuid5fifwjc@linutronix.de
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Joel Fernandes <joel@joelfernandes.org>
The kvm-test-1-run.sh script says 'test -z "last_ts"' which always
evaluates to true (AKA zero) regardless of the value of the last_ts shell
variable. This commit therefore inserts the needed dollar sign ("$").
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
When trace_printk() is used, a message including "BUG" is printed to
the console, which fools the rcutorture scripting into believing that
the corresponding test scenario failed. This commit therefore filters
out this particular instance of "BUG", thus avoiding the false-positive
test-failure report.
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
The current rcutorture scripts unconditionally do "make clean", which is
a good way of getting the needed testing done despite any imperfections in
Makefile dependency tracking. However, this can be a bit irritating when
repeatedly running a single scenario after small changes, for example,
when debugging a problem that affects only a single scenario. This commit
therefore adds a --trust-make argument that suppresses the "make clean".
Even when using ccache, this speeds up kernel builds by up to almost an
order of magnitude on my laptop.
Reported-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
Currently, rcutorture will use relatively few CPUs to build the kernel
on a busy system, which is often as it should be. However, if the user
has used the --cpus argument to dedicate a specified number of CPUs to
this torture test, it would be good if the kernel build also made use
of them. This commit therefore changes the cpus2use.sh script to use
--cpus when specified and to do the idleness calculations otherwise.
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
For historical reasons, rcutorture places its build products in a
tools/testing/selftests/rcutorture/b1 directory using the O= kbuild
command-line argument. However, doing this requires that the source
directory be pristine: Not just "make clean" pristine, but instead "make
mrproper" (or, equivalently, "make distclean") pristine. Therefore,
rcutorture executes a "make mrproper" before each build. Unfortunately,
"make mrproper" has the side effect of removing pretty much everything,
including tags files and cscope databases, which can be inconvenient
to people whose workflow centers around a single source tree.
This commit therefore makes rcutorture do the build directly in the
source directory, removing the need for "make mrproper". This works
because all needed build products are moved to their proper place in the
"res" directory immediately after the build completes, so that multiple
rcutorture kernels can still run concurrently.
Reported-by: Peter Zijlstra <peterz@infradead.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
Currently qemu output appears on standard output, but is inaccessible
later on. This commit therefore captures this output and causes
kvm-recheck.sh to output this output if QEMU gave a non-zero non-137
exit code. (And exit code of 137 indicates that QEMU was killed, in
which case we want to know about the hang rather than the fact that
QEMU was killed.)
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
In one of my rcutorture tests the TSC clocksource got marked unstable
due to a large difference in the TSC value. I'm not sure if the guest
run for a long time with disabled interrupts or if the host was very
busy and didn't schedule the guest for some time.
I took a look on the qemu/KVM options and decided to update the options:
- Use kvm{32|64} as CPU. We could probably use `host' (like ARM does)
for maximum available features but since we don't run any userland I'm
not sure if it makes any difference.
- Drop the "noapic" option. There is no history why the APIC was disabled,
I see no reason for it. Once old qemu versions fade away, we can add
"x2apic=on,tsc-deadline=on,hypervisor=on,tsc_adjust=on".
- Additional config options. It ensures that the kernel knowns that it
runs as a kvm guest and can use virt devices like the kvm-clock as
clocksource. The kvm-clock was the main motivation here.
- I didn't add a random HW device. It would make the random device ready
earlier (not it doesn't complete the initialisation at all) but I
doubt that there is any need for this.
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
[ paulmck: The world is not quite ready for CONFIG_PARAVIRT_SPINLOCKS=y
and x2apic, so they are omitted for the time being. ]
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
I have been showing off a trivial RCU implementation for non-preemptive
environments for some time now:
#define rcu_read_lock()
#define rcu_read_unlock()
#define rcu_dereference(p) READ_ONCE(p)
#define rcu_assign_pointer(p, v) smp_store_release(&(p), (v))
void synchronize_rcu(void)
{
int cpu;
for_each_online_cpu(cpu)
sched_setaffinity(current->pid, cpumask_of(cpu));
}
Trivial or not, as the old saying goes, "if it ain't tested, it don't
work!". This commit therefore adds a "trivial" flavor to rcutorture
and a corresponding TRIVIAL test scenario. This variant does not handle
CPU hotplug, which is unconditionally enabled on x86 for post-v5.1-rc3
kernels, which is why the TRIVIAL.boot says "rcutorture.onoff_interval=0".
This commit actually does handle CONFIG_PREEMPT=y kernels, but only
because it turns back the Linux-kernel clock in order to provide these
alternative definitions (or the moral equivalent thereof):
#define rcu_read_lock() preempt_disable()
#define rcu_read_unlock() preempt_enable()
In CONFIG_PREEMPT=n kernels without debugging, these are equivalent to
empty macros give or take a compiler barrier. However, the have been
successfully tested with actual empty macros as well.
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
[ paulmck: Fix symbol issue reported by kbuild test robot <lkp@intel.com>. ]
[ paulmck: Work around sched_setaffinity() issue noted by Andrea Parri. ]
[ paulmck: Add rcutorture.shuffle_interval=0 to TRIVIAL.boot to fix
interaction with shuffler task noted by Peter Zijlstra. ]
Tested-by: Andrea Parri <andrea.parri@amarulasolutions.com>
Because TREE01 can end up running more vCPUs that physical CPUs,
hammering these shortchanged CPUs with tight loops containing call_rcu()
invocations seems a bit like overkill. This commit therefore exempts
TREE01 from rcutorture's forward-progress testing.
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
This commit provides a rudimentary Makefile that runs a 10-minute
rcutorture test on scenario TREE01. This must be run on a system capable
of spawning virtual machines and with everything installed to permit
building Linux kernels.
Reported-by: Shuah Khan <shuah@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
This commit causes both kvm-find-errors.sh and kvm-recheck.sh to provide
an exit status based on whether or not errors were located. In the
case of kvm-recheck.sh, this will be the error status of the last run.
This change allows these commands to be used in scripting and Makefiles
to automatically report failed rcutorture runs.
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
jitter.sh currently does not add CPU0 to the list of CPUs for adding of
jitter. Let us add it to this list even when it is not hot-pluggable.
Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
The rcutorture jitter.sh script selects a random CPU but does not check
if it is offline or online. This leads to taskset errors many times. On
my machine, hyper threading is disabled so half the cores are offline
causing taskset errors a lot of times. Let us fix this by checking from
only the online CPUs on the system.
Cc: rcu@vger.kernel.org
Signed-off-by: Joel Fernandes (Google) <joel@joelfernandes.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
-----BEGIN PGP SIGNATURE-----
iQJEBAABCAAuFiEEwPw5LcreJtl1+l5K99NY+ylx4KYFAlzR0AAQHGF4Ym9lQGtl
cm5lbC5kawAKCRD301j7KXHgpo0MD/47D1kBK9rGzkAwIz1Jkh1Qy/ITVaDJzmHJ
UP5uncQsgKFLKMR1LbRcrWtmk2MwFDNULGbteHFeCYE1ypCrTgpWSp5+SJluKd1Q
hma9krLSAXO9QiSaZ4jafshXFIZxz6IjakOW8c9LrT80Ze47yh7AxiLwDafcp/Jj
x6NW790qB7ENDtfarDkZk14NCS8HGLRHO5B21LB+hT0Kfbh0XZaLzJdj7Mck1wPA
VT8hL9mPuA++AjF7Ra4kUjwSakgmajTa3nS2fpkwTYdztQfas7x5Jiv7FWxrrelb
qbabkNkWKepcHAPEiZR7o53TyfCucGeSK/jG+dsJ9KhNp26kl1ci3frl5T6PfVMP
SPPDjsKIHs+dqFrU9y5rSGhLJqewTs96hHthnLGxyF67+5sRb5+YIy+dcqgiyc/b
TUVyjCD6r0cO2q4v9VhwnhOyeBUA9Rwbu8nl7JV5Q45uG7qI4BC39l1jfubMNDPO
GLNGUUzb6ER7z6lYINjRSF2Jhejsx8SR9P7jhpb1Q7k/VvDDxO1T4FpwvqWFz9+s
Gn+s6//+cA6LL+42eZkQjvwF2CUNE7TaVT8zdb+s5HP1RQkZToqUnsQCGeRTrFni
RqWXfW9o9+awYRp431417oMdX/LvLGq9+ZtifRk9DqDcowXevTaf0W2RpplWSuiX
RcCuPeLAVg==
=Ot0g
-----END PGP SIGNATURE-----
Merge tag 'for-5.2/block-20190507' of git://git.kernel.dk/linux-block
Pull block updates from Jens Axboe:
"Nothing major in this series, just fixes and improvements all over the
map. This contains:
- Series of fixes for sed-opal (David, Jonas)
- Fixes and performance tweaks for BFQ (via Paolo)
- Set of fixes for bcache (via Coly)
- Set of fixes for md (via Song)
- Enabling multi-page for passthrough requests (Ming)
- Queue release fix series (Ming)
- Device notification improvements (Martin)
- Propagate underlying device rotational status in loop (Holger)
- Removal of mtip32xx trim support, which has been disabled for years
(Christoph)
- Improvement and cleanup of nvme command handling (Christoph)
- Add block SPDX tags (Christoph)
- Cleanup/hardening of bio/bvec iteration (Christoph)
- A few NVMe pull requests (Christoph)
- Removal of CONFIG_LBDAF (Christoph)
- Various little fixes here and there"
* tag 'for-5.2/block-20190507' of git://git.kernel.dk/linux-block: (164 commits)
block: fix mismerge in bvec_advance
block: don't drain in-progress dispatch in blk_cleanup_queue()
blk-mq: move cancel of hctx->run_work into blk_mq_hw_sysfs_release
blk-mq: always free hctx after request queue is freed
blk-mq: split blk_mq_alloc_and_init_hctx into two parts
blk-mq: free hw queue's resource in hctx's release handler
blk-mq: move cancel of requeue_work into blk_mq_release
blk-mq: grab .q_usage_counter when queuing request from plug code path
block: fix function name in comment
nvmet: protect discovery change log event list iteration
nvme: mark nvme_core_init and nvme_core_exit static
nvme: move command size checks to the core
nvme-fabrics: check more command sizes
nvme-pci: check more command sizes
nvme-pci: remove an unneeded variable initialization
nvme-pci: unquiesce admin queue on shutdown
nvme-pci: shutdown on timeout during deletion
nvme-pci: fix psdt field for single segment sgls
nvme-multipath: don't print ANA group state by default
nvme-multipath: split bios with the ns_head bio_set before submitting
...
Currently support for 64-bit sector_t and blkcnt_t is optional on 32-bit
architectures. These types are required to support block device and/or
file sizes larger than 2 TiB, and have generally defaulted to on for
a long time. Enabling the option only increases the i386 tinyconfig
size by 145 bytes, and many data structures already always use
64-bit values for their in-core and on-disk data structures anyway,
so there should not be a large change in dynamic memory usage either.
Dropping this option removes a somewhat weird non-default config that
has cause various bugs or compiler warnings when actually used.
Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
The scripting must supply the CONFIG_INITRAMFS_SOURCE Kconfig option
so that kbuild can find the desired initrd, but the configcheck.sh
script gets confused by this option because it takes a string instead
of the expected y/n/m. This causes checkconfig.sh to complain about
CONFIG_INITRAMFS_SOURCE in the torture-test output (though not in the
summary). As more people use rcutorture, the resulting confusion is
an increasing concern.
This commit therefore suppresses this false-positive warning by filtering
CONFIG_INITRAMFS_SOURCE from within the checkconfig.sh script.
Reported-by: Joel Fernandes <joel@joelfernandes.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
Replace the license boiler plate with a SPDX license identifier.
While in the area, update an email address and add copyright notices.
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
As suggested by Ingo, this header file might benefit other tools than
just rcutorture. For now it's quite limited, but is easy to extend, so
exposing it into tools/include/nolibc/ will make it much easier to
adopt by other tools.
The mkinitrd.sh script in rcutorture was updated to use this new location.
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Signed-off-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
Ingo rightfully asked for a bit more documentation in the nolibc header,
so this patch adds some explanation about its purpose, how it's made, and
how to use it.
Cc: Ingo Molnar <mingo@kernel.org>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
Reviewed-by: Joey Pabalinas <joeypabalinas@gmail.com>
Reviewed-by: Randy Dunlap <rdunlap@infradead.org>
A few macros had their rightmost backslash misaligned, and the pollfd
struct definition resisted the previous code reindent. Nothing else
changed.
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Signed-off-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
Reviewed-by: Joey Pabalinas <joeypabalinas@gmail.com>
A last-minute checkpatch cleanup caused most of list of clobbered
registers to be lost in the MIPS syscall definition. Although this code
is not yet used on MIPS, it is nevertheless better to fix it before it
does get used.
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Signed-off-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
Why a Bourne-shell "for" loop? And why 192 instances of "a"? This commit
adds a shell comment to present the answer to these mysteries. It also
uses a series of factor-of-four Bourne-shell assignments to make it
easy to see how many instances there are, replacing the earlier wall of
'a' characters.
Reported-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
[ paulmck: Fix wrong-variable bugs noted by Andrea Parri. ]
bug.2018.11.12a: Get rid of BUG_ON() and friends
consolidate.2018.12.01a: Continued RCU flavor-consolidation cleanup
doc.2018.11.12a: Documentation updates
fixes.2018.11.12a: Miscellaneous fixes
initrd.2018.11.08b: Automate creation of rcutorture initrd
sil.2018.11.12a: Remove more spin_unlock_wait() calls
Now that synchronize_rcu() waits for preempt-disable regions of code
as well as RCU read-side critical sections, synchronize_sched() can be
replaced by synchronize_rcu(). This commit therefore makes this change,
even though it is but a comment.
Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
This reduces the size of the init executable from ~800 kB to ~800 bytes
on x86_64. This is only implemented for x86_64, i386, arm and arm64.
Others not tested.
Signed-off-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This is a definition of the most common syscalls needed in minimalist
init executables, allowing to statically build them with no external
dependencies. It is sufficient in its current form to build rcutorture's
init on x86_64, i386, arm, and arm64. Others have not been ported or
tested. Updates may be found here :
http://git.formilux.org/?p=people/willy/nolibc.git
Signed-off-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
If the build fails, we can end up with an empty initrd directory which
prevents the build script from operating again. Better rely on the
resulting init executable instead.
Signed-off-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Strip using -s on the compiler command line instead of calling the "strip"
utility as the latter isn't necessarily compatible with the target arch.
Signed-off-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This adds the CROSS_COMPILE environment to the initrd.sh script's
gcc command to enable cross compilation.
Reported-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, the initrd/init script and executable remain blocked almost
all the time. However, it is necessary to test nohz_full userspace
execution, which both variants of initrd/init fail to do. This commit
therefore causes initrd/init to spend about a millisecond per second
executing in userspace.
Reported-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The support for creating initrd directories using dracut is a great
improvement over having to always hand-create them, it is a bit annoying
to have to install some otherwise irrelevant package just to be able to
run rcutorture. This commit therefore adds support for creating initrd
directories on systems innocent of dracut. You do need gcc, but then
again you need that to build the kernel (or to build llvm) in any case.
The idea is to create an initrd directory containing nothing but a
statically linked binary having a for-loop over a long-term sleep().
The result is a Linux kernel with almost no userspace: even the
time-honored /dev, /lib, /tmp, and /usr directories are gone. In fact,
the only directory present is "/", but only because I don't know how to
get rid of it, at least short of not having an initrd in the first place.
Although statically linked binaries are much maligned, and rightly so,
their disadvantages seem to be irrelevant for this particular use case.
From https://www.akkadia.org/drepper/no_static_linking.html:
1. Fixes are difficult to apply to hordes of widely scattered
statically linked binaries. But in this case, there is only one
binary, but there would otherwise be no fewer than four libraries.
2. Security measures like local address randomization cannot be used.
Prudence prevents me from asserting that it is impossible to
base a remote attack on a networking-free rcutorture instance.
Nevertheless, bonus points to the first person who comes up with
such an attack!
3. More efficient use of physical memory. Not in this case, given
that libc is 1.8MB and the statically linked binary "only" 800K.
4. Features such as locales, name service switch (NSS),
internationalized domain names (IDN) tool, and so on require
dynamic linking. Bonus points to the first person coming up
with a valid rcutorture use case requiring these features in
its initrd.
5. Accidental violations of (L)GPL. Actually, this change actually
helps -avoid- such violations by reducing the temptation to
pass around tarballs of rcutorture-ready initrd directories.
After all, the rcutorture scripts automatically create an initrd
directory for you, so why bother with the tarballs?
6. Tools and hacks like ltrace, LD_PRELOAD, LD_PROFILE, and LD_AUDIT
don't work. Again, bonus points to the first person coming up
with a valid rcutorture use case requiring these features in
its initrd.
Nevertheless, the script will use dracut if available, and will create the
statically linked binary only when dracut are missing. Those preferring
the smaller initrd directory resulting from the statically linked binary
(like me) are free to hand-edit mkinitrd.sh to remove the code using
dracut. ;-)
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The rcutorture scripts currently expect the user to create the
tools/testing/selftests/rcutorture/initrd directory. Should the user
fail to do this, the kernel build will fail with obscure and confusing
error messages. This commit therefore adds explicit checks for the
tools/testing/selftests/rcutorture/initrd directory, and if not present,
creates one on systems on which dracut is installed. If this directory
could not be created, a less obscure error message is emitted and the
test is aborted.
Suggested-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Connor Shu <Connor.Shu@ibm.com>
[ paulmck: Adapt the script to fit into the rcutorture framework and
severely abbreviate the initrd/init script. ]
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Now that SRCU permits call_srcu() to be invoked at early boot, this
commit ensures that the rcutorture scripting tests early boot call_srcu().
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Now that the RCU-bh and RCU-sched update-side functions are simple
wrappers around their RCU counterparts, there isn't a whole lot of
point in testing them. This commit therefore removes the self-test
capability and removes the corresponding kernel-boot parameters.
It also updates the various rcutorture .boot files to remove the
kernel boot parameters that call for testing RCU-bh and RCU-sched.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Now that there is only one RCU flavor to rule them all, the TREE06
and TREE08 test scenarios are redundant. This commit therefore removes
them. Later changes will rebalance and renumber the tests.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The old approach placed all the build products into the b* directories,
which meant that some of these build products needed to be copied to
the proper directory in the res hierarchy. The new approach leaves
things like .config and the .o files in the b1 directory, but directs
build output and diagnostics directly to the proper directory in the
res hierarchy. Unfortunately, one of the copies was still carried out,
which could (and sometimes did) overwrite the build output and diagnostics
with obsolete output remaining in the b1 directory.
This commit therefore removes the offending "cp" command.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Some RCU bugs have been sensitive to the frequency of CPU-hotplug
operations, which have been gradually increased over time. But this
frequency is now at the one-second lower limit that can be specified using
the rcutorture.onoff_interval kernel parameter. This commit therefore
changes the units of rcutorture.onoff_interval from seconds to jiffies,
and also sets the value specified for this kernel parameter in the TREE03
rcutorture scenario to 200, which is 200 milliseconds for HZ=1000.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The main race with the early part of grace-period initialization appears
to be with CPU hotplug. To more fully open this race window, this commit
moves the rcu_gp_slow() from the beginning of the early initialization
loop to follow that loop, thus widening the race window, especially for
the rcu_node structures that are initialized last. This commit also
expands rcutree.gp_preinit_delay from 3 to 12, giving the same overall
delay in the grace period, but concentrated in the spot where it will
do the most good.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Although warnings about close calls are printed by kvm-recheck.sh,
kvm-find-errors.sh currently ignores them. This could easily result
in someone failing to investigate close calls, so this commit makes
them visible to kvm-find-errors.sh.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The TREE08-T.boot file was used to provide alternative options for
debugging, but things have changed, it has not kept up, and it has not
been used or missed. This commit therefore removes it.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The torture scripting currently builds each kernel from a set of parallel
runs in its own build directory. This can waste quite a bit of space when
running large numbers of concurrent scenarios, and pointlessly given that
the builds are run sequentially (albeit with a largish -j number passed to
"make"). This commit therefore places all build-command output in the
results directory, then does all builds in a single "b1" build directory.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, kvm-find-errors.sh looks only for build errors ("error:"),
so this commit makes it also locate build warnings ("warning:").
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Nicholas Piggin <npiggin@gmail.com>
With the addition of the end-of-test state, it is not uncommon for the
kvm.sh summary lines to overflow 80 characters. This commit therefore
applies abbreviations in order to make the line fit into 80 characters
with high probability.
And yes, I did make heavy use of punched cards back in the day, so 80
columns it is for my xterms! ;-)
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Nicholas Piggin <npiggin@gmail.com>
This commit adds the end-of-test test, if present in the console output,
to the kvm.sh test summary that is printed by kvm-recheck.sh. Note that
this only applies to rcutorture console output.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Nicholas Piggin <npiggin@gmail.com>
The rcutorture scripting scans the console output twice, once to look
for various sorts of hangs and again to find warnings and panics.
Unfortunately, only the output of the second scan gets written to the
console.log.diags file, which can cause hangs to be overlooked.
This commit therefore folds the parse-torture.sh script (which looks
for hangs) into the parse-console.sh script (which looks for warnings
and panics). This allows both types of failure information to be
added to console.log.diags, while still reliably removing this file
when it proves to be empty.
This also fixes a long-standing bug where rcuperf log files would
unconditionally complain about a hang.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Nicholas Piggin <npiggin@gmail.com>
This commit adds a script that allows viewing the build and/or
console output from failed rcutorture, locktorture, or rcuperf runs.
This replaces a time-honored but inefficient manual procedure that uses
cut and paste.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tested-by: Nicholas Piggin <npiggin@gmail.com>
The default values for nreader and nwriter are apparently not all that
user-friendly, resulting in people doing scalability tests that ran all
runs at large scale. This commit therefore makes both the nreaders and
nwriters module default to the number of CPUs, and adds a comment to
rcuperf.c stating that the number of CPUs should be specified using the
nr_cpus kernel boot parameter. This commit also eliminates the redundant
rcuperf scripting specification of default values for these parameters.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The rcuperf trace-event processing counted every "done" trace event
as a piggyback, which is incorrect because the task that started the
grace period didn't piggyback at all. This commit fixes this problem
by recording the task that started a given grace period and ignoring
that task's "done" record for that grace period.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The rcuperf event-trace processing assumes that expedited grace periods
start and end on the same task, an assumption that was violated by moving
expedited grace-period processing to workqueues. This commit removes
this now-fallacious assumption from rcuperf's event-trace processing.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The purpose of jitter is to expose concurrency bugs due to invalid
assumptions about forward progress. There is usually little point
in jitter when measuring performance. This commit therefore defaults
jitter off when running rcuperf. You can override this by specifying
the kvm.sh "--jitter" argument -after- the "--torture rcuperf"
argument. No idea why you would want this, but if you do, that is
how you do it.
One example of a conccurrency bug that this jitter might expose is one
in which the developer assumed that a given short region of code would be
guaranteed to execute within some short time limit. Such assumptions are
invalid in virtualized environments because the hupervisor can preempt
the guest OS at any point, even when the guest OS thinks that it has
disabled interrupts.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The 512 megabyte memory size has served quite well, but more memory
is required when using large trace buffers on large systems. This
commit therefore adds a --memory argument to the kvm.sh script, which
allows the memory size to be specified on the command line, for example,
"--memory 768", --memory 800M", or "--memory 2G".
Reported-by: Sebastian Siewior <bigeasy@linutronix.de>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds support of the qemu command qemu-system-aarch64
to rcutorture.
Signed-off-by: Lihao Liang <lianglihao@huawei.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The kvm.sh header comment is a bit of a relic, so this commit brings
it up to date.
Reported-by: Lihao Liang <lianglihao@huawei.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Commit 6f1982fedd ("sched/isolation: Handle the nohz_full= parameter")
broke CONFIG_NO_HZ_FULL_ALL=y kernels. This breakage is due to the code
under CONFIG_NO_HZ_FULL_ALL failing to invoke the shiny new housekeeping
functions. This means that rcutorture scenario TREE04 now emits RCU CPU
stall warnings due to the RCU grace-period kthreads not being awakened
at a time of their choosing, or perhaps even not at all:
[ 27.731422] rcu_bh kthread starved for 21001 jiffies! g18446744073709551369 c18446744073709551368 f0x0 RCU_GP_WAIT_FQS(3) ->state=0x402 ->cpu=3
[ 27.731423] rcu_bh I14936 9 2 0x80080000
[ 27.731435] Call Trace:
[ 27.731440] __schedule+0x31a/0x6d0
[ 27.731442] schedule+0x31/0x80
[ 27.731446] schedule_timeout+0x15a/0x320
[ 27.731453] ? call_timer_fn+0x130/0x130
[ 27.731457] rcu_gp_kthread+0x66c/0xea0
[ 27.731458] ? rcu_gp_kthread+0x66c/0xea0
Because no one has complained about CONFIG_NO_HZ_FULL_ALL=y being broken,
I hypothesize that no one is in fact using it, other than rcutorture.
This commit therefore eliminates CONFIG_NO_HZ_FULL_ALL and updates
rcutorture's config files to instead use the nohz_full= kernel parameter
to put the desired CPUs into nohz_full mode.
Fixes: 6f1982fedd ("sched/isolation: Handle the nohz_full= parameter")
Reported-by: kernel test robot <xiaolong.ye@intel.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Frederic Weisbecker <frederic@kernel.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Chris Metcalf <cmetcalf@mellanox.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Luiz Capitulino <lcapitulino@redhat.com>
Cc: Mike Galbraith <efault@gmx.de>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Rik van Riel <riel@redhat.com>
Cc: Wanpeng Li <kernellwp@gmail.com>
Cc: Ingo Molnar <mingo@kernel.org>
Cc: John Stultz <john.stultz@linaro.org>
Cc: Jonathan Corbet <corbet@lwn.net>
The purpose of torture_runnable is to allow rcutorture and locktorture
to be started and stopped via sysfs when they are built into the kernel
(as in not compiled as loadable modules). However, the 0444 permissions
for both instances of torture_runnable prevent this use case from ever
being put into practice. Given that there have been no complaints
about this deficiency, it is reasonable to conclude that no one actually
makes use of this sysfs capability. The perf_runnable module parameter
for rcuperf is in the same situation.
This commit therefore removes both torture_runnable instances as well
as perf_runnable.
Reported-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Check for build-directory existence and write permissions are provided in
both 'kvm-test-1-run.sh' an 'kvm-build.sh'. Because the 'kvm-build.sh'
is dependent on 'kvm-test-1-run.sh' ('kvm-build.sh' uses variables that
defined from its caller.), these checks are unnecessarily duplicated.
This commit therefore removes the check in from the 'kvm-build.sh' script.
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Inclusions of 'functions.sh' from 'kvm-test-1-run.sh' and
'kvm-recheck*.sh' use its absolute path. Because the directory containing
'functions.sh' is already in PATH, the full path is unnecessary. This
commit therefore simplifies the inclusions to use the short relative path.
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Both the 'kvm.sh' and 'kvm-test-1-run.sh' scripts log messages by printing
the message to 'stdout' and then also printing it into the log file.
Generation of the message thus occurs twice, once for 'stdout' and once
for the log file. Moreover, many of the messages contain 'date' output,
which results in date being invoked twice (once for stdout print, once
for log file write). As a result, the date information in stdout and
log file can differ, which could cause confusion.
This commit therefore simplifies the logging procedure by using 'tee'.
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The kvm-recheck-(lock|rcu|rcuperf).sh scripts check whether the
user-specified results directory exists. If not, it prints out error
message that says the specified directory is unreadable. To make the
message more precise, this commit adds a readability check.
Fixes: 2193e1604e ("rcutorture: Abstract kvm-recheck.sh")
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The 'kvm.sh' rcutorture script requires that it be invoked from the top
of Linux-kernel source tree. It is just a subtle restriction, but users
using it for the first time could forget the restriction and be confused.
Moreover, it makes commands a little longer, which can be frustrating.
This commit therefore lets users invoke the script from any location.
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The '--qemu-args' option's help text is wrongly copied from '--qemu-cmd'
option and its argument type description message format is inconsistent
with other arguments. This commit fixes the usage and type messages to
be consistent with others.
Fixes: e9ce640001 ("rcutorture: Add --qemu-args argument to kvm.sh")
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The variable `alldone` is defined but not used within an awk script.
This commit therefore removes it.
Fixes:53954671033d ("rcutorture: Do better bin packing")
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The 'config2frag.sh' script is not used, so this commit removes it.
Fixes: c87b9c601a ("rcutorture: Add KVM-based test framework")
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The 'configinit.sh' script checks the format of optional argument for the
build directory, printing an error message if the format is not valid.
However, the error message uses the wrong variable, indicating an empty
string even though the user entered a non-empty (but erroneous) string.
This commit fixes the script to use the correct variable.
Fixes: c87b9c601a ("rcutorture: Add KVM-based test framework")
Signed-off-by: SeongJae Park <sj38.park@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Because %p prints "(null)" and %pK prints "0000000000000000" or (on
32-bit systems) "00000000", this commit adjusts torture-test scripting
accordingly.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Pull core locking updates from Ingo Molnar:
"The main changes in this cycle are:
- Another attempt at enabling cross-release lockdep dependency
tracking (automatically part of CONFIG_PROVE_LOCKING=y), this time
with better performance and fewer false positives. (Byungchul Park)
- Introduce lockdep_assert_irqs_enabled()/disabled() and convert
open-coded equivalents to lockdep variants. (Frederic Weisbecker)
- Add down_read_killable() and use it in the VFS's iterate_dir()
method. (Kirill Tkhai)
- Convert remaining uses of ACCESS_ONCE() to
READ_ONCE()/WRITE_ONCE(). Most of the conversion was Coccinelle
driven. (Mark Rutland, Paul E. McKenney)
- Get rid of lockless_dereference(), by strengthening Alpha atomics,
strengthening READ_ONCE() with smp_read_barrier_depends() and thus
being able to convert users of lockless_dereference() to
READ_ONCE(). (Will Deacon)
- Various micro-optimizations:
- better PV qspinlocks (Waiman Long),
- better x86 barriers (Michael S. Tsirkin)
- better x86 refcounts (Kees Cook)
- ... plus other fixes and enhancements. (Borislav Petkov, Juergen
Gross, Miguel Bernal Marin)"
* 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (70 commits)
locking/x86: Use LOCK ADD for smp_mb() instead of MFENCE
rcu: Use lockdep to assert IRQs are disabled/enabled
netpoll: Use lockdep to assert IRQs are disabled/enabled
timers/posix-cpu-timers: Use lockdep to assert IRQs are disabled/enabled
sched/clock, sched/cputime: Use lockdep to assert IRQs are disabled/enabled
irq_work: Use lockdep to assert IRQs are disabled/enabled
irq/timings: Use lockdep to assert IRQs are disabled/enabled
perf/core: Use lockdep to assert IRQs are disabled/enabled
x86: Use lockdep to assert IRQs are disabled/enabled
smp/core: Use lockdep to assert IRQs are disabled/enabled
timers/hrtimer: Use lockdep to assert IRQs are disabled/enabled
timers/nohz: Use lockdep to assert IRQs are disabled/enabled
workqueue: Use lockdep to assert IRQs are disabled/enabled
irq/softirqs: Use lockdep to assert IRQs are disabled/enabled
locking/lockdep: Add IRQs disabled/enabled assertion APIs: lockdep_assert_irqs_enabled()/disabled()
locking/pvqspinlock: Implement hybrid PV queued/unfair locks
locking/rwlocks: Fix comments
x86/paravirt: Set up the virt_spin_lock_key after static keys get initialized
block, locking/lockdep: Assign a lock_class per gendisk used for wait_for_completion()
workqueue: Remove now redundant lock acquisitions wrt. workqueue flushes
...
Pull RCU updates from Ingo Molnar:
"The main changes in this cycle are:
- Documentation updates
- RCU CPU stall-warning updates
- Torture-test updates
- Miscellaneous fixes
Size wise the biggest updates are to documentation. Excluding
documentation most of the code increase comes from a single commit
which expands debugging"
* 'core-rcu-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (24 commits)
srcu: Add parameters to SRCU docbook comments
doc: Rewrite confusing statement about memory barriers
memory-barriers.txt: Fix typo in pairing example
rcu/segcblist: Include rcupdate.h
rcu: Add extended-quiescent-state testing advice
rcu: Suppress lockdep false-positive ->boost_mtx complaints
rcu: Do not include rtmutex_common.h unconditionally
torture: Provide TMPDIR environment variable to specify tmpdir
rcutorture: Dump writer stack if stalled
rcutorture: Add interrupt-disable capability to stall-warning tests
rcu: Suppress RCU CPU stall warnings while dumping trace
rcu: Turn off tracing before dumping trace
rcu: Make RCU CPU stall warnings check for irq-disabled CPUs
sched,rcu: Make cond_resched() provide RCU quiescent state
sched: Make resched_cpu() unconditional
irq_work: Map irq_work_on_queue() to irq_work_on() in !SMP
rcu: Create call_rcu_tasks() kthread at boot time
rcu: Fix up pending cbs check in rcu_prepare_for_idle
memory-barriers: Rework multicopy-atomicity section
memory-barriers: Replace uses of "transitive"
...
Many source files in the tree are missing licensing information, which
makes it harder for compliance tools to determine the correct license.
By default all files without license information are under the default
license of the kernel, which is GPL version 2.
Update the files which contain no license information with the 'GPL-2.0'
SPDX license identifier. The SPDX identifier is a legally binding
shorthand, which can be used instead of the full boiler plate text.
This patch is based on work done by Thomas Gleixner and Kate Stewart and
Philippe Ombredanne.
How this work was done:
Patches were generated and checked against linux-4.14-rc6 for a subset of
the use cases:
- file had no licensing information it it.
- file was a */uapi/* one with no licensing information in it,
- file was a */uapi/* one with existing licensing information,
Further patches will be generated in subsequent months to fix up cases
where non-standard license headers were used, and references to license
had to be inferred by heuristics based on keywords.
The analysis to determine which SPDX License Identifier to be applied to
a file was done in a spreadsheet of side by side results from of the
output of two independent scanners (ScanCode & Windriver) producing SPDX
tag:value files created by Philippe Ombredanne. Philippe prepared the
base worksheet, and did an initial spot review of a few 1000 files.
The 4.13 kernel was the starting point of the analysis with 60,537 files
assessed. Kate Stewart did a file by file comparison of the scanner
results in the spreadsheet to determine which SPDX license identifier(s)
to be applied to the file. She confirmed any determination that was not
immediately clear with lawyers working with the Linux Foundation.
Criteria used to select files for SPDX license identifier tagging was:
- Files considered eligible had to be source code files.
- Make and config files were included as candidates if they contained >5
lines of source
- File already had some variant of a license header in it (even if <5
lines).
All documentation files were explicitly excluded.
The following heuristics were used to determine which SPDX license
identifiers to apply.
- when both scanners couldn't find any license traces, file was
considered to have no license information in it, and the top level
COPYING file license applied.
For non */uapi/* files that summary was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 11139
and resulted in the first patch in this series.
If that file was a */uapi/* path one, it was "GPL-2.0 WITH
Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was:
SPDX license identifier # files
---------------------------------------------------|-------
GPL-2.0 WITH Linux-syscall-note 930
and resulted in the second patch in this series.
- if a file had some form of licensing information in it, and was one
of the */uapi/* ones, it was denoted with the Linux-syscall-note if
any GPL family license was found in the file or had no licensing in
it (per prior point). Results summary:
SPDX license identifier # files
---------------------------------------------------|------
GPL-2.0 WITH Linux-syscall-note 270
GPL-2.0+ WITH Linux-syscall-note 169
((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21
((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17
LGPL-2.1+ WITH Linux-syscall-note 15
GPL-1.0+ WITH Linux-syscall-note 14
((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5
LGPL-2.0+ WITH Linux-syscall-note 4
LGPL-2.1 WITH Linux-syscall-note 3
((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3
((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1
and that resulted in the third patch in this series.
- when the two scanners agreed on the detected license(s), that became
the concluded license(s).
- when there was disagreement between the two scanners (one detected a
license but the other didn't, or they both detected different
licenses) a manual inspection of the file occurred.
- In most cases a manual inspection of the information in the file
resulted in a clear resolution of the license that should apply (and
which scanner probably needed to revisit its heuristics).
- When it was not immediately clear, the license identifier was
confirmed with lawyers working with the Linux Foundation.
- If there was any question as to the appropriate license identifier,
the file was flagged for further research and to be revisited later
in time.
In total, over 70 hours of logged manual review was done on the
spreadsheet to determine the SPDX license identifiers to apply to the
source files by Kate, Philippe, Thomas and, in some cases, confirmation
by lawyers working with the Linux Foundation.
Kate also obtained a third independent scan of the 4.13 code base from
FOSSology, and compared selected files where the other two scanners
disagreed against that SPDX file, to see if there was new insights. The
Windriver scanner is based on an older version of FOSSology in part, so
they are related.
Thomas did random spot checks in about 500 files from the spreadsheets
for the uapi headers and agreed with SPDX license identifier in the
files he inspected. For the non-uapi files Thomas did random spot checks
in about 15000 files.
In initial set of patches against 4.14-rc6, 3 files were found to have
copy/paste license identifier errors, and have been fixed to reflect the
correct identifier.
Additionally Philippe spent 10 hours this week doing a detailed manual
inspection and review of the 12,461 patched files from the initial patch
version early this week with:
- a full scancode scan run, collecting the matched texts, detected
license ids and scores
- reviewing anything where there was a license detected (about 500+
files) to ensure that the applied SPDX license was correct
- reviewing anything where there was no detection but the patch license
was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
SPDX license was correct
This produced a worksheet with 20 files needing minor correction. This
worksheet was then exported into 3 different .csv files for the
different types of files to be modified.
These .csv files were then reviewed by Greg. Thomas wrote a script to
parse the csv files and add the proper SPDX tag to the file, in the
format that the file expected. This script was further refined by Greg
based on the output to detect more types of files automatically and to
distinguish between header and source .c files (which need different
comment types.) Finally Greg ran the script using the .csv files to
generate the patches.
Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org>
Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com>
Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
For several reasons, it is desirable to use {READ,WRITE}_ONCE() in
preference to ACCESS_ONCE(), and new code is expected to use one of the
former. So far, there's been no reason to change most existing uses of
ACCESS_ONCE(), as these aren't currently harmful.
However, for some features it is necessary to instrument reads and
writes separately, which is not possible with ACCESS_ONCE(). This
distinction is critical to correct operation.
The bulk of the kernel code can be transformed via Coccinelle to use
{READ,WRITE}_ONCE(), though this only modifies users of ACCESS_ONCE(),
and not the implementation itself. As such, it has the potential to
break homebrew ACCESS_ONCE() macros seen in some user code in the kernel
tree (e.g. the virtio code, as fixed in commit ea9156fb3b).
To avoid fragility if/when that transformation occurs, this patch
reworks the definitions of {READ,WRITE}_ONCE() in the rcutorture formal
tests, and removes the unused ACCESS_ONCE() helper. There should be no
functional change as a result of this patch.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: davem@davemloft.net
Cc: linux-arch@vger.kernel.org
Cc: mpe@ellerman.id.au
Cc: shuah@kernel.org
Cc: snitzer@redhat.com
Cc: thor.thayer@linux.intel.com
Cc: tj@kernel.org
Cc: viro@zeniv.linux.org.uk
Cc: will.deacon@arm.com
Link: http://lkml.kernel.org/r/1508792849-3115-13-git-send-email-paulmck@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Both rcutorture and locktorture currently place temporary files in /tmp,
in keeping with decades-long tradition. However, sometimes it is useful
to specify an alternative temporary directory, for example, for space
or performance reasons. This commit therefore causes the torture-test
scripting to use the path specified in the TMPDIR environment variable,
or to fall back to traditional /tmp if this variable is not set.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The actual use of TASKS_RCU is only when PREEMPT, otherwise RCU-sched
is used instead. This commit therefore makes synchronize_rcu_tasks()
and call_rcu_tasks() available always, but mapped to synchronize_sched()
and call_rcu_sched(), respectively, when !PREEMPT. This approach also
allows some #ifdefs to be removed from rcutorture.
Reported-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Masami Hiramatsu <mhiramat@kernel.org>
Acked-by: Ingo Molnar <mingo@kernel.org>
The maxcpus= kernel boot parameter limits the number of CPUs brought
online at boot time, but it does nothing to prevent additional CPUs
from being brought up later. Placing a hard cap on the total number
of CPUs is instead the job of the nr_cpus= boot parameter. This commit
therefore switches the configfrag_boot_cpus() shell function from maxcpus=
to nr_cpus=. This commit also adds a nr_cpus=43 kernel parameter to RCU's
TREE01 test scenario, but retains the maxcpus=8 kernel parameter in order
to test the ability of RCU expedited grace periods to handle new CPUs
coming online for the first time during grace-period initialization.
Finally, this commit makes the torture scheduling allow maxcpus= to
override other means of specifying the number of CPUs to allow for.
This last works because the torture kernel modules size their workloads
based on the number of CPUs present at the start of the test, not the
ultimate number of CPUs.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Strings used in event tracing need to be specially handled, for example,
being copied to the trace buffer instead of being pointed to by the trace
buffer. Although the TPS() macro can be used to "launder" pointed-to
strings, this might not be all that effective within a loadable module.
This commit therefore copies rcutorture's strings to the trace buffer.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Steven Rostedt <rostedt@goodmis.org>
Currently, rcutorture groups runs in batches, building each scenario in
a given batch, then invoking qemu to run all the kernels in the batch.
Of course, if a given scenario's kernel fails to build, there is no qemu
run for that scenario. And if all of the kernels in a given batch fail
to build, there are no runs, and rcutorture immediately starts on the
next batch.
But not if --jitter has been specified, which it is by default. In this
case, the jitter scripts are started unconditionally, and rcutorture
waits for them to complete, even though there are no kernels to run.
This commit therefore checks for this situation, and refuses to start
jitter unless at least one of the kernels in the batch built successfully.
This saves substantial time when all scenarios' kernels fail to build,
particularly if a long --duration was specified.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, testing a variant of an existing scenario requires editing
that scenario's file or creating a new scenario file. This is messy
and error prone with respect to changes to scenarios.
This commit therefore adds a --kconfig argument to kvm.sh, so that
'--kconfig "CONFIG_RCU_TRACE=y CONFIG_RCU_EQS_DEBUG=n" will override those
two Kconfig options. In addition, there is now clear precedence:
the config fragment overrides CFcommon, and the --kconfig argument
overrides both.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit selects CONFIG_PROVE_LOCKING for the SRCU-u scenario
to get better test coverage.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Classic SRCU is no more, so this commit removes the corresponding
rcutorture boot-parameters file.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Tiny RCU's job is to be tiny, so this commit removes its RCU CPU
stall warning code. After this, there is no longer any need for
rcu_sched_ctrlblk and rcu_bh_ctrlblk to be in tiny_plugin.h, so this
commit also moves them to tiny.c.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The CONFIG_RCU_NOCB_CPU_ALL, CONFIG_RCU_NOCB_CPU_NONE, and
CONFIG_RCU_NOCB_CPU_ZERO Kconfig options are used only in testing and
are redundant with the rcu_nocbs= boot parameter. This commit therefore
removes these three Kconfig options and adjusts the rcutorture scripts
to use the boot parameter instead.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
RCU's debugfs tracing used to be the only reasonable low-level debug
information available, but ftrace and event tracing has since surpassed
the RCU debugfs level of usefulness. This commit therefore removes
RCU's debugfs tracing.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Classic SRCU was only ever intended to be a fallback in case of issues
with Tree/Tiny SRCU, and the latter two are doing quite well in testing.
This commit therefore removes Classic SRCU.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The sparse-based checking for non-RCU accesses to RCU-protected pointers
has been around for a very long time, and it is now the only type of
sparse-based checking that is optional. This commit therefore makes
it unconditional.
Reported-by: Ingo Molnar <mingo@kernel.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Fengguang Wu <fengguang.wu@intel.com>
The PROVE_RCU_REPEATEDLY Kconfig option was initially added due to
the volume of messages from PROVE_RCU: Doing just one per boot would
have required excessive numbers of boots to locate them all. However,
PROVE_RCU messages are now relatively rare, so there is no longer any
reason to need more than one such message per boot. This commit therefore
removes the PROVE_RCU_REPEATEDLY Kconfig option.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Ingo Molnar <mingo@kernel.org>
The NO_HZ_FULL_SYSIDLE full-system-idle capability was added in 2013
by commit 0edd1b1784 ("nohz_full: Add full-system-idle state machine"),
but has not been used. This commit therefore removes it.
If it turns out to be needed later, this commit can always be reverted.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Ingo Molnar <mingo@kernel.org>
Acked-by: Linus Torvalds <torvalds@linux-foundation.org>
Anything that can be done with the RCU_KTHREAD_PRIO Kconfig option can
also be done with the rcutree.kthread_prio kernel boot parameter.
This commit therefore removes this Kconfig option.
Reported-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Rik van Riel <riel@redhat.com>
The RCU_TORTURE_TEST_SLOW_PREINIT, RCU_TORTURE_TEST_SLOW_PREINIT_DELAY,
RCU_TORTURE_TEST_SLOW_PREINIT_DELAY, RCU_TORTURE_TEST_SLOW_INIT,
RCU_TORTURE_TEST_SLOW_INIT_DELAY, RCU_TORTURE_TEST_SLOW_CLEANUP,
and RCU_TORTURE_TEST_SLOW_CLEANUP_DELAY Kconfig options are only
useful for torture testing, and there are the rcutree.gp_cleanup_delay,
rcutree.gp_init_delay, and rcutree.gp_preinit_delay kernel boot parameters
that rcutorture can use instead. The effect of these parameters is to
artificially slow down grace period initialization and cleanup in order
to make some types of race conditions happen more often.
This commit therefore simplifies Tree RCU a bit by removing the Kconfig
options and adding the corresponding kernel parameters to rcutorture's
.boot files instead. However, this commit also leaves out the kernel
parameters for TREE02, TREE04, and TREE07 in order to have about the
same number of tests slowed as not slowed. TREE01, TREE03, TREE05,
and TREE06 are slowed, and the rest are not slowed.
Reported-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, when running from a git archive, the testid.txt file contains
only the branch name, the output of "git status", and the SHA-1 of
the current HEAD. This is useful, but does not uniquely identify the
source code that was built. This commit therefore adds the output of
"git diff HEAD", which means that if two testid.txt files compare equal,
they correspond to exactly the same source code. Give or take the
possibility of SHA-1 collisions, that is. ;-)
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Most OS distribution have awk in /usr/bin not in /bin
Without this patch, kernel-devsrc fails to build as
runtime dependency for srcu-cbmc script /bin/awk is
not found.
Signed-off-by: Kushwaha, Priyalee <priyalee.kushwaha@intel.com>
Acked-by: Lance Roy <ldr709@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Given that the plan is to retire Classic SRCU in the near future, this
commit reduces the number of CPUs dedicated to testing Classic SRCU.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds a TINY rcuperf test scenario, which allows performance
testing of Tiny RCU and Tiny SRCU.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds a Kconfig-fragment file for Classic SRCU to ease
performance comparisons with Tree SRCU.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The TREE and TREE54 rcuperf scenarios' Kconfig fragment files specified
conflicting values for CONFIG_RCU_TRACE. This commit therefore removes
the =n line in favor of the =y line.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
A number of the rcutorture test scenarios were not using the desired
Kconfig options because dependencies were preventing the selections in the
Kconfig-fragment files from being honored. This commit therefore updates
the Kconfig-fragment files to account for these changes in dependencies.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The rcutorture scripting handles the CONFIG_*_TORTURE_TEST Kconfig
options specially, and therefore greps them out of the Kconfig-fragment
files. Unfortunately, a poor choice of grep pattern means that the
CONFIG_RCU_TORTURE_TEST_SLOW_CLEANUP, CONFIG_RCU_TORTURE_TEST_SLOW_INIT,
and CONFIG_RCU_TORTURE_TEST_SLOW_PREINIT Kconfig options are also grepped
out, preventing rcutorture from using them. This commit therefore fixes
the offending grep pattern to focus only on the CONFIG_*_TORTURE_TEST
Kconfig options.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
A robust combination of paranoia and cowardice has resulted in retaining
Classic SRCU (CONFIG_CLASSIC_SRCU) as a backup for the shiny new Tiny
and Tree SRCU implementations. If it is to be a viable backup, it of
course needs to be tested. This commit therefore adds an rcutorture
scenario named SRCU-C for Classic SRCU. This commit also adds this
scenario to the set that are run by default.
Once sufficient good experience has accumulated for Tiny and Tree SRCU,
this test will be removed, along with the Classic SRCU implementation
itself.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds an SRCU-t rcutorture scenario for the new Tiny SRCU
implementation, removing the need to pass the --bootargs parameter to
kvm.sh to run Tiny SRCU tests. This commit also adds SRCU-t to the set
of scenarios that are run by default.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Kconfig "select" clauses can defeat Kconfig-fragment file attempts to
clear a given Kconfig variable, and dependencies can defeat attempts to
set a given Kconfig variable. Because "select" clauses and dependencies
can be added at any time, there needs to be a way to verify that the
Kconfig-fragment file's requests were honored. And there is, except
that it is buggy. This commit therefore provides the needed fix.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds a test for a three-level srcu_node tree for Tree SRCU
in the existing SRCU-P scenario. This requires enabling CONFIG_RCU_EXPERT,
so the CONFIG_RCU_EXPERT=n scenario is now SRCU-N. The reason for using
SRCU-P for the tall tree is that preemption raises the possibility of
locating more bugs than does the non-preemptive SRCU-N.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Back when SRCU was simpler, there wasn't much need for lockdep.
However, with Tree SRCU, it is needed. This commit therefore adds
CONFIG_PROVE_LOCKING to the SRCU-P scenario.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Currently, the rcutorture scripting will give an error message if
running a duplicate scenario that happens also to have a non-existent
build directory (b1, b2, ... in the rcutorture directory). Worse yet, if
the build directory has already been created and used for a real build,
the script will silently grab the wrong Kconfig fragment, which could
cause confusion to the poor sap (me) analyzing old test results. At
least the actual test runs correctly...
This commit therefore accesses the Kconfig fragment from the results
directory corresponding to the first of the duplicate scenarios, for
which a build was actually carried out. This prevents both the messages
and at least one form of later confusion.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Pull locking updates from Ingo Molnar:
"The main changes in this cycle were:
- Implement wraparound-safe refcount_t and kref_t types based on
generic atomic primitives (Peter Zijlstra)
- Improve and fix the ww_mutex code (Nicolai Hähnle)
- Add self-tests to the ww_mutex code (Chris Wilson)
- Optimize percpu-rwsems with the 'rcuwait' mechanism (Davidlohr
Bueso)
- Micro-optimize the current-task logic all around the core kernel
(Davidlohr Bueso)
- Tidy up after recent optimizations: remove stale code and APIs,
clean up the code (Waiman Long)
- ... plus misc fixes, updates and cleanups"
* 'locking-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (50 commits)
fork: Fix task_struct alignment
locking/spinlock/debug: Remove spinlock lockup detection code
lockdep: Fix incorrect condition to print bug msgs for MAX_LOCKDEP_CHAIN_HLOCKS
lkdtm: Convert to refcount_t testing
kref: Implement 'struct kref' using refcount_t
refcount_t: Introduce a special purpose refcount type
sched/wake_q: Clarify queue reinit comment
sched/wait, rcuwait: Fix typo in comment
locking/mutex: Fix lockdep_assert_held() fail
locking/rtmutex: Flip unlikely() branch to likely() in __rt_mutex_slowlock()
locking/rwsem: Reinit wake_q after use
locking/rwsem: Remove unnecessary atomic_long_t casts
jump_labels: Move header guard #endif down where it belongs
locking/atomic, kref: Implement kref_put_lock()
locking/ww_mutex: Turn off __must_check for now
locking/atomic, kref: Avoid more abuse
locking/atomic, kref: Use kref_get_unless_zero() more
locking/atomic, kref: Kill kref_sub()
locking/atomic, kref: Add kref_read()
locking/atomic, kref: Add KREF_INIT()
...
This commit creates a formal/srcu-cbmc directory containing scripts that
pull SRCU in from the source code, filter it to remove things that CBMC
cannot handle, and run a series of verifications on it. This has a number
of shortcomings:
1. It does not yet hook into the upper-level self-test Makefiles.
2. It tests only a single scenario, store buffering.
3. There is no gcc-based syntax-error prefiltering.
Nevertheless, it does fully verify a piece of SRCU under a moderately
weak memory model (PSO).
Signed-off-by: Lance Roy <ldr709@gmail.com>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The RCU torture tests currently do not run any Tiny RCU scenarios for
CONFIG_DEBUG_OBJECTS_RCU_HEAD=y. This is a hole in the test, given
that someone might need this in real life and given that Tiny RCU uses
different callback-handling code than does Tree RCU. This commit
therefore enables this Kconfig option for scenario TINY02.
Reported-by: "Ahmed, Iftekhar" <ahmedi@oregonstate.edu>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit runs TREE04 and TREE08 with CONFIG_RCU_EQS_DEBUG=y,
enabling dyntick-counter checking on those two tests.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit sets CONFIG_DEBUG_LOCK_ALLOC but not CONFIG_PROVE_LOCKING
for TREE08 in order to have at least one test with this configuration.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit enables the CONFIG_DEBUG_OBJECTS_RCU_HEAD Kconfig option
in TREE02 in order to do at least some testing with this enabled.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit moves CONFIG_RCU_TORTURE_TEST_SLOW_CLEANUP,
CONFIG_RCU_TORTURE_TEST_SLOW_INIT, and CONFIG_RCU_TORTURE_TEST_SLOW_PREINIT
from CFcommon to all of the TREE scenarios other than TREE08 and TREE09
in order to do at least some testing without these Kconfig options set.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit adds CONFIG_PROVE_RCU_REPEATEDLY=y, which has been untested
for quite some time.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit verifies coverage of testing with CONFIG_RCU_STALL_COMMON=n.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Although ww_mutexes degenerate into mutexes, it would be useful to
torture the deadlock handling between multiple ww_mutexes in addition to
torturing the regular mutexes.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Maarten Lankhorst <dev@mblankhorst.nl>
Cc: Nicolai Hähnle <nhaehnle@gmail.com>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Link: http://lkml.kernel.org/r/20161201114711.28697-3-chris@chris-wilson.co.uk
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Currently, if the --jitter flag specifies jitter for a --build-only
run, the system will obediently build a kernel, refuse to launch it,
launch the requested number of jitter processes, and wait for the
specified kernel run time, which defaults to 30 minutes. This is
of course quite pointless.
This commit therefore disables jitter on build-only runs.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
This commit enables jitter by default. It may be manually disabled
by passing "--jitter 0" to kvm.sh.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Because recent testing shows that "-soundhw pcspkr" is no longer required
in the kernel boot arguments, this commit drops this qemu argument.
Reported-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Do not restrict the cpu type to POWER7 for QEMU as we have POWER8 now.
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The option "-soundhw pcspk" gives me a error on PPC as follow:
qemu-system-ppc64: ISA bus not available for pcspk
This means this option doesn't work on ppc by default. So simply make
this an x86-specific option via identify_qemu_args().
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
The vmlinux image is available for all the architectures, and suitable
for running a KVM guest by QEMU, besides, we used to copy the vmlinux
to $resdir anyway. Therefore it makes sense to use it as the fallback
kernel image for rcutorture KVM tests.
This patch makes identify_boot_image() return vmlinux if
${TORTURE_BOOT_IMAGE} is not set on non-x86 architectures, also fixes
several places that hard-code "bzImage" as $KERNEL.
This also fixes a problem that PPC doesn't have a bzImage file as build
results.
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Using dracut is another way to get an initramfs for KVM-based RCU
torture tests, which is more flexible than using the host's initramfs
image, because modules and binaries may be added or removed via dracut
command options. So add an example in the document, in case that there
are some situations where host's initramfs couldn't be used.
Signed-off-by: Boqun Feng <boqun.feng@gmail.com>
Reviewed-by: Josh Triplett <josh@joshtriplett.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
This commit adds a string of the form "Starves: 10" to the summary
line for error conditions found in the console output.
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
