Go to file
Andrey Konovalov 44383cef54 kasan: allow sampling page_alloc allocations for HW_TAGS
As Hardware Tag-Based KASAN is intended to be used in production, its
performance impact is crucial.  As page_alloc allocations tend to be big,
tagging and checking all such allocations can introduce a significant
slowdown.

Add two new boot parameters that allow to alleviate that slowdown:

- kasan.page_alloc.sample, which makes Hardware Tag-Based KASAN tag only
  every Nth page_alloc allocation with the order configured by the second
  added parameter (default: tag every such allocation).

- kasan.page_alloc.sample.order, which makes sampling enabled by the first
  parameter only affect page_alloc allocations with the order equal or
  greater than the specified value (default: 3, see below).

The exact performance improvement caused by using the new parameters
depends on their values and the applied workload.

The chosen default value for kasan.page_alloc.sample.order is 3, which
matches both PAGE_ALLOC_COSTLY_ORDER and SKB_FRAG_PAGE_ORDER.  This is
done for two reasons:

1. PAGE_ALLOC_COSTLY_ORDER is "the order at which allocations are deemed
   costly to service", which corresponds to the idea that only large and
   thus costly allocations are supposed to sampled.

2. One of the workloads targeted by this patch is a benchmark that sends
   a large amount of data over a local loopback connection. Most multi-page
   data allocations in the networking subsystem have the order of
   SKB_FRAG_PAGE_ORDER (or PAGE_ALLOC_COSTLY_ORDER).

When running a local loopback test on a testing MTE-enabled device in sync
mode, enabling Hardware Tag-Based KASAN introduces a ~50% slowdown. 
Applying this patch and setting kasan.page_alloc.sampling to a value
higher than 1 allows to lower the slowdown.  The performance improvement
saturates around the sampling interval value of 10 with the default
sampling page order of 3.  This lowers the slowdown to ~20%.  The slowdown
in real scenarios involving the network will likely be better.

Enabling page_alloc sampling has a downside: KASAN misses bad accesses to
a page_alloc allocation that has not been tagged.  This lowers the value
of KASAN as a security mitigation.

However, based on measuring the number of page_alloc allocations of
different orders during boot in a test build, sampling with the default
kasan.page_alloc.sample.order value affects only ~7% of allocations.  The
rest ~93% of allocations are still checked deterministically.

Link: https://lkml.kernel.org/r/129da0614123bb85ed4dd61ae30842b2dd7c903f.1671471846.git.andreyknvl@google.com
Signed-off-by: Andrey Konovalov <andreyknvl@google.com>
Reviewed-by: Marco Elver <elver@google.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Dmitry Vyukov <dvyukov@google.com>
Cc: Evgenii Stepanov <eugenis@google.com>
Cc: Jann Horn <jannh@google.com>
Cc: Mark Brand <markbrand@google.com>
Cc: Peter Collingbourne <pcc@google.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2023-01-18 17:12:45 -08:00
arch mm/uffd: always wr-protect pte in pte|pmd_mkuffd_wp() 2023-01-18 17:12:37 -08:00
block block-6.2-2023-01-13 2023-01-13 17:41:19 -06:00
certs certs: make system keyring depend on built-in x509 parser 2022-09-24 04:31:18 +09:00
crypto This update includes the following changes: 2022-12-14 12:31:09 -08:00
Documentation kasan: allow sampling page_alloc allocations for HW_TAGS 2023-01-18 17:12:45 -08:00
drivers - Fix the EDAC device's confusion in the polling setting units 2023-01-15 07:12:58 -06:00
fs mpage: use b_folio in do_mpage_readpage() 2023-01-18 17:12:41 -08:00
include kasan: allow sampling page_alloc allocations for HW_TAGS 2023-01-18 17:12:45 -08:00
init Sync with v6.2-rc4 2023-01-18 16:52:20 -08:00
io_uring io_uring: lock overflowing for IOPOLL 2023-01-13 07:32:46 -07:00
ipc Non-MM patches for 6.2-rc1. 2022-12-12 17:28:58 -08:00
kernel mm/memfd: add MFD_NOEXEC_SEAL and MFD_EXEC 2023-01-18 17:12:37 -08:00
lib lib/test_vmalloc.c: add parameter use_huge for fix_size_alloc_test 2023-01-18 17:12:41 -08:00
LICENSES LICENSES: Add the copyleft-next-0.3.1 license 2022-11-08 15:44:01 +01:00
mm kasan: allow sampling page_alloc allocations for HW_TAGS 2023-01-18 17:12:45 -08:00
net Including fixes from rxrpc. 2023-01-12 18:20:44 -06:00
rust rust: types: add Opaque type 2022-12-04 01:59:16 +01:00
samples Char/Misc driver changes for 6.2-rc1 2022-12-16 03:49:24 -08:00
scripts kernel hardening fixes for v6.2-rc4 2023-01-14 10:04:00 -06:00
security kernel hardening fixes for v6.2-rc1 2022-12-23 12:00:24 -08:00
sound sound fixes for 6.2-rc4 2023-01-13 08:20:29 -06:00
tools selftests/damon/sysfs: test filters directory 2023-01-18 17:12:44 -08:00
usr usr/gen_init_cpio.c: remove unnecessary -1 values from int file 2022-10-03 14:21:44 -07:00
virt KVM: Ensure lockdep knows about kvm->lock vs. vcpu->mutex ordering rule 2023-01-11 13:32:21 -05:00
.clang-format iommufd for 6.2 2022-12-14 09:15:43 -08:00
.cocciconfig
.get_maintainer.ignore get_maintainer: add Alan to .get_maintainer.ignore 2022-08-20 15:17:44 -07:00
.gitattributes .gitattributes: use 'dts' diff driver for dts files 2019-12-04 19:44:11 -08:00
.gitignore .gitignore: ignore *.rpm 2022-12-30 17:22:14 +09:00
.mailmap Sync with v6.2-rc4 2023-01-18 16:52:20 -08:00
.rustfmt.toml rust: add .rustfmt.toml 2022-09-28 09:02:20 +02:00
COPYING COPYING: state that all contributions really are covered by this file 2020-02-10 13:32:20 -08:00
CREDITS MAINTAINERS: zram: zsmalloc: Add an additional co-maintainer 2022-12-15 16:37:49 -08:00
Kbuild Kbuild updates for v6.1 2022-10-10 12:00:45 -07:00
Kconfig kbuild: ensure full rebuild when the compiler is updated 2020-05-12 13:28:33 +09:00
MAINTAINERS Sync with v6.2-rc4 2023-01-18 16:52:20 -08:00
Makefile Linux 6.2-rc4 2023-01-15 09:22:43 -06:00
README Drop all 00-INDEX files from Documentation/ 2018-09-09 15:08:58 -06:00

Linux kernel

There are several guides for kernel developers and users. These guides can be rendered in a number of formats, like HTML and PDF. Please read Documentation/admin-guide/README.rst first.

In order to build the documentation, use make htmldocs or make pdfdocs. The formatted documentation can also be read online at:

https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory, several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the requirements for building and running the kernel, and information about the problems which may result by upgrading your kernel.