
The commit 93e72b3c61
("squashfs: migrate from ll_rw_block usage
to BIO") removed caching of compressed blocks in SquashFS, causing fio
performance regression in workloads with repeated file reads. Without
caching, every read triggers disk I/O, severely impacting performance in
tools like fio.
This patch introduces a new CONFIG_SQUASHFS_COMP_CACHE_FULL Kconfig option
to enable caching of all compressed blocks, restoring performance to
pre-BIO migration levels. When enabled, all pages in a BIO are cached in
the page cache, reducing disk I/O for repeated reads. The fio test
results with this patch confirm the performance restoration:
For example, fio tests (iodepth=1, numjobs=1,
ioengine=psync) show a notable performance restoration:
Disable CONFIG_SQUASHFS_COMP_CACHE_FULL:
IOPS=815, BW=102MiB/s (107MB/s)(6113MiB/60001msec)
Enable CONFIG_SQUASHFS_COMP_CACHE_FULL:
IOPS=2223, BW=278MiB/s (291MB/s)(16.3GiB/59999msec)
The tradeoff is increased memory usage due to caching all compressed
blocks. The CONFIG_SQUASHFS_COMP_CACHE_FULL option allows users to enable
this feature selectively, balancing performance and memory usage for
workloads with frequent repeated reads.
Link: https://lkml.kernel.org/r/20250521072559.2389-1-chanho.min@lge.com
Signed-off-by: Chanho Min <chanho.min@lge.com>
Reviewed-by Phillip Lougher <phillip@squashfs.org.uk>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
10 KiB
SPDX-License-Identifier: GPL-2.0-only
config SQUASHFS tristate "SquashFS 4.0 - Squashed file system support" depends on BLOCK help Saying Y here includes support for SquashFS 4.0 (a Compressed Read-Only File System). Squashfs is a highly compressed read-only filesystem for Linux. It uses zlib, lz4, lzo, xz or zstd compression to compress both files, inodes and directories. Inodes in the system are very small and all blocks are packed to minimise data overhead. Block sizes greater than 4K are supported up to a maximum of 1 Mbytes (default block size 128K). SquashFS 4.0 supports 64 bit filesystems and files (larger than 4GB), full uid/gid information, hard links and timestamps.
Squashfs is intended for general read-only filesystem use, for
archival use (i.e. in cases where a .tar.gz file may be used), and in
embedded systems where low overhead is needed. Further information
and tools are available from github.com/plougher/squashfs-tools.
If you want to compile this as a module ( = code which can be
inserted in and removed from the running kernel whenever you want),
say M here. The module will be called squashfs. Note that the root
file system (the one containing the directory /) cannot be compiled
as a module.
If unsure, say N.
choice prompt "File decompression options" depends on SQUASHFS help Squashfs now supports two options for decompressing file data. Traditionally Squashfs has decompressed into an intermediate buffer and then memcopied it into the page cache. Squashfs now supports the ability to decompress directly into the page cache.
If unsure, select "Decompress file data into an intermediate buffer"
config SQUASHFS_FILE_CACHE bool "Decompress file data into an intermediate buffer" help Decompress file data into an intermediate buffer and then memcopy it into the page cache.
config SQUASHFS_FILE_DIRECT bool "Decompress files directly into the page cache" help Directly decompress file data into the page cache. Doing so can significantly improve performance because it eliminates a memcpy and it also removes the lock contention on the single buffer.
endchoice
config SQUASHFS_DECOMP_SINGLE depends on SQUASHFS def_bool n
config SQUASHFS_DECOMP_MULTI depends on SQUASHFS def_bool n
config SQUASHFS_DECOMP_MULTI_PERCPU depends on SQUASHFS def_bool n
config SQUASHFS_CHOICE_DECOMP_BY_MOUNT bool "Select the parallel decompression mode during mount" depends on SQUASHFS default n select SQUASHFS_DECOMP_SINGLE select SQUASHFS_DECOMP_MULTI select SQUASHFS_DECOMP_MULTI_PERCPU select SQUASHFS_MOUNT_DECOMP_THREADS help Compile all parallel decompression modes and specify the decompression mode by setting "threads=" during mount. default Decompressor parallelisation is SQUASHFS_DECOMP_SINGLE
choice prompt "Select decompression parallel mode at compile time" depends on SQUASHFS depends on !SQUASHFS_CHOICE_DECOMP_BY_MOUNT help Squashfs now supports three parallelisation options for decompression. Each one exhibits various trade-offs between decompression performance and CPU and memory usage.
If in doubt, select "Single threaded compression"
config SQUASHFS_COMPILE_DECOMP_SINGLE bool "Single threaded compression" select SQUASHFS_DECOMP_SINGLE help Traditionally Squashfs has used single-threaded decompression. Only one block (data or metadata) can be decompressed at any one time. This limits CPU and memory usage to a minimum.
config SQUASHFS_COMPILE_DECOMP_MULTI bool "Use multiple decompressors for parallel I/O" select SQUASHFS_DECOMP_MULTI help By default Squashfs uses a single decompressor but it gives poor performance on parallel I/O workloads when using multiple CPU machines due to waiting on decompressor availability.
If you have a parallel I/O workload and your system has enough memory,
using this option may improve overall I/O performance.
This decompressor implementation uses up to two parallel
decompressors per core. It dynamically allocates decompressors
on a demand basis.
config SQUASHFS_COMPILE_DECOMP_MULTI_PERCPU bool "Use percpu multiple decompressors for parallel I/O" select SQUASHFS_DECOMP_MULTI_PERCPU help By default Squashfs uses a single decompressor but it gives poor performance on parallel I/O workloads when using multiple CPU machines due to waiting on decompressor availability.
This decompressor implementation uses a maximum of one
decompressor per core. It uses percpu variables to ensure
decompression is load-balanced across the cores.
endchoice
config SQUASHFS_MOUNT_DECOMP_THREADS bool "Add the mount parameter 'threads=' for squashfs" depends on SQUASHFS depends on SQUASHFS_DECOMP_MULTI default n help Use threads= to set the decompression parallel mode and the number of threads. If SQUASHFS_CHOICE_DECOMP_BY_MOUNT=y threads=<single|multi|percpu|1|2|3|...> else threads=<2|3|...> The upper limit is num_online_cpus() * 2.
config SQUASHFS_XATTR bool "Squashfs XATTR support" depends on SQUASHFS help Saying Y here includes support for extended attributes (xattrs). Xattrs are name:value pairs associated with inodes by the kernel or by users (see the attr(5) manual page).
If unsure, say N.
config SQUASHFS_COMP_CACHE_FULL bool "Enable full caching of compressed blocks" depends on SQUASHFS default n help This option enables caching of all compressed blocks, Without caching, repeated reads of the same files trigger excessive disk I/O, significantly reducinng performance in workloads like fio-based benchmarks.
For example, fio tests (iodepth=1, numjobs=1, ioengine=psync) show:
With caching: IOPS=2223, BW=278MiB/s (291MB/s)
Without caching: IOPS=815, BW=102MiB/s (107MB/s)
Enabling this option restores performance to pre-regression levels by
caching all compressed blocks in the page cache, reducing disk I/O for
repeated reads. However, this increases memory usage, which may be a
concern in memory-constrained environments.
Enable this option if your workload involves frequent repeated reads and
memory usage is not a limiting factor. If unsure, say N.
config SQUASHFS_ZLIB bool "Include support for ZLIB compressed file systems" depends on SQUASHFS select ZLIB_INFLATE default y help ZLIB compression is the standard compression used by Squashfs file systems. It offers a good trade-off between compression achieved and the amount of CPU time and memory necessary to compress and decompress.
If unsure, say Y.
config SQUASHFS_LZ4 bool "Include support for LZ4 compressed file systems" depends on SQUASHFS select LZ4_DECOMPRESS help Saying Y here includes support for reading Squashfs file systems compressed with LZ4 compression. LZ4 compression is mainly aimed at embedded systems with slower CPUs where the overheads of zlib are too high.
LZ4 is not the standard compression used in Squashfs and so most
file systems will be readable without selecting this option.
If unsure, say N.
config SQUASHFS_LZO bool "Include support for LZO compressed file systems" depends on SQUASHFS select LZO_DECOMPRESS help Saying Y here includes support for reading Squashfs file systems compressed with LZO compression. LZO compression is mainly aimed at embedded systems with slower CPUs where the overheads of zlib are too high.
LZO is not the standard compression used in Squashfs and so most
file systems will be readable without selecting this option.
If unsure, say N.
config SQUASHFS_XZ bool "Include support for XZ compressed file systems" depends on SQUASHFS select XZ_DEC help Saying Y here includes support for reading Squashfs file systems compressed with XZ compression. XZ gives better compression than the default zlib compression, at the expense of greater CPU and memory overhead.
XZ is not the standard compression used in Squashfs and so most
file systems will be readable without selecting this option.
If unsure, say N.
config SQUASHFS_ZSTD bool "Include support for ZSTD compressed file systems" depends on SQUASHFS select ZSTD_DECOMPRESS help Saying Y here includes support for reading Squashfs file systems compressed with ZSTD compression. ZSTD gives better compression than the default ZLIB compression, while using less CPU.
ZSTD is not the standard compression used in Squashfs and so most
file systems will be readable without selecting this option.
If unsure, say N.
config SQUASHFS_4K_DEVBLK_SIZE bool "Use 4K device block size?" depends on SQUASHFS help By default Squashfs sets the dev block size (sb_min_blocksize) to 1K or the smallest block size supported by the block device (if larger). This, because blocks are packed together and unaligned in Squashfs, should reduce latency.
This, however, gives poor performance on MTD NAND devices where
the optimal I/O size is 4K (even though the devices can support
smaller block sizes).
Using a 4K device block size may also improve overall I/O
performance for some file access patterns (e.g. sequential
accesses of files in filesystem order) on all media.
Setting this option will force Squashfs to use a 4K device block
size by default.
If unsure, say N.
config SQUASHFS_EMBEDDED bool "Additional option for memory-constrained systems" depends on SQUASHFS help Saying Y here allows you to specify cache size.
If unsure, say N.
config SQUASHFS_FRAGMENT_CACHE_SIZE int "Number of fragments cached" if SQUASHFS_EMBEDDED depends on SQUASHFS default "3" help By default SquashFS caches the last 3 fragments read from the filesystem. Increasing this amount may mean SquashFS has to re-read fragments less often from disk, at the expense of extra system memory. Decreasing this amount will mean SquashFS uses less memory at the expense of extra reads from disk.
Note there must be at least one cached fragment. Anything
much more than three will probably not make much difference.