Pavel Begunkov 873ebaf3c1 io_uring: fix overflow resched cqe reordering ... [ Upstream commit a7d755ed9ce9738af3db602eb29d32774a180bc7 ] Leaving the CQ critical section in the middle of a overflow flushing can cause cqe reordering since the cache cq pointers are reset and any new cqe emitters that might get called in between are not going to be forced into io_cqe_cache_refill(). Fixes: eac2ca2d68 ("io_uring: check if we need to reschedule during overflow flush") Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Link: https://lore.kernel.org/r/90ba817f1a458f091f355f407de1c911d2b93bbf.1747483784.git.asml.silence@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk> Signed-off-by: Sasha Levin <sashal@kernel.org>		2025-05-29 11:03:17 +02:00
..
advise.c	io_uring/advise: support 64-bit lengths	2024-06-16 14:54:55 -06:00
advise.h
alloc_cache.h	io_uring/alloc_cache: switch to array based caching	2024-04-15 08:10:25 -06:00
cancel.c	io_uring: fix warnings on shadow variables	2024-04-15 08:10:26 -06:00
cancel.h	io_uring: fix cancellation overwriting req->flags	2024-06-13 19:25:28 -06:00
epoll.c	io_uring: undeprecate epoll_ctl support	2023-05-26 20:22:41 -06:00
epoll.h
eventfd.c	io_uring/eventfd: ensure io_eventfd_signal() defers another RCU period	2025-01-17 13:40:58 +01:00
eventfd.h	io_uring/eventfd: move eventfd handling to separate file	2024-06-16 14:54:55 -06:00
fdinfo.c	io_uring/fdinfo: annotate racy sq/cq head/tail reads	2025-05-29 11:03:11 +02:00
fdinfo.h
filetable.c	io_uring/filetable: don't unnecessarily clear/reset bitmap	2024-05-08 08:27:45 -06:00
filetable.h	io_uring: expand main struct io_kiocb flags to 64-bits	2024-02-08 13:27:03 -07:00
fs.c	io_uring/fs: consider link->flags when getting path for LINKAT	2023-11-20 09:01:42 -07:00
fs.h
futex.c	futex: Pass in task to futex_queue()	2025-03-22 12:54:14 -07:00
futex.h	io_uring/alloc_cache: switch to array based caching	2024-04-15 08:10:25 -06:00
io_uring.c	io_uring: fix overflow resched cqe reordering	2025-05-29 11:03:17 +02:00
io_uring.h	io_uring/sqpoll: ensure task state is TASK_RUNNING when running task_work	2024-10-17 08:38:04 -06:00
io-wq.c	io-wq: backoff when retrying worker creation	2025-03-22 12:54:19 -07:00
io-wq.h	io_uring/io-wq: make io_wq_work flags atomic	2024-06-16 14:54:55 -06:00
kbuf.c	io_uring/kbuf: reject zero sized provided buffers	2025-04-20 10:15:40 +02:00
kbuf.h	io_uring/kbuf: add support for incremental buffer consumption	2024-08-29 08:44:58 -06:00
Makefile	io_uring: add GCOV_PROFILE_URING Kconfig option	2024-08-30 10:52:02 -06:00
memmap.c	io_uring: check for overflows in io_pin_pages	2024-12-05 14:02:50 +01:00
memmap.h	io_uring: move mapping/allocation helpers to a separate file	2024-04-15 08:10:26 -06:00
msg_ring.c	io_uring/msg: initialise msg request opcode	2025-05-29 11:02:03 +02:00
msg_ring.h	io_uring/msg_ring: add an alloc cache for io_kiocb entries	2024-06-24 08:39:55 -06:00
napi.c	io_uring: user registered clockid for wait timeouts	2024-08-25 08:27:01 -06:00
napi.h	io_uring/napi: postpone napi timeout adjustment	2024-08-25 08:27:01 -06:00
net.c	io_uring/net: fix io_req_post_cqe abuse by send bundle	2025-04-20 10:15:40 +02:00
net.h	io_uring: Introduce IORING_OP_LISTEN	2024-06-19 07:57:21 -06:00
nop.c	io_uring: support to inject result for NOP	2024-05-10 06:09:45 -06:00
nop.h
notif.c	io_uring/notif: disable LAZY_WAKE for linked notifs	2024-04-30 13:06:27 -06:00
notif.h	io_uring/notif: implement notification stacking	2024-04-22 19:31:18 -06:00
opdef.c	io_uring: Fix probe of disabled operations	2024-06-19 08:58:00 -06:00
opdef.h	io_uring: Fix probe of disabled operations	2024-06-19 08:58:00 -06:00
openclose.c	io_uring: enable audit and restrict cred override for IORING_OP_FIXED_FD_INSTALL	2024-01-23 15:25:14 -07:00
openclose.h	io_uring/openclose: add support for IORING_OP_FIXED_FD_INSTALL	2023-12-12 07:42:57 -07:00
poll.c	io_uring/net: don't retry connect operation on EPOLLERR	2025-02-17 10:05:38 +01:00
poll.h	io_uring/poll: shrink alloc cache size to 32	2024-04-15 08:10:25 -06:00
refs.h	io_uring: always do atomic put from iowq	2025-05-02 07:59:21 +02:00
register.c	io_uring: clean up a type in io_uring_register_get_file()	2024-09-16 12:04:10 -06:00
register.h	io_uring: clean up a type in io_uring_register_get_file()	2024-09-16 12:04:10 -06:00
rsrc.c	io_uring/rsrc: require cloned buffers to share accounting contexts	2025-02-01 18:39:40 +01:00
rsrc.h	io_uring/rsrc: change ubuf->ubuf_end to length tracking	2024-09-15 09:15:22 -06:00
rw.c	block: add a rq_list type	2025-04-25 10:48:06 +02:00
rw.h	io_uring/alloc_cache: switch to array based caching	2024-04-15 08:10:25 -06:00
slist.h	io_uring: silence variable ‘prev’ set but not used warning	2023-03-09 10:10:58 -07:00
splice.c	splice: return type ssize_t from all helpers	2023-12-12 16:19:59 +01:00
splice.h
sqpoll.c	io_uring/sqpoll: Increase task_work submission batch size	2025-05-18 08:24:57 +02:00
sqpoll.h	io_uring/sqpoll: statistics of the true utilization of sq threads	2024-03-01 06:28:19 -07:00
statx.c	vfs: retire user_path_at_empty and drop empty arg from getname_flags	2024-06-05 17:03:57 +02:00
statx.h
sync.c	io_uring: for requests that require async, force it	2023-01-29 15:18:26 -07:00
sync.h
tctx.c	io_uring/tctx: work around xa_store() allocation error issue	2024-12-14 20:04:10 +01:00
tctx.h	io_uring: simplify __io_uring_add_tctx_node	2022-10-07 12:25:30 -06:00
timeout.c	io_uring/timeout: fix multishot updates	2025-01-17 13:40:51 +01:00
timeout.h	io_uring: remove unused return from io_disarm_next	2022-09-21 13:15:01 -06:00
truncate.c	io_uring: add support for ftruncate	2024-02-09 09:04:39 -07:00
truncate.h	io_uring: add support for ftruncate	2024-02-09 09:04:39 -07:00
uring_cmd.c	io_uring/uring_cmd: remove dead req_has_async_data() check	2025-02-21 14:01:20 +01:00
uring_cmd.h	io_uring/alloc_cache: switch to array based caching	2024-04-15 08:10:25 -06:00
waitid.c	io_uring/waitid: don't abuse io_tw_state	2025-02-21 14:01:20 +01:00
waitid.h	io_uring: add IORING_OP_WAITID support	2023-09-21 12:04:45 -06:00
xattr.c	vfs: retire user_path_at_empty and drop empty arg from getname_flags	2024-06-05 17:03:57 +02:00
xattr.h