Go to file
Peter Zijlstra d03b481743 sched: Fix stop_one_cpu_nowait() vs hotplug
[ Upstream commit f0498d2a54 ]

Kuyo reported sporadic failures on a sched_setaffinity() vs CPU
hotplug stress-test -- notably affine_move_task() remains stuck in
wait_for_completion(), leading to a hung-task detector warning.

Specifically, it was reported that stop_one_cpu_nowait(.fn =
migration_cpu_stop) returns false -- this stopper is responsible for
the matching complete().

The race scenario is:

	CPU0					CPU1

					// doing _cpu_down()

  __set_cpus_allowed_ptr()
    task_rq_lock();
					takedown_cpu()
					  stop_machine_cpuslocked(take_cpu_down..)

					<PREEMPT: cpu_stopper_thread()
					  MULTI_STOP_PREPARE
					  ...
    __set_cpus_allowed_ptr_locked()
      affine_move_task()
        task_rq_unlock();

  <PREEMPT: cpu_stopper_thread()\>
    ack_state()
					  MULTI_STOP_RUN
					    take_cpu_down()
					      __cpu_disable();
					      stop_machine_park();
						stopper->enabled = false;
					 />
   />
	stop_one_cpu_nowait(.fn = migration_cpu_stop);
          if (stopper->enabled) // false!!!

That is, by doing stop_one_cpu_nowait() after dropping rq-lock, the
stopper thread gets a chance to preempt and allows the cpu-down for
the target CPU to complete.

OTOH, since stop_one_cpu_nowait() / cpu_stop_queue_work() needs to
issue a wakeup, it must not be ran under the scheduler locks.

Solve this apparent contradiction by keeping preemption disabled over
the unlock + queue_stopper combination:

	preempt_disable();
	task_rq_unlock(...);
	if (!stop_pending)
	  stop_one_cpu_nowait(...)
	preempt_enable();

This respects the lock ordering contraints while still avoiding the
above race. That is, if we find the CPU is online under rq-lock, the
targeted stop_one_cpu_nowait() must succeed.

Apply this pattern to all similar stop_one_cpu_nowait() invocations.

Fixes: 6d337eab04 ("sched: Fix migrate_disable() vs set_cpus_allowed_ptr()")
Reported-by: "Kuyo Chang (張建文)" <Kuyo.Chang@mediatek.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Tested-by: "Kuyo Chang (張建文)" <Kuyo.Chang@mediatek.com>
Link: https://lkml.kernel.org/r/20231010200442.GA16515@noisy.programming.kicks-ass.net
Signed-off-by: Sasha Levin <sashal@kernel.org>
2023-11-20 11:58:52 +01:00
arch iov_iter, x86: Be consistent about the __user tag on copy_mc_to_user() 2023-11-20 11:58:52 +01:00
block blk-throttle: check for overflow in calculate_bytes_allowed 2023-10-20 18:38:17 -06:00
certs certs: Reference revocation list for all keyrings 2023-08-17 20:12:41 +00:00
crypto KEYS: asymmetric: Fix sign/verify on pkcs1pad without a hash 2023-10-18 12:27:10 +08:00
Documentation dt-bindings: serial: rs485: Add rs485-rts-active-high 2023-11-08 11:56:24 +01:00
drivers hwmon: (nct6775) Fix incorrect variable reuse in fan_div calculation 2023-11-20 11:58:51 +01:00
fs eventfs: Use simple_recursive_removal() to clean up dentries 2023-11-08 11:56:21 +01:00
include sched/topology: Fix sched_numa_find_nth_cpu() in non-NUMA case 2023-11-20 11:58:51 +01:00
init workqueue: Changes for v6.6 2023-09-01 16:06:32 -07:00
io_uring assorted fixes all over the place 2023-10-27 16:44:58 -10:00
ipc Add x86 shadow stack support 2023-08-31 12:20:12 -07:00
kernel sched: Fix stop_one_cpu_nowait() vs hotplug 2023-11-20 11:58:52 +01:00
lib maple_tree: add GFP_KERNEL to allocations in mas_expected_entries() 2023-10-18 12:12:41 -07:00
LICENSES LICENSES: Add the copyleft-next-0.3.1 license 2022-11-08 15:44:01 +01:00
mm numa: Generalize numa_map_to_online_node() 2023-11-20 11:58:51 +01:00
net netfilter pull request 23-10-25 2023-10-25 16:02:06 -07:00
rust rust: docs: fix logo replacement 2023-10-19 16:40:00 +02:00
samples VFIO updates for v6.6-rc1 2023-08-30 20:36:01 -07:00
scripts kbuild: remove stale code for 'source' symlink in packaging scripts 2023-10-01 23:06:06 +09:00
security KEYS: trusted: Remove redundant static calls usage 2023-10-10 11:19:43 -07:00
sound ASoC: SOF: sof-pci-dev: Fix community key quirk detection 2023-11-08 11:56:24 +01:00
tools objtool: Propagate early errors 2023-11-20 11:58:52 +01:00
usr initramfs: Encode dependency on KBUILD_BUILD_TIMESTAMP 2023-06-06 17:54:49 +09:00
virt ARM: 2023-09-07 13:52:20 -07:00
.clang-format iommu: Add for_each_group_device() 2023-05-23 08:15:51 +02:00
.cocciconfig
.get_maintainer.ignore
.gitattributes .gitattributes: set diff driver for Rust source code files 2023-05-31 17:48:25 +02:00
.gitignore kbuild: rpm-pkg: rename binkernel.spec to kernel.spec 2023-07-25 00:59:33 +09:00
.mailmap 20 hotfixes. 12 are cc:stable and the remainder address post-6.5 issues 2023-10-24 09:52:16 -10:00
.rustfmt.toml
COPYING COPYING: state that all contributions really are covered by this file 2020-02-10 13:32:20 -08:00
CREDITS USB: Remove Wireless USB and UWB documentation 2023-08-09 14:17:32 +02:00
Kbuild Kbuild updates for v6.1 2022-10-10 12:00:45 -07:00
Kconfig
MAINTAINERS Char/Misc driver fixes for 6.6-final 2023-10-28 07:51:27 -10:00
Makefile Linux 6.6.1 2023-11-08 11:56:25 +01:00
README

Linux kernel

There are several guides for kernel developers and users. These guides can be rendered in a number of formats, like HTML and PDF. Please read Documentation/admin-guide/README.rst first.

In order to build the documentation, use make htmldocs or make pdfdocs. The formatted documentation can also be read online at:

https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory, several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the requirements for building and running the kernel, and information about the problems which may result by upgrading your kernel.