Commit 2fcfe595 authored by zhidao su's avatar zhidao su Committed by Tejun Heo
Browse files

sched_ext: Use WRITE_ONCE() for the write side of scx_enable helper pointer



scx_enable() uses double-checked locking to lazily initialize a static
kthread_worker pointer. The fast path reads helper locklessly:

    if (!READ_ONCE(helper)) {          // lockless read -- no helper_mutex

The write side initializes helper under helper_mutex, but previously
used a plain assignment:

        helper = kthread_run_worker(0, "scx_enable_helper");
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                 plain write -- KCSAN data race with READ_ONCE() above

Since READ_ONCE() on the fast path and the plain write on the
initialization path access the same variable without a common lock,
they constitute a data race. KCSAN requires that all sides of a
lock-free access use READ_ONCE()/WRITE_ONCE() consistently.

Use a temporary variable to stage the result of kthread_run_worker(),
and only WRITE_ONCE() into helper after confirming the pointer is
valid. This avoids a window where a concurrent caller on the fast path
could observe an ERR pointer via READ_ONCE(helper) before the error
check completes.

Fixes: b06ccbab ("sched_ext: Fix starvation of scx_enable() under fair-class saturation")
Signed-off-by: default avatarzhidao su <suzhidao@xiaomi.com>
Acked-by: default avatarAndrea Righi <arighi@nvidia.com>
Signed-off-by: default avatarTejun Heo <tj@kernel.org>
parent 57ccf5cc
Loading
Loading
Loading
Loading
+5 −4
Original line number Diff line number Diff line
@@ -5258,13 +5258,14 @@ static int scx_enable(struct sched_ext_ops *ops, struct bpf_link *link)
	if (!READ_ONCE(helper)) {
		mutex_lock(&helper_mutex);
		if (!helper) {
			helper = kthread_run_worker(0, "scx_enable_helper");
			if (IS_ERR_OR_NULL(helper)) {
				helper = NULL;
			struct kthread_worker *w =
				kthread_run_worker(0, "scx_enable_helper");
			if (IS_ERR_OR_NULL(w)) {
				mutex_unlock(&helper_mutex);
				return -ENOMEM;
			}
			sched_set_fifo(helper->task);
			sched_set_fifo(w->task);
			WRITE_ONCE(helper, w);
		}
		mutex_unlock(&helper_mutex);
	}