Commit 9adfcef3 authored by zhidao su's avatar zhidao su Committed by Tejun Heo
Browse files

sched_ext: Use READ_ONCE() for the read side of dsq->nr update



scx_bpf_dsq_nr_queued() reads dsq->nr via READ_ONCE() without holding
any lock, making dsq->nr a lock-free concurrently accessed variable.
However, dsq_mod_nr(), the sole writer of dsq->nr, only uses
WRITE_ONCE() on the write side without the matching READ_ONCE() on the
read side:

    WRITE_ONCE(dsq->nr, dsq->nr + delta);
                        ^^^^^^^
                        plain read -- KCSAN data race

The KCSAN documentation requires that if one accessor uses READ_ONCE()
or WRITE_ONCE() on a variable to annotate lock-free access, all other
accesses must also use the appropriate accessor. A plain read on the
right-hand side of WRITE_ONCE() leaves the pair incomplete and will
trigger KCSAN warnings.

Fix by using READ_ONCE() for the read side of the update:

    WRITE_ONCE(dsq->nr, READ_ONCE(dsq->nr) + delta);

This is consistent with scx_bpf_dsq_nr_queued() and makes the
concurrent access annotation complete and KCSAN-clean.

Signed-off-by: default avatarzhidao su <suzhidao@xiaomi.com>
Signed-off-by: default avatarTejun Heo <tj@kernel.org>
parent 032e084f
Loading
Loading
Loading
Loading
+6 −2
Original line number Diff line number Diff line
@@ -976,8 +976,12 @@ static bool scx_dsq_priq_less(struct rb_node *node_a,

static void dsq_mod_nr(struct scx_dispatch_q *dsq, s32 delta)
{
	/* scx_bpf_dsq_nr_queued() reads ->nr without locking, use WRITE_ONCE() */
	WRITE_ONCE(dsq->nr, dsq->nr + delta);
	/*
	 * scx_bpf_dsq_nr_queued() reads ->nr without locking. Use READ_ONCE()
	 * on the read side and WRITE_ONCE() on the write side to properly
	 * annotate the concurrent lockless access and avoid KCSAN warnings.
	 */
	WRITE_ONCE(dsq->nr, READ_ONCE(dsq->nr) + delta);
}

static void refill_task_slice_dfl(struct scx_sched *sch, struct task_struct *p)