Commit 2f2ea770 authored by Tejun Heo's avatar Tejun Heo
Browse files

sched_ext: Use dsq->first_task instead of list_empty() in dispatch_enqueue() FIFO-tail 



dispatch_enqueue()'s FIFO-tail path used list_empty(&dsq->list) to decide
whether to set dsq->first_task on enqueue. dsq->list can contain parked BPF
iterator cursors (SCX_DSQ_LNODE_ITER_CURSOR), so list_empty() is not a
reliable "no real task" check. If the last real task is unlinked while a
cursor is parked, first_task becomes NULL; the next FIFO-tail enqueue then
sees list_empty() == false and skips the first_task update, leaving
scx_bpf_dsq_peek() returning NULL for a non-empty DSQ.

Test dsq->first_task directly, which already tracks only real tasks and is
maintained under dsq->lock.

Fixes: 44f5c8ec ("sched_ext: Add lockless peek operation for DSQs")
Cc: stable@vger.kernel.org # v6.19+
Reported-by: default avatarChris Mason <clm@meta.com>
Signed-off-by: default avatarTejun Heo <tj@kernel.org>
Reviewed-by: default avatarAndrea Righi <arighi@nvidia.com>
Cc: Ryan Newton <newton@meta.com>
parent cc2a387d

kernel/sched/ext.c

+6 −4
Original line number Diff line number Diff line
@@ -1495,11 +1495,13 @@ static void dispatch_enqueue(struct scx_sched *sch, struct rq *rq, 
			if (!(dsq->id & SCX_DSQ_FLAG_BUILTIN))

				rcu_assign_pointer(dsq->first_task, p);

		} else {

			bool was_empty;



			was_empty = list_empty(&dsq->list);

			/*

			 * dsq->list can contain parked BPF iterator cursors, so

			 * list_empty() here isn't a reliable proxy for "no real

			 * task in the DSQ". Test dsq->first_task directly.

			 */

			list_add_tail(&p->scx.dsq_list.node, &dsq->list);

			if (was_empty && !(dsq->id & SCX_DSQ_FLAG_BUILTIN))

			if (!dsq->first_task && !(dsq->id & SCX_DSQ_FLAG_BUILTIN))

				rcu_assign_pointer(dsq->first_task, p);

		}

	}