Commit 30dc2f70 authored Nov 28, 2025 by Kumar Kartikeya Dwivedi Committed by Alexei Starovoitov Nov 29, 2025

rqspinlock: Disable spinning for trylock fallback

The original trylock fallback was inherited from qspinlock, and then
reused for the reentrant NMIs while the slow path is active. However,
under contention, it is very unlikely for the trylock to succeed in
taking the lock. In addition, a trylock also has no fairness guarantees,
and thus is prone to starvation issues under extreme scenarios.

The original qspinlock had no choice in terms of returning an error the
caller; if the node count was breached, it had to fall back to trylock
to attempt to take the lock. In case of rqspinlock, we do have the
option of returning to the user. Thus, simply attempt the trylock once,
and instead of spinning, return an error in case the lock cannot be
taken.

This ends up significantly reducing the time spent in the trylock
fallback, since we no longer wait for the timeout duration trying to
aimlessly acquire the lock when there's a high-probability that under
contention, it won't be available to us anyway.

Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
Link: https://lore.kernel.org/r/20251128232802.1031906-5-memxor@gmail.com

Signed-off-by: Alexei Starovoitov <ast@kernel.org>

parent 81d5a6a4

kernel/bpf/rqspinlock.c

+8 −10

Original line number	Diff line number	Diff line
		@@ -450,20 +450,18 @@ int __lockfunc resilient_queued_spin_lock_slowpath(rqspinlock_t *lock, u32 val)
		* not be nested NMIs taking spinlocks. That may not be true in
		* some architectures even though the chance of needing more than
		* 4 nodes will still be extremely unlikely. When that happens,
		* we fall back to spinning on the lock directly without using
		* any MCS node. This is not the most elegant solution, but is
		* simple enough.
		* we fall back to attempting a trylock operation without using
		* any MCS node. Unlike qspinlock which cannot fail, we have the
		* option of failing the slow path, and under contention, such a
		* trylock spinning will likely be treated unfairly due to lack of
		* queueing, hence do not spin.
		*/
		if (unlikely(idx >= _Q_MAX_NODES \|\| (in_nmi() && idx > 0))) {
		lockevent_inc(lock_no_node);
		RES_RESET_TIMEOUT(ts, RES_DEF_TIMEOUT);
		while (!queued_spin_trylock(lock)) {
		if (RES_CHECK_TIMEOUT(ts, ret, ~0u)) {
		lockevent_inc(rqspinlock_lock_timeout);
		if (!queued_spin_trylock(lock)) {
		ret = -EDEADLK;
		goto err_release_node;
		}
		cpu_relax();
		}
		goto release;
		}