Commit 8f870e6e authored by Paul E. McKenney's avatar Paul E. McKenney
Browse files

srcu: Block less aggressively for expedited grace periods 



Commit 282d8998 ("srcu: Prevent expedited GPs and blocking readers
from consuming CPU") fixed a problem where a long-running expedited SRCU
grace period could block kernel live patching.  It did so by giving up
on expediting once a given SRCU expedited grace period grew too old.

Unfortunately, this added excessive delays to boots of virtual embedded
systems specifying "-bios QEMU_EFI.fd" to qemu.  This commit therefore
makes the transition away from expediting less aggressive, increasing
the per-grace-period phase number of non-sleeping polls of readers from
one to three and increasing the required grace-period age from one jiffy
(actually from zero to one jiffies) to two jiffies (actually from one
to two jiffies).

Fixes: 282d8998 ("srcu: Prevent expedited GPs and blocking readers from consuming CPU")
Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
Reported-by: default avatarZhangfei Gao <zhangfei.gao@linaro.org>
Reported-by: default avatarchenxiang (M)" <chenxiang66@hisilicon.com>
Cc: Shameerali Kolothum Thodi  <shameerali.kolothum.thodi@huawei.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: default avatarNeeraj Upadhyay <quic_neeraju@quicinc.com>
Link: https://lore.kernel.org/all/20615615-0013-5adc-584f-2b1d5c03ebfc@linaro.org/
parent a111daf0

kernel/rcu/srcutree.c

+13 −7
Original line number Diff line number Diff line
@@ -513,7 +513,7 @@ static bool srcu_readers_active(struct srcu_struct *ssp) 


#define SRCU_INTERVAL		1	// Base delay if no expedited GPs pending.

#define SRCU_MAX_INTERVAL	10	// Maximum incremental delay from slow readers.

#define SRCU_MAX_NODELAY_PHASE	1	// Maximum per-GP-phase consecutive no-delay instances.

#define SRCU_MAX_NODELAY_PHASE	3	// Maximum per-GP-phase consecutive no-delay instances.

#define SRCU_MAX_NODELAY	100	// Maximum consecutive no-delay instances.



/*
@@ -522,17 +522,23 @@ static bool srcu_readers_active(struct srcu_struct *ssp) 
 */

static unsigned long srcu_get_delay(struct srcu_struct *ssp)

{

	unsigned long gpstart;

	unsigned long j;

	unsigned long jbase = SRCU_INTERVAL;



	if (ULONG_CMP_LT(READ_ONCE(ssp->srcu_gp_seq), READ_ONCE(ssp->srcu_gp_seq_needed_exp)))

		jbase = 0;

	if (rcu_seq_state(READ_ONCE(ssp->srcu_gp_seq)))

		jbase += jiffies - READ_ONCE(ssp->srcu_gp_start);

	if (rcu_seq_state(READ_ONCE(ssp->srcu_gp_seq))) {

		j = jiffies - 1;

		gpstart = READ_ONCE(ssp->srcu_gp_start);

		if (time_after(j, gpstart))

			jbase += j - gpstart;

		if (!jbase) {

			WRITE_ONCE(ssp->srcu_n_exp_nodelay, READ_ONCE(ssp->srcu_n_exp_nodelay) + 1);

			if (READ_ONCE(ssp->srcu_n_exp_nodelay) > SRCU_MAX_NODELAY_PHASE)

				jbase = 1;

		}

	}

	return jbase > SRCU_MAX_INTERVAL ? SRCU_MAX_INTERVAL : jbase;

}