Commit 6f0e6c15 authored by Frederic Weisbecker's avatar Frederic Weisbecker Committed by Paul E. McKenney
Browse files

context_tracking: Take IRQ eqs entrypoints over RCU



The RCU dynticks counter is going to be merged into the context tracking
subsystem. Prepare with moving the IRQ extended quiescent states
entrypoints to context tracking. For now those are dumb redirection to
existing RCU calls.

[ paulmck: Apply Stephen Rothwell feedback from -next. ]
[ paulmck: Apply Nathan Chancellor feedback. ]

Acked-by: default avatarPaul E. McKenney <paulmck@kernel.org>
Signed-off-by: default avatarFrederic Weisbecker <frederic@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
Cc: Uladzislau Rezki <uladzislau.rezki@sony.com>
Cc: Joel Fernandes <joel@joelfernandes.org>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Nicolas Saenz Julienne <nsaenz@kernel.org>
Cc: Marcelo Tosatti <mtosatti@redhat.com>
Cc: Xiongfeng Wang <wangxiongfeng2@huawei.com>
Cc: Yu Liao <liaoyu15@huawei.com>
Cc: Phil Auld <pauld@redhat.com>
Cc: Paul Gortmaker<paul.gortmaker@windriver.com>
Cc: Alex Belits <abelits@marvell.com>
Signed-off-by: default avatarPaul E. McKenney <paulmck@kernel.org>
Reviewed-by: default avatarNicolas Saenz Julienne <nsaenzju@redhat.com>
Tested-by: default avatarNicolas Saenz Julienne <nsaenzju@redhat.com>
parent e67198cc
Loading
Loading
Loading
Loading
+5 −5
Original line number Diff line number Diff line
@@ -1844,10 +1844,10 @@ that meets this requirement.

Furthermore, NMI handlers can be interrupted by what appear to RCU to be
normal interrupts. One way that this can happen is for code that
directly invokes rcu_irq_enter() and rcu_irq_exit() to be called
directly invokes ct_irq_enter() and ct_irq_exit() to be called
from an NMI handler. This astonishing fact of life prompted the current
code structure, which has rcu_irq_enter() invoking
rcu_nmi_enter() and rcu_irq_exit() invoking rcu_nmi_exit().
code structure, which has ct_irq_enter() invoking
rcu_nmi_enter() and ct_irq_exit() invoking rcu_nmi_exit().
And yes, I also learned of this requirement the hard way.

Loadable Modules
@@ -2195,7 +2195,7 @@ scheduling-clock interrupt be enabled when RCU needs it to be:
   sections, and RCU believes this CPU to be idle, no problem. This
   sort of thing is used by some architectures for light-weight
   exception handlers, which can then avoid the overhead of
   rcu_irq_enter() and rcu_irq_exit() at exception entry and
   ct_irq_enter() and ct_irq_exit() at exception entry and
   exit, respectively. Some go further and avoid the entireties of
   irq_enter() and irq_exit().
   Just make very sure you are running some of your tests with
@@ -2226,7 +2226,7 @@ scheduling-clock interrupt be enabled when RCU needs it to be:
+-----------------------------------------------------------------------+
| **Answer**:                                                           |
+-----------------------------------------------------------------------+
| One approach is to do ``rcu_irq_exit();rcu_irq_enter();`` every so    |
| One approach is to do ``ct_irq_exit();ct_irq_enter();`` every so      |
| often. But given that long-running interrupt handlers can cause other |
| problems, not least for response time, shouldn't you work to keep     |
| your interrupt handler's runtime within reasonable bounds?            |
+2 −2
Original line number Diff line number Diff line
@@ -98,11 +98,11 @@ warnings:

-	A low-level kernel issue that either fails to invoke one of the
	variants of rcu_user_enter(), rcu_user_exit(), ct_idle_enter(),
	ct_idle_exit(), rcu_irq_enter(), or rcu_irq_exit() on the one
	ct_idle_exit(), ct_irq_enter(), or ct_irq_exit() on the one
	hand, or that invokes one of them too many times on the other.
	Historically, the most frequent issue has been an omission
	of either irq_enter() or irq_exit(), which in turn invoke
	rcu_irq_enter() or rcu_irq_exit(), respectively.  Building your
	ct_irq_enter() or ct_irq_exit(), respectively.  Building your
	kernel with CONFIG_RCU_EQS_DEBUG=y can help track down these types
	of issues, which sometimes arise in architecture-specific code.

+1 −1
Original line number Diff line number Diff line
@@ -782,7 +782,7 @@ config HAVE_CONTEXT_TRACKING_USER
	  Syscalls need to be wrapped inside user_exit()-user_enter(), either
	  optimized behind static key or through the slow path using TIF_NOHZ
	  flag. Exceptions handlers must be wrapped as well. Irqs are already
	  protected inside rcu_irq_enter/rcu_irq_exit() but preemption or signal
	  protected inside ct_irq_enter/ct_irq_exit() but preemption or signal
	  handling on irq exit still need to be protected.

config HAVE_CONTEXT_TRACKING_USER_OFFSTACK
+3 −3
Original line number Diff line number Diff line
@@ -41,7 +41,7 @@ static __always_inline void __enter_from_kernel_mode(struct pt_regs *regs)

	if (!IS_ENABLED(CONFIG_TINY_RCU) && is_idle_task(current)) {
		lockdep_hardirqs_off(CALLER_ADDR0);
		rcu_irq_enter();
		ct_irq_enter();
		trace_hardirqs_off_finish();

		regs->exit_rcu = true;
@@ -76,7 +76,7 @@ static __always_inline void __exit_to_kernel_mode(struct pt_regs *regs)
		if (regs->exit_rcu) {
			trace_hardirqs_on_prepare();
			lockdep_hardirqs_on_prepare();
			rcu_irq_exit();
			ct_irq_exit();
			lockdep_hardirqs_on(CALLER_ADDR0);
			return;
		}
@@ -84,7 +84,7 @@ static __always_inline void __exit_to_kernel_mode(struct pt_regs *regs)
		trace_hardirqs_on();
	} else {
		if (regs->exit_rcu)
			rcu_irq_exit();
			ct_irq_exit();
	}
}

+1 −1
Original line number Diff line number Diff line
@@ -1526,7 +1526,7 @@ DEFINE_IDTENTRY_RAW_ERRORCODE(exc_page_fault)

	/*
	 * Entry handling for valid #PF from kernel mode is slightly
	 * different: RCU is already watching and rcu_irq_enter() must not
	 * different: RCU is already watching and ct_irq_enter() must not
	 * be invoked because a kernel fault on a user space address might
	 * sleep.
	 *
Loading