Commit 6ded46b5 authored Oct 07, 2024 by Oliver Upton Committed by Marc Zyngier Oct 08, 2024

KVM: arm64: nv: Keep reference on stage-2 MMU when scheduled out

If a vCPU is scheduling out and not in WFI emulation, it is highly
likely it will get scheduled again soon and reuse the MMU it had before.
Dropping the MMU at vcpu_put() can have some unfortunate consequences,
as the MMU could get reclaimed and used in a different context, forcing
another 'cold start' on an otherwise active MMU.

Avoid that altogether by keeping a reference on the MMU if the vCPU is
scheduling out, ensuring that another vCPU cannot reclaim it while the
current vCPU is away. Since there are more MMUs than vCPUs, this does
not affect the guarantee that an unused MMU is available at any time.

Furthermore, this makes the vcpu->arch.hw_mmu ~stable in preemptible
code, at least for where it matters in the stage-2 abort path. Yes, the
MMU can change across WFI emulation, but there isn't even a use case
where this would matter.

Signed-off-by: Oliver Upton <oliver.upton@linux.dev>
Link: https://lore.kernel.org/r/20241007233028.2236133-2-oliver.upton@linux.dev

Signed-off-by: Marc Zyngier <maz@kernel.org>

parent ae8f8b37

arch/arm64/kvm/nested.c

+18 −3

Original line number	Diff line number	Diff line
		@@ -663,6 +663,13 @@ void kvm_init_nested_s2_mmu(struct kvm_s2_mmu *mmu)

		void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu)
		{
		/*
		* The vCPU kept its reference on the MMU after the last put, keep
		* rolling with it.
		*/
		if (vcpu->arch.hw_mmu)
		return;

		if (is_hyp_ctxt(vcpu)) {
		vcpu->arch.hw_mmu = &vcpu->kvm->arch.mmu;
		} else {
		@@ -674,11 +681,19 @@ void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu)

		void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu)
		{
		if (kvm_is_nested_s2_mmu(vcpu->kvm, vcpu->arch.hw_mmu)) {
		/*
		* Keep a reference on the associated stage-2 MMU if the vCPU is
		* scheduling out and not in WFI emulation, suggesting it is likely to
		* reuse the MMU sometime soon.
		*/
		if (vcpu->scheduled_out && !vcpu_get_flag(vcpu, IN_WFI))
		return;

		if (kvm_is_nested_s2_mmu(vcpu->kvm, vcpu->arch.hw_mmu))
		atomic_dec(&vcpu->arch.hw_mmu->refcnt);

		vcpu->arch.hw_mmu = NULL;
		}
		}

		/*
		* Returns non-zero if permission fault is handled by injecting it to the next