Commit 0d10fe75 authored by Uros Bizjak's avatar Uros Bizjak Committed by Joerg Roedel
Browse files

iommu/amd: Use try_cmpxchg64 in alloc_pte and free_clear_pte 



Use try_cmpxchg64 instead of cmpxchg64 (*ptr, old, new) != old in
alloc_pte and free_clear_pte.  cmpxchg returns success in ZF flag, so this
change saves a compare after cmpxchg (and related move instruction
in front of cmpxchg). Also, remove racy explicit assignment to pteval
when cmpxchg fails, this is what try_cmpxchg does implicitly from
*pte in an atomic way.

Signed-off-by: default avatarUros Bizjak <ubizjak@gmail.com>
Cc: Joerg Roedel <joro@8bytes.org>
Cc: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Cc: Will Deacon <will@kernel.org>
Link: https://lore.kernel.org/r/20220525145416.10816-1-ubizjak@gmail.com


Signed-off-by: default avatarJoerg Roedel <jroedel@suse.de>
parent a111daf0

drivers/iommu/amd/io_pgtable.c

+2 −4
Original line number Diff line number Diff line
@@ -258,7 +258,7 @@ static u64 *alloc_pte(struct protection_domain *domain, 
			__npte = PM_LEVEL_PDE(level, iommu_virt_to_phys(page));



			/* pte could have been changed somewhere. */

			if (cmpxchg64(pte, __pte, __npte) != __pte)

			if (!try_cmpxchg64(pte, &__pte, __npte))

				free_page((unsigned long)page);

			else if (IOMMU_PTE_PRESENT(__pte))

				*updated = true;
@@ -341,10 +341,8 @@ static void free_clear_pte(u64 *pte, u64 pteval, struct list_head *freelist) 
	u64 *pt;

	int mode;



	while (cmpxchg64(pte, pteval, 0) != pteval) {

	while (!try_cmpxchg64(pte, &pteval, 0))

		pr_warn("AMD-Vi: IOMMU pte changed since we read it\n");

		pteval = *pte;

	}



	if (!IOMMU_PTE_PRESENT(pteval))

		return;