Commit c0c6ccd9 authored by David Hildenbrand (Arm)'s avatar David Hildenbrand (Arm) Committed by Andrew Morton
Browse files

mm: fix __vm_normal_page() to handle missing support for pmd_special()/pud_special()

On x86 32-bit with THP enabled, zap_huge_pmd() is seen to generate a
"WARNING: mm/memory.c:735 at __vm_normal_page+0x6a/0x7d", from the
VM_WARN_ON_ONCE(is_zero_pfn(pfn) || is_huge_zero_pfn(pfn)); followed by
"BUG: Bad rss-counter state"s, then later "BUG: Bad page state"s when
reclaim gets to call shrink_huge_zero_folio_scan().

It's as if the _PAGE_SPECIAL bit never got set in the huge_zero pmd: and
indeed, whereas pte_special() and pte_mkspecial() are subject to a
dedicated CONFIG_ARCH_HAS_PTE_SPECIAL, pmd_special() and pmd_mkspecial()
are subject to CONFIG_ARCH_SUPPORTS_PMD_PFNMAP, which is never enabled on
any 32-bit architecture.

While the problem was exposed through commit d80a9cb1
("mm/huge_memory: add and use normal_or_softleaf_folio_pmd()"), it was an
oversight in commit af385388 ("mm/memory: factor out common code from
vm_normal_page_*()") and would result in other problems:
* huge zero folio accounted in smaps, pagemap (PAGE_IS_FILE) and
  numamaps as file-backed THP
* folio_walk_start() returning the folio even without FW_ZEROPAGE set.
  Callers seem to tolerate that, though.

... and triggering the VM_WARN_ON_ONE(), although never reported so far.

To fix it, teach vm_normal_page_pmd()/vm_normal_page_pud() to consider
whether pmd_special/pud_special is actually implemented.

Link: https://lore.kernel.org/20260430-pmd_special-v1-1-dbcbcfd72c20@kernel.org


Fixes: af385388 ("mm/memory: factor out common code from vm_normal_page_*()")
Signed-off-by: default avatarDavid Hildenbrand (Arm) <david@kernel.org>
Reported-by: default avatarHugh Dickins <hughd@google.com>
Closes: https://lore.kernel.org/r/74a75b59-2e13-3985-ee99-d5521f39df2a@google.com


Reported-by: default avatarBibo Mao <maobibo@loongson.cn>
Closes: https://lore.kernel.org/r/20260430041121.2839350-1-maobibo@loongson.cn


Debugged-by: default avatarHugh Dickins <hughd@google.com>
Reviewed-by: default avatarLance Yang <lance.yang@linux.dev>
Tested-by: default avatarBibo Mao <maobibo@loongson.cn>
Reviewed-by: default avatarBaolin Wang <baolin.wang@linux.alibaba.com>
Reviewed-by: default avatarOscar Salvador <osalvador@suse.de>
Reviewed-by: default avatarLorenzo Stoakes <ljs@kernel.org>
Cc: Liam R. Howlett <liam@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Vlastimil Babka <vbabka@kernel.org>
Cc: <stable@vger.kernel.org>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
parent 03a2cc17
Loading
Loading
Loading
Loading
+19 −3
Original line number Diff line number Diff line
@@ -612,6 +612,21 @@ static void print_bad_page_map(struct vm_area_struct *vma,
	dump_stack();
	add_taint(TAINT_BAD_PAGE, LOCKDEP_NOW_UNRELIABLE);
}

static inline bool pgtable_level_has_pxx_special(enum pgtable_level level)
{
	switch (level) {
	case PGTABLE_LEVEL_PTE:
		return IS_ENABLED(CONFIG_ARCH_HAS_PTE_SPECIAL);
	case PGTABLE_LEVEL_PMD:
		return IS_ENABLED(CONFIG_ARCH_SUPPORTS_PMD_PFNMAP);
	case PGTABLE_LEVEL_PUD:
		return IS_ENABLED(CONFIG_ARCH_SUPPORTS_PUD_PFNMAP);
	default:
		return false;
	}
}

#define print_bad_pte(vma, addr, pte, page) \
	print_bad_page_map(vma, addr, pte_val(pte), page, PGTABLE_LEVEL_PTE)

@@ -684,7 +699,7 @@ static inline struct page *__vm_normal_page(struct vm_area_struct *vma,
		unsigned long addr, unsigned long pfn, bool special,
		unsigned long long entry, enum pgtable_level level)
{
	if (IS_ENABLED(CONFIG_ARCH_HAS_PTE_SPECIAL)) {
	if (pgtable_level_has_pxx_special(level)) {
		if (unlikely(special)) {
#ifdef CONFIG_FIND_NORMAL_PAGE
			if (vma->vm_ops && vma->vm_ops->find_normal_page)
@@ -699,8 +714,9 @@ static inline struct page *__vm_normal_page(struct vm_area_struct *vma,
			return NULL;
		}
		/*
		 * With CONFIG_ARCH_HAS_PTE_SPECIAL, any special page table
		 * mappings (incl. shared zero folios) are marked accordingly.
		 * With working pte_special()/pmd_special()..., any special page
		 * table mappings (incl. shared zero folios) are marked
		 * accordingly.
		 */
	} else {
		if (unlikely(vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP))) {