Commit c6307674 authored by Uladzislau Rezki (Sony)'s avatar Uladzislau Rezki (Sony) Committed by Andrew Morton
Browse files

mm: kvmalloc: add non-blocking support for vmalloc 

Extend __kvmalloc_node_noprof() to handle non-blocking GFP flags
(GFP_NOWAIT and GFP_ATOMIC).  Previously such flags were rejected,
returning NULL.  With this change:

- kvmalloc() can fall back to vmalloc() if non-blocking contexts;
- for non-blocking allocations the VM_ALLOW_HUGE_VMAP option is
  disabled, since the huge mapping path still contains might_sleep();
- documentation update to reflect that GFP_NOWAIT and GFP_ATOMIC
  are now supported.

Link: https://lkml.kernel.org/r/20251007122035.56347-11-urezki@gmail.com


Signed-off-by: default avatarUladzislau Rezki (Sony) <urezki@gmail.com>
Acked-by: default avatarMichal Hocko <mhocko@suse.com>
Reviewed-by: default avatarBaoquan He <bhe@redhat.com>
Cc: Alexander Potapenko <glider@google.com>
Cc: Andrey Ryabinin <ryabinin.a.a@gmail.com>
Cc: Marco Elver <elver@google.com>
Cc: Michal Hocko <mhocko@kernel.org>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
parent 0667b209

mm/slub.c

+13 −6
Original line number Diff line number Diff line
@@ -7090,7 +7090,7 @@ static gfp_t kmalloc_gfp_adjust(gfp_t flags, size_t size) 
 * Uses kmalloc to get the memory but if the allocation fails then falls back

 * to the vmalloc allocator. Use kvfree for freeing the memory.

 *

 * GFP_NOWAIT and GFP_ATOMIC are not supported, neither is the __GFP_NORETRY modifier.

 * GFP_NOWAIT and GFP_ATOMIC are supported, the __GFP_NORETRY modifier is not.

 * __GFP_RETRY_MAYFAIL is supported, and it should be used only if kmalloc is

 * preferable to the vmalloc fallback, due to visible performance drawbacks.

 *
@@ -7099,6 +7099,7 @@ static gfp_t kmalloc_gfp_adjust(gfp_t flags, size_t size) 
void *__kvmalloc_node_noprof(DECL_BUCKET_PARAMS(size, b), unsigned long align,

			     gfp_t flags, int node)

{

	bool allow_block;

	void *ret;



	/*
@@ -7111,16 +7112,22 @@ void *__kvmalloc_node_noprof(DECL_BUCKET_PARAMS(size, b), unsigned long align, 
	if (ret || size <= PAGE_SIZE)

		return ret;



	/* non-sleeping allocations are not supported by vmalloc */

	if (!gfpflags_allow_blocking(flags))

		return NULL;



	/* Don't even allow crazy sizes */

	if (unlikely(size > INT_MAX)) {

		WARN_ON_ONCE(!(flags & __GFP_NOWARN));

		return NULL;

	}



	/*

	 * For non-blocking the VM_ALLOW_HUGE_VMAP is not used

	 * because the huge-mapping path in vmalloc contains at

	 * least one might_sleep() call.

	 *

	 * TODO: Revise huge-mapping path to support non-blocking

	 * flags.

	 */

	allow_block = gfpflags_allow_blocking(flags);



	/*

	 * kvmalloc() can always use VM_ALLOW_HUGE_VMAP,

	 * since the callers already cannot assume anything
@@ -7128,7 +7135,7 @@ void *__kvmalloc_node_noprof(DECL_BUCKET_PARAMS(size, b), unsigned long align, 
	 * protection games.

	 */

	return __vmalloc_node_range_noprof(size, align, VMALLOC_START, VMALLOC_END,

			flags, PAGE_KERNEL, VM_ALLOW_HUGE_VMAP,

			flags, PAGE_KERNEL, allow_block ? VM_ALLOW_HUGE_VMAP:0,

			node, __builtin_return_address(0));

}

EXPORT_SYMBOL(__kvmalloc_node_noprof);