Commit 45012241 authored by Suren Baghdasaryan's avatar Suren Baghdasaryan Committed by Andrew Morton
Browse files

mm/slab: introduce SLAB_NO_OBJ_EXT to avoid obj_ext creation 

Slab extension objects can't be allocated before slab infrastructure is
initialized.  Some caches, like kmem_cache and kmem_cache_node, are
created before slab infrastructure is initialized.  Objects from these
caches can't have extension objects.  Introduce SLAB_NO_OBJ_EXT slab flag
to mark these caches and avoid creating extensions for objects allocated
from these slabs.

Link: https://lkml.kernel.org/r/20240321163705.3067592-9-surenb@google.com


Signed-off-by: default avatarSuren Baghdasaryan <surenb@google.com>
Reviewed-by: default avatarKees Cook <keescook@chromium.org>
Reviewed-by: default avatarPasha Tatashin <pasha.tatashin@soleen.com>
Reviewed-by: default avatarVlastimil Babka <vbabka@suse.cz>
Tested-by: default avatarKees Cook <keescook@chromium.org>
Cc: Alexander Viro <viro@zeniv.linux.org.uk>
Cc: Alex Gaynor <alex.gaynor@gmail.com>
Cc: Alice Ryhl <aliceryhl@google.com>
Cc: Andreas Hindborg <a.hindborg@samsung.com>
Cc: Benno Lossin <benno.lossin@proton.me>
Cc: "Björn Roy Baron" <bjorn3_gh@protonmail.com>
Cc: Boqun Feng <boqun.feng@gmail.com>
Cc: Christoph Lameter <cl@linux.com>
Cc: Dennis Zhou <dennis@kernel.org>
Cc: Gary Guo <gary@garyguo.net>
Cc: Kent Overstreet <kent.overstreet@linux.dev>
Cc: Miguel Ojeda <ojeda@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Tejun Heo <tj@kernel.org>
Cc: Wedson Almeida Filho <wedsonaf@gmail.com>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
parent 768c33be

include/linux/slab.h

+10 −0
Original line number Diff line number Diff line
@@ -56,6 +56,9 @@ enum _slab_flag_bits { 
#endif

	_SLAB_OBJECT_POISON,

	_SLAB_CMPXCHG_DOUBLE,

#ifdef CONFIG_SLAB_OBJ_EXT

	_SLAB_NO_OBJ_EXT,

#endif

	_SLAB_FLAGS_LAST_BIT

};
@@ -202,6 +205,13 @@ enum _slab_flag_bits { 
#endif

#define SLAB_TEMPORARY		SLAB_RECLAIM_ACCOUNT	/* Objects are short-lived */



/* Slab created using create_boot_cache */

#ifdef CONFIG_SLAB_OBJ_EXT

#define SLAB_NO_OBJ_EXT		__SLAB_FLAG_BIT(_SLAB_NO_OBJ_EXT)

#else

#define SLAB_NO_OBJ_EXT		__SLAB_FLAG_UNUSED

#endif



/*

 * ZERO_SIZE_PTR will be returned for zero sized kmalloc requests.

 *

mm/slub.c

+3 −2
Original line number Diff line number Diff line
@@ -5693,7 +5693,8 @@ void __init kmem_cache_init(void) 
		node_set(node, slab_nodes);



	create_boot_cache(kmem_cache_node, "kmem_cache_node",

		sizeof(struct kmem_cache_node), SLAB_HWCACHE_ALIGN, 0, 0);

			sizeof(struct kmem_cache_node),

			SLAB_HWCACHE_ALIGN | SLAB_NO_OBJ_EXT, 0, 0);



	hotplug_memory_notifier(slab_memory_callback, SLAB_CALLBACK_PRI);
@@ -5703,7 +5704,7 @@ void __init kmem_cache_init(void) 
	create_boot_cache(kmem_cache, "kmem_cache",

			offsetof(struct kmem_cache, node) +

				nr_node_ids * sizeof(struct kmem_cache_node *),

		       SLAB_HWCACHE_ALIGN, 0, 0);

			SLAB_HWCACHE_ALIGN | SLAB_NO_OBJ_EXT, 0, 0);



	kmem_cache = bootstrap(&boot_kmem_cache);

	kmem_cache_node = bootstrap(&boot_kmem_cache_node);