Files
linux-cryptodev-2.6/include/linux
Alexei Starovoitov af92793e52 slab: Introduce kmalloc_nolock() and kfree_nolock().
kmalloc_nolock() relies on ability of local_trylock_t to detect
the situation when per-cpu kmem_cache is locked.

In !PREEMPT_RT local_(try)lock_irqsave(&s->cpu_slab->lock, flags)
disables IRQs and marks s->cpu_slab->lock as acquired.
local_lock_is_locked(&s->cpu_slab->lock) returns true when
slab is in the middle of manipulating per-cpu cache
of that specific kmem_cache.

kmalloc_nolock() can be called from any context and can re-enter
into ___slab_alloc():
  kmalloc() -> ___slab_alloc(cache_A) -> irqsave -> NMI -> bpf ->
    kmalloc_nolock() -> ___slab_alloc(cache_B)
or
  kmalloc() -> ___slab_alloc(cache_A) -> irqsave -> tracepoint/kprobe -> bpf ->
    kmalloc_nolock() -> ___slab_alloc(cache_B)

Hence the caller of ___slab_alloc() checks if &s->cpu_slab->lock
can be acquired without a deadlock before invoking the function.
If that specific per-cpu kmem_cache is busy the kmalloc_nolock()
retries in a different kmalloc bucket. The second attempt will
likely succeed, since this cpu locked different kmem_cache.

Similarly, in PREEMPT_RT local_lock_is_locked() returns true when
per-cpu rt_spin_lock is locked by current _task_. In this case
re-entrance into the same kmalloc bucket is unsafe, and
kmalloc_nolock() tries a different bucket that is most likely is
not locked by the current task. Though it may be locked by a
different task it's safe to rt_spin_lock() and sleep on it.

Similar to alloc_pages_nolock() the kmalloc_nolock() returns NULL
immediately if called from hard irq or NMI in PREEMPT_RT.

kfree_nolock() defers freeing to irq_work when local_lock_is_locked()
and (in_nmi() or in PREEMPT_RT).

SLUB_TINY config doesn't use local_lock_is_locked() and relies on
spin_trylock_irqsave(&n->list_lock) to allocate,
while kfree_nolock() always defers to irq_work.

Note, kfree_nolock() must be called _only_ for objects allocated
with kmalloc_nolock(). Debug checks (like kmemleak and kfence)
were skipped on allocation, hence obj = kmalloc(); kfree_nolock(obj);
will miss kmemleak/kfence book keeping and will cause false positives.
large_kmalloc is not supported by either kmalloc_nolock()
or kfree_nolock().

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Reviewed-by: Harry Yoo <harry.yoo@oracle.com>
Signed-off-by: Vlastimil Babka <vbabka@suse.cz>
2025-09-29 09:42:36 +02:00
..
2025-07-23 01:38:56 -04:00
2025-06-15 21:19:08 +05:30
2025-07-21 18:18:51 +01:00
2025-07-31 11:28:03 -04:00
2025-07-31 18:23:53 -07:00
2025-05-21 13:41:03 +02:00
2025-05-21 13:39:45 +02:00
2025-07-07 12:24:50 +02:00
2025-07-19 18:59:57 -07:00
2025-06-11 13:39:52 -04:00
2025-06-27 16:38:02 -07:00
2025-07-22 15:57:02 +00:00
2025-07-16 14:28:21 +02:00
2025-07-19 18:59:51 -07:00
2025-07-24 19:12:32 -07:00
2025-05-22 11:07:05 +02:00
2025-08-02 12:06:10 -07:00
2025-06-11 11:57:14 -07:00
2025-07-02 17:18:01 +01:00
2025-07-17 06:01:16 -06:00
2025-07-13 16:38:24 -07:00
2025-07-09 22:41:56 -07:00
2025-06-17 18:18:46 -07:00
2025-07-14 15:20:02 -07:00
2025-07-14 15:20:02 -07:00
2025-07-08 19:11:57 -04:00
2025-07-31 16:11:43 -05:00
2025-05-21 16:46:37 +02:00
2025-07-09 19:32:30 -07:00
2025-06-19 14:28:24 +02:00
2025-06-19 14:28:24 +02:00
2025-07-22 18:07:11 +02:00
2025-07-01 12:29:29 +02:00
2025-06-13 08:47:18 +02:00
2025-07-10 09:39:18 +02:00
2025-07-19 10:17:56 +05:30
2025-07-23 11:56:02 +02:00
2025-07-21 17:48:32 -07:00
2025-06-25 15:12:17 -07:00
2025-06-26 09:44:45 -07:00
2025-07-17 11:26:56 +02:00
2025-08-02 12:01:37 -07:00