Commit daa01d95 authored by Caleb Sander Mateos's avatar Caleb Sander Mateos Committed by Jens Axboe
Browse files

io_uring/rsrc: skip atomic refcount for uncloned buffers 



io_buffer_unmap() performs an atomic decrement of the io_mapped_ubuf's
reference count in case it has been cloned into another io_ring_ctx's
registered buffer table. This is an expensive operation and unnecessary
in the common case that the io_mapped_ubuf is only registered once.
Load the reference count first and check whether it's 1. In that case,
skip the atomic decrement and immediately free the io_mapped_ubuf.

Signed-off-by: default avatarCaleb Sander Mateos <csander@purestorage.com>
Link: https://lore.kernel.org/r/20250619143435.3474028-1-csander@purestorage.com


Signed-off-by: default avatarJens Axboe <axboe@kernel.dk>
parent e448d578

io_uring/rsrc.c

+4 −2
Original line number Diff line number Diff line
@@ -135,8 +135,10 @@ static void io_free_imu(struct io_ring_ctx *ctx, struct io_mapped_ubuf *imu) 


static void io_buffer_unmap(struct io_ring_ctx *ctx, struct io_mapped_ubuf *imu)

{

	if (unlikely(refcount_read(&imu->refs) > 1)) {

		if (!refcount_dec_and_test(&imu->refs))

			return;

	}



	if (imu->acct_pages)

		io_unaccount_mem(ctx, imu->acct_pages);