Commit 14cebf68 authored by Will Deacon's avatar Will Deacon Committed by Christoph Hellwig
Browse files

swiotlb: Reinstate page-alignment for mappings >= PAGE_SIZE 



For swiotlb allocations >= PAGE_SIZE, the slab search historically
adjusted the stride to avoid checking unaligned slots. This had the
side-effect of aligning large mapping requests to PAGE_SIZE, but that
was broken by 0eee5ae1 ("swiotlb: fix slot alignment checks").

Since this alignment could be relied upon drivers, reinstate PAGE_SIZE
alignment for swiotlb mappings >= PAGE_SIZE.

Reported-by: default avatarMichael Kelley <mhklinux@outlook.com>
Signed-off-by: default avatarWill Deacon <will@kernel.org>
Reviewed-by: default avatarRobin Murphy <robin.murphy@arm.com>
Reviewed-by: default avatarPetr Tesarik <petr.tesarik1@huawei-partners.com>
Tested-by: default avatarNicolin Chen <nicolinc@nvidia.com>
Tested-by: default avatarMichael Kelley <mhklinux@outlook.com>
Signed-off-by: default avatarChristoph Hellwig <hch@lst.de>
parent afc5aa46

kernel/dma/swiotlb.c

+11 −7
Original line number Diff line number Diff line
@@ -1014,6 +1014,17 @@ static int swiotlb_search_pool_area(struct device *dev, struct io_tlb_pool *pool 
	BUG_ON(!nslots);

	BUG_ON(area_index >= pool->nareas);



	/*

	 * Historically, swiotlb allocations >= PAGE_SIZE were guaranteed to be

	 * page-aligned in the absence of any other alignment requirements.

	 * 'alloc_align_mask' was later introduced to specify the alignment

	 * explicitly, however this is passed as zero for streaming mappings

	 * and so we preserve the old behaviour there in case any drivers are

	 * relying on it.

	 */

	if (!alloc_align_mask && !iotlb_align_mask && alloc_size >= PAGE_SIZE)

		alloc_align_mask = PAGE_SIZE - 1;



	/*

	 * Ensure that the allocation is at least slot-aligned and update

	 * 'iotlb_align_mask' to ignore bits that will be preserved when
@@ -1028,13 +1039,6 @@ static int swiotlb_search_pool_area(struct device *dev, struct io_tlb_pool *pool 
	 */

	stride = get_max_slots(max(alloc_align_mask, iotlb_align_mask));



	/*

	 * For allocations of PAGE_SIZE or larger only look for page aligned

	 * allocations.

	 */

	if (alloc_size >= PAGE_SIZE)

		stride = umax(stride, PAGE_SHIFT - IO_TLB_SHIFT + 1);



	spin_lock_irqsave(&area->lock, flags);

	if (unlikely(nslots > pool->area_nslabs - area->used))

		goto not_found;