Commit ea5e101f authored by Pankaj Raghav's avatar Pankaj Raghav Committed by Andrew Morton
Browse files

block: use largest_zero_folio in __blkdev_issue_zero_pages() 

Use largest_zero_folio() in __blkdev_issue_zero_pages().  On systems with
CONFIG_PERSISTENT_HUGE_ZERO_FOLIO enabled, we will end up sending larger
bvecs instead of multiple small ones.

Noticed a 4% increase in performance on a commercial NVMe SSD which does
not support OP_WRITE_ZEROES.  The device's MDTS was 128K.  The performance
gains might be bigger if the device supports bigger MDTS.

Link: https://lkml.kernel.org/r/20250811084113.647267-6-kernel@pankajraghav.com


Signed-off-by: default avatarPankaj Raghav <p.raghav@samsung.com>
Acked-by: default avatarLorenzo Stoakes <lorenzo.stoakes@oracle.com>
Acked-by: default avatarDavid Hildenbrand <david@redhat.com>
Reviewed-by: default avatarHannes Reinecke <hare@suse.de>
Cc: Kiryl Shutsemau <kirill@shutemov.name>
Cc: Baolin Wang <baolin.wang@linux.alibaba.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: "Darrick J. Wong" <djwong@kernel.org>
Cc: Dev Jain <dev.jain@arm.com>
Cc: Jens Axboe <axboe@kernel.dk>
Cc: Liam Howlett <liam.howlett@oracle.com>
Cc: Luis Chamberalin <mcgrof@kernel.org>
Cc: Mariano Pache <npache@redhat.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Michal Hocko <mhocko@suse.com>
Cc: Mike Rapoport <rppt@kernel.org>
Cc: "Ritesh Harjani (IBM)" <ritesh.list@gmail.com>
Cc: Ryan Roberts <ryan.roberts@arm.com>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Thomas Gleinxer <tglx@linutronix.de>
Cc: Vlastimil Babka <vbabka@suse.cz>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
parent 415a0fd6

block/blk-lib.c

+8 −7
Original line number Diff line number Diff line
@@ -196,6 +196,8 @@ static void __blkdev_issue_zero_pages(struct block_device *bdev, 
		sector_t sector, sector_t nr_sects, gfp_t gfp_mask,

		struct bio **biop, unsigned int flags)

{

	struct folio *zero_folio = largest_zero_folio();



	while (nr_sects) {

		unsigned int nr_vecs = __blkdev_sectors_to_bio_pages(nr_sects);

		struct bio *bio;
@@ -208,15 +210,14 @@ static void __blkdev_issue_zero_pages(struct block_device *bdev, 
			break;



		do {

			unsigned int len, added;

			unsigned int len;



			len = min_t(sector_t,

				PAGE_SIZE, nr_sects << SECTOR_SHIFT);

			added = bio_add_page(bio, ZERO_PAGE(0), len, 0);

			if (added < len)

			len = min_t(sector_t, folio_size(zero_folio),

				    nr_sects << SECTOR_SHIFT);

			if (!bio_add_folio(bio, zero_folio, len, 0))

				break;

			nr_sects -= added >> SECTOR_SHIFT;

			sector += added >> SECTOR_SHIFT;

			nr_sects -= len >> SECTOR_SHIFT;

			sector += len >> SECTOR_SHIFT;

		} while (nr_sects);



		*biop = bio_chain_and_submit(*biop, bio);