Commit fc4b909c authored by Joshua Hahn's avatar Joshua Hahn Committed by Andrew Morton
Browse files

mm/page_alloc: batch page freeing in decay_pcp_high 

It is possible for pcp->count - pcp->high to exceed pcp->batch by a lot. 
When this happens, we should perform batching to ensure that
free_pcppages_bulk isn't called with too many pages to free at once and
starve out other threads that need the pcp or zone lock.

Since we are still only freeing the difference between the initial
pcp->count and pcp->high values, there should be no change to how many
pages are freed.

Link: https://lkml.kernel.org/r/20251014145011.3427205-3-joshua.hahnjy@gmail.com


Signed-off-by: default avatarJoshua Hahn <joshua.hahnjy@gmail.com>
Suggested-by: default avatarChris Mason <clm@fb.com>
Suggested-by: default avatarAndrew Morton <akpm@linux-foundation.org>
Co-developed-by: default avatarJohannes Weiner <hannes@cmpxchg.org>
Reviewed-by: default avatarVlastimil Babka <vbabka@suse.cz>
Cc: Brendan Jackman <jackmanb@google.com>
Cc: "Kirill A. Shutemov" <kirill@shutemov.name>
Cc: Michal Hocko <mhocko@suse.com>
Cc: SeongJae Park <sj@kernel.org>
Cc: Suren Baghdasaryan <surenb@google.com>
Cc: Zi Yan <ziy@nvidia.com>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
parent 0acc67c4

mm/page_alloc.c

+6 −3
Original line number Diff line number Diff line
@@ -2559,7 +2559,7 @@ static int rmqueue_bulk(struct zone *zone, unsigned int order, 
 */

bool decay_pcp_high(struct zone *zone, struct per_cpu_pages *pcp)

{

	int high_min, to_drain, batch;

	int high_min, to_drain, to_drain_batched, batch;

	bool todo = false;



	high_min = READ_ONCE(pcp->high_min);
@@ -2577,11 +2577,14 @@ bool decay_pcp_high(struct zone *zone, struct per_cpu_pages *pcp) 
	}



	to_drain = pcp->count - pcp->high;

	if (to_drain > 0) {

	while (to_drain > 0) {

		to_drain_batched = min(to_drain, batch);

		spin_lock(&pcp->lock);

		free_pcppages_bulk(zone, to_drain, pcp, 0);

		free_pcppages_bulk(zone, to_drain_batched, pcp, 0);

		spin_unlock(&pcp->lock);

		todo = true;



		to_drain -= to_drain_batched;

	}



	return todo;