Commit b0377ee8 authored by Sergey Senozhatsky's avatar Sergey Senozhatsky Committed by Andrew Morton
Browse files

zram: do not slot_free() written-back slots 

slot_free() basically completely resets the slots by clearing all of
its flags and attributes.  While zram_writeback_complete() restores
some of flags back (those that are necessary for async read
decompression) we still lose a lot of slot's metadata.  For example,
slot's ac-time, or ZRAM_INCOMPRESSIBLE.

More importantly, restoring flags/attrs requires extra attention as
some of the flags are directly affecting zram device stats.  And the
original code did not pay that attention.  Namely ZRAM_HUGE slots
handling in zram_writeback_complete().  The call to slot_free() would
decrement ->huge_pages, however when zram_writeback_complete() restored
the slot's ZRAM_HUGE flag, it would not get reflected in an incremented
->huge_pages.  So when the slot would finally get freed, slot_free()
would decrement ->huge_pages again, leading to underflow.

Fix this by open-coding the required memory free and stats updates in
zram_writeback_complete(), rather than calling the destructive
slot_free().  Since we now preserve the ZRAM_HUGE flag on written-back
slots (for the deferred decompression path), we also update slot_free()
to skip decrementing ->huge_pages if ZRAM_WB is set.

Link: https://lkml.kernel.org/r/20260320023143.2372879-1-senozhatsky@chromium.org
Link: https://lkml.kernel.org/r/20260319034912.1894770-1-senozhatsky@chromium.org


Fixes: d38fab60 ("zram: introduce compressed data writeback")
Signed-off-by: default avatarSergey Senozhatsky <senozhatsky@chromium.org>
Acked-by: default avatarMinchan Kim <minchan@kernel.org>
Cc: Brian Geffon <bgeffon@google.com>
Cc: Richard Chang <richardycc@google.com>
Signed-off-by: default avatarAndrew Morton <akpm@linux-foundation.org>
parent 26f775a0

drivers/block/zram/zram_drv.c

+14 −25
Original line number Diff line number Diff line
@@ -917,9 +917,8 @@ static void zram_account_writeback_submit(struct zram *zram) 


static int zram_writeback_complete(struct zram *zram, struct zram_wb_req *req)

{

	u32 size, index = req->pps->index;

	int err, prio;

	bool huge;

	u32 index = req->pps->index;

	int err;



	err = blk_status_to_errno(req->bio.bi_status);

	if (err) {
@@ -946,28 +945,13 @@ static int zram_writeback_complete(struct zram *zram, struct zram_wb_req *req) 
		goto out;

	}



	if (zram->compressed_wb) {

		/*

		 * ZRAM_WB slots get freed, we need to preserve data required

		 * for read decompression.

		 */

		size = get_slot_size(zram, index);

		prio = get_slot_comp_priority(zram, index);

		huge = test_slot_flag(zram, index, ZRAM_HUGE);

	}



	slot_free(zram, index);

	set_slot_flag(zram, index, ZRAM_WB);

	clear_slot_flag(zram, index, ZRAM_IDLE);

	if (test_slot_flag(zram, index, ZRAM_HUGE))

		atomic64_dec(&zram->stats.huge_pages);

	atomic64_sub(get_slot_size(zram, index), &zram->stats.compr_data_size);

	zs_free(zram->mem_pool, get_slot_handle(zram, index));

	set_slot_handle(zram, index, req->blk_idx);



	if (zram->compressed_wb) {

		if (huge)

			set_slot_flag(zram, index, ZRAM_HUGE);

		set_slot_size(zram, index, size);

		set_slot_comp_priority(zram, index, prio);

	}



	atomic64_inc(&zram->stats.pages_stored);

	set_slot_flag(zram, index, ZRAM_WB);



out:

	slot_unlock(zram, index);
@@ -2010,8 +1994,13 @@ static void slot_free(struct zram *zram, u32 index) 
	set_slot_comp_priority(zram, index, 0);



	if (test_slot_flag(zram, index, ZRAM_HUGE)) {

		clear_slot_flag(zram, index, ZRAM_HUGE);

		/*

		 * Writeback completion decrements ->huge_pages but keeps

		 * ZRAM_HUGE flag for deferred decompression path.

		 */

		if (!test_slot_flag(zram, index, ZRAM_WB))

			atomic64_dec(&zram->stats.huge_pages);

		clear_slot_flag(zram, index, ZRAM_HUGE);

	}



	if (test_slot_flag(zram, index, ZRAM_WB)) {