Merge tag 'linux_kselftest-kunit-6.18-rc1' of...

Parameterized Testing

~~~~~~~~~~~~~~~~~~~~~

The table-driven testing pattern is common enough that KUnit has special

support for it.

To run a test case against multiple inputs, KUnit provides a parameterized

testing framework. This feature formalizes and extends the concept of

table-driven tests discussed previously.

By reusing the same ``cases`` array from above, we can write the test as a

"parameterized test" with the following.

A KUnit test is determined to be parameterized if a parameter generator function

is provided when registering the test case. A test user can either write their

own generator function or use one that is provided by KUnit. The generator

function is stored in  ``kunit_case->generate_params`` and can be set using the

macros described in the section below.

To establish the terminology, a "parameterized test" is a test which is run

multiple times (once per "parameter" or "parameter run"). Each parameter run has

both its own independent ``struct kunit`` (the "parameter run context") and

access to a shared parent ``struct kunit`` (the "parameterized test context").

Passing Parameters to a Test

^^^^^^^^^^^^^^^^^^^^^^^^^^^^

There are three ways to provide the parameters to a test:

Array Parameter Macros:

   KUnit provides special support for the common table-driven testing pattern.

   By applying either ``KUNIT_ARRAY_PARAM`` or ``KUNIT_ARRAY_PARAM_DESC`` to the

   ``cases`` array from the previous section, we can create a parameterized test

   as shown below:

.. code-block:: c

		const char *str;

		const char *sha1;

	};

	const struct sha1_test_case cases[] = {

	static const struct sha1_test_case cases[] = {

		{

			.str = "hello world",

			.sha1 = "2aae6c35c94fcfb415dbe95f408b9ce91ee846ed",

		{}

	};

Custom Parameter Generator Function:

   The generator function is responsible for generating parameters one-by-one

   and has the following signature:

   ``const void* (*)(struct kunit *test, const void *prev, char *desc)``.

   You can pass the generator function to the ``KUNIT_CASE_PARAM``

   or ``KUNIT_CASE_PARAM_WITH_INIT`` macros.

   The function receives the previously generated parameter as the ``prev`` argument

   (which is ``NULL`` on the first call) and can also access the parameterized

   test context passed as the ``test`` argument. KUnit calls this function

   repeatedly until it returns ``NULL``, which signifies that a parameterized

   test ended.

   Below is an example of how it works:

.. code-block:: c

	#define MAX_TEST_BUFFER_SIZE 8

	// Example generator function. It produces a sequence of buffer sizes that

	// are powers of two, starting at 1 (e.g., 1, 2, 4, 8).

	static const void *buffer_size_gen_params(struct kunit *test, const void *prev, char *desc)

	{

		long prev_buffer_size = (long)prev;

		long next_buffer_size = 1; // Start with an initial size of 1.

		// Stop generating parameters if the limit is reached or exceeded.

		if (prev_buffer_size >= MAX_TEST_BUFFER_SIZE)

			return NULL;

		// For subsequent calls, calculate the next size by doubling the previous one.

		if (prev)

			next_buffer_size = prev_buffer_size << 1;

		return (void *)next_buffer_size;

	}

	// Simple test to validate that kunit_kzalloc provides zeroed memory.

	static void buffer_zero_test(struct kunit *test)

	{

		long buffer_size = (long)test->param_value;

		// Use kunit_kzalloc to allocate a zero-initialized buffer. This makes the

		// memory "parameter run managed," meaning it's automatically cleaned up at

		// the end of each parameter run.

		int *buf = kunit_kzalloc(test, buffer_size * sizeof(int), GFP_KERNEL);

		// Ensure the allocation was successful.

		KUNIT_ASSERT_NOT_NULL(test, buf);

		// Loop through the buffer and confirm every element is zero.

		for (int i = 0; i < buffer_size; i++)

			KUNIT_EXPECT_EQ(test, buf[i], 0);

	}

	static struct kunit_case buffer_test_cases[] = {

		KUNIT_CASE_PARAM(buffer_zero_test, buffer_size_gen_params),

		{}

	};

Runtime Parameter Array Registration in the Init Function:

   For scenarios where you might need to initialize a parameterized test, you

   can directly register a parameter array to the parameterized test context.

   To do this, you must pass the parameterized test context, the array itself,

   the array size, and a ``get_description()`` function to the

   ``kunit_register_params_array()`` macro. This macro populates

   ``struct kunit_params`` within the parameterized test context, effectively

   storing a parameter array object. The ``get_description()`` function will

   be used for populating parameter descriptions and has the following signature:

   ``void (*)(struct kunit *test, const void *param, char *desc)``. Note that it

   also has access to the parameterized test context.

      .. important::

         When using this way to register a parameter array, you will need to

         manually pass ``kunit_array_gen_params()`` as the generator function to

         ``KUNIT_CASE_PARAM_WITH_INIT``. ``kunit_array_gen_params()`` is a KUnit

         helper that will use the registered array to generate the parameters.

	 If needed, instead of passing the KUnit helper, you can also pass your

	 own custom generator function that utilizes the parameter array. To

	 access the parameter array from within the parameter generator

	 function use ``test->params_array.params``.

   The ``kunit_register_params_array()`` macro should be called within a

   ``param_init()`` function that initializes the parameterized test and has

   the following signature ``int (*)(struct kunit *test)``. For a detailed

   explanation of this mechanism please refer to the "Adding Shared Resources"

   section that is after this one. This method supports registering both

   dynamically built and static parameter arrays.

   The code snippet below shows the ``example_param_init_dynamic_arr`` test that

   utilizes ``make_fibonacci_params()`` to create a dynamic array, which is then

   registered using ``kunit_register_params_array()``. To see the full code

   please refer to lib/kunit/kunit-example-test.c.

.. code-block:: c

	/*

	* Example of a parameterized test param_init() function that registers a dynamic

	* array of parameters.

	*/

	static int example_param_init_dynamic_arr(struct kunit *test)

	{

		size_t seq_size;

		int *fibonacci_params;

		kunit_info(test, "initializing parameterized test\n");

		seq_size = 6;

		fibonacci_params = make_fibonacci_params(test, seq_size);

		if (!fibonacci_params)

			return -ENOMEM;

		/*

		* Passes the dynamic parameter array information to the parameterized test

		* context struct kunit. The array and its metadata will be stored in

		* test->parent->params_array. The array itself will be located in

		* params_data.params.

		*/

		kunit_register_params_array(test, fibonacci_params, seq_size,

					example_param_dynamic_arr_get_desc);

		return 0;

	}

	static struct kunit_case example_test_cases[] = {

		/*

		 * Note how we pass kunit_array_gen_params() to use the array we

		 * registered in example_param_init_dynamic_arr() to generate

		 * parameters.

		 */

		KUNIT_CASE_PARAM_WITH_INIT(example_params_test_with_init_dynamic_arr,

					   kunit_array_gen_params,

					   example_param_init_dynamic_arr,

					   example_param_exit_dynamic_arr),

		{}

	};

Adding Shared Resources

^^^^^^^^^^^^^^^^^^^^^^^

All parameter runs in this framework hold a reference to the parameterized test

context, which can be accessed using the parent ``struct kunit`` pointer. The

parameterized test context is not used to execute any test logic itself; instead,

it serves as a container for shared resources.

It's possible to add resources to share between parameter runs within a

parameterized test by using ``KUNIT_CASE_PARAM_WITH_INIT``, to which you pass

custom ``param_init()`` and ``param_exit()`` functions. These functions run once

before and once after the parameterized test, respectively.

The ``param_init()`` function, with the signature ``int (*)(struct kunit *test)``,

can be used for adding resources to the ``resources`` or ``priv`` fields of

the parameterized test context, registering the parameter array, and any other

initialization logic.

The ``param_exit()`` function, with the signature ``void (*)(struct kunit *test)``,

can be used to release any resources that were not parameterized test managed (i.e.

not automatically cleaned up after the parameterized test ends) and for any other

exit logic.

Both ``param_init()`` and ``param_exit()`` are passed the parameterized test

context behind the scenes. However, the test case function receives the parameter

run context. Therefore, to manage and access shared resources from within a test

case function, you must use ``test->parent``.

For instance, finding a shared resource allocated by the Resource API requires

passing ``test->parent`` to ``kunit_find_resource()``. This principle extends to

all other APIs that might be used in the test case function, including

``kunit_kzalloc()``, ``kunit_kmalloc_array()``, and others (see

Documentation/dev-tools/kunit/api/test.rst and the

Documentation/dev-tools/kunit/api/resource.rst).

.. note::

   The ``suite->init()`` function, which executes before each parameter run,

   receives the parameter run context. Therefore, any resources set up in

   ``suite->init()`` are cleaned up after each parameter run.

The code below shows how you can add the shared resources. Note that this code

utilizes the Resource API, which you can read more about here:

Documentation/dev-tools/kunit/api/resource.rst. To see the full version of this

code please refer to lib/kunit/kunit-example-test.c.

.. code-block:: c

	static int example_resource_init(struct kunit_resource *res, void *context)

	{

		... /* Code that allocates memory and stores context in res->data. */

	}

	/* This function deallocates memory for the kunit_resource->data field. */

	static void example_resource_free(struct kunit_resource *res)

	{

		kfree(res->data);

	}

	/* This match function locates a test resource based on defined criteria. */

	static bool example_resource_alloc_match(struct kunit *test, struct kunit_resource *res,

						 void *match_data)

	{

		return res->data && res->free == example_resource_free;

	}

	/* Function to initialize the parameterized test. */

	static int example_param_init(struct kunit *test)

	{

		int ctx = 3; /* Data to be stored. */

		void *data = kunit_alloc_resource(test, example_resource_init,

						  example_resource_free,

						  GFP_KERNEL, &ctx);

		if (!data)

			return -ENOMEM;

		kunit_register_params_array(test, example_params_array,

					    ARRAY_SIZE(example_params_array));

		return 0;

	}

	/* Example test that uses shared resources in test->resources. */

	static void example_params_test_with_init(struct kunit *test)

	{

		int threshold;

		const struct example_param *param = test->param_value;

		/*  Here we pass test->parent to access the parameterized test context. */

		struct kunit_resource *res = kunit_find_resource(test->parent,

								 example_resource_alloc_match,

								 NULL);

		threshold = *((int *)res->data);

		KUNIT_ASSERT_LE(test, param->value, threshold);

		kunit_put_resource(res);

	}

	static struct kunit_case example_test_cases[] = {

		KUNIT_CASE_PARAM_WITH_INIT(example_params_test_with_init, kunit_array_gen_params,

					   example_param_init, NULL),

		{}

	};

As an alternative to using the KUnit Resource API for sharing resources, you can

place them in ``test->parent->priv``. This serves as a more lightweight method

for resource storage, best for scenarios where complex resource management is

not required.

As stated previously ``param_init()`` and ``param_exit()`` get the parameterized

test context. So, you can directly use ``test->priv`` within ``param_init/exit``

to manage shared resources. However, from within the test case function, you must

navigate up to the parent ``struct kunit`` i.e. the parameterized test context.

Therefore, you need to use ``test->parent->priv`` to access those same

resources.

The resources placed in ``test->parent->priv`` will need to be allocated in

memory to persist across the parameter runs. If memory is allocated using the

KUnit memory allocation APIs (described more in the "Allocating Memory" section

below), you won't need to worry about deallocation. The APIs will make the memory

parameterized test 'managed', ensuring that it will automatically get cleaned up

after the parameterized test concludes.

The code below demonstrates example usage of the ``priv`` field for shared

resources:

.. code-block:: c

	static const struct example_param {

		int value;

	} example_params_array[] = {

		{ .value = 3, },

		{ .value = 2, },

		{ .value = 1, },

		{ .value = 0, },

	};

	/* Initialize the parameterized test context. */

	static int example_param_init_priv(struct kunit *test)

	{

		int ctx = 3; /* Data to be stored. */

		int arr_size = ARRAY_SIZE(example_params_array);

		/*

		 * Allocate memory using kunit_kzalloc(). Since the `param_init`

		 * function receives the parameterized test context, this memory

		 * allocation will be scoped to the lifetime of the parameterized test.

		 */

		test->priv = kunit_kzalloc(test, sizeof(int), GFP_KERNEL);

		/* Assign the context value to test->priv.*/

		*((int *)test->priv) = ctx;

		/* Register the parameter array. */

		kunit_register_params_array(test, example_params_array, arr_size, NULL);

		return 0;

	}

	static void example_params_test_with_init_priv(struct kunit *test)

	{

		int threshold;

		const struct example_param *param = test->param_value;

		/* By design, test->parent will not be NULL. */

		KUNIT_ASSERT_NOT_NULL(test, test->parent);

		/* Here we use test->parent->priv to access the shared resource. */

		threshold = *(int *)test->parent->priv;

		KUNIT_ASSERT_LE(test, param->value, threshold);

	}

	static struct kunit_case example_tests[] = {

		KUNIT_CASE_PARAM_WITH_INIT(example_params_test_with_init_priv,

					   kunit_array_gen_params,

					   example_param_init_priv, NULL),

		{}

	};

Allocating Memory

-----------------

 *

 * Return: pointer to the next parameter or NULL if no more parameters

 */

const void *xe_pci_graphics_ip_gen_param(const void *prev, char *desc)

const void *xe_pci_graphics_ip_gen_param(struct kunit *test, const void *prev, char *desc)

{

	return graphics_ip_gen_params(prev, desc);

	return graphics_ip_gen_params(test, prev, desc);

}

EXPORT_SYMBOL_IF_KUNIT(xe_pci_graphics_ip_gen_param);

 *

 * Return: pointer to the next parameter or NULL if no more parameters

 */

const void *xe_pci_media_ip_gen_param(const void *prev, char *desc)

const void *xe_pci_media_ip_gen_param(struct kunit *test, const void *prev, char *desc)

{

	return media_ip_gen_params(prev, desc);

	return media_ip_gen_params(test, prev, desc);

}

EXPORT_SYMBOL_IF_KUNIT(xe_pci_media_ip_gen_param);

 *

 * Return: pointer to the next parameter or NULL if no more parameters

 */

const void *xe_pci_id_gen_param(const void *prev, char *desc)

const void *xe_pci_id_gen_param(struct kunit *test, const void *prev, char *desc)

{

	const struct pci_device_id *pci = pci_id_gen_params(prev, desc);

	const struct pci_device_id *pci = pci_id_gen_params(test, prev, desc);

	return pci->driver_data ? pci : NULL;

}

 * Return: pointer to the next &struct xe_device ready to be used as a parameter

 *         or NULL if there are no more Xe devices on the system.

 */

const void *xe_pci_live_device_gen_param(const void *prev, char *desc)

const void *xe_pci_live_device_gen_param(struct kunit *test, const void *prev, char *desc)

{

	const struct xe_device *xe = prev;

	struct device *dev = xe ? xe->drm.dev : NULL;

#define _XE_PCI_TEST_H_

#include <linux/types.h>

#include <kunit/test.h>

#include "xe_platform_types.h"

#include "xe_sriov_types.h"

int xe_pci_fake_device_init(struct xe_device *xe);

const void *xe_pci_graphics_ip_gen_param(const void *prev, char *desc);

const void *xe_pci_media_ip_gen_param(const void *prev, char *desc);

const void *xe_pci_id_gen_param(const void *prev, char *desc);

const void *xe_pci_live_device_gen_param(const void *prev, char *desc);

const void *xe_pci_graphics_ip_gen_param(struct kunit *test, const void *prev, char *desc);

const void *xe_pci_media_ip_gen_param(struct kunit *test, const void *prev, char *desc);

const void *xe_pci_id_gen_param(struct kunit *test, const void *prev, char *desc);

const void *xe_pci_live_device_gen_param(struct kunit *test, const void *prev, char *desc);

#endif

 * @name:     the name of the test case.

 * @generate_params: the generator function for parameterized tests.

 * @attr:     the attributes associated with the test

 * @param_init: The init function to run before a parameterized test.

 * @param_exit: The exit function to run after a parameterized test.

 *

 * A test case is a function with the signature,

 * ``void (*)(struct kunit *)``

struct kunit_case {

	void (*run_case)(struct kunit *test);

	const char *name;

	const void* (*generate_params)(const void *prev, char *desc);

	const void* (*generate_params)(struct kunit *test,

				       const void *prev, char *desc);

	struct kunit_attributes attr;

	int (*param_init)(struct kunit *test);

	void (*param_exit)(struct kunit *test);

	/* private: internal use only. */

	enum kunit_status status;

		  .generate_params = gen_params,				\

		  .attr = attributes, .module_name = KBUILD_MODNAME}

/**

 * KUNIT_CASE_PARAM_WITH_INIT - Define a parameterized KUnit test case with custom

 * param_init() and param_exit() functions.

 * @test_name: The function implementing the test case.

 * @gen_params: The function to generate parameters for the test case.

 * @init: A reference to the param_init() function to run before a parameterized test.

 * @exit: A reference to the param_exit() function to run after a parameterized test.

 *

 * Provides the option to register param_init() and param_exit() functions.

 * param_init/exit will be passed the parameterized test context and run once

 * before and once after the parameterized test. The init function can be used

 * to add resources to share between parameter runs, pass parameter arrays,

 * and any other setup logic. The exit function can be used to clean up resources

 * that were not managed by the parameterized test, and any other teardown logic.

 *

 * Note: If you are registering a parameter array in param_init() with

 * kunit_register_param_array() then you need to pass kunit_array_gen_params()

 * to this as the generator function.

 */

#define KUNIT_CASE_PARAM_WITH_INIT(test_name, gen_params, init, exit)		\

		{ .run_case = test_name, .name = #test_name,			\

		  .generate_params = gen_params,				\

		  .param_init = init, .param_exit = exit,			\

		  .module_name = KBUILD_MODNAME}

/**

 * struct kunit_suite - describes a related collection of &struct kunit_case

 *

	struct kunit_suite * const *end;

};

/* Stores the pointer to the parameter array and its metadata. */

struct kunit_params {

	/*

	 * Reference to the parameter array for a parameterized test. This

	 * is NULL if a parameter array wasn't directly passed to the

	 * parameterized test context struct kunit via kunit_register_params_array().

	 */

	const void *params;

	/* Reference to a function that gets the description of a parameter. */

	void (*get_description)(struct kunit *test, const void *param, char *desc);

	size_t num_params;

	size_t elem_size;

};

/**

 * struct kunit - represents a running instance of a test.

 *

 * @priv: for user to store arbitrary data. Commonly used to pass data

 *	  created in the init function (see &struct kunit_suite).

 * @parent: reference to the parent context of type struct kunit that can

 *	    be used for storing shared resources.

 * @params_array: for storing the parameter array.

 *

 * Used to store information about the current context under which the test

 * is running. Most of this data is private and should only be accessed

 * indirectly via public functions; the one exception is @priv which can be

 * used by the test writer to store arbitrary data.

 * indirectly via public functions; the exceptions are @priv, @parent and

 * @params_array which can be used by the test writer to store arbitrary data,

 * access the parent context, and to store the parameter array, respectively.

 */

struct kunit {

	void *priv;

	struct kunit *parent;

	struct kunit_params params_array;

	/* private: internal use only. */

	const char *name; /* Read only after initialization! */

struct kunit_suite_set kunit_merge_suite_sets(struct kunit_suite_set init_suite_set,

		struct kunit_suite_set suite_set);

const void *kunit_array_gen_params(struct kunit *test, const void *prev, char *desc);

#if IS_BUILTIN(CONFIG_KUNIT)

int kunit_run_all_tests(void);

#else

 * Define function @name_gen_params which uses @array to generate parameters.

 */

#define KUNIT_ARRAY_PARAM(name, array, get_desc)						\

	static const void *name##_gen_params(const void *prev, char *desc)			\

	static const void *name##_gen_params(struct kunit *test,				\

					     const void *prev, char *desc)			\

	{											\

		typeof((array)[0]) *__next = prev ? ((typeof(__next)) prev) + 1 : (array);	\

		if (!prev)									\

			kunit_register_params_array(test, array, ARRAY_SIZE(array), NULL);	\

		if (__next - (array) < ARRAY_SIZE((array))) {					\

			void (*__get_desc)(typeof(__next), char *) = get_desc;			\

			if (__get_desc)								\

 * Define function @name_gen_params which uses @array to generate parameters.

 */

#define KUNIT_ARRAY_PARAM_DESC(name, array, desc_member)					\

	static const void *name##_gen_params(const void *prev, char *desc)			\

	static const void *name##_gen_params(struct kunit *test,				\

					     const void *prev, char *desc)			\

	{											\

		typeof((array)[0]) *__next = prev ? ((typeof(__next)) prev) + 1 : (array);	\

		if (!prev)									\

			kunit_register_params_array(test, array, ARRAY_SIZE(array), NULL);	\

		if (__next - (array) < ARRAY_SIZE((array))) {					\

			strscpy(desc, __next->desc_member, KUNIT_PARAM_DESC_SIZE);		\

			return __next;								\

		return NULL;									\

	}

/**

 * kunit_register_params_array() - Register parameter array for a KUnit test.

 * @test: The KUnit test structure to which parameters will be added.

 * @array: An array of test parameters.

 * @param_count: Number of parameters.

 * @get_desc: Function that generates a string description for a given parameter

 * element.

 *

 * This macro initializes the @test's parameter array data, storing information

 * including the parameter array, its count, the element size, and the parameter

 * description function within `test->params_array`.

 *

 * Note: If using this macro in param_init(), kunit_array_gen_params()

 * will then need to be manually provided as the parameter generator function to

 * KUNIT_CASE_PARAM_WITH_INIT(). kunit_array_gen_params() is a KUnit

 * function that uses the registered array to generate parameters

 */

#define kunit_register_params_array(test, array, param_count, get_desc)				\

	do {											\

		struct kunit *_test = (test);							\

		const typeof((array)[0]) * _params_ptr = &(array)[0];				\

		_test->params_array.params = _params_ptr;					\

		_test->params_array.num_params = (param_count);					\

		_test->params_array.elem_size = sizeof(*_params_ptr);				\

		_test->params_array.get_description = (get_desc);				\

	} while (0)

// TODO(dlatypov@google.com): consider eventually migrating users to explicitly

// include resource.h themselves if they need it.

#include <kunit/resource.h>

 * The thread counts are chosen to cover potentially interesting boundaries and

 * corner cases (2 to 5), and then stress the system with larger counts.

 */

static const void *nthreads_gen_params(const void *prev, char *desc)

static const void *nthreads_gen_params(struct kunit *test, const void *prev, char *desc)

{

	long nthreads = (long)prev;