Commit 12079a59 authored by Breno Leitao's avatar Breno Leitao Committed by Paolo Abeni
Browse files

net: Implement fault injection forcing skb reallocation 

Introduce a fault injection mechanism to force skb reallocation. The
primary goal is to catch bugs related to pointer invalidation after
potential skb reallocation.

The fault injection mechanism aims to identify scenarios where callers
retain pointers to various headers in the skb but fail to reload these
pointers after calling a function that may reallocate the data. This
type of bug can lead to memory corruption or crashes if the old,
now-invalid pointers are used.

By forcing reallocation through fault injection, we can stress-test code
paths and ensure proper pointer management after potential skb
reallocations.

Add a hook for fault injection in the following functions:

 * pskb_trim_rcsum()
 * pskb_may_pull_reason()
 * pskb_trim()

As the other fault injection mechanism, protect it under a debug Kconfig
called CONFIG_FAIL_SKB_REALLOC.

This patch was *heavily* inspired by Jakub's proposal from:
https://lore.kernel.org/all/20240719174140.47a868e6@kernel.org/



CC: Akinobu Mita <akinobu.mita@gmail.com>
Suggested-by: default avatarJakub Kicinski <kuba@kernel.org>
Signed-off-by: default avatarBreno Leitao <leitao@debian.org>
Reviewed-by: default avatarAkinobu Mita <akinobu.mita@gmail.com>
Acked-by: default avatarPaolo Abeni <pabeni@redhat.com>
Acked-by: default avatarGuillaume Nault <gnault@redhat.com>
Link: https://patch.msgid.link/20241107-fault_v6-v6-1-1b82cb6ecacd@debian.org


Signed-off-by: default avatarPaolo Abeni <pabeni@redhat.com>
parent 12f077a7

Documentation/admin-guide/kernel-parameters.txt

+1 −0
Original line number Diff line number Diff line
@@ -1546,6 +1546,7 @@ 
	failslab=

	fail_usercopy=

	fail_page_alloc=

	fail_skb_realloc=

	fail_make_request=[KNL]

			General fault injection mechanism.

			Format: <interval>,<probability>,<space>,<times>

Documentation/fault-injection/fault-injection.rst

+40 −0
Original line number Diff line number Diff line
@@ -45,6 +45,32 @@ Available fault injection capabilities 
  ALLOW_ERROR_INJECTION() macro, by setting debugfs entries

  under /sys/kernel/debug/fail_function. No boot option supported.



- fail_skb_realloc



  inject skb (socket buffer) reallocation events into the network path. The

  primary goal is to identify and prevent issues related to pointer

  mismanagement in the network subsystem.  By forcing skb reallocation at

  strategic points, this feature creates scenarios where existing pointers to

  skb headers become invalid.



  When the fault is injected and the reallocation is triggered, cached pointers

  to skb headers and data no longer reference valid memory locations. This

  deliberate invalidation helps expose code paths where proper pointer updating

  is neglected after a reallocation event.



  By creating these controlled fault scenarios, the system can catch instances

  where stale pointers are used, potentially leading to memory corruption or

  system instability.



  To select the interface to act on, write the network name to

  /sys/kernel/debug/fail_skb_realloc/devname.

  If this field is left empty (which is the default value), skb reallocation

  will be forced on all network interfaces.



  The effectiveness of this fault detection is enhanced when KASAN is

  enabled, as it helps identify invalid memory references and use-after-free

  (UAF) issues.



- NVMe fault injection



  inject NVMe status code and retry flag on devices permitted by setting
@@ -216,6 +242,19 @@ configuration of fault-injection capabilities. 
	use a negative errno, you better use 'printf' instead of 'echo', e.g.:

	$ printf %#x -12 > retval



- /sys/kernel/debug/fail_skb_realloc/devname:



        Specifies the network interface on which to force SKB reallocation.  If

        left empty, SKB reallocation will be applied to all network interfaces.



        Example usage::



          # Force skb reallocation on eth0

          echo "eth0" > /sys/kernel/debug/fail_skb_realloc/devname



          # Clear the selection and force skb reallocation on all interfaces

          echo "" > /sys/kernel/debug/fail_skb_realloc/devname



Boot option

^^^^^^^^^^^
@@ -227,6 +266,7 @@ use the boot option:: 
	fail_usercopy=

	fail_make_request=

	fail_futex=

	fail_skb_realloc=

	mmc_core.fail_request=<interval>,<probability>,<space>,<times>



proc entries

include/linux/skbuff.h

+9 −0
Original line number Diff line number Diff line
@@ -2682,6 +2682,12 @@ static inline void skb_assert_len(struct sk_buff *skb) 
#endif /* CONFIG_DEBUG_NET */

}



#if defined(CONFIG_FAIL_SKB_REALLOC)

void skb_might_realloc(struct sk_buff *skb);

#else

static inline void skb_might_realloc(struct sk_buff *skb) {}

#endif



/*

 *	Add data to an sk_buff

 */
@@ -2782,6 +2788,7 @@ static inline enum skb_drop_reason 
pskb_may_pull_reason(struct sk_buff *skb, unsigned int len)

{

	DEBUG_NET_WARN_ON_ONCE(len > INT_MAX);

	skb_might_realloc(skb);



	if (likely(len <= skb_headlen(skb)))

		return SKB_NOT_DROPPED_YET;
@@ -3240,6 +3247,7 @@ static inline int __pskb_trim(struct sk_buff *skb, unsigned int len) 


static inline int pskb_trim(struct sk_buff *skb, unsigned int len)

{

	skb_might_realloc(skb);

	return (len < skb->len) ? __pskb_trim(skb, len) : 0;

}
@@ -3994,6 +4002,7 @@ int pskb_trim_rcsum_slow(struct sk_buff *skb, unsigned int len); 


static inline int pskb_trim_rcsum(struct sk_buff *skb, unsigned int len)

{

	skb_might_realloc(skb);

	if (likely(len >= skb->len))

		return 0;

	return pskb_trim_rcsum_slow(skb, len);

lib/Kconfig.debug

+10 −0
Original line number Diff line number Diff line
@@ -2115,6 +2115,16 @@ config FAIL_SUNRPC 
	  Provide fault-injection capability for SunRPC and

	  its consumers.



config FAIL_SKB_REALLOC

	bool "Fault-injection capability forcing skb to reallocate"

	depends on FAULT_INJECTION_DEBUG_FS

	help

	  Provide fault-injection capability that forces the skb to be

	  reallocated, catching possible invalid pointers to the skb.



	  For more information, check

	  Documentation/dev-tools/fault-injection/fault-injection.rst



config FAULT_INJECTION_CONFIGFS

	bool "Configfs interface for fault-injection capabilities"

	depends on FAULT_INJECTION

net/core/Makefile

+1 −0
Original line number Diff line number Diff line
@@ -46,3 +46,4 @@ obj-$(CONFIG_OF) += of_net.o 
obj-$(CONFIG_NET_TEST) += net_test.o

obj-$(CONFIG_NET_DEVMEM) += devmem.o

obj-$(CONFIG_DEBUG_NET_SMALL_RTNL) += rtnl_net_debug.o

obj-$(CONFIG_FAIL_SKB_REALLOC) += skb_fault_injection.o