Commit 8989d328 authored by Jakub Sitnicki's avatar Jakub Sitnicki Committed by Martin KaFai Lau
Browse files

net: Helper to move packet data and metadata after skb_push/pull 



Lay groundwork for fixing BPF helpers available to TC(X) programs.

When skb_push() or skb_pull() is called in a TC(X) ingress BPF program, the
skb metadata must be kept in front of the MAC header. Otherwise, BPF
programs using the __sk_buff->data_meta pseudo-pointer lose access to it.

Introduce a helper that moves both metadata and a specified number of
packet data bytes together, suitable as a drop-in replacement for
memmove().

Signed-off-by: default avatarJakub Sitnicki <jakub@cloudflare.com>
Signed-off-by: default avatarMartin KaFai Lau <martin.lau@kernel.org>
Link: https://patch.msgid.link/20251105-skb-meta-rx-path-v4-1-5ceb08a9b37b@cloudflare.com
parent a0c3aefb

include/linux/skbuff.h

+75 −0
Original line number Diff line number Diff line
@@ -4564,6 +4564,81 @@ static inline void skb_metadata_clear(struct sk_buff *skb) 
	skb_metadata_set(skb, 0);

}



/**

 * skb_data_move - Move packet data and metadata after skb_push() or skb_pull().

 * @skb: packet to operate on

 * @len: number of bytes pushed or pulled from &sk_buff->data

 * @n: number of bytes to memmove() from pre-push/pull &sk_buff->data

 *

 * Moves @n bytes of packet data, can be zero, and all bytes of skb metadata.

 *

 * Assumes metadata is located immediately before &sk_buff->data prior to the

 * push/pull, and that sufficient headroom exists to hold it after an

 * skb_push(). Otherwise, metadata is cleared and a one-time warning is issued.

 *

 * Prefer skb_postpull_data_move() or skb_postpush_data_move() to calling this

 * helper directly.

 */

static inline void skb_data_move(struct sk_buff *skb, const int len,

				 const unsigned int n)

{

	const u8 meta_len = skb_metadata_len(skb);

	u8 *meta, *meta_end;



	if (!len || (!n && !meta_len))

		return;



	if (!meta_len)

		goto no_metadata;



	meta_end = skb_metadata_end(skb);

	meta = meta_end - meta_len;



	if (WARN_ON_ONCE(meta_end + len != skb->data ||

			 meta_len > skb_headroom(skb))) {

		skb_metadata_clear(skb);

		goto no_metadata;

	}



	memmove(meta + len, meta, meta_len + n);

	return;



no_metadata:

	memmove(skb->data, skb->data - len, n);

}



/**

 * skb_postpull_data_move - Move packet data and metadata after skb_pull().

 * @skb: packet to operate on

 * @len: number of bytes pulled from &sk_buff->data

 * @n: number of bytes to memmove() from pre-pull &sk_buff->data

 *

 * See skb_data_move() for details.

 */

static inline void skb_postpull_data_move(struct sk_buff *skb,

					  const unsigned int len,

					  const unsigned int n)

{

	DEBUG_NET_WARN_ON_ONCE(len > INT_MAX);

	skb_data_move(skb, len, n);

}



/**

 * skb_postpush_data_move - Move packet data and metadata after skb_push().

 * @skb: packet to operate on

 * @len: number of bytes pushed onto &sk_buff->data

 * @n: number of bytes to memmove() from pre-push &sk_buff->data

 *

 * See skb_data_move() for details.

 */

static inline void skb_postpush_data_move(struct sk_buff *skb,

					  const unsigned int len,

					  const unsigned int n)

{

	DEBUG_NET_WARN_ON_ONCE(len > INT_MAX);

	skb_data_move(skb, -len, n);

}



struct sk_buff *skb_clone_sk(struct sk_buff *skb);



#ifdef CONFIG_NETWORK_PHY_TIMESTAMPING