Commit 5b2b7dec authored by Eric Dumazet's avatar Eric Dumazet Committed by Jakub Kicinski
Browse files

net: add add indirect call wrapper in skb_release_head_state() 



While stress testing UDP senders on a host with expensive indirect
calls, I found cpus processing TX completions where showing
a very high cost (20%) in sock_wfree() due to
CONFIG_MITIGATION_RETPOLINE=y.

Take care of TCP and UDP TX destructors and use INDIRECT_CALL_3() macro.

Signed-off-by: default avatarEric Dumazet <edumazet@google.com>
Reviewed-by: default avatarKuniyuki Iwashima <kuniyu@google.com>
Reviewed-by: default avatarToke Høiland-Jørgensen <toke@redhat.com>
Tested-by: default avatarJamal Hadi Salim <jhs@mojatatu.com>
Acked-by: default avatarJamal Hadi Salim <jhs@mojatatu.com>
Link: https://patch.msgid.link/20251014171907.3554413-3-edumazet@google.com


Signed-off-by: default avatarJakub Kicinski <kuba@kernel.org>
parent 56cef47c

net/core/skbuff.c

+10 −1
Original line number Diff line number Diff line
@@ -1136,7 +1136,16 @@ void skb_release_head_state(struct sk_buff *skb) 
	skb_dst_drop(skb);

	if (skb->destructor) {

		DEBUG_NET_WARN_ON_ONCE(in_hardirq());

		skb->destructor(skb);

#ifdef CONFIG_INET

		INDIRECT_CALL_3(skb->destructor,

				tcp_wfree, __sock_wfree, sock_wfree,

				skb);

#else

		INDIRECT_CALL_1(skb->destructor,

				sock_wfree,

				skb);



#endif

	}

#if IS_ENABLED(CONFIG_NF_CONNTRACK)

	nf_conntrack_put(skb_nfct(skb));