Commit c9525d24 authored by Jason Xing's avatar Jason Xing Committed by Martin KaFai Lau
Browse files

bpf: Add BPF_SOCK_OPS_TSTAMP_SENDMSG_CB callback 



This patch introduces a new callback in tcp_tx_timestamp() to correlate
tcp_sendmsg timestamp with timestamps from other tx timestamping
callbacks (e.g., SND/SW/ACK).

Without this patch, BPF program wouldn't know which timestamps belong
to which flow because of no socket lock protection. This new callback
is inserted in tcp_tx_timestamp() to address this issue because
tcp_tx_timestamp() still owns the same socket lock with
tcp_sendmsg_locked() in the meanwhile tcp_tx_timestamp() initializes
the timestamping related fields for the skb, especially tskey. The
tskey is the bridge to do the correlation.

For TCP, BPF program hooks the beginning of tcp_sendmsg_locked() and
then stores the sendmsg timestamp at the bpf_sk_storage, correlating
this timestamp with its tskey that are later used in other sending
timestamping callbacks.

Signed-off-by: default avatarJason Xing <kerneljasonxing@gmail.com>
Signed-off-by: default avatarMartin KaFai Lau <martin.lau@kernel.org>
Reviewed-by: default avatarWillem de Bruijn <willemb@google.com>
Link: https://patch.msgid.link/20250220072940.99994-11-kerneljasonxing@gmail.com
parent b3b81e6b

include/uapi/linux/bpf.h

+5 −0
Original line number Diff line number Diff line
@@ -7049,6 +7049,11 @@ enum { 
					 * when SK_BPF_CB_TX_TIMESTAMPING

					 * feature is on.

					 */

	BPF_SOCK_OPS_TSTAMP_SENDMSG_CB,	/* Called when every sendmsg syscall

					 * is triggered. It's used to correlate

					 * sendmsg timestamp with corresponding

					 * tskey.

					 */

};



/* List of TCP states. There is a build check in net/ipv4/tcp.c to detect

net/ipv4/tcp.c

+4 −0
Original line number Diff line number Diff line
@@ -496,6 +496,10 @@ static void tcp_tx_timestamp(struct sock *sk, struct sockcm_cookie *sockc) 
		if (tsflags & SOF_TIMESTAMPING_TX_RECORD_MASK)

			shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1;

	}



	if (cgroup_bpf_enabled(CGROUP_SOCK_OPS) &&

	    SK_BPF_CB_FLAG_TEST(sk, SK_BPF_CB_TX_TIMESTAMPING) && skb)

		bpf_skops_tx_timestamping(sk, skb, BPF_SOCK_OPS_TSTAMP_SENDMSG_CB);

}



static bool tcp_stream_is_readable(struct sock *sk, int target)

tools/include/uapi/linux/bpf.h

+5 −0
Original line number Diff line number Diff line
@@ -7049,6 +7049,11 @@ enum { 
					 * when SK_BPF_CB_TX_TIMESTAMPING

					 * feature is on.

					 */

	BPF_SOCK_OPS_TSTAMP_SENDMSG_CB,	/* Called when every sendmsg syscall

					 * is triggered. It's used to correlate

					 * sendmsg timestamp with corresponding

					 * tskey.

					 */

};



/* List of TCP states. There is a build check in net/ipv4/tcp.c to detect