Commit 91b7fbf3 authored by Eduard Zingerman's avatar Eduard Zingerman Committed by Andrii Nakryiko
Browse files

bpf, x86, riscv, arm: no_caller_saved_registers for bpf_get_smp_processor_id() 



The function bpf_get_smp_processor_id() is processed in a different
way, depending on the arch:
- on x86 verifier replaces call to bpf_get_smp_processor_id() with a
  sequence of instructions that modify only r0;
- on riscv64 jit replaces call to bpf_get_smp_processor_id() with a
  sequence of instructions that modify only r0;
- on arm64 jit replaces call to bpf_get_smp_processor_id() with a
  sequence of instructions that modify only r0 and tmp registers.

These rewrites satisfy attribute no_caller_saved_registers contract.
Allow rewrite of no_caller_saved_registers patterns for
bpf_get_smp_processor_id() in order to use this function as a canary
for no_caller_saved_registers tests.

Signed-off-by: default avatarEduard Zingerman <eddyz87@gmail.com>
Link: https://lore.kernel.org/r/20240722233844.1406874-4-eddyz87@gmail.com


Signed-off-by: default avatarAlexei Starovoitov <ast@kernel.org>
Signed-off-by: default avatarAndrii Nakryiko <andrii@kernel.org>
parent 5b5f51bf

kernel/bpf/helpers.c

+1 −0
Original line number Diff line number Diff line
@@ -158,6 +158,7 @@ const struct bpf_func_proto bpf_get_smp_processor_id_proto = { 
	.func		= bpf_get_smp_processor_id,

	.gpl_only	= false,

	.ret_type	= RET_INTEGER,

	.allow_nocsr	= true,

};



BPF_CALL_0(bpf_get_numa_node_id)

kernel/bpf/verifier.c

+9 −2
Original line number Diff line number Diff line
@@ -16130,8 +16130,15 @@ static u32 helper_nocsr_clobber_mask(const struct bpf_func_proto *fn) 
 */

static bool verifier_inlines_helper_call(struct bpf_verifier_env *env, s32 imm)

{

	switch (imm) {

#ifdef CONFIG_X86_64

	case BPF_FUNC_get_smp_processor_id:

		return env->prog->jit_requested && bpf_jit_supports_percpu_insn();

#endif

	default:

		return false;

	}

}









/* GCC and LLVM define a no_caller_saved_registers function attribute.









 * This attribute means that function scratches only some of








@@ -20834,7 +20841,7 @@ static int do_misc_fixups(struct bpf_verifier_env *env)







#if defined(CONFIG_X86_64) && !defined(CONFIG_UML)









		/* Implement bpf_get_smp_processor_id() inline. */









		if (insn->imm == BPF_FUNC_get_smp_processor_id &&









		    prog->jit_requested && bpf_jit_supports_percpu_insn()) {









		    verifier_inlines_helper_call(env, insn->imm)) {









			/* BPF_FUNC_get_smp_processor_id inlining is an









			 * optimization, so if pcpu_hot.cpu_number is ever









			 * changed in some incompatible and hard to support