Commit 54e3eae8 authored by Miguel Ojeda's avatar Miguel Ojeda
Browse files

Merge patch series "`syn` support" 

This patch series introduces support for `syn` (and its dependencies):

    Syn is a parsing library for parsing a stream of Rust tokens into a
    syntax tree of Rust source code.

    Currently this library is geared toward use in Rust procedural
    macros, but contains some APIs that may be useful more generally.

It is the most downloaded Rust crate (according to crates.io), and it
is also used by the Rust compiler itself. Having such support allows to
greatly simplify writing complex macros such as `pin-init`. We will use
it in the `macros` crate too.

Benno has already prepared the `pin-init` version based on this, and on
top of that, we will be able to simplify the `macros` crate too. I think
Jesung is working on updating the `TryFrom` and `Into` upcoming derive
macros to use `syn` too.

The series starts with a few preparation commits (two fixes were already
merged in mainline that were discovered by this series), then each crate
is added. Finally, support for using the new crates from our `macros`
crate is introduced.

This has been a long time coming, e.g. even before Rust for Linux was
merged into the Linux kernel, Gary and Benno have wanted to use `syn`.
The first iterations of this, from 2022 and 2023 (with `serde` too,
another popular crate), are at:

    https://github.com/Rust-for-Linux/linux/pull/910
    https://github.com/Rust-for-Linux/linux/pull/1007

After those, we considered picking these from the distributions where
possible. However, after discussing it, it is not really worth the
complexity: vendoring makes things less complex and is less fragile.

In particular, we avoid having to support and test several versions,
we avoid having to introduce Cargo just to properly fetch the right
versions from the registry, we can easily customize the crates if needed
(e.g. dropping the `unicode_idents` dependency like it is done in this
series) and we simplify the configuration of the build for users for
which the "default" paths/registries would not have worked.

Moreover, nowadays, the ~57k lines introduced are not that much compared
to years ago (it dwarfed the actual Rust kernel code). Moreover, back
then it wasn't clear the Rust experiment would be a success, so it would
have been a bit pointless/risky to add many lines for nothing. Our macro
needs were also smaller in the early days.

So, finally, in Kangrejos 2025 we discussed going with the original,
simpler approach. Thus here it is the result.

There should not be many updates needed for these, and even if there
are, they should not be too big, e.g. +7k -3k lines across the 3 crates
in the last year.

Note that `syn` does not have all the features enabled, since we do not
need them so far, but they can easily be enabled just adding them to the
list.

Link: https://patch.msgid.link/20251124151837.2184382-1-ojeda@kernel.org


Signed-off-by: default avatarMiguel Ojeda <ojeda@kernel.org>
parents 7a0eae4d 52ba807f

.gitignore

+1 −0
Original line number Diff line number Diff line
@@ -41,6 +41,7 @@ 
*.o.*

*.patch

*.pyc

*.rlib

*.rmeta

*.rpm

*.rsi

Makefile

+7 −0
Original line number Diff line number Diff line
@@ -1826,10 +1826,17 @@ rusttest: prepare 
	$(Q)$(MAKE) $(build)=rust $@



# Formatting targets

#

# Generated files as well as vendored crates are skipped.

PHONY += rustfmt rustfmtcheck



rustfmt:

	$(Q)find $(srctree) $(RCS_FIND_IGNORE) \

		\( \

			-path $(srctree)/rust/proc-macro2 \

			-o -path $(srctree)/rust/quote \

			-o -path $(srctree)/rust/syn \

		\) -prune -o \

		-type f -a -name '*.rs' -a ! -name '*generated*' -print \

		| xargs $(RUSTFMT) $(rustfmt_flags)

rust/Makefile

+132 −15
Original line number Diff line number Diff line
@@ -27,6 +27,8 @@ endif 


obj-$(CONFIG_RUST) += exports.o



always-$(CONFIG_RUST) += libproc_macro2.rlib libquote.rlib libsyn.rlib



always-$(CONFIG_RUST_KERNEL_DOCTESTS) += doctests_kernel_generated.rs

always-$(CONFIG_RUST_KERNEL_DOCTESTS) += doctests_kernel_generated_kunit.c
@@ -60,11 +62,61 @@ rustdoc_test_quiet=--test-args -q 
rustdoc_test_kernel_quiet=>/dev/null

endif



core-cfgs = \

    --cfg no_fp_fmt_parse

cfgs-to-flags = $(patsubst %,--cfg='%',$1)



core-cfgs := \

    no_fp_fmt_parse



core-edition := $(if $(call rustc-min-version,108700),2024,2021)



core-skip_flags := \

    --edition=2021 \

    -Wunreachable_pub \

    -Wrustdoc::unescaped_backticks



core-flags := \

    --edition=$(core-edition) \

    $(call cfgs-to-flags,$(core-cfgs))



proc_macro2-cfgs := \

    feature="proc-macro" \

    wrap_proc_macro \

    $(if $(call rustc-min-version,108800),proc_macro_span_file proc_macro_span_location)



# Stable since Rust 1.79.0: `feature(proc_macro_byte_character,proc_macro_c_str_literals)`.

proc_macro2-flags := \

    --cap-lints=allow \

    -Zcrate-attr='feature(proc_macro_byte_character,proc_macro_c_str_literals)' \

    $(call cfgs-to-flags,$(proc_macro2-cfgs))



quote-cfgs := \

    feature="proc-macro"



quote-skip_flags := \

    --edition=2021



quote-flags := \

    --edition=2018 \

    --cap-lints=allow \

    --extern proc_macro2 \

    $(call cfgs-to-flags,$(quote-cfgs))



# `extra-traits`, `fold` and `visit` may be enabled if needed.

syn-cfgs := \

    feature="clone-impls" \

    feature="derive" \

    feature="full" \

    feature="parsing" \

    feature="printing" \

    feature="proc-macro" \

    feature="visit-mut"



syn-flags := \

    --cap-lints=allow \

    --extern proc_macro2 \

    --extern quote \

    $(call cfgs-to-flags,$(syn-cfgs))



# `rustdoc` did not save the target modifiers, thus workaround for

# the time being (https://github.com/rust-lang/rust/issues/144521).

rustdoc_modifiers_workaround := $(if $(call rustc-min-version,108800),-Cunsafe-allow-abi-mismatch=fixed-x18)
@@ -114,16 +166,33 @@ rustdoc: rustdoc-core rustdoc-macros rustdoc-compiler_builtins \ 
	$(Q)for f in $(rustdoc_output)/static.files/rustdoc-*.css; do \

		echo ".logo-container > img { object-fit: contain; }" >> $$f; done



rustdoc-proc_macro2: private rustdoc_host = yes

rustdoc-proc_macro2: private rustc_target_flags = $(proc_macro2-flags)

rustdoc-proc_macro2: $(src)/proc-macro2/lib.rs rustdoc-clean FORCE

	+$(call if_changed,rustdoc)



rustdoc-quote: private rustdoc_host = yes

rustdoc-quote: private rustc_target_flags = $(quote-flags)

rustdoc-quote: private skip_flags = $(quote-skip_flags)

rustdoc-quote: $(src)/quote/lib.rs rustdoc-clean rustdoc-proc_macro2 FORCE

	+$(call if_changed,rustdoc)



rustdoc-syn: private rustdoc_host = yes

rustdoc-syn: private rustc_target_flags = $(syn-flags)

rustdoc-syn: $(src)/syn/lib.rs rustdoc-clean rustdoc-quote FORCE

	+$(call if_changed,rustdoc)



rustdoc-macros: private rustdoc_host = yes

rustdoc-macros: private rustc_target_flags = --crate-type proc-macro \

    --extern proc_macro

rustdoc-macros: $(src)/macros/lib.rs rustdoc-clean FORCE

    --extern proc_macro --extern proc_macro2 --extern quote --extern syn

rustdoc-macros: $(src)/macros/lib.rs rustdoc-clean rustdoc-proc_macro2 \

    rustdoc-quote rustdoc-syn FORCE

	+$(call if_changed,rustdoc)



# Starting with Rust 1.82.0, skipping `-Wrustdoc::unescaped_backticks` should

# not be needed -- see https://github.com/rust-lang/rust/pull/128307.

rustdoc-core: private skip_flags = --edition=2021 -Wrustdoc::unescaped_backticks

rustdoc-core: private rustc_target_flags = --edition=$(core-edition) $(core-cfgs)

rustdoc-core: private skip_flags = $(core-skip_flags)

rustdoc-core: private rustc_target_flags = $(core-flags)

rustdoc-core: $(RUST_LIB_SRC)/core/src/lib.rs rustdoc-clean FORCE

	+$(call if_changed,rustdoc)
@@ -161,8 +230,8 @@ rustdoc-clean: FORCE 
quiet_cmd_rustc_test_library = $(RUSTC_OR_CLIPPY_QUIET) TL $<

      cmd_rustc_test_library = \

	OBJTREE=$(abspath $(objtree)) \

	$(RUSTC_OR_CLIPPY) $(rust_common_flags) \

		@$(objtree)/include/generated/rustc_cfg $(rustc_target_flags) \

	$(RUSTC_OR_CLIPPY) $(filter-out $(skip_flags),$(rust_common_flags) $(rustc_target_flags)) \

		@$(objtree)/include/generated/rustc_cfg \

		--crate-type $(if $(rustc_test_library_proc),proc-macro,rlib) \

		--out-dir $(objtree)/$(obj)/test --cfg testlib \

		-L$(objtree)/$(obj)/test \
@@ -174,9 +243,24 @@ rusttestlib-build_error: $(src)/build_error.rs FORCE 
rusttestlib-ffi: $(src)/ffi.rs FORCE

	+$(call if_changed,rustc_test_library)



rusttestlib-macros: private rustc_target_flags = --extern proc_macro

rusttestlib-proc_macro2: private rustc_target_flags = $(proc_macro2-flags)

rusttestlib-proc_macro2: $(src)/proc-macro2/lib.rs FORCE

	+$(call if_changed,rustc_test_library)



rusttestlib-quote: private skip_flags = $(quote-skip_flags)

rusttestlib-quote: private rustc_target_flags = $(quote-flags)

rusttestlib-quote: $(src)/quote/lib.rs rusttestlib-proc_macro2 FORCE

	+$(call if_changed,rustc_test_library)



rusttestlib-syn: private rustc_target_flags = $(syn-flags)

rusttestlib-syn: $(src)/syn/lib.rs rusttestlib-quote FORCE

	+$(call if_changed,rustc_test_library)



rusttestlib-macros: private rustc_target_flags = --extern proc_macro \

    --extern proc_macro2 --extern quote --extern syn

rusttestlib-macros: private rustc_test_library_proc = yes

rusttestlib-macros: $(src)/macros/lib.rs FORCE

rusttestlib-macros: $(src)/macros/lib.rs \

    rusttestlib-proc_macro2 rusttestlib-quote rusttestlib-syn FORCE

	+$(call if_changed,rustc_test_library)



rusttestlib-pin_init_internal: private rustc_target_flags = --cfg kernel \
@@ -257,7 +341,8 @@ quiet_cmd_rustc_test = $(RUSTC_OR_CLIPPY_QUIET) T $< 
rusttest: rusttest-macros



rusttest-macros: private rustc_target_flags = --extern proc_macro \

	--extern macros --extern kernel --extern pin_init

    --extern macros --extern kernel --extern pin_init \

    --extern proc_macro2 --extern quote --extern syn

rusttest-macros: private rustdoc_test_target_flags = --crate-type proc-macro

rusttest-macros: $(src)/macros/lib.rs \

    rusttestlib-macros rusttestlib-kernel rusttestlib-pin_init FORCE
@@ -410,18 +495,47 @@ $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE 
$(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE

	$(call if_changed,exports)



quiet_cmd_rustc_procmacrolibrary = $(RUSTC_OR_CLIPPY_QUIET) PL $@

      cmd_rustc_procmacrolibrary = \

	$(if $(skip_clippy),$(RUSTC),$(RUSTC_OR_CLIPPY)) \

		$(filter-out $(skip_flags),$(rust_common_flags) $(rustc_target_flags)) \

		--emit=dep-info,link --crate-type rlib -O \

		--out-dir $(objtree)/$(obj) -L$(objtree)/$(obj) \

		--crate-name $(patsubst lib%.rlib,%,$(notdir $@)) $<; \

	mv $(objtree)/$(obj)/$(patsubst lib%.rlib,%,$(notdir $@)).d $(depfile); \

	sed -i '/^\#/d' $(depfile)



$(obj)/libproc_macro2.rlib: private skip_clippy = 1

$(obj)/libproc_macro2.rlib: private rustc_target_flags = $(proc_macro2-flags)

$(obj)/libproc_macro2.rlib: $(src)/proc-macro2/lib.rs FORCE

	+$(call if_changed_dep,rustc_procmacrolibrary)



$(obj)/libquote.rlib: private skip_clippy = 1

$(obj)/libquote.rlib: private skip_flags = $(quote-skip_flags)

$(obj)/libquote.rlib: private rustc_target_flags = $(quote-flags)

$(obj)/libquote.rlib: $(src)/quote/lib.rs $(obj)/libproc_macro2.rlib FORCE

	+$(call if_changed_dep,rustc_procmacrolibrary)



$(obj)/libsyn.rlib: private skip_clippy = 1

$(obj)/libsyn.rlib: private rustc_target_flags = $(syn-flags)

$(obj)/libsyn.rlib: $(src)/syn/lib.rs $(obj)/libquote.rlib FORCE

	+$(call if_changed_dep,rustc_procmacrolibrary)



quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@

      cmd_rustc_procmacro = \

	$(RUSTC_OR_CLIPPY) $(rust_common_flags) $(rustc_target_flags) \

		-Clinker-flavor=gcc -Clinker=$(HOSTCC) \

		-Clink-args='$(call escsq,$(KBUILD_PROCMACROLDFLAGS))' \

		--emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \

		--crate-type proc-macro \

		--crate-type proc-macro -L$(objtree)/$(obj) \

		--crate-name $(patsubst lib%.$(libmacros_extension),%,$(notdir $@)) \

		@$(objtree)/include/generated/rustc_cfg $<



# Procedural macros can only be used with the `rustc` that compiled it.

$(obj)/$(libmacros_name): $(src)/macros/lib.rs FORCE

$(obj)/$(libmacros_name): private rustc_target_flags = \

    --extern proc_macro2 --extern quote --extern syn

$(obj)/$(libmacros_name): $(src)/macros/lib.rs $(obj)/libproc_macro2.rlib \

    $(obj)/libquote.rlib $(obj)/libsyn.rlib FORCE

	+$(call if_changed_dep,rustc_procmacro)



$(obj)/$(libpin_init_internal_name): private rustc_target_flags = --cfg kernel
@@ -444,6 +558,9 @@ quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L 
rust-analyzer:

	$(Q)MAKEFLAGS= $(srctree)/scripts/generate_rust_analyzer.py \

		--cfgs='core=$(core-cfgs)' $(core-edition) \

		--cfgs='proc_macro2=$(proc_macro2-cfgs)' \

		--cfgs='quote=$(quote-cfgs)' \

		--cfgs='syn=$(syn-cfgs)' \

		$(realpath $(srctree)) $(realpath $(objtree)) \

		$(rustc_sysroot) $(RUST_LIB_SRC) $(if $(KBUILD_EXTMOD),$(srcroot)) \

		> rust-project.json
@@ -499,9 +616,9 @@ $(obj)/helpers/helpers.o: $(src)/helpers/helpers.c $(recordmcount_source) FORCE 
$(obj)/exports.o: private skip_gendwarfksyms = 1



$(obj)/core.o: private skip_clippy = 1

$(obj)/core.o: private skip_flags = --edition=2021 -Wunreachable_pub

$(obj)/core.o: private skip_flags = $(core-skip_flags)

$(obj)/core.o: private rustc_objcopy = $(foreach sym,$(redirect-intrinsics),--redefine-sym $(sym)=__rust$(sym))

$(obj)/core.o: private rustc_target_flags = --edition=$(core-edition) $(core-cfgs)

$(obj)/core.o: private rustc_target_flags = $(core-flags)

$(obj)/core.o: $(RUST_LIB_SRC)/core/src/lib.rs \

    $(wildcard $(objtree)/include/config/RUSTC_VERSION_TEXT) FORCE

	+$(call if_changed_rule,rustc_library)

rust/proc-macro2/README.md

0 → 100644
+13 −0
Original line number Diff line number Diff line 
# `proc-macro2`



These source files come from the Rust `proc-macro2` crate, version

1.0.101 (released 2025-08-16), hosted in the

<https://github.com/dtolnay/proc-macro2> repository, licensed under

"Apache-2.0 OR MIT" and only modified to add the SPDX license

identifiers and to remove the `unicode-ident` dependency.



For copyright details, please see:



    https://github.com/dtolnay/proc-macro2/blob/1.0.101/README.md#license

    https://github.com/dtolnay/proc-macro2/blob/1.0.101/LICENSE-APACHE

    https://github.com/dtolnay/proc-macro2/blob/1.0.101/LICENSE-MIT

rust/proc-macro2/detection.rs

0 → 100644
+77 −0
Original line number Diff line number Diff line 
// SPDX-License-Identifier: Apache-2.0 OR MIT



use core::sync::atomic::{AtomicUsize, Ordering};

use std::sync::Once;



static WORKS: AtomicUsize = AtomicUsize::new(0);

static INIT: Once = Once::new();



pub(crate) fn inside_proc_macro() -> bool {

    match WORKS.load(Ordering::Relaxed) {

        1 => return false,

        2 => return true,

        _ => {}

    }



    INIT.call_once(initialize);

    inside_proc_macro()

}



pub(crate) fn force_fallback() {

    WORKS.store(1, Ordering::Relaxed);

}



pub(crate) fn unforce_fallback() {

    initialize();

}



#[cfg(not(no_is_available))]

fn initialize() {

    let available = proc_macro::is_available();

    WORKS.store(available as usize + 1, Ordering::Relaxed);

}



// Swap in a null panic hook to avoid printing "thread panicked" to stderr,

// then use catch_unwind to determine whether the compiler's proc_macro is

// working. When proc-macro2 is used from outside of a procedural macro all

// of the proc_macro crate's APIs currently panic.

//

// The Once is to prevent the possibility of this ordering:

//

//     thread 1 calls take_hook, gets the user's original hook

//     thread 1 calls set_hook with the null hook

//     thread 2 calls take_hook, thinks null hook is the original hook

//     thread 2 calls set_hook with the null hook

//     thread 1 calls set_hook with the actual original hook

//     thread 2 calls set_hook with what it thinks is the original hook

//

// in which the user's hook has been lost.

//

// There is still a race condition where a panic in a different thread can

// happen during the interval that the user's original panic hook is

// unregistered such that their hook is incorrectly not called. This is

// sufficiently unlikely and less bad than printing panic messages to stderr

// on correct use of this crate. Maybe there is a libstd feature request

// here. For now, if a user needs to guarantee that this failure mode does

// not occur, they need to call e.g. `proc_macro2::Span::call_site()` from

// the main thread before launching any other threads.

#[cfg(no_is_available)]

fn initialize() {

    use std::panic::{self, PanicInfo};



    type PanicHook = dyn Fn(&PanicInfo) + Sync + Send + 'static;



    let null_hook: Box<PanicHook> = Box::new(|_panic_info| { /* ignore */ });

    let sanity_check = &*null_hook as *const PanicHook;

    let original_hook = panic::take_hook();

    panic::set_hook(null_hook);



    let works = panic::catch_unwind(proc_macro::Span::call_site).is_ok();

    WORKS.store(works as usize + 1, Ordering::Relaxed);



    let hopefully_null_hook = panic::take_hook();

    panic::set_hook(original_hook);

    if sanity_check != &*hopefully_null_hook {

        panic!("observed race condition in proc_macro2::inside_proc_macro");

    }

}