My changes for "Module Declarations Shouldn’t be Macros" paper broke
the following testcase. The backup handling intentionally tries to
drop CPP_PRAGMA_EOL token if things go wrong, which is desirable for the
case where we haven't committed to the module preprocessing directive
(i.e. changed the first token to the magic one). In that case there is
no preprocessing directive start and so CPP_PRAGMA_EOL would be wrong.
If there is a premature new-line after we've changed the first token though,
we shouldn't drop CPP_PRAGMA_EOL, because otherwise we ICE in the FE.
While clang++ and MSVC accept the testcase, in my reading it is incorrect
at least in the C++23 and newer wordings and I think the changes have been
a DR, https://eel.is/c++draft/cpp.module has no exception for new-lines
and https://eel.is/c++draft/cpp.pre#1.sentence-2 says that new-line (unless
deleted during phase 2 when after backslash) ends the preprocessing
directive.
The patch arranges for eol being set only in the not_module case.
2025-08-03 Jakub Jelinek <jakub@redhat.com>
PR c++/120845
libcpp/
* lex.cc (cpp_maybe_module_directive): Move eol variable declaration
to the start of the function, initialize to false and only set it to
peek->type == CPP_PRAGMA_EOL in the not_module case. Formatting fix.
gcc/testsuite/
* g++.dg/modules/cpp-21.C: New test.
(cherry picked from commit a68f416a70)
The backport of the PR108900 fix to 14 branch broke building chromium
because static_assert (__LINE__ == expected_line_number, ""); now triggers
as the __LINE__ values are off by one.
This isn't the case on the trunk and 15 branch because we've switched
to 64-bit location_t and so one actually needs far longer header files
to trigger it.
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=120061#c11https://gcc.gnu.org/bugzilla/show_bug.cgi?id=120061#c12
contain (large) testcases in patch form which show on the 14 branch
that the first one used to fail before the PR108900 backport and now
works correctly, while the second one attempts to match the chromium
behavior and it used to pass before the PR108900 backport and now it
FAILs.
The two testcases show rare problematic cases, because
do_include_common -> parse_include -> check_eol -> check_eol_1 ->
cpp_get_token_1 -> _cpp_lex_token -> _cpp_lex_direct -> linemap_line_start
triggers there
/* Allocate the new line_map. However, if the current map only has a
single line we can sometimes just increase its column_bits instead. */
if (line_delta < 0
|| last_line != ORDINARY_MAP_STARTING_LINE_NUMBER (map)
|| SOURCE_COLUMN (map, highest) >= (1U << (column_bits - range_bits))
|| ( /* We can't reuse the map if the line offset is sufficiently
large to cause overflow when computing location_t values. */
(to_line - ORDINARY_MAP_STARTING_LINE_NUMBER (map))
>= (((uint64_t) 1)
<< (CHAR_BIT * sizeof (linenum_type) - column_bits)))
|| range_bits < map->m_range_bits)
map = linemap_check_ordinary
(const_cast <line_map *>
(linemap_add (set, LC_RENAME,
ORDINARY_MAP_IN_SYSTEM_HEADER_P (map),
ORDINARY_MAP_FILE_NAME (map),
to_line)));
and so creates a new ordinary map on the line right after the
(problematic) #include line.
Now, in the spot that r14-11679-g8a884140c2bcb7 patched,
pfile->line_table->highest_location in all 3 tests (also
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=120061#c13
) is before the decrement the start of the line after the #include line and so
the decrement is really desirable in that case to put highest_location
somewhere on the line where the #include actually is.
But at the same time it is also undesirable, because if we do decrement it,
then linemap_add LC_ENTER called from _cpp_do_file_change will then
/* Generate a start_location above the current highest_location.
If possible, make the low range bits be zero. */
location_t start_location = set->highest_location + 1;
unsigned range_bits = 0;
if (start_location < LINE_MAP_MAX_LOCATION_WITH_COLS)
range_bits = set->default_range_bits;
start_location += (1 << range_bits) - 1;
start_location &= ~((1 << range_bits) - 1);
linemap_assert (!LINEMAPS_ORDINARY_USED (set)
|| (start_location
>= MAP_START_LOCATION (LINEMAPS_LAST_ORDINARY_MAP (set))));
and we can end up with the new LC_ENTER ordinary map having the same
start_location as the preceding LC_RENAME one.
Next thing that happens is computation of included_from:
if (reason == LC_ENTER)
{
if (set->depth == 0)
map->included_from = 0;
else
/* The location of the end of the just-closed map. */
map->included_from
= (((map[0].start_location - 1 - map[-1].start_location)
& ~((1 << map[-1].m_column_and_range_bits) - 1))
+ map[-1].start_location);
The normal case (e.g. with the testcase included at the start of this comment) is
that map[-1] starts somewhere earlier and so map->included_from computation above
nicely computes location_t which expands to the start of the #include line.
With r14-11679 reverted, for #c11 as well as #c12
map[0].start_location == map[-1].start_location above, and so it is
((location_t) -1 & ~((1 << map[-1].m_column_and_range_bits) - 1)))
+ map[-1].start_location,
which happens to be start of the #include line.
For #c11 map[0].start_location is 0x500003a0 and map[-1] has
m_column_and_range_bits 7 and map[-2] has m_column_and_range_bits 12 and
map[0].included_from is set to 0x50000320.
For #c12 map[0].start_location is 0x606c0402 and map[-2].start_location is
0x606c0400 and m_column_and_range_bits is 0 for all 3 maps.
map[0].included_from is set to 0x606c0401.
The last important part is again in linemap_add when doing LC_LEAVE:
/* (MAP - 1) points to the map we are leaving. The
map from which (MAP - 1) got included should be the map
that comes right before MAP in the same file. */
from = linemap_included_from_linemap (set, map - 1);
/* A TO_FILE of NULL is special - we use the natural values. */
if (to_file == NULL)
{
to_file = ORDINARY_MAP_FILE_NAME (from);
to_line = SOURCE_LINE (from, from[1].start_location);
sysp = ORDINARY_MAP_IN_SYSTEM_HEADER_P (from);
}
Here it wants to compute the right to_line which ought to be the line after
the #include directive.
On the #c11 testcase that doesn't work correctly though, because
map[-1].included_from is 0x50000320, from[0] for that is LC_ENTER with
start_location 0x4080 and m_column_and_range_bits 12 but note that we've
earlier computed map[-1].start_location + (-1 & 0xffffff80) and so only
decreased by 7 bits, so to_line is still on the line with #include and not
after it. In the #c12 that doesn't happen, all the ordinary maps involved
there had 0 m_column_and_range_bits and so this computes correct line.
Below is a fix for the trunk including testcases using the
location_overflow_plugin hack to simulate the bugs without needing huge
files (in the 14 case it is just 330KB and almost 10MB, but in the 15
case it would need to be far bigger).
The pre- r15-9018 trunk has
FAIL: gcc.dg/plugin/location-overflow-test-pr116047.c -fplugin=./location_overflow_plugin.so scan-file static_assert[^\n\r]*6[^\n\r]*== 6
and current trunk
FAIL: gcc.dg/plugin/location-overflow-test-pr116047.c -fplugin=./location_overflow_plugin.so scan-file static_assert[^\n\r]*6[^\n\r]*== 6
FAIL: gcc.dg/plugin/location-overflow-test-pr120061.c -fplugin=./location_overflow_plugin.so scan-file static_assert[^\n\r]*5[^\n\r]*== 5
and with the patch everything PASSes.
I'll post afterwards a 14 version of the patch.
The patch reverts the r15-9018 change, because it is incorrect,
we really need to decrement it even when crossing ordinary map
boundaries, so that the location is not on the line after the #include
line but somewhere on the #include line. It also patches two spots
in linemap_add mentioned above to make sure we get correct locations
both in the included_from location_t when doing LC_ENTER (second
line-map.cc hunk) and when doing LC_LEAVE to compute the right to_line
(first line-map.cc hunk), both in presence of an added LC_RENAME
with the same start_location as the following LC_ENTER (i.e. the
problematic cases).
The LC_ENTER hunk is mostly to ensure included_form location_t is
at the start of the #include line (column 0), without it we can
decrease include_from not enough and end up at some random column
in the middle of the line, because it is masking away
map[-1].m_column_and_range_bits bits even when in the end the resulting
include_from location_t will be found in map[-2] map with perhaps
different m_column_and_range_bits. That alone doesn't fix the bug
though.
The more important is the LC_LEAVE hunk and the problem there is
caused by linemap_line_start not actually doing
r = set->highest_line + (line_delta << map->m_column_and_range_bits);
when adding a new map (the LC_RENAME one because we need to switch to
different number of directly encoded ranges, or columns, etc.).
So, in the original PR108900 case that
to_line = SOURCE_LINE (from, from[1].start_location);
doesn't do the right thing, from there is the last < 0x50000000 map
with m_column_and_range_bits 12, from[1] is the first one above it
and map[-1].included_from is the correct location of column 0 on
the #include line, but as the new LC_RENAME map has been created without
actually increasing highest_location to be on the new line (we've just
set to_line of the new LC_RENAME map to the correct line),
to_line = SOURCE_LINE (from, from[1].start_location);
stays on the same source line. I've tried to just replace that with
to_line = SOURCE_LINE (from, linemap_included_from (map - 1)) + 1;
i.e. just find out the #include line from map[-1].included_from and
add 1 to it, unfortunately that breaks the
c-c++-common/cpp/line-4.c
test where we expect to stay on the same 0 line for LC_LEAVE from
<command line> and gcc.dg/cpp/trad/Wunused.c, gcc.dg/cpp/trad/builtins.c
and c-c++-common/analyzer/named-constants-via-macros-traditional.c tests
all with -traditional-cpp preprocessing where to_line is also off-by-one
from the expected one.
So, this patch instead conditionalizes it, uses the
to_line = SOURCE_LINE (from, linemap_included_from (map - 1)) + 1;
way only if from[1] is a LC_RENAME map (rather than the usual
LC_ENTER one), that should limit it to the problematic cases of when
parse_include peeked after EOL and had to create LC_RENAME map with
the same start_location as the LC_ENTER after it.
Some further justification for the LC_ENTER hunk, using the
https://gcc.gnu.org/pipermail/gcc-patches/2025-May/682774.html testcase
(old is 14 before r14-11679, vanilla current 14 and new with the 14 patch)
I get
$ /usr/src/gcc-14/obj/gcc/cc1.old -quiet -std=c23 pr116047.c -nostdinc
In file included from pr116047-1.h:327677:21,
from pr116047.c:4:
pr116047-2.h:1:1: error: unknown type name ‘a’
1 | a b c;
| ^
pr116047-2.h:1:5: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘c’
1 | a b c;
| ^
pr116047-1.h:327677:1: error: static assertion failed: ""
327677 | #include "pr116047-2.h"
| ^~~~~~~~~~~~~
$ /usr/src/gcc-14/obj/gcc/cc1.vanilla -quiet -std=c23 pr116047.c -nostdinc
In file included from pr116047-1.h:327678,
from pr116047.c:4:
pr116047-2.h:1:1: error: unknown type name ‘a’
1 | a b c;
| ^
pr116047-2.h:1:5: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘c’
1 | a b c;
| ^
$ /usr/src/gcc-14/obj/gcc/cc1.new -quiet -std=c23 pr116047.c -nostdinc
In file included from pr116047-1.h:327677,
from pr116047.c:4:
pr116047-2.h:1:1: error: unknown type name ‘a’
1 | a b c;
| ^
pr116047-2.h:1:5: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘c’
1 | a b c;
| ^
pr116047-1.h has on lines 327677+327678:
#include "pr116047-2.h"
static_assert (__LINE__ == 327678, "");
so the static_assert failure is something that was dealt mainly in the
LC_LEAVE hunk and files.cc reversion, but please have a look at the
In file included from lines.
14.2 emits correct line (#include "pr116047-2.h" is indeed on line
327677) but some random column in there (which is not normally printed
for smaller headers; 21 is the . before extension in the filename).
Current trunk emits incorrect line (327678 instead of 327677, clearly
it didn't decrement).
And the patched compiler emits the right line with no column, as would
be printed if I remove e.g. 300000 newlines from the file.
2025-05-07 Jakub Jelinek <jakub@redhat.com>
PR preprocessor/108900
PR preprocessor/116047
PR preprocessor/120061
* files.cc (_cpp_stack_file): Revert 2025-03-28 change.
* line-map.cc (linemap_add): Use
SOURCE_LINE (from, linemap_included_from (map - 1)) + 1; instead of
SOURCE_LINE (from, from[1].start_location); to compute to_line
for LC_LEAVE. For LC_ENTER included_from computation, look at
map[-2] or even lower if map[-1] has the same start_location as
map[0].
* gcc.dg/plugin/plugin.exp: Add location-overflow-test-pr116047.c
and location-overflow-test-pr120061.c.
* gcc.dg/plugin/location_overflow_plugin.cc (plugin_init): Don't error
on unknown values, instead just break. Handle 0x4fHHHHHH arguments
differently.
* gcc.dg/plugin/location-overflow-test-pr116047.c: New test.
* gcc.dg/plugin/location-overflow-test-pr116047-1.h: New test.
* gcc.dg/plugin/location-overflow-test-pr116047-2.h: New test.
* gcc.dg/plugin/location-overflow-test-pr120061.c: New test.
* gcc.dg/plugin/location-overflow-test-pr120061-1.h: New test.
* gcc.dg/plugin/location-overflow-test-pr120061-2.h: New test.
(cherry picked from commit edf745dc51)
The following testcase ICEs after emitting one pedwarn (about using
__VA_ARGS__ in a place where it shouldn't be used) and one error.
The error is emitted by _cpp_save_parameter where it sees the node
has been used already earlier. But unlike the other _cpp_save_parameter
caller which does goto out; if it returns false, this call with explicit
__VA_ARGS__ doesn't and if it increments number of parameters etc. after
the error, we then try to unsave it twice.
The following patch fixes it by doing the goto out in that case too,
the macro will then not be considered as variable arguments macro,
but for error recovery I think that is fine.
The other option would be before the other _cpp_save_parameter caller
check if the node is pfile->spec_nodes.n__VA_ARGS__ and in that case
also error and goto out, but that seems more expensive than this for
the common case that the macro definition is correct.
2025-04-09 Jakub Jelinek <jakub@redhat.com>
PR preprocessor/118674
* macro.cc (parse_params) <case CPP_ELLIPSIS>: If _cpp_save_parameter
failed for __VA_ARGS__, goto out.
* gcc.dg/cpp/pr118674.c: New test.
The libcpp left shift handling implements (partially) the C99-C23
wording where shifts are UB if shift count is negative, or too large,
or shifting left a negative value or shifting left non-negative value
results in something not representable in the result type (in the
preprocessor case that is intmax_t).
libcpp actually implements left shift by negative count as right shifts
by negation of the count and similarly right shifts by negative count
as left shifts by negation (not ok), sets overflow for too large shift
count (ok), doesn't check for negative values on left shift (not ok)
and checks correctly for the non-representable ones otherwise (ok).
Now, C++11 to C++17 has different behavior, whereas in C99-C23 1 << 63
in preprocessor is invalid, in C++11-17 it is valid, but 3 << 63 is
not. The wording is that left shift of negative value is UB (like in C)
and signed non-negative left shift is UB if the result isn't representable
in corresponding unsigned type (so uintmax_t for libcpp).
And then C++20 and newer says all left shifts are well defined with the
exception of bad shift counts.
In -fsanitize=undefined we handle these by
/* For signed x << y, in C99 and later, the following:
(unsigned) x >> (uprecm1 - y)
if non-zero, is undefined. */
and
/* For signed x << y, in C++11 to C++17, the following:
x < 0 || ((unsigned) x >> (uprecm1 - y))
if > 1, is undefined. */
Now, we are late in GCC 15 development, so I think making the preprocessor
more strict than it is now is undesirable, so will defer setting overflow
flag for the shifts by negative count, or shifts by negative value left.
The following patch just makes some previously incorrectly rejected or
warned cases valid for C++11-17 and even more for C++20 and later.
2025-04-04 Jakub Jelinek <jakub@redhat.com>
PR preprocessor/119391
* expr.cc (num_lshift): Add pfile argument. Don't set num.overflow
for !num.unsignedp in C++20 or later unless n >= precision. For
C++11 to C++17 set it if orig >> (precision - 1 - n) as logical
shift results in value > 1.
(num_binary_op): Pass pfile to num_lshift.
(num_div_op): Likewise.
* g++.dg/cpp/pr119391.C: New test.
This patch addresses an issue in the C preprocessor where incorrect
line number information is generated when processing files with a
large number of lines. The problem arises from improper handling
of location intervals in the line map, particularly when locations
exceed LINE_MAP_MAX_LOCATION_WITH_PACKED_RANGES.
By ensuring that the highest location is not decremented if it
would move to a different ordinary map, this fix resolves
the line number discrepancies observed in certain test cases.
This change improves the accuracy of line number reporting, benefiting
users relying on precise code coverage and debugging information.
libcpp/ChangeLog:
PR preprocessor/108900
* files.cc (_cpp_stack_file): Do not decrement highest_location
across distinct maps.
Signed-off-by: Jeremy Bettis <jbettis@google.com>
Signed-off-by: Yash Shinde <Yash.Shinde@windriver.com>
In r15-4286 I've introduced a typo, part of the change was
- cpp_error (pfile, CPP_DL_ERROR, "'\\o' not followed by '{'");
+ cpp_error (pfile, CPP_DL_ERROR, "%<\\o%> not followed by %<}%>");
which turned { into }. This patch fixes it back.
2025-03-12 Jakub Jelinek <jakub@redhat.com>
PR preprocessor/119202
* charset.cc (convert_oct): Fix up typo in diagnostics about \o
not followed by {.
Now that the #embed paper has been voted in, the following patch
removes the pedwarn for C++26 on it (and adjusts pedwarn warning for
older C++ versions) and predefines __cpp_pp_embed FTM.
Also, the patch changes cpp_error to cpp_pedwarning with for C++
-Wc++26-extensions guarding, and for C add -Wc11-c23-compat warning
about #embed.
I believe we otherwise implement everything in the paper already,
except I'm really confused by the
[Example:
#embed <data.dat> limit(__has_include("a.h"))
#if __has_embed(<data.dat> limit(__has_include("a.h")))
// ill-formed: __has_include [cpp.cond] cannot appear here
#endif
— end example]
part. My reading of both C23 and C++ with the P1967R14 paper in
is that the first case (#embed with __has_include or __has_embed in its
clauses) is what is clearly invalid and so the ill-formed note should be
for #embed. And the __has_include/__has_embed in __has_embed is actually
questionable.
Both C and C++ have something like
"The identifiers __has_include, __has_embed, and __has_c_attribute
shall not appear in any context not mentioned in this subclause."
or
"The identifiers __has_include and __has_cpp_attribute shall not appear
in any context not mentioned in this subclause."
(into which P1967R14 adds __has_embed) in the conditional inclusion
subclause. #embed is defined in a different one, so using those in there
is invalid (unless "using the rules specified for conditional inclusion"
wording e.g. in limit clause overrides that).
The reason why I think it is fuzzy for __has_embed is that __has_embed
is actually defined in the Conditional inclusion subclause (so that
would mean one can use __has_include, __has_embed and __has_*attribute
in there) but its clauses are described in a different one.
GCC currently accepts
#embed __FILE__ limit (__has_include (<stdarg.h>))
#if __has_embed (__FILE__ limit (__has_include (<stdarg.h>)))
#endif
#embed __FILE__ limit (__has_embed (__FILE__))
#if __has_embed (__FILE__ limit (__has_embed (__FILE__)))
#endif
Note, it isn't just about limit clause, but also about
prefix/suffix/if_empty, except that in those cases the "using the rules
specified for conditional inclusion" doesn't apply.
In any case, I'd hope that can be dealt with incrementally (and should
be handled the same for both C and C++).
2025-02-28 Jakub Jelinek <jakub@redhat.com>
libcpp/
* include/cpplib.h (enum cpp_warning_reason): Add
CPP_W_CXX26_EXTENSIONS enumerator.
* init.cc (lang_defaults): Set embed for GNUCXX26 and CXX26.
* directives.cc (do_embed): Adjust pedwarn wording for embed in C++,
use cpp_pedwarning instead of cpp_error and add CPP_W_C11_C23_COMPAT
warning of cpp_pedwarning hasn't diagnosed anything.
gcc/c-family/
* c.opt (Wc++26-extensions): Add CppReason(CPP_W_CXX26_EXTENSIONS).
* c-cppbuiltin.cc (c_cpp_builtins): Predefine __cpp_pp_embed=202502
for C++26.
gcc/testsuite/
* g++.dg/cpp/embed-1.C: Adjust for pedwarn wording change and don't
expect any error for C++26.
* g++.dg/cpp/embed-2.C: Adjust for pedwarn wording change and don't
expect any warning for C++26.
* g++.dg/cpp26/feat-cxx26.C: Test __cpp_pp_embed value.
* gcc.dg/cpp/embed-17.c: New test.
It seems that tokens_buff_new() has always been allocating the virtual
location buffer 4 times larger than intended, and now that location_t is
64-bit, it is 8 times larger. Fixed.
libcpp/ChangeLog:
* macro.cc (tokens_buff_new): Fix length argument to XNEWVEC.
GCC Administrator