This patch converts the bidi::vec to use a struct so that we can
capture location_t values for the bidirectional control characters.
Before:
Wbidi-chars-1.c: In function ‘main’:
Wbidi-chars-1.c:6:43: warning: unpaired UTF-8 bidirectional control character detected [-Wbidi-chars=]
6 | /*<U+202E> } <U+2066>if (isAdmin)<U+2069> <U+2066> begin admins only */
| ^
Wbidi-chars-1.c:9:28: warning: unpaired UTF-8 bidirectional control character detected [-Wbidi-chars=]
9 | /* end admins only <U+202E> { <U+2066>*/
| ^
After:
Wbidi-chars-1.c: In function ‘main’:
Wbidi-chars-1.c:6:43: warning: unpaired UTF-8 bidirectional control characters detected [-Wbidi-chars=]
6 | /*<U+202E> } <U+2066>if (isAdmin)<U+2069> <U+2066> begin admins only */
| ~~~~~~~~ ~~~~~~~~ ^
| | | |
| | | end of bidirectional context
| U+202E (RIGHT-TO-LEFT OVERRIDE) U+2066 (LEFT-TO-RIGHT ISOLATE)
Wbidi-chars-1.c:9:28: warning: unpaired UTF-8 bidirectional control characters detected [-Wbidi-chars=]
9 | /* end admins only <U+202E> { <U+2066>*/
| ~~~~~~~~ ~~~~~~~~ ^
| | | |
| | | end of bidirectional context
| | U+2066 (LEFT-TO-RIGHT ISOLATE)
| U+202E (RIGHT-TO-LEFT OVERRIDE)
Signed-off-by: David Malcolm <dmalcolm@redhat.com>
gcc/testsuite/ChangeLog:
PR preprocessor/103026
* c-c++-common/Wbidi-chars-ranges.c: New test.
libcpp/ChangeLog:
PR preprocessor/103026
* lex.c (struct bidi::context): New.
(bidi::vec): Convert to a vec of context rather than unsigned
char.
(bidi::ctx_at): Rename to...
(bidi::pop_kind_at): ...this and reimplement for above change.
(bidi::current_ctx): Update for change to vec.
(bidi::current_ctx_ucn_p): Likewise.
(bidi::current_ctx_loc): New.
(bidi::on_char): Update for usage of context struct. Add "loc"
param and pass it when pushing contexts.
(get_location_for_byte_range_in_cur_line): New.
(get_bidi_utf8): Rename to...
(get_bidi_utf8_1): ...this, reintroducing...
(get_bidi_utf8): ...as a wrapper, setting *OUT when the result is
not NONE.
(get_bidi_ucn): Rename to...
(get_bidi_ucn_1): ...this, reintroducing...
(get_bidi_ucn): ...as a wrapper, setting *OUT when the result is
not NONE.
(class unpaired_bidi_rich_location): New.
(maybe_warn_bidi_on_close): Use unpaired_bidi_rich_location when
reporting on unpaired bidi chars. Split into singular vs plural
spellings.
(maybe_warn_bidi_on_char): Pass in a location_t rather than a
const uchar * and use it when emitting warnings, and when calling
bidi::on_char.
(_cpp_skip_block_comment): Capture location when kind is not NONE
and pass it to maybe_warn_bidi_on_char.
(skip_line_comment): Likewise.
(forms_identifier_p): Likewise.
(lex_raw_string): Likewise.
(lex_string): Likewise.
Signed-off-by: David Malcolm <dmalcolm@redhat.com>
This flags rich_locations associated with -Wbidi-chars= so that
non-ASCII bytes will be escaped when printing the source lines
(using the diagnostics support I added in
r12-4825-gbd5e882cf6e0def3dd1bc106075d59a303fe0d1e).
In particular, this ensures that the printed source lines will
be pure ASCII, and thus the visual ordering of the characters
will be the same as the logical ordering.
Before:
Wbidi-chars-1.c: In function ‘main’:
Wbidi-chars-1.c:6:43: warning: unpaired UTF-8 bidirectional control character detected [-Wbidi-chars=]
6 | /* } if (isAdmin) begin admins only */
| ^
Wbidi-chars-1.c:9:28: warning: unpaired UTF-8 bidirectional control character detected [-Wbidi-chars=]
9 | /* end admins only { */
| ^
Wbidi-chars-11.c:6:15: warning: UTF-8 vs UCN mismatch when closing a context by "U+202C (POP DIRECTIONAL FORMATTING)" [-Wbidi-chars=]
6 | int LRE__PDF_\u202c;
| ^
Wbidi-chars-11.c:8:19: warning: UTF-8 vs UCN mismatch when closing a context by "U+202C (POP DIRECTIONAL FORMATTING)" [-Wbidi-chars=]
8 | int LRE_\u202a_PDF__;
| ^
Wbidi-chars-11.c:10:28: warning: UTF-8 vs UCN mismatch when closing a context by "U+202C (POP DIRECTIONAL FORMATTING)" [-Wbidi-chars=]
10 | const char *s1 = "LRE__PDF_\u202c";
| ^
Wbidi-chars-11.c:12:33: warning: UTF-8 vs UCN mismatch when closing a context by "U+202C (POP DIRECTIONAL FORMATTING)" [-Wbidi-chars=]
12 | const char *s2 = "LRE_\u202a_PDF_";
| ^
After:
Wbidi-chars-1.c: In function ‘main’:
Wbidi-chars-1.c:6:43: warning: unpaired UTF-8 bidirectional control character detected [-Wbidi-chars=]
6 | /*<U+202E> } <U+2066>if (isAdmin)<U+2069> <U+2066> begin admins only */
| ^
Wbidi-chars-1.c:9:28: warning: unpaired UTF-8 bidirectional control character detected [-Wbidi-chars=]
9 | /* end admins only <U+202E> { <U+2066>*/
| ^
Wbidi-chars-11.c:6:15: warning: UTF-8 vs UCN mismatch when closing a context by "U+202C (POP DIRECTIONAL FORMATTING)" [-Wbidi-chars=]
6 | int LRE_<U+202A>_PDF_\u202c;
| ^
Wbidi-chars-11.c:8:19: warning: UTF-8 vs UCN mismatch when closing a context by "U+202C (POP DIRECTIONAL FORMATTING)" [-Wbidi-chars=]
8 | int LRE_\u202a_PDF_<U+202C>_;
| ^
Wbidi-chars-11.c:10:28: warning: UTF-8 vs UCN mismatch when closing a context by "U+202C (POP DIRECTIONAL FORMATTING)" [-Wbidi-chars=]
10 | const char *s1 = "LRE_<U+202A>_PDF_\u202c";
| ^
Wbidi-chars-11.c:12:33: warning: UTF-8 vs UCN mismatch when closing a context by "U+202C (POP DIRECTIONAL FORMATTING)" [-Wbidi-chars=]
12 | const char *s2 = "LRE_\u202a_PDF_<U+202C>";
| ^
libcpp/ChangeLog:
PR preprocessor/103026
* lex.c (maybe_warn_bidi_on_close): Use a rich_location
and call set_escape_on_output (true) on it.
(maybe_warn_bidi_on_char): Likewise.
Signed-off-by: David Malcolm <dmalcolm@redhat.com>
Normal preprocessing, -fdirectives-only preprocessing before the Nathan's
rewrite, and all other compilers I've tried on godbolt treat even \*/
as end of a block comment, but the new -fdirectives-only handling doesn't.
2021-11-17 Jakub Jelinek <jakub@redhat.com>
PR preprocessor/103130
* lex.c (cpp_directive_only_process): Treat even \*/ as end of block
comment.
* c-c++-common/cpp/dir-only-9.c: New test.
From a link below:
"An issue was discovered in the Bidirectional Algorithm in the Unicode
Specification through 14.0. It permits the visual reordering of
characters via control sequences, which can be used to craft source code
that renders different logic than the logical ordering of tokens
ingested by compilers and interpreters. Adversaries can leverage this to
encode source code for compilers accepting Unicode such that targeted
vulnerabilities are introduced invisibly to human reviewers."
More info:
https://nvd.nist.gov/vuln/detail/CVE-2021-42574https://trojansource.codes/
This is not a compiler bug. However, to mitigate the problem, this patch
implements -Wbidi-chars=[none|unpaired|any] to warn about possibly
misleading Unicode bidirectional control characters the preprocessor may
encounter.
The default is =unpaired, which warns about improperly terminated
bidirectional control characters; e.g. a LRE without its corresponding PDF.
The level =any warns about any use of bidirectional control characters.
This patch handles both UCNs and UTF-8 characters. UCNs designating
bidi characters in identifiers are accepted since r204886. Then r217144
enabled -fextended-identifiers by default. Extended characters in C/C++
identifiers have been accepted since r275979. However, this patch still
warns about mixing UTF-8 and UCN bidi characters; there seems to be no
good reason to allow mixing them.
We warn in different contexts: comments (both C and C++-style), string
literals, character constants, and identifiers. Expectedly, UCNs are ignored
in comments and raw string literals. The bidirectional control characters
can nest so this patch handles that as well.
I have not included nor tested this at all with Fortran (which also has
string literals and line comments).
Dave M. posted patches improving diagnostic involving Unicode characters.
This patch does not make use of this new infrastructure yet.
PR preprocessor/103026
gcc/c-family/ChangeLog:
* c.opt (Wbidi-chars, Wbidi-chars=): New option.
gcc/ChangeLog:
* doc/invoke.texi: Document -Wbidi-chars.
libcpp/ChangeLog:
* include/cpplib.h (enum cpp_bidirectional_level): New.
(struct cpp_options): Add cpp_warn_bidirectional.
(enum cpp_warning_reason): Add CPP_W_BIDIRECTIONAL.
* internal.h (struct cpp_reader): Add warn_bidi_p member
function.
* init.c (cpp_create_reader): Set cpp_warn_bidirectional.
* lex.c (bidi): New namespace.
(get_bidi_utf8): New function.
(get_bidi_ucn): Likewise.
(maybe_warn_bidi_on_close): Likewise.
(maybe_warn_bidi_on_char): Likewise.
(_cpp_skip_block_comment): Implement warning about bidirectional
control characters.
(skip_line_comment): Likewise.
(forms_identifier_p): Likewise.
(lex_identifier): Likewise.
(lex_string): Likewise.
(lex_raw_string): Likewise.
gcc/testsuite/ChangeLog:
* c-c++-common/Wbidi-chars-1.c: New test.
* c-c++-common/Wbidi-chars-2.c: New test.
* c-c++-common/Wbidi-chars-3.c: New test.
* c-c++-common/Wbidi-chars-4.c: New test.
* c-c++-common/Wbidi-chars-5.c: New test.
* c-c++-common/Wbidi-chars-6.c: New test.
* c-c++-common/Wbidi-chars-7.c: New test.
* c-c++-common/Wbidi-chars-8.c: New test.
* c-c++-common/Wbidi-chars-9.c: New test.
* c-c++-common/Wbidi-chars-10.c: New test.
* c-c++-common/Wbidi-chars-11.c: New test.
* c-c++-common/Wbidi-chars-12.c: New test.
* c-c++-common/Wbidi-chars-13.c: New test.
* c-c++-common/Wbidi-chars-14.c: New test.
* c-c++-common/Wbidi-chars-15.c: New test.
* c-c++-common/Wbidi-chars-16.c: New test.
* c-c++-common/Wbidi-chars-17.c: New test.
This patch adds support to GCC's diagnostic subsystem for escaping certain
bytes and Unicode characters when quoting source code.
Specifically, this patch adds a new flag rich_location::m_escape_on_output
which is a hint from a diagnostic that non-ASCII bytes in the pertinent
lines of the user's source code should be escaped when printed.
The patch sets this for the following diagnostics:
- when complaining about stray bytes in the program (when these
are non-printable)
- when complaining about "null character(s) ignored");
- for -Wnormalized= (and generate source ranges for such warnings)
The escaping is controlled by a new option:
-fdiagnostics-escape-format=[unicode|bytes]
For example, consider a diagnostic involing a source line containing the
string "before" followed by the Unicode character U+03C0 ("GREEK SMALL
LETTER PI", with UTF-8 encoding 0xCF 0x80) followed by the byte 0xBF
(a stray UTF-8 trailing byte), followed by the string "after", where the
diagnostic highlights the U+03C0 character.
By default, this line will be printed verbatim to the user when
reporting a diagnostic at it, as:
beforeπXafter
^
(using X for the stray byte to avoid putting invalid UTF-8 in this
commit message)
If the diagnostic sets the "escape" flag, it will be printed as:
before<U+03C0><BF>after
^~~~~~~~
with -fdiagnostics-escape-format=unicode (the default), or as:
before<CF><80><BF>after
^~~~~~~~
if the user supplies -fdiagnostics-escape-format=bytes.
This only affects how the source is printed; it does not affect
how column numbers that are printed (as per -fdiagnostics-column-unit=
and -fdiagnostics-column-origin=).
gcc/c-family/ChangeLog:
* c-lex.c (c_lex_with_flags): When complaining about non-printable
CPP_OTHER tokens, set the "escape on output" flag.
gcc/ChangeLog:
* common.opt (fdiagnostics-escape-format=): New.
(diagnostics_escape_format): New enum.
(DIAGNOSTICS_ESCAPE_FORMAT_UNICODE): New enum value.
(DIAGNOSTICS_ESCAPE_FORMAT_BYTES): Likewise.
* diagnostic-format-json.cc (json_end_diagnostic): Add
"escape-source" attribute.
* diagnostic-show-locus.c
(exploc_with_display_col::exploc_with_display_col): Replace
"tabstop" param with a cpp_char_column_policy and add an "aspect"
param. Use these to compute m_display_col accordingly.
(struct char_display_policy): New struct.
(layout::m_policy): New field.
(layout::m_escape_on_output): New field.
(def_policy): New function.
(make_range): Update for changes to exploc_with_display_col ctor.
(default_print_decoded_ch): New.
(width_per_escaped_byte): New.
(escape_as_bytes_width): New.
(escape_as_bytes_print): New.
(escape_as_unicode_width): New.
(escape_as_unicode_print): New.
(make_policy): New.
(layout::layout): Initialize new fields. Update m_exploc ctor
call for above change to ctor.
(layout::maybe_add_location_range): Update for changes to
exploc_with_display_col ctor.
(layout::calculate_x_offset_display): Update for change to
cpp_display_width.
(layout::print_source_line): Pass policy
to cpp_display_width_computation. Capture cpp_decoded_char when
calling process_next_codepoint. Move printing of source code to
m_policy.m_print_cb.
(line_label::line_label): Pass in policy rather than context.
(layout::print_any_labels): Update for change to line_label ctor.
(get_affected_range): Pass in policy rather than context, updating
calls to location_compute_display_column accordingly.
(get_printed_columns): Likewise, also for cpp_display_width.
(correction::correction): Pass in policy rather than tabstop.
(correction::compute_display_cols): Pass m_policy rather than
m_tabstop to cpp_display_width.
(correction::m_tabstop): Replace with...
(correction::m_policy): ...this.
(line_corrections::line_corrections): Pass in policy rather than
context.
(line_corrections::m_context): Replace with...
(line_corrections::m_policy): ...this.
(line_corrections::add_hint): Update to use m_policy rather than
m_context.
(line_corrections::add_hint): Likewise.
(layout::print_trailing_fixits): Likewise.
(selftest::test_display_widths): New.
(selftest::test_layout_x_offset_display_utf8): Update to use
policy rather than tabstop.
(selftest::test_one_liner_labels_utf8): Add test of escaping
source lines.
(selftest::test_diagnostic_show_locus_one_liner_utf8): Update to
use policy rather than tabstop.
(selftest::test_overlapped_fixit_printing): Likewise.
(selftest::test_overlapped_fixit_printing_utf8): Likewise.
(selftest::test_overlapped_fixit_printing_2): Likewise.
(selftest::test_tab_expansion): Likewise.
(selftest::test_escaping_bytes_1): New.
(selftest::test_escaping_bytes_2): New.
(selftest::diagnostic_show_locus_c_tests): Call the new tests.
* diagnostic.c (diagnostic_initialize): Initialize
context->escape_format.
(convert_column_unit): Update to use default character width policy.
(selftest::test_diagnostic_get_location_text): Likewise.
* diagnostic.h (enum diagnostics_escape_format): New enum.
(diagnostic_context::escape_format): New field.
* doc/invoke.texi (-fdiagnostics-escape-format=): New option.
(-fdiagnostics-format=): Add "escape-source" attribute to examples
of JSON output, and document it.
* input.c (location_compute_display_column): Pass in "policy"
rather than "tabstop", passing to
cpp_byte_column_to_display_column.
(selftest::test_cpp_utf8): Update to use cpp_char_column_policy.
* input.h (class cpp_char_column_policy): New forward decl.
(location_compute_display_column): Pass in "policy" rather than
"tabstop".
* opts.c (common_handle_option): Handle
OPT_fdiagnostics_escape_format_.
* selftest.c (temp_source_file::temp_source_file): New ctor
overload taking a size_t.
* selftest.h (temp_source_file::temp_source_file): Likewise.
gcc/testsuite/ChangeLog:
* c-c++-common/diagnostic-format-json-1.c: Add regexp to consume
"escape-source" attribute.
* c-c++-common/diagnostic-format-json-2.c: Likewise.
* c-c++-common/diagnostic-format-json-3.c: Likewise.
* c-c++-common/diagnostic-format-json-4.c: Likewise, twice.
* c-c++-common/diagnostic-format-json-5.c: Likewise.
* gcc.dg/cpp/warn-normalized-4-bytes.c: New test.
* gcc.dg/cpp/warn-normalized-4-unicode.c: New test.
* gcc.dg/encoding-issues-bytes.c: New test.
* gcc.dg/encoding-issues-unicode.c: New test.
* gfortran.dg/diagnostic-format-json-1.F90: Add regexp to consume
"escape-source" attribute.
* gfortran.dg/diagnostic-format-json-2.F90: Likewise.
* gfortran.dg/diagnostic-format-json-3.F90: Likewise.
libcpp/ChangeLog:
* charset.c (convert_escape): Use encoding_rich_location when
complaining about nonprintable unknown escape sequences.
(cpp_display_width_computation::::cpp_display_width_computation):
Pass in policy rather than tabstop.
(cpp_display_width_computation::process_next_codepoint): Add "out"
param and populate *out if non-NULL.
(cpp_display_width_computation::advance_display_cols): Pass NULL
to process_next_codepoint.
(cpp_byte_column_to_display_column): Pass in policy rather than
tabstop. Pass NULL to process_next_codepoint.
(cpp_display_column_to_byte_column): Pass in policy rather than
tabstop.
* errors.c (cpp_diagnostic_get_current_location): New function,
splitting out the logic from...
(cpp_diagnostic): ...here.
(cpp_warning_at): New function.
(cpp_pedwarning_at): New function.
* include/cpplib.h (cpp_warning_at): New decl for rich_location.
(cpp_pedwarning_at): Likewise.
(struct cpp_decoded_char): New.
(struct cpp_char_column_policy): New.
(cpp_display_width_computation::cpp_display_width_computation):
Replace "tabstop" param with "policy".
(cpp_display_width_computation::process_next_codepoint): Add "out"
param.
(cpp_display_width_computation::m_tabstop): Replace with...
(cpp_display_width_computation::m_policy): ...this.
(cpp_byte_column_to_display_column): Replace "tabstop" param with
"policy".
(cpp_display_width): Likewise.
(cpp_display_column_to_byte_column): Likewise.
* include/line-map.h (rich_location::escape_on_output_p): New.
(rich_location::set_escape_on_output): New.
(rich_location::m_escape_on_output): New.
* internal.h (cpp_diagnostic_get_current_location): New decl.
(class encoding_rich_location): New.
* lex.c (skip_whitespace): Use encoding_rich_location when
complaining about null characters.
(warn_about_normalization): Generate a source range when
complaining about improperly normalized tokens, rather than just a
point, and use encoding_rich_location so that the source code
is escaped on printing.
* line-map.c (rich_location::rich_location): Initialize
m_escape_on_output.
Signed-off-by: David Malcolm <dmalcolm@redhat.com>
Both #pragma and _Pragma ended up as CPP_PRAGMA. Presumably since
r131819 (2008, GCC 4.3) for PR34692, pragmas are not expanded in
macro arguments but are output as is before. From the old bug report,
that was to fix usage like
FOO (
#pragma GCC diagnostic
)
However, that change also affected _Pragma such that
BAR (
"1";
_Pragma("omp ..."); )
yielded
#pragma omp ...
followed by what BAR expanded too, possibly including '"1";'.
This commit adds a flag, PRAGMA_OP, to tokens to make the two
distinguishable - and include again _Pragma in the expanded arguments.
libcpp/ChangeLog:
PR c++/102409
* directives.c (destringize_and_run): Add PRAGMA_OP to the
CPP_PRAGMA token's flags to mark is as coming from _Pragma.
* include/cpplib.h (PRAGMA_OP): #define, to be used with token flags.
* macro.c (collect_args): Only handle CPP_PRAGMA special if PRAGMA_OP
is set.
gcc/testsuite/ChangeLog:
* c-c++-common/gomp/pragma-1.c: New test.
* c-c++-common/gomp/pragma-2.c: New test.
This patch implements C++23 P2334R1, which is easy because Joseph has done
all the hard work for C2X already.
Unlike the C N2645 paper, the C++ P2334R1 contains one important addition
(but not in the normative text):
"While this is a new preprocessor feature and cannot be treated as a defect
report, implementations that support older versions of the standard are
encouraged to implement this feature in the older language modes as well
as C++23."
so there are different variants how to implement it.
One is ignoring that sentence and only implementing it
for -std=c++23/-std=gnu++23 like it is only implemented for -std=c2x.
Another option would be to implement it also in the older GNU modes but
not in the C/CXX modes (but it would be strange if we did that just for
C++ and not for C).
Yet another option is to enable it unconditionally.
And yet another option would be to enable it unconditionally but emit
a warning (or pedwarn) when it is seen.
Note, when it is enabled for the older language modes, as Joseph wrote
in the c11-elifdef-1.c testcase, it can result e.g. in rejecting previously
valid code:
#define A
#undef B
#if 0
#elifdef A
#error "#elifdef A applied"
#endif
#if 0
#elifndef B
#error "#elifndef B applied"
#endif
Note, seems clang went the enable it unconditionally in all standard
versions of both C and C++, no warnings or anything whatsoever, so
essentially treated it as a DR that changed behavior of e.g. the above code.
After feedback, this option enables #elifdef/#elifndef for -std=c2x
and -std=c++2{b,3} and enables it also for -std=gnu*, but for GNU modes
older than C2X or C++23 if -pedantic it emits a pedwarn on the directives
that either would be rejected in the corresponding -std=c* modes, e.g.
#if 1
#elifdef A // pedwarn if -pedantic
#endif
or when the directives would be silently accepted, but when they are
recognized it changes behavior, so e.g.
#define A
#if 0
#elifdef A // pedwarn if -pedantic
#define M 1
#endif
It won't pedwarn if the directives would be silently ignored and wouldn't
change anything, like:
#define A
#if 0
#elifndef A
#define M 1
#endif
or
#undef B
#if 0
#elifdef B
#define M 1
#endif
2021-10-06 Jakub Jelinek <jakub@redhat.com>
libcpp/
* init.c (lang_defaults): Implement P2334R1, enable elifdef for
-std=c++23 and -std=gnu++23.
* directives.c (_cpp_handle_directive): Support elifdef/elifndef if
either CPP_OPTION (pfile, elifdef) or !CPP_OPTION (pfile, std).
(do_elif): For older non-std modes if pedantic pedwarn about
#elifdef/#elifndef directives that change behavior.
gcc/testsuite/
* gcc.dg/cpp/gnu11-elifdef-1.c: New test.
* gcc.dg/cpp/gnu11-elifdef-2.c: New test.
* gcc.dg/cpp/gnu11-elifdef-3.c: New test.
* gcc.dg/cpp/gnu11-elifdef-4.c: New test.
* g++.dg/cpp/elifdef-1.C: New test.
* g++.dg/cpp/elifdef-2.C: New test.
* g++.dg/cpp/elifdef-3.C: New test.
* g++.dg/cpp/elifdef-4.C: New test.
* g++.dg/cpp/elifdef-5.C: New test.
* g++.dg/cpp/elifdef-6.C: New test.
* g++.dg/cpp/elifdef-7.C: New test.
The following patch implements the
P1949R7 - C++ Identifier Syntax using Unicode Standard Annex 31
paper. We already allow UTF-8 characters in the source, so that part
is already implemented, so IMHO all we need to do is pedwarn instead of
just warn for the (default) -Wnormalize=nfc (or for -Wnormalize={id,nkfc})
if the character is not in NFC and to use the unicode XID_Start and
XID_Continue derived code properties to find out what characters are allowed
(the standard actually adds U+005F to XID_Start, but we are handling the
ASCII compatible characters differently already and they aren't allowed
in UCNs in identifiers). Instead of hardcoding the large tables
in ucnid.tab, this patch makes makeucnid.c read them from the Unicode
tables (13.0.0 version at this point).
For non-pedantic mode, we accept as 2nd+ char in identifiers a union
of valid characters in all supported modes, but for the 1st char it
was actually pedantically requiring that it is not any of the characters
that may not appear in the currently chosen standard as the first character.
This patch changes it such that also what is allowed at the start of an
identifier is a union of characters valid at the start of an identifier
in any of the pedantic modes.
2021-09-01 Jakub Jelinek <jakub@redhat.com>
PR c++/100977
libcpp/
* include/cpplib.h (struct cpp_options): Add cxx23_identifiers.
* charset.c (CXX23, NXX23): New enumerators.
(CID, NFC, NKC, CTX): Renumber.
(ucn_valid_in_identifier): Implement P1949R7 - use CXX23 and
NXX23 flags for cxx23_identifiers. For start character in
non-pedantic mode, allow characters that are allowed as start
characters in any of the supported language modes, rather than
disallowing characters allowed only as non-start characters in
current mode but for characters from other language modes allowing
them even if they are never allowed at start.
* init.c (struct lang_flags): Add cxx23_identifiers.
(lang_defaults): Add cxx23_identifiers column.
(cpp_set_lang): Initialize CPP_OPTION (pfile, cxx23_identifiers).
* lex.c (warn_about_normalization): If cxx23_identifiers, use
cpp_pedwarning_with_line instead of cpp_warning_with_line for
"is not in NFC" diagnostics.
* makeucnid.c: Adjust usage comment.
(CXX23, NXX23): New enumerators.
(all_languages): Add CXX23.
(not_NFC, not_NFKC, maybe_not_NFC): Renumber.
(read_derivedcore): New function.
(write_table): Print also CXX23 and NXX23 columns.
(main): Require 5 arguments instead of 4, call read_derivedcore.
* ucnid.h: Regenerated using Unicode 13.0.0 files.
gcc/testsuite/
* g++.dg/cpp23/normalize1.C: New test.
* g++.dg/cpp23/normalize2.C: New test.
* g++.dg/cpp23/normalize3.C: New test.
* g++.dg/cpp23/normalize4.C: New test.
* g++.dg/cpp23/normalize5.C: New test.
* g++.dg/cpp23/normalize6.C: New test.
* g++.dg/cpp23/normalize7.C: New test.
* g++.dg/cpp23/ucnid-1-utf8.C: New test.
* g++.dg/cpp23/ucnid-2-utf8.C: New test.
* gcc.dg/cpp/ucnid-4.c: Don't expect
"not valid at the start of an identifier" errors.
* gcc.dg/cpp/ucnid-4-utf8.c: Likewise.
* gcc.dg/cpp/ucnid-5-utf8.c: New test.
> We want to remove the latter <placemarker> but not the former one, and
> the patch adds the vaopt_padding_tokens counter for it to control
> how many placemarkers are removed on vaopt_state::END.
> As can be seen in #c1 and #c2 of the PR, I've tried various approaches,
> but neither worked out for all the cases except the posted one.
I notice that the second placemarker you mention is avoid_paste, which seems
relevant. This seems to also work, at least it doesn't seem to break any of
the va_opt tests.
2021-09-01 Jason Merrill <jason@redhat.com>
* macro.c (replace_args): When __VA_OPT__ is on the LHS of ##,
remove trailing avoid_paste tokens.
So, besides missing #__VA_OPT__ patch for which I've posted patch last week,
P1042R1 introduced some placemarker changes for __VA_OPT__, most notably
the addition of before "removal of placemarker tokens," rescanning ...
and the
#define H4(X, ...) __VA_OPT__(a X ## X) ## b
H4(, 1) // replaced by a b
example mentioned there where we replace it currently with ab
The following patch are the minimum changes (except for the
__builtin_expect) that achieve the same preprocessing between current
clang++ and patched gcc on all the testcases I've tried (i.e. gcc __VA_OPT__
testsuite in c-c++-common/cpp/va-opt* including the new test and the clang
clang/test/Preprocessor/macro_va_opt* testcases).
At one point I was trying to implement the __VA_OPT__(args) case as if
for non-empty __VA_ARGS__ it expanded as if __VA_OPT__( and ) were missing,
but from the tests it seems that is not how it should work, in particular
if after (or before) we have some macro argument and it is not followed
(or preceded) by ##, then it should be macro expanded even when __VA_OPT__
is after ## or ) is followed by ##. And it seems that not removing any
padding tokens isn't possible either, because the expansion of the arguments
typically has a padding token at the start and end and those at least
according to the testsuite need to go. It is unclear if it would be enough
to remove just one or if all padding tokens should be removed.
Anyway, e.g. the previous removal of all padding tokens at the end of
__VA_OPT__ is undesirable, as it e.g. eats also the padding tokens needed
for the H4 example from the paper.
2021-09-01 Jakub Jelinek <jakub@redhat.com>
PR preprocessor/101488
* macro.c (replace_args): Fix up handling of CPP_PADDING tokens at the
start or end of __VA_OPT__ arguments when preceeded or followed by ##.
* c-c++-common/cpp/va-opt-3.c: Adjust expected output.
* c-c++-common/cpp/va-opt-7.c: New test.
Adds the logic to handle -finput-charset in layout_get_source_line(), so that
source lines are converted from their input encodings prior to being output by
diagnostics machinery. Also adds the ability to strip a UTF-8 BOM similarly.
gcc/c-family/ChangeLog:
PR other/93067
* c-opts.c (c_common_input_charset_cb): New function.
(c_common_post_options): Call new function
diagnostic_initialize_input_context().
gcc/d/ChangeLog:
PR other/93067
* d-lang.cc (d_input_charset_callback): New function.
(d_init): Call new function
diagnostic_initialize_input_context().
gcc/fortran/ChangeLog:
PR other/93067
* cpp.c (gfc_cpp_post_options): Call new function
diagnostic_initialize_input_context().
gcc/ChangeLog:
PR other/93067
* coretypes.h (typedef diagnostic_input_charset_callback): Declare.
* diagnostic.c (diagnostic_initialize_input_context): New function.
* diagnostic.h (diagnostic_initialize_input_context): Declare.
* input.c (default_charset_callback): New function.
(file_cache::initialize_input_context): New function.
(file_cache_slot::create): Added ability to convert the input
according to the input context.
(file_cache::file_cache): Initialize the new input context.
(class file_cache_slot): Added new m_alloc_offset member.
(file_cache_slot::file_cache_slot): Initialize the new member.
(file_cache_slot::~file_cache_slot): Handle potentially offset buffer.
(file_cache_slot::maybe_grow): Likewise.
(file_cache_slot::needs_read_p): Handle NULL fp, which is now possible.
(file_cache_slot::get_next_line): Likewise.
* input.h (class file_cache): Added input context member.
libcpp/ChangeLog:
PR other/93067
* charset.c (init_iconv_desc): Adapt to permit PFILE argument to
be NULL.
(_cpp_convert_input): Likewise. Also move UTF-8 BOM logic to...
(cpp_check_utf8_bom): ...here. New function.
(cpp_input_conversion_is_trivial): New function.
* files.c (read_file_guts): Allow PFILE argument to be NULL. Add
INPUT_CHARSET argument as an alternate source of this information.
(read_file): Pass the new argument to read_file_guts.
(cpp_get_converted_source): New function.
* include/cpplib.h (struct cpp_converted_source): Declare.
(cpp_get_converted_source): Declare.
(cpp_input_conversion_is_trivial): Declare.
(cpp_check_utf8_bom): Declare.
gcc/testsuite/ChangeLog:
PR other/93067
* gcc.dg/diagnostic-input-charset-1.c: New test.
* gcc.dg/diagnostic-input-utf8-bom.c: New test.
The following patch implements C++20 # __VA_OPT__ (...) support.
Testcases cover what I came up with myself and what LLVM has for #__VA_OPT__
in its testsuite and the string literals are identical between the two
compilers on the va-opt-5.c testcase.
2021-08-17 Jakub Jelinek <jakub@redhat.com>
libcpp/
* macro.c (vaopt_state): Add m_stringify member.
(vaopt_state::vaopt_state): Initialize it.
(vaopt_state::update): Overwrite it.
(vaopt_state::stringify): New method.
(stringify_arg): Replace arg argument with first, count arguments
and add va_opt argument. Use first instead of arg->first and
count instead of arg->count, for va_opt add paste_tokens handling.
(paste_tokens): Fix up len calculation. Don't spell rhs twice,
instead use %.*s to supply lhs and rhs spelling lengths. Don't call
_cpp_backup_tokens here.
(paste_all_tokens): Call it here instead.
(replace_args): Adjust stringify_arg caller. For vaopt_state::END
if stringify is true handle __VA_OPT__ stringification.
(create_iso_definition): Handle # __VA_OPT__ similarly to # macro_arg.
gcc/testsuite/
* c-c++-common/cpp/va-opt-5.c: New test.
* c-c++-common/cpp/va-opt-6.c: New test.
The following testcase ICEs in cpp_sys_macro_p, because cpp_sys_macro_p
is called for a builtin macro which doesn't use node->value.macro union
member but a different one and so dereferencing it ICEs.
As the testcase is distilled from contemporary glibc headers, it means
basically -Wtraditional now ICEs on almost everything.
The fix can be either the patch below, return true for builtin macros,
or we could instead return false for builtin macros, or the fix could
be also (untested):
--- libcpp/expr.c 2021-05-07 10:34:46.345122608 +0200
+++ libcpp/expr.c 2021-08-12 09:54:01.837556365 +0200
@@ -783,13 +783,13 @@ cpp_classify_number (cpp_reader *pfile,
/* Traditional C only accepted the 'L' suffix.
Suppress warning about 'LL' with -Wno-long-long. */
- if (CPP_WTRADITIONAL (pfile) && ! cpp_sys_macro_p (pfile))
+ if (CPP_WTRADITIONAL (pfile))
{
int u_or_i = (result & (CPP_N_UNSIGNED|CPP_N_IMAGINARY));
int large = (result & CPP_N_WIDTH) == CPP_N_LARGE
&& CPP_OPTION (pfile, cpp_warn_long_long);
- if (u_or_i || large)
+ if ((u_or_i || large) && ! cpp_sys_macro_p (pfile))
cpp_warning_with_line (pfile, large ? CPP_W_LONG_LONG : CPP_W_TRADITIONAL,
virtual_location, 0,
"traditional C rejects the \"%.*s\" suffix",
The builtin macros at least currently don't add any suffixes
or numbers -Wtraditional would like to warn about. For floating
point suffixes, -Wtraditional calls cpp_sys_macro_p only right
away before emitting the warning, but in the above case the ICE
is because cpp_sys_macro_p is called even if the number doesn't
have any suffixes (that is I think always for builtin macros
right now).
2021-08-12 Jakub Jelinek <jakub@redhat.com>
PR preprocessor/101638
* macro.c (cpp_sys_macro_p): Return true instead of
crashing on builtin macros.
* gcc.dg/cpp/pr101638.c: New test.
The following patch (incremental to the makeucnid.c fix) regenerates
ucnid.h with https://www.unicode.org/Public/13.0.0/ucd/ files.
2021-08-05 Jakub Jelinek <jakub@redhat.com>
PR c++/100977
* ucnid.h: Regenerated using Unicode 13.0.0 files.
I've noticed in ucnid.h two adjacent lines that had all flags and combine
values identical and as such were supposed to be merged.
This is due to a bug in makeucnid.c, which records last_flag,
last_combine and really_safe of what has just been printed, but
because of a typo mishandles it for last_combine, always compares against
the combining_value[0] which is 0.
This has two effects on the table, one is that often the table is
unnecessarily large, as for non-zero .combine every character has its own
record instead of adjacent characters with the same flags and combine
being merged. This means larger tables.
The other is that sometimes the last char that has combine set doesn't
actually have it in the tables, because the code is printing entries only
upon seeing the next character and if that character does have
combining_value of 0 and flags are otherwise the same as previously printed,
it will not print anything.
The following patch fixes that, for clarity what exactly it affects
I've regenerated with the same Unicode files as last time it has
been regenerated.
2021-08-05 Jakub Jelinek <jakub@redhat.com>
PR c++/100977
* makeucnid.c (write_table): Fix computation of last_combine.
* ucnid.h: Regenerated using Unicode 6.3.0 files.
The patch for 96391 changed linemap_compare_locations to give up on
comparing locations from macro expansions if we don't have column
information. But in this testcase, the BOILERPLATE macro is multiple lines
long, so we do want to compare locations within the macro. So this patch
moves the LINE_MAP_MAX_LOCATION_WITH_COLS check inside the block, to use it
for failing gracefully.
PR c++/100796
PR preprocessor/96391
libcpp/ChangeLog:
* line-map.c (linemap_compare_locations): Only use comparison with
LINE_MAP_MAX_LOCATION_WITH_COLS to avoid abort.
gcc/testsuite/ChangeLog:
* g++.dg/plugin/location-overflow-test-pr100796.c: New test.
* g++.dg/plugin/plugin.exp: Run it.
The toolchain provided by ST for stm32 has had support for
__FILENAME__ for a while, but clang/llvm has recently implemented
support for __FILE_NAME__, so it seems better to use the same macro
name in GCC.
It happens that the ST patch is similar to the one proposed in PR
c/42579.
Given these input files:
::::::::::::::
mydir/myinc.h
::::::::::::::
char* mystringh_file = __FILE__;
char* mystringh_filename = __FILE_NAME__;
char* mystringh_base_file = __BASE_FILE__;
::::::::::::::
mydir/mysrc.c
::::::::::::::
char* mystring_file = __FILE__;
char* mystring_filename = __FILE_NAME__;
char* mystring_base_file = __BASE_FILE__;
we produce:
$ gcc mydir/mysrc.c -I . -E
char* mystringh_file = "./mydir/myinc.h";
char* mystringh_filename = "myinc.h";
char* mystringh_base_file = "mydir/mysrc.c";
char* mystring_file = "mydir/mysrc.c";
char* mystring_filename = "mysrc.c";
char* mystring_base_file = "mydir/mysrc.c";
2021-05-20 Christophe Lyon <christophe.lyon@linaro.org>
Torbjörn Svensson <torbjorn.svensson@st.com>
PR c/42579
libcpp/
* include/cpplib.h (cpp_builtin_type): Add BT_FILE_NAME entry.
* init.c (builtin_array): Likewise.
* macro.c (_cpp_builtin_macro_text): Add support for BT_FILE_NAME.
gcc/
* doc/cpp.texi (Common Predefined Macros): Document __FILE_NAME__.
gcc/testsuite/
* c-c++-common/spellcheck-reserved.c: Add tests for __FILE_NAME__.
* c-c++-common/cpp/file-name-1.c: New test.
As can be seen on the testcases, before the -fdirectives-only preprocessing
rewrite the preprocessor would assume // comments are terminated by the
end of file even when newline wasn't there, but now we error out.
The following patch restores the previous behavior.
2021-05-20 Jakub Jelinek <jakub@redhat.com>
PR preprocessor/100646
* lex.c (cpp_directive_only_process): Treat end of file as termination
for !is_block comments.
* gcc.dg/cpp/pr100646-1.c: New test.
* gcc.dg/cpp/pr100646-2.c: New test.
If a header doesn't end with a new-line, with -fdirectives-only we right now
preprocess it as
int i = 1;# 2 "pr100392.c" 2
i.e. the line directive isn't on the next line, which means we fail to parse
it when compiling.
GCC 10 and earlier libcpp/directives-only.c had for this:
if (!pfile->state.skipping && cur != base)
{
/* If the file was not newline terminated, add rlimit, which is
guaranteed to point to a newline, to the end of our range. */
if (cur[-1] != '\n')
{
cur++;
CPP_INCREMENT_LINE (pfile, 0);
lines++;
}
cb->print_lines (lines, base, cur - base);
}
and we have the assertion
/* Files always end in a newline or carriage return. We rely on this for
character peeking safety. */
gcc_assert (buffer->rlimit[0] == '\n' || buffer->rlimit[0] == '\r');
So, this patch just does readd the more less same thing, so that we emit
a newline after the inline even when it wasn't there before.
2021-05-12 Jakub Jelinek <jakub@redhat.com>
PR preprocessor/100392
* lex.c (cpp_directive_only_process): If buffer doesn't end with '\n',
add buffer->rlimit[0] character to the printed range and
CPP_INCREMENT_LINE and increment line_count.
* gcc.dg/cpp/pr100392.c: New test.
* gcc.dg/cpp/pr100392.h: New file.
C2X adds #elifdef and #elifndef preprocessor directives; these have
also been proposed for C++. Implement these directives in libcpp
accordingly.
In this implementation, #elifdef and #elifndef are treated as
non-directives for any language version other than c2x and gnu2x (if
the feature is accepted for C++, it can trivially be enabled for
relevant C++ versions). In strict conformance modes for prior
language versions, this is required, as illustrated by the
c11-elifdef-1.c test added.
Bootstrapped with no regressions for x86_64-pc-linux-gnu.
libcpp/
* include/cpplib.h (struct cpp_options): Add elifdef.
* init.c (struct lang_flags): Add elifdef.
(lang_defaults): Update to include elifdef initializers.
(cpp_set_lang): Set elifdef for pfile based on language.
* directives.c (STDC2X, ELIFDEF): New macros.
(EXTENSION): Increase value to 3.
(DIRECTIVE_TABLE): Add #elifdef and #elifndef.
(_cpp_handle_directive): Do not treat ELIFDEF directives as
directives for language versions without the #elifdef feature.
(do_elif): Handle #elifdef and #elifndef.
(do_elifdef, do_elifndef): New functions.
gcc/testsuite/
* gcc.dg/cpp/c11-elifdef-1.c, gcc.dg/cpp/c2x-elifdef-1.c,
gcc.dg/cpp/c2x-elifdef-2.c: New tests.
The libcpp function cpp_avoid_paste is used to insert whitespace in
preprocessed output where needed to avoid two consecutive
preprocessing tokens, that logically (e.g. when stringized) do not
have whitespace between them, from being incorrectly lexed as one when
the preprocessed input is reread by a compiler.
This fails to allow for digit separators, so meaning that invalid
code, that has a CPP_NUMBER (from a macro expansion) followed by a
character literal, can result in preprocessed output with a valid use
of digit separators, so that required syntax errors do not occur when
compiling with -save-temps. Fix this by handling that case in
cpp_avoid_paste (as with other cases in cpp_avoid_paste, this doesn't
try to check whether the language version in use supports digit
separators; it's always OK to have unnecessary whitespace in
preprocessed output).
Note: there are other cases, with various kinds of wide character or
string literal following a CPP_NUMBER, where spurious pasting of
preprocessing tokens can occur but the sequence of tokens remains
invalid both before and after that pasting. Maybe cpp_avoid_paste
should also handle those cases (and similar cases after a CPP_NAME),
to ensure the sequence of preprocessing tokens in preprocessed output
is exactly right, whether or not it affects whether syntax errors
occur. This patch only addresses the case with digit separators where
invalid code can fail to be diagnosed without the space inserted.
Bootstrapped with no regressions for x86_64-pc-linux-gnu.
libcpp/
* lex.c (cpp_avoid_paste): Do not allow pasting CPP_NUMBER with
CPP_CHAR.
gcc/testsuite/
* g++.dg/cpp1y/digit-sep-paste.C, gcc.dg/c2x-digit-separators-3.c:
New tests.
C2X adds digit separators, as in C++. Enable them accordingly in
libcpp and c-lex.c. Some basic tests are added that digit separators
behave as expected for C2X and are properly disabled for C11; further
test coverage is included in the existing g++.dg/cpp1y/digit-sep*.C
tests.
Bootstrapped with no regressions for x86_64-pc-linux-gnu.
gcc/c-family/
* c-lex.c (interpret_float): Handle digit separators for C2X.
libcpp/
* init.c (lang_defaults): Enable digit separators for GNUC2X and
STDC2X.
gcc/testsuite/
* gcc.dg/c11-digit-separators-1.c,
gcc.dg/c2x-digit-separators-1.c, gcc.dg/c2x-digit-separators-2.c:
New tests.
Since the r0-85991-ga25a8f3be322fe0f838947b679f73d6efc2a412c
https://gcc.gnu.org/legacy-ml/gcc-patches/2008-02/msg01329.html
changes, so that we handle macros inside of pragmas that should expand
macros, during preprocessing we print those pragmas token by token,
with CPP_PRAGMA printed as
fputs ("#pragma ", print.outf);
if (space)
fprintf (print.outf, "%s %s", space, name);
else
fprintf (print.outf, "%s", name);
where name is some identifier (so e.g. print
#pragma omp parallel
or
#pragma omp for
etc.). Because it ends in an identifier, we need to handle it like
an identifier (i.e. CPP_NAME) for the decision whether a space needs
to be emitted in between that #pragma whatever or #pragma whatever whatever
and following token, otherwise the attached testcase is preprocessed as
#pragma omp forreduction(+:red)
rather than
#pragma omp for reduction(+:red)
The cpp_avoid_paste function is only called for this purpose.
2021-05-07 Jakub Jelinek <jakub@redhat.com>
PR c/100450
* lex.c (cpp_avoid_paste): Handle token1 CPP_PRAGMA like CPP_NAME.
* c-c++-common/gomp/pr100450.c: New test.
When the preprocessor lexes preprocessing numbers in lex_number, it
accepts digit separators in more cases than actually permitted in
pp-numbers by the standard syntax.
One thing this accepts is adjacent digit separators; there is some
code to reject those later, but as noted in bug 83873 it fails to
cover the case of adjacent digit separators within a floating-point
exponent. Accepting adjacent digit separators only results in a
missing diagnostic, not in valid code being rejected or being accepted
with incorrect semantics, because the correct lexing in such a case
would have '' start the following preprocessing tokens, and no valid
preprocessing token starts '' while ' isn't valid on its own as a
preprocessing token either. So this patch fixes that case by moving
the error for adjacent digit separators to lex_number (allowing a more
specific diagnostic than if '' were excluded from the pp-number
completely).
Other cases inappropriately accepted involve digit separators before
'.', 'e+', 'e-', 'p+' or 'p-' (or corresponding uppercase variants).
In those cases, as shown by the test digit-sep-pp-number.C added, this
can result in valid code being wrongly rejected as a result of too
many characters being included in the pp-number. So this case is
fixed by terminating the pp-number at the correct character according
to the standard. That test also covers the case where a digit
separator was followed by an identifier-nondigit that is not a
nondigit (e.g. a UCN); that case was already handled correctly.
Bootstrapped with no regressions for x86_64-pc-linux-gnu.
libcpp/
PR c++/83873
PR preprocessor/97604
* lex.c (lex_number): Reject adjacent digit separators here. Do
not allow digit separators before '.' or an exponent with sign.
* expr.c (cpp_classify_number): Do not check for adjacent digit
separators here.
gcc/testsuite/
PR c++/83873
PR preprocessor/97604
* g++.dg/cpp1y/digit-sep-neg-2.C,
g++.dg/cpp1y/digit-sep-pp-number.C: New tests.
* g++.dg/cpp1y/digit-sep-line-neg.C, g++.dg/cpp1y/digit-sep-neg.C:
Adjust expected messages.
As reported in bug 82359, the preprocessor does not allow C++ digit
separators in the line number in a #line directive, despite the
standard syntax for that directive using digit-sequence which allows
digit separators.
There is some confusion in that bug about whether C++ is meant to
allow digit separators there or not, but the last comment there
suggests they are meant to be allowed, and the version of digit
separators accepted for C2X at the March meeting explicitly mentions
digit separators in the #line specification to avoid any ambiguity
there.
This patch thus adds code to handle digit separators in the line
number in #line, as part of the preparation for enabling digit
separators in C2X mode. The code changed does not contain any
conditionals for whether digit separators are supported in the chosen
language version, because that was handled earlier in pp-number lexing
and if they aren't supported they won't appear in the string passed to
that function. It does however make sure not to allow adjacent digit
separators because those are only handled at a later stage of lexing
at present. (Problems with how certain source character sequences
involving digit separators that don't actually match the pp-number
syntax get lexed as a pp-number and only diagnosed later, if at all,
are bugs 83873 and 97604, to be addressed separately.)
Making the change in this location will have the effect of allowing
digit separators in the "# <line-number> <file> <flags>" form of
directive as well as #line; I don't think that's a problem.
Bootstrapped with no regressions for x86_64-pc-linux-gnu.
libcpp/
PR preprocessor/82359
* directives.c (strtolinenum): Handle digit separators.
gcc/testsuite/
PR preprocessor/82359
* g++.dg/cpp1y/digit-sep-line.C,
g++.dg/cpp1y/digit-sep-line-neg.C: New tests.
This reverts a s/column_offset/column/ change in the fix for PR99446.
2021-04-19 Richard Biener <rguenther@suse.de>
PR preprocessor/100142
libcpp/
* line-map.c (linemap_position_for_loc_and_offset): Revert
unintended s/column_offset/column/ change.
gcc/testsuite/
* gcc.dg/pr100142.c: New testcase.
* g++.dg/diagnostic/pr72803.C: Revert last change.
This ICE was because when adjusting a column offset we could advance
into a linemap for a different file. We only checked the next line
map was not for a line further advanced in any file, forgetting that
it could be for an earlier line in a different file. The testcase
needed adjusting as column 512 was unrepresentable, once that was
taken into consideration.
PR preprocessor/99446
libcpp/
* line-map.c (line-map.c): Do not advance to linemaps for
different files.
gcc/testsuite/
* g++.dg/diagnostic/pr72803.C: Adjust expected column.
The problem is that the new IS_MACRO_LOC macro:
inline bool
IS_MACRO_LOC (location_t loc)
{
return !IS_ORDINARY_LOC (loc) && !IS_ADHOC_LOC (loc);
}
is not fully correct since the position of the macro lines is not fixed:
/* Returns the lowest location [of a token resulting from macro
expansion] encoded in this line table. */
inline location_t
LINEMAPS_MACRO_LOWEST_LOCATION (const line_maps *set)
{
return LINEMAPS_MACRO_USED (set)
? MAP_START_LOCATION (LINEMAPS_LAST_MACRO_MAP (set))
: MAX_LOCATION_T + 1;
}
In Ada, LINEMAPS_MACRO_USED is false so LINEMAPS_MACRO_LOWEST_LOCATION is
MAX_LOCATION_T + 1, but IS_MACRO_LOC nevertheless returns true for anything
in the range [LINE_MAP_MAX_LOCATION; MAX_LOCATION_T], thus yielding an ICE
in linemap_macro_map_lookup for very large files.
libcpp/
* include/line-map.h (IS_MACRO_LOC): Delete.
* line-map.c (linemap_location_from_macro_expansion_p): Test
LINEMAPS_MACRO_LOWEST_LOCATION of the linemap.
gcc/cp/
* module.cc (ordinary_loc_of): Test LINEMAPS_MACRO_LOWEST_LOCATION
of the linemap.
(module_state::write_location): Likewise.
PR c/99323 describes an ICE due to a failed assertion deep inside the
fix-it printing machinery, where the fix-it hints on one line have not
been properly sorted in layout's constructor.
The underlying issue occurs when multiple fix-it hints affect a line
wider that LINE_MAP_MAX_COLUMN_NUMBER, where the location_t values for
characters after that threshold fall back to having column zero.
It's not meaningful to try to handle fix-it hints without column
information, so this patch rejects them as they are added to the
rich_location, falling back to the "no fix-it hints on this diagnostic"
case, fixing the crash.
gcc/ChangeLog:
PR c/99323
* diagnostic-show-locus.c
(selftest::test_one_liner_many_fixits_2): Fix accidental usage of
column 0.
gcc/testsuite/ChangeLog:
PR c/99323
* gcc.dg/pr99323-1.c: New test.
* gcc.dg/pr99323-2.c: New test.
libcpp/ChangeLog:
PR c/99323
* line-map.c (rich_location::maybe_add_fixit): Reject fix-it hints
at column 0.
This fixes some issues with macro maps. We were incorrectly
calculating the number of macro expansions in a location span, and I
had a workaround that partially covered that up. Further, while macro
location spans are monotonic, that is not true of ordinary location
spans. Thus we need to insert an indirection array when binary
searching the latter. (We load ordinary locations before loading
imports, but macro locations afterwards. We make sure an import
location is de-macrofied, if needed.)
PR c++/98718
gcc/cp/
* module.cc (ool): New indirection vector.
(loc_spans::maybe_propagate): Location is not optional.
(loc_spans::open): Likewise. Assert monotonically advancing.
(module_for_ordinary_loc): Use ool indirection vector.
(module_state::write_prepare_maps): Do not count empty macro
expansions. Elide empty spans.
(module_state::write_macro_maps): Skip empty expansions.
(ool_cmp): New qsort comparator.
(module_state::write): Create and destroy ool vector.
(name_pending_imports): Fix dump push/pop.
(preprocess_module): Likewise. Add more dumping.
(preprocessed_module): Likewise.
libcpp/
* include/line-map.h
* line-map.c
gcc/testsuite/
* g++.dg/modules/pr98718_a.C: New.
* g++.dg/modules/pr98718_b.C: New.
When we read preprocessed source, we deal with a couple of special
location lines at the start of the file. These provide information
about the original filename of the source and the current directory,
so we can process the source in the same manner. When updating that
code, I had a somewhat philosophical question: Should the line table
contain evidence of the filename the user provided to the compiler? I
figured to leave it there, as it did no harm. But this defect shows
an issue. It's in the line table and our (non optimizing) line table
serializer emits that filename. Which means if one re-preprocesses
the original source to a differently-named intermediate file, the
resultant CMI is different. Boo. That's a difference that doesn't
matter, except the CRC matching then fails. We should elide the
filename, so that one can preprocess to mktemp intermediate filenames
for whatever reason.
This patch takes the approach of expunging it from the line table --
so the line table will end up with exactly the same form. That seems
a better bet than trying to fix up mismatching line tables in CMI
emission.
PR c++/99072
libcpp/
* init.c (read_original_filename): Expunge all evidence of the
original filename.
gcc/testsuite/
* g++.dg/modules/pr99072.H: New.
This defect really required building header-units and include translation
of pieces of the standard library. This adds smarts to the modules
test harness to do that -- accept .X files as the source file, but
provide '-x c++-system-header $HDR' in the options. The .X file will
be considered by the driver to be a linker script and ignored (with a
warning).
Using this we can add 2 tests that end up building list_initializer
and iostream, along with a test that iostream's build
include-translates list_initializer's #include. That discovered a set
of issues with the -flang-info-include-translate=HDR handling, also
fixed and documented here.
PR c++/99023
gcc/cp/
* module.cc (canonicalize_header_name): Use
cpp_probe_header_unit.
(maybe_translate_include): Fix note_includes comparison.
(init_modules): Fix note_includes string termination.
libcpp/
* include/cpplib.h (cpp_find_header_unit): Rename to ...
(cpp_probe_header_unit): ... this.
* internal.h (_cp_find_header_unit): Declare.
* files.c (cpp_find_header_unit): Break apart to ..
(test_header_unit): ... this, and ...
(_cpp_find_header_unit): ... and, or and ...
(cpp_probe_header_unit): ... this.
* macro.c (cpp_get_token_1): Call _cpp_find_header_unit.
gcc/
* doc/invoke.texi (flang-info-include-translate): Document header
lookup behaviour.
gcc/testsuite/
* g++.dg/modules/modules.exp: Bail on cross-testing. Add support
for .X files.
* g++.dg/modules/pr99023_a.X: New.
* g++.dg/modules/pr99023_b.X: New.
We make sure files end in \n by placing one at the limit of the buffer
(just past the end of what is read). We need to do the same for
buffers generated via include-translation. Fortunately they have
space.
libcpp/
* files.c (_cpp_stack_file): Make buffers end in unread \n.
gcc/testsuite/
* g++.dg/modules/pr99050_a.H: New.
* g++.dg/modules/pr99050_b.C: New.
PR preprocessor/96391 describes an ICE in the C++ frontend on:
#define CONST const
#define VOID void
typedef CONST VOID *PCVOID;
where the typedef line occurs after enough code has been compiled
that location_t values are beyond LINE_MAP_MAX_LOCATION_WITH_COLS,
and hence no column numbers are available.
The issue occurs in linemap_compare_locations when comparing the
locations of the "const" and "void" tokens.
Upon resolving the LRK_MACRO_EXPANSION_POINT, both have the same
location_t, the line of the "typedef" (with no column), and so
the l0 == l1 clause is triggered, but they are not from the
same macro expansion, leading first_map_in_common to return NULL
and triggering the "abort" condition.
This patch fixes the issue by checking when the two macro expansion
point location_t values are equal that the value
<= LINE_MAP_MAX_LOCATION_WITH_COLS and thus has column information,
fixing the issue.
gcc/testsuite/ChangeLog:
PR preprocessor/96391
* g++.dg/plugin/location-overflow-test-pr96391.c: New test.
* g++.dg/plugin/plugin.exp (plugin_test_list): Add it,
using the location_overflow_plugin.c from gcc.dg/plugin.
libcpp/ChangeLog:
PR preprocessor/96391
* line-map.c (linemap_compare_locations): Require that
the location be <= LINE_MAP_MAX_LOCATION_WITH_COLS when
treating locations as coming from the same macro expansion.
The following patch uses make_signed_t<size_t> instead of
make_signed<size_t>::type in the diagnostics, because the former is shorter.
It is true that one can't use make_signed<size_t>::type in C++11 code (which
is why I haven't changed it in the testcase which is c++11 effective
target), but the message talks about C++23 and make_signed_t is a C++14 and
later feature, so I think it is fine.
2021-02-04 Jakub Jelinek <jakub@redhat.com>
* expr.c (cpp_classify_number): Use make_signed_t<size_t> instead of
make_signed<size_t>::type in the diagnostics.
* g++.dg/warn/Wsize_t-literals.C: Expect make_signed_t<size_t> instead
of make_signed<size_t>::type in the diagnostics.
GCC 11 ICEs on all -fdirectives-only preprocessing when the files don't end
with a newline.
The problem is in the assertion, for empty TUs buffer->cur == buffer->rlimit
and so buffer->rlimit[-1] access triggers UB in the preprocessor, for
non-empty TUs it refers to the last character in the file, which can be
anything.
The preprocessor adds a '\n' character (or '\r', in particular if the
user file ends with '\r' then it adds another '\r' rather than '\n'), but
that is added after the limit, i.e. at buffer->rlimit[0].
Now, if the routine handles occassional bumping of pos to buffer->rlimit + 1,
I think it is just the assert that needs changing, usually we read from *pos
if pos < limit and then e.g. if it is '\r', look at the following character
(which could be one of those '\n' or '\r' at buffer->rlimit[0]). There is
also the case where for '\\' before the limit we read following character
and if it is '\n', do one thing, if it is '\r' read another character.
But in that case if '\\' was the last char in the TU, the limit char will be
'\n', so we are ok.
2021-02-03 Jakub Jelinek <jakub@redhat.com>
PR preprocessor/98882
* lex.c (cpp_directive_only_process): Don't assert that rlimit[-1]
is a newline, instead assert that rlimit[0] is either newline or
carriage return. When seeing '\\' followed by '\r', check limit
before accessing pos[1].
* gcc.dg/cpp/pr98882.c: New test.
Integer literal suffixes for signed size ('z') and unsigned size
(some permutation od 'zu') are provided as a language addition.
gcc/c-family/ChangeLog:
* c-cppbuiltin.c (c_cpp_builtins): Define __cpp_size_t_suffix.
* c-lex.c (interpret_integer): Set node type for size literal.
libcpp/ChangeLog:
* expr.c (interpret_int_suffix): Detect 'z' integer suffix.
(cpp_classify_number): Compat warning for use of 'z' suffix.
* include/cpplib.h (struct cpp_options): New flag.
(enum cpp_warning_reason): New flag.
(CPP_N_USERDEF): Comment C++0x -> C++11.
(CPP_N_SIZE_T): New flag for cpp_classify_number.
* init.c (cpp_set_lang): Initialize new flag.
gcc/testsuite/ChangeLog:
* g++.dg/cpp0x/udlit-shadow-neg.C: Test for 'z' and 'zu' shadowing.
* g++.dg/cpp23/feat-cxx2b.C: New test.
* g++.dg/cpp23/size_t-literals.C: New test.
* g++.dg/warn/Wsize_t-literals.C: New test.
Derived from the changes that added C++2a support in 2017.
r8-3237-g026a79f70cf33f836ea5275eda72d4870a3041e5
No C++23 features are added here.
Use of -std=c++23 sets __cplusplus to 202100L.
$ g++ -std=c++23 -dM -E -x c++ - < /dev/null | grep cplusplus
#define __cplusplus 202100L
gcc/
* doc/cpp.texi (__cplusplus): Document value for -std=c++23
or -std=gnu++23.
* doc/invoke.texi: Document -std=c++23 and -std=gnu++23.
* dwarf2out.c (highest_c_language): Recognise C++20 and C++23.
(gen_compile_unit_die): Recognise C++23.
gcc/c-family/
* c-common.h (cxx_dialect): Add cxx23 as a dialect.
* c.opt: Add options for -std=c++23, std=c++2b, -std=gnu++23
and -std=gnu++2b
* c-opts.c (set_std_cxx23): New.
(c_common_handle_option): Set options when -std=c++23 is enabled.
(c_common_post_options): Adjust comments.
(set_std_cxx20): Likewise.
gcc/testsuite/
* lib/target-supports.exp (check_effective_target_c++2a):
Check for C++2a or C++23.
(check_effective_target_c++20_down): New.
(check_effective_target_c++23_only): New.
(check_effective_target_c++23): New.
* g++.dg/cpp23/cplusplus.C: New.
libcpp/
* include/cpplib.h (c_lang): Add CXX23 and GNUCXX23.
* init.c (lang_defaults): Add rows for CXX23 and GNUCXX23.
(cpp_init_builtins): Set __cplusplus to 202100L for C++23.
For deferred macros we also need a new field on the macro itself, so
that the module machinery can determine the macro was imported. Also
the documentation for the hashnode's deferred field was incomplete.
libcpp/
* include/cpplib.h (struct cpp_macro): Add imported_p field.
(struct cpp_hashnode): Tweak deferred field documentation.
* macro.c (_cpp_new_macro): Clear new field.
(cpp_get_deferred_macro, get_deferred_or_lazy_macro): Assert
more.
The preprocessor check for overflow (of linenum_type = unsigned int)
when reading the line number in a #line directive is incomplete; it
checks "reg < reg_prev" which doesn't cover all cases where
multiplying by 10 overflowed. Fix this by checking for overflow
before rather than after it occurs (using essentially the same logic
as used by e.g. glibc printf when reading width and precision values
from strings).
Bootstrapped with no regressions for x86_64-pc-linux-gnu.
libcpp/
2020-11-27 Joseph Myers <joseph@codesourcery.com>
PR preprocessor/97602
* directives.c (strtolinenum): Check for overflow before it
occurs. Correct comment.
gcc/testsuite/
2020-11-27 Joseph Myers <joseph@codesourcery.com>
PR preprocessor/97602
* gcc.dg/cpp/line9.c, gcc.dg/cpp/line10.c: New tests.
Deferred macros are needed for C++ modules. Header units may export
macro definitions and undefinitions. These are resolved lazily at the
point of (potential) use. (The language specifies that, it's not just
a useful optimization.) Thus, identifier nodes grow a 'deferred'
field, which fortunately doesn't expand the structure on 64-bit
systems as there was padding there. This is non-zero on NT_MACRO
nodes, if the macro is deferred. When such an identifier is lexed, it
is resolved via a callback that I added recently. That will either
provide the macro definition, or discover it there was an overriding
undef. Either way the identifier is no longer a deferred macro.
Notice it is now possible for NT_MACRO nodes to have a NULL macro
expansion.
libcpp/
* include/cpplib.h (struct cpp_hashnode): Add deferred field.
(cpp_set_deferred_macro): Define.
(cpp_get_deferred_macro): Declare.
(cpp_macro_definition): Reformat, add overload.
(cpp_macro_definition_location): Deal with deferred macro.
(cpp_alloc_token_string, cpp_compare_macro): Declare.
* internal.h (_cpp_notify_macro_use): Return bool
(_cpp_maybe_notify_macro_use): Likewise.
* directives.c (do_undef): Check macro is not undef before
warning.
(do_ifdef, do_ifndef): Deal with deferred macro.
* expr.c (parse_defined): Likewise.
* lex.c (cpp_allocate_token_string): Break out of ...
(create_literal): ... here. Call it.
(cpp_maybe_module_directive): Deal with deferred macro.
* macro.c (cpp_get_token_1): Deal with deferred macro.
(warn_of_redefinition): Deal with deferred macro.
(compare_macros): Rename to ...
(cpp_compare_macro): ... here. Make extern.
(cpp_get_deferred_macro): New.
(_cpp_notify_macro_use): Deal with deferred macro, return bool
indicating definedness.
(cpp_macro_definition): Deal with deferred macro.
This adds the capability to locate the main file on the user or system
include paths. That's extremely useful to users building header
units. Searching has to be requiested (plain header-unit compilation
will not search). Also, to make include_next work as expected when
building a header unit, we add a mechanism to retrofit a non-searched
source file as one on the include path.
libcpp/
* include/cpplib.h (enum cpp_main_search): New.
(struct cpp_options): Add main_search field.
(cpp_main_loc): Declare.
(cpp_retrofit_as_include): Declare.
* internal.h (struct cpp_reader): Add main_loc field.
(_cpp_in_main_source_file): Not main if main is a header.
* init.c (cpp_read_main_file): Use main_search option to locate
main file. Set main_loc
* files.c (cpp_retrofit_as_include): New.
In preparing module patch 7 I realized there was a cleanup I could
make to simplify it. This is that cleanup. Also, when doing the
cleanup I noticed some macros had been turned into inline functions,
but not renamed to the preprocessors internal namespace
(_cpp_$INTERNAL rather than cpp_$USER). Thus, this renames those
functions, deletes an internal field of the file structure, and
determines whether we're in the main file by comparing to
pfile->main_file, the _cpp_file of the main file.
libcpp/
* internal.h (cpp_in_system_header): Rename to ...
(_cpp_in_system_header): ... here.
(cpp_in_primary_file): Rename to ...
(_cpp_in_main_source_file): ... here. Compare main_file equality
and check main_search value.
* lex.c (maybe_va_opt_error, _cpp_lex_direct): Adjust for rename.
* macro.c (_cpp_builtin_macro_text): Likewise.
(replace_args): Likewise.
* directives.c (do_include_next): Likewise.
(do_pragma_once, do_pragma_system_header): Likewise.
* files.c (struct _cpp_file): Delete main_file field.
(pch_open): Check pfile->main_file equality.
(make_cpp_file): Drop cpp_reader parm, don't set main_file.
(_cpp_find_file): Adjust.
(_cpp_stack_file): Check pfile->main_file equality.
(struct report_missing_guard_data): Add cpp_reader field.
(report_missing_guard): Check pfile->main_file equality.
(_cpp_report_missing_guards): Adjust.
C++20 modules introduces a new kind of preprocessor directive -- a
module directive. These are directives but without the leading '#'.
We have to detect them by sniffing the start of a logical line. When
detected we replace the initial identifiers with unspellable tokens
and pass them through to the language parser the same way deferred
pragmas are. There's a PRAGMA_EOL at the logical end of line too.
One additional complication is that we have to do header-name lexing
after the initial tokens, and that requires changes in the macro-aware
piece of the preprocessor. The above sniffer sets a counter in the
lexer state, and that triggers at the appropriate point. We then do
the same header-name lexing that occurs on a #include directive or
has_include pseudo-macro. Except that the header name ends up in the
token stream.
A couple of token emitters need to deal with the new token possibility.
gcc/c-family/
* c-lex.c (c_lex_with_flags): CPP_HEADER_NAMEs can now be seen.
libcpp/
* include/cpplib.h (struct cpp_options): Add module_directives
option.
(NODE_MODULE): New node flag.
(struct cpp_hashnode): Make rid-code a bitfield, increase bits in
flags and swap with type field.
* init.c (post_options): Create module-directive identifier nodes.
* internal.h (struct lexer_state): Add directive_file_token &
n_modules fields. Add module node enumerator.
* lex.c (cpp_maybe_module_directive): New.
(_cpp_lex_token): Call it.
(cpp_output_token): Add '"' around CPP_HEADER_NAME token.
(do_peek_ident, do_peek_module): New.
(cpp_directives_only): Detect module-directive lines.
* macro.c (cpp_get_token_1): Deal with directive_file_token
triggering.
This is slightly different to the original patch I posted. This adds
separate module target and dependency functions (rather than a single
bi-modal function).
libcpp/
* include/cpplib.h (struct cpp_options): Add modules to
dep-options.
* include/mkdeps.h (deps_add_module_target): Declare.
(deps_add_module_dep): Declare.
* mkdeps.c (class mkdeps): Add modules, module_name, cmi_name,
is_header_unit fields. Adjust cdtors.
(deps_add_module_target, deps_add_module_dep): New.
(make_write): Write module dependencies, if enabled.
These two callbacks are needed for C++ modules. The first is for
handling macros from header-units. These are resolved lazily. The
second is for include-translation -- whether a #include gets turned
into a header-unit import.
libcpp/
* include/cpplib.h (struct cpp_callbacks): Add
user_deferred_macro & translate_include.
This patch adds LC_MODULE as a map kind, used to indicate a c++
module. Unlike a regular source file, it only contains a single
location, and the source locations in that module are represented by
ordinary locations whose 'included_from' location is the module.
It also exposes some entry points that modules will use to create
blocks of line maps.
In the original posting, I'd missed the deletion of the
linemap_enter_macro from internal.h. That's included here.
libcpp/
* include/line-map.h (enum lc_reason): Add LC_MODULE.
(MAP_MODULE_P): New.
(line_map_new_raw): Declare.
(linemap_enter_macro): Move declaration from internal.h
(linemap_module_loc, linemap_module_reparent)
(linemap_module_restore): Declare.
(linemap_lookup_macro_indec): Declare.
* internal.h (linemap_enter_macro): Moved to line-map.h.
* line-map.c (linemap_new_raw): New, broken out of ...
(new_linemap): ... here. Call it.
(LAST_SOURCE_LINE_LOCATION): New.
(liemap_module_loc, linemap_module_reparent)
(linemap_module_restore): New.
(linemap_lookup_macro_index): New, broken out of ...
(linemap_macro_map_lookup): ... here. Call it.
(linemap_dump): Add module dump.
As Jakub points out, we only ever pass a single variadic parm (if at
all), so just an optional arg is fine.
PR preprocessor/97858
libcpp/
* mkdeps.c (munge): Drop varadic args, we only ever use one.
C2x adds binary integer constants (approved at the last WG14 meeting,
though not yet added to the working draft in git). Configure libcpp
to consider these a standard feature in C2x mode, with appropriate
updates to diagnostics including support for diagnosing them with
-std=c2x -Wc11-c2x-compat.
Bootstrapped with no regressions for x86_64-pc-linux-gnu.
gcc/testsuite/
2020-11-13 Joseph Myers <joseph@codesourcery.com>
* gcc.dg/binary-constants-2.c, gcc.dg/binary-constants-3.c,
gcc.dg/system-binary-constants-1.c: Update expected diagnostics.
* gcc.dg/c11-binary-constants-1.c,
gcc.dg/c11-binary-constants-2.c, gcc.dg/c2x-binary-constants-1.c,
gcc.dg/c2x-binary-constants-2.c, gcc.dg/c2x-binary-constants-3.c:
New tests.
libcpp/
2020-11-13 Joseph Myers <joseph@codesourcery.com>
* expr.c (cpp_classify_number): Update diagnostic for binary
constants for C. Also diagnose binary constants for
-Wc11-c2x-compat.
* init.c (lang_defaults): Enable binary constants for GNUC2X and
STDC2X.
C2x adds the __has_c_attribute preprocessor operator, similar to C++
__has_cpp_attribute.
GCC implements __has_cpp_attribute as exactly equivalent to
__has_attribute. (The documentation says they differ regarding the
values returned for standard attributes, but that's actually only a
matter of the particular nonzero value returned not being specified in
the documentation for __has_attribute; the implementation makes no
distinction between the two.)
I don't think having them exactly equivalent is actually correct,
either for __has_cpp_attribute or for __has_c_attribute.
Specifically, I think it is only correct for __has_cpp_attribute or
__has_c_attribute to return nonzero if the given attribute is
supported, with the particular pp-tokens passed to __has_cpp_attribute
or __has_c_attribute, with [[]] syntax, not if it's only accepted in
__attribute__ or with gnu:: added in [[]]. For example, they should
return nonzero for gnu::packed, but zero for plain packed, because
[[gnu::packed]] is accepted but [[packed]] is ignored as not a
standard attribute.
This patch implements that for __has_c_attribute, leaving any changes
to __has_cpp_attribute for the C++ maintainers. A new
BT_HAS_STD_ATTRIBUTE is added for __has_c_attribute (which I think,
based on the above, would actually be correct to use for
__has_cpp_attribute as well). The code in c_common_has_attribute that
deals with scopes has its C++ conditional removed; instead, whether
the language is C or C++ is used only to determine the numeric values
returned for standard attributes (and which standard attributes are
handled there at all). A new argument is passed to
c_common_has_attribute to distinguish BT_HAS_STD_ATTRIBUTE from
BT_HAS_ATTRIBUTE, and that argument is used to stop attributes with no
scope specified from being accepted with __has_c_attribute unless they
are one of the known standard attributes and so handled specially.
Although the standard specify constants ending with 'L' as the values
for the standard attributes, there is no correctness issue with the
lack of code in GCC to add that 'L' to the expansion:
__has_c_attribute and __has_cpp_attribute are expanded in #if after
other macro expansion has occurred, with no semantics being specified
if they occur outside #if, so there is no way for a conforming program
to inspect the exact text of the expansion of those macros, only to
use the resulting pp-number in a #if expression, where long and int
have the same set of values.
Bootstrapped with no regressions for x86_64-pc-linux-gnu.
gcc/
2020-11-12 Joseph Myers <joseph@codesourcery.com>
* doc/cpp.texi (__has_attribute): Document when scopes are allowed
for C.
(__has_c_attribute): New.
gcc/c-family/
2020-11-12 Joseph Myers <joseph@codesourcery.com>
* c-lex.c (c_common_has_attribute): Take argument std_syntax.
Allow scope for C. Handle standard attributes for C. Do not
accept unscoped attributes if std_syntax and not handled as
standard attributes.
* c-common.h (c_common_has_attribute): Update prototype.
gcc/testsuite/
2020-11-12 Joseph Myers <joseph@codesourcery.com>
* gcc.dg/c2x-has-c-attribute-1.c, gcc.dg/c2x-has-c-attribute-2.c,
gcc.dg/c2x-has-c-attribute-3.c, gcc.dg/c2x-has-c-attribute-4.c:
New tests.
libcpp/
2020-11-12 Joseph Myers <joseph@codesourcery.com>
* include/cpplib.h (struct cpp_callbacks): Add bool argument to
has_attribute.
(enum cpp_builtin_type): Add BT_HAS_STD_ATTRIBUTE.
* init.c (builtin_array): Add __has_c_attribute.
(cpp_init_special_builtins): Handle BT_HAS_STD_ATTRIBUTE.
* macro.c (_cpp_builtin_macro_text): Handle BT_HAS_STD_ATTRIBUTE.
Update call to has_attribute for BT_HAS_ATTRIBUTE.
* traditional.c (fun_like_macro): Handle BT_HAS_STD_ATTRIBUTE.
gcc/c-family
PR pch/86674
* c-pch.c (c_common_valid_pch): Use cpp_warning with CPP_W_INVALID_PCH
reason to fix -Werror=invalid-pch and -Wno-error=invalid-pch switches.
libcpp
PR pch/86674
* files.c (_cpp_find_file): Use CPP_DL_NOTE not CPP_DL_ERROR in call to
cpp_error.
generated_cpp_wcwidth.h was regenerated using Unicode 13.0.0 data files. No
material changes to the parsing scripts (either GCC- or glibc-sourced) were
necessary; glibc's utf8_gen.py was tweaked slightly by glibc and matched here.
contrib/ChangeLog:
* unicode/EastAsianWidth.txt: Update to Unicode 13.0.0.
* unicode/PropList.txt: Likewise.
* unicode/README: Likewise.
* unicode/UnicodeData.txt: Likewise.
* unicode/from_glibc/unicode_utils.py: Update to latest glibc version.
* unicode/from_glibc/utf8_gen.py: Likewise.
libcpp/ChangeLog:
* generated_cpp_wcwidth.h: Regenerated from Unicode 13.0.0 data.
Joseph pointed me at cb_get_source_date_epoch, which allows repeatable
builds and solves a FIXME I had on the modules branch. Unfortunately
it's used exclusively to generate __DATE__ and __TIME__ values, which
fallback to using a time(2) call. It'd be nicer if the preprocessor
made whatever time value it determined available to the rest of the
compiler. So this patch adds a new cpp_get_date function, which
abstracts the call to the get_source_date_epoch hook, or uses time
directly. The value is cached. Thus the timestamp I end up putting
on CMI files matches __DATE__ and __TIME__ expansions. That seems
worthwhile.
libcpp/
* include/cpplib.h (enum class CPP_time_kind): New.
(cpp_get_date): Declare.
* internal.h (struct cpp_reader): Replace source_date_epoch with
time_stamp and time_stamp_kind.
* init.c (cpp_create_reader): Initialize them.
* macro.c (_cpp_builtin_macro_text): Use cpp_get_date.
(cpp_get_date): Broken out from _cpp_builtin_macro_text and
genericized.
This patch moves the generation of PRAGMA_EOF earlier, to when we set
need_line, rather than when we try and get the next line. It also
prevents peeking past a PRAGMA token.
libcpp/
* lex.c (cpp_peek_token): Do not peek past CPP_PRAGMA.
(_cpp_lex_direct): Handle EOF in pragma when setting need_line,
not when needing a line.
I noticed a fencepost error in the preprocessor. We should be
checking if the next char is at the limit, not the current char (which
can't be, because we're looking at it).
libcpp/
* lex.c (_cpp_clean_line): Fix DOS off-by-one error.
This patch cleans up the interface to the dependency generation a
little. We now only check the option in one place, and the
cpp_get_deps function returns nullptr if there are no dependencies. I
also reworded the -MT and -MQ help text to be make agnostic -- as
there are ideas about emitting, say, JSON.
libcpp/
* include/mkdeps.h: Include cpplib.h
(deps_write): Adjust first parm type.
* mkdeps.c: Include internal.h
(make_write): Adjust first parm type. Check phony option
directly.
(deps_write): Adjust first parm type.
* init.c (cpp_read_main_file): Use get_deps.
* directives.c (cpp_get_deps): Check option before initializing.
gcc/c-family/
* c.opt (MQ,MT): Reword description to be make-agnostic.
gcc/fortran/
* cpp.c (gfc_cpp_add_dep): Only add dependency if we're recording
them.
(gfc_cpp_init): Likewise for target.
Our macro use hook passes a location, but doesn't recieve it from the
using location. This patch adds the extra location_t parameter and
passes it though.
A second cleanup is breaking out the macro comparison code from the
redefinition warning. That;ll turn out useful for modules.
Finally, there's a filename comparison needed for the location
optimization of rewinding from line 2 (occurs during the emission of
builtin macros).
libcpp/
* internal.h (_cpp_notify_macro_use): Add location parm.
(_cpp_maybe_notify_macro_use): Likewise.
* directives.c (_cpp_do_file_change): Check we've not changed file
when optimizing a rewind.
(do_ifdef): Pass location to _cpp_maybe_notify_macro_use.
(do_ifndef): Likewise. Delete obsolete comment about powerpc.
* expr.c (parse_defined): Pass location to
_cpp_maybe_notify_macro_use.
* macro.c (enter_macro_context): Likewise.
(warn_of_redefinition): Break out helper function. Call it.
(compare_macros): New function broken out of warn_of_redefinition.
(_cpp_new_macro): Zero all fields.
(_cpp_notify_macro_use): Add location parameter.
My previous attempt at fixing this was incorrect. The problem occurs
earlier in that _cpp_lex_direct processes the unwinding EOF needs in
collect_args mode. This patch changes it not to do that, in the same
way as directive parsing works. Also collect_args shouldn't push_back
such fake EOFs, and neither should funlike_invocation_p.
libcpp/
* lex.c (_cpp_lex_direct): Do not complete EOF processing when
parsing_args.
* macro.c (collect_args): Do not unwind fake EOF.
(funlike_invocation_p): Do not unwind fake EOF.
(cpp_context): Replace abort with gcc_assert.
gcc/testsuite/
* gcc.dg/cpp/endif.c: Move to ...
* c-c++-common/cpp/endif.c: ... here.
* gcc.dg/cpp/endif.h: Move to ...
* c-c++-common/cpp/endif.h: ... here.
* c-c++-common/cpp/eof-2.c: Adjust diagnostic.
* c-c++-common/cpp/eof-3.c: Adjust diagnostic.
We inject EOF tokens between macro argument lists, but had
confused/stale logic in the non-fn invocation. Renamed the magic
'eof' token, as it's now only used for macro argument termination.
Always rewind the non-OPEN_PAREN token.
libcpp/
* internal.h (struct cpp_reader): Rename 'eof' field to 'endarg'.
* init.c (cpp_create_reader): Adjust.
* macro.c (collect_args): Use endarg for separator. Always rewind
in the not-fn case.
gcc/testsuite/
* c-c++-common/cpp/pr97471.c: New.
Using the tokenizer to sniff for an initial line marker for
preprocessed input is a little brittle, particularly with
-fdirectives-only. If there is no marker we'll happily munch initial
comments. This patch directly sniffs the buffer. This is safe
because the initial line marker was machine generated and must be
right at the beginning of the file. Anything else is not such a line
marker. The same is true for the initial directory marker. For that
tokenizing the string is simplest, but at that point it's either a
regular line marker or a directory marker. If it's a regular marker,
unwinding tokens is fine.
libcpp/
* internal.h (enum include_type): Rename IT_MAIN_INJECT to
IT_PRE_MAIN.
* init.c (cpp_read_main_file): If there is no line marker, adjust
the initial line marker.
(read_original_filename): Return bool, peek the buffer directly
before trying to tokenize.
(read_original_directory): Likewise. Directly prod the string
literal.
* files.c (_cpp_stack_file): Adjust for IT_PRE_MAIN change.
libcpp has two specialized altivec implementations of search_line_fast,
one for power8+ and the other one otherwise.
Both use __attribute__((altivec(vector))) and the GCC builtins rather than
altivec.h and the APIs from there, which is fine, but should be restricted
to when libcpp is built with GCC, so that it can be relied on.
The second elif is
and thus e.g. when built with clang it isn't picked, but the first one was
just guarded with
and so according to the bugreporter clang fails miserably on that.
The following patch fixes that by adding the same GCC_VERSION requirement
as the second version. I don't know where the 4.5 in there comes from and
the exact version doesn't matter that much, as long as it is above 4.2 that
clang pretends to be and smaller or equal to 4.8 as the oldest gcc we
support as bootstrap compiler ATM.
Furthermore, the patch fixes the comment, the version it is talking about is
not pre-GCC 5, but actually the GCC 5+ one.
2020-09-26 Jakub Jelinek <jakub@redhat.com>
PR bootstrap/97163
* lex.c (search_line_fast): Only use _ARCH_PWR8 Altivec version
for GCC >= 4.5.
Currently the -Wmisleading-indentation warning doesn't do any analysis
when the guarded statement or the statement after it is produced by a
macro. This means we warn for:
if (flag)
foo ();
bar ();
but not for:
#define BAR bar
if (flag)
foo ();
BAR ();
This patch extends the -Wmisleading-indentation implementation to
support analyzing such statements and their tokens. This is done in the
"natural" way by resolving the location of each of the three tokens to
the token's macro expansion point. (Additionally, if the tokens all
resolve to the same macro expansion point then we instead use their
locations within the macro definition.) When these resolved locations
are all different, then we can proceed with applying the warning
heuristics to them as if no macros were involved.
gcc/c-family/ChangeLog:
PR c/80076
* c-indentation.c (should_warn_for_misleading_indentation): Move
declarations of local variables closer to their first use.
Handle virtual token locations by resolving them to their
respective macro expansion points. If all three tokens are
produced from the same macro expansion, then instead use their
loci within the macro definition.
gcc/objc/ChangeLog:
PR c/80076
* objc-gnu-runtime-abi-01.c
(gnu_runtime_abi_01_get_class_super_ref): Reduce indentation of
misleadingly indented return statements.
* objc-next-runtime-abi-01.c
(next_runtime_abi_01_get_class_super_ref): Likewise.
gcc/ChangeLog:
PR c/80076
* gensupport.c (alter_attrs_for_subst_insn) <case SET_ATTR>:
Reduce indentation of misleadingly indented code fragment.
* lra-constraints.c (multi_block_pseudo_p): Likewise.
* sel-sched-ir.c (merge_fences): Likewise.
libcpp/ChangeLog:
PR c/80076
* include/line-map.h (first_map_in_common): Declare.
* line-map.c (first_map_in_common): Remove static.
gcc/testsuite/ChangeLog:
PR c/80076
* c-c++-common/Wmisleading-indentation-5.c: New test.
This fixes compilation of codepaths for dos-like filesystems
with Clang. When built with clang, it treats C input files as C++
when the compiler driver is invoked in C++ mode, triggering errors
when the return value of strchr() on a pointer to const is assigned
to a pointer to non-const variable.
This matches similar variables outside of the ifdefs for dos-like
path handling.
2020-09-07 Martin Storsjö <martin@martin.st>
gcc/
* dwarf2out.c (file_name_acquire): Make a strchr return value
pointer to const.
libcpp/
* files.c (remap_filename): Make a strchr return value pointer
to const.
With --enable-cet, require CET support only for the final GCC build.
Don't enable CET without CET support for non-bootstrap build, in stage1
nor for build support.
config/
PR bootstrap/96202
* cet.m4 (GCC_CET_HOST_FLAGS): Don't enable CET without CET
support in stage1 nor for build support.
gcc/
PR bootstrap/96202
* configure: Regenerated.
libbacktrace/
PR bootstrap/96202
* configure: Regenerated.
libcc1/
PR bootstrap/96202
* configure: Regenerated.
libcpp/
PR bootstrap/96202
* configure: Regenerated.
libdecnumber/
PR bootstrap/96202
* configure: Regenerated.
libiberty/
PR bootstrap/96202
* configure: Regenerated.
lto-plugin/
PR bootstrap/96202
* configure: Regenerated.
Traditional cpp (used by fortran) didn;t know about the new
__has_include__ implementation. Hey, since when did traditional cpp
grow __has_include__? That wasn't in knr!
libcpp/
* init.c (builtin_array): Add xref comment.
* traditional.c (fun_like_macro): Add HAS_INCLUDE codes.
gcc/testsuite/
* c-c++-common/cpp/has-include-1-traditional.c: New.
For (invalid) newline inside of the raw string literal delimiter, doing
continue means we skip the needed processing of newlines. Instead of
duplicating that, this patch just doesn't continue for those.
2020-07-28 Jakub Jelinek <jakub@redhat.com>
PR preprocessor/96323
* lex.c (lex_raw_string): For c == '\n' don't continue after reporting
an prefix delimiter error.
* c-c++-common/cpp/pr96323.c: New test.
We can always use the final map to get the return value, rather than
conditionally only when there was an immediate line directive.
libcpp/
* init.c (cpp_read_main_file): Always use the last map for
the return value.
I found the linemap logic dealing with running out of column numbers
confusing. There's no need for completely separate code blocks there,
as we can rely on the masking operations working all the way down to
zero bits. The two binary searches for linemap lookups could do with
modernization of placing the var decls at their initialization point.
(These two searches work in opposite directions, and while lower_bound
would work there, the caching got in the way and I decided to be
conservative.)
libcpp/
* line-map.c (linemap_add): Simplify column overflow calculation.
Add comment about range and column bit init.
(linemap_ordinary_map_lookup): Refactor for RAII
(linemap_macro_map_lookup): Likewise.
Supports conversion of tabs to spaces when outputting diagnostics. Also
adds -fdiagnostics-column-unit and -fdiagnostics-column-origin options to
control how the column number is output, thereby resolving the two PRs.
gcc/c-family/ChangeLog:
PR other/86904
* c-indentation.c (should_warn_for_misleading_indentation): Get
global tabstop from the new source.
* c-opts.c (c_common_handle_option): Remove handling of -ftabstop, which
is now a common option.
* c.opt: Likewise.
gcc/ChangeLog:
PR preprocessor/49973
PR other/86904
* common.opt: Handle -ftabstop here instead of in c-family
options. Add -fdiagnostics-column-unit= and
-fdiagnostics-column-origin= options.
* opts.c (common_handle_option): Handle the new options.
* diagnostic-format-json.cc (json_from_expanded_location): Add
diagnostic_context argument. Use it to convert column numbers as per
the new options.
(json_from_location_range): Likewise.
(json_from_fixit_hint): Likewise.
(json_end_diagnostic): Pass the new context argument to helper
functions above. Add "column-origin" field to the output.
(test_unknown_location): Add the new context argument to calls to
helper functions.
(test_bad_endpoints): Likewise.
* diagnostic-show-locus.c
(exploc_with_display_col::exploc_with_display_col): Support
tabstop parameter.
(layout_point::layout_point): Make use of class
exploc_with_display_col.
(layout_range::layout_range): Likewise.
(struct line_bounds): Clarify that the units are now always
display columns. Rename members accordingly. Add constructor.
(layout::print_source_line): Add support for tab expansion.
(make_range): Adapt to class layout_range changes.
(layout::maybe_add_location_range): Likewise.
(layout::layout): Adapt to class exploc_with_display_col changes.
(layout::calculate_x_offset_display): Support tabstop parameter.
(layout::print_annotation_line): Adapt to struct line_bounds changes.
(layout::print_line): Likewise.
(line_label::line_label): Add diagnostic_context argument.
(get_affected_range): Likewise.
(get_printed_columns): Likewise.
(layout::print_any_labels): Adapt to struct line_label changes.
(class correction): Add m_tabstop member.
(correction::correction): Add tabstop argument.
(correction::compute_display_cols): Use m_tabstop.
(class line_corrections): Add m_context member.
(line_corrections::line_corrections): Add diagnostic_context argument.
(line_corrections::add_hint): Use m_context to handle tabstops.
(layout::print_trailing_fixits): Adapt to class line_corrections
changes.
(test_layout_x_offset_display_utf8): Support tabstop parameter.
(test_layout_x_offset_display_tab): New selftest.
(test_one_liner_colorized_utf8): Likewise.
(test_tab_expansion): Likewise.
(test_diagnostic_show_locus_one_liner_utf8): Call the new tests.
(diagnostic_show_locus_c_tests): Likewise.
(test_overlapped_fixit_printing): Adapt to helper class and
function changes.
(test_overlapped_fixit_printing_utf8): Likewise.
(test_overlapped_fixit_printing_2): Likewise.
* diagnostic.h (enum diagnostics_column_unit): New enum.
(struct diagnostic_context): Add members for the new options.
(diagnostic_converted_column): Declare.
(json_from_expanded_location): Add new context argument.
* diagnostic.c (diagnostic_initialize): Initialize new members.
(diagnostic_converted_column): New function.
(maybe_line_and_column): Be willing to output a column of 0.
(diagnostic_get_location_text): Convert column number as per the new
options.
(diagnostic_report_current_module): Likewise.
(assert_location_text): Add origin and column_unit arguments for
testing the new functionality.
(test_diagnostic_get_location_text): Test the new functionality.
* doc/invoke.texi: Document the new options and behavior.
* input.h (location_compute_display_column): Add tabstop argument.
* input.c (location_compute_display_column): Likewise.
(test_cpp_utf8): Add selftests for tab expansion.
* tree-diagnostic-path.cc (default_tree_make_json_for_path): Pass the
new context argument to json_from_expanded_location().
libcpp/ChangeLog:
PR preprocessor/49973
PR other/86904
* include/cpplib.h (struct cpp_options): Removed support for -ftabstop,
which is now handled by diagnostic_context.
(class cpp_display_width_computation): New class.
(cpp_byte_column_to_display_column): Add optional tabstop argument.
(cpp_display_width): Likewise.
(cpp_display_column_to_byte_column): Likewise.
* charset.c
(cpp_display_width_computation::cpp_display_width_computation): New
function.
(cpp_display_width_computation::advance_display_cols): Likewise.
(compute_next_display_width): Removed and implemented this
functionality in a new function...
(cpp_display_width_computation::process_next_codepoint): ...here.
(cpp_byte_column_to_display_column): Added tabstop argument.
Reimplemented in terms of class cpp_display_width_computation.
(cpp_display_column_to_byte_column): Likewise.
* init.c (cpp_create_reader): Remove handling of -ftabstop, which is now
handled by diagnostic_context.
gcc/testsuite/ChangeLog:
PR preprocessor/49973
PR other/86904
* c-c++-common/Wmisleading-indentation-3.c: Adjust expected output
for new defaults.
* c-c++-common/Wmisleading-indentation.c: Likewise.
* c-c++-common/diagnostic-format-json-1.c: Likewise.
* c-c++-common/diagnostic-format-json-2.c: Likewise.
* c-c++-common/diagnostic-format-json-3.c: Likewise.
* c-c++-common/diagnostic-format-json-4.c: Likewise.
* c-c++-common/diagnostic-format-json-5.c: Likewise.
* c-c++-common/missing-close-symbol.c: Likewise.
* g++.dg/diagnostic/bad-binary-ops.C: Likewise.
* g++.dg/parse/error4.C: Likewise.
* g++.old-deja/g++.brendan/crash11.C: Likewise.
* g++.old-deja/g++.pt/overload2.C: Likewise.
* g++.old-deja/g++.robertl/eb109.C: Likewise.
* gcc.dg/analyzer/malloc-paths-9.c: Likewise.
* gcc.dg/bad-binary-ops.c: Likewise.
* gcc.dg/format/branch-1.c: Likewise.
* gcc.dg/format/pr79210.c: Likewise.
* gcc.dg/plugin/diagnostic-test-expressions-1.c: Likewise.
* gcc.dg/plugin/diagnostic-test-string-literals-1.c: Likewise.
* gcc.dg/redecl-4.c: Likewise.
* gfortran.dg/diagnostic-format-json-1.F90: Likewise.
* gfortran.dg/diagnostic-format-json-2.F90: Likewise.
* gfortran.dg/diagnostic-format-json-3.F90: Likewise.
* go.dg/arrayclear.go: Add a comment explaining why adding a
comment was necessary to work around a dejagnu bug.
* c-c++-common/diagnostic-units-1.c: New test.
* c-c++-common/diagnostic-units-2.c: New test.
* c-c++-common/diagnostic-units-3.c: New test.
* c-c++-common/diagnostic-units-4.c: New test.
* c-c++-common/diagnostic-units-5.c: New test.
* c-c++-common/diagnostic-units-6.c: New test.
* c-c++-common/diagnostic-units-7.c: New test.
* c-c++-common/diagnostic-units-8.c: New test.
With C++ module header units it becomes important to distinguish
between macros defined in forced headers (& commandline & builtins)
from those defined in the header file being processed. We weren't
making that easy because we treated the builtins and command-line
locations somewhat file-like, with incrementing line numbers, and
showing them as included from line 1 of the main file. This patch does
3 things:
0) extend the idiom that 'line 0' of a file means 'the file as a whole'
1) builtins and command-line macros are shown as-if included from line zero.
2) when emitting preprocessed output we keep resetting the line number
so that re-reading that preprocessed output will get the same set of
locations for the command line etc.
For instance the new c-c++-common/cpp/line-2.c test, now emits
In file included from <command-line>:
./line-2.h:4:2: error: #error wrong
4 | #error wrong
| ^~~~~
line-2.c:3:11: error: macro "bill" passed 1 arguments, but takes just 0
3 | int bill(1);
| ^
In file included from <command-line>:
./line-2.h:3: note: macro "bill" defined here
3 | #define bill() 2
|
Before it told you about including from <command-line>:31.
the preprocessed output looks like:
...
(There's a new optimization in do_line_marker to stop each of these
line markers causing a new line map. We can simply rewind the
location, and keep using the same line map.)
libcpp/
* directives.c (do_linemarker): Optimize rewinding to line zero.
* files.c (_cpp_stack_file): Start on line zero when about to inject
headers.
(cpp_push_include, cpp_push_default_include): Use highest_line as
the location.
* include/cpplib.h (cpp_read_main_file): Add injecting parm.
* init.c (cpp_read_main_file): Likewise, inform _cpp_stack_file.
* internal.h (enum include_type): Add IT_MAIN_INJECT.
gcc/c-family/
* c-opts.c (c_common_post_options): Add 'injecting' arg to
cpp_read_main_file.
(c_finish_options): Add linemap_line_start calls for builtin and cmd
maps. Force token position to line_table's highest line.
* c-ppoutput.c (print_line_1): Refactor, print line zero.
(cb_define): Always increment source line.
gcc/testsuite/
* c-c++-common/cpp/line-2.c: New.
* c-c++-common/cpp/line-2.h: New.
* c-c++-common/cpp/line-3.c: New.
* c-c++-common/cpp/line-4.c: New.
* c-c++-common/cpp/line-4.h: New.
AIX supports "FAT" libraries containing 32 bit and 64 bit objects
(similar to Darwin), but commands for manipulating libraries do not
default to accept both 32 bit and 64 bit object files. While updating
the AIX configuration to support building and running GCC as a 64 bit
application, I have encountered some build libraries that hard code
AR=ar instead of testing the environment.
This patch adds AR_CHECK_TOOL(AR, ar) to configure.ac for the two
libraries and updates Makefile.in to accept the substitution.
2020-05-23 David Edelsohn <dje.gcc@gmail.com>
libcpp/ChangeLog:
* Makefile.in (AR): Substitute @AR@.
* configure.ac (CHECK_PROG AR): New.
* configure: Regenerate.
libdecnumber/ChangeLog:
* Makefile.in (AR): Substitute @AR@.
* configure.ac (CHECK_PROG AR): New.
* configure: Regenerate.
_cpp_find_file has 3 bool arguments, at most one of which is ever set.
Ripe for replacing with a 4-state enum. Also, this is C++, so
'typedef struct Foo Foo' is unnecessary.
* internal.h (typedef _cpp_file): Delete, unnecessary in C++.
(enum _cpp_find_file_kind): New.
(_cpp_find_file): Use it, not 3 bools.
* files.c (_cpp_find_file): Use _cpp_find_file_kind enum, not
bools.
(cpp_make_system_header): Break overly long line.
(_cpp_stack_include, _cpp_fake_include)
(_cpp_do_file_change, _cpp_compare_file_date, _cpp_has_header): Adjust.
* init.c (cpp_read_main): Adjust _cpp_find_file call.
This fixes a bunch of poorly formatted decls, marks some getters as
PURE, deletes some C-relevant bool hackery, and finally uses a
passed-in location rather than deducing a closely-related but not
necessarily the same location.
* include/cpplib.h (cpp_get_otions, cpp_get_callbacks)
(cpp_get_deps): Mark as PURE.
* include/line-map.h (get_combined_adhoc_loc)
(get_location_from_adhoc_loc, get_pure_location): Reformat decls.
* internal.h (struct lexer_state): Clarify comment.
* system.h: Remove now-unneeded bool hackery.
* files.c (_cpp_find_file): Store LOC not highest_location.
pr95149 is a false positive static analysis checker. But it
encouranged me to fix raw string lexing, which does contain a
complicated macro and pointers to local variables. The
reimplementation does away with that macro. Part of the complication
is we need to undo some of the fresh line processing -- trigraph notes
and escaped line continuations. But the undone characters need to go
through the raw string processing, as they can legitimately be part of
the prefix marker. however, in this reformulation we only process one
line marker at a time[*], so there's a limited number of undone
characters. We can arrange the buffering to make sure we don't split
such an append sequence, and then simply take the characters from the
append buffer.
The prefix scanner had a switch statement, which I discovered was not
optimized as well as an if of a bunch of explicit comparisons (pr
95208 filed).
Finally I adjusted the failure mode. When we get a bad prefix, we lex
up until the next '"', thus often swallowing the whole raw string.
Previously we'd bail and then the lexer would usually generate stupid
tokens, particularly when meeting the ending '"'.
libcpp/
* lex.c (struct lit_accum): New.
(bufring_append): Replace by lit_accum::append.
(lex_raw_string): Reimplement, using fragments of the old version.
(lex_string): Adjust lex_raw_string call.
gcc/testsuite/
* c-c++-common/raw-string-14.c: Adjust errors.
* c-c++-common/raw-string-16.c: Likewise.
* c-c++-common/raw-string-5.c: Likewise.
This was another latent case of us losing an EOF token, but succeeding
anyway. Since my patch to make us pay more attention to EOFs it came
to light. We also need to keep the EOF if we fall off the end of the
main file. Forced includes look like regular nested includes at this
point.
PR preprocessor/95182
libcpp/
* macro.c (collect_args): Preserve EOFif we fell out of the main
file.
(cpp_get_token_1): Reformat a couple of short lines.
C++20 isn't final quite yet, but all that remains is formalities, so let's
go ahead and change all the references.
I think for the next C++ standard we can just call it C++23 rather than
C++2b, since the committee has been consistent about time-based releases
rather than feature-based.
gcc/c-family/ChangeLog
2020-05-13 Jason Merrill <jason@redhat.com>
* c.opt (std=c++20): Make c++2a the alias.
(std=gnu++20): Likewise.
* c-common.h (cxx_dialect): Change cxx2a to cxx20.
* c-opts.c: Adjust.
* c-cppbuiltin.c: Adjust.
* c-ubsan.c: Adjust.
* c-warn.c: Adjust.
gcc/cp/ChangeLog
2020-05-13 Jason Merrill <jason@redhat.com>
* call.c, class.c, constexpr.c, constraint.cc, decl.c, init.c,
lambda.c, lex.c, method.c, name-lookup.c, parser.c, pt.c, tree.c,
typeck2.c: Change cxx2a to cxx20.
libcpp/ChangeLog
2020-05-13 Jason Merrill <jason@redhat.com>
* include/cpplib.h (enum c_lang): Change CXX2A to CXX20.
* init.c, lex.c: Adjust.
My recent C++ parser change to pay attention to EOF location uncovered
a separate bug. The preprocesor's EOF logic would set the EOF
location to be the beginning of the last line of text in the file --
not the 'line' after that, which contains no characters. Mostly.
This fixes things so that when we attempt to read the last line of the
main file, we don't pop the buffer until the tokenizer has a chance to
create an EOF token with the correct location information. It is then
responsible for popping the buffer. As it happens, raw string literal
tokenizing contained a bug -- it would increment the line number
prematurely, because it cached buffer->cur in a local variable, but
checked buffer->cur before updating it to figure out if it was at end
of file. We fix up that too.
The EOF token intentionally doesn't have a column number -- it's not a
position on a line, it's a non-existant line.
The testsuite churn is just correcting the EOF location diagnostics.
libcpp/
PR preprocessor/95013
* lex.c (lex_raw_string): Process line notes before incrementing.
Correct incrementing condition. Adjust for new
_cpp_get_fresh_line EOF behaviour.
(_cpp_get_fresh_line): Do not pop buffer at EOF, increment line
instead.
(_cpp_lex_direct): Adjust for new _cpp_get_fresh_line behaviour.
(cpp_directive_only_process): Assert we got a fresh line.
* traditional.c (_cpp_read_logical_line_trad): Adjust for new
_cpp_get_fresh_line behaviour.
gcc/testsuite/
* c-c++-common/goacc/pr79428-1.c: Adjust EOF diagnostic location.
* c-c++-common/gomp/pr79428-2.c: Likewise.
* g++.dg/cpp0x/decltype63.C: Likewise.
* g++.dg/cpp0x/gen-attrs-64.C: Likewise.
* g++.dg/cpp0x/pr68726.C: Likewise.
* g++.dg/cpp0x/pr78341.C: Likewise.
* g++.dg/cpp1y/pr65202.C: Likewise.
* g++.dg/cpp1y/pr65340.C: Likewise.
* g++.dg/cpp1y/pr68578.C: Likewise.
* g++.dg/cpp1z/class-deduction44.C: Likewise.
* g++.dg/diagnostic/unclosed-extern-c.C: Likewise.
* g++.dg/diagnostic/unclosed-function.C: Likewise.
* g++.dg/diagnostic/unclosed-namespace.C: Likewise.
* g++.dg/diagnostic/unclosed-struct.C: Likewise.
* g++.dg/ext/pr84598.C: Likewise.
* g++.dg/other/switch4.C: Likewise.
* g++.dg/parse/attr4.C: Likewise.
* g++.dg/parse/cond4.C: Likewise.
* g++.dg/parse/crash10.C: Likewise.
* g++.dg/parse/crash18.C: Likewise.
* g++.dg/parse/crash27.C: Likewise.
* g++.dg/parse/crash34.C: Likewise.
* g++.dg/parse/crash35.C: Likewise.
* g++.dg/parse/crash52.C: Likewise.
* g++.dg/parse/crash59.C: Likewise.
* g++.dg/parse/crash61.C: Likewise.
* g++.dg/parse/crash67.C: Likewise.
* g++.dg/parse/error14.C: Likewise.
* g++.dg/parse/error56.C: Likewise.
* g++.dg/parse/invalid1.C: Likewise.
* g++.dg/parse/parameter-declaration-1.C: Likewise.
* g++.dg/parse/parser-pr28152-2.C: Likewise.
* g++.dg/parse/parser-pr28152.C: Likewise.
* g++.dg/parse/pr68722.C: Likewise.
* g++.dg/pr46852.C: Likewise.
* g++.dg/pr46868.C: Likewise.
* g++.dg/template/crash115.C: Likewise.
* g++.dg/template/crash43.C: Likewise.
* g++.dg/template/crash90.C: Likewise.
* g++.dg/template/error-recovery1.C: Likewise.
* g++.dg/template/error57.C: Likewise.
* g++.old-deja/g++.other/crash31.C: Likewise.
* gcc.dg/empty-source-2.c: Likewise.
* gcc.dg/empty-source-3.c: Likewise.
* gcc.dg/noncompile/pr30552-3.c: Likewise.
* gcc.dg/noncompile/pr35447-1.c: Likewise.
* gcc.dg/pr20245-1.c: Likewise.
* gcc.dg/pr28419.c: Likewise.
* gcc.dg/rtl/truncated-rtl-file.c: Likewise.
* gcc.dg/unclosed-init.c: Likewise.
* obj-c++.dg/property/property-neg-6.mm: Likewise.
* obj-c++.dg/syntax-error-10.mm: Likewise.
* obj-c++.dg/syntax-error-8.mm: Likewise.
* obj-c++.dg/syntax-error-9.mm: Likewise.
Since on Intel CET enabled host, dlopen in Intel CET enabled applications
fails on shared libraries which aren't Intel CET enabled, compile with
-fcf-protection on Intel CET enabled host when jit is enabled to enable
Intel CET on libgccjit.
* Makefile.in (CET_HOST_FLAGS): New.
(COMPILER): Add $(CET_HOST_FLAGS).
* configure.ac: Add GCC_CET_HOST_FLAGS(CET_HOST_FLAGS) and
AC_SUBST(CET_HOST_FLAGS). Clear CET_HOST_FLAGS if jit isn't
enabled.
* aclocal.m4: Regenerated.
* configure: Likewise.
The existing directives-only code (a) punched a hole through the
libcpp interface and (b) didn't support raw string literals. This
reimplements this preprocessing mode. I added a proper callback
interface, and adjusted c-ppoutput to use it. Sadly I cannot get rid
of the libcpp/internal.h include for unrelated reasons.
The new scanner is in lex.x, and works doing some backwards scanning
when it finds a charater of interest. This reduces the number of
cases one has to deal with in forward scanning. It may have different
failure mode than forward scanning on bad tokenization.
Finally, Moved some cpp tests from the c-specific dg.gcc/cpp directory
to the c-c++-common/cpp shared directory,
libcpp/
* directives-only.c: Delete.
* Makefile.in (libcpp_a_OBJS, libcpp_a_SOURCES): Remove it.
* include/cpplib.h (enum CPP_DO_task): New enum.
(cpp_directive_only_preprocess): Declare.
* internal.h (_cpp_dir_only_callbacks): Delete.
(_cpp_preprocess_dir_only): Delete.
* lex.c (do_peek_backslask, do_peek_next, do_peek_prev): New.
(cpp_directives_only_process): New implementation.
gcc/c-family/
Reimplement directives only processing.
* c-ppoutput.c (token_streamer): Ne.
(directives_only_cb): New. Swallow ...
(print_lines_directives_only): ... this.
(scan_translation_unit_directives_only): Reimplment using the
published interface.
gcc/testsuite/
* gcc.dg/cpp/counter-[23].c: Move to c-c+_-common/cpp.
* gcc.dg/cpp/dir-only-*: Likewise.
* c-c++-common/cpp/dir-only-[78].c: New.
I've noticed we claim in cxx-status.html that we implement P1042R1,
but it seems we don't implement any of the changes from there.
The following patch implements just the change that __VA_OPT__ determines
whether to expand to nothing or the enclosed tokens no longer based on
whether there were any tokens passed to __VA_ARGS__, but whether __VA_ARGS__
expands to any tokens (from testing apparently it has to be non-CPP_PADDING
tokens).
I'm afraid I'm completely lost about the padding preservation/removal
changes that are also in the paper, so haven't touched that part.
2020-02-14 Jakub Jelinek <jakub@redhat.com>
Partially implement P1042R1: __VA_OPT__ wording clarifications
PR preprocessor/92319
* macro.c (expand_arg): Move declarations before vaopt_state
definition.
(class vaopt_state): Move enum update_type definition earlier. Remove
m_allowed member, add m_arg and m_update members.
(vaopt_state::vaopt_state): Change last argument from bool any_args
to macro_arg *arg, initialize m_arg and m_update instead of m_allowed.
(vaopt_state::update): When bumping m_state from 1 to 2 and m_update
is ERROR, determine if __VA_ARGS__ expansion has any non-CPP_PADDING
tokens and set m_update to INCLUDE if it has any, DROP otherwise.
Return m_update instead of m_allowed ? INCLUDE : DROP in m_state >= 2.
(replace_args, create_iso_definition): Adjust last argument to
vaopt_state ctor.
* c-c++-common/cpp/va-opt-4.c: New test.
The standard says http://eel.is/c++draft/cpp.cond#7.sentence-2 that
__has_include can't appear at arbitrary places in the source. As we have
not recognized __has_include* outside of preprocessing directives in the
past, accepting it there now would be a regression. The patch does still
allow it in #define if it is then used in preprocessing directives, I guess
that use isn't strictly valid either, but clang seems to accept it.
2020-02-04 Jakub Jelinek <jakub@redhat.com>
* macro.c (builtin_has_include): Diagnose __has_include* use outside
of preprocessing directives.
* c-c++-common/cpp/has-include-1.c: New test.
* c-c++-common/cpp/has-include-next-1.c: New test.
* c-c++-common/gomp/has-include-1.c: New test.
Some of the following testcases ICE, because one of the cpp_get_token
calls in builtin_has_include reads the CPP_EOF token but the caller isn't
aware that CPP_EOF has been reached and will do another cpp_get_token.
get_token_no_padding is something that is use by the
has_attribute/has_builtin callbacks, which will first peek and will not
consume CPP_EOF (but will consume other tokens). The !SEEN_EOL ()
check on the other side doesn't work anymore and isn't really needed,
as we don't consume the EOF. The change adds one further error to the
pr88974.c testcase, if we wanted just one error per __has_include,
we could add some boolean whether we've emitted errors already and
only emit the first one we encounter (not implemented).
2020-02-04 Jakub Jelinek <jakub@redhat.com>
PR preprocessor/93545
* macro.c (cpp_get_token_no_padding): New function.
(builtin_has_include): Use it instead of cpp_get_token. Don't check
SEEN_EOL.
* c-c++-common/cpp/pr88974.c: Expect another diagnostics during error
recovery.
* c-c++-common/cpp/pr93545-1.c: New test.
* c-c++-common/cpp/pr93545-2.c: New test.
* c-c++-common/cpp/pr93545-3.c: New test.
* c-c++-common/cpp/pr93545-4.c: New test.
This commit:
commit e7c26e04b2 (tjteru/master)
Date: Wed Jan 22 14:54:26 2020 +0000
gcc: Add new configure options to allow static libraries to be selected
contains a couple of issues. First I failed to correctly regenerate
all of the configure files it should have done. Second, there was a
mistake in lib-link.m4, one of the conditions didn't use pure sh
syntax, I wrote this:
if x$lib_type = xauto || x$lib_type = xshared; then
When I should have written this:
if test "x$lib_type" = "xauto" || test "x$lib_type" = "xshared"; then
These issues were raised on the mailing list in these messages:
https://gcc.gnu.org/ml/gcc-patches/2020-01/msg01827.htmlhttps://gcc.gnu.org/ml/gcc-patches/2020-01/msg01921.html
config/ChangeLog:
* lib-link.m4 (AC_LIB_LINKFLAGS_BODY): Update shell syntax.
gcc/ChangeLog:
* configure: Regenerate.
intl/ChangeLog:
* configure: Regenerate.
libcpp/ChangeLog:
* configure: Regenerate.
libstdc++-v3/ChangeLog:
* configure: Regenerate.
The clever hack of '#define __has_include __has_include' breaks -dD
and -fdirectives-only, because that emits definitions. This turns
__has_include into a proper builtin macro. Thus it's never emitted
via -dD, and because use outside of directive processing is undefined,
we can just expand it anywhere.
PR preprocessor/93452
* internal.h (struct spec_nodes): Drop n__has_include{,_next}.
* directives.c (lex_macro_node): Don't check __has_include redef.
* expr.c (eval_token): Drop __has_include eval.
(parse_has_include): Move to ...
* macro.c (builtin_has_include): ... here.
(_cpp_builtin_macro_text): Eval __has_include{,_next}.
* include/cpplib.h (enum cpp_builtin_type): Add BT_HAS_INCLUDE{,_NEXT}.
* init.c (builtin_array): Add them.
(cpp_init_builtins): Drop __has_include{,_next} init here ...
* pch.c (cpp_read_state): ... and here.
* traditional.c (enum ls): Drop has_include states ...
(_cpp_scan_out_logical_line): ... and here.
The motivation behind this change is to make it easier for a user to
link against static libraries on a target where dynamic libraries are
the default library type (for example GNU/Linux).
Further, my motivation is really for linking libraries into GDB,
however, the binutils-gdb/config/ directory is a copy of gcc/config/
so changes for GDB need to be approved by the GCC project first.
After making this change in the gcc/config/ directory I've run
autoreconf on all of the configure scripts in the GCC tree and a
couple have been updated, so I'll use one of these to describe what my
change does.
Consider libcpp, this library links against libiconv. Currently if
the user builds on a system with both static and dynamic libiconv
installed then autotools will pick up the dynamic libiconv by
default. This is almost certainly the right thing to do.
However, if the user wants to link against static libiconv then things
are a little harder, they could remove the dynamic libiconv from their
system, but this is probably a bad idea (other things might depend on
that library), or the user can build their own version of libiconv,
install it into a unique prefix, and then configure gcc using the
--with-libiconv-prefix=DIR flag. This works fine, but is somewhat
annoying, the static library available, I just can't get autotools to
use it.
My change then adds a new flag --with-libiconv-type=TYPE, where type
is either auto, static, or shared. The default auto, ensures we keep
the existing behaviour unchanged.
If the user configures with --with-libiconv-type=static then the
configure script will ignore any dynamic libiconv it finds, and will
only look for a static libiconv, if no static libiconv is found then
the configure will continue as though there is no libiconv at all
available.
Similarly a user can specify --with-libiconv-type=shared and force the
use of shared libiconv, any static libiconv will be ignored.
As I've implemented this change within the AC_LIB_LINKFLAGS_BODY macro
then only libraries configured using the AC_LIB_LINKFLAGS or
AC_LIB_HAVE_LINKFLAGS macros will gain the new configure flag.
If this is accepted into GCC then there will be follow on patches for
binutils and GDB to regenerate some configure scripts in those
projects.
For GCC only two configure scripts needed updated after this commit,
libcpp and libstdc++-v3, both of which link against libiconv.
config/ChangeLog:
* lib-link.m4 (AC_LIB_LINKFLAGS_BODY): Add new
--with-libXXX-type=... option. Use this to guide the selection of
either a shared library or a static library.
libcpp/ChangeLog:
* configure: Regenerate.
libstdc++-v3/ChangeLog:
* configure: Regenerate.
I noticed, but ignored this code when addressing p80005, but having
fixed up defined(X) on the modules branch, I could see where it came
from, and it's obviously wrong as we've just pulled out a string
contant from the token.
* expr.c (parse_has_include): Remove bogus controlling macro code.
__has_include is funky in that it is macro-like from the POV of #ifdef and
friends, but lexes its parenthesize argument #include-like. We were
failing the second part of that, because we used a forwarding macro to an
internal name, and hence always lexed the argument in macro-parameter
context. We componded that by not setting the right flag when lexing, so
it didn't even know. Mostly users got lucky.
This reimplements the handline.
1) Remove the forwarding, but declare object-like macros that
expand to themselves. This satisfies the #ifdef requirement
2) Correctly set angled_brackets when lexing the parameter. This tells
the lexer (a) <...> is a header name and (b) "..." is too (not a string).
3) Remove the in__has_include lexer state, just tell find_file that that's
what's happenning, so it doesn't emit an error.
We lose the (undocumented) ability to #undef __has_include. That may well
have been an accident of implementation. There are no tests for it.
We gain __has_include behaviour for all users of the preprocessors -- not
just the C-family ones that defined a forwarding macro.
libcpp/
PR preprocessor/80005
* include/cpplib.h (BT_HAS_ATTRIBUTE): Fix comment.
* internal.h (struct lexer_state): Delete in__has_include field.
(struct spec_nodes): Rename n__has_include{,_next}__ fields.
(_cpp_defined_macro_p): New.
(_cpp_find_file): Add has_include parm.
* directives.c (lex_macro_node): Combine defined,
__has_inline{,_next} checking.
(do_ifdef, do_ifndef): Use _cpp_defined_macro_p.
(_cpp_init_directives): Refactor.
* expr.c (parse_defined): Use _cpp_defined_macro_p.
(eval_token): Adjust parse_has_include calls.
(parse_has_include): Add OP parameter. Reimplement.
* files.c (_cpp_find_file): Add HAS_INCLUDE parm. Use it to
inhibit error message.
(_cpp_stack_include): Adjust _cpp_find_file call.
(_cpp_fake_include, _cpp_compare_file_date): Likewise.
(open_file_failed): Remove in__has_include check.
(_cpp_has_header): Adjust _cpp_find_file call.
* identifiers.c (_cpp_init_hashtable): Don't init
__has_include{,_next} here ...
* init.c (cpp_init_builtins): ... init them here. Define as
macros.
(cpp_read_main_file): Adjust _cpp_find_file call.
* pch.c (cpp_read_state): Adjust __has_include{,_next} access.
* traditional.c (_cpp_scan_out_locgical_line): Likewise.
gcc/c-family/
PR preprocessor/80005
* c-cppbuiltins.c (c_cpp_builtins): Don't define __has_include{,_next}.
gcc/testsuite/
PR preprocessor/80005
* g++.dg/cpp1y/feat-cxx14.C: Adjust.
* g++.dg/cpp1z/feat-cxx17.C: Adjust.
* g++.dg/cpp2a/feat-cxx2a.C: Adjust.
* g++.dg/cpp/pr80005.C: New.
the preprocessor evaluator has a skip_eval counter, but we weren't
checking it after parsing has_include(foo), but before looking for
foo. Resulting in unnecessary io for 'FALSE_COND && has_include <foo>'
PR preprocessor/93306
* expr.c (parse_has_include): Refactor. Check skip_eval before
looking.
This suppresses an array out of bounds warning in mkdeps.c as proposed
by Martin Sebor in the bugzilla.
array subscript 2 is outside array bounds of ‘const char [2]’
Since this warning does occur during bootstrap it currently breaks
werror builds on IBM Z.
The problem can be reproduced also on x86_64 by changing the inlining
threshold using: --param max-inline-insns-auto=80
Bootstrapped and regression tested on x86_64 and IBM Z.
libcpp/ChangeLog:
2020-01-16 Andreas Krebbel <krebbel@linux.ibm.com>
PR tree-optimization/92176
* mkdeps.c (deps_add_default_target): Avoid calling apply_vpath to
suppress an array out of bounds warning.
This patch adds support for associating a "diagnostic_path" with a
diagnostic: a sequence of events predicted by the compiler that leads to
the problem occurring, with their locations in the user's source,
text descriptions, and stack information (for handling interprocedural
paths).
For example, the following (hypothetical) error has a 3-event
intraprocedural path:
test.c: In function 'demo':
test.c:29:5: error: passing NULL as argument 1 to 'PyList_Append' which
requires a non-NULL parameter
29 | PyList_Append(list, item);
| ^~~~~~~~~~~~~~~~~~~~~~~~~
'demo': events 1-3
|
| 25 | list = PyList_New(0);
| | ^~~~~~~~~~~~~
| | |
| | (1) when 'PyList_New' fails, returning NULL
| 26 |
| 27 | for (i = 0; i < count; i++) {
| | ~~~
| | |
| | (2) when 'i < count'
| 28 | item = PyLong_FromLong(random());
| 29 | PyList_Append(list, item);
| | ~~~~~~~~~~~~~~~~~~~~~~~~~
| | |
| | (3) when calling 'PyList_Append', passing NULL from (1) as argument 1
|
The patch adds a new "%@" format code for printing event IDs, so that
in the above, the description of event (3) mentions event (1), showing
the user where the bogus NULL value comes from (the event IDs are
colorized to draw the user's attention to them).
There is a separation between data vs presentation: the above shows how
the diagnostic-printing code has consolidated the path into a single run
of events, since all the events are near each other and within the same
function; more complicated examples (such as interprocedural paths)
might be printed as multiple runs of events.
Examples of how interprocedural paths are printed can be seen in the
test suite (which uses a plugin to exercise the code without relying
on specific warnings using this functionality).
Other output formats include
- JSON,
- printing each event as a separate "note", and
- to not emit paths.
gcc/ChangeLog:
* Makefile.in (OBJS): Add tree-diagnostic-path.o.
* common.opt (fdiagnostics-path-format=): New option.
(diagnostic_path_format): New enum.
(fdiagnostics-show-path-depths): New option.
* coretypes.h (diagnostic_event_id_t): New forward decl.
* diagnostic-color.c (color_dict): Add "path".
* diagnostic-event-id.h: New file.
* diagnostic-format-json.cc (json_from_expanded_location): Make
non-static.
(json_end_diagnostic): Call context->make_json_for_path if it
exists and the diagnostic has a path.
(diagnostic_output_format_init): Clear context->print_path.
* diagnostic-path.h: New file.
* diagnostic-show-locus.c (colorizer::set_range): Special-case
when printing a run of events in a diagnostic_path so that they
all get the same color.
(layout::m_diagnostic_path_p): New field.
(layout::layout): Initialize it.
(layout::print_any_labels): Don't colorize the label text for an
event in a diagnostic_path.
(gcc_rich_location::add_location_if_nearby): Add
"restrict_to_current_line_spans" and "label" params. Pass the
former to layout.maybe_add_location_range; pass the latter
when calling add_range.
* diagnostic.c: Include "diagnostic-path.h".
(diagnostic_initialize): Initialize context->path_format and
context->show_path_depths.
(diagnostic_show_any_path): New function.
(diagnostic_path::interprocedural_p): New function.
(diagnostic_report_diagnostic): Call diagnostic_show_any_path.
(simple_diagnostic_path::num_events): New function.
(simple_diagnostic_path::get_event): New function.
(simple_diagnostic_path::add_event): New function.
(simple_diagnostic_event::simple_diagnostic_event): New ctor.
(simple_diagnostic_event::~simple_diagnostic_event): New dtor.
(debug): New overload taking a diagnostic_path *.
* diagnostic.def (DK_DIAGNOSTIC_PATH): New.
* diagnostic.h (enum diagnostic_path_format): New enum.
(json::value): New forward decl.
(diagnostic_context::path_format): New field.
(diagnostic_context::show_path_depths): New field.
(diagnostic_context::print_path): New callback field.
(diagnostic_context::make_json_for_path): New callback field.
(diagnostic_show_any_path): New decl.
(json_from_expanded_location): New decl.
* doc/invoke.texi (-fdiagnostics-path-format=): New option.
(-fdiagnostics-show-path-depths): New option.
(-fdiagnostics-color): Add "path" to description of default
GCC_COLORS; describe it.
(-fdiagnostics-format=json): Document how diagnostic paths are
represented in the JSON output format.
* gcc-rich-location.h (gcc_rich_location::add_location_if_nearby):
Add optional params "restrict_to_current_line_spans" and "label".
* opts.c (common_handle_option): Handle
OPT_fdiagnostics_path_format_ and
OPT_fdiagnostics_show_path_depths.
* pretty-print.c: Include "diagnostic-event-id.h".
(pp_format): Implement "%@" format code for printing
diagnostic_event_id_t *.
(selftest::test_pp_format): Add tests for "%@".
* selftest-run-tests.c (selftest::run_tests): Call
selftest::tree_diagnostic_path_cc_tests.
* selftest.h (selftest::tree_diagnostic_path_cc_tests): New decl.
* toplev.c (general_init): Initialize global_dc->path_format and
global_dc->show_path_depths.
* tree-diagnostic-path.cc: New file.
* tree-diagnostic.c (maybe_unwind_expanded_macro_loc): Make
non-static. Drop "diagnostic" param in favor of storing the
original value of "where" and re-using it.
(virt_loc_aware_diagnostic_finalizer): Update for dropped param of
maybe_unwind_expanded_macro_loc.
(tree_diagnostics_defaults): Initialize context->print_path and
context->make_json_for_path.
* tree-diagnostic.h (default_tree_diagnostic_path_printer): New
decl.
(default_tree_make_json_for_path): New decl.
(maybe_unwind_expanded_macro_loc): New decl.
gcc/c-family/ChangeLog:
* c-format.c (local_event_ptr_node): New.
(PP_FORMAT_CHAR_TABLE): Add entry for "%@".
(init_dynamic_diag_info): Initialize local_event_ptr_node.
* c-format.h (T_EVENT_PTR): New define.
gcc/testsuite/ChangeLog:
* gcc.dg/format/gcc_diag-10.c (diagnostic_event_id_t): New
typedef.
(test_diag): Add coverage of "%@".
* gcc.dg/plugin/diagnostic-path-format-default.c: New test.
* gcc.dg/plugin/diagnostic-path-format-inline-events-1.c: New test.
* gcc.dg/plugin/diagnostic-path-format-inline-events-2.c: New test.
* gcc.dg/plugin/diagnostic-path-format-inline-events-3.c: New test.
* gcc.dg/plugin/diagnostic-path-format-none.c: New test.
* gcc.dg/plugin/diagnostic-test-paths-1.c: New test.
* gcc.dg/plugin/diagnostic-test-paths-2.c: New test.
* gcc.dg/plugin/diagnostic-test-paths-3.c: New test.
* gcc.dg/plugin/diagnostic-test-paths-4.c: New test.
* gcc.dg/plugin/diagnostic_plugin_test_paths.c: New.
* gcc.dg/plugin/plugin.exp: Add the new plugin and test cases.
libcpp/ChangeLog:
* include/line-map.h (class diagnostic_path): New forward decl.
(rich_location::get_path): New accessor.
(rich_location::set_path): New function.
(rich_location::m_path): New field.
* line-map.c (rich_location::rich_location): Initialize m_path.
From-SVN: r280142
libcpp's label_text class wraps a text buffer, along with a flag to
determine if it "owns" the buffer.
The existing ctor exposed this directly, but I found it difficult
to remember the sense of flag, so this patch hides the ctor, in
favor of static member functions "borrow" and "take", to make
the effect on ownership explicit in the name.
gcc/c-family/ChangeLog:
* c-format.c (range_label_for_format_type_mismatch::get_text):
Replace label_text ctor called with true with label_text::take.
gcc/c/ChangeLog:
* c-objc-common.c (range_label_for_type_mismatch::get_text):
Replace label_text ctor calls.
gcc/cp/ChangeLog:
* error.c (range_label_for_type_mismatch::get_text): Replace
label_text ctor calls with label_text::borrow.
gcc/ChangeLog:
* gcc-rich-location.c
(maybe_range_label_for_tree_type_mismatch::get_text): Replace
label_text ctor call with label_text::borrow.
* gcc-rich-location.h (text_range_label::get_text): Replace
label_text ctor called with false with label_text::borrow.
libcpp/ChangeLog:
* include/line-map.h (label_text::label_text): Make private.
(label_text::borrow): New.
(label_text::take): New.
(label_text::take_or_copy): New.
From-SVN: r279153
contrib/ChangeLog
2019-12-09 Lewis Hyatt <lhyatt@gmail.com>
PR preprocessor/49973
* unicode/from_glibc/unicode_utils.py: Support script from
glibc (commit 464cd3) to extract character widths from Unicode data
files.
* unicode/from_glibc/utf8_gen.py: Likewise.
* unicode/UnicodeData.txt: Unicode v. 12.1.0 data file.
* unicode/EastAsianWidth.txt: Likewise.
* unicode/PropList.txt: Likewise.
* unicode/gen_wcwidth.py: New utility to generate
libcpp/generated_cpp_wcwidth.h with help from the glibc support
scripts and the Unicode data files.
* unicode/unicode-license.txt: Added.
* unicode/README: New explanatory file.
libcpp/ChangeLog
2019-12-09 Lewis Hyatt <lhyatt@gmail.com>
PR preprocessor/49973
* generated_cpp_wcwidth.h: New file generated by
../contrib/unicode/gen_wcwidth.py, supports new cpp_wcwidth function.
* charset.c (compute_next_display_width): New function to help
implement display columns.
(cpp_byte_column_to_display_column): Likewise.
(cpp_display_column_to_byte_column): Likewise.
(cpp_wcwidth): Likewise.
* include/cpplib.h (cpp_byte_column_to_display_column): Declare.
(cpp_display_column_to_byte_column): Declare.
(cpp_wcwidth): Declare.
(cpp_display_width): New function.
gcc/ChangeLog
2019-12-09 Lewis Hyatt <lhyatt@gmail.com>
PR preprocessor/49973
* input.c (location_compute_display_column): New function to help with
multibyte awareness in diagnostics.
(test_cpp_utf8): New self-test.
(input_c_tests): Call the new test.
* input.h (location_compute_display_column): Declare.
* diagnostic-show-locus.c: Pervasive changes to add multibyte awareness
to all classes and functions.
(enum column_unit): New enum.
(class exploc_with_display_col): New class.
(class layout_point): Convert m_column member to array m_columns[2].
(layout_range::contains_point): Add col_unit argument.
(test_layout_range_for_single_point): Pass new argument.
(test_layout_range_for_single_line): Likewise.
(test_layout_range_for_multiple_lines): Likewise.
(line_bounds::convert_to_display_cols): New function.
(layout::get_state_at_point): Add col_unit argument.
(make_range): Use empty filename rather than dummy filename.
(get_line_width_without_trailing_whitespace): Rename to...
(get_line_bytes_without_trailing_whitespace): ...this.
(test_get_line_width_without_trailing_whitespace): Rename to...
(test_get_line_bytes_without_trailing_whitespace): ...this.
(class layout): m_exploc changed to exploc_with_display_col from
plain expanded_location.
(layout::get_linenum_width): New accessor member function.
(layout::get_x_offset_display): Likewise.
(layout::calculate_linenum_width): New subroutine for the constuctor.
(layout::calculate_x_offset_display): Likewise.
(layout::layout): Use the new subroutines. Add multibyte awareness.
(layout::print_source_line): Add multibyte awareness.
(layout::print_line): Likewise.
(layout::print_annotation_line): Likewise.
(line_label::line_label): Likewise.
(layout::print_any_labels): Likewise.
(layout::annotation_line_showed_range_p): Likewise.
(get_printed_columns): Likewise.
(class line_label): Rename m_length to m_display_width.
(get_affected_columns): Rename to...
(get_affected_range): ...this; add col_unit argument and multibyte
awareness.
(class correction): Add m_affected_bytes and m_display_cols
members. Rename m_len to m_byte_length for clarity. Add multibyte
awareness throughout.
(correction::insertion_p): Add multibyte awareness.
(correction::compute_display_cols): New function.
(correction::ensure_terminated): Use new member name m_byte_length.
(line_corrections::add_hint): Add multibyte awareness.
(layout::print_trailing_fixits): Likewise.
(layout::get_x_bound_for_row): Likewise.
(test_one_liner_simple_caret_utf8): New self-test analogous to the one
with _utf8 suffix removed, testing multibyte awareness.
(test_one_liner_caret_and_range_utf8): Likewise.
(test_one_liner_multiple_carets_and_ranges_utf8): Likewise.
(test_one_liner_fixit_insert_before_utf8): Likewise.
(test_one_liner_fixit_insert_after_utf8): Likewise.
(test_one_liner_fixit_remove_utf8): Likewise.
(test_one_liner_fixit_replace_utf8): Likewise.
(test_one_liner_fixit_replace_non_equal_range_utf8): Likewise.
(test_one_liner_fixit_replace_equal_secondary_range_utf8): Likewise.
(test_one_liner_fixit_validation_adhoc_locations_utf8): Likewise.
(test_one_liner_many_fixits_1_utf8): Likewise.
(test_one_liner_many_fixits_2_utf8): Likewise.
(test_one_liner_labels_utf8): Likewise.
(test_diagnostic_show_locus_one_liner_utf8): Likewise.
(test_overlapped_fixit_printing_utf8): Likewise.
(test_overlapped_fixit_printing): Adapt for changes to
get_affected_columns, get_printed_columns and class corrections.
(test_overlapped_fixit_printing_2): Likewise.
(test_linenum_sep): New constant.
(test_left_margin): Likewise.
(test_offset_impl): Helper function for new test.
(test_layout_x_offset_display_utf8): New test.
(diagnostic_show_locus_c_tests): Call new tests.
gcc/testsuite/ChangeLog:
2019-12-09 Lewis Hyatt <lhyatt@gmail.com>
PR preprocessor/49973
* gcc.dg/plugin/diagnostic_plugin_test_show_locus.c
(test_show_locus): Tweak so that expected output is the same as
before the diagnostic-show-locus.c changes.
* gcc.dg/cpp/pr66415-1.c: Likewise.
From-SVN: r279137
C2x adds u8'' character constants to C. This patch adds the
corresponding GCC support.
Most of the support was already present for C++ and just needed
enabling for C2x. However, in C2x these constants have type unsigned
char, which required corresponding adjustments in the compiler and the
preprocessor to give them that type for C.
For C, it seems clear to me that having type unsigned char means the
constants are unsigned in the preprocessor (and thus treated as having
type uintmax_t in #if conditionals), so this patch implements that. I
included a conditional in the libcpp change to avoid affecting
signedness for C++, but I'm not sure if in fact these constants should
also be unsigned in the preprocessor for C++ in which case that
!CPP_OPTION (pfile, cplusplus) conditional would not be needed.
Bootstrapped with no regressions on x86_64-pc-linux-gnu.
gcc/c:
* c-parser.c (c_parser_postfix_expression)
(c_parser_check_literal_zero): Handle CPP_UTF8CHAR.
* gimple-parser.c (c_parser_gimple_postfix_expression): Likewise.
gcc/c-family:
* c-lex.c (lex_charconst): Make CPP_UTF8CHAR constants unsigned
char for C.
gcc/testsuite:
* gcc.dg/c11-utf8char-1.c, gcc.dg/c2x-utf8char-1.c,
gcc.dg/c2x-utf8char-2.c, gcc.dg/c2x-utf8char-3.c,
gcc.dg/gnu2x-utf8char-1.c: New tests.
libcpp:
* charset.c (narrow_str_to_charconst): Make CPP_UTF8CHAR constants
unsigned for C.
* init.c (lang_defaults): Set utf8_char_literals for GNUC2X and
STDC2X.
From-SVN: r278265
PR c++/91370 - Implement P1041R4 and P1139R2 - Stronger Unicode reqs
* charset.c (narrow_str_to_charconst): Add TYPE argument. For
CPP_UTF8CHAR diagnose whenever number of chars is > 1, using
CPP_DL_ERROR instead of CPP_DL_WARNING.
(wide_str_to_charconst): For CPP_CHAR16 or CPP_CHAR32, use
CPP_DL_ERROR instead of CPP_DL_WARNING when multiple char16_t
or char32_t chars are needed.
(cpp_interpret_charconst): Adjust narrow_str_to_charconst caller.
* g++.dg/cpp1z/utf8-neg.C: Expect errors rather than -Wmultichar
warnings.
* g++.dg/ext/utf16-4.C: Expect errors rather than warnings.
* g++.dg/ext/utf32-4.C: Likewise.
* g++.dg/cpp2a/ucn2.C: New test.
From-SVN: r277929
There are three major pieces to this support: scalar operator<=>,
synthesis of comparison operators, and rewritten/reversed overload
resolution (e.g. a < b becomes 0 > b <=> a).
Unlike other defaulted functions, where we use synthesized_method_walk to
semi-simulate what the definition of the function will be like, this patch
determines the characteristics of a comparison operator by trying to define
it.
My handling of non-dependent rewritten operators in templates can still use
some work: build_min_non_dep_op_overload can't understand the rewrites and
crashes, so I'm avoiding it for now by clearing *overload. This means we'll
do name lookup again at instantiation time, which can incorrectly mean a
different result. I'll poke at this more in stage 3.
I'm leaving out a fourth section ("strong structural equality") even though
I've implemented it, because it seems likely to change radically tomorrow.
Thanks to Tim van Deurzen and Jakub for implementing lexing of the <=>
operator, and Jonathan for the initial <compare> header.
gcc/cp/
* cp-tree.h (struct lang_decl_fn): Add maybe_deleted bitfield.
(DECL_MAYBE_DELETED): New.
(enum special_function_kind): Add sfk_comparison.
(LOOKUP_REWRITTEN, LOOKUP_REVERSED): New.
* call.c (struct z_candidate): Add rewritten and reversed methods.
(add_builtin_candidate): Handle SPACESHIP_EXPR.
(add_builtin_candidates): Likewise.
(add_candidates): Don't add a reversed candidate if the parms are
the same.
(add_operator_candidates): Split out from build_new_op_1. Handle
rewritten and reversed candidates.
(add_candidate): Swap conversions of reversed candidate.
(build_new_op_1): Swap them back. Build a second operation for
rewritten candidates.
(extract_call_expr): Handle rewritten calls.
(same_fn_or_template): New.
(joust): Handle rewritten and reversed candidates.
* class.c (add_implicitly_declared_members): Add implicit op==.
(classtype_has_op, classtype_has_defaulted_op): New.
* constexpr.c (cxx_eval_binary_expression): Handle SPACESHIP_EXPR.
(cxx_eval_constant_expression, potential_constant_expression_1):
Likewise.
* cp-gimplify.c (genericize_spaceship): New.
(cp_genericize_r): Use it.
* cp-objcp-common.c (cp_common_init_ts): Handle SPACESHIP_EXPR.
* decl.c (finish_function): Handle deleted function.
* decl2.c (grokfield): SET_DECL_FRIEND_CONTEXT on defaulted friend.
(mark_used): Check DECL_MAYBE_DELETED. Remove assumption that
defaulted functions are non-static members.
* error.c (dump_expr): Handle SPACESHIP_EXPR.
* method.c (type_has_trivial_fn): False for sfk_comparison.
(enum comp_cat_tag, struct comp_cat_info_t): New types.
(comp_cat_cache): New array variable.
(lookup_comparison_result, lookup_comparison_category)
(is_cat, cat_tag_for, spaceship_comp_cat)
(spaceship_type, genericize_spaceship)
(common_comparison_type, early_check_defaulted_comparison)
(comp_info, build_comparison_op): New.
(synthesize_method): Handle sfk_comparison. Handle deleted.
(get_defaulted_eh_spec, maybe_explain_implicit_delete)
(explain_implicit_non_constexpr, implicitly_declare_fn)
(defaulted_late_check, defaultable_fn_check): Handle sfk_comparison.
* name-lookup.c (get_std_name_hint): Add comparison categories.
* tree.c (special_function_p): Add sfk_comparison.
* typeck.c (cp_build_binary_op): Handle SPACESHIP_EXPR.
2019-11-05 Tim van Deurzen <tim@kompiler.org>
Add new tree code for the spaceship operator.
gcc/cp/
* cp-tree.def: Add new tree code.
* operators.def: New binary operator.
* parser.c: Add new token and tree code.
libcpp/
* cpplib.h: Add spaceship operator for C++.
* lex.c: Implement conditional lexing of spaceship operator for C++20.
2019-11-05 Jonathan Wakely <jwakely@redhat.com>
libstdc++-v3/
* libsupc++/compare: New header.
* libsupc++/Makefile.am (std_HEADERS): Add compare.
* include/std/version: Define __cpp_lib_three_way_comparison.
* include/std/functional: #include <compare>.
From-SVN: r277865
https://gcc.gnu.org/ml/gcc-patches/2019-10/msg01080.html
looking up a line map takes a non-constant line_maps object, which is confusing.
This makes the caching fields mutable, so permits a constant object, as one might expect for a lookup.
* include/line-map.h (struct maps_info_ordinary): Make cache
mutable.
(struct maps_info_macro): Likewise.
(LINEMAPS_CACHE): Remove non-ref accessor. Constify ref accessor.
(LINEMAPS_ORDINARY_CACHE, LINEMAPS_MACRO_CACHE): Likewise.
(LINEMAPS_ORDINARY_MAP_AT, LINEMAPS_MACRO_MAP_AT): Use
LINEMAPS_USED and LINEMAPS_MAP_AT.
(linemap_lookup): Constify line_map arg.
linemap.c (linemap_ordinary_map_lookup, linemap_macro_map_lookup):
Constify line_map arg.
From-SVN: r276994
ISO C2x adds decimal floating point as an optional standard feature.
This patch accordingly makes GCC accept DFP constants (DF, DD, DL, df,
dd, dl suffixes) in strict C2X mode, with a pedwarn-if-pedantic for
older standards and a warning with -Wc11-c2x-compat even in C2x mode
(which in turn requires -Wc11-c2x-compat to be newly passed through to
libcpp).
Bootstrapped with no regressions on x86_64-pc-linux-gnu.
gcc/c-family:
* c.opt (Wc11-c2x-compat): Add CPP(cpp_warn_c11_c2x_compat)
CppReason(CPP_W_C11_C2X_COMPAT).
gcc/testsuite:
* gcc.dg/dfp/c11-constants-1.c, gcc.dg/dfp/c11-constants-2.c,
gcc.dg/dfp/c2x-constants-1.c, gcc.dg/dfp/c2x-constants-2.c: New
tests.
* gcc.dg/dfp/constants-pedantic.c: Use -std=gnu17 explicitly.
Update expected diagnostics.
libcpp:
* include/cpplib.h (struct cpp_options): Add dfp_constants and
cpp_warn_c11_c2x_compat.
(enum cpp_warning_reason): Add CPP_W_C11_C2X_COMPAT.
* init.c (struct lang_flags): Add dfp_constants.
(lang_defaults): Set dfp_constants to 1 for GNUC2X and STDC2X and
0 for other languages.
(cpp_set_lang): Set dfp_constants from language.
(cpp_create_reader): Set cpp_warn_c11_c2x_compat to -1.
* expr.c (interpret_float_suffix): Mention DFP constants as C2X in
comment.
(cpp_classify_number): Do not diagnose DFP constants for languages
setting dfp_constants, unless cpp_warn_c11_c2x_compat.
From-SVN: r276908
GCC Administrator