git
/
gcc
mirror of git://gcc.gnu.org/git/gcc.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

ipa, cgraph: Enable constant propagation to OpenMP kernels. 

This patch enables constant propagation to outlined OpenMP kernels.
It does so using a new function attribute called ' callback' (note the
space).

The attribute ' callback' captures the notion of a function calling one
of its arguments with some of its parameters as arguments.  An OpenMP
example of such function is GOMP_parallel.
We implement the attribute with new callgraph edges called callback
edges. They are imaginary edges pointing from the caller of the function
with the attribute (e.g. caller of GOMP_parallel) to the body function
itself (e.g. the outlined OpenMP body).  They share their call statement
with the edge from which they are derived (direct edge caller -> GOMP_parallel
in this case).  These edges allow passes such as ipa-cp to see the hidden
call site to the body function and optimize the function accordingly.

To illustrate on an example, the body GOMP_parallel looks something
like this:

void GOMP_parallel (void (*fn) (void *), void *data, /* ... */)
{
  /* ... */
  fn (data);
  /* ... */
}

If we extend it with the attribute ' callback(1, 2)', we express that the
function calls its first argument and passes it its second argument.
This is represented in the call graph in this manner:

             direct                         indirect
caller -----------------> GOMP_parallel ---------------> fn
  |
  ----------------------> fn
          callback

The direct edge is then the callback-carrying edge, all new edges
are the derived callback edges.
While constant propagation is the main focus of this patch, callback
edges can be useful for different passes (for example, they improve icf
for OpenMP kernels), as they allow for address redirection.
If the outlined body function gets optimized and cloned, from body_fn to
body_fn.optimized, the callback edge allows us to replace the
address in the arguments list:

GOMP_parallel (body_fn, &data_struct, /* ... */);

becomes

GOMP_parallel (body_fn.optimized, &data_struct, /* ... */);

This redirection is possible for any function with the attribute.

This callback attribute implementation is partially compatible with
clang's implementation. Its semantics, arguments and argument indexing style are
the same, but we represent an unknown argument position with 0
(precedent set by attributes such as 'format'), while clang uses -1 or '?'.
We use the index 1 for the 'this' pointer in member functions, clang
uses 0. We also allow for multiple callback attributes on the same function,
while clang only allows one.

The attribute is currently for GCC internal use only, thanks to the
space in its name.  Originally, it was supposed to be called
'callback' like its clang counterpart, but we cannot use this name, as
clang uses non-standard indexing style, leading to inconsistencies.  The
attribute will be introduced into the public API as 'gnu::callback_only'
in a future patch.

The attribute allows us to propagate constants into body functions of
OpenMP constructs. Currently, GCC won't propagate the value 'c' into the
OpenMP body in the following example:

int a[100];
void test(int c) {
#pragma omp parallel for
  for (int i = 0; i < c; i++) {
    if (!__builtin_constant_p(c)) {
      __builtin_abort();
    }
    a[i] = i;
  }
}
int main() {
  test(100);
  return a[5] - 5;
}

With this patch, the body function will get cloned and the constant 'c'
will get propagated.

Some functions may utilize the attribute's infrastructure without being
declared with it, for example GOMP_task.  These functions are special
cases and use the special case functions found in attr-callback.h.  Special
cases use the attribute under certain circumstances, for example
GOMP_task uses it when the copy function is not being used required.

gcc/ChangeLog:

	* Makefile.in: Add attr-callback.o to OBJS.
	* builtin-attrs.def (ATTR_CALLBACK): Callback attr identifier.
	(DEF_CALLBACK_ATTRIBUTE): Macro for callback attr creation.
	(GOMP): Attr for libgomp functions.
	(ATTR_CALLBACK_GOMP_LIST): ATTR_NOTHROW_LIST with GOMP callback
	attr added.
	* cgraph.cc (cgraph_add_edge_to_call_site_hash): Always hash the
	callback-carrying edge.
	(cgraph_node::get_edge): Always return the callback-carrying
	edge.
	(cgraph_edge::set_call_stmt): Add cascade for callback edges.
	(symbol_table::create_edge): Allow callback edges to share call
	stmts, initialize new flags.
	(cgraph_edge::make_callback): New method, derives a new callback
	edge.
	(cgraph_edge::get_callback_carrying_edge): New method.
	(cgraph_edge::first_callback_edge): Likewise.
	(cgraph_edge::next_callback_edge): Likewise.
	(cgraph_edge::purge_callback_edges): Likewise.
	(cgraph_edge::redirect_callee): When redirecting a callback
	edge, redirect its ref as well.
	(cgraph_edge::redirect_call_stmt_to_callee): Add callback edge
	redirection logic, set update_derived_edges to true hwne
	redirecting the carrying edge.
	(cgraph_node::remove_callers): Add cascade for callback edges.
	(cgraph_edge::dump_edge_flags): Print callback flags.
	(cgraph_node::verify_node): Add sanity checks for callback
	edges.
	* cgraph.h: Add new 1 bit flags and 16 bit callback_id to
	cgraph_edge class.
	* cgraphclones.cc (cgraph_edge::clone): Copy over callback data.
	* cif-code.def (CALLBACK_EDGE): Add CIF_CALLBACK_EDGE code.
	* ipa-cp.cc (purge_useless_callback_edges): New function,
	deletes callback edges when necessary.
	(ipcp_decision_stage): Call purge_useless_callback_edges.
	* ipa-fnsummary.cc (ipa_call_summary_t::duplicate): Add
	an exception for callback edges.
	(analyze_function_body): Copy over summary from carrying to
	callback edge.
	* ipa-inline-analysis.cc (do_estimate_growth_1): Skip callback
	edges when estimating growth.
	* ipa-inline-transform.cc (inline_transform): Add redirection
	cascade for callback edges.
	* ipa-param-manipulation.cc
	(drop_decl_attribute_if_params_changed_p): New function.
	(ipa_param_adjustments::build_new_function_type): Add
	args_modified out param.
	(ipa_param_adjustments::adjust_decl): Drop callback attrs when
	modifying args.
	* ipa-param-manipulation.h: Adjust decl of
	build_new_function_type.
	* ipa-prop.cc (ipa_duplicate_jump_function): Add decl.
	(init_callback_edge_summary): New function.
	(ipa_compute_jump_functions_for_edge): Add callback edge
	creation logic.
	* lto-cgraph.cc (lto_output_edge): Stream out callback data.
	(input_edge): Input callback data.
	* omp-builtins.def (BUILT_IN_GOMP_PARALLEL_LOOP_STATIC): Use new
	attr list.
	(BUILT_IN_GOMP_PARALLEL_LOOP_GUIDED): Likewise.
	(BUILT_IN_GOMP_PARALLEL_LOOP_NONMONOTONIC_DYNAMIC): Likewise.
	(BUILT_IN_GOMP_PARALLEL_LOOP_NONMONOTONIC_RUNTIME): Likewise.
	(BUILT_IN_GOMP_PARALLEL): Likewise.
	(BUILT_IN_GOMP_PARALLEL_SECTIONS): Likewise.
	(BUILT_IN_GOMP_TEAMS_REG): Likewise.
	* tree-core.h (ECF_CB_1_2): New constant for callback(1,2).
	* tree-inline.cc (copy_bb): Copy callback edges when copying the
	carrying edge.
	(redirect_all_calls): Redirect callback edges.
	* tree.cc (set_call_expr_flags): Create callback attr according
	to the ECF_CB flag.
	* attr-callback.cc: New file.
	* attr-callback.h: New file.

gcc/c-family/ChangeLog:

	* c-attribs.cc: Define callback attr.

gcc/fortran/ChangeLog:

	* f95-lang.cc (ATTR_CALLBACK_GOMP_LIST): New attr list
	corresponding to the list in builtin-attrs.def.

gcc/testsuite/ChangeLog:

	* gcc.dg/ipa/ipcp-cb-spec1.c: New test.
	* gcc.dg/ipa/ipcp-cb-spec2.c: New test.
	* gcc.dg/ipa/ipcp-cb1.c: New test.

Signed-off-by: Josef Melcr <jmelcr02@gmail.com>
This commit is contained in:
Josef Melcr 2025-10-16 16:25:29 +02:00
parent cdb08b4bd2
commit 7cd91c7c42
25 changed files with 1166 additions and 42 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
gcc/Makefile.in
View File

@ -1853,6 +1853,7 @@ OBJS = \
web.o \
wide-int.o \
wide-int-print.o \
attr-callback.o \
$(out_object_file) \
$(ANALYZER_OBJS) \
$(EXTRA_OBJS) \

367
gcc/attr-callback.cc Normal file
View File

@ -0,0 +1,367 @@
/* Callback attribute handling
Copyright (C) 2025 Free Software Foundation, Inc.
Contributed by Josef Melcr <jmelcr@gcc.gnu.org>
This file is part of GCC.
GCC is free software; you can redistribute it and/or modify
under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.
GCC is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING3. If not see
<http://www.gnu.org/licenses/>. */
#include "config.h"
#include "system.h"
#include "coretypes.h"
#include "backend.h"
#include "tree.h"
#include "gimple.h"
#include "alloc-pool.h"
#include "cgraph.h"
#include "diagnostic.h"
#include "builtins.h"
#include "options.h"
#include "gimple-range.h"
#include "attribs.h"
#include "attr-callback.h"
/* Returns a callback attribute with callback index FN_IDX, and ARG_COUNT
arguments specified by VA_ARGS. */
tree
callback_build_attr (unsigned fn_idx, unsigned arg_count...)
{
va_list args;
va_start (args, arg_count);
tree cblist = NULL_TREE;
tree *pp = &cblist;
unsigned i;
for (i = 0; i < arg_count; i++)
{
int num = va_arg (args, int);
tree tnum = build_int_cst (integer_type_node, num);
*pp = build_tree_list (NULL, tnum PASS_MEM_STAT);
pp = &TREE_CHAIN (*pp);
}
cblist
= tree_cons (NULL_TREE, build_int_cst (integer_type_node, fn_idx), cblist);
tree attr
= tree_cons (get_identifier (CALLBACK_ATTR_IDENT), cblist, NULL_TREE);
return attr;
}
/* Returns TRUE if a function should be treated as if it had a callback
attribute despite the DECL not having it. STMT can be passed NULL
if the call statement is not available at the time, for example WPA, but it
should be called with the statement itself whenever possible. */
bool
callback_is_special_cased (tree decl, gcall *stmt)
{
if (fndecl_built_in_p (decl, BUILT_IN_GOMP_TASK))
{
if (stmt)
return gimple_call_arg (stmt, 2) == null_pointer_node;
return true;
}
return false;
}
/* Returns an attribute for a special cased function. */
tree
callback_special_case_attr (tree decl)
{
if (fndecl_built_in_p (decl, BUILT_IN_GOMP_TASK))
return callback_build_attr (1, 1, 2);
gcc_unreachable ();
}
/* Given an instance of callback attribute, return the 0-based
index of the called function in question. */
int
callback_get_fn_index (tree cb_attr)
{
tree args = TREE_VALUE (cb_attr);
int idx = TREE_INT_CST_LOW (TREE_VALUE (args)) - 1;
return idx;
}
/* For a given callback pair, retrieves the callback attribute used
to create E from the callee of CARRYING. */
tree
callback_fetch_attr_by_edge (cgraph_edge *e, cgraph_edge *carrying)
{
gcc_checking_assert (e->call_stmt == carrying->call_stmt
&& e->lto_stmt_uid == carrying->lto_stmt_uid);
if (callback_is_special_cased (carrying->callee->decl, e->call_stmt))
return callback_special_case_attr (carrying->callee->decl);
tree cb_attr = lookup_attribute (CALLBACK_ATTR_IDENT,
DECL_ATTRIBUTES (carrying->callee->decl));
gcc_checking_assert (cb_attr);
tree res = NULL_TREE;
for (; cb_attr;
cb_attr = lookup_attribute (CALLBACK_ATTR_IDENT, TREE_CHAIN (cb_attr)))
{
unsigned id = callback_get_fn_index (cb_attr);
if (id == e->callback_id)
{
res = cb_attr;
break;
}
}
gcc_checking_assert (res != NULL_TREE);
return res;
}
/* Given an instance of callback attribute, return the 0-base indices
of arguments passed to the callback. For a callback function taking
n parameters, returns a vector of n indices of their values in the parameter
list of it's caller. Indices with unknown positions contain -1. */
auto_vec<int>
callback_get_arg_mapping (cgraph_edge *e, cgraph_edge *carrying)
{
tree attr = callback_fetch_attr_by_edge (e, carrying);
gcc_checking_assert (attr);
tree args = TREE_VALUE (attr);
auto_vec<int> res;
tree it;
/* Skip over the first argument, which denotes
which argument is the called function. */
for (it = TREE_CHAIN (args); it != NULL_TREE; it = TREE_CHAIN (it))
{
int idx = TREE_INT_CST_LOW (TREE_VALUE (it));
/* Subtract 1 to account for 1-based indexing. If the value is unknown,
use constant -1 instead. */
idx = idx == CB_UNKNOWN_POS ? -1 : idx - 1;
res.safe_push (idx);
}
return res;
}
/* For a callback pair, returns the 0-based index of the address of
E's callee in the argument list of CARRYING's callee decl. */
int
callback_fetch_fn_position (cgraph_edge *e, cgraph_edge *carrying)
{
tree attr = callback_fetch_attr_by_edge (e, carrying);
return callback_get_fn_index (attr);
}
/* Returns the element at index idx in the list or NULL_TREE if
the list isn't long enough. NULL_TREE is used as the endpoint. */
static tree
get_nth_list_elem (tree list, unsigned idx)
{
tree res = NULL_TREE;
unsigned i = 0;
tree it;
for (it = list; it != NULL_TREE; it = TREE_CHAIN (it), i++)
{
if (i == idx)
{
res = TREE_VALUE (it);
break;
}
}
return res;
}
/* Handle a "callback" attribute; arguments as in
struct attribute_spec.handler. */
tree
handle_callback_attribute (tree *node, tree name, tree args,
int ARG_UNUSED (flags), bool *no_add_attrs)
{
tree decl = *node;
if (TREE_CODE (decl) != FUNCTION_DECL)
{
error_at (DECL_SOURCE_LOCATION (decl),
"%qE attribute can only be used on functions", name);
*no_add_attrs = true;
}
tree cb_fn_idx_node = TREE_VALUE (args);
if (TREE_CODE (cb_fn_idx_node) != INTEGER_CST)
{
error_at (DECL_SOURCE_LOCATION (decl),
"argument specifying callback function position is not an "
"integer constant");
*no_add_attrs = true;
return NULL_TREE;
}
/* We have to use the function type for validation, as
DECL_ARGUMENTS returns NULL at this point. */
int callback_fn_idx = TREE_INT_CST_LOW (cb_fn_idx_node);
tree decl_type_args = TYPE_ARG_TYPES (TREE_TYPE (decl));
tree it;
int decl_nargs = list_length (decl_type_args);
for (it = decl_type_args; it != NULL_TREE; it = TREE_CHAIN (it))
if (it == void_list_node)
{
--decl_nargs;
break;
}
if (callback_fn_idx == CB_UNKNOWN_POS)
{
error_at (DECL_SOURCE_LOCATION (decl),
"callback function position cannot be marked as unknown");
*no_add_attrs = true;
return NULL_TREE;
}
--callback_fn_idx;
if (callback_fn_idx >= decl_nargs)
{
error_at (DECL_SOURCE_LOCATION (decl),
"callback function position out of range");
*no_add_attrs = true;
return NULL_TREE;
}
/* Search for the type of the callback function
in parameters of the original function. */
tree cfn = get_nth_list_elem (decl_type_args, callback_fn_idx);
if (cfn == NULL_TREE)
{
error_at (DECL_SOURCE_LOCATION (decl),
"could not retrieve callback function from arguments");
*no_add_attrs = true;
return NULL_TREE;
}
tree cfn_pointee_type = TREE_TYPE (cfn);
if (TREE_CODE (cfn) != POINTER_TYPE
|| TREE_CODE (cfn_pointee_type) != FUNCTION_TYPE)
{
error_at (DECL_SOURCE_LOCATION (decl),
"argument no. %d is not an address of a function",
callback_fn_idx + 1);
*no_add_attrs = true;
return NULL_TREE;
}
tree type_args = TYPE_ARG_TYPES (cfn_pointee_type);
/* Compare the length of the list of argument indices
and the real number of parameters the callback takes. */
unsigned cfn_nargs = list_length (TREE_CHAIN (args));
unsigned type_nargs = list_length (type_args);
for (it = type_args; it != NULL_TREE; it = TREE_CHAIN (it))
if (it == void_list_node)
{
--type_nargs;
break;
}
if (cfn_nargs != type_nargs)
{
error_at (DECL_SOURCE_LOCATION (decl),
"argument number mismatch, %d expected, got %d", type_nargs,
cfn_nargs);
*no_add_attrs = true;
return NULL_TREE;
}
unsigned curr = 0;
tree cfn_it;
/* Validate type compatibility of the arguments passed
from caller function to callback. "it" is used to step
through the parameters of the caller, "cfn_it" is
stepping through the parameters of the callback. */
for (it = type_args, cfn_it = TREE_CHAIN (args); curr < type_nargs;
it = TREE_CHAIN (it), cfn_it = TREE_CHAIN (cfn_it), curr++)
{
if (TREE_CODE (TREE_VALUE (cfn_it)) != INTEGER_CST)
{
error_at (DECL_SOURCE_LOCATION (decl),
"argument no. %d is not an integer constant", curr + 1);
*no_add_attrs = true;
continue;
}
int arg_idx = TREE_INT_CST_LOW (TREE_VALUE (cfn_it));
/* No need to check for type compatibility,
if we don't know what we are passing. */
if (arg_idx == CB_UNKNOWN_POS)
continue;
arg_idx -= 1;
/* Report an error if the position is out of bounds,
but we can still check the rest of the arguments. */
if (arg_idx >= decl_nargs)
{
error_at (DECL_SOURCE_LOCATION (decl),
"callback argument index %d is out of range", arg_idx + 1);
*no_add_attrs = true;
continue;
}
tree arg_type = get_nth_list_elem (decl_type_args, arg_idx);
tree expected_type = TREE_VALUE (it);
/* Check the type of the value we are about to pass ("arg_type")
for compatibility with the actual type the callback function
expects ("expected_type"). */
if (!types_compatible_p (expected_type, arg_type))
{
error_at (DECL_SOURCE_LOCATION (decl),
"argument type at index %d is not compatible with callback "
"argument type at index %d",
arg_idx + 1, curr + 1);
*no_add_attrs = true;
continue;
}
}
/* Check that the decl does not already have a callback attribute describing
the same argument. */
it = lookup_attribute (CALLBACK_ATTR_IDENT, DECL_ATTRIBUTES (decl));
for (; it; it = lookup_attribute (CALLBACK_ATTR_IDENT, TREE_CHAIN (it)))
if (callback_get_fn_index (it) == callback_fn_idx)
{
error_at (DECL_SOURCE_LOCATION (decl),
"function declaration has multiple callback attributes "
"describing argument no. %d",
callback_fn_idx + 1);
*no_add_attrs = true;
break;
}
return NULL_TREE;
}
/* Returns TRUE if E is considered useful in the callgraph, FALSE otherwise. If
this predicate returns FALSE, then E wasn't used to optimize its callee and
can be safely removed from the callgraph. */
bool
callback_edge_useful_p (cgraph_edge *e)
{
gcc_checking_assert (e->callback);
/* If the edge is not pointing towards a clone, it is no longer useful as its
entire purpose is to produce clones of callbacks. */
if (!e->callee->clone_of)
return false;
return true;
}
/* Returns the number of arguments the callback function described by ATTR
takes. */
size_t
callback_num_args (tree attr)
{
tree args = TREE_VALUE (attr);
size_t res = 0;
tree it;
for (it = TREE_CHAIN (args); it != NULL_TREE; it = TREE_CHAIN (it), ++res)
;
return res;
}

78
gcc/attr-callback.h Normal file
View File

@ -0,0 +1,78 @@
/* Callback attribute handling
Copyright (C) 2025 Free Software Foundation, Inc.
Contributed by Josef Melcr <jmelcr@gcc.gnu.org>
This file is part of GCC.
GCC is free software; you can redistribute it and/or modify
under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.
GCC is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with GCC; see the file COPYING3. If not see
<http://www.gnu.org/licenses/>. */
#ifndef ATTR_CALLBACK_H
#define ATTR_CALLBACK_H
enum callback_position
{
/* Value used when an argument of a callback function
is unknown or when multiple values may be used. */
CB_UNKNOWN_POS = 0
};
#define CALLBACK_ATTR_IDENT " callback"
/* Returns a callback attribute with callback index FN_IDX, and ARG_COUNT
arguments specified by VA_ARGS. */
tree callback_build_attr (unsigned fn_idx, unsigned arg_count...);
/* Returns TRUE if a function should be treated as if it had a callback
attribute despite the DECL not having it. STMT can be passed NULL
if the call statement is not available at the time, for example WPA, but it
should be called with the statement itself whenever possible. */
bool callback_is_special_cased (tree decl, gcall *stmt);
/* Returns an attribute for a special cased function. */
tree callback_special_case_attr (tree decl);
/* Given an instance of callback attribute, return the 0-based
index of the called function in question. */
int callback_get_fn_index (tree cb_attr);
/* For a given callback pair, retrieves the callback attribute used
to create E from the callee of CARRYING. */
tree callback_fetch_attr_by_edge (cgraph_edge *e, cgraph_edge *carrying);
/* Given an instance of callback attribute, return the 0-base indices
of arguments passed to the callback. For a callback function taking
n parameters, returns a vector of n indices of their values in the parameter
list of it's caller. Indices with unknown positions contain -1. */
auto_vec<int> callback_get_arg_mapping (cgraph_edge *e, cgraph_edge *carrying);
/* For a callback pair, returns the 0-based index of the address of
E's callee in the argument list of CARRYING's callee decl. */
int callback_fetch_fn_position (cgraph_edge *e, cgraph_edge *carrying);
/* Handle a "callback" attribute; arguments as in
struct attribute_spec.handler. */
tree handle_callback_attribute (tree *node, tree name, tree args, int flags,
bool *no_add_attrs);
/* Returns TRUE if E is considered useful in the callgraph, FALSE otherwise. If
this predicate returns FALSE, then E wasn't used to optimize its callee and
can be safely removed from the callgraph. */
bool callback_edge_useful_p (cgraph_edge *e);
/* Returns the number of arguments the callback function described by ATTR
takes. */
size_t callback_num_args (tree attr);
#endif /* ATTR_CALLBACK_H */

11
gcc/builtin-attrs.def
View File

@ -130,6 +130,7 @@ DEF_ATTR_IDENT (ATTR_TM_TMPURE, "transaction_pure")
DEF_ATTR_IDENT (ATTR_RETURNS_TWICE, "returns_twice")
DEF_ATTR_IDENT (ATTR_RETURNS_NONNULL, "returns_nonnull")
DEF_ATTR_IDENT (ATTR_WARN_UNUSED_RESULT, "warn_unused_result")
DEF_ATTR_IDENT (ATTR_CALLBACK, " callback")
DEF_ATTR_TREE_LIST (ATTR_NOVOPS_LIST, ATTR_NOVOPS, ATTR_NULL, ATTR_NULL)
@ -430,6 +431,16 @@ DEF_FORMAT_ATTRIBUTE_NOTHROW(STRFMON,3,3_4)
#undef DEF_FORMAT_ATTRIBUTE_NOTHROW
#undef DEF_FORMAT_ATTRIBUTE_BOTH
/* Construct callback attributes for GOMP builtins. */
#define DEF_CALLBACK_ATTRIBUTE(TYPE, CA, VALUES) \
DEF_ATTR_TREE_LIST (ATTR_CALLBACK_##TYPE##_##CA##_##VALUES, ATTR_CALLBACK,\
ATTR_##CA, ATTR_LIST_##VALUES)
DEF_CALLBACK_ATTRIBUTE(GOMP, 1, 2)
DEF_ATTR_TREE_LIST(ATTR_CALLBACK_GOMP_LIST, ATTR_CALLBACK,
ATTR_CALLBACK_GOMP_1_2, ATTR_NOTHROW_LIST)
#undef DEF_CALLBACK_ATTRIBUTE
/* Transactional memory variants of the above. */
DEF_ATTR_TREE_LIST (ATTR_TM_NOTHROW_LIST,

3
gcc/c-family/c-attribs.cc
View File

@ -49,6 +49,7 @@ along with GCC; see the file COPYING3. If not see
#include "tree-pretty-print.h"
#include "gcc-rich-location.h"
#include "gcc-urlifier.h"
#include "attr-callback.h"
static tree handle_packed_attribute (tree *, tree, tree, int, bool *);
static tree handle_nocommon_attribute (tree *, tree, tree, int, bool *);
@ -484,6 +485,8 @@ const struct attribute_spec c_common_gnu_attributes[] =
handle_tm_attribute, NULL },
{ "transaction_may_cancel_outer", 0, 0, false, true, false, false,
handle_tm_attribute, NULL },
{ CALLBACK_ATTR_IDENT, 1, -1, true, false, false, false,
handle_callback_attribute, NULL },
/* ??? These two attributes didn't make the transition from the
Intel language document to the multi-vendor language document. */
{ "transaction_pure", 0, 0, false, true, false, false,

290
gcc/cgraph.cc
View File

@ -69,6 +69,7 @@ along with GCC; see the file COPYING3. If not see
#include "tree-nested.h"
#include "symtab-thunks.h"
#include "symtab-clones.h"
#include "attr-callback.h"
/* FIXME: Only for PROP_loops, but cgraph shouldn't have to know about this. */
#include "tree-pass.h"
@ -871,11 +872,22 @@ cgraph_add_edge_to_call_site_hash (cgraph_edge *e)
one indirect); always hash the direct one. */
if (e->speculative && e->indirect_unknown_callee)
return;
/* We always want to hash the carrying edge of a callback, not the edges
pointing to the callbacks themselves, as their call statement doesn't
exist. */
if (e->callback)
return;
cgraph_edge **slot = e->caller->call_site_hash->find_slot_with_hash
(e->call_stmt, cgraph_edge_hasher::hash (e->call_stmt), INSERT);
if (*slot)
{
gcc_assert (((cgraph_edge *)*slot)->speculative);
cgraph_edge *edge = (cgraph_edge *) *slot;
gcc_assert (edge->speculative || edge->has_callback);
if (edge->has_callback)
/* If the slot is already occupied, then the hashed edge is the
callback-carrying edge, which is desired behavior, so we can safely
return. */
gcc_checking_assert (edge == e);
if (e->callee && (!e->prev_callee
|| !e->prev_callee->speculative
|| e->prev_callee->call_stmt != e->call_stmt))
@ -919,6 +931,13 @@ cgraph_node::get_edge (gimple *call_stmt)
n++;
}
/* We want to work with the callback-carrying edge whenever possible. When it
comes to callback edges, a call statement might have multiple callback
edges attached to it. These can be easily obtained from the carrying edge
instead. */
if (e && e->callback)
e = e->get_callback_carrying_edge ();
if (n > 100)
{
call_site_hash = hash_table<cgraph_edge_hasher>::create_ggc (120);
@ -931,15 +950,16 @@ cgraph_node::get_edge (gimple *call_stmt)
return e;
}
/* Change field call_stmt of edge E to NEW_STMT. If UPDATE_SPECULATIVE and E
/* Change field call_stmt of edge E to NEW_STMT. If UPDATE_DERIVED_EDGES and E
is any component of speculative edge, then update all components.
Speculations can be resolved in the process and EDGE can be removed and
deallocated. Return the edge that now represents the call. */
speculations can be resolved in the process and edge can be removed and
deallocated. if update_derived_edges and e is a part of a callback pair,
update all associated edges and return their carrying edge. return the edge
that now represents the call. */
cgraph_edge *
cgraph_edge::set_call_stmt (cgraph_edge *e, gcall *new_stmt,
bool update_speculative)
bool update_derived_edges)
{
tree decl;
@ -955,7 +975,7 @@ cgraph_edge::set_call_stmt (cgraph_edge *e, gcall *new_stmt,
/* Speculative edges has three component, update all of them
when asked to. */
if (update_speculative && e->speculative
if (update_derived_edges && e->speculative
/* If we are about to resolve the speculation by calling make_direct
below, do not bother going over all the speculative edges now. */
&& !new_direct_callee)
@ -991,6 +1011,27 @@ cgraph_edge::set_call_stmt (cgraph_edge *e, gcall *new_stmt,
if (new_direct_callee)
e = make_direct (e, new_direct_callee);
/* When updating a callback or a callback-carrying edge, update every edge
involved. */
if (update_derived_edges && (e->callback || e->has_callback))
{
cgraph_edge *current, *next, *carrying;
carrying = e->has_callback ? e : e->get_callback_carrying_edge ();
current = e->first_callback_edge ();
if (current)
{
for (cgraph_edge *d = current; d; d = next)
{
next = d->next_callback_edge ();
cgraph_edge *d2 = set_call_stmt (d, new_stmt, false);
gcc_assert (d2 == d);
}
}
carrying = set_call_stmt (carrying, new_stmt, false);
return carrying;
}
/* Only direct speculative edges go to call_site_hash. */
if (e->caller->call_site_hash
&& (!e->speculative || !e->indirect_unknown_callee)
@ -1036,7 +1077,7 @@ symbol_table::create_edge (cgraph_node *caller, cgraph_node *callee,
construction of call stmt hashtable. */
cgraph_edge *e;
gcc_checking_assert (!(e = caller->get_edge (call_stmt))
|| e->speculative);
|| e->speculative || e->has_callback || e->callback);
gcc_assert (is_gimple_call (call_stmt));
}
@ -1063,6 +1104,9 @@ symbol_table::create_edge (cgraph_node *caller, cgraph_node *callee,
edge->indirect_info = NULL;
edge->indirect_inlining_edge = 0;
edge->speculative = false;
edge->has_callback = false;
edge->callback = false;
edge->callback_id = 0;
edge->indirect_unknown_callee = indir_unknown_callee;
if (call_stmt && caller->call_site_hash)
cgraph_add_edge_to_call_site_hash (edge);
@ -1286,6 +1330,119 @@ cgraph_edge::make_speculative (cgraph_node *n2, profile_count direct_count,
return e2;
}
/* Create a callback edge calling N2. Callback edges
never get turned into actual calls, they are just used
as clues and allow for optimizing functions which do not
have any callsites during compile time, e.g. functions
passed to standard library functions.
The edge will be attached to the same call statement as
the callback-carrying edge, which is the instance this method
is called on.
callback_id is used to pair the returned edge with the attribute that
originated it.
Return the resulting callback edge. */
cgraph_edge *
cgraph_edge::make_callback (cgraph_node *n2, unsigned int callback_id)
{
cgraph_node *n = caller;
cgraph_edge *e2;
has_callback = true;
e2 = n->create_edge (n2, call_stmt, count);
if (dump_file)
fprintf (
dump_file,
"Created callback edge %s -> %s belonging to carrying edge %s -> %s\n",
e2->caller->dump_name (), e2->callee->dump_name (), caller->dump_name (),
callee->dump_name ());
e2->inline_failed = CIF_CALLBACK_EDGE;
e2->callback = true;
e2->callback_id = callback_id;
if (TREE_NOTHROW (n2->decl))
e2->can_throw_external = false;
else
e2->can_throw_external = can_throw_external;
e2->lto_stmt_uid = lto_stmt_uid;
n2->mark_address_taken ();
return e2;
}
/* Returns the callback_carrying edge of a callback edge on which
it is called on or NULL when no such edge can be found.
An edge is taken to be the callback-carrying if it has it's has_callback
flag set and the edges share their call statements. */
cgraph_edge *
cgraph_edge::get_callback_carrying_edge ()
{
gcc_checking_assert (callback);
cgraph_edge *e;
for (e = caller->callees; e; e = e->next_callee)
{
if (e->has_callback && e->call_stmt == call_stmt
&& e->lto_stmt_uid == lto_stmt_uid)
break;
}
return e;
}
/* Returns the first callback edge in the list of callees of the caller node.
Note that the edges might be in arbitrary order. Must be called on a
callback or callback-carrying edge. */
cgraph_edge *
cgraph_edge::first_callback_edge ()
{
gcc_checking_assert (has_callback || callback);
cgraph_edge *e = NULL;
for (e = caller->callees; e; e = e->next_callee)
{
if (e->callback && e->call_stmt == call_stmt
&& e->lto_stmt_uid == lto_stmt_uid)
break;
}
return e;
}
/* Given a callback edge, returns the next callback edge belonging to the same
carrying edge. Must be called on a callback edge, not the callback-carrying
edge. */
cgraph_edge *
cgraph_edge::next_callback_edge ()
{
gcc_checking_assert (callback);
cgraph_edge *e = NULL;
for (e = next_callee; e; e = e->next_callee)
{
if (e->callback && e->call_stmt == call_stmt
&& e->lto_stmt_uid == lto_stmt_uid)
break;
}
return e;
}
/* When called on a callback-carrying edge, removes all of its attached callback
edges and sets has_callback to FALSE. */
void
cgraph_edge::purge_callback_edges ()
{
gcc_checking_assert (has_callback);
cgraph_edge *e, *next;
for (e = first_callback_edge (); e; e = next)
{
next = e->next_callback_edge ();
cgraph_edge::remove (e);
}
has_callback = false;
}
/* Speculative call consists of an indirect edge and one or more
direct edge+ref pairs.
@ -1521,12 +1678,27 @@ void
cgraph_edge::redirect_callee (cgraph_node *n)
{
bool loc = callee->comdat_local_p ();
cgraph_node *old_callee = callee;
/* Remove from callers list of the current callee. */
remove_callee ();
/* Insert to callers list of the new callee. */
set_callee (n);
if (callback)
{
/* When redirecting a callback callee, redirect its ref as well. */
ipa_ref *old_ref = caller->find_reference (old_callee, call_stmt,
lto_stmt_uid, IPA_REF_ADDR);
gcc_checking_assert(old_ref);
old_ref->remove_reference ();
ipa_ref *new_ref = caller->create_reference (n, IPA_REF_ADDR, call_stmt);
new_ref->lto_stmt_uid = lto_stmt_uid;
if (!old_callee->referred_to_p ())
old_callee->address_taken = 0;
}
if (!inline_failed)
return;
if (!loc && n->comdat_local_p ())
@ -1643,6 +1815,27 @@ cgraph_edge::redirect_call_stmt_to_callee (cgraph_edge *e,
|| decl == e->callee->decl)
return e->call_stmt;
/* When redirecting a callback edge, all we need to do is replace
the original address with the address of the function we are
redirecting to. */
if (e->callback)
{
cgraph_edge *carrying = e->get_callback_carrying_edge ();
if (!callback_is_special_cased (carrying->callee->decl, e->call_stmt)
&& !lookup_attribute (CALLBACK_ATTR_IDENT,
DECL_ATTRIBUTES (carrying->callee->decl)))
/* Callback attribute is removed if the dispatching function changes
signature, as the indices wouldn't be correct anymore. These edges
will get cleaned up later, ignore their redirection for now. */
return e->call_stmt;
int fn_idx = callback_fetch_fn_position (e, carrying);
tree previous_arg = gimple_call_arg (e->call_stmt, fn_idx);
location_t loc = EXPR_LOCATION (previous_arg);
tree new_addr = build_fold_addr_expr_loc (loc, e->callee->decl);
gimple_call_set_arg (e->call_stmt, fn_idx, new_addr);
return e->call_stmt;
}
if (decl && ipa_saved_clone_sources)
{
tree *p = ipa_saved_clone_sources->get (e->callee);
@ -1752,7 +1945,9 @@ cgraph_edge::redirect_call_stmt_to_callee (cgraph_edge *e,
maybe_remove_unused_call_args (DECL_STRUCT_FUNCTION (e->caller->decl),
new_stmt);
e->caller->set_call_stmt_including_clones (e->call_stmt, new_stmt, false);
/* Update callback edges if setting the carrying edge's statement, or else
their pairing would fall apart. */
e->caller->set_call_stmt_including_clones (e->call_stmt, new_stmt, e->has_callback);
if (symtab->dump_file)
{
@ -1944,6 +2139,17 @@ cgraph_node::remove_callers (void)
for (e = callers; e; e = f)
{
f = e->next_caller;
/* When removing a callback-carrying edge, remove all its attached edges
as well. */
if (e->has_callback)
{
cgraph_edge *cbe, *next_cbe = NULL;
for (cbe = e->first_callback_edge (); cbe; cbe = next_cbe)
{
next_cbe = cbe->next_callback_edge ();
cgraph_edge::remove (cbe);
}
}
symtab->call_edge_removal_hooks (e);
e->remove_caller ();
symtab->free_edge (e);
@ -2253,6 +2459,10 @@ cgraph_edge::dump_edge_flags (FILE *f)
{
if (speculative)
fprintf (f, "(speculative) ");
if (callback)
fprintf (f, "(callback) ");
if (has_callback)
fprintf (f, "(has_callback) ");
if (!inline_failed)
fprintf (f, "(inlined) ");
if (call_stmt_cannot_inline_p)
@ -3866,6 +4076,8 @@ cgraph_node::verify_node (void)
if (gimple_has_body_p (e->caller->decl)
&& !e->caller->inlined_to
&& !e->speculative
&& !e->callback
&& !e->has_callback
/* Optimized out calls are redirected to __builtin_unreachable. */
&& (e->count.nonzero_p ()
|| ! e->callee->decl
@ -4071,7 +4283,12 @@ cgraph_node::verify_node (void)
}
if (!e->indirect_unknown_callee)
{
if (e->verify_corresponds_to_fndecl (decl))
/* Callback edges violate this assertion
because their call statement doesn't exist,
their associated statement belongs to the
callback-dispatching function. */
if (!e->callback
&& e->verify_corresponds_to_fndecl (decl))
{
error ("edge points to wrong declaration:");
debug_tree (e->callee->decl);
@ -4113,7 +4330,58 @@ cgraph_node::verify_node (void)
for (e = callees; e; e = e->next_callee)
{
if (!e->aux && !e->speculative)
if (!e->callback && e->callback_id)
{
error ("non-callback edge has callback_id set");
error_found = true;
}
if (e->callback && e->has_callback)
{
error ("edge has both callback and has_callback set");
error_found = true;
}
if (e->callback)
{
if (!e->get_callback_carrying_edge ())
{
error ("callback edge %s->%s has no callback-carrying",
identifier_to_locale (e->caller->name ()),
identifier_to_locale (e->callee->name ()));
error_found = true;
}
}
if (e->has_callback
&& !callback_is_special_cased (e->callee->decl, e->call_stmt))
{
int ncallbacks = 0;
int nfound_edges = 0;
for (tree cb = lookup_attribute (CALLBACK_ATTR_IDENT, DECL_ATTRIBUTES (
e->callee->decl));
cb; cb = lookup_attribute (CALLBACK_ATTR_IDENT, TREE_CHAIN (cb)),
ncallbacks++)
;
for (cgraph_edge *cbe = callees; cbe; cbe = cbe->next_callee)
{
if (cbe->callback && cbe->call_stmt == e->call_stmt
&& cbe->lto_stmt_uid == e->lto_stmt_uid)
{
nfound_edges++;
}
}
if (ncallbacks < nfound_edges)
{
error ("callback edge %s->%s callback edge count mismatch, "
"expected at most %d, found %d",
identifier_to_locale (e->caller->name ()),
identifier_to_locale (e->callee->name ()), ncallbacks,
nfound_edges);
}
}
if (!e->aux && !e->speculative && !e->callback && !e->has_callback)
{
error ("edge %s->%s has no corresponding call_stmt",
identifier_to_locale (e->caller->name ()),

53
gcc/cgraph.h
View File

@ -1738,12 +1738,14 @@ public:
/* Remove EDGE from the cgraph. */
static void remove (cgraph_edge *edge);
/* Change field call_stmt of edge E to NEW_STMT. If UPDATE_SPECULATIVE and E
is any component of speculative edge, then update all components.
/* Change field call_stmt of edge E to NEW_STMT. If UPDATE_DERIVED_EDGES and
E is any component of speculative edge, then update all components.
Speculations can be resolved in the process and EDGE can be removed and
deallocated. Return the edge that now represents the call. */
deallocated. Return the edge that now represents the call. If
UPDATE_DERIVED_EDGES and E is a part of a callback edge, update all
associated edges and return the callback-carrying edge. */
static cgraph_edge *set_call_stmt (cgraph_edge *e, gcall *new_stmt,
bool update_speculative = true);
bool update_derived_edges = true);
/* Redirect callee of the edge to N. The function does not update underlying
call expression. */
@ -1769,6 +1771,32 @@ public:
cgraph_edge *make_speculative (cgraph_node *n2, profile_count direct_count,
unsigned int speculative_id = 0);
/* Create a callback edge, representing an indirect call to n2
passed to a function by argument. Sets has_callback flag of the original
edge. Both edges are attached to the same call statement. Returns created
callback edge. */
cgraph_edge *make_callback (cgraph_node *n2, unsigned int callback_hash);
/* Returns the callback-carrying edge of a callback edge or NULL, if such edge
cannot be found. An edge is considered callback-carrying, if it has it's
has_callback flag set and shares it's call statement with the edge
this method is caled on. */
cgraph_edge *get_callback_carrying_edge ();
/* Returns the first callback edge in the list of callees of the caller node.
Note that the edges might be in arbitrary order. Must be called on a
callback or callback-carrying edge. */
cgraph_edge *first_callback_edge ();
/* Given a callback edge, returns the next callback edge belonging to the same
callback-carrying edge. Must be called on a callback edge, not the
callback-carrying edge. */
cgraph_edge *next_callback_edge ();
/* When called on a callback-carrying edge, removes all of its attached
callback edges and sets has_callback to FALSE. */
void purge_callback_edges ();
/* Speculative call consists of an indirect edge and one or more
direct edge+ref pairs. Speculative will expand to the following sequence:
@ -1990,6 +2018,23 @@ public:
Optimizers may later redirect direct call to clone, so 1) and 3)
do not need to necessarily agree with destination. */
unsigned int speculative : 1;
/* Edges with CALLBACK flag represent indirect calls to functions passed
to their callers by argument. This is useful in cases, where the body
of these caller functions is not known, e. g. qsort in glibc or
GOMP_parallel in libgomp. These edges are never made into real calls,
but are used instead to optimize these callback functions and later replace
their addresses with their optimized versions. Edges with this flag set
share their call statement with their callback-carrying edge. */
unsigned int callback : 1;
/* Edges with this flag set have one or more callback edges attached. They
share their call statements with this edge. This flag represents the fact
that the callee of this edge takes a function and it's parameters by
argument and calls it at a later time. */
unsigned int has_callback : 1;
/* Used to pair callback edges and the attributes that originated them
together. Currently the index of the callback argument, retrieved
from the attribute. */
unsigned int callback_id : 16;
/* Set to true when caller is a constructor or destructor of polymorphic
type. */
unsigned in_polymorphic_cdtor : 1;

3
gcc/cgraphclones.cc
View File

@ -144,6 +144,9 @@ cgraph_edge::clone (cgraph_node *n, gcall *call_stmt, unsigned stmt_uid,
new_edge->can_throw_external = can_throw_external;
new_edge->call_stmt_cannot_inline_p = call_stmt_cannot_inline_p;
new_edge->speculative = speculative;
new_edge->callback = callback;
new_edge->has_callback = has_callback;
new_edge->callback_id = callback_id;
new_edge->in_polymorphic_cdtor = in_polymorphic_cdtor;
/* Update IPA profile. Local profiles need no updating in original. */

5
gcc/cif-code.def
View File

@ -142,3 +142,8 @@ DEFCIFCODE(EXTERN_LIVE_ONLY_STATIC, CIF_FINAL_ERROR,
/* We proved that the call is unreachable. */
DEFCIFCODE(UNREACHABLE, CIF_FINAL_ERROR,
N_("unreachable"))
/* Callback edges cannot be inlined, as the corresponding call
statement does not exist. */
DEFCIFCODE(CALLBACK_EDGE, CIF_FINAL_ERROR,
N_("callback edges cannot be inlined"))

1
gcc/fortran/f95-lang.cc
View File

@ -580,6 +580,7 @@ gfc_builtin_function (tree decl)
#define ATTR_COLD_NORETURN_NOTHROW_LEAF_LIST \
(ECF_COLD | ECF_NORETURN | \
ECF_NOTHROW | ECF_LEAF)
#define ATTR_CALLBACK_GOMP_LIST (ECF_CB_1_2 | ATTR_NOTHROW_LIST)
#define ATTR_PURE_NOTHROW_LIST (ECF_PURE | ECF_NOTHROW)
static void

73
gcc/ipa-cp.cc
View File

@ -131,7 +131,7 @@ along with GCC; see the file COPYING3. If not see
#include "dbgcnt.h"
#include "symtab-clones.h"
#include "gimple-range.h"
#include "attr-callback.h"
/* Allocation pools for values and their sources in ipa-cp. */
@ -6214,6 +6214,72 @@ identify_dead_nodes (struct cgraph_node *node)
}
}
/* Removes all useless callback edges from the callgraph. Useless callback
edges might mess up the callgraph, because they might be impossible to
redirect and so on, leading to crashes. Their usefulness is evaluated
through callback_edge_useful_p. */
static void
purge_useless_callback_edges ()
{
if (dump_file)
fprintf (dump_file, "\nPurging useless callback edges:\n");
cgraph_edge *e;
cgraph_node *node;
FOR_EACH_FUNCTION_WITH_GIMPLE_BODY (node)
{
for (e = node->callees; e; e = e->next_callee)
{
if (e->has_callback)
{
if (dump_file)
fprintf (dump_file, "\tExamining callbacks of edge %s -> %s:\n",
e->caller->dump_name (), e->callee->dump_name ());
if (!lookup_attribute (CALLBACK_ATTR_IDENT,
DECL_ATTRIBUTES (e->callee->decl))
&& !callback_is_special_cased (e->callee->decl, e->call_stmt))
{
if (dump_file)
fprintf (
dump_file,
"\t\tPurging callbacks, because the callback-dispatching"
"function no longer has any callback attributes.\n");
e->purge_callback_edges ();
continue;
}
cgraph_edge *cbe, *next;
for (cbe = e->first_callback_edge (); cbe; cbe = next)
{
next = cbe->next_callback_edge ();
if (!callback_edge_useful_p (cbe))
{
if (dump_file)
fprintf (dump_file,
"\t\tCallback edge %s -> %s not deemed "
"useful, removing.\n",
cbe->caller->dump_name (),
cbe->callee->dump_name ());
cgraph_edge::remove (cbe);
}
else
{
if (dump_file)
fprintf (dump_file,
"\t\tKept callback edge %s -> %s "
"because it looks useful.\n",
cbe->caller->dump_name (),
cbe->callee->dump_name ());
}
}
}
}
}
if (dump_file)
fprintf (dump_file, "\n");
}
/* The decision stage. Iterate over the topological order of call graph nodes
TOPO and make specialized clones if deemed beneficial. */
@ -6244,6 +6310,11 @@ ipcp_decision_stage (class ipa_topo_info *topo)
if (change)
identify_dead_nodes (node);
}
/* Currently, the primary use of callback edges is constant propagation.
Constant propagation is now over, so we have to remove unused callback
edges. */
purge_useless_callback_edges ();
}
/* Look up all VR and bits information that we have discovered and copy it

24
gcc/ipa-fnsummary.cc
View File

@ -990,7 +990,10 @@ ipa_call_summary_t::duplicate (struct cgraph_edge *src,
info->predicate = NULL;
edge_set_predicate (dst, srcinfo->predicate);
info->param = srcinfo->param.copy ();
if (!dst->indirect_unknown_callee && src->indirect_unknown_callee)
if (!dst->indirect_unknown_callee && src->indirect_unknown_callee
/* Don't subtract the size when dealing with callback pairs, since the
edge has no real size. */
&& !src->has_callback && !dst->callback)
{
info->call_stmt_size -= (eni_size_weights.indirect_call_cost
- eni_size_weights.call_cost);
@ -3107,6 +3110,25 @@ analyze_function_body (struct cgraph_node *node, bool early)
es, es3);
}
}
/* If dealing with a carrying edge, copy its summary over to its
attached edges as well. */
if (edge->has_callback)
{
cgraph_edge *cbe;
for (cbe = edge->first_callback_edge (); cbe;
cbe = cbe->next_callback_edge ())
{
ipa_call_summary *es2 = ipa_call_summaries->get (cbe);
es2 = ipa_call_summaries->get_create (cbe);
ipa_call_summaries->duplicate (edge, cbe, es, es2);
/* Unlike speculative edges, callback edges have no real
size or time; the call doesn't exist. Reflect that in
their summaries. */
es2->call_stmt_size = 0;
es2->call_stmt_time = 0;
}
}
}
/* TODO: When conditional jump or switch is known to be constant, but

5
gcc/ipa-inline-analysis.cc
View File

@ -417,6 +417,11 @@ do_estimate_growth_1 (struct cgraph_node *node, void *data)
{
gcc_checking_assert (e->inline_failed);
/* Don't count callback edges into growth, since they are never inlined
anyway. */
if (e->callback)
continue;
if (cgraph_inline_failed_type (e->inline_failed) == CIF_FINAL_ERROR
|| !opt_for_fn (e->caller->decl, optimize))
{

12
gcc/ipa-inline-transform.cc
View File

@ -845,7 +845,17 @@ inline_transform (struct cgraph_node *node)
if (!e->inline_failed)
has_inline = true;
next = e->next_callee;
cgraph_edge::redirect_call_stmt_to_callee (e);
if (e->has_callback)
{
/* Redirect callback edges when redirecting their carrying edge. */
cgraph_edge *cbe;
cgraph_edge::redirect_call_stmt_to_callee (e);
for (cbe = e->first_callback_edge (); cbe;
cbe = cbe->next_callback_edge ())
cgraph_edge::redirect_call_stmt_to_callee (cbe);
}
else
cgraph_edge::redirect_call_stmt_to_callee (e);
}
node->remove_all_references ();

37
gcc/ipa-param-manipulation.cc
View File

@ -50,6 +50,7 @@ along with GCC; see the file COPYING3. If not see
#include "sreal.h"
#include "ipa-cp.h"
#include "ipa-prop.h"
#include "attr-callback.h"
/* Actual prefixes of different newly synthetized parameters. Keep in sync
with IPA_PARAM_PREFIX_* defines. */
@ -308,6 +309,16 @@ drop_type_attribute_if_params_changed_p (tree name)
return false;
}
/* Return TRUE if the attribute should be dropped in the decl it is sitting on
changes. Primarily affects attributes working with the decls arguments. */
static bool
drop_decl_attribute_if_params_changed_p (tree name)
{
if (is_attribute_p (CALLBACK_ATTR_IDENT, name))
return true;
return false;
}
/* Build and return a function type just like ORIG_TYPE but with parameter
types given in NEW_PARAM_TYPES - which can be NULL if, but only if,
ORIG_TYPE itself has NULL TREE_ARG_TYPEs. If METHOD2FUNC is true, also make
@ -488,11 +499,12 @@ ipa_param_adjustments::method2func_p (tree orig_type)
performing all atored modifications. TYPE_ORIGINAL_P should be true when
OLD_TYPE refers to the type before any IPA transformations, as opposed to a
type that can be an intermediate one in between various IPA
transformations. */
transformations. Set pointee of ARGS_MODIFIED (if provided) to TRUE if the
type's arguments were changed. */
tree
ipa_param_adjustments::build_new_function_type (tree old_type,
bool type_original_p)
ipa_param_adjustments::build_new_function_type (
tree old_type, bool type_original_p, bool *args_modified /* = NULL */)
{
auto_vec<tree,16> new_param_types, *new_param_types_p;
if (prototype_p (old_type))
@ -518,6 +530,8 @@ ipa_param_adjustments::build_new_function_type (tree old_type,
|| get_original_index (index) != (int)index)
modified = true;
if (args_modified)
*args_modified = modified;
return build_adjusted_function_type (old_type, new_param_types_p,
method2func_p (old_type), m_skip_return,
@ -536,10 +550,11 @@ ipa_param_adjustments::adjust_decl (tree orig_decl)
{
tree new_decl = copy_node (orig_decl);
tree orig_type = TREE_TYPE (orig_decl);
bool args_modified = false;
if (prototype_p (orig_type)
|| (m_skip_return && !VOID_TYPE_P (TREE_TYPE (orig_type))))
{
tree new_type = build_new_function_type (orig_type, false);
tree new_type = build_new_function_type (orig_type, false, &args_modified);
TREE_TYPE (new_decl) = new_type;
}
if (method2func_p (orig_type))
@ -556,6 +571,20 @@ ipa_param_adjustments::adjust_decl (tree orig_decl)
if (m_skip_return)
DECL_IS_MALLOC (new_decl) = 0;
/* If the decl's arguments changed, we might need to drop some attributes. */
if (args_modified && DECL_ATTRIBUTES (new_decl))
{
tree t = DECL_ATTRIBUTES (new_decl);
tree *last = &DECL_ATTRIBUTES (new_decl);
DECL_ATTRIBUTES (new_decl) = NULL;
for (; t; t = TREE_CHAIN (t))
if (!drop_decl_attribute_if_params_changed_p (get_attribute_name (t)))
{
*last = copy_node (t);
TREE_CHAIN (*last) = NULL;
last = &TREE_CHAIN (*last);
}
}
return new_decl;
}

3
gcc/ipa-param-manipulation.h
View File

@ -229,7 +229,8 @@ public:
/* Return if the first parameter is left intact. */
bool first_param_intact_p ();
/* Build a function type corresponding to the modified call. */
tree build_new_function_type (tree old_type, bool type_is_original_p);
tree build_new_function_type (tree old_type, bool type_is_original_p,
bool *args_modified = NULL);
/* Build a declaration corresponding to the target of the modified call. */
tree adjust_decl (tree orig_decl);
/* Fill a vector marking which parameters are intact by the described

102
gcc/ipa-prop.cc
View File

@ -61,6 +61,8 @@ along with GCC; see the file COPYING3. If not see
#include "value-range-storage.h"
#include "vr-values.h"
#include "lto-streamer.h"
#include "attribs.h"
#include "attr-callback.h"
/* Function summary where the parameter infos are actually stored. */
ipa_node_params_t *ipa_node_params_sum = NULL;
@ -324,6 +326,10 @@ ipa_get_param_decl_index (class ipa_node_params *info, tree ptree)
return ipa_get_param_decl_index_1 (info->descriptors, ptree);
}
static void
ipa_duplicate_jump_function (cgraph_edge *src, cgraph_edge *dst,
ipa_jump_func *src_jf, ipa_jump_func *dst_jf);
/* Populate the param_decl field in parameter DESCRIPTORS that correspond to
NODE. */
@ -2416,6 +2422,18 @@ skip_a_safe_conversion_op (tree t)
return t;
}
/* Initializes ipa_edge_args summary of CBE given its callback-carrying edge.
This primarily means allocating the correct amount of jump functions. */
static inline void
init_callback_edge_summary (struct cgraph_edge *cbe, tree attr)
{
ipa_edge_args *cb_args = ipa_edge_args_sum->get_create (cbe);
size_t jf_vec_length = callback_num_args(attr);
vec_safe_grow_cleared (cb_args->jump_functions,
jf_vec_length, true);
}
/* Compute jump function for all arguments of callsite CS and insert the
information in the jump_functions array in the ipa_edge_args corresponding
to this callsite. */
@ -2441,6 +2459,7 @@ ipa_compute_jump_functions_for_edge (struct ipa_func_body_info *fbi,
if (ipa_func_spec_opts_forbid_analysis_p (cs->caller))
return;
auto_vec<cgraph_edge*> callback_edges;
for (n = 0; n < arg_num; n++)
{
struct ipa_jump_func *jfunc = ipa_get_ith_jump_func (args, n);
@ -2519,10 +2538,57 @@ ipa_compute_jump_functions_for_edge (struct ipa_func_body_info *fbi,
arg = skip_a_safe_conversion_op (arg);
if (is_gimple_ip_invariant (arg)
|| (VAR_P (arg)
&& is_global_var (arg)
&& TREE_READONLY (arg)))
ipa_set_jf_constant (jfunc, arg, cs);
|| (VAR_P (arg) && is_global_var (arg) && TREE_READONLY (arg)))
{
ipa_set_jf_constant (jfunc, arg, cs);
if (TREE_CODE (arg) == ADDR_EXPR)
{
tree pointee = TREE_OPERAND (arg, 0);
if (TREE_CODE (pointee) == FUNCTION_DECL && !cs->callback
&& cs->callee)
{
/* Argument is a pointer to a function. Look for a callback
attribute describing this argument. */
tree callback_attr
= lookup_attribute (CALLBACK_ATTR_IDENT,
DECL_ATTRIBUTES (cs->callee->decl));
for (; callback_attr;
callback_attr
= lookup_attribute (CALLBACK_ATTR_IDENT,
TREE_CHAIN (callback_attr)))
if (callback_get_fn_index (callback_attr) == n)
break;
/* If no callback attribute is found, check if the function is
a special case. */
if (!callback_attr
&& callback_is_special_cased (cs->callee->decl, call))
{
callback_attr
= callback_special_case_attr (cs->callee->decl);
/* Check if the special attribute describes the correct
attribute, as a special cased function might have
multiple callbacks. */
if (callback_get_fn_index (callback_attr) != n)
callback_attr = NULL;
}
/* If a callback attribute describing this pointer is found,
create a callback edge to the pointee function to
allow for further optimizations. */
if (callback_attr)
{
cgraph_node *kernel_node
= cgraph_node::get_create (pointee);
unsigned callback_id = n;
cgraph_edge *cbe
= cs->make_callback (kernel_node, callback_id);
init_callback_edge_summary (cbe, callback_attr);
callback_edges.safe_push (cbe);
}
}
}
}
else if (!is_gimple_reg_type (TREE_TYPE (arg))
&& TREE_CODE (arg) == PARM_DECL)
{
@ -2580,6 +2646,34 @@ ipa_compute_jump_functions_for_edge (struct ipa_func_body_info *fbi,
|| POINTER_TYPE_P (param_type)))
determine_known_aggregate_parts (fbi, call, arg, param_type, jfunc);
}
if (!callback_edges.is_empty ())
{
/* For every callback edge, fetch jump functions of arguments
passed to them and copy them over to their respective summaries.
This avoids recalculating them for every callback edge, since their
arguments are just passed through. */
unsigned j;
for (j = 0; j < callback_edges.length (); j++)
{
cgraph_edge *callback_edge = callback_edges[j];
ipa_edge_args *cb_summary
= ipa_edge_args_sum->get_create (callback_edge);
auto_vec<int> arg_mapping
= callback_get_arg_mapping (callback_edge, cs);
unsigned i;
for (i = 0; i < arg_mapping.length (); i++)
{
if (arg_mapping[i] == -1)
continue;
class ipa_jump_func *src
= ipa_get_ith_jump_func (args, arg_mapping[i]);
class ipa_jump_func *dst = ipa_get_ith_jump_func (cb_summary, i);
ipa_duplicate_jump_function (cs, callback_edge, src, dst);
}
}
}
if (!useful_context)
vec_free (args->polymorphic_call_contexts);
}

6
gcc/lto-cgraph.cc
View File

@ -274,6 +274,9 @@ lto_output_edge (struct lto_simple_output_block *ob, struct cgraph_edge *edge,
bp_pack_value (&bp, edge->speculative_id, 16);
bp_pack_value (&bp, edge->indirect_inlining_edge, 1);
bp_pack_value (&bp, edge->speculative, 1);
bp_pack_value (&bp, edge->callback, 1);
bp_pack_value (&bp, edge->has_callback, 1);
bp_pack_value (&bp, edge->callback_id, 16);
bp_pack_value (&bp, edge->call_stmt_cannot_inline_p, 1);
gcc_assert (!edge->call_stmt_cannot_inline_p
|| edge->inline_failed != CIF_BODY_NOT_AVAILABLE);
@ -1539,6 +1542,9 @@ input_edge (class lto_input_block *ib, vec<symtab_node *> nodes,
edge->indirect_inlining_edge = bp_unpack_value (&bp, 1);
edge->speculative = bp_unpack_value (&bp, 1);
edge->callback = bp_unpack_value(&bp, 1);
edge->has_callback = bp_unpack_value(&bp, 1);
edge->callback_id = bp_unpack_value(&bp, 16);
edge->lto_stmt_uid = stmt_id;
edge->speculative_id = speculative_id;
edge->inline_failed = inline_failed;

24
gcc/omp-builtins.def
View File

@ -358,35 +358,35 @@ DEF_GOMP_BUILTIN (BUILT_IN_GOMP_LOOP_ULL_ORDERED_RUNTIME_NEXT,
DEF_GOMP_BUILTIN (BUILT_IN_GOMP_PARALLEL_LOOP_STATIC,
"GOMP_parallel_loop_static",
BT_FN_VOID_OMPFN_PTR_UINT_LONG_LONG_LONG_LONG_UINT,
ATTR_NOTHROW_LIST)
ATTR_CALLBACK_GOMP_LIST)
DEF_GOMP_BUILTIN (BUILT_IN_GOMP_PARALLEL_LOOP_DYNAMIC,
"GOMP_parallel_loop_dynamic",
BT_FN_VOID_OMPFN_PTR_UINT_LONG_LONG_LONG_LONG_UINT,
ATTR_NOTHROW_LIST)
ATTR_CALLBACK_GOMP_LIST)
DEF_GOMP_BUILTIN (BUILT_IN_GOMP_PARALLEL_LOOP_GUIDED,
"GOMP_parallel_loop_guided",
BT_FN_VOID_OMPFN_PTR_UINT_LONG_LONG_LONG_LONG_UINT,
ATTR_NOTHROW_LIST)
ATTR_CALLBACK_GOMP_LIST)
DEF_GOMP_BUILTIN (BUILT_IN_GOMP_PARALLEL_LOOP_RUNTIME,
"GOMP_parallel_loop_runtime",
BT_FN_VOID_OMPFN_PTR_UINT_LONG_LONG_LONG_UINT,
ATTR_NOTHROW_LIST)
ATTR_CALLBACK_GOMP_LIST)
DEF_GOMP_BUILTIN (BUILT_IN_GOMP_PARALLEL_LOOP_NONMONOTONIC_DYNAMIC,
"GOMP_parallel_loop_nonmonotonic_dynamic",
BT_FN_VOID_OMPFN_PTR_UINT_LONG_LONG_LONG_LONG_UINT,
ATTR_NOTHROW_LIST)
ATTR_CALLBACK_GOMP_LIST)
DEF_GOMP_BUILTIN (BUILT_IN_GOMP_PARALLEL_LOOP_NONMONOTONIC_GUIDED,
"GOMP_parallel_loop_nonmonotonic_guided",
BT_FN_VOID_OMPFN_PTR_UINT_LONG_LONG_LONG_LONG_UINT,
ATTR_NOTHROW_LIST)
ATTR_CALLBACK_GOMP_LIST)
DEF_GOMP_BUILTIN (BUILT_IN_GOMP_PARALLEL_LOOP_NONMONOTONIC_RUNTIME,
"GOMP_parallel_loop_nonmonotonic_runtime",
BT_FN_VOID_OMPFN_PTR_UINT_LONG_LONG_LONG_UINT,
ATTR_NOTHROW_LIST)
ATTR_CALLBACK_GOMP_LIST)
DEF_GOMP_BUILTIN (BUILT_IN_GOMP_PARALLEL_LOOP_MAYBE_NONMONOTONIC_RUNTIME,
"GOMP_parallel_loop_maybe_nonmonotonic_runtime",
BT_FN_VOID_OMPFN_PTR_UINT_LONG_LONG_LONG_UINT,
ATTR_NOTHROW_LIST)
ATTR_CALLBACK_GOMP_LIST)
DEF_GOMP_BUILTIN (BUILT_IN_GOMP_LOOP_END, "GOMP_loop_end",
BT_FN_VOID, ATTR_NOTHROW_LEAF_LIST)
DEF_GOMP_BUILTIN (BUILT_IN_GOMP_LOOP_END_CANCEL, "GOMP_loop_end_cancel",
@ -409,10 +409,10 @@ DEF_GOMP_BUILTIN (BUILT_IN_GOMP_INTEROP, "GOMP_interop",
BT_FN_VOID_INT_INT_PTR_PTR_PTR_INT_PTR_INT_PTR_UINT_PTR,
ATTR_NOTHROW_LIST)
DEF_GOMP_BUILTIN (BUILT_IN_GOMP_PARALLEL, "GOMP_parallel",
BT_FN_VOID_OMPFN_PTR_UINT_UINT, ATTR_NOTHROW_LIST)
BT_FN_VOID_OMPFN_PTR_UINT_UINT, ATTR_CALLBACK_GOMP_LIST)
DEF_GOMP_BUILTIN (BUILT_IN_GOMP_PARALLEL_REDUCTIONS,
"GOMP_parallel_reductions",
BT_FN_UINT_OMPFN_PTR_UINT_UINT, ATTR_NOTHROW_LIST)
BT_FN_UINT_OMPFN_PTR_UINT_UINT, ATTR_CALLBACK_GOMP_LIST)
DEF_GOMP_BUILTIN (BUILT_IN_GOMP_TASK, "GOMP_task",
BT_FN_VOID_OMPFN_PTR_OMPCPYFN_LONG_LONG_BOOL_UINT_PTR_INT_PTR,
ATTR_NOTHROW_LIST)
@ -430,7 +430,7 @@ DEF_GOMP_BUILTIN (BUILT_IN_GOMP_SECTIONS_NEXT, "GOMP_sections_next",
BT_FN_UINT, ATTR_NOTHROW_LEAF_LIST)
DEF_GOMP_BUILTIN (BUILT_IN_GOMP_PARALLEL_SECTIONS,
"GOMP_parallel_sections",
BT_FN_VOID_OMPFN_PTR_UINT_UINT_UINT, ATTR_NOTHROW_LIST)
BT_FN_VOID_OMPFN_PTR_UINT_UINT_UINT, ATTR_CALLBACK_GOMP_LIST)
DEF_GOMP_BUILTIN (BUILT_IN_GOMP_SECTIONS_END, "GOMP_sections_end",
BT_FN_VOID, ATTR_NOTHROW_LEAF_LIST)
DEF_GOMP_BUILTIN (BUILT_IN_GOMP_SECTIONS_END_CANCEL,
@ -471,7 +471,7 @@ DEF_GOMP_BUILTIN (BUILT_IN_GOMP_TARGET_MAP_INDIRECT_PTR,
DEF_GOMP_BUILTIN (BUILT_IN_GOMP_TEAMS4, "GOMP_teams4",
BT_FN_BOOL_UINT_UINT_UINT_BOOL, ATTR_NOTHROW_LIST)
DEF_GOMP_BUILTIN (BUILT_IN_GOMP_TEAMS_REG, "GOMP_teams_reg",
BT_FN_VOID_OMPFN_PTR_UINT_UINT_UINT, ATTR_NOTHROW_LIST)
BT_FN_VOID_OMPFN_PTR_UINT_UINT_UINT, ATTR_CALLBACK_GOMP_LIST)
DEF_GOMP_BUILTIN (BUILT_IN_GOMP_TASKGROUP_REDUCTION_REGISTER,
"GOMP_taskgroup_reduction_register",
BT_FN_VOID_PTR, ATTR_NOTHROW_LEAF_LIST)

19
gcc/testsuite/gcc.dg/ipa/ipcp-cb-spec1.c Normal file
View File

@ -0,0 +1,19 @@
/* Test that GOMP_task is special cased when cpyfn is NULL. */
/* { dg-do run } */
/* { dg-options "-O3 -fopenmp -flto -std=gnu99 -fdump-ipa-cp-details" } */
/* { dg-require-effective-target fopenmp } */
/* { dg-require-effective-target lto } */
void test(int c) {
for (int i = 0; i < c; i++)
if (!__builtin_constant_p(c))
__builtin_abort();
}
int main() {
#pragma omp task
test(7);
return 0;
}
/* { dg-final { scan-wpa-ipa-dump "Creating a specialized node of main._omp_fn" "cp" } } */

21
gcc/testsuite/gcc.dg/ipa/ipcp-cb-spec2.c Normal file
View File

@ -0,0 +1,21 @@
/* Check that GOMP_task doesn't produce callback edges when cpyfn is not
NULL. */
/* { dg-do run } */
/* { dg-options "-O3 -fopenmp -flto -std=gnu99 -fdump-ipa-cp-details" } */
/* { dg-require-effective-target fopenmp } */
/* { dg-require-effective-target lto } */
void test(int *a) {
for (int i = 0; i < 100; i++) {
a[i] = i;
}
}
int main() {
int a[100];
__builtin_memset (a, 0, sizeof (a));
#pragma omp task
test (a);
}
/* { dg-final { scan-ipa-dump-not "Created callback edge" "cp" } } */

25
gcc/testsuite/gcc.dg/ipa/ipcp-cb1.c Normal file
View File

@ -0,0 +1,25 @@
/* Test that we can propagate constants into outlined OpenMP kernels.
This tests the underlying callback attribute and its related edges. */
/* { dg-do run } */
/* { dg-options "-O3 -fopenmp -flto -std=gnu99 -fdump-ipa-cp-details" } */
/* { dg-require-effective-target fopenmp } */
/* { dg-require-effective-target lto } */
int a[100];
void test(int c) {
#pragma omp parallel for
for (int i = 0; i < c; i++) {
if (!__builtin_constant_p(c)) {
__builtin_abort();
}
a[i] = i;
}
}
int main() {
test(100);
return a[5] - 5;
}
/* { dg-final { scan-wpa-ipa-dump "Creating a specialized node of test._omp_fn" "cp" } } */
/* { dg-final { scan-wpa-ipa-dump "Aggregate replacements: 0\\\[0]=100\\(by_ref\\)" "cp" } } */

7
gcc/tree-core.h
View File

@ -98,6 +98,13 @@ struct die_struct;
/* Nonzero if this is a function expected to end with an exception. */
#define ECF_XTHROW (1 << 16)
/* Flags for various callback attribute combinations. These constants are only
meant to be used for the construction of builtin functions. They were only
added because Fortran uses them for attributes of builtins. */
/* callback(1, 2) */
#define ECF_CB_1_2 (1 << 17)
/* Call argument flags. */
/* Nonzero if the argument is not used by the function. */

27
gcc/tree-inline.cc
View File

@ -2359,6 +2359,19 @@ copy_bb (copy_body_data *id, basic_block bb,
indirect->count
= copy_basic_block->count.apply_probability (prob);
}
/* If edge is a callback-carrying edge, copy all its
attached edges as well. */
else if (edge->has_callback)
{
edge
= edge->clone (id->dst_node, call_stmt,
gimple_uid (stmt), num, den, true);
cgraph_edge *e;
for (e = old_edge->first_callback_edge (); e;
e = e->next_callback_edge ())
edge = e->clone (id->dst_node, call_stmt,
gimple_uid (stmt), num, den, true);
}
else
{
edge = edge->clone (id->dst_node, call_stmt,
@ -3051,8 +3064,18 @@ redirect_all_calls (copy_body_data * id, basic_block bb)
{
if (!id->killed_new_ssa_names)
id->killed_new_ssa_names = new hash_set<tree> (16);
cgraph_edge::redirect_call_stmt_to_callee (edge,
id->killed_new_ssa_names);
cgraph_edge::redirect_call_stmt_to_callee (
edge, id->killed_new_ssa_names);
if (edge->has_callback)
{
/* When redirecting a carrying edge, we need to redirect its
attached edges as well. */
cgraph_edge *cbe;
for (cbe = edge->first_callback_edge (); cbe;
cbe = cbe->next_callback_edge ())
cgraph_edge::redirect_call_stmt_to_callee (
cbe, id->killed_new_ssa_names);
}
if (stmt == last && id->call_stmt && maybe_clean_eh_stmt (stmt))
gimple_purge_dead_eh_edges (bb);

11
gcc/tree.cc
View File

@ -75,6 +75,7 @@ along with GCC; see the file COPYING3. If not see
#include "dfp.h"
#include "asan.h"
#include "ubsan.h"
#include "attr-callback.h"
/* Names of tree components.
Used for printing out the tree and error messages. */
@ -10000,7 +10001,15 @@ set_call_expr_flags (tree decl, int flags)
DECL_ATTRIBUTES (decl)
= tree_cons (get_identifier ("expected_throw"),
NULL, DECL_ATTRIBUTES (decl));
/* Looping const or pure is implied by noreturn.
if (flags & ECF_CB_1_2)
{
tree attr = callback_build_attr (1, 1, 2);
TREE_CHAIN (attr) = DECL_ATTRIBUTES (decl);
DECL_ATTRIBUTES (decl) = attr;
}
/* Looping const or pure is implied by noreturn.
There is currently no way to declare looping const or looping pure alone. */
gcc_assert (!(flags & ECF_LOOPING_CONST_OR_PURE)
|| ((flags & ECF_NORETURN) && (flags & (ECF_CONST | ECF_PURE))));