Re: Update LTO plugin interface

[Dave Korn <dave dot korn dot cygwin at gmail dot com>]

On 01/12/2010 23:06, Cary Coutant wrote:

>> My suspicion is that the LTO plugin can only introduce a small bounded
>> set of new symbol references, namely those which we assume can be
>> satisified from -lc or -lgcc.  Is that true?
> 
> Exactly. 

  Potentially also gcov, ssp, mudflap?

> The plugin API, as implemented in gold (not sure about gnu ld), does
> maintain the original order of input files as far as symbol binding is
> concerned. When IR files are claimed, the plugin provides the list of
> symbols defined and referenced, and the linker builds the symbol table
> as if those files were linked in at that particular spot in the
> command line. When the compiler provides real definitions of those
> symbols later, the real definitions simply replace the "placeholders"
> that were left in the linker's symbol table. 

  We just ran into a new problem with that:

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=45375#c12

  (Brief summary: no symbol type info is conveyed in the LTO symtab, and in
the ELF linker, _bfd_elf_merge_symbol errors out when the real symbol is TLS
and the LTO symtab one didn't have the same type, and it does so too early for
the plugin api's multiple_definition linker callback hook to do anything about
it.)

> As Ian noted, if the compiler introduces new references that weren't
> there before, the new references must be from a limited set of
> libcalls that the backend can introduce, and those should all be
> resolved with an extra pass through -lc or -lgcc. That's not exactly
> pretty, but I don't see how it destroys the notion of link order --
> the only way those new symbols could have been resolved differently is
> if a user library interposed definitions for the libcall, and those
> certainly can't be what the compiler intended to bind to. In PR 12248,
> I think it's questionable to claim that the compiler-introduced call
> to __udivdi3 should not resolve to the version in libgcc. Sure, I
> understand it's useful for library developers while debugging and
> testing, but an ordinary user certainly can't count on his own
> definition of that routine to get called -- the compiler might
> generate the division inline, or call a different specialized version.
> All of these routines are outside the user's namespace, and we should
> be able to optimize without regard for what the user's libraries might
> contain.

  I tend to follow this theory.  I think that the current approach should be
sufficient, but I think we probably need to arrange a few more pass-throughs:
for (some/all of?) the libraries I mentioned above, and HJ pointed out another
relevant issue: we should pass-through the crt endfiles as well, because the
new object files introduced by LTO might pull in new dynamic references from
libc et. al., and apparently you mustn't do that after crtn.o has been linked.

http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42690#c27

  I was going to test something like the attached.


    cheers,
      DaveK

Index: gcc/gcc.c
===================================================================
--- gcc/gcc.c	(revision 167334)
+++ gcc/gcc.c	(working copy)
@@ -264,7 +264,9 @@ static const char *print_asm_header_spec_function
 static const char *compare_debug_dump_opt_spec_function (int, const char **);
 static const char *compare_debug_self_opt_spec_function (int, const char **);
 static const char *compare_debug_auxbase_opt_spec_function (int, const char **);
+static const char *gen_pass_through_spec (int, const char **, bool, bool);
 static const char *pass_through_libs_spec_func (int, const char **);
+static const char *pass_through_objs_spec_func (int, const char **);
 
 /* The Specs Language
 
@@ -637,7 +639,9 @@ proper position among the other output files.  */
     -plugin %(linker_plugin_file) \
     -plugin-opt=%(lto_wrapper) \
     -plugin-opt=-fresolution=%u.res \
+    %{fprofile-arcs|fprofile-generate*|coverage:-plugin-opt=-pass-through=-lgcov} \
     %{!nostdlib:%{!nodefaultlibs:%:pass-through-libs(%(link_gcc_c_sequence))}} \
+    %{!A:%{!nostdlib:%{!nostartfiles:%:pass-through-objs(%E)}}}
     } \
     %{flto*:%<fcompare-debug*} \
     %{flto*} %l " LINK_PIE_SPEC \
@@ -648,7 +652,7 @@ proper position among the other output files.  */
     %(mflib) " STACK_SPLIT_SPEC "\
     %{fprofile-arcs|fprofile-generate*|coverage:-lgcov}\
     %{!nostdlib:%{!nodefaultlibs:%(link_ssp) %(link_gcc_c_sequence)}}\
-    %{!A:%{!nostdlib:%{!nostartfiles:%E}}} %{T*} }}}}}}"
+    %{!fuse-linker-plugin:%{!A:%{!nostdlib:%{!nostartfiles:%E}}}} %{T*} }}}}}}"
 #endif
 
 #ifndef LINK_LIBGCC_SPEC
@@ -1221,6 +1225,7 @@ static const struct spec_function static_spec_func
   { "compare-debug-self-opt",	compare_debug_self_opt_spec_function },
   { "compare-debug-auxbase-opt", compare_debug_auxbase_opt_spec_function },
   { "pass-through-libs",	pass_through_libs_spec_func },
+  { "pass-through-objs",	pass_through_objs_spec_func },
 #ifdef EXTRA_SPEC_FUNCTIONS
   EXTRA_SPEC_FUNCTIONS
 #endif
@@ -8171,8 +8176,8 @@ compare_debug_auxbase_opt_spec_function (int arg,
    prepended with the plugin option to cause them to be passed through
    to the final link after all the new object files have been added.  */
 
-const char *
-pass_through_libs_spec_func (int argc, const char **argv)
+static const char *
+gen_pass_through_spec (int argc, const char **argv, bool do_libs, bool do_objs)
 {
   char *prepended = xstrdup (" ");
   int n;
@@ -8185,7 +8190,7 @@ compare_debug_auxbase_opt_spec_function (int arg,
       /* Anything that isn't an option is a full path to an output
          file; pass it through if it ends in '.a'.  Among options,
 	 pass only -l.  */
-      if (argv[n][0] == '-' && argv[n][1] == 'l')
+      if (do_libs && argv[n][0] == '-' && argv[n][1] == 'l')
 	{
 	  const char *lopt = argv[n] + 2;
 	  /* Handle both joined and non-joined -l options.  If for any
@@ -8198,13 +8203,32 @@ compare_debug_auxbase_opt_spec_function (int arg,
 	  prepended = concat (prepended, "-plugin-opt=-pass-through=-l",
 		lopt, " ", NULL);
 	}
-      else if (!strcmp (".a", argv[n] + strlen (argv[n]) - 2))
+      else if (do_libs && !strcmp (".a", argv[n] + strlen (argv[n]) - 2))
 	{
 	  prepended = concat (prepended, "-plugin-opt=-pass-through=",
 		argv[n], " ", NULL);
 	}
+      else if (do_objs && !strcmp (TARGET_OBJECT_SUFFIX,
+				  argv[n] + strlen (argv[n])
+				  - strlen (TARGET_OBJECT_SUFFIX)))
+	{
+	  prepended = concat (prepended, "-plugin-opt=-pass-through=",
+		argv[n], " ", NULL);
+	}
       if (prepended != old)
 	free (old);
     }
   return prepended;
 }
+
+static const char *
+pass_through_libs_spec_func (int argc, const char **argv)
+{
+  return gen_pass_through_spec (argc, argv, true, false);
+}
+
+static const char *
+pass_through_objs_spec_func (int argc, const char **argv)
+{
+  return gen_pass_through_spec (argc, argv, false, true);
+}