Re: [PATCH] PR ld/20103: Skip an archive element if not added by linker

["H.J. Lu" <hjl dot tools at gmail dot com>]

On Mon, May 23, 2016 at 7:29 AM, Alan Modra <amodra@gmail.com> wrote:
> On Wed, May 18, 2016 at 05:10:55AM -0700, H.J. Lu wrote:
>> On Tue, May 17, 2016 at 11:49 PM, Alan Modra <amodra@gmail.com> wrote:
>> > On Tue, May 17, 2016 at 07:24:08PM -0700, H.J. Lu wrote:
>> >> During archive rescan to resolve symbol references for files added by
>> >> LTO, linker add_archive_element callback is called to check if an
>> >> archive element should added.  After all IR symbols have been claimed,
>> >> linker won't claim new IR symbols and shouldn't add the LTO archive
>> >> element.  This patch updates linker add_archive_element callback to
>> >> return FALSE when seeing an LTO archive element during rescan and
>> >> changes ELF linker to skip such archive element.  Other linker backends
>> >> may also need similar update.
>> >>
>> >> Tested on x86.  OK for master?
>> >
>> > Not without a lot more explanation and justification why this is
>> > correct, in particular the plugin_call_claim_file change.
>> >
>>
>> plugin_call_claim_file is only called from plugin_object_p to check
>> if an input is IR, not really claim symbols. which is done by
>> plugin_call_all_symbols_read, which sets no_more_claiming.  Whether
>> an input is an IR file is independent of if there is more IR symbols to
>> claim.
>>
>> During archive rescan, if plugin_object_p rejects an IR archive element,
>> linker will treat it as a non-IR element, which leads to error.  In stead
>> plugin_object_p should always accept an IR object,  We won't claim the
>> IR symbols when no_more_claiming is set. My change just makes linker
>> consistent on whether an input file is an IR file.
>
> OK, I've now had a look at everything and I think your patch is OK
> if you edit every file that calls add_archive_element.  You can't
> change a pass/fail return status to add/don't add and not change all
> users of the function.
>

Here is the updated patch.  OK for master?

-- 
H.J.

From f490c2d5e99955a284d105b19d27015cd117aee0 Mon Sep 17 00:00:00 2001
From: "H.J. Lu" <hjl.tools@gmail.com>
Date: Tue, 17 May 2016 07:59:52 -0700
Subject: [PATCH] Skip an archive element if not added by linker

During archive rescan to resolve symbol references for files added by
LTO, linker add_archive_element callback is called to check if an
archive element should added.  After all IR symbols have been claimed,
linker won't claim new IR symbols and shouldn't add the LTO archive
element.  This patch updates linker add_archive_element callback to
return FALSE when seeing an LTO archive element during rescan and
changes ELF linker to skip such archive element.

bfd/

	PR ld/20103
	* cofflink.c (coff_link_check_archive_element): Return TRUE if
	linker add_archive_element callback returns FALSE.
	* ecoff.c (ecoff_link_check_archive_element): Likewise.
	* elf64-ia64-vms.c (elf64_vms_link_add_archive_symbols): Skip
	archive element if linker add_archive_element callback returns
	FALSE.
	* elflink.c (elf_link_add_archive_symbols): Likewise.
	* pdp11.c (aout_link_check_ar_symbols): Likewise.
	* vms-alpha.c (alpha_vms_link_add_archive_symbols): Likewise.
	* xcofflink.c (xcoff_link_check_dynamic_ar_symbols): Likewise.
	(xcoff_link_check_ar_symbols): Likewise.

ld/

	PR ld/20103
	* ldmain.c (add_archive_element): Don't claim new IR symbols
	after all IR symbols have been claimed.
	* plugin.c (plugin_call_claim_file): Remove no_more_claiming
	check.
	* testsuite/ld-plugin/lto.exp (pr20103): New proc.
	Run PR ld/20103 tests.
	* testsuite/ld-plugin/pr20103a.c: New file.
	* testsuite/ld-plugin/pr20103b.c: Likewise.
	* testsuite/ld-plugin/pr20103c.c: Likewise.
---
 bfd/cofflink.c                    |  3 +-
 bfd/ecoff.c                       |  4 +-
 bfd/elf64-ia64-vms.c              |  2 +-
 bfd/elflink.c                     |  2 +-
 bfd/pdp11.c                       |  2 +-
 bfd/vms-alpha.c                   |  2 +-
 bfd/xcofflink.c                   |  4 +-
 ld/ldmain.c                       | 12 +++++-
 ld/plugin.c                       |  2 -
 ld/testsuite/ld-plugin/lto.exp    | 85 +++++++++++++++++++++++++++++++++++++++
 ld/testsuite/ld-plugin/pr20103a.c |  8 ++++
 ld/testsuite/ld-plugin/pr20103b.c |  3 ++
 ld/testsuite/ld-plugin/pr20103c.c |  6 +++
 13 files changed, 123 insertions(+), 12 deletions(-)
 create mode 100644 ld/testsuite/ld-plugin/pr20103a.c
 create mode 100644 ld/testsuite/ld-plugin/pr20103b.c
 create mode 100644 ld/testsuite/ld-plugin/pr20103c.c

diff --git a/bfd/cofflink.c b/bfd/cofflink.c
index 4756fc3..b8a85b0 100644
--- a/bfd/cofflink.c
+++ b/bfd/cofflink.c
@@ -212,8 +212,9 @@ coff_link_check_archive_element (bfd *abfd,
   if (h->type != bfd_link_hash_undefined)
     return TRUE;
 
+  /* Include this element?  */
   if (!(*info->callbacks->add_archive_element) (info, abfd, name, &abfd))
-    return FALSE;
+    return TRUE;
   *pneeded = TRUE;
 
   return coff_link_add_object_symbols (abfd, info);
diff --git a/bfd/ecoff.c b/bfd/ecoff.c
index 031abdf..d618572 100644
--- a/bfd/ecoff.c
+++ b/bfd/ecoff.c
@@ -3544,9 +3544,9 @@ ecoff_link_check_archive_element (bfd *abfd,
   if (h->type != bfd_link_hash_undefined)
     return TRUE;
 
-  /* Include this element.  */
+  /* Include this element?  */
   if (!(*info->callbacks->add_archive_element) (info, abfd, name, &abfd))
-    return FALSE;
+    return TRUE;
   *pneeded = TRUE;
 
   return ecoff_link_add_object_symbols (abfd, info);
diff --git a/bfd/elf64-ia64-vms.c b/bfd/elf64-ia64-vms.c
index 760e1db..4482646 100644
--- a/bfd/elf64-ia64-vms.c
+++ b/bfd/elf64-ia64-vms.c
@@ -5361,7 +5361,7 @@ elf64_vms_link_add_archive_symbols (bfd *abfd, struct bfd_link_info *info)
 	 to include it.  We don't need to check anything.  */
       if (! (*info->callbacks->add_archive_element) (info, element,
                                                      h->root.string, &element))
-	return FALSE;
+	continue;
       if (! elf64_vms_link_add_object_symbols (element, info))
 	return FALSE;
 
diff --git a/bfd/elflink.c b/bfd/elflink.c
index 1569e93..b4efd5a 100644
--- a/bfd/elflink.c
+++ b/bfd/elflink.c
@@ -5283,7 +5283,7 @@ elf_link_add_archive_symbols (bfd *abfd, struct bfd_link_info *info)
 
 	  if (!(*info->callbacks
 		->add_archive_element) (info, element, symdef->name, &element))
-	    goto error_return;
+	    continue;
 	  if (!bfd_link_add_symbols (element, info))
 	    goto error_return;
 
diff --git a/bfd/pdp11.c b/bfd/pdp11.c
index bf0cfc3..1f40be5 100644
--- a/bfd/pdp11.c
+++ b/bfd/pdp11.c
@@ -2608,7 +2608,7 @@ aout_link_check_ar_symbols (bfd *abfd,
 	     However, it might be correct.  */
 	  if (!(*info->callbacks
 		->add_archive_element) (info, abfd, name, subsbfd))
-	    return FALSE;
+	    continue;
 	  *pneeded = TRUE;
 	  return TRUE;
 	}
diff --git a/bfd/vms-alpha.c b/bfd/vms-alpha.c
index e6cfc1f..449354d 100644
--- a/bfd/vms-alpha.c
+++ b/bfd/vms-alpha.c
@@ -8202,7 +8202,7 @@ alpha_vms_link_add_archive_symbols (bfd *abfd, struct bfd_link_info *info)
 	 to include it.  We don't need to check anything.  */
       if (!(*info->callbacks
 	    ->add_archive_element) (info, element, h->root.string, &element))
-	return FALSE;
+	continue;
       if (!alpha_vms_link_add_object_symbols (element, info))
 	return FALSE;
 
diff --git a/bfd/xcofflink.c b/bfd/xcofflink.c
index f6030f1..ca40096 100644
--- a/bfd/xcofflink.c
+++ b/bfd/xcofflink.c
@@ -2291,7 +2291,7 @@ xcoff_link_check_dynamic_ar_symbols (bfd *abfd,
 	{
 	  if (!(*info->callbacks
 		->add_archive_element) (info, abfd, name, subsbfd))
-	    return FALSE;
+	    continue;
 	  *pneeded = TRUE;
 	  return TRUE;
 	}
@@ -2363,7 +2363,7 @@ xcoff_link_check_ar_symbols (bfd *abfd,
 	    {
 	      if (!(*info->callbacks
 		    ->add_archive_element) (info, abfd, name, subsbfd))
-		return FALSE;
+		continue;
 	      *pneeded = TRUE;
 	      return TRUE;
 	    }
diff --git a/ld/ldmain.c b/ld/ldmain.c
index 21133ab..7d04be2 100644
--- a/ld/ldmain.c
+++ b/ld/ldmain.c
@@ -796,12 +796,22 @@ add_archive_element (struct bfd_link_info *info,
      BFD, but we still want to output the original BFD filename.  */
   orig_input = *input;
 #ifdef ENABLE_PLUGINS
-  if (link_info.lto_plugin_active && !no_more_claiming)
+  if (link_info.lto_plugin_active)
     {
       /* We must offer this archive member to the plugins to claim.  */
       plugin_maybe_claim (input);
       if (input->flags.claimed)
 	{
+	  if (no_more_claiming)
+	    {
+	      /* Don't claim new IR symbols after all IR symbols have
+		 been claimed.  */
+	      if (trace_files || verbose)
+		info_msg ("%I: no new IR symbols to claimi\n",
+			  &orig_input);
+	      input->flags.claimed = 0;
+	      return FALSE;
+	    }
 	  input->flags.claim_archive = TRUE;
 	  *subsbfd = input->the_bfd;
 	}
diff --git a/ld/plugin.c b/ld/plugin.c
index a60ffca..c951995 100644
--- a/ld/plugin.c
+++ b/ld/plugin.c
@@ -1033,8 +1033,6 @@ plugin_call_claim_file (const struct ld_plugin_input_file *file, int *claimed)
 {
   plugin_t *curplug = plugins_list;
   *claimed = FALSE;
-  if (no_more_claiming)
-    return 0;
   while (curplug && !*claimed)
     {
       if (curplug->claim_file_handler)
diff --git a/ld/testsuite/ld-plugin/lto.exp b/ld/testsuite/ld-plugin/lto.exp
index 6330a17..7a13abb 100644
--- a/ld/testsuite/ld-plugin/lto.exp
+++ b/ld/testsuite/ld-plugin/lto.exp
@@ -460,4 +460,89 @@ if { [is_elf_format] } {
     run_ld_link_exec_tests [] $lto_run_elf_tests
 }
 
+proc pr20103 {cflags libs} {
+    global CC
+
+    set testname "PR ld/20103 ($cflags $libs)"
+    set exec_output [run_host_cmd "$CC" "$cflags $libs"]
+    if { [ regexp "undefined reference to `dead'" $exec_output ] } {
+        pass "$testname (1)"
+    } {
+        fail "$testname (1)"
+    }
+    if { [ regexp "plugin needed to handle lto object" $exec_output ] } {
+        fail "$testname (2)"
+    } {
+        pass "$testname (2)"
+    }
+}
+
+if { [check_lto_fat_available] } {
+    run_cc_link_tests [list \
+	[list \
+	    "Build fatpr20103a.a" \
+	    "$plug_opt" "-flto -ffat-lto-objects" \
+	    {pr20103a.c} {} "fatpr20103a.a"
+	] \
+	[list \
+	    "Build fatpr20103b.a" \
+	    "$plug_opt" "-flto -ffat-lto-objects" \
+	    {pr20103b.c} {} "fatpr20103b.a"
+	] \
+	[list \
+	    "Build fatpr20103c.a" \
+	    "$plug_opt" "-flto -ffat-lto-objects" \
+	    {pr20103c.c} {} "fatpr20103c.a" \
+	] \
+	[list \
+	    "Build thinpr20103a.a" \
+	    "$plug_opt" "-flto -fno-fat-lto-objects" \
+	    {pr20103a.c} {} "thinpr20103a.a"
+	] \
+	[list \
+	    "Build thinpr20103b.a" \
+	    "$plug_opt" "-flto -fno-fat-lto-objects" \
+	    {pr20103b.c} {} "thinpr20103b.a"
+	] \
+	[list \
+	    "Build thinpr20103c.a" \
+	    "$plug_opt" "-flto -fno-fat-lto-objects" \
+	    {pr20103c.c} {} "thinpr20103c.a" \
+	] \
+	[list \
+	    "Build pr20103a" \
+	    "-O2 -flto -Wl,--start-group tmpdir/thinpr20103a.a tmpdir/thinpr20103b.a tmpdir/thinpr20103c.a -Wl,--end-group" \
+	    "-O2 -flto" \
+	    {dummy.c} {} "pr20103a" \
+	] \
+	[list \
+	    "Build pr20103b" \
+	    "-O2 -flto -Wl,--start-group tmpdir/fatpr20103a.a tmpdir/fatpr20103b.a tmpdir/fatpr20103c.a -Wl,--end-group" \
+	    "-O2 -flto" \
+	    {dummy.c} {} "pr20103b" \
+	] \
+	[list \
+	    "Build pr20103c" \
+	    "-O2 -Wl,--start-group tmpdir/fatpr20103a.a tmpdir/fatpr20103b.a tmpdir/fatpr20103c.a -Wl,--end-group" \
+	    "-O2" \
+	    {dummy.c} {} "pr20103c" \
+	] \
+    ]
+    pr20103 "-O2 -flto" "tmpdir/thinpr20103a.a tmpdir/thinpr20103b.a tmpdir/thinpr20103c.a"
+    pr20103 "-O2 -flto" "tmpdir/fatpr20103a.a tmpdir/fatpr20103b.a tmpdir/fatpr20103c.a"
+    pr20103 "-O2" "tmpdir/fatpr20103a.a tmpdir/fatpr20103b.a tmpdir/fatpr20103c.a"
+
+    if { [at_least_gcc_version 4 9] } {
+	run_cc_link_tests [list \
+	    [list \
+		"Build pr20103d" \
+		"-O2 -Wl,--start-group tmpdir/thinpr20103a.a tmpdir/thinpr20103b.a tmpdir/thinpr20103c.a -Wl,--end-group" \
+		"-O2" \
+		{dummy.c} {} "pr20103d" \
+	    ] \
+	]
+	pr20103 "-O2" "tmpdir/thinpr20103a.a tmpdir/thinpr20103b.a tmpdir/thinpr20103c.a"
+    }
+}
+
 restore_notify
diff --git a/ld/testsuite/ld-plugin/pr20103a.c b/ld/testsuite/ld-plugin/pr20103a.c
new file mode 100644
index 0000000..6ebfd2c
--- /dev/null
+++ b/ld/testsuite/ld-plugin/pr20103a.c
@@ -0,0 +1,8 @@
+void live();
+
+int
+main ()
+{
+    live();
+    return 0;
+}
diff --git a/ld/testsuite/ld-plugin/pr20103b.c b/ld/testsuite/ld-plugin/pr20103b.c
new file mode 100644
index 0000000..ead122d
--- /dev/null
+++ b/ld/testsuite/ld-plugin/pr20103b.c
@@ -0,0 +1,3 @@
+void dead()
+{
+}
diff --git a/ld/testsuite/ld-plugin/pr20103c.c b/ld/testsuite/ld-plugin/pr20103c.c
new file mode 100644
index 0000000..fb94e02
--- /dev/null
+++ b/ld/testsuite/ld-plugin/pr20103c.c
@@ -0,0 +1,6 @@
+extern void dead ();
+
+void live()
+{
+  dead ();
+}
-- 
2.5.5