Bug 12291

Summary: "ld -r" doesn't work with mixed IR/non-IR objects
Product: binutils Reporter: H.J. Lu <hjl.tools>
Component: ldAssignee: H.J. Lu <hjl.tools>
Status: NEW ---    
Severity: normal CC: ian, law, nickc, sam
Priority: P2    
Version: 2.22   
Target Milestone: ---   
See Also: https://sourceware.org/bugzilla/show_bug.cgi?id=27122
Host: Target:
Build: Last reconfirmed:

Description H.J. Lu 2010-12-06 22:28:36 UTC
[hjl@gnu-6 pr-1]$ cat foo.c
#include <stdio.h>

void foo(void)
{
  printf ("hello foo\n");
}
[hjl@gnu-6 pr-1]$ cat main.c
extern void foo(void);

int main(void)
{
  foo();
  return 0;
}
[hjl@gnu-6 pr-1]$ make
/usr/gcc-4.6/bin/gcc -B./ -c -O -flto -fuse-linker-plugin main.c -o main.o
/usr/gcc-4.6/bin/gcc -B./ -c -O foo.c -o foo.o
./ld -r -o prog.o main.o foo.o
/usr/gcc-4.6/bin/gcc -B./ -flto -fuse-linker-plugin prog.o -o prog
/tmp/ccAWLRVh.ltrans0.ltrans.o: In function `main':
ccAWLRVh.ltrans0.o:(.text.startup+0x5): undefined reference to `foo'
collect2: ld returned 1 exit status
make: *** [prog] Error 1
[hjl@gnu-6 pr-1]$
Comment 1 H.J. Lu 2010-12-06 23:01:17 UTC
One approach

--
We generate a prog.o that contains an object that is the relocation of real foo.o and main.o (used if prog.o is linked without IR support), that also contains IR for main.o, and within the IR the object code for main.o.  So the interesting case is when the relocatable object is again compiled in such a way that the IR is read out from prog.o.  The Intel compiler will take the IR from prog.o, and compile it with any other IR present in archives or other objects (none in this particular case), and will create a real object from the IR, this is then added to a list of objects that the driver will pass to the linker along with the object code for main.o which the compiler extracts to a new temporary object that is added to the list of objects.
--
Comment 2 H.J. Lu 2010-12-15 19:17:03 UTC
The proposal is at

http://www.kernel.org/pub/linux/devel/gcc/lto/mixed-IR/mixed-IR.pdf

It is implemented on hjl/lto-mixed branch at

http://git.kernel.org/?p=devel/binutils/hjl/x86.git;a=summary
Comment 3 Dmitry Gorbachev 2011-01-21 21:55:59 UTC
ld.ian (gold with patch <http://sourceware.org/ml/binutils/2011-01/msg00270.html>) silently uses non-IR part of prog.o
Comment 4 H.J. Lu 2011-01-21 22:27:45 UTC
(In reply to comment #3)
> ld.ian (gold with patch
> <http://sourceware.org/ml/binutils/2011-01/msg00270.html>) silently uses non-IR
> part of prog.o

What does gold generate for

/usr/gcc-4.6/bin/gcc -B./ -c -O -flto -fuse-linker-plugin main.c -o main.o
/usr/gcc-4.6/bin/gcc -B./ -c -O foo.c -o foo.o
./ld -r -o prog.o main.o foo.o

Does prog.o contain both IR and non-IR sections?
Comment 5 Dmitry Gorbachev 2011-01-22 00:43:24 UTC
> Does prog.o contain both IR and non-IR sections?

No. I think a new bug report should be opened for gold.

When I use ld.bfd (or ld.hjl) for partial link step and ld.ian for final link, everything works.
Comment 6 H.J. Lu 2020-10-13 14:07:15 UTC
(In reply to H.J. Lu from comment #2)
> The proposal is at
> 
> http://www.kernel.org/pub/linux/devel/gcc/lto/mixed-IR/mixed-IR.pdf

Here is the proposal:

Link with mixed IR/non-IR objects

* 2 kinds of object files
  o non-IR object file has
    * non-IR sections
  o IR object file has
    * IR sections
    * non-IR sections
    * The output of "ld -r" with mixed IR/non-IR objects should work with:
        o Compilers/linkers with IR support.
	o Compilers/linkers without IR support.
* Add the mixed object file which has
  o IR sections
  o non-IR sections:
    * Object codes from IR sections.
    * Object codes from non-IR object files.
  o Object-only section:
    * With section name ".gnu_object_only" and SHT_GNU_OBJECT_ONLY type
    on ELF:
    #define SHT_GNU_OBJECT_ONLY 0x6ffffff8	/* Object only */
    * Contain non-IR object file.
    * Input is discarded after link.
* Linker action:
  o Classify each input object file:
    * If there is a ".gnu_object_only" section, it is a mixed object file.
    * If there is a IR section, it is an IR object file.
    * Otherwise, it is a non-IR object file.
  o Relocatable non-IR link:
    * Prepare for an object-only output.
    * Prepare for a regular output.
    * For each mixed object file:
      * Add IR and non-IR sections to the regular output.
      * For object-only section:
	* Extract object only file.
	* Add it to the object-only output.
	* Discard object-only section.
    * For each IR object file:
      * Add IR and non-IR sections to the regular output.
    * For each non-IR object file:
      * Add non-IR sections to the regular output.
      * Add non-IR sections to the object-only output.
    * Final output:
      * If there are IR objects, non-IR objects and the object-only
      output isn't empty:
	* Put the object-only output into the object-only section.
	* Add the object-only section to the regular output.
	* Remove the object-only output.
  o Normal link and relocatable IR link:
    * Prepare for output.
    * IR link:
      * For each mixed object file:
	* Compile and add IR sections to the output.
	* Discard non-IR sections.
	* Object-only section:
	  * Extract object only file.
	  * Add it to the output.
	  * Discard object-only section.
      * For each IR object file:
        * Compile and add IR sections to the output.
	* Discard non-IR sections.
      * For each non-IR object file:
	* Add non-IR sections to the output.
    * Non-IR link:
      * For each mixed object file:
	* Add non-IR sections to the output.
	* Discard IR sections and object-only section.
      * For each IR object file:
	* Add non-IR sections to the output.
	* Discard IR sections.
      * For each non-IR object file:
	* Add non-IR sections to the output.

> 
> It is implemented on hjl/lto-mixed branch at
> 
> http://git.kernel.org/?p=devel/binutils/hjl/x86.git;a=summary

It has been moved at:

https://gitlab.com/x86-binutils/binutils-gdb/-/tree/users/hjl/lto-mixed/master
Comment 7 Nick Clifton 2020-10-20 16:23:46 UTC
Hi H.J.

(In reply to H.J. Lu from comment #6)

> Here is the proposal:

> It has been moved at:
> 
> https://gitlab.com/x86-binutils/binutils-gdb/-/tree/users/hjl/lto-mixed/
> master

I like the proposal.

Do you think that the branch sources are ready for review and 
merging/pulling into mainline ?

Cheers
  Nick
Comment 8 H.J. Lu 2020-10-20 17:15:00 UTC
(In reply to Nick Clifton from comment #7)
> Hi H.J.
> 
> (In reply to H.J. Lu from comment #6)
> 
> > Here is the proposal:
> 
> > It has been moved at:
> > 
> > https://gitlab.com/x86-binutils/binutils-gdb/-/tree/users/hjl/lto-mixed/
> > master
> 
> I like the proposal.
> 
> Do you think that the branch sources are ready for review and 
> merging/pulling into mainline ?
> 

Yes, I will submit a patch set.
Comment 9 Sam James 2022-07-22 22:28:46 UTC
(In reply to H.J. Lu from comment #8)
> (In reply to Nick Clifton from comment #7)
> > Hi H.J.
> > 
> > (In reply to H.J. Lu from comment #6)
> > 
> > > Here is the proposal:
> > 
> > > It has been moved at:
> > > 
> > > https://gitlab.com/x86-binutils/binutils-gdb/-/tree/users/hjl/lto-mixed/
> > > master
> > 
> > I like the proposal.
> > 
> > Do you think that the branch sources are ready for review and 
> > merging/pulling into mainline ?
> > 
> 
> Yes, I will submit a patch set.

Did anything happen w/ this? Thanks!