Created attachment 12820 [details] GITHUB download GNU-ld-for-MicrosoftCOFF-to-LinuxELF Hi all, when using uninitialized global variables (that are placed in the .BSS section) the linker assigns wrong addresses in the .ELF binary. From the same .OBJ the Microsoft LINK.EXE assigns correct addresses in the .EXE binary. I have created a comprehensive documentation on my GITHUB account for that particular .BSS bug: https://github.com/KilianKegel/GNU-ld-for-MicrosoftCOFF-to-LinuxELF#bss-bug All .OBJ, .EXE, .ELF, .DIS (disassembler), .MAP files are kept there. This is the one single bug already reported in https://sourceware.org/bugzilla/show_bug.cgi?id=26547 Regards, Kilian
[hjl@gnu-cfl-2 ldBugDemo0]$ cat x.S .text .align 16 .globl getaddr1 .def getaddr1; .scl 2; .type 32; .endef getaddr1: leaq var(%rip), %rax ret .comm var, 1 # 1 [hjl@gnu-cfl-2 ldBugDemo0]$ ./as -o x.o x.S [hjl@gnu-cfl-2 ldBugDemo0]$ ./objdump -dwr x.o x.o: file format pe-x86-64 Disassembly of section .text: 0000000000000000 <getaddr1>: 0: 48 8d 05 00 00 00 00 lea 0x0(%rip),%rax # 7 <getaddr1+0x7>3: R_X86_64_PC32 var-0x1 7: c3 ret 8: 90 nop 9: 90 nop a: 90 nop b: 90 nop c: 90 nop d: 90 nop e: 90 nop f: 90 nop [hjl@gnu-cfl-2 ldBugDemo0]$ ./ld x.o ./ld: warning: cannot find entry symbol _start; defaulting to 0000000000401000 [hjl@gnu-cfl-2 ldBugDemo0]$ objdump -dw a.out a.out: file format elf64-x86-64 Disassembly of section .text: 0000000000401000 <getaddr1>: 401000: 48 8d 05 f8 0f 00 00 lea 0xff8(%rip),%rax # 401fff <getaddr1+0xfff> ^^^^ Wrong address. 401007: c3 ret 401008: 90 nop 401009: 90 nop 40100a: 90 nop 40100b: 90 nop 40100c: 90 nop 40100d: 90 nop 40100e: 90 nop 40100f: 90 nop [hjl@gnu-cfl-2 ldBugDemo0]$ ./nm a.out 0000000000402000 B __bss_start 0000000000402000 B _edata 0000000000402008 B _end 0000000000401000 T getaddr1 U _start 0000000000402000 B var [hjl@gnu-cfl-2 ldBugDemo0]$
Please provide Windows object file of: [hjl@gnu-cfl-2 ldBugDemo0]$ cat bar.c extern void bar (void); void foo (void) { bar (); } [hjl@gnu-cfl-2 ldBugDemo0]$
Hi H.J.Lu, thanks a lot. in ldBugDemo0 the bug appears, because "char var" is kept unintilialized. in ldBugDemo1 the bug does not appear , because "char var" is intilialized. in ldBugDemo2 the bug does not appear , because "char var" is intilialized. If you check the screen shot from the diff tool: https://github.com/KilianKegel/GNU-ld-for-MicrosoftCOFF-to-LinuxELF/blob/master/bssbug.png you can see, that on the left side var has the faulty address 403013 in getaddr1() line 8 but the correct address 403014 in getaddr2() line 14. THIS IS THE .BSS bug! The work around is to initialize var to 0. In that case the linker doesn't create a "common symbol". Instead the compiler creates an initialized var placed in .BSS that is not "common". In ldBugDemo2 var is uninitialized too, but is placed in a separate file. In that case the .BSS bug doesn't occure despite var is a "common symbol" in .BSS. https://github.com/KilianKegel/GNU-ld-for-MicrosoftCOFF-to-LinuxELF/blob/master/bssbugMAP.png Regards, Kilian
Created attachment 12823 [details] bar.c with /Od (opt dis) /O1 (opt ena) and disassemblies and objdumps .DMP: objdumps with the Microsoft DUMPBIN.EXE /all .DIS: obj disassemblies with Microsoft DUMPBIN.EXE /disasm .OBJ: object modules O1-> optimizer enabled Od-> optimizer disasbled
(In reply to Kilian Kegel from comment #4) > Created attachment 12823 [details] > bar.c with /Od (opt dis) /O1 (opt ena) and disassemblies and objdumps > > .DMP: objdumps with the Microsoft DUMPBIN.EXE /all > .DIS: obj disassemblies with Microsoft DUMPBIN.EXE /disasm > .OBJ: object modules > > O1-> optimizer enabled > Od-> optimizer disasbled I need obj file.
Created attachment 12825 [details] added .OBJ sorry, my mistake... :-(
Please try users/hjl/pr26583/master branch https://gitlab.com/x86-binutils/binutils-gdb/-/commits/users/hjl/pr26583/master
Hi H.J.Lu, thank you for your quick response. Regrettably I had a lot of trouble building your branch https://gitlab.com/x86-binutils/binutils-gdb/-/commits/users/hjl/pr26583/master because of missing, bison, m4 in my UBUNTU 20.04 build environment. So could you please do the testing? In the ldBugDemo0 directory just type ld --output=program.elf -Map=program.elf.map --entry="begin" *.obj and objdump -d program.elf (from M.BAT batch file included in the directory) and check if ... 0000000000401000 <getaddr1>: 401000: 48 8d 05 0c 20 00 00 lea 0x200c(%rip),%rax #403013 <begin+0x1ff3> 401007: c3 retq 401008: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1) 40100f: 00 0000000000401010 <getaddr2>: 401010: 48 8d 05 fd 1f 00 00 lea 0x1ffd(%rip),%rax #403014 <__bss_start> 401017: c3 retq 401018: 0f 1f 84 00 00 00 00 nopl 0x0(%rax,%rax,1) 40101f: 00 ... the LEA instruction in getaddr1() and getaddr2() loads the SAME address. In the sample above they return 403013 vs 403014. This is different and THIS IS THE .BSS RELOCATION BUG! Instead I have downloaded the binutils source from GNU (2.35, 2.34 and 2.21) and I was able to build and test, but all of them report: "getaddr1.obj: file not recognized: file format not recognized" Afterwards I checked recent versions of Debian, RedHat, Suse and Ubuntu with different versions of binutils included in each particular distribution and ALL OF THEM were able to link .OBJ to .ELF, but regrettably with the .BSS relocation error we are talking about. Regards, Kilian
Created attachment 12837 [details] wrong relocation Wrong address relocation Original binutils 2.34 from UBUNTU 20.04 The address returned by getaddr1() and getaddr2() is different
Created attachment 12838 [details] correct relocation Correct address relocation Original binutils 2.34 from CYGWIN The address returned by getaddr1() and getaddr2() is equal
(In reply to Kilian Kegel from comment #8) > Hi H.J.Lu, > > thank you for your quick response. > > Regrettably I had a lot of trouble building your branch > > https://gitlab.com/x86-binutils/binutils-gdb/-/commits/users/hjl/pr26583/ > master > > because of missing, bison, m4 in my UBUNTU 20.04 build environment. Please install the missing programs on your machine. > So could you please do the testing? In the ldBugDemo0 directory just type > > ld --output=program.elf -Map=program.elf.map --entry="begin" *.obj > > and > > objdump -d program.elf It should be fixed with my patch. Please verify it.
Created attachment 12841 [details] GNU ld (GNU Binutils) 2.35.50.20200909 result Hi H.J.Lu, now I was able to build, install and run your fix. (Please have a look onto the appended screen shot.) Regrettably it does not accept the COFF .OBJ files. This is the same behavior as with my own binutil-rebuilds of ver. 2.34, 2.35 and 2.21 from https://ftp.gnu.org/gnu/binutils Let me summarize: 1) LD (2.34) of CYGWIN doesn't have that bug at all. Both of my bugs (.BSS and OPTIMIZATION) are just gone. The link result is correct! But regrettably it creates the wrong executable format (.EXE instead of .ELF) 2) All binutils of my own rebuild (your fix, 2.21, 2.34, and 2.35) doesn't accept the COFF .OBJ fileformat and stop with the same error message: "getaddr1.obj: file not recognized: file format not recognized" 3) all original ld/binutil versions that comes with Debian, Suse, RedHat and Ubuntu accept the COFF .OBJ but creates erroneous .ELF That sounds to me that a) Due to (1) MS COFF .OBJ file format is already implemented somewhere in the binutils source code base but just not activated/ enabled/pulled in (maybe a different variant of COFF .OBJ is activated instead) b) Due to (2) and (3) it needs some additional magic configuration at build time to accept MS COFF .OBJ, that the binutil maintainers at Debian, RedHad, Suse and Ubuntu certainly know... Thanks, Kilian
Please configure binutils with --enable-targets=x86_64-linux,x86_64-pep
Hi H.J.Lu, no it works! That is great! ;-) Your implementation resolves all of the bugs (2) that I noticed so far 26583, 26584 and 26547 (.BSS and OPTIMIZATION bug). So I will continue my project soon: Port of Torito C Library to Linux - that means to create ANSI C (C89/C90) applications with Visual Studio 2019 tool chain for 1. UEFI Shell 2. Linux x86-64 3. Windows10 64 (for validation of (1) and (2) only) https://github.com/KilianKegel/torito-C-Library To create the Linux .ELF executables the LINK.EXE is replaced by a GNU ld, running in a WSL2 VM (Windows subsystem for Linux) on the Windows Build Machine. To get a deeper understanding please tell me 1. Is Microsoft COFF support really (hopefully) a requirement of GNU/ld? 2. Are there probably other COFF variants that may be affected by that fix 3. Will this fix also appear in the next public release of BINUTILS? Thanks a lot, Kilian
(In reply to Kilian Kegel from comment #14) > Hi H.J.Lu, > > To get a deeper understanding please tell me > 1. Is Microsoft COFF support really (hopefully) a requirement of GNU/ld? Linking in Microsoft COFF input for ELF output is very limited. Not all Microsoft COFF inputs are compatible with ELF output. > 2. Are there probably other COFF variants that may be affected by that fix This fix only affects ELF output with Microsoft COFF inputs. > 3. Will this fix also appear in the next public release of BINUTILS? > I am targeting 2.36 release.
The master branch has been updated by H.J. Lu <hjl@sourceware.org>: https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;h=36068e2fa5460e6cfa1f5f359df6f37c497aab50 commit 36068e2fa5460e6cfa1f5f359df6f37c497aab50 Author: H.J. Lu <hjl.tools@gmail.com> Date: Wed Sep 16 07:11:16 2020 -0700 elf/x86-64: Adjust relocation for PE/x86-64 inputs PE linker calls _bfd_relocate_contents to resolve relocation, instead of bfd_perform_relocation. But ELF linker calls bfd_perform_relocation, not _bfd_relocate_contents. When linking PE/x86-64 inputs to generate ELF output, we need to adjust PE/x86-64 relocations in bfd_perform_relocation. Enable PE/x86-64 in bfd together with PEI/x86-64. Update run_ld_link_tests to handle bzip2 binary inputs. bfd/ PR ld/26583 * config.bfd (targ64_selvecs, targ_selvecs): Add x86_64_pe_vec to x86_64_pei_vec. * reloc.c: Include "coff/internal.h". (bfd_perform_relocation): Adjust relocation for PE/x86-64 inputs. ld/ PR ld/26583 * testsuite/ld-x86-64/pe-x86-64-1.od: New file. * testsuite/ld-x86-64/pe-x86-64-1a.obj.bz2: Likewise. * testsuite/ld-x86-64/pe-x86-64-1b.obj.bz2: Likewise. * testsuite/ld-x86-64/pe-x86-64-1c.obj.bz2: Likewise. * testsuite/ld-x86-64/pe-x86-64-2.od: Likewise. * testsuite/ld-x86-64/pe-x86-64-2a.obj.bz2: Likewise. * testsuite/ld-x86-64/pe-x86-64-2b.obj.bz2: Likewise. * testsuite/ld-x86-64/pe-x86-64-2c.obj.bz2: Likewise. * testsuite/ld-x86-64/pe-x86-64-3.od: Likewise. * testsuite/ld-x86-64/pe-x86-64-3a.obj.bz2: Likewise. * testsuite/ld-x86-64/pe-x86-64-3b.obj.bz2: Likewise. * testsuite/ld-x86-64/pe-x86-64-3c.obj.bz2: Likewise. * testsuite/ld-x86-64/pe-x86-64-3d.obj.bz2: Likewise. * testsuite/ld-x86-64/pe-x86-64-4.od: Likewise. * testsuite/ld-x86-64/pe-x86-64-4a.obj.bz2: Likewise. * testsuite/ld-x86-64/pe-x86-64-4b.obj.bz2: Likewise. * testsuite/ld-x86-64/pe-x86-64-4c.obj.bz2: Likewise. * testsuite/ld-x86-64/pe-x86-64-4d.obj.bz2: Likewise. * testsuite/ld-x86-64/pe-x86-64.exp: Likewise. * testsuite/lib/ld-lib.exp (run_ld_link_tests): Handle bz2 binary inputs.
Fixed for 2.36.
*** Bug 26547 has been marked as a duplicate of this bug. ***
*** Bug 26584 has been marked as a duplicate of this bug. ***