Re: Help needed to track down bug: linking Linux kernel with gold creates unbootable kernel

[John Reiser <jreiser at bitwagon dot com>]

> Here you go, looks like gold dropped 1/3 of the kernel:
> -rwxr-xr-x 1 edwin edwin 11M Apr 10 15:38 vmlinux.gold
> -rwxr-xr-x 1 edwin edwin 17M Apr 10 15:37 vmlinux.bfd

Not necessarily.  The 6M difference might be explained by a
combination of alignment in the file and/or symbol tables and/or
debugging information that need not be relevant to execution,
particularly for a kernel.

> [Nr] Name Type Addr Off Size ES Flags Lk Inf A
> -[ 1] .text PROGBITS ffffffff81000000 00001000 003d1c55 0 AX 0 0 4096
> +[ 1] .text PROGBITS ffffffff81000000 00200000 003d1c55 0 AX 0 0 4096

The difference in file offset between 0x1000 (4 KiB) and 0x200000 (2 MiB)
probably accounts for just under 2 MiB of zeroes in the file.  Such a
difference for file offset of Elf64_Shdr need not be relevant to execution
of a Linux kernel, as long as the 0x3d1c55 bytes of content are identical.

> alignment differences: [KEY:  -: gold;  +: ld]
> Program Headers:
>   Type   Offset           VirtAddr           PhysAddr  FileSiz   MemSiz Flg  Align
> - LOAD 0x001000 0xffffffff81000000 0x0000000001000000 0x5f2000 0x5f2000 R E 0x1000
> - LOAD 0x5f3000 0xffffffff81600000 0x0000000001600000 0x183220 0x183220 RWE 0x1000
> - LOAD 0x777000 0xffffffffff600000 0x0000000001784000 0x000888 0x000888 R E 0x1000
> - LOAD 0x778000 0x0000000000000000 0x0000000001785000 0x014628 0x014628 RW  0x1000
> - LOAD 0x78d000 0xffffffff8179a000 0x000000000179a000 0x071000 0x456d000 RWE 0x1000
> - NOTE 0x3d2c58 0xffffffff813d1c58 0x00000000013d1c58 0x00003c 0x00003c 0x4
> + LOAD 0x200000 0xffffffff81000000 0x0000000001000000 0x5ef000 0x5ef000 R E 0x200000
> + LOAD 0x800000 0xffffffff81600000 0x0000000001600000 0x183220 0x183220 RWE 0x200000
> + LOAD 0xa00000 0xffffffffff600000 0x0000000001784000 0x000888 0x000888 R E 0x200000
> + LOAD 0xc00000 0x0000000000000000 0x0000000001785000 0x014628 0x014628 RW  0x200000
> + LOAD 0xd9a000 0xffffffff8179a000 0x000000000179a000 0x071000 0x456d000 RWE 0x200000
> + NOTE 0x5d1c58 0xffffffff813d1c58 0x00000000013d1c58 0x000024 0x000024 0x4

The differing .p_align values of 0x1000 vs 0x200000 indicate that
gold has a bug interpreting the commands from the linker script
for alignment of Elf64_Phdr.  The .p_align applies to the .p_vaddr
and .p_paddr (which are related to address space at execution),
and not necessarily to .p_offset (which is related to file storage.)
For an ordinary ET_EXEC application or ET_DYN shared lib these are
connected by mmap(): the actual hardware page size must divide all
three of (.p_vaddr - .p_offset), (.p_paddr - .p_offset), .p_align.
For a Linux kernel they are connected by the boot loader: .p_vaddr
and .p_paddr must be divisible by the actual hardware page size,
but it could be OK for .p_offset to be anything as long as the
content (the interval of loaded bytes) was identical.  This looser
restriction requires a boot loader that processes each PT_LOAD
independently as "real bytes."  If the boot loader tries to do more
than one PT_LOAD at a time without carefully checking that the adjacency
in the input stream is compliant with the adjacency in the address space,
then that is a problem.  Also, if the boot loader is loading the
address space of some virtual machine by using mmap() on the host
machine, without a fallback for the case when mmap() fails, then
the more-stringent restrictions of "ordinary ET_EXEC" are relevant,
and thus gold's differing .p_align is an underlying bug.

The differing .p_filsz and .p_memsz of 0x3c vs 0x24 for the PT_NOTE
indicate that gold may have a bug there.  Examine the contents
(see Elf64_Nhdr in /usr/include/elf.h) to determine the added/omitted/
merged/changed content.

The differing .p_filesz and .p_memsz of 0x5f2000 vs 0x5ef000 for
the first PT_LOAD is a bug in gold.

--