Re: Regarding "Inconsistency detected by ld.so" 32 Bit Elf

["Carlos O'Donell" <carlos at redhat dot com>]

On 05/10/2013 03:07 PM, Karthikeyan Shanmugam wrote:
> Thanks Carlos.
> 
>> It is a problem. Less people will be able to help you if you can't
>> duplicate the problem on a common architecture. There is nothing
>> target dependent about this problem. You should equally be able to
>> produce an example program that shows the problem on x86-64.
> 
> I'll give a try once again.

Thanks.

>>>> I know that QEMU has to do something similar. There are probably so
>>>> few applications that need to do this that there is no good accepted
>>>> standard way to accomplish this.
> 
> Yes. I've checked QEMU code and linker script. They are modifying
> only the executable start address and Text segment in the custom
> linker script but no new/custom section which reserves VMA space .

Have you tried doing something similar?

>> Let me be clear here on two points.
>>
>> I don't know what you mean by "detect-frce relocate."
> 
> For example below is the elf segments output captured for libc library.
> 
>   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
>   PHDR           
>   INTERP         
>       [Requesting program interpreter: /lib/ld.so.1]
>   LOAD           0x000000 0x0fe40000 0x0fe40000 0x158608 0x158608 R E 0x10000
>   LOAD           0x15930c 0x0ffa930c 0x0ffa930c 0x0460c 0x071f0 RW  0x10000
>   DYNAMIC       
>   NOTE           
>   TLS            
>   GNU_EH_FRAME  
>   GNU_STACK      
>   GNU_RELRO      
> 
> Here, the first load segment base address is non-zero, so just
> curious to know whether prelink has some mechanism to detect and
> relocate only these libraries?

Why does prelink have to detect anything? It just reads the ELF file
and knows exactly the layout of the library.

>> I also don't know what you mean by "non-zero PT_LOAD segment libraries."
> 
> For. Example in above case libc holds the non-zero first load segment.

I will assume you mean libraries with PT_LOAD segments that have non-zero VMA.

>> The size is always known. Look at the size of the loadable segments,
>> and round up to page sizes and appropriate alignment.
>>
> 
> I've captured one sample output in which I noticed few discrepancies. Could you please clarify them? 
> 
> LIBC:
> =====
> 
> /proc/<pid>/maps Output:
> -------------------------
> 0fe40000-0ff99000 r-xp 00000000 00:0f 2392980    /lib/libc-2.11.1.so

Text.

> 0ff99000-0ffa9000 ---p 00159000 00:0f 2392980    /lib/libc-2.11.1.so

Gap.

> 0ffa9000-0ffad000 r--p 00159000 00:0f 2392980    /lib/libc-2.11.1.so

TLS + GNU_RELRO.

> 0ffad000-0ffae000 rw-p 0015d000 00:0f 2392980    /lib/libc-2.11.1.so
> 0ffae000-0ffb1000 rw-p 00000000 00:00 0

R/W Data.

> 
> Map Segments Difference:
> ------------------------
> 0ffb1000 - 0fe40000 = 1476 KB

OK, or 0x171000 bytes.

> readelf Output:
> ---------------
> 
> Program Headers:
>   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
>   PHDR           0x000034 0x0fe40034 0x0fe40034 0x00140 0x00140 R E 0x4
>   INTERP         0x14a7f0 0x0ff8a7f0 0x0ff8a7f0 0x0000d 0x0000d R   0x4
>       [Requesting program interpreter: /lib/ld.so.1]
>   LOAD           0x000000 0x0fe40000 0x0fe40000 0x158608 0x158608 R E 0x10000
>   LOAD           0x15930c 0x0ffa930c 0x0ffa930c 0x0460c 0x071f0 RW  0x10000
>   DYNAMIC        0x15ac60 0x0ffaac60 0x0ffaac60 0x000f8 0x000f8 RW  0x4
>   NOTE           0x000174 0x0fe40174 0x0fe40174 0x00020 0x00020 R   0x4
>   TLS            0x15930c 0x0ffa930c 0x0ffa930c 0x00008 0x00044 R   0x4
>   GNU_EH_FRAME   0x14a800 0x0ff8a800 0x0ff8a800 0x01cdc 0x01cdc R   0x4
>   GNU_STACK      0x000000 0x00000000 0x00000000 0x00000 0x00000 RW  0x4
>   GNU_RELRO      0x15930c 0x0ffa930c 0x0ffa930c 0x03cf4 0x03cf4 R   0x1
> 
> 
> Load Segments:
> 
>   Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
>   LOAD           0x000000 0x0fe40000 0x0fe40000 0x158608 0x158608 R E 0x10000
>   LOAD           0x15930c 0x0ffa930c 0x0ffa930c 0x0460c 0x071f0 RW  0x10000
>
> Memsize : 0x158608 - 160000 (Alignment)
>         : 0x071f0  - 010000  (Alignment)

That's not what alignment means, please read the ELF standard.

See:

p_vaddr % p_align == p_offset % p_align

The are *congruent* with respect to the alignment, not aligned by it.

>              -------
>              170000 = ~1472 KB = ~1476 (Assuming Page size of Linux 4 KB)
> 
> Size (1476) is matching with the memory allocation (map pointed above).

As a correct example:

0x0ffb1000 - 0x0fe40000 = 0x171000 bytes.

PT_LOAD #1 
  0x0fe40000 modulo 0x10000 is 0x0, thus p_vaddr % p_align == p_offset % p_align.
  Load from file at file offset 0x0 up to 0x158608 with zero padding because Memsiz == FileSiz.
  Mark 0x0fe40000 to round_up(0x0fe40000 + Memsiz, page size) as R|E e.g. 0x0fe40000 to 0x0ff99000 are R|E

GAP
  Mark from 0x0ff99000 to start of TLS 0xffa9000 as !(R|W|E).

PT_LOAD #2

  0x0ffa930c modulo 0x10000 is 0x930c, 0x15930c modulo 0x10000 is 0x930c, thus p_vaddr % p_align == p_offset % p_align.
  Load from file at file offset 0x15930c up to 0x15d918 with 0x2be4 zero padding in memory.
  Mark round_down (0x0ffa930c, page size) to round_up (0x0ffa930c + 0x071f0, page size) as RW e.g. 0x0ffa9000 to 0xffb1000.

TLS + GNU_RELRO
  Mark round_down (0x0ffa930c, page size) to round_up (0x0ffa930c + GNU_RELRO, page size) as R e.g. 0xffa9000 to 0ffad000 as R.

Thus total memory size is 0xffb1000 - 0x0fe40000 or 0x171000 bytes or 1476kb.

> ====
> 
> LIBNSL:
> ======
> 
> /proc/<pid>/maps Output:
> -------------------------
> 0f880000-0f895000 r-xp 00000000 00:0f 2392922    /lib/libnsl-2.11.1.so
> 0f895000-0f8a5000 ---p 00015000 00:0f 2392922    /lib/libnsl-2.11.1.so
> 0f8a5000-0f8a6000 r--p 00015000 00:0f 2392922    /lib/libnsl-2.11.1.so
> 0f8a6000-0f8a7000 rw-p 00016000 00:0f 2392922    /lib/libnsl-2.11.1.so
> 0f8a7000-0f8a9000 rw-p 00000000 00:00 0
> 
> Map Segments Difference:
> ------------------------
> 0f8a9000 - 0f880000 = 164 KB
> 
> readelf Output:
> ---------------
> 
> Program Headers:
>   Type           Offset   VirtAddr   PhysAddr   FileSiz  MemSiz   Flg Align
>   PHDR           0x000034 0x0f880034 0x0f880034 0x000100 0x000100 R E 0x4
>   INTERP         0x0143c4 0x0f8943c4 0x0f8943c4 0x00000d 0x00000d R   0x4
>         [Requesting program interpreter: /lib/ld.so.1]
>   LOAD           0x000000 0x0f880000 0x0f880000 0x014ed0 0x014ed0 R E 0x10000
>   LOAD           0x015abc 0x0f8a5abc 0x0f8a5abc 0x0006f4 0x002c00 RW  0x10000
>   DYNAMIC        0x015af4 0x0f8a5af4 0x0f8a5af4 0x0000e8 0x0000e8 RW  0x4
>   NOTE           0x000134 0x0f880134 0x0f880134 0x000020 0x000020 R   0x4
>   GNU_STACK      0x000000 0x00000000 0x00000000 0x000000 0x000000 RW  0x4
>   GNU_RELRO      0x015abc 0x0f8a5abc 0x0f8a5abc 0x000544 0x000544 R   0x1
> 
> Load Segments:
> ---------------
> 
>   LOAD           0x000000 0x0f880000 0x0f880000 0x014ed0 0x014ed0 R E 0x10000
>   LOAD           0x015abc 0x0f8a5abc 0x0f8a5abc 0x0006f4 0x002c00 RW  0x10000
> 
> Memsize : 0x014ed0 - 020000 (Alignment)
>         : 0x002c00 - 010000 (Alignment)

Wrong. See ELF standard comments above.

>              -------
>              030000 = ~192 KB
> 
> Here the load segments mem size is not matching with the above memory
> allocation. Please correct me if the above calculation is wrong.
> 
> I'm doing this to make sure maximum memory segment is allocated to
> shared libraries so that reposition can be avoided as I'm planning
> for relocation.

PT_LOAD #1 starts at 0x0f880000.

PT_LOAD #2 ends at round_up (0x0f8a5abc + 0x002c00, page size) = 0xf8a9000

Thus total memory size is 0xf8a9000 - 0x0f880000 or 0x29000 bytes or 164kb

>> The location is not known, the kernel decides that.
> 
> Can you please provide me some pointers or docs related to this ?
> More interested to understand how Kernel decides on ELF placement
> from code perspective.

There are no docs, you need to read the source for mmap :-)

Cheers,
Carlos.