Re: RFH: Annotating ELF binaries

[Carlos O'Donell <carlos at redhat dot com>]

On 01/18/2017 12:02 PM, Nick Clifton wrote:
> Hi Carlos,
> 
>> I've added 2 questions to the Toolchain/Watermark wiki but will post them
>> here for posterity:
> 
> Thanks - I'll try answering them here first, and if my answers make sense
> then I will update the wiki.
> 
>> (1) What happened to SHT_GNU_ATTRIBUTES and how does it relate to what 
>>     you are proposing?
> 
> Good question, and unfortunately I do not know the answer.  The problem
> is, I have been unable to locate any documentation that describes 
> SHT_GNU_ATTRIBUTES and how it is supposed to be used.
> 
> I think that the two schemes are quite similar, although this new proposal
> is intended to be able to cope with attributes that only apply to part of
> an executable and not necessarily the executable as a whole.  (Also, IMHO,
> my proposal has better documentation...)
 
I'm including Joseph Myers in this discussion.

It's very very well documented by ARM.

The point of SHT_GNU_ATTRIBUTES is to mark a section as containing
GNU-based Attributes for the object in question. Rather than have a generic
SHT_NOTE. The GNU-based attributes can be applied to the file, section, or
symbol (and extended further).

Attributes can be anything you want them to be. Today most of the GNU-attributes
are ABI related and the static linker uses them at link time to issue errors
when objects of mixed incompatible ABIs are attempted to be used together.
That is to say they are Tag_File-based attributes (see binutils source for this).

The generic section which holds attributes is SHT_GNU_ATTRIBUTES, but machines
can override this e.g. SHT_ARM_ATTRIBUTES for ARM-specific attributes.

The ELF attributes section design was originally based on the need at the time 
from ARM for ARM EABI Attributes.

See 4.3.6 Build Attributes:

http://infocenter.arm.com/help/topic/com.arm.doc.ihi0044f/IHI0044F_aaelf.pdf

.gnu.attributes (section)

	* <format version>, currently 'A' (0x41)
	* <section length> (4 byte unsigned int in ELF endian order of all subsequent data)
		* "vendor name" ("GNU") for generic + NULL pointer.
		* <file-tag> <szie> (all things which are file scope) <attribute>
		* <section-tag> (all things which are section scope) <size> <section #> <attribute>
		* <symbol-tag> (all things which are symbol scope) <size> <symbol #> <attribute>

The ARM documentation goes into a lot of detail about what attributes it
supports.

Given that we have a GNU-generic version of this... might we not extend things
to a new Tag_Range and keep all the existing machinery we already have in binutils
for processing all of these attributes?

Again, the SHT_GNU_ATTRIBUTES section was never designed to be loaded by the
dynamic linker, and it's the reason I recommend you look at the design and think
about how we can avoid another attributes format that is incompatible with the
one we already have in binutils.

Harmonize the designs please :-)

>> (2) What is being done to ensure the attributes are space and time
>>     efficient for dynamic link comparison in the dynamic linker? 
>>     Speed of checking 10,000 DSOs (scalability) for ABI compatibility is 
>>     going to be a very important requirement.
> 
> I believe that H.J's design for the dynamic link notes does take efficiency
> into consideration, but I will leave that for him to comment on further.

It does, mostly be using bits, and AND/OR/EQ handling.

I have yet to see exactly how it works.

I might suggest we version the section like .gnu.attributes so we can
increment the version if we have a flag day changing the layout.

> One thing that I have already done for the static notes is to implement a
> new option for objcopy called "--merge-notes" which eliminates redundancies.
> Theoretically this option could be extended to work with the dynamic notes
> too, helping to make them as space efficient as possible.

OK.

> Another possibility is that the linker could be extended so that when it
> creates a dynamic executable it also inserts a "master" dynamic linker note,
> which contains all of the information that the dynamic linker will need,
> without it having to search through all of the shared libraries used by the
> application.  (This does assume that the shared libraries examined at static
> link time are the same ones that are loaded/used at dynamic link time).
 
We can't assume that unfortunately.
 
>> (b) Loadable notes and space/time efficiency vs. non-loadable notes and
>>     static analysis tools.
>>
>> Run-time checking of properties is radically different from offline
>> checking of properties and we absolutely need two different designs to
>> meet these needs. However, if we could weld the two together in a compatible
>> way, that would be great. For example if the dynamic loader could map from
>> a 'run-time property' to a 'link-time property' to increase the verbosity
>> of the error in a failure scenario, then that might be beneficial.
> 
> I think that this might not be easy to do in a way that both imposes a low
> code-increase cost on the dynamic linker and a keep-the-dynamic-link-notes-small
> space requirement.  There is no harm in investigating though.

What I would like to see is parity between the notes the dynamic loader
is using and the notes the static linker is using to generate warnings or outright
link failures.

>> If we
>> could translate 'link-time notes' into 'a collection of run-time properties' in
>> a semi-automatic fashion given strict rules about the notes application,
>> then that would also be awesome.
> 
> Now this might well be feasible.  I am thinking of another new option to objcopy
> here that examines the static notes and generates dynamic notes from them.  This
> should be quite straightforward, provided that the static notes have captures
> the right information.

Agreed.

-- 
Cheers,
Carlos.