Re: RFE: dl_iterate_phdr, r_debug.r_brk usability for introspection

["Carlos O'Donell" <carlos at redhat dot com>]

On 09/23/2015 02:53 PM, OndÅej BÃlka wrote:
> On Wed, Sep 16, 2015 at 12:21:51AM -0400, Rich Felker wrote:
>> On Tue, Sep 15, 2015 at 08:54:40PM -0700, jreiser@bitwagon.com wrote:
>>> On Tue, September 15, 2015 Rich Felker wrote:
>>>> On Sun, Sep 13, 2015 at 09:45:22AM -0700, John Reiser wrote:
>>>>
>>>>> Counting and reporting the number of calls to dlopen and dlclose
>>>>> would enhance the usability of dl_iterate_phdr.  A related change to
>>>>> r_debug.r_brk would make life easier for live introspection.
>>>>
>>>> Can you offer some intended usage cases? I'm not seeing how this
>>>> "enhances the usability".
>>>
>>> Today it is cumbersome for an app to determine whether two invocations
>>> of dl_iterate_phdr() generate the same list of loaded modules,
>>> or to verify quickly the common cases when one is a subset of the other.
>>> The app must test for equality of two sets.  This requires
>>> storage whose size is unknown in advance, and some non-trivial code.
>>> In contrast, if the number of calls to dlopen and dlclose are tracked
>>> and reported, then in most common uses and environments the required storage
>>> is two counters, and the code is trivial.  That's enhanced usability.
>>> [Of course the worst case still requires general set difference;
>>> but a very large fraction of practical cases do not.]
>>>
>>> Suppose you are integrating code, shared libraries, and dynamic modules
>>> from several sources.  The combination has a bug, and you want to locate it.
>>> You identify some critical nodes and arcs in the high-level call graph,
>>> and insert some introspecting code which calls dl_iterate_phdr.  The lists
>>> of loaded modules at various times gives you significant information
>>> about the state of the memory space, but making use of the information
>>> is difficult because it is hard to compute the deltas between adjacent
>>> samples.  But if the counts of dlopen and dlclose are tracked and reported,
>>> then in most practical cases it becomes much easier.
>>
>> Thanks for explaining.
>>
>>>> The type unsigned is way too small for this and subject to overflow.
>>>> Counters like this need to be at least 64-bit.
>>>
>>> Claiming that more than 64 bits might be necessary is buffoonery; stop it.
>>> In practice, overflowing a plain unsigned counter of the number of
>>> dlopen or dlclose in a non-toy application takes a very long time.
>>
>> I didn't claim that more than 64 bits are needed. I claimed that more
>> than 32 bits are needed. It does take a while to overflow 32 bits, but
>> not as long as you think (probably possible in ~1 month if you hammer
>> it, and realistically on a scale of around a year), and low-quality
>> interfaces that can overflow after a few weeks or months of uptime
>> should not be added to glibc.
>>
> That is quite absurd that you need to wait that long. When you want to
> find such issues you will debug/run application with added
> dl_iterate_phdr and it wont take you so long to find problem. 

I disagree, but this disagreement is solely based on my experience debugging
larger applications.

I agree with Rich that more than 32-bits is required here, and the next
increment is logically 64-bits.

Cheers,
Carlos.