Re: [PATCH 1/2][RFC] #17645, fix slow DSO sorting behavior in dynamic loader

[Chung-Lin Tang <chunglin_tang at mentor dot com>]

On 2019/8/5 6:45 PM, Florian Weimer wrote:
> > > > What I did in the ld.so code patch was add a second pass of sorting
> > > > that ignores runtime deps, prioritizing link dependencies; this
> > > > appears to also be what prior discussion pointed towards, see more
> > > > details in that 2nd email with the actual code patch.
> > > I wonder if it makes sense to disentangle this (desirable) functional
> > > change from the rest (which sould be purely an optimization).
> > By "functional change" here, are you referring to the testing framework,
> > or the described ld.so destructor behavior I described above?
> The destructor behavior.

Well, I'm definitely not suggesting adding the two-pass sorting described
above with the current sorting algorithm (even if it should be relatively
straightforward to do so)

The entire #17645 issue is due to the current algorithm becoming prohibitively
slow in certain pathological cases. Trying to fix the destructor behavior that
way without replacing the current sorting algorithm will greatly exacerbate
the performance problem.

> > > Is it even necessary to re-sort on dlclose?  Is the original dependency
> > > order available somewhere?  Then we could make it explicit that the
> > > destructor order is the reverse of the constructor order (for the
> > > objects unloaded).  Or is there a corner case which causes an expected
> > > divergence?
> > Dynamic loaded objects could add more relocation dependencies, and
> > thus augment the dependency relations (by adding more constraints), so
> > a final sort should still be required.
> Yes, these dynamically added relocation dependencies could mean that
> fewer objects than had been loaded by the dlopen can be freed with
> dlclose.  But if we disregard those relocation dependencies for
> destructor order sorting, wouldn't be the sorted result equivalent to
> the constructor order?

Relocation dependencies are not completely disregarded during destruction,
just that they're prioritized lower than static link dependencies (when
dependence cycles cause ambiguity in determining a single ordering), hence
the two passes of sorting. Besides that, dlopen'ed but not dlclose'd objects
also need to be processed along, so any existing already-computed ordering
is probably not enough in the general case.

Chung-Lin