Re: parallelized 'ld'?

[Michael Meissner <bin-mail at the-meissners dot org>]

On Fri, Jul 18, 2003 at 12:44:28AM +1000, Mathew Boorman wrote:
> On Tue, 2003-07-15 at 09:56, Kris Warkentin wrote:
> > I don't know much about the subject one way or another but an interesting
> > whitepaper on a related subject, Incremental Linking is here:
> > 
> > http://www.usenix.org/events/osdi2000/wiess2000/full_papers/mikulin/mikulin.
> > pdf
> > 
> > I have a feeling that the gains from this would be similar to parallelizing
> > (when only a few objects are changed) and perhaps, in the case of GNU ld,
> > easier to implement.
> 
> Hi, I have been lurking with a plan to improve ld's performance, I was
> expecting incremental linking would be the way to do it.  I had not
> found the above paper though, it certainly makes the design effort
> simpler.  I was not ready to de-lurk yet, but the issue turned up!
> 
> In our environment the biggest performance limitation of ld has simply
> been the file io, which is very noticeable when over nfs on slow LANs.
> For Solaris we turned on incremental linking which really helped, for
> Linux we messed with our source control tools to get the object files
> stored locally.
> 
> I only realised how much faster g++/ld/gdb could be when I used MS
> Visual Studio again and could get debug/edit/build/debug cycles in sub 1
> second. Of course msvc uses a whole bunch of tricks to achieve that, but
> incremental linking is probably the next step given the g++ changes in
> progress.

That reminds of another possible speedup that has been talked about over the
years -- namely not copying the debug information from the object files to the
program file, but having the debugger being able to find the object files and
read it directly.  If you compile for debugging, the size of the debug
information usually swamps the size of the binary code.  I believe Solaris does
this.  This obviously would have to be a coordinated effort between the binary
utilities, GDB, and possibly GCC.

Another thing that would help is coalescing common declarations from .h file
debug information for libraries or ld -r objects into one place.  Recently on
the GCC list there have been patches not to put out debug information for
things that aren't referenced in this module under switch control.

> The other big factor was edit in place(name?), where code changes are
> patched into the executable running in the debugger.  To implement that
> you need to add the new version of the changed functions to the image
> and link them in for future calls. The msvc one is much smarter than
> that,  doing the duplication only if the changed functions are already
> down the call stack, and manipulating the stack appropriately if the
> change is to the currently active function.

The problem with things like this is whether static variables get
reinitialized, and whether things grow in size, and tracking down pointers to
the old objects.

-- 
Michael Meissner
email: gnu@the-meissners.org
http://www.the-meissners.org