Re: PATCH: 2 stage BFD linker for LTO plugin

[Cary Coutant <ccoutant at google dot com>]

> Here is an alternative proposal, with a patch for gold.
>
> We add a new plugin vector: LDPT_REGISTER_RESCAN_ARCHIVE_HOOK. ?Like
> LDPT_REGISTER_CLAIM_FILE_HOOK, this gives the plugin the address of a
> function which can register a plugin function: rescan_archive.
>
> typedef
> enum ld_plugin_status
> (*ld_plugin_rescan_archive_handler) (
> ?const struct ld_plugin_input_file *file, int *rescan);
>
> If the plugin registers this hook, the linker will call the hook for
> each archive that it sees. ?If the hook sets the *rescan variable to a
> non-zero value, then the linker will rescan the archive after calling
> the all_symbols_read hook. ?The archive will be rescanned using the same
> position dependent options as when it was originally scanned. ?In
> particular, if the archive occurred within --start-group/--end-group,
> the entire group will be rescanned.
>
> The point of this patch is that the known problems with the LTO plugin
> are when the plugin introduces a previously unknown symbol which must be
> satisfied from some archive. ?The new symbol is introduced when the
> all_symbols_hook read hook calls add_input_file to add a new object file
> to the link, and the new object file refers to the new symbol. ?Since
> the symbol was not previously seen, if the definition should come from
> an archive, the archive will not have been searched. ?Hence the test
> case in GCC PR 42690 comment #32.
>
> Fortunately, we know that gcc is not going to introduce arbitrary new
> symbol references. ?The current system hacks around the problem by using
> a special -pass-through option which directs the plugin to add specific
> archives to the link, namely -lc and -lgcc. ?That works for most cases,
> but not when gcc introduces a symbol which comes from -lm. ?Still,
> despite the -lm issue, we do not have to concern ourselves with
> arbitrary archives, because gcc is not going to act arbitrarily.
>
> Also we know that the problem can only occur with archives, not with
> shared objects.
>
> The rescan_archive proposal fixes the problems and obviates the need to
> use -pass-through. ?It avoids the unnecessary cost of a complete relink.
>
> I've appended a patch for gold which implements this proposal. ?I've
> also appended a patch for lto-plugin. ?This patched lto-plugin does not
> use -pass-through when rescan_archive is available. ?It rescans libc.a,
> libgcc.a, and libm.a. ?It handles the PR 42690 comment #32 test case
> correctly.
>
> Any thoughts on this approach?

Looks good to me. I'd still prefer something based on a list of symbol
names that the backend might introduce calls to, but I'll concede that
this is far more practical.

I think Dave mentioned in the other thread that libgcov, libssp, and
libmudflap might also need to be rescanned:

>>> My suspicion is that the LTO plugin can only introduce a small bounded
>>> set of new symbol references, namely those which we assume can be
>>> satisified from -lc or -lgcc.  Is that true?
>>
>> Exactly.
>
> Potentially also gcov, ssp, mudflap?

Should we mark the pass-through option in lto-plugin as obsolescent?

-cary