Re: runtime loader replaces argv[0]

["Carlos O'Donell" <carlos at redhat dot com>]

On 05/15/2015 06:04 PM, Patrick Donnelly wrote:
> Hi,
> 
> My colleagues and I are using the runtime loader in preservation
> software for scientific applications. To be specific, we preserve the
> runtime loader implicitly used by the software so that it can be run
> again in the future, possibly on a completely different platform (but
> using the same or newer kernel). In this context, we are using Linux
> but the problems exists for all kernels.
> 
> The problem we see is that the glibc loader unconditionally replaces
> argv[0]. So if I want to run an application on a different platform
> that uses a different loader, I would do this instead:
> 
> execve("./preserved/ld-linux.so", ["argv[0]" /* from original
> experiment */, "./preserved/application.exe" /* physical exe */,
> "argv[1]", ...], [...]);

You are executing the loader directly, but doing so as if you
were the kernel, by using a distinct argv[0], which will trigger
the loader to treat argv[0] as if it was *real* and existing on
the filesystem.
 
> I would expect the loader to simply strip off its argv[1] and leave
> the rest alone. Unfortunately, it sets argv[0] to argv[1], as if we
> ran:
> 
> execve("./preserved/application.exe", ["./preserved/application.exe",
> "argv[1]", ...], [...]);

AFAIK there is no such mode to achieve this.

> This prevents us from passing the original argv[0] to the application.

Yes.

Nobody has ever asked for such a feature before.

> Ironically enough, the software that is most affected by this is the
> loader itself. If argv[0] is not a real path (i.e. beginning with ./
> or /), then it will readlink("/proc/self/exe") to determine the
> physical path. In the original experiment with an argv[0] of "foo",
> that's exactly what it does. In the rerun experiment with the
> preserved loader, argv[0] is "./preserved/application.exe". So the
> loader uses this path incorrectly in subsequent library lookups,
> ultimately causing the application to fail.

This is because you've tricked the loader into thinking it's the kernel
doing the execution of a normal binary, which *must* exist on the
filesystem in order to be executed.

As I said before, this is a unique feature that hasn't been requested
before and therefore you can't preserve argv[0] if you change the name
of the binary to be executed without also preserving the filesystem
layout.

> So as for my question, is there a viable workaround for this or does
> anyone have some other suggestion?

Use patchelf to modify the binary PT_INTERP to use the preserved loader.

or

Maintain the application filesystem layout using a chroot

There are many other aspects of the preserved application you are not
maintaining also, and eventually your preserved application will fail
if it relies on features tied to the OS or filesystem. See the recent
discussion on posix_fallocate for an example of this, where the system
call isn't available on NFS (uses somewhat unsafe fallback in glibc),
but is for other local filesystems.

Cheers,
Carlos.