Re: Unexec gurus?

[Telford Tendys <telford at eng dot uts dot edu dot au>]

On Sat, Feb 26, 2000 at 05:20:50PM -0500, Jim Blandy wrote:
> 
> Just for kicks, I thought I'd offer an explanation of *why*
> initialized variables in dynamically linked libraries get overwritten
> when you dump and re-exec.

Well unexec is notoriously system dependent and shared library
management is too so any explanantion is almost always going to
be either wrong or incomplete.

Anyhow, I only use linux so I'm just commenting from that particular
point of view, solaris, BDS, etc may do other things. Also note that
linux has changed over from a.out format to ELF format and pretty
much no one uses a.out any more. However, there are still a.out
compatibility things around which may taint various mechanisms.

> Suppose you compile foo.c to foo.o file, and foo.c references some
> global int variable, var.  The compiler has no idea whether the definition of
> var is going to come from some other .o file, or some shared
> library.  So it emits completely straightforward code, which assumes
> that the address of var is some constant, to be resolved at static
> link time.  The .o file will contain appropriate relocs, telling the
> static linker how to plug in var's address once it knows it.

I think this is pretty standard.

> Now, suppose it turns out that var lives in a shared library, which
> provides an initialized value for it.  We won't know the shared
> library's address until run-time, when the dynamic linker will load it
> wherever it pleases.  But we can't go through the executable and patch
> up all the references to var at run-time, because that would make the
> executable unshareable, to some extent, which works against the whole
> purpose of shared libraries --- to save memory.  And we can't make the
> executable reference var through some indirection table, since we must
> compile foo.o before we realize var is coming from a shared library.
> So we're kind of stuck.

This sounds kind of strange. If you have a linux system handy, take
a look at /proc/nnn/maps and see that every shared library has two images,
one being the read only part and one being the read/write part.
The mechanism that you describe would not logically cooperate with the
model of having a double map for every library. Personally, I see the
double map as being a bit more sensible. Linux supports copy-on-write
pages (to get fork to work properly you pretty much HAVE to support
this, and once you have it, might as well use it lots) so if read/write
map MIGHT be shared if the library variables don't change. In practice
most libraries have much bigger code maps than data maps (most libraries
have almost no data to speak of but the minimum map size is one page).

> For the sake of references to var in the shared lib itself, the
> dynamic linker stores the newly initialized copy's address in the
> global offset table or someplace like that; I forget.  The shared
> library itself was compiled with the -PIC flag, so any references to
> var in the shared library will go through this table.

This sounds like a total mess. I'm not saying it isn't done like
that because I haven't checked.

> So, in general, if you unexec an executable linked dynamically against
> some shared library, any variable initialized in that shared library
> will get re-initialized each time you run the dumped executable.

I guess that depends on how unexec works, seems to me that if it
dumps its own code and stack then the library variables are not even
stored in the dump! If it scans its own /proc/nnn/map file, finds
every block that is writable and dumps that block then at least it
has a full copy of its state (I doubt that many unexecs do this because
they don't want to bloat themselves any worse than they already are and
even if they do, it is system specific, on linux for example, the /proc
directory may not even be mounted).

Trying to restore those library variables is dangerous. Firstly,
they may contain pointers and the libraries may move so you can't just
splurt back into the same place that you found it. Secondly, the library
may be a different version! Variables may be in completely different
locations or have different meanings. Trying to outsmart the library
authors is bordering on the ridiculous.

> It's exactly this sort of crap that made me so hesitant to support
> unexec.  The idea of unexec is seductively simple, and it'll seem to
> work, but it runs afoul of the modern run-time environment in so many
> different ways, you're just asking for trouble.  And it's neverending:
> today, it's this; tomorrow, you'll find something else.  It would be
> much better if we could find some way to get instantaneous startup
> that doesn't require us to violate the run-time's abstractions so
> utterly.  That's my opinion.

Agreed, unexec is a funky trick that works for some special cases.
As a general rule it doesn't work at all. However, a full unexec is
not actually what the users want... all they want is a fast startup
that gets them back the working environment that they had before.
Emacs uses an unexec to achieve this but there are other ways to
do it. My personal belief if that any data structures that are not
temporary and that use pointers are flawed. A lot of people disagree
with me on this but most processors handle *p and p[ i ] and p[ i + 3 ]
at the same speed (or close to it) so the days of trying to reduce
everything down to *p are gone (for the moment at least).

There may be ways to run through the tree of cons cells and construct
some encoded format that can reconstruct a similar tree at a later date.
If you HAVE to base your data on pointers then that is probably the
best way to handle a fast startup. Obviously, the startup state must
be restricted in some special ways -- no files can stay open for
example -- and these restrictions have to be spelled out to the users
so that they don't attempt the impossible.

	- Tel