posix_spawn's usage of fork versus vfork

[Mike Axiak <mike at axiak dot net>]

Hello,

I'm not sure if this is the list to ask about this, but I figure
someone might be able to point me in the right direction :-).
I've recently hit an issue in a Java environment in linux wherein
executing small processes tries to allocate too much virtual memory
[1].
The issue really comes down to competing architectural designs, but I
have to make the project I'm working on work, and have to come up with
a solution.

The issue I'm seeing is this: as you know, there are two calls to fork
a process: vfork() and fork(). Fork will duplicate the virtual memory
space and perform a copy-on-write in order to maintain program space
in the child process. This is a problem for applications with large
monolithic address spaces, as the duplication of virtual memory space
can easily fail in normal conditions (imagine a single process with a
4GB heap trying to run a 10KB shell program). From what I can tell,
vfork does not do any of this, and will merely move execution to
another loaded program. Now, if I want to run an external program
performing a fork/pipe redirect/exec, I have to have some way of
passing the fd redirect information to the child process.

It appears that posix_spawn is uniquely defined to assist in this
situation, because I can just define whatever FD redirects I have to
take, then just let it decide how to fork. The problem is, whenever I
ask for /any/ fd redirects, it appears to want to fork(). That is,
I've only had it run vfork() when I handle the fd redirects myself. As
a result of this, there seem to be two ways to call a program without
copying address space and still opening pipes:

  1) Using an small external program to perform the requisite fd
redirects and run an exec (This seems to work)
  2) Perhaps using environment variables to store the necessary
information (Note: I haven't tested this idea, and don't know if it
works), then perform a vfork manually and do the fd redirects.

Surely other people have hit this particular issue before? I hope that
I am missing an obvious feature of one of these functions, and that
someone can just point me in the right direction. Should I just be
content with (1) as my solution? Should I even try (2)?

Thanks,
Mike


1: http://bugs.sun.com/view_bug.do?bug_id=5049299