Re: Chroot testsuite

["Joseph S. Myers" <joseph at codesourcery dot com>]

On Mon, 13 Jan 2014, Ryan Arnold wrote:

> On Sat, Jan 11, 2014 at 6:40 PM, Joseph S. Myers
> <joseph@codesourcery.com> wrote:
> > On Sat, 11 Jan 2014, Ryan Arnold wrote:
> >
> >> > glibc testing should not assume any particular packages on the host; it
> >> > should work even if what you get from ssh is just a busybox shell (with no
> >> > privileges to run chroot).
> >>
> >> So for the general case of builddir tests being turned into
> >> host/install tests (i.e., not integration tests), this means that
> >> glibc would copy the local install_root contents onto the remote
> >> machine where the tests will then be executed there by invoking the
> >> test via the dynamic linker <install_root>/elf/ld[64].so <testcase>?
> >
> > Yes.  (Well, copying should be optional - it's not needed when you have a
> > shared filesystem, but Brooks has requested support for testing without
> > such a shared filesystem, which certainly makes sense as well.)
> 
> For remote host tests, a post-install-hook specified as an environment
> variable would give all the power that's needed.  If the hook isn't
> present, then make install will work as it does now, with or without
> install_root being present.  If it is present then the
> post_install_hook can be called after each local install step.  A
> canned hook could live in scripts/.
> 
> I suppose a post-make-hook could do something similar for the builddir
> if someone wants to pursue that.

I don't understand what you are suggesting.

"make install" is supposed to convert from the source-tree and build-tree 
locations of files that end up in a glibc installation, to the locations 
of those files in an installation, and to copy the files accordingly.  I 
don't think it makes sense to hook it in any way.  Copying an installation 
tree from one system to another should be a completely separate step.

I'd be quite happy for "make install" to be the *only* thing that needs to 
know the aforementioned mapping.  That is, for build-tree testing to start 
by running "make install install_root=<build-dir>/install-tests", and 
copying libgcc_s.so.* / libstdc++.so.* from the relevant directory in the 
GCC installation (or from a location specified by the user running "make 
check"), before using those locations in all ld.so invocations, 
environment variables, etc. - this doesn't require GLIBC_SYSROOT insofar 
as existing tests don't need GLIBC_SYSROOT, though of course GLIBC_SYSROOT 
would simplify things.

> Running remote host tests would require a host-test-wrapper= that
> copies each testcase to a staging directory, and executes it there.
> This wrapper would be similar to the existing test-wrapper= which is
> used when remotely executing builddir tests except that
> cross-test-ssh.sh doesn't need to copy the testcase to the remote
> machine (yet?) since it currently expects a shared filesystem.

A way to specify a wrapper with a more complicated interface to list files 
to copy in each direction seems a reasonable thing to add.

> If host-test-wrapper isn't present and host tests are run, then the
> system just assumes they're to be installed into an
> install_root/host_tests staging directory and invoked there indirectly
> through the install_root/<path_to_lib>/ld.so.  This requires that make
> host_check take an install_root option.
> 
> If install_root isn't present in this case, what's the default
> behavior?  I suppose the host tests are installed into some staging
> directory and then executed, as dynamically linked, against the system
> libraries.  Is that useful?

It does not make sense to link (static-linker) tests with new libc and run 
them with old libc.  It may make sense in principle to run a new libc's 
testsuite with old libc, but then tests should be compiled with old libc's 
headers and linked with old libc.

I don't understand your host_check concept, but while an externally 
specified install_root may be useful I'd assume the default would be 
installing into a standard location within the build tree.

> >> > I would not be adverse to the normal glibc testsuite moving to installing
> >> > files in a staging directory within the build tree using install_root,
> >> > then running tests the same way they would be run with a previously built
> >> > and installed glibc, provided this does not require any extra software or
> >> > privileges for chroot.
> >>
> >> I think we've sorted out that this is possible to do this for existing
> >> tests that aren't meant to be integration tests that presume they're
> >> executing in the specified prefix.
> >
> > Note however there might still be concerns about how different ways of
> > testing exercise different code paths - see the discussion from when
> > --enable-hardcoded-path-in-tests was added, that resulted in it being a
> > non-default configure option rather than the only way to run tests.  (An
> > actual chroot would of course test paths through the dynamic linker that
> > are closer to how people normally use glibc than either of the existing
> > approaches, or GLIBC_SYSROOT, is.)
> 
> So are these integration tests that are maintained outside of glibc,
> or glibc tests that presumes an ABI compatible chroot is setup
> (somewhere--native or remote)?

I don't understand your question.  I'm talking about how normal, existing 
glibc tests are run, or might be run in future.

> Do you recall what the concerns with providing GLIBC_SYSROOT were?  I
> imagine the ease of bug reproducibility goes down for OS distributors
> because they'll have to have additional verification that
> GLIBC_SYSROOT is correctly set.

See Roland's comments in the previous discussions, since he was the one 
raising concerns.

> So if we do add a GLIBC_SYSROOT option.. how does this change host and
> builddir testing?

Hypothetically, it would add a third way of running tests, (a) as at 
present, (b) --enable-hardcoded-path-in-tests, (c) installing in a staging 
directory within the build tree and using GLIBC_SYSROOT.  There's also 
potential for (d) installing in a staging directory within the build tree 
but using existing ld.so invocations and environment variables, just with 
simpler lists of paths (possibly as a replacement for (a)).

-- 
Joseph S. Myers
joseph@codesourcery.com