Re: RFC: Test hook for nss_files testing

["Carlos O'Donell" <carlos at redhat dot com>]

On 12/06/2015 03:22 PM, Florian Weimer wrote:
> The attached patch adds a test hook which allows us to select different
> files (not those under /etc) for testing nss_files inside the build
> tree.  It contains a regression test for bug 17363 as an example.
> 
> If the direction is acceptable, I'll add further tests.

[snip]

> There are also a few additional relocations (for the string constants),
> but they should not matter.

I have two concerns.

(1) Security.

What security implications are there in exposing this interface?

(2) Test what we ship.

We need to get away from build-tree testing and move to installed tree
testing to verify that we are testing is what we are shipping.

The testing would look like this:

- Setup an installed tree.
- Setup the test.
- Run the test in some kind of isolation with configuration changes
  made to the sysroot that would otherwise be impossible on the host.
- Return status.
- Repeat for all tests that need a sysroot e.g. ldconfig, network, nss...

That is not to say that whitebox hook-based testing is bad, but it
deviates from (1) and (2) in ways which make me uncomfortable.

A more appealing alternative would be to run the test under a systemtap
script which did all the work of updating the paths to the databases
without the hook changes.

Thoughts?

Cheers,
Carlos.