[PATCH] Add compile testing to glibc test framework.

[Carlos O'Donell <carlos at redhat dot com>]

The goal of this patch is to add compile testing to the glibc
test framework.

The changes allow you to add C source files that the test framework
can compile and verify the compiler exits with a zero exit status for
PASS. While a non-zero exit status by the compiler indicates a FAIL.
The support for UNSUPPORTED is handled in the normal way for other
tests that are conditionally disabled (can't be run and thus can't
exit with code 77). Only C source files are currently supported
because that's all I needed to support today, but it could be extended
to compile any type of source target. I wanted to approach the work
incrementally.

The test is being compiled under the glibc test framework, which
is as close as we currently get to building real user applications.

For example I use the test to compile various glibc/linux header
order combinations e.g.

cat sysdeps/unix/sysv/linux/tst-ipv6-compat1.c
#include <netinet/in.h>
#include <linux/in6.h>

int
main (void)
{
  return 0;
}

git diff sysdeps/unix/sysv/linux/Makefile
diff --git a/sysdeps/unix/sysv/linux/Makefile b/sysdeps/unix/sysv/linux/Makefile
index c57575f..dad556c 100644
--- a/sysdeps/unix/sysv/linux/Makefile
+++ b/sysdeps/unix/sysv/linux/Makefile
@@ -203,3 +203,9 @@ tests += tst-setgetname tst-align-clone tst-getpid1 \
        tst-thread-affinity-pthread tst-thread-affinity-pthread2 \
        tst-thread-affinity-sched
 endif
+
+ifeq ($(subdir),inet)
+# Add tests for Linux and glibc header compatibility.
+tests-compile += tst-ipv6-compat1 tst-ipv6-compat2 tst-ipv6-compat3
+TEST_SRC_DIR = ../sysdeps/unix/sysv/linux/
+endif

Previously these kinds of tests would result in the testsuite failing
to run to completion because compiler errors are treated as harness
failures.

I tried a variety of solutions to implement this, but settled on the
present set of two variables: tests-compile and TESTS_SRC_DIR as a way
to avoid needing anything like sysd-rules to find the test sources.
I did not want to unduly slow down build test cycles. The static
pattern rule is pretty concise and optimal.

Thoughts?

-- 
Cheers,
Carlos.

2015-06-06  Carlos O'Donell  <carlos@redhat.com>

	* Rules [ifeq ($(run-built-tests),no)] (tests): Add $(tests-compile).
	[ifneq ($(run-built-tests),no)] (tests): Add $(tests-compile:%=$(objpfx)%.out).
	(tests): Add $(tests-compile) to merge result list.
	[ifeq ($(build-programs),yes)] (binaries-all-tests): Add $(tests-compile).
	[ifneq "$(strip $(tests-compile))" ""] (tc-OUTPUT_OPTION-file): Define.
	(tc-OUTPUT_OPTION): Define.
	(tc-compile-mkdeps-flags): Define.
	(tc-compile.c): Define.
	($(tests-compile:%=$(objpfx)%.out)): Define static pattern rule.

diff --git a/Rules b/Rules
index 8306d36..e92a9ff 100644
--- a/Rules
+++ b/Rules
@@ -91,10 +91,10 @@ else
 others: $(addprefix $(objpfx),$(extra-objs))
 endif
 ifeq ($(run-built-tests),no)
-tests: $(addprefix $(objpfx),$(tests) $(test-srcs)) $(tests-special)
+tests: $(addprefix $(objpfx),$(tests) $(test-srcs)) $(tests-compile) $(tests-special)
 xtests: tests $(xtests-special)
 else
-tests: $(tests:%=$(objpfx)%.out) $(tests-special)
+tests: $(tests:%=$(objpfx)%.out) $(tests-compile:%=$(objpfx)%.out) $(tests-special)
 xtests: tests $(xtests:%=$(objpfx)%.out) $(xtests-special)
 endif
 
@@ -102,7 +102,7 @@ tests-special-notdir = $(patsubst $(objpfx)%, %, $(tests-special))
 xtests-special-notdir = $(patsubst $(objpfx)%, %, $(xtests-special))
 tests:
 	$(..)scripts/merge-test-results.sh -s $(objpfx) $(subdir) \
-	  $(sort $(tests) $(tests-special-notdir:.out=)) \
+	  $(sort $(tests) $(tests-special-notdir:.out=) $(tests-compile)) \
 	  > $(objpfx)subdir-tests.sum
 xtests:
 	$(..)scripts/merge-test-results.sh -s $(objpfx) $(subdir) \
@@ -111,7 +111,7 @@ xtests:
 
 ifeq ($(build-programs),yes)
 binaries-all-notests = $(others) $(sysdep-others)
-binaries-all-tests = $(tests) $(xtests) $(test-srcs)
+binaries-all-tests = $(tests) $(xtests) $(test-srcs) $(tests-compile)
 binaries-all = $(binaries-all-notests) $(binaries-all-tests)
 binaries-static-notests = $(others-static)
 binaries-static-tests = $(tests-static) $(xtests-static)
@@ -198,6 +198,45 @@ $(objpfx)%.out: /dev/null $(objpfx)%	# Make it 2nd arg for canned sequence.
 	$(make-test-out) > $@; \
 	$(evaluate-test)
 
+# tests-compile provides a way for developers to test compiling a singular
+# C source file under the environment of the glibc build.  This is useful
+# when testing system invariants like compatibility between headers files
+# provided by the linux kernel headers (--with-headers).  While it is
+# arguable that such tests, as invariants, should go in configure scripts,
+# that would lead to a large number of unruly tests in configure.ac.
+# Having the compatibility tests as part of the regression testsuite is far
+# more manageable from a maintenance perspective.
+#
+# There are three things you need to do to add a compilation test to glibc:
+# * Create the source file to compile.
+#   - It should compile without error in order for the test to PASS.
+#   - It should compile with error in order for the test to FAIL.
+#
+# * Add the test name to 'tests-compile'
+#   - e.g.
+#     ifeq ($(subdir),inet)
+#     tests-compile += tst-ipv6-compat
+#     ...
+#
+# * Set 'TEST_SRC_DIR' to the relative path to the test sources (no trailing
+#   slash):
+#     ...
+#     TEST_SRC_DIR = ../sysdeps/unix/sysv/linux
+#     endif
+#
+# You will now have an additional test that is compiled during 'make check',
+# with compiler output to $(objpfx)/<test name>.out as you would expect for
+# a normal test.
+ifneq "$(strip $(tests-compile))" ""
+tc-OUTPUT_OPTION-file = $(objpfx)$(notdir $(@:%.out=%.o))
+tc-OUTPUT_OPTION = -o $(tc-OUTPUT_OPTION-file)
+tc-compile-mkdep-flags = -MD -MP -MF $(tc-OUTPUT_OPTION-file).dt -MT $(tc-OUTPUT_OPTION-file)
+tc-compile.c = $(CC) $(TEST_SRC_DIR)/$(notdir $(@:%.out=%.c)) -c $(CFLAGS) $(CPPFLAGS)
+$(tests-compile:%=$(objpfx)%.out): $(objpfx)%.out : $(TEST_SRC_DIR)/%.c Makefile
+	$(tc-compile.c) $(tc-OUTPUT_OPTION) $(tc-compile-mkdep-flags) \
+	>& $@; $(evaluate-test)
+endif
+
 # tests-unsupported lists tests that we will not try to build at all in
 # this configuration.  Note this runs every time because it does not
 # actually create its target.  The dependency on Makefile is meant to