Testing against Gnulib's test suite

Contents

  1. Testing against Gnulib's test suite
    1. How to run the test suite
    2. Known problems on Linux
    3. Apparently fixed on Linux as of glibc 2.38
    4. Known problems on Hurd
    5. Investigating new problems

GNU gnulib has a growing test suite of POSIX and glibc headers and functions. As of 2012, it covers ca. 40 header files and ca. 280 functions. These tests are known to pass on glibc platforms with 6 different CPU types, as well as on major Unix systems (from Mac OS X to Cygwin), modulo platform specific bugs on these non-glibc systems.

The tests are generally written according to the POSIX standard. Many tested behaviours are also "common sense". Only few tests could be characterized as "hairy". It is suitable to test glibc against this test suite, as a sanity check.

How to run the test suite

1. On a development machine (a machine with GNU tools, including gperf), as unprivileged user, checkout GNU gnulib https://savannah.gnu.org/projects/gnulib and run 

$ GNULIB_CHECKOUT=`pwd`
$ ${GNULIB_CHECKOUT}/gnulib-tool --create-testdir --dir=/tmp/testdir \
                                 --with-tests --single-configure \
                                 $(${GNULIB_CHECKOUT}/posix-modules)

2. Copy the /tmp/testdir directory to the target machine, if it is a different machine.

3. In /tmp/testdir: For testing a glibc installed in /usr, run 

$ ./configure CPPFLAGS="-Wall"

For testing a glibc built from source with --prefix=$GLIBC_INSTALL_DIR, run 

$ ./configure CC="gcc -Wl,-rpath=$GLIBC_INSTALL_DIR/lib64:$GLIBC_INSTALL_DIR/usr/lib64 -Wl,--dynamic-linker=$GLIBC_INSTALL_DIR/lib64/ld-linux-x86-64.so.2" CPPFLAGS="-nostdinc -I`gcc -print-file-name=include-fixed` -I$GLIBC_INSTALL_DIR/usr/include -I`gcc -print-file-name=include` -Wall"

4. Run 

$ make

Verify that Gnulib has built no replacement/workaround code (gllib/*.o files) - if so, this indicates problems in the libc. 

$ (cd gllib; ls -1 $(${GNULIB_CHECKOUT}/posix-modules | sed -e 's|-posix$||' | sort -u | grep -v 'nonblocking' | sed -e 's|$|.o|') 2>/dev/null )

5. Run 

make check

Known problems on Linux

btowc.o

Observed with glibc-2.38. 

configure:46783: checking whether btowc is consistent with mbrtowc in the C locale ... no

From documentation: In the C or POSIX locales, this function is not consistent with Gnulib’s mbrtowc and can return WEOF: glibc 2.35, MirOS BSD #10. Thus, expected.

dprintf.o

checking whether printf supports infinite 'long double' arguments... no

glibc's fprintf produces random output when 'long double' values outside the IEEE range are given. Gnulib developers don't find this is OK and prefer "nan" output.

Related:

This applies to dprintf.o, fprintf.o, printf.o, snprintf.o, sprintf.o, vdprintf.o, vfprintf.o, vprintf.o, vsnprintf.o, vsprintf.o

fclose.o

checking whether fflush works on input streams... no

This is due to

This applies to fclose.o, fflush.o, fseek.o, fseeko.o

fcntl.o

This is due to 

checking whether fcntl understands F_DUPFD_CLOEXEC... needs runtime check

The Linux kernel only added F_DUPFD_CLOEXEC in 2.6.24, so Gnulib always replace it to support the semantics on older kernels that failed with EINVAL. Glibc should do the same.

fflush.o

See fclose.o

fopen.o

Observed with glibc 2.38. Needs analysis.

fprintf.o

See dprintf.o

fseek.o

See fclose.o

fseeko.o

See fclose.o

ioctl.o

checking for ioctl with POSIX signature... no

Glibc has a wrong parameter type: 'unsigned long' rather than 'int'.

math.o

Needs analysis

mbrlen.o

This is a known bug:

This affects mbrtowc.o, mbrlen.o, c32rtomb.o, mbrtoc32.o

mbrtoc16.o

Needs analysis

mbrtoc32.o

See mbrlen.o

mbrtowc.o

See mbrlen.o

mbsnrtowcs.o

Seen with glibc 2.38. Needs analysis.

mbsrtowcs.o

Seen with glibc 2.38. Needs analysis.

mbstowcs.o

Seen with glibc 2.38. Needs analysis.

memset_explicit.o

Seen with glibc 2.38.

From documentation: This function is missing in glibc.

mktime.o

Seen with glibc 2.38. May be related to (?) 

configure:81341: checking for __mktime_internal
configure:81341: gcc -o conftest -g -O2 -Wall  conftest.c  >&5
/usr/lib/gcc/x86_64-pc-linux-gnu/13/../../../../x86_64-pc-linux-gnu/bin/ld: /tmp/ccrjo63V.o: in function `main':
/home/dilfridge/gnulib_testdir/conftest.c:694: undefined reference to `__mktime_internal'
collect2: error: ld returned 1 exit status

Needs analysis.

nanosleep.o

This is due to 

checking for working nanosleep... no (mishandles large arguments)

This function mishandles large arguments when interrupted by a signal on 64-bit Linux platforms. Gnulib has a workaround, Glibc does not.

printf.o

See dprintf.o

snprintf.o

See dprintf.o

sprintf.o

See dprintf.o

strerror_r.o

This is due to 

checking for strerror_r with POSIX signature... no

When _GNU_SOURCE is defined, glibc defines strerror_r with a non-POSIX declaration.

sys_socket.o

Needs analysis

time.o

Seen with glibc 2.38.

From documentation: not consistent with gettimeofday and timespec_get on some platforms: glibc 2.31 or newer on Linux

unistd.o

Needs analysis

vdprintf.o

See dprintf.o

vfprintf.o

See dprintf.o

vprintf.o

See dprintf.o

vsnprintf.o

See dprintf.o

vsprintf.o

See dprintf.o

wcsstr.o

Seen with glibc 2.38

From documentation: This function has quadratic instead of linear worst-case complexity on some platforms: glibc 2.37

wctype-h.o

Needs analysis.

Apparently fixed on Linux as of glibc 2.38

futimens.o

(fixed) This is due to 

checking whether futimens works... needs runtime check

glob.o

(fixed) This is due to 

checking whether glob lists broken symlinks... no

isfinite.o

(fixed) This is due to 

checking whether isfinite(long double) works... no

linkat.o

(fixed) This is due to 

checking whether linkat(,AT_SYMLINK_FOLLOW) works... need runtime check

regex.o

(fixed) This is due to a bug in a gnulib test case that misses escaping regular expressions in the *.m4 tests.

utimensat.o

(fixed) This is due to 

checking whether utimensat works... needs runtime check

Known problems on Hurd

TODO

Investigating new problems

When an expected .o file is built, or when a check fails, look at

If your conclusion is that it's a bug in Gnulib, please report it at bug-gnulib at gnu dot org.

None: Testing/Gnulib (last edited 2023-07-24 20:35:06 by AndreasHüttel)