When I run the testsuite on x86_64-pc-linux-gnu with a soft RLIMIT_AS of 7680000, malloc/tst-malloc-thread-exit fails like so: error: pthread_create: Resource temporarily unavailable Didn't expect signal from child: got `Aborted' It aborts with a more or less useless stack trace from one of the exploded threads some time after 'Resource temporarily unavailable' messages have started appearing (indicating that the ulimit has been hit), but not immediately: gdb shows a screenful of thread creations, intermittent 'Resource temporarily unavailable' and thread exits before halting with this not-too-useful backtrace: (gdb) bt #0 0x00007ffff7a6d1b7 in raise () from /usr/src/glibc/x86_64-mutilate/libc.so.6 #1 0x00007ffff7a6e60a in __GI_abort () at abort.c:89 #2 0x000000000040180b in ?? () #3 0x0000000000000000 in ?? () If the RLIMIT_AS is lifted to unlimited, the failure goes away. The thread stack for the aborted thread exists at this point, according to /proc/$pid/maps, and the registers show that %esp is pointing inside it, so it's not something obvious like ulimits preventing thread stack allocation. The thread it halts at always appears to have been one that was created shortly before the ulimit was hit: presumably the creation succeeded without ulimit errors, because more threads are shown as created before the errors start appearing, so I'm guessing this is malloc() hitting trouble, not pthread_create(). (However, this might just be GDB attaching to threads out of creation order.)
To reproduce with a glibc after commit 2a38688932243b5b16fb12d84c7ac1138ce50363, you need to drop the RLIMIT_AS further for the failure to happen, but it still happens: $ ulimit -v 500000 $ env GCONV_PATH=/usr/src/glibc/x86_64-mutilate/iconvdata LOCPATH=/usr/src/glibc/x86_64-mutilate/localedata LC_ALL=C /usr/src/glibc/x86_64-mutilate/elf/ld-linux-x86-64.so.2 --library-path /usr/src/glibc/x86_64-mutilate:/usr/src/glibc/x86_64-mutilate/math:/usr/src/glibc/x86_64-mutilate/elf:/usr/src/glibc/x86_64-mutilate/dlfcn:/usr/src/glibc/x86_64-mutilate/nss:/usr/src/glibc/x86_64-mutilate/nis:/usr/src/glibc/x86_64-mutilate/rt:/usr/src/glibc/x86_64-mutilate/resolv:/usr/src/glibc/x86_64-mutilate/crypt:/usr/src/glibc/x86_64-mutilate/mathvec:/usr/src/glibc/x86_64-mutilate/nptl /usr/src/glibc/x86_64-mutilate/malloc/tst-malloc-thread-exit error: pthread_create: Resource temporarily unavailable Didn't expect signal from child: got `Aborted'
Running with my recently-posted --enable-stack-protector=all does *not* trigger a stack-protection abort when this testcase abort()s, which suggests that whatever is going wrong when the stack is smashed does not test the canary, so probably no function returns after the corruption (though if the code is modifying control flow before the assert depending on values in the corrupted stack, bad things could still happen, of course -- e.g. smashing a function pointer and then calling it: but libio is the most likely place to do that, I guess, and this testcase outputs nothing so is fairly likely to be safe from libio-related problems).
(In reply to Nick Alcock from comment #2) > Running with my recently-posted --enable-stack-protector=all does *not* > trigger a stack-protection abort when this testcase abort()s, which suggests > that whatever is going wrong when the stack is smashed This is an explicit abort call because a thread creation fails. The error message clearly indicates that the process has run out of resources. It is not caused by any kind of memory corruption. I don't think this is a bug in the test. The test needs a few threads to exercise the malloc implementation, and each thread has a certain address space requirement. If you constrain the environment too much, the test cannot run in a meaningful way.
Oh good, that was what I hoped was the case. Phew. :) thanks for checking! (I have rarely been so happy to have a bug marked invalid.)