Re: segfault with pthread_cancel() and PTHREAD_STACK_MIN on armv7

[Adhemerval Zanella <adhemerval dot zanella at linaro dot org>]

On 10/02/2017 17:23, Michael Weiser wrote:
> Hi,
> 
> I run Gentoo Linux on Cubieboard2s (armv7v, le and be) with the
> distribution's glibc-2.23. The whole system recompiled itself without
> problems using gcc-6.3.0. This gcc is hardened, i.e. configured amongst
> others with --enable-default-ssp.
> 
> Now I've run into a peculiar problem with pthreads, manifesting itself
> in ntpd segfaulting upon startup. Turns out for very specific reasons
> they spawn a thread doing an endless loop of sleep(10)'s and cancel
> that. Also, they set the thread's stack size to PTHREAD_STACK_MIN.
> (rationale and code can be seen here:
> https://github.com/ntp-project/ntp/blob/stable/ntpd/ntpd.c#L252)
> 
> After isolating the code into the attached testcase I found that
> increasing the thread's stack size by 6640 bytes or eliminating
> pthread_cancel() altogehter both make the segfault go away.
> 
> The same testcase runs fine on an otherwise identical x86_64 install of
> Gentoo compiled with the same kind of hardened gcc-6.3.0.
> 
> So I guess my question is: Is pthread_cancel() supposed to work with a
> stack size of PTHREAD_STACK_MIN on every platform?
> 
> What might be causing my platform to require 6640 bytes more stack to
> succeed?
> 

GLIBC implements thread cancellation through a signal (SIGCANCEL/__SIGRTMIN), 
so when a thread is cancelled a glibc internal signal handler
(sigcancel_handler) is executed.  Since glibc does not use an alternate
stack for signal handling (sigaltstack), the thread own stack will be
used to actually run the handle and subsequent stack trace.

It will really depend of the stack usage when the signal arrives and also
on compiler options and version, but in either way if you intend to handle
signal (and cancellation will fall in this category) you will need to take
in consideration the signal stack size on pthread stack size.

I could not reproduce the issue with your testscase in my environment
(kernel 3.8.11, gcc 4.8.4, ld 2.24), but for arm you can see that
PTHREAD_STACK_MIN is 16k so glibc will allocate in fact a 12k stack size
due the required guard page:

b6ef3000-b6ef4000 ---p 00000000 00:00 0 
b6ef4000-b6efb000 rw-p 00000000 00:00 0          [stack:10745]

(this is an output of the same test using glibc master on armv7l).

Also the tests when executing the SIGCANCEL handle shows:

stacktrace 										-	stack pointer
#0  sigcancel_handler (sig=32, si=0xb6ef74b0, ctx=0xb6ef7530) at nptl-init.c:187	-	sp=0xb6ef74b0
#1  <signal handler called>
#2  0xb6f6ddc2 in nanosleep () at ../sysdeps/unix/syscall-template.S:84			-	sp=0xb6ef7820
#3  0xb6f6dcc2 in __sleep (seconds=0) at ../sysdeps/posix/sleep.c:55			-	sp=0xb6ef7828
#4  0x00008726 in ?? ()

So for syscall plus signal handling at least 888 bytes will be required just
to call the signal handling.  It in fact will require a slight more stack
allocation due pthread_unwind call for actual cancellation (it will be around
1152 bytes more or less).

In a short, if you are using PTHREAD_STACK_MIN and using cancellation I 
recommend to either install a alternate signal stack [1] and take in
consideration {MIN}SIGSTKSZ while setting the thread stack size.

[1] https://www.gnu.org/software/libc/manual/html_node/Signal-Stack.html