[Bug threads/10757] GDB does not attach all threads of a multithreaded process => inferior gets SIGTRAP

["pedro at codesourcery dot com" <sourceware-bugzilla at sourceware dot org>]

------- Additional Comments From pedro at codesourcery dot com  2009-10-12 00:56 -------
Subject: Re:  GDB does not attach all threads of a multithreaded process => inferior gets SIGTRAP

On my machine (ubuntu 8.04, 2.6.24-24-generic x86_64) I haven't
managed to get a SIGTRAP, but I see badness nonetheless.

>ulimit -s 32
>~/manythreads2 20&
[1] 26139
>./gdb --pid $(pidof manythreads2)
(...)
[New Thread 0x4d686950 (LWP 418)]
[New Thread 0x40b46950 (LWP 434)]

warning: Can't attach LWP 449: Operation not permitted
[New Thread 0x4de87950 (LWP 476)]
[New Thread 0x4fe8b950 (LWP 489)]
[New Thread 0x4be83950 (LWP 510)]
[New Thread 0x51e8f950 (LWP 529)]

warning: Can't attach LWP -1: No such process
[New Thread 0x49e7f950 (LWP 537)]
(...)

>./gdb -ex "set pagination off" --pid $(pidof manythreads2)
GNU gdb (GDB) 7.0.50.20091011-cvs
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
/home/pedro/.gdbinit:35: Error in sourced command file:
Undefined set remote command: "username gemini".  Try "help set remote".
Attaching to process 8699
Reading symbols from /home/pedro/manythreads2...done.
Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done.
[Thread debugging using libthread_db enabled]
[New Thread 0x4087f950 (LWP 3678)]
[New Thread 0x40481950 (LWP 3698)]

warning: Can't attach LWP 3697: No such process
[New Thread 0x4065b950 (LWP 3733)]
[New Thread 0x41ecc950 (LWP 3753)]
[New Thread 0x404b1950 (LWP 3752)]
[New Thread 0x41bc7950 (LWP 3750)]
[New Thread 0x403f5950 (LWP 3748)]
[New Thread 0x40353950 (LWP 3746)]
[New Thread 0x40ae0950 (LWP 3745)]
[New Thread 0x40322950 (LWP 3744)]
[New Thread 0x4040d950 (LWP 3743)]
[New Thread 0x401a2950 (LWP 3742)]
[New Thread 0x403b5950 (LWP 3741)]
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
0x00007f7cdd611b81 in nanosleep () from /lib/libc.so.6
Setting up the environment for debugging gdb.
Function "internal_error" not defined.
Make breakpoint pending on future shared library load? (y or [n]) [answered N; input not from terminal]
Function "info_command" not defined.
Make breakpoint pending on future shared library load? (y or [n]) [answered N; input not from terminal]
/home/pedro/gdb/baseline/build/gdb/.gdbinit:8: Error in sourced command file:
No breakpoint number 0.
(gdb) b foo
During symbol reading, DW_AT_name missing from DW_TAG_base_type.
During symbol reading, debug info gives in-file macro definition with zero line 0: __STDC__ 1.
Breakpoint 1 at 0x40077c: file manythreads2.c, line 10.
(gdb) commands
Type commands for when breakpoint 1 is hit, one per line.
End with a line saying just "end".
>c
>end
(gdb) c
Continuing.
[Thread 0x403b5950 (LWP 3741) exited]
[Thread 0x401a2950 (LWP 3742) exited]
[Thread 0x4040d950 (LWP 3743) exited]
[Thread 0x40322950 (LWP 3744) exited]
[Thread 0x40ae0950 (LWP 3745) exited]
[Thread 0x40353950 (LWP 3746) exited]
[Thread 0x403f5950 (LWP 3748) exited]
[Thread 0x41bc7950 (LWP 3750) exited]
[Thread 0x404b1950 (LWP 3752) exited]
[Thread 0x41ecc950 (LWP 3753) exited]
[Thread 0x40481950 (LWP 3698) exited]
[Thread 0x4087f950 (LWP 3678) exited]

<ctrl-c> doesn't work.

>cat /proc/8699/status | grep State
State:  Z (zombie)

and 

>ls /proc/8699/task/
13546/ 13547/ 13548/ 13549/ 13550/ 13551/ 13552/ 13553/ 13554/ 3733/  8699/

and they're all zombie.


I'm suspecting something like this:

> There could also
> be an problem with new threads being added to glibc's
> pthreads list just while thread_db is iterating
> over them so that the iterator could miss new threads.

Look here:

>./gdb -ex "set pagination off" --pid $(pidof manythreads2)
GNU gdb (GDB) 7.0.50.20091011-cvs
(...)
Attaching to process 14235
Reading symbols from /home/pedro/manythreads2...done.
Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done.
[Thread debugging using libthread_db enabled]
[New Thread 0x40019950 (LWP 31166)]
[New Thread 0x403d4950 (LWP 31184)]
[New Thread 0x4162e950 (LWP 31200)]
[New Thread 0x41f75950 (LWP 31214)]
[New Thread 0x41636950 (LWP 31226)]
[New Thread 0x41666950 (LWP 31238)]
[New Thread 0x4163e950 (LWP 31237)]
[New Thread 0x418bc950 (LWP 31236)]
[New Thread 0x4055b950 (LWP 31281)]
[New Thread 0x403cc950 (LWP 31299)]
[New Thread 0x41646950 (LWP 31315)]
[New Thread 0x4166e950 (LWP 31329)]
[New Thread 0x40a5a950 (LWP 31341)]
[New Thread 0x41144950 (LWP 31353)]
[New Thread 0x4071f950 (LWP 31352)]
[New Thread 0x41656950 (LWP 31347)]
[New Thread 0x41626950 (LWP 31279)]
[New Thread 0x4165e950 (LWP 31260)]
[New Thread 0x404df950 (LWP 31248)]
[New Thread 0x41776950 (LWP 31232)]
Loaded symbols for /lib/libpthread.so.0
Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
0x00007fe54bab4b81 in nanosleep () from /lib/libc.so.6
Setting up the environment for debugging gdb.
Function "internal_error" not defined.
Make breakpoint pending on future shared library load? (y or [n]) [answered N; input not from terminal]
Function "info_command" not defined.
Make breakpoint pending on future shared library load? (y or [n]) [answered N; input not from terminal]
/home/pedro/gdb/baseline/build/gdb/.gdbinit:8: Error in sourced command file:
No breakpoint number 0.


Now, if we had attached and managed to freeze all threads, "info threads"
wouldn't show "New Thread"s, but it does:

(gdb) info threads
[New Thread 0x4000e950 (LWP 25558)]
[New Thread 0x4167e950 (LWP 25557)]
[New Thread 0x4002e950 (LWP 25556)]
[New Thread 0x403e4950 (LWP 25394)]
[New Thread 0x403ec950 (LWP 25560)]
[New Thread 0x41d49950 (LWP 31359)]
[New Thread 0x40727950 (LWP 31354)]
...
  28 Thread 0x40727950 (LWP 31354)  0x00007fe54bd80796 in pthread_join () from /lib/libpthread.so.0
  27 Thread 0x41d49950 (LWP 31359)  0x00007fe54bd80796 in pthread_join () from /lib/libpthread.so.0
...


This means that GDB hadn't been attached to those threads yet.  They
were still running free.  One of those would be your SIGTRAP thread.
If one does enough "info threads" (perhaps one or two more), GDB should
manage to attach to all threads (and stop reporting "New Threads"s).

After that, you shouldn't be able to reproduce the SIGTRAP.

I guess one way to fix it would be to to have try_thread_db_load_1
know that we're attaching, and make it keep looking for new
threads until no new thread comes out.


-- 


http://sourceware.org/bugzilla/show_bug.cgi?id=10757

------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.