This is the mail archive of the
systemtap@sourceware.org
mailing list for the systemtap project.
Re: Segmentation Fault: Task died at uprobe probepoint
On Mon, Jun 14, 2010 at 6:33 PM, Frank Ch. Eigler <fche@redhat.com> wrote:
> Gaurav Dhiman <dimanuec@gmail.com> writes:
>
>> [...]
>> The following commit is on the head of the git log I have (I don't
>> know how to get the exact systemtap version):
>>
>> commit b275171e26c0cf9441858209ac62e70b8ddda71d
>> [...]
>
> That's perfectly specific, thanks.
>
>> [...]
>> Following are the logs from the Pass 5 of the run with stap -vvvt
>> -DDEBUG_UPROBES:
>>
>> Pass 5: starting run.
>> [...]
>> stapio:stp_main_loop:571 probe_start() returned 0
>> stap_uprobe_change_plus:67: +uprobe spec 0 idx 0 process uprobe-test[25763] addr 0000000000400517 pp process("/home/gdhiman/tests/uprobe-test").function("test_func@/home/gdhiman/tests/uprobe-test.c:4")
>> test begin!!
>> test_func called!
>> i=0x5
>> stap_uprobe_change_minus:220: -uprobe spec 0 idx 0 process uprobe-test[25763] reloc 0000000000400517 pp process("/home/gdhiman/tests/uprobe-test").function("test_func@/home/gdhiman/tests/uprobe-test.c:4")
>>
>> The program receives a SIGSEGV signal at this point.
>
> OK, something is wrong here. ?See that -uprobe line? ?It means stap
> has for some reason determined that it's time to remove the uprobe,
> but that should not happen until either the script or the target
> program exits.
>
> This will make things more verbose, but could you try adding
> -DDEBUG_TASK_FINDER and -DDEBUG_TASK_FINDER_VMA to the stap
> command line?
>
You are right, that looks suspicious; it is actually removing the
probe, when it should not be doing it. Here is the more verbose
output:
stap_uprobe_process_found:246: +proc pid 27237 stf ffffffffa001dce0
ffffffffa001dce0 path /home/gdhiman/tests/uprobe-test
stap_uprobe_change_plus:67: +uprobe spec 0 idx 0 process
uprobe-test[27237] addr 0000000000400517 pp
process("/home/gdhiman/tests/uprobe-test").function("test_func@/home/gdhiman/tests/uprobe-test.c:4")
test begin!!
_stp_tf_exec_cb:29: tsk 27237:27237 , register_p: 1, process_p: 1
__stp_call_mmap_callbacks:599: pid 27237, a/l/o/p/path 0x400000
0x1000 0x0 r-xp /home/gdhiman/tests/uprobe-test
_stp_tf_mmap_cb:51: mmap_cb: tsk 27237:27237 path
/home/gdhiman/tests/uprobe-test, addr 0x00400000, length 0x00001000,
offset 0x0, flags 0x8001875
_stp_tf_mmap_cb:63: vm_cb: matched path
/home/gdhiman/tests/uprobe-test to module (for sec: .absolute)
__stp_call_mmap_callbacks:599: pid 27237, a/l/o/p/path 0x600000
0x2000 0x0 rw-p /home/gdhiman/tests/uprobe-test
_stp_tf_mmap_cb:51: mmap_cb: tsk 27237:27237 path
/home/gdhiman/tests/uprobe-test, addr 0x00600000, length 0x00002000,
offset 0x0, flags 0x8101873
__stp_call_mmap_callbacks:599: pid 27237, a/l/o/p/path 0x7faadb80e000
0x20000 0x0 r-xp /lib/ld-2.9.so
_stp_tf_mmap_cb:51: mmap_cb: tsk 27237:27237 path /lib/ld-2.9.so, addr
0x7faadb80e000, length 0x00020000, offset 0x0, flags 0x8000875
_stp_tf_mmap_cb:92: registered 'ld-2.9.so' for 27237 (res:0)
__stp_call_mmap_callbacks:599: pid 27237, a/l/o/p/path 0x7faadba2d000
0x2000 0x1f000 rw-p /lib/ld-2.9.so
_stp_tf_mmap_cb:51: mmap_cb: tsk 27237:27237 path /lib/ld-2.9.so, addr
0x7faadba2d000, length 0x00002000, offset 0x1f000, flags 0x8100873
__stp_utrace_task_finder_target_syscall_exit:1423: tsk 27237 found
mmap(0x0), returned 0x7faadba2c000
__stp_utrace_task_finder_target_syscall_exit:1423: tsk 27237 found
mmap(0x0), returned 0x7faadba2a000
__stp_utrace_task_finder_target_syscall_exit:1423: tsk 27237 found
mmap(0x0), returned 0x7faadba25000
__stp_call_mmap_callbacks:599: pid 27237, a/l/o/p/path 0x7faadba25000
0x5000 0x0 r--p /etc/ld.so.cache
_stp_tf_mmap_cb:51: mmap_cb: tsk 27237:27237 path /etc/ld.so.cache,
addr 0x7faadba25000, length 0x00005000, offset 0x0, flags 0x8000071
__stp_utrace_task_finder_target_syscall_exit:1423: tsk 27237 found
mmap(0x0), returned 0x7faadb49c000
__stp_call_mmap_callbacks:599: pid 27237, a/l/o/p/path 0x7faadb49c000
0x372000 0x0 r-xp /lib/libc-2.9.so
_stp_tf_mmap_cb:51: mmap_cb: tsk 27237:27237 path /lib/libc-2.9.so,
addr 0x7faadb49c000, length 0x00372000, offset 0x0, flags 0x8000075
_stp_tf_mmap_cb:92: registered 'libc-2.9.so' for 27237 (res:0)
__stp_utrace_task_finder_target_syscall_exit:1423: tsk 27237 found
mmap(0x0), returned 0x7faadb804000
__stp_call_mmap_callbacks:599: pid 27237, a/l/o/p/path 0x7faadb804000
0x5000 0x168000 rw-p /lib/libc-2.9.so
_stp_tf_mmap_cb:51: mmap_cb: tsk 27237:27237 path /lib/libc-2.9.so,
addr 0x7faadb804000, length 0x00005000, offset 0x168000, flags
0x8100073
__stp_utrace_task_finder_target_syscall_exit:1423: tsk 27237 found
mmap(0x0), returned 0x7faadb809000
__stp_utrace_task_finder_target_syscall_exit:1423: tsk 27237 found
mmap(0x0), returned 0x7faadba24000
__stp_utrace_task_finder_target_syscall_exit:1423: tsk 27237 found
mmap(0x0), returned 0x7faadba23000
__stp_utrace_task_finder_target_syscall_exit:1423: tsk 27237 found
munmap(0x7faadba25000), returned 0x0
test_func called!
i=0x5
stap_uprobe_process_found:246: -proc pid 27237 stf ffffffffa001dce0
ffffffffa001dce0 path /home/gdhiman/tests/uprobe-test
stap_uprobe_change_minus:220: -uprobe spec 0 idx 0 process
uprobe-test[27237] reloc 0000000000400517 pp
process("/home/gdhiman/tests/uprobe-test").function("test_func@/home/gdhiman/tests/uprobe-test.c:4")
_stp_tf_exec_cb:29: tsk 27237:27237 , register_p: 0, process_p: 1
>
> I meant the uprobe-test binary, just to see what instructions were
> in the vicinity of the test_func function. ?Just that area.
>
Here it is:
000000000040050c <test_func>:
40050c: 55 push %rbp
40050d: 48 89 e5 mov %rsp,%rbp
400510: 48 83 ec 10 sub $0x10,%rsp
400514: 89 7d fc mov %edi,-0x4(%rbp)
400517: 8b 75 fc mov -0x4(%rbp),%esi
40051a: bf 3c 06 40 00 mov $0x40063c,%edi
40051f: b8 00 00 00 00 mov $0x0,%eax
400524: e8 cf fe ff ff callq 4003f8 <printf@plt>
400529: 8b 45 fc mov -0x4(%rbp),%eax
40052c: c9 leaveq
40052d: c3 retq
000000000040052e <main>:
40052e: 55 push %rbp
40052f: 48 89 e5 mov %rsp,%rbp
400532: 48 83 ec 10 sub $0x10,%rsp
400536: bf 05 00 00 00 mov $0x5,%edi
40053b: e8 cc ff ff ff callq 40050c <test_func>
400540: 89 45 fc mov %eax,-0x4(%rbp)
400543: b8 00 00 00 00 mov $0x0,%eax
400548: c9 leaveq
400549: c3 retq
40054a: 90 nop
40054b: 90 nop
40054c: 90 nop
40054d: 90 nop
40054e: 90 nop
40054f: 90 nop
Thanks,
-Gaurav