This is the mail archive of the
systemtap@sourceware.org
mailing list for the systemtap project.
Re: Getting userspace stack traces with userspace probes ?
- From: Mark Wielaard <mjw at redhat dot com>
- To: "Daniel P. Berrange" <berrange at redhat dot com>
- Cc: systemtap at sources dot redhat dot com
- Date: Wed, 26 Jan 2011 17:27:32 +0100
- Subject: Re: Getting userspace stack traces with userspace probes ?
- References: <20110126161758.GI18839@redhat.com>
Hi Daniel,
On Wed, 2011-01-26 at 16:17 +0000, Daniel P. Berrange wrote:
> This is printing what I expect. I now want to get associated
> userspace stack traces, so I tried adding
>
> print_ustack(ubacktrace())
>
> And / or
>
> print_ubacktrace()
>
> To all the probe points, all it ever prints for a stack trace is
>
> 0x3992a52e10 : virDomainObjUnref+0x0/0x100 [libvirt.so.0.8.4]
> 0x45900f [libvirtd+0x5900f/0xe8000]
^- this is a very poor hint from systemtap that it found an
address in a module (libvirtd) but could find the symbol
(and by extension the unwind table).
> Which is far too small compared to what GDB says the stack
> trace is, eg I'm expecting at least 6-8 levels deep
>
> (gdb) bt
> #0 virDomainObjUnref (dom=0x1b49390) at conf/domain_conf.c:828
> #1 0x0000000000470f33 in qemuMonitorFree (mon=0x1cb6640) at qemu/qemu_monitor.c:202
> #2 qemuMonitorUnref (mon=0x1cb6640) at qemu/qemu_monitor.c:221
> #3 0x0000000000416ebf in virEventCleanupHandles () at event.c:548
> #4 0x0000000000417c8e in virEventRunOnce () at event.c:612
> #5 0x0000000000418593 in qemudOneLoop () at libvirtd.c:2226
> #6 0x00000000004188b9 in qemudRunLoop (opaque=0x1b42850) at libvirtd.c:2335
> #7 0x0000003980e07761 in start_thread (arg=0x7f6e63423700) at pthread_create.c:301
> #8 0x00000039806e098d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115
>
> Am I using the wrong functions for getting stack traces,
> or is it just not possible to get reliable stacks from
> userspace probes ?
You are doing the right things, but stap isn't smart enough to figure it
all out on its own. The simplest way to tell stap you want all symbols
and unwind tables is to say something like:
$ stap -d /usr/sbin/libvirtd --ldd [...]
> FYI, I'm using systemtap-1.3-3.fc13.x86_64 and
> kernel 2.6.34.7-66.fc13.x86_64
systemtap 1.4 (just released) should at least give you a hint what to
add if it cannot unwind further.
Cheers,
Mark