This is the mail archive of the
systemtap@sourceware.org
mailing list for the systemtap project.
Re: [HELP] ubacktrace() problem
- From: fche at redhat dot com (Frank Ch. Eigler)
- To: Jinhua Luo <luajit dot io at gmail dot com>
- Cc: systemtap at sourceware dot org
- Date: Tue, 12 Jan 2016 09:32:13 -0500
- Subject: Re: [HELP] ubacktrace() problem
- Authentication-results: sourceware.org; auth=none
- References: <CAAc9rOzFmkSKJCmi=0=Vggvaf3-VnwgfHFPfBxG475NKso3arQ at mail dot gmail dot com>
Hi -
luajit.io wrote:
> [...]
> See https://github.com/openresty/nginx-systemtap-toolkit#sample-bt-off-cpu
> for the script.
It's a perl-generated stap script/invocation:
https://github.com/openresty/nginx-systemtap-toolkit/blob/master/sample-bt-off-cpu
> It fails to unwind some user backtrace (which is a big fraction of the
> total samples), and just show some hex adddress, e.g. 0xb77a5d40.
>
> I check that address belongs to the vdso mapping. [...]
Please check if your kernel dbgsym packages include the vdso debuginfo
files. In Fedora for example:
% rpm -ql kernel-debuginfo | grep vdso
/usr/lib/debug/lib/modules/4.2.6-200.fc22.x86_64/vdso
/usr/lib/debug/lib/modules/4.2.6-200.fc22.x86_64/vdso/vdso32-int80.so.debug
/usr/lib/debug/lib/modules/4.2.6-200.fc22.x86_64/vdso/vdso32-syscall.so.debug
/usr/lib/debug/lib/modules/4.2.6-200.fc22.x86_64/vdso/vdso32-sysenter.so.debug
/usr/lib/debug/lib/modules/4.2.6-200.fc22.x86_64/vdso/vdso64.so.debug
These may be necessary to be able to unwind through the kernel-user
transition (if a process is interrupted mid-syscall). I seem to
recall that, years ago, the Fedora kernel team had some struggles to
ensure that these userspace libraries' debug data was generated &
included properly. Maybe the Debian/Ubuntu folks are about to have
the same struggle.
> And, I am also get confused about the warnings:
>
> WARNING: Too many CFI instuctions
> WARNING: DWARF expression stack underflow in CFI
> WARNING: no or bad debug frame hdr
> WARNING: No binary search table for eh frame, doing slow linear search
> for stap_b9bfb422b7bd73acc448d7f775aaacd_37477
These generally reflect imperfections in the gcc-generated unwind
information. These are not something a user can do anything about; we
should probably stop warning about them (or give more information like
the suspect module name).
- FChE