Re: [RFC] systemtap: begin the process of using proper kernel APIs (part1: use kprobe symbol_name/offset instead of address)

[James Bottomley <James dot Bottomley at HansenPartnership dot com>]

On Wed, 2008-07-16 at 20:49 -0500, James Bottomley wrote:
> On Wed, 2008-07-16 at 20:05 -0400, Masami Hiramatsu wrote:
> > Hmm, here is what I got with your patch;
> > $ stap --kelf -e 'probe kernel.function("do_open"){}' -p2
> > # probes
> > kernel.function("do_open@arch/x86/kernel/apm_32.c:1557") /* pc=<do_open+0x0> */ /* <- kernel.function("do_open") */
> > kernel.function("do_open@fs/block_dev.c:928") /* pc=<do_open+0x0> */ /* <- kernel.function("do_open") */
> > kernel.function("do_open@fs/nfsctl.c:24") /* pc=<sys_nfsservctl+0x55> */ /* <- kernel.function("do_open") */
> > kernel.function("do_open@ipc/mqueue.c:630") /* pc=<do_open+0x0> */ /* <- kernel.function("do_open") */
> > 
> > Without your patch;
> > $ stap -e 'probe kernel.function("do_open"){}' -p2
> > # probes
> > kernel.function("do_open@arch/x86/kernel/apm_32.c:1557") /* pc=0x10382 */ /* <- kernel.function("do_open") */
> > kernel.function("do_open@fs/block_dev.c:928") /* pc=0xa0750 */ /* <- kernel.function("do_open") */
> > kernel.function("do_open@fs/nfsctl.c:24") /* pc=0xa6411 */ /* <- kernel.function("do_open") */
> > kernel.function("do_open@ipc/mqueue.c:630") /* pc=0xc55a6 */ /* <- kernel.function("do_open") */
> > 
> > Obviously, the 3rd "do_open" is fully inlined, so it can be
> > correctly handled by kprobes, because it has different
> > symbol(sys_nfsservctl). However, other "do_open" have
> > same symbol(do_open). these will be put on same
> > address (at the first symbol on kallsyms list).
> > 
> > So, we need a bridge for the gap of function addresses
> > between kallsyms and dwarf.
> 
> You mean this particular problem:
> 
> hobholes:/home/jejb/git/BUILD-2.6# grep do_open /proc/kallsyms 
> c01af160 t do_open
> c01d5d40 t do_open
> 
> It's certainly a material defect in the current API.  I'll think about
> it and see if I can come up with a solution.

OK, thought about it.  There seem to be two possible solutions

     1. Get systemtap always to offset from non-static functions.  This
        will use the standard linker to ensure uniqueness (a module
        qualifier will still need to be added to the struct kprobe for
        lookup, since modules can duplicate unexported kernel symbols).
     2. Add the filename as a discriminator for duplicate symbols in the
        kallsyms program (would still need module qualifier).  This is
        appealing because the path name would be printed in the kernel
        trace to help with oops tracking

This is where negotiations come in.  To me 2. looks to be better because
it will help us with oops tracking.  On the other hand, it's usually
pretty obvious from the stack trace context which files the duplicate
symbols are actually in; what do other people think?

James