Re: user mode backtrace

[Vara Prasad <prasadav at us dot ibm dot com>]

David Boreham wrote:

> I'd like to get a stack trace for the process that made the
> system call I'm probing (I'm looking at filesystem access
> typically, so reads/writes/syncs etc). The systemtap backtrace
> function appears to only get the kernel mode stack which
> is not much use to me. I was wondering if anyone had
> discovered a good solution to this problem already ?

If i read your above message correctly you would like to put probes in 
the kernel but you want to get a full stack which includes kernel mode 
stack and as well as user mode stack, correct.

Martine, as a first step, can we dump the stack pointer addresses for 
the user space stack as part of our back trace functionality when 
someone says give me full stack. I understand stack unwinding is not 
always reliable in x86 without frame pointers but we have the same 
problem with the kernel as well and we can deal with this also in the 
same fashion. I think it is safe to assume that the most interesting 
part of the user space stack will be in memory not paged out when we are 
in the kernel handler.  Are there some other complications that i am 
missing?

David, if you want to put probes in the user space itself and do back 
trace we are working on user space probing patch that will let you  see 
the stack just in the application space only. We may be able to give 
some thing for you to play within couple of weeks if you are interested 
in user space probing.

> I was thinking perhaps I could invoke pstack (gdb)
> on the current pid/tid. But I'm worried that doing so
> might deadlock since the process is inside a system
> call.
>
> I'm looking at a very large application that beats up on
> the filesystem, in case you're wondering why I want to do
> this. It's so large that nobody is quite sure what code
> access which files, when and why.


I can completely understand why it is difficult to figure out which code 
path you are taking that is one of the reasons why we are working on 
user space probes.

My next question to you is we are looking to put good load on the system 
while running probes, is this filesystem application is something that 
you can share and it is easy for us to run outside of your environment. 
If this is an application that you can not share outside can you help us 
run this application while putting lots of probes touching various parts 
of the system. We can give you scripts, if you like, that will place 
probes in many parts of the system or specifically in the areas this 
application touches the most. Please let us know.

> Thanks.