Re: static instrumentation for kernel

[Mathieu Desnoyers <compudj at krystal dot dyndns dot org>]

* Frank Ch. Eigler (fche@redhat.com) wrote:
> Hi -
> 
> > [...]
> > >    SYSTEMTAP_PROBE(name) 
> > >    SYSTEMTAP_PROBE_N(name,arg1) // arg1 castable to int64_t numeric
> > >    SYSTEMTAP_PROBE_NS(name,arg1,arg2) // arg2 castable to char* string
> > >
> 
> > It seems to somehow limit the variations of parameters that can be
> > passed as argument : one could need a particular macro for a
> > (int64_t,void*,int32_t,byte).  We can imagine thousands of
> > variations. Or maybe the goal is just to allow a small subset of
> > types ?
> 
> Since systemtap uses (approximately) only two data types, the
> combinatorial explosion is not so bad.  Integers and pointers are
> castable to int64_t, and char* strings can get passed as is.  So we
> would have 2 ** (maximum-arity) macros to generate, say 256.
> 

If you don't need the event logging to be efficient with regards to minimal
type sizes, it could be acceptable.

Still, I like your idea to reuse the logging functions per types. It can save
precious memory space, at the cost of a call.

> 
> 
> > The problem with the if() { call } scheme comes when you want to
> > deactivate the probe. [...]  When the probe is removed, a thread
> > could clearly end up calling a bad function pointer by being stalled
> > between the if and the call. [...]
> 
> Thanks for pointing out that such concurrency issues have to be
> investigated.  Since session startup/shutdown is not as time critical
> as probe execution is, something as conservative as the
> djprobes-inspired IPI/quiescence could do the trick here.
> 

With the same limitation concerning the preemption. I suggest that you disable
preemption explicitely around the if statement, so this probe can be inserted in
this context.

> 
> > > [...] The indirect call would be somewhat slower but much simpler
> > > than a djprobe, and much faster than a kprobe.  [...]
> 
> > But still slower than an inlined function. Inlining has this
> > interesting point: it doesn't have to build up the function call by
> > putting the arguments on the stack and does not do the call itself.
> 
> Indeed, but if the code to be inlined is to be changeable (i.e., let
> the user write an arbitrary probe handler), then this can't work
> without kernel recompilation/rebooting each time.
> 

Well, it's partly true. What about an inline function for the fast case (event
logging) and an optionnal call to a dynamically settable function for special
purposes ?

> 
> > If you want to do static tracing, you should really investigate the
> > fastest solutions available.
> 
> The thing is, I don't just want to do *tracing*.
> 

The LTTng mechanism does a little more than just "tracing". Its logging
functions can easily used to replace your SystemTAP stp_printf mechanism which
seems to be a bit limited. From what I see in your CVS repository, this logging
function really has to be called with interrupts deactivated to be reentrant
(string.c:_stp_sprintf(). Therefore, it can't log a non maskable interrupt
handler without risks of corruption.

The principal goals of LTTng is to offer a completely reentrant logging
function that can write all types known to the C language, plus a little more.


> 
> > [...]  And did I say that we have a modular trace viewer (LTTV) ?
> 
> Emitting data in a compatible form could be very useful for systemtap
> users.
> 

Yes, I think it would. But it could be also very interesting to see SystemTAP
log through the LTTng mechanism. It would make it able to log from execution
contexts currently non probe safe.


Mathieu


OpenPGP public key:              http://krystal.dyndns.org:8080/key/compudj.gpg
Key fingerprint:     8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68