Re: feedback from google

[Karim Yaghmour <karim at opersys dot com>]

Chen, Brad wrote:
> Earlier we were looking for stronger justification for 
> scrutinizing kprobes performance; why "high-frequency" 
> probes executing over 5KHz would matter to anybody. I 
> was talking with a friend at Google today (Dick Sites)
> about Systemtap and he jumped on this immediately as 
> an issue. He suggested a couple examples off the top 
> of his head of things we'd want to monitor that pushed 
> well above the 1KHz event frequency range:
> - context switches. Just moving the mouse around on 
> the desktop can generate context switches at over 2KHz. 
> Peak context switch rates are probably over 100K events 
> per second; are we ready?
> - network interrupts. A Gb Ethernet receiving full-sized 
> packets at full rates would generate about 80K network 
> interrupts per second. Smaller packets could be worse.
> Actually, if you just do the arithmetic of peak rates 
> for things like page faults, disk requests, system calls 
> etc. it's easy to find cases that involve interrupt
> rates much higher than 5KHz.

That's the reason why things like LTT and relayfs are necessary.
kprobes is nice for adding random instrumentation, and systemtap
does give it an edge. But generating one debug int per event and
having serialized execution of probes is not something that is
viable, as the gentleman from Google correctly points out. Having
a set of pre-defined, statically-included, and dynamically-
selectable events is the only solution for the kind of situation
where thousands and thousands of events are generated per second.

Karim
-- 
Author, Speaker, Developer, Consultant
Pushing Embedded and Real-Time Linux Systems Beyond the Limits
http://www.opersys.com || karim@opersys.com || 1-866-677-4546