Re: Dynamic djprobe (and summary of Q&A)

[Karim Yaghmour <karim at opersys dot com>]

Frank Ch. Eigler wrote:
> I wonder if the "_all_" is not quite as important as that.  For
> someone so strictly inclined, the translator could be made to assert
> djprobe availability for all spots, and know what to do with an
> "unable" error.  For someone else who's not so absolutist, it may be
> enough to know which probes ended up using which breakpoint mechanism.

I could be wrong, but my experience has been what I just described.
Again, it's all about how the debugging process works. If I'm going
to end up spending a considerable amount of time figuring out where and
where not my djprobed points will be useful, I might as well just
add those markers all over the place, use kexec to start the newly
rebuilt kernel, and then dynamically hook to those marked areas I
want to look at selectively.

> We may be able to measure the overheads to allow factoring out of the
> differences.

Maybe, but you still won't be able to avoid the actual runtime costs.

> There does not seem to be a resource constraint in concurrently
> inserted djprobes, so switching dynamically, such as on the grounds of
> frequent invocation, is probably not helpful.  Similarly, the djprobes
> applicability predicates don't seem random, so results would be
> reproducible from run to run.

I'm not sure how this argument applies to what I describe.

> I suspect such a black & white dichotomy does not apply widely.

Yes, this is a perception difference. If you look at what people are
actually doing right now, though, you will notice that that's exactly
what's happening.

>  But
> even if it does, it is not relevant to properties of the Hitachi
> djprobes code.

Sorry, I don't see this statement as substantiated. Sure, what people
are currently doing doesn't change the properties of djprobes, but
if you look at the applicability of djprobes to how people use such
things, one can only conclude that it can only be applied to those
same cases where kprobes is already applicable.

>  Once one is willing to put markers into the kernel
> source and recompile and reboot, then one doesn't even have to use
> djprobes per se to go fast (much faster than int3).

Actually I think things are slightly different. If you intend to do
high-speed tracing of a certain portion of the kernel, then you can
insert assembly instructions as buffer for later inserting dynamically-
generated code for jmping elsewhere. That's a perfectly valid approach,
and one that had successfully been implemented by the kernel hooks
mechanism.

As for the time it takes to recompile-reboot, this is mitigated by
the imminent inclusion of kexec. And before I'm told "yes, but, we
can't always reboot", the fact of the matter is that if you're in
the process of doing high-speed tracing, it's likely because you intend
to optimize a bit of code in the kernel, and are therefore willing
to reboot anyway once you have an update or have a test system
replicating the live system where you're actually sandboxing your
tests, so static instrumention should not be an issue. And if you're in
the process of trying to find a fault on a live system, then kprobes is
a perfectly viable way of not having to reboot, albeit it isn't as fast
as statically-inserted probe points.

In sum, I'm just saying that the real advantage of adding djprobe is
outweighed by the limitations to the actual range of cases where it will
be an advantage. Granted, it's a nice optimization when it can be used,
but it certainly falls short of reaching its stated goal: provide a
generic high-speed probing mechanism.

But, in fact, the decision is not mine. I'm just voicing concerns here.
You're free to do as you wish. It'd be really interesting, though, to
see an anlysis of how many of the total assembly instructions in an
x86 kernel can use djprobe vs. those that can't, once the complete
list of limitations is known. For now, I guess, an analysis of how
many instructions are less than 5 bytes vs. those that are equal and
larger should be food for thought.

Karim
-- 
Author, Speaker, Developer, Consultant
Pushing Embedded and Real-Time Linux Systems Beyond the Limits
http://www.opersys.com || karim@opersys.com || 1-866-677-4546