Re: Implementing a generic binary trace interface.

["Jose R. Santos" <jrs at us dot ibm dot com>]

Frank Ch. Eigler wrote:

> zanussi wrote:
>
> > [...]
>
> Thank you for your rather detailed proposal.
>
>
> > [...]
> > probe kernel.function("sys_open")
> > {
> > 	data.hook_id = HOOKID_OPEN
> > 	fill_in_common(data)
> > 	data.flags = $flags;
> > [...]
> > }
> >
> > function fill_in_common(data)
> > {
> > 	data.timestamp = timestamp_us();
> > }
>
> From this discussion, it seems to me that the sole use of this sort of
> struct construct is for purposes of tracing.  There are no language
> elements proposed for generically extracting information such as field
> names / values.  Have you thought about what happens in connection
> with conditionals / loops?  How should functions be type checked when
> they take structs?  And return them?  And how are these connected to
> arrays, when they are also passed by reference?  Can they be global?
> Why not?
>
> This seems to me as a too special-purpose and of too limited
> application for it to show up in the script language in this
> misleadingly general-looking form.
>
> Here is a possible way of accomplishing the same goal - lightweight
> binary tracing.  No new misleading notation, just special functions,
> like printf* today.
>  
I wonder, would local arrays be less misleading in your opinion.  If the 
array could contain multiple types and be use as both an argument and 
return value, this would also work for tracing.  The use of arrays does 
seem to better fit the SystemTap coding style.

> # probe kernel.function("sys_open")
> # {
> # 	trace("hook_id", HOOKID_OPEN)
> # 	trace_common()
> # 	trace("flags", $flags)
> # 	[...]
> # }
> # 
> # function trace_common()
> # {
> # 	trace("timestamp", timestamp_us())
> # }
>
> The translator could generate similar code to your sketch: a field
> assignment in a synthesized buffer struct for each executed trace().
> It could avoid widening $target values to systemtap types, or include
> explicit encoding/casting directives.  It could infer a "flush" at the
> end of a probe that calls trace().  It could invent field names if you
> don't care to give one for each call site.
>
> For that matter, this could all be hidden in printf itself, with
> special formatting directives.  (Don't assume that this would
> necessarily invoke the kernel vsnprintf.  The translator could be a
> smarter than to call the runtime _stp_printf function.)
>
>  
Printf is the current mechanism that is used in LKET.  While it works, I 
am finding the code does not look a clean as it could with a better 
interface. BTI does offer nice interface that is very easy to decode and 
easy to implement.  Then new gBTI is better, but still somewhat not as 
flexible to the any type of data that we may want to put.

> By the way, in either the structy or printy design, how do you expect
> the actual binary struct metadata to be used at the end?  How would
> the binary trace data be decoded?
>
> - FChE
>
>  
Thats up to the trace format to provide enough information to decode the 
trace.  All of the hooks will have some fixed size data followed by data 
that is specific to the hook.  For example:

(int)HookID|(long long)Timestamp|(int)CPU|(int)pid|(long)sizeof 
hook|.........

We can do partial decoding of all the fixed size items and as long as we 
have the size of the hook somewhere in there, we can calculate the 
offset of the next hook.  Its up to the user space tool to know what 
each trace hook represents and do decoding of the data that cant be 
interpreted by a generic decoder.


-JRS