Re: proposed instruction trace support in SystemTap

[Maynard Johnson <maynardj at us dot ibm dot com>]

Frank Ch. Eigler wrote:
> Maynard Johnson <maynardj@us.ibm.com> writes:
>
>
> > [...]
> > I work with Dave, and I told him I would cover for him on this issue
> > while he's away from the office for a bit.  
>
>
> Thanks!
>
>
> > > [...] We could have a
> > > guard expression like dtrace's /.../ - though we would probably just
> > > spell it thusly:
> > >    probe PROBEPOINT if (expr) { } [...]
> >
> > Yes, I think this construct could be very useful.
>
>
> OK, anyone dissenting?
>
>
> > > [...]
> >
> > In fact, the raw trace data is well-defined by the existing ITrace
> > tool I mentioned above.  Of course, this definition is
> > negotiable. [...]
>
>
> My point is more whether this definition ought to be hidden, and be
> hard-coded for interoperation with your itrace post-processing tool.
As in "instruction trace tool", I envision a small number of pre-defined 
data formats that the user could select from that would provide, say, 2 
or 3 different levels of detail in the trace.  One of these formats 
would likely be a superset of the others in order to provide complete 
detail.  Examples:  1) branch-level trace data format; 2) 
instruction-level trace data format (basically, a branch-level trace 
plus load/store addresses, etc.)

These pre-defined formats should be public to facilitate development of 
post-processing tools.  Example 1: An existing pipeline analyzer tool 
uses an instruction-level trace to help performance analysts identify 
the causes of stalls.  An intermediate post-processing tool reads the 
raw trace data, fills in the gaps from the sparse trace data, and 
converts to a particular binary format (qtrace) that is then consumed by 
the pipeline analyzer tool.  Example 2: A higher level analysis tool 
could use some subset of the trace data represented in XML, which could 
be provided by an intermediate post-processing tool.

If feasible and practical, we could allow the user to specify an 
arbitrary data format.  This could be a future enhancement.
> > > Is there an existing kernel API for management of these
> > > [single-stepping] registers/fields?
> >
> > Unfortunately, not that I'm aware of. [...]
>
>
> OK, then how is it done at all?  You just set saved control register
> bits by hand?  Available on how many architectures?  How is the
> callback received?  How does this handle issues such as more than one
> itrace session operating at a time?
Currently, the Performance Inspector ITrace supports i386 (Intel/AMD), 
x86_64, and PPC64.  The tracing enablement is, of course, unique per 
architecture.  For example, on PPC64, the MSR_SE or MSR_BE bit is set to 
enable single step exception or branch exception, respectively.  The 
global variable '__debugger_sstep' (declared in 
include/asm-powerpc/system.h) is set to the appropriate callback 
function.  We also use kprobes to set up handlers for things like task 
switch and process exit.  I'm not as familiar with the details for x86*, 
but I do know the main exception handler for ITrace is set by changing 
entry #1 in the interrupt vector table.

As for handling the case of more than one ITrace session being active at 
once -- no, there's nothing that prevents that from happening.  This is 
certainly an issue that should be addressed.

Regards,
-Maynard
> - FChE