Re: analysis of polling in systemtap

[Martin Hunt <hunt at redhat dot com>]

On Mon, 2005-12-19 at 14:29 -0500, Frank Ch. Eigler wrote:
> hunt wrote:
> 
> > [...]  Unfortunately if an error is hit, the error message is
> > sometimes cleared before the script exits. [...]  Error and warning
> > messages are sent over a separate channel and are displayed
> > immediately.  [...]
> 
> The error/warning messages already go to stderr, if I read the code
> correctly.  A script that generates lots of stdout + stderr traffic
> may need to be diagnosed the same way as for unix tools generally:
> separate stderr redirection.

Yes, that's why it is done that way. Still, the problem is where error
messages scroll off the screen before the script terminates, leaving the
user puzzled about what happened. I was thinking maybe if the script
terminates abnormally, the last line printed before exiting should be an
error summary.

> > [...]
> > 	sortn(called, 10, SORT_VALUE)
> >       printa(called, 10, "%1d called %2s\t%d times")
> >       delete called
> > [...]
> 
> One obvious problem with this is the race and possible mismatches
> between the "sortn" and "printa" calls.  

The same race condition exists with the generated code from foreach. A
write lock is needed for the sort and a readlock for the foreach loop.
There is a small race in there.  The only real risk is from someone
running multiple timer probes sorting the same data. In that case, they
deserve the confusing output they will probably get.  System stability
is not at risk.
 
> Compared to this, the
> proposed sorted-limited-foreach is indeed somewhat slower, but
> likewise requires the same number of script lines:
> 
>      foreach ([x,y] in called- limit 10) 
>        printf("%d called %s\t%d times", x, y, foo[x,y] )

This isn't bad either. It gets more complicated when you want stats and
histograms.

>      # there is also a race here.
>      delete called

Yes, this is a much more serious race. I've been thinking about how to
solve that. The best way to fix this from an efficiency and reliability
point of view is to simply add another flag to the runtime print and
aggregation commands to clear the map as it is printed. I don't have any
good ideas about how this would made part of the systemtap language.

Martin