Re: Using systemtap to validate manual code inspection.

[OndÅej BÃlka <neleai at seznam dot cz>]

On Tue, May 07, 2013 at 08:26:54AM -0400, Carlos O'Donell wrote:
> On 05/07/2013 05:14 AM, Siddhesh Poyarekar wrote:
> > On Mon, May 06, 2013 at 03:41:10AM -0400, Carlos O'Donell wrote:
> >> I recently had to do some manual code inspection
> >> for a bug which has no reproducer (patch forthcoming).
> >>
> >> The manual inspection found real bugs, but I wanted
> >> to get more coverage for the failure scenarios.
> >>
> >> I wrote up my experience using systemtap to inject
> >> failures into glibc:
> >> http://sourceware.org/glibc/wiki/Testing/WhiteBox
> >>
> >> Looking forward to white box and unit tests ;-)
> > 
> > This is really neat, but the problem here is not limited to systemtap
> > needing root privileges.  The more subtle problem is the dependency of
> > these tests on line numbers within code.  The tests will have to be
> > continually updated as the code gets updated and that may blow out of
> > proportion as the number of such tests per source file grows.
> > 
> > It would be nicer if it were possible to add annotations within source
> > that systemtap picks up and set probes at those points.  I'm not
> > talking about probe points, since they're additional code within the
> > library and not just source level annotations.  Something like this:
> > 
> > some_func (int *in)
> > {
> >   /** SYSPROBE:probe=simulate_crash;file=simulate_crash.stp */
> >   glob = *in;
> > }
> > 
> > The problem with doing this is that it could quickly clutter up code
> > with white box testing annotations.
> 
> The implementation details of adding white box testing
> to glibc is certainly a complex and difficult topic that
> will require some designing :-)
> 
> Line numbers are certainly a pain to update.
> 
> We really want an automated way to generate the .stp
> file from an annotation and have that become a whitebox
> test associated with a particular test.
> 
> e.g.
> 
> * Annotate source.
> * Auto-generate .stp files from source.
> * Run test-stpN with .stp file active.
> * Observe PASS/FAIL.
> 
You can do this without annotating source. Use diff approach, for probe
copy fragment consisting of three lines before and three after with
relevant code, for example bug with freed memory could be writen as:

START_DIFF

some_func (int *in)
{
+ PROBE{ 
+ free(in);
+ }
glob = *in;
}

END_DIFF

Three lines are in 99.9% cases enough to uniquely determine position in code, when ambiguous
write bigger fragment.

Ondra