Re: [ECOS] libc-time-clock test doesn't seem to be written correctly??

[Brij Bihari Pandey <fuzzhead012 at yahoo dot com>]

Hi Jonathan,

> > Another related thought - shouldn't the analysis of going through valid
> >  results and comparing against average -- be done using floating point
> >  arithmetic for mean/err computation and comparision?
>
> Why bother?
Say if things are pretty stable and the the counter values are mostly 3 but for
some samples where there are 2 (even a single 2 does it), summation will turn
out to Not exact multiple of 3, hence integer mean calculation will give the
mean as 2, so notion of mean is kinda way off rather than being close to actual
mean.

Also if clock is pretty stable but counter values turn out to be on smaller
side, then test will incorrectly cause failures for tolerance values of say
40%,  because err(100*(3-2)/2) > TOLERANCE(40).

Now, after you have applied (mean+1) fix for division by zero, the above issue
(still remains though) shifts to different examples.

In general, if the clock is stable but value returned by clock_loop is in
smaller range, the way test is written, it will cause incorrect test failure
messages. for example, values returned by clock_loop say in closed range [0-4],
mean being 2, will cause test failures even for 100% TOLERANCE. while same
won't happen even for a 40% TOLERANCE when the range is [20-24]. 

This was why, I had suggested the following in earlier mail.
> > - Should test fail on first err value, that is not within TOLERANCE? It
> > may be better if failure-tolerance (another parameter) is taken on 
> > number-of-samples not within TOLERANCE limit, to decide the
> > test-failure.
> 
> If the system disappears into the middle of nowhere for a long time before 
> coming back even once, then something needs investigating. It may be 
> alright, but that's not up to this test to know.
That's right reason to look for, but test seems to be overdoing in deciding
test failures.

I will put forth another suggestion. Rather than looking for percentage
tolerance, what if we look for how much +/- from the average we can tolerate? 

> > Though I don't quite get - - What is meant by stability of clock? - In
> > what way the nature of test ensures it is testing stability of clock?
> 
> Roughly equal period between clock increments.
Sorry, but I still don't get - how "equal period between clock increments"
governs consistent/stable (from the point of view of test) return values from
clock_loop.

brij


__________________________________
Do you Yahoo!?
SBC Yahoo! DSL - Now only $29.95 per month!
http://sbc.yahoo.com

-- 
Before posting, please read the FAQ: http://sources.redhat.com/fom/ecos
and search the list archive: http://sources.redhat.com/ml/ecos-discuss