Re: [PATCH] Preheat CPU in benchtests

[Petr Baudis <pasky at ucw dot cz>]

On Tue, Apr 23, 2013 at 02:11:41PM -0300, Adhemerval Zanella wrote:
> On 23-04-2013 12:17, Ondřej Bílka wrote:
> > On Tue, Apr 23, 2013 at 07:22:16AM -0700, Andi Kleen wrote:
> >> Ondřej Bílka <neleai@seznam.cz> writes:
> >>
> >>> Benchmarks now are affected by cpu scaling when initialy run at low
> >>> frequency.
> >>>
> >>> Following benchmark runs nonsensial loop first to ensure that benchmark
> >>> are measured at maximal frequency. This greatly cuts time needed to
> >>> get accurate results.

It seems to me pre-heating for a fixed time period (e.g. 500ms) would be
safer than pre-heating for a fixed number of cycles. However, I'm not
sure about the exact CPU frequency governor rules usually employed.

> >> FWIW it's generally safer to disable frequency scaling explicitely
> >> through sysfs (but that needs root), as the reaction time of the
> >> p-state governour can be unpredictable.
> > Which needs root, so it would request typing password each time you run 
> >  automated benchmarks.
> >
> I see it should be up to developer to setup the environment and to report
> its findings and configuration used. Maybe we might add hooks though 
> env. vars or additional logic on the Makefile/script that runs the benchmark
> (to bind cpu/memory, setup machine scaling, etc.), but I don't think it
> should in benchmark logic to setup such things.

Maybe we should just test whether the conditions are right, i.e. if
frequency scaling is disabled; if we detect a problem, print a fat
warning so that the user knows their results aren't reliable, plus
print an one-liner suggestion for the user to run to fix the situation?

-- 
				Petr "Pasky" Baudis
	For every complex problem there is an answer that is clear,
	simple, and wrong.  -- H. L. Mencken