Re: [PATCH 2/4] Perf test framework

[Doug Evans <dje at google dot com>]

On Fri, Nov 1, 2013 at 12:34 AM, Yao Qi <yao@codesourcery.com> wrote:
> This patch adds a basic framework to do performance testing for GDB.
> perftest.py is about the test case, testresult.py is about test
> results, and how are they saved.  reporter.py is about how results
> are reported (in what format).  measure.py is about measuring the
> execution of tests by a collection of measurements.
>
> In V4:
>
>  - Rename MeasurementCPUTime to MeasurementCpuTime,
>  - Add 'pass' in empty method,
>  - Simplify string comparison in perftest.exp.
>  - Rename GDB_PERFORMANCE to GDB_PERFTEST_MODE and rename
>    GDB_PERFORMANCE_TIMEOUT to GDB_PERFTEST_TIMEOUT.
>
> In V3, there are some changes,
>
>  - Add wall time measurement, cpu time measurement and vmsize
>    measurement.
>  - Rename SingleStatisticTestCase to TestCaseWithBasicMeasurements,
>    which measures cpu time, wall time, and memory (vmsize).
>  - GDB_PERFORMANCE=run|compile|both to control the mode of perf
>    testing.
>  - New GDB_PERFORMANCE_TIMEOUT to specify the timeout.
>  - Split proc prepare to proc compile and startup.
>  - Disable GC while doing measurements.
>
> In V2, there are several changes to address Doug and Sanimir's
> comments.
>
>  - Add copyright header and docstring in perftest/__init__.py
>  - Remove config.py.
>  - Fix docstring format.
>  - Rename classes "SingleVariable" to "SingleStatistic".
>  - Don't extend gdb.Function in class TestCase.  Add a new method run
>    to run the test case so that we can pass parameters to test.
>  - Allow to customize whether to warm up and to append test log.
>  - Move time measurement into test harness.  Add a new class
>    Measurement for a specific measurement and a new class Measure to
>    measure them for a given test case.
>  - A new class ResultFactory to create instances of TestResult.
>  - New file lib/perftest.exp, which is to do some preparations and
>    cleanups to simplify each *.exp file.
>  - Skip compilation step if GDB_PERFORMANCE_SKIP_COMPILE is set.
>
> gdb/testsuite/
>
>         * lib/perftest.exp: New.
>         * gdb.perf/lib/perftest/__init__.py: New.
>         * gdb.perf/lib/perftest/measure.py: New.
>         * gdb.perf/lib/perftest/perftest.py: New.
>         * gdb.perf/lib/perftest/reporter.py: New.
>         * gdb.perf/lib/perftest/testresult.py: New.

Ok by me.
[Might want to give it a day or two to see if anyone else has any
final comments.]

Thanks for the perseverance!