Re: [PATCH] New numbers in the benchtests.

[Carlos O'Donell <carlos at redhat dot com>]

On 12/14/2017 08:20 PM, Ben Woodard wrote:
> A customer who has asked not to be named that has had some problems
> with performance for some inputs of certain libm functions. We felt
> that a good idea would be to capture and preserve these test cases so
> that we could track and measure the performance for these cases over
> time. These values evidently came from analyzing performance problems
> with actual running codes. The intent is that more test cases will be
> added over time.
> 
> The fast and the slow cases are separated in the test results so that
> we could not only track any improvements in the worst case performance
> but also detect any degregation in the performance of the optimal
> cases should we change the algorithm or split the range different.
This looks good to me, and I've always been suggesting that hardware
and software vendors should contribute important numbers upstream so
we can see which classes of users are impacted. I'm happy to see this
kind of thing go upstream. Some might argue these numbers should stay
downstream in RHEL, but I think it does a disservice. With my volunteer
hat on I'd like to know how my changes impact all sorts of users.

> Here are the results of that part of the modified tests:
> 
>   "exp": {
>    "": {
>     "duration": 3.10837e+10,
>     "iterations": 5.654e+06,
>     "max": 37073.6,
>     "min": 61.904,
>     "mean": 5497.65
>    },
>    "144bits": {
>     "duration": 2.9752e+10,
>     "iterations": 1.83e+06,
>     "max": 36458.4,
>     "min": 159.793,
>     "mean": 16257.9
>    },
>    "768bits": {
>     "duration": 2.98869e+10,
>     "iterations": 105000,
>     "max": 307814,
>     "min": 277808,
>     "mean": 284637
>    },
>    "redhat-slow-customer-cases": {
>     "duration": 2.90641e+10,
>     "iterations": 241000,
>     "max": 173768,
>     "min": 118393,
>     "mean": 120598
>    },
>    "redhat-fast-customer-cases": {
>     "duration": 2.89308e+10,
>     "iterations": 1.94562e+08,
>     "max": 285.431,
>     "min": 143.328,
>     "mean": 148.697
>    }
>   },
> 
>   "pow": {
>    "": {
>     "duration": 2.91114e+10,
>     "iterations": 5.9899e+07,
>     "max": 1340.38,
>     "min": 149.229,
>     "mean": 486.009
>    },
>    "240bits": {
>     "duration": 3.76374e+10,
>     "iterations": 400000,
>     "max": 110951,
>     "min": 76769.9,
>     "mean": 94093.4
>    },
>    "768bits": {
>     "duration": 1.54488e+11,
>     "iterations": 101000,
>     "max": 1.60541e+06,
>     "min": 742756,
>     "mean": 1.52959e+06
>    },
>    "redhat-slow-customer-cases": {
>     "duration": 3.02998e+10,
>     "iterations": 42000,
>     "max": 790097,
>     "min": 692942,
>     "mean": 721425
>    },
>    "redhat-fast-customer-cases": {
>     "duration": 2.89989e+10,
>     "iterations": 4.8063e+07,
>     "max": 1339.97,
>     "min": 564.295,
>     "mean": 603.352
>    }
>   },
> 
> Signed-off-by: Ben Woodard <woodard@redhat.com>

Modulo the nits below, you get my Reviewed-by.

Reviewed-by: Carlos O'Donell <carlos@redhat.com>

> ---
>  benchtests/exp-inputs |  9 +++++++++
>  benchtests/pow-inputs | 12 ++++++++++++
>  2 files changed, 21 insertions(+)
> 
> diff --git a/benchtests/exp-inputs b/benchtests/exp-inputs
> index aff3fb42f1..b94e19c4f5 100644
> --- a/benchtests/exp-inputs
> +++ b/benchtests/exp-inputs
> @@ -587,3 +587,12 @@
>  0x1.0000015853da7p0
>  0x1.0000098e5e007p0
>  0x1.0000099a1ac59p0
> +
> +# Contributed based on customer reports
> +# Ben Woodard woodard@redhat.com

Please use standard '<>' bracing to make this easier to parse in the future.

e.g.

Contact: Ben Woodard <woodard@redhat.com>

> +## name: redhat-slow-customer-cases
> +0x1.0p-53
> +## name: redhat-fast-customer-cases
> +0x1.0p-52
> +0x1.0p+0
> +0x1.999999999999Ap-4
> diff --git a/benchtests/pow-inputs b/benchtests/pow-inputs
> index 78f8ac73d5..cea50edea7 100644
> --- a/benchtests/pow-inputs
> +++ b/benchtests/pow-inputs
> @@ -509,3 +509,15 @@
>  0x1.f8b79758182dap-884, 0x1.ed6174093fca4p-6
>  0x1.fa5c677254961p133, -0x1.c91962524971ep-1
>  0x1.ff0544adacb78p649, -0x1.6c17c3a7210e2p-1
> +
> +
> +# Contributed based on customer reports
> +# Ben Woodard woodard@redhat.com

Contact: Ben Woodard <woodard@redhat.com>

> +## name: redhat-slow-customer-cases
> +0x1.fffffffffffffp-1, 0x1.8p+0
> +0x1.ffffffffffffdp-1, 0x1.8p+0
> +0x1.ffffffffffff7p-1, 0x1.8p+0
> +## name: redhat-fast-customer-cases
> +0x1.ffffffffffffep-1, 0x1.8p+0
> +0x1.ffffffffffffcp-1, 0x1.8p+0
> +0x1.999999999999ap-4, 0x1.8p+0
> 


-- 
Cheers,
Carlos.