Re: [PATCH V2] benchtests: Expand range of tests names in schema.json

["Rodriguez Bahena, Victor" <victor dot rodriguez dot bahena at intel dot com>]

-----Original Message-----
From: Siddhesh Poyarekar <siddhesh@gotplt.org>
Date: Wednesday, November 22, 2017 at 4:18 AM
To: Victor Rodriguez Bahena <victor.rodriguez.bahena@intel.com>,
"libc-alpha@sourceware.org" <libc-alpha@sourceware.org>
Subject: Re: [PATCH V2] benchtests: Expand range of tests names in
schema.json

>On Tuesday 21 November 2017 11:34 PM, Victor Rodriguez wrote:
>> When executing bench-math the benchmark output is invalid with this
>> error msg:
>> 
>> Invalid benchmark output: 'workload-spec2006.wrf' does not match any of
>> the regexes: '^[_a-zA-Z0-9]*$¹ or Invalid benchmark output: Additional
>> properties are not allowed ('workload-spec2006.wrf' was unexpected)
>> 
>> This patch changes regex in benchout.schema.json to accept symbols in
>> benchmark tests names. This patch came after error running
>> workload-spec2006.wrf, that detects an error in validation script, which
>> does not expected the hyphen and the dot. Other commond symbols like:
>>"," and
>> "=" are added to prevent future errors.
>
>Lets not add , and = since they're typically not part of names.  OK with
>the regex only adding a hyphen and dot.

Now I remember why I added “=" and “,: There is another test with these
symbolism its parameters:

Invalid benchmark output: 'stack=1024,guard=1', 'stack=1024,guard=2',
'stack=2048,guard=1', 'stack=2048,guard=2' do not match any of the
regexes: '^[_a-zA-Z0-9.-]*$'
make[1]: *** [Makefile:215: bench-func] Error 65


Full log: https://hastebin.com/menofisuki.pl

From : BENCHSET="bench-pthread bench-math malloc-thread”

Should I add , and = to the regex or fix the name of the test?

Thanks 

Victor 

>
>Thanks,
>Siddhesh
>
>> 
>> 	(VERSION): Set to 2.27
>>         * benchtests/scripts/benchout.schema.json: Fix regex to accept a
>>         wider range of tests names.
>> 
>> Signed-off-by: Victor Rodriguez <victor.rodriguez.bahena@intel.com>
>> ---
>>  ChangeLog                               | 6 ++++++
>>  benchtests/scripts/benchout.schema.json | 2 +-
>>  2 files changed, 7 insertions(+), 1 deletion(-)
>> 
>> diff --git a/ChangeLog b/ChangeLog
>> index 8dbfc7e..f29d180 100644
>> --- a/ChangeLog
>> +++ b/ChangeLog
>> @@ -1,3 +1,9 @@
>> +2017-11-15  Victor Rodriguez  <victor.rodriguez.bahena@intel.com>
>> +
>> +	(VERSION): Set to 2.27
>> +	* benchtests/scripts/benchout.schema.json: Fix regex to accept a
>>wider range
>> +	of tests names.
>> +
>>  2017-08-02  Siddhesh Poyarekar  <siddhesh@sourceware.org>
>>  
>>  	* version.h (RELEASE): Set to "stable"
>> diff --git a/benchtests/scripts/benchout.schema.json
>>b/benchtests/scripts/benchout.schema.json
>> index affb7c1..815ad97 100644
>> --- a/benchtests/scripts/benchout.schema.json
>> +++ b/benchtests/scripts/benchout.schema.json
>> @@ -13,7 +13,7 @@
>>            "title": "Function names",
>>            "type": "object",
>>            "patternProperties": {
>> -            "^[_a-zA-Z0-9]*$": {
>> +            "^[_a-zA-Z0-9,=.-]*$": {
>>                "title": "Function variants",
>>                "type": "object",
>>                "properties": {
>>