On 10/10/2017 07:51 PM, Simon Marchi wrote:
The get_integer_valueof outputs the value it has read as part of the
test name. This causes test names to vary from run to run, and adds
some noise when diffing test results. e.g.:
-PASS: gdb.threads/process-dies-while-detaching.exp: multi-process:
continue: killed outside: get integer valueof "mypid" (28770)
+PASS: gdb.threads/process-dies-while-detaching.exp: multi-process:
continue: killed outside: get integer valueof "mypid" (32238)
Ideally, test result diffing tools would be ignoring
" (foo)" at the end test names, because that part is
not considered a part of the test name itself. It's
extra info. That's where we write "(timeout)", "(eof)",
etc. So for instance:
-PASS: gdb.threads/process-dies-while-detaching.exp: multi-process:
continue: killed outside: get integer valueof "mypid"
+FAIL: gdb.threads/process-dies-while-detaching.exp: multi-process:
continue: killed outside: get integer valueof "mypid" (timeout)
... this should be considered a regression, instead of a new FAIL and
a missing PASS.
I think the buildbot diffing script is correctly ignoring that
tail end "(foo)"s.
That said, I think it's still a good idea to make the gdb.sum output
more stable across passing runs, so that plain /usr/bin/diff also
works reasonable well.