This is the mail archive of the
gdb-patches@sourceware.org
mailing list for the GDB project.
Re: relying on testsuite results
- From: Tom Tromey <tromey at redhat dot com>
- To: Thiago Jung Bauermann <bauerman at br dot ibm dot com>
- Cc: Paul Pluzhnikov <ppluzhnikov at google dot com>, gdb-patches at sourceware dot org
- Date: Mon, 06 Apr 2009 13:50:48 -0600
- Subject: Re: relying on testsuite results
- References: <200901121846.51709.pedro@codesourcery.com> <200902062335.17737.pedro@codesourcery.com> <8ac60eac0902082223q2192830cu8b75f6424fca6c68@mail.gmail.com> <200902092216.54762.pedro@codesourcery.com> <8ac60eac0904061159v4deb9d48n1cf791463e587e54@mail.gmail.com> <1239045480.8871.11.camel@localhost.localdomain>
- Reply-to: tromey at redhat dot com
>>>>> "Thiago" == Thiago Jung Bauermann <bauerman@br.ibm.com> writes:
Thiago> So the most use I get from the testsuite is to run regression
Thiago> tests on each patch I submit, and tediously eyeball the diff
Thiago> looking to see if any of the PASS<->FAIL flips actuallly mean
Thiago> something.
Yeah, me too.
These non-deterministic tests may add something, in one scenario: if
they start failing consistently, and somebody notices that, then they
may provide information about a real regression.
I'm sure I won't notice if they start failing consistently, though.
I already ignore them.
I generally do full regression tests for each patch. ("Generally"
because I occasionally forget, or something goes wrong and I don't
notice.) This is nice, although slow, and unfortunately also open to
some kinds of major failure (e.g., a change of compiler negatively
affecting baseline results).
Tom