This is the mail archive of the
mailing list for the Mauve project.
Re: Mauve wishlist
>>>>> "David" == David Gilbert <email@example.com> writes:
>> Another suggestion that Tom Fitzsimmons had was to change the way we
>> count the number of tests. Counting each invocation of the test()
>> method rather than each call to harness.check() has two benefits:
David> We can work around that by ensuring that all the tests run linearly
David> (no if-else branches - I've written a large number of tests this way
David> and not found it to be a limitation, but I don't know what lurks in
David> the depths of the older Mauve tests). There is still the problem that
David> an exception being thrown during a test means some checks don't get
David> run, but a new Mauve comparison report (not yet developed, although
David> I've done a little experimenting with it) could highlight those.
I've always tried to write tests the way you suggest, but the
exception problem turns out to be a real one, preventing test
stability in some cases.
One thing I like about this current proposal is that it automates test
stability -- the only failure modes possible are if a test hangs or if
the VM crashes.
As far as having more granular information -- we can still print a
message when a check() fails. A command line option to the test
harness could control this, for instance. I think we don't want to
just print a plain 'FAIL', we want some explanation; the detailed info
could go there.