This is the mail archive of the
mailing list for the Mauve project.
Re: [Q] Number of unit tests in Mauve and of assertions in Classpath?
- From: Markus Gälli <markus dot gaelli at iam dot unibe dot ch>
- To: Sascha Brawer <brawer at dandelis dot ch>
- Cc: Stephen Crawley <crawley at dstc dot edu dot au>,<mauve-discuss at sources dot redhat dot com>
- Date: Thu, 11 Mar 2004 11:51:48 +0100
- Subject: Re: [Q] Number of unit tests in Mauve and of assertions in Classpath?
Right. In my experience many tests are on the same level also. There
you basically don't
roughly 145 [Mauve tests] are currently failing.
I guess this 145 are not independent, meaning, if you fixed one, many
others would be fixed too. Right?
In some cases, there exist multiple tests for the same functionality.
example the tests for solving quadratic and cubic equations, where the
testee is called with a bunch of somewhat random test equations. You'll
have the same situation with any tests that are feeding random input
into a function and check whether the result is as expected. Any
would be sort of mutually dependent: As soon as you fix any of these
failures, the other failures are likely to disappear immediately
it's usually the same code that is failing). Stated in more abstract
terms, the "fixing order" is only a partial, not a total ordering on
care with which you begin, as any exception will do. I think that you
can get more
out of the ones which form an inclusion hierarchy of coverage sets
that one test is concerned with a more abstract view than the other but
running the same inner parts.
That was what I meant. I hope I am not that sloppy in my paper... ;-)
I am just writing a paper stating that it makes sense to sort failing
tests by size of covered methods beginning with the smallest.
Interesting... Why would this be an improvement -- because you assume
that smaller methods get invoked by larger ones, and the smaller
would be the specific point of failure? In that case, couldn't you just
sort the failed tests based on a (post-order) traversal of the observed
call graph? Or, this may be easier to obtain, based on the total number
of instructions/bytecodes executed by the testee and its callees?
It should have been: "The size of the sets of covered method
Well, IYou can read an old version here:
guess the reason for sorting by method size will be stated in your
Would you mind posting a link to your paper on this list when you've
published it? Thanks in advance!
Will hopefully publish a published version Real Soon Now (TM)
Difficult to introduce that to Object? Could be made painless (in
Smalltalk at least)
My motivation for asking here, was that if you had assertions (which I
surely understand that you don't have), more of the failing
tests would fail at the most specific assertion, making my sorting a
bit more useless.
This seems like a reasonable assumption, but I wouldn't know how to
But hey, you just gave me a wonderful argument: In some big and
important projects, assertions are even not an option!
Well, I guess we will eventually use assertions in Classpath. There's
law saying we must not use 1.4 language features. The reason why we
haven't been using assertions in the past is the lack of support by
Java compilers. Which would be easy enough to fix (a quick-and-dirty
would be just to ignore the assert statement).
Thanks a lot.
Mauve simply takes a large text file that lists the names of test
classes, loads the respective class (which must implement the
gnu.testlet.Testlet interface) and invokes its "test" method. So the
"sorting" is just the ordering in the text file. If you sorted this
in any particular way, the output (= failing tests) would be sorted in
the same order.
- Do you know any (as responsive :-) java open source community with
some big project and which
uses both, JDK 1.4 (thus theoretically assertions) and unit tests?
I guess most JUnit users would fall into that category, but I
unfortunately don't know any specific examples.