This is the mail archive of the mauve-discuss@sourceware.cygnus.com mailing list for the Mauve project. See the Mauve home page for more information.


[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index] [Subject Index] [Author Index] [Thread Index]

Re: Project Mauve: A Free Java Regression Test and Compatibility Package



>>>>> "Godmar" == Godmar Back <gback@cs.utah.edu> writes:

README> You'll notice that each test has a public constructor.

Godmar> That wouldn't allow me to test JNI compatibility, plus it
Godmar> would allow me to run any of the tests that deal the
Godmar> complexities of the threading and initialization parts.  (For
Godmar> instance: what happens if you get an
Godmar> ExceptionInInitializerError in the class whose main() method
Godmar> is invoked, etc.)

I'm afraid I don't understand either of these objections.

What is it about this approach that makes it impossible to test JNI
compatibility?

For the second point: are you concerned about testing for exceptions
thrown in `main' in particular?  Or do you want to be able to test
constructors that cause ExceptionInInitializerError to be thrown?  You
can certainly test the latter; you just need two classes.

Can you give me an actual example of a test you can't write in this
framework?  I mean, based on one of the above things?  I know there
are tests you can't write -- for instance, you can't test that a
running daemon thread won't prevent the test program from exiting.
Also, you can't test finalization on exit.

I guess that's too bad.  I personally don't like it, but there it is
anyway.  Some of the untestable things really approach being tests of
the runtime and not the library per se.  I don't know if these really
fall within mauve's remit.

Or maybe you're asking about what happens if there is a bug in the
test case's constructor.  All the existing constructors are empty.
That seems like a wise course to take in general.  So I'd say this
isn't an important consideration.

Godmar> I would prefer something that would simply diff the expected
Godmar> against the actual output, and voila.

I'm with you there, but most everybody else was against this.

Anyway, the plan is to add some code to the harness to let you do this
in a certain way -- you'll be able to embed the expected output in the
test case, and the harness will do the comparison.  This isn't written
yet.

Godmar> It would also be nice if it didn't require anything but
Godmar> /bin/sh to run.

I believe we currently require GNU make, sh, a Java runtime, and a
Java compiler.  It would probably be possible to get rid of the GNU
make dependency.  I'm not particularly motivated to do it.

Godmar> On the other hand, I would like it if I could use it to run a
Godmar> group of tests or single tests quickly.

You can do this.  Feed the class list you care about into the test
harness' stdin.

Note also that SimpleTestHarness is just a simple, generic test
harness.  You can write a more complicated one if you have different
needs.  We're going to write another harness in-house so we can do
DejaGNU-based embedded system testing (though we might use the simple
harness for ordinary testing)

Godmar> That is not to say that test using Class.forName etc. would be
Godmar> useless: in fact, they would probably test durability if the
Godmar> engine isn't shut down between tests.

Right now we run all the tests in a single harness invocation.
However, this isn't really fixed.  There is some info in README about
an idea to change this (grouping).  If you want, you could run a
separate invocation of the harness for each test.

Note that being able to run big batches of tests all at once is nice
for us gcj developers.  That's because we have to relink the test
harness when we compile.  If we put a main() into every test, then
we'd at least double the time it takes to run the test suite.

Tom