This is the mail archive of the mailing list for the Mauve project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: New Harness, version 1

Anthony Balkissoon wrote:

Okay, the new Harness is checked in, please use it and comment on
features you would like added.

Running tests is now much easier, please use the harness script in the
top folder.  ./harness -help will print a help message, and the README
file gives more detailed instructions, but it is very intuitive.  Simply
specify the tests or folders you want to run.

"./harness javax.swing" will run all the tests in

It is important to specify the VM you wish to use to run the tests.  If
you always (or almost always) run the Mauve tests using the same VM, you
should export the environment variable MAUVEVM.  For example, I have
MAUVEVM=jamvm, so the above command would run all the javax.swing tests
on JamVM.  Alternately, you can use the -vm option (say, if you want to
run against Sun to make sure the tests pass, or just for comparison):

"./harness -vm PATH-TO-EXECUTABLE" javax.swing"

This will override the MAUVEVM variable.  If neither of these options is
used, the tests will be run on the system "java" VM, whatever you have
that set to.

Comments are greatly appreciated.

Hi Anthony,

I gave this a quick run - it's very nice! I really like how easy you've made it to run all the tests for one class or package.

I have a few comments:

- is there an easy way to run all the tests? Maybe something like "./harness gnu.testlet -vm jamvm"? Perhaps I missed something obvious, but if this is possible then you could add it to the README.

- compiling the code manually was straightforward, but perhaps it could be done in one step with "make" or "ant", just to make it as simple as possible?

- this runner ignores the tags in the tests (JDK1.0, JDK1.1, GUI etc.) unless you use the -file option. Do you have any plans to add support for the flags? I don't know how much it matters, as I get the feeling that the tags are not used a lot.

It would be nice to generate some HTML output for easy casual review, along the lines of:

Even better than that, though, would be to get some test coverage tool working and have that generate some HTML reports that we can use to monitor our progress...



Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]