Re: Harness patch to allow code coverage

[Thomas Fitzsimmons <fitzsim at redhat dot com>]

David Gilbert wrote:
> Anthony Balkissoon wrote:
>
> > Hi David,
> >
> > yes, it's working now, I have a report viewable internally (ie - Red
> > Hat).  I have no public space so I couldn't put them up anywhere else.
> > They will be up on builder.classpath.org soon, Tom Fitzsimmons is now
> > working on that. He also wants to have a combined Japi/code coverage
> > report so you can link to the source code right from the Japi scores and
> > see which lines are being tested.  This is all coming soon.
> >
> > --Tony
> >
> >  
> Great, I'm looking forward to that.  Another thing that would be great 
> is just a set of results for Mauve generated nightly.  From time to 
> time, I run this one (that's linked from the GNU Classpath home page):
>
> http://www.object-refinery.com/classpath/mauve/report/
>
> ...but the code that generates this is quite flaky...a replacement would 
> be very welcome!

Yes, we need a nightly script to generate Mauve results.  But the report page 
should be more detailed, laid out like this:

Package:	Total:	 	Passed: 	Failed:
java.awt 	1284		1270		14

And the Failed number should be a link to a list of the specific failing tests, 
with the failure messages produced by the new harness:

Package: java.awt

FAIL: gnu.testlet.java.awt.Frame.size1
  line 54: got 29 but expected 53
  line 45: boolean passed to check was false
  line 45: boolean passed to check was false
  line 45: boolean passed to check was false

That would tell people which tests to work on.

We also need a percentage bar beside each package in the JAPI output, indicating 
the number passing and failing tests.

Tom