This is the mail archive of the mailing list for the Mauve project.

Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Updating Mauve tags

The Sun's swing did have as many as 6514 bugs through the history, despite many of them are fixed
now. Together with the new features and improvements, each new major release inevitably brings
some regressions that are later fixed. J2SE 5.0 <> already had six updates in the past. They do make some
mistakes, same as we do, same as any java development group inevitably does. This may be especially
true for tests that were succeeding in the past but fail with the newest version.

Because of that reason I would not suggest any automated removal or inactivation of the tests just
because they fail on Sun's or any other java implementation. If the implemented behavior clearly
mismatches the Sun's API specification, this is probably just a bug that is likely to be fixed - probably in the
next minor release.

Some tests can be invalid and can be inactivated or removed (probably better altered, making them valid).
However I think that this work (a difficult work) must be done manually. The typical reason can be if we find
the similar problem in the Sun's bug reports and the Sun states that this is not a bug or it will never be fixed,
or if we realize that the former author of the test clearly misinterprets the Sun's API standard.

If needed, we could probably simply have and use the list of tests that are known to pass with the final
releases of the JDK 1.2, 1.3 and 1.4 (for the 1.5, such list would need the regular updates).


Roman Kennke wrote:

Hi Mark,

I see that we have a concept of tags in Mauve. That is a collection of
keys at the top of each test class. This way we can filter the tests.
ATM we have tags for the JDK versions like JDK1.4 JDK1.3 and so on and a
couple of other tags. However, it seems that they are not maintained in
a usable way, so most people simply include every tag that they can
think of (that is what's done in batch_run for example) to run all

Why do you feel they aren't maintained in a usable way?

This was caused by a misunderstanding of the usage/meaning of those tags. I was thinking that when a test has the tag JDK1.x, that this test is meant to PASS under a JDK1.x-ish JDK. As Michael and others have pointed out on IRC this is not the case. If I want to test a JDK1.3-sh (for example) environment I should include JDK1.0 JDK1.1 JDK1.2 and JDK1.3 tags in my keys.

The problem that I am seeing is when a test that is written to PASS
under 1.4 fails under 1.5. There are lots of those tests in the
testsuite for the javax.swing package. So my plan would have been to tag
all tests that pass under JDK1.5 with the 1.5 tag and those that don't
only with JDK1.4 or whatever is ok. Since the tags are not meant to be
used that way, maybe we can do it different. Could we extend the
choose-classes script to detect !JDK1.x tags in the tag header of java
source files and don't include the test in a JDK1.x test run?


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]