Re: Two more AI bugs

[Stan Shebs <shebs at apple dot com>]

Jim Kingdon wrote:
> 
> > Fixing this crash is actually the first item in the TODO list that
> > Stan left for the next release. However, running all.g causes a hard
> > freeze under MacOS so I haven't got around to debugging it yet.
> 
> OK, I've checked in a fix to change this from a core dump to a
> run_error; that should be sufficient.
> 
> Now, as to where we stand on tests in general, what we have now seems
> like it is pretty hard to use.  A few of the problems are:

Hey!  You're criticizing my code! :-)

> * They are way too slow to run them with any frequency.  Some of this
>   would be easy to fix (for example, I see little reason for
>   test-acts.sh to run for every game in lib).

You could do some pruning, although you'd have to be careful not
to miss the oddballs.  test-acts.sh for instance will find cases
where obscure GDL doesn't play nice with particular actions.

> * I don't see any mechanism for comparing the result of a test with
>   some kind of expected result.  This is pretty crucial, because
>   unless you get a pass/fail it is hard to run the tests frequently
>   and know what they are telling you.

Most of the tests are just looking for crashers.  The save/restore
tests diff multiple saves and expect them to be (nearly) identical.
It would be spiffy to have the AI tests return pass/fail ("crossed
the strait in less than 20 turns") - mostly I just watched them,
not very efficient I suppose.

> * I'd also probably prefer tests which link into xconq.  It makes it
>   easier to test internal APIs and test one module at a time.
> 
> The
> http://cppunit.sourceforge.net/cppunit_cookbook.html#cppunit_cookbook
> page is an introduction to the style of unit test which I'm thinking
> of.

I looked at it, but dunno.  Most of Xconq's components aren't
really that complicated (at least if you only count the code that
is actually used by real games :-) ), and many of the interesting
algorithms, such as terrain generation or contour line tracing, tend
to have to be evaluated visually.  I've always found the interactions
to be the hardest to test - the task-level machinery usually works
OK in isolation, and it's the AI's randomly-chosen bad handling of a
task completion that causes problems.

On the other hand, the sole purpose of the "randomly" action argument
in testacts.inp is to run linked-in code to randomize data going
into actions...

Stan