Re: [ECOS] test case programs

[Jonathan Larmour <jlarmour at redhat dot com>]

Andrew Lunn wrote:
> 
> Hi Folks
> 
> Is there any documentation about writing test cases and about the test
> farm RedHat has?

Alas no for both.

> I've seen the include file testcase.h which gives some clues, but more
> information would be good. Does it matter if a test program produces
> other output than using the macros defined there.

No, just that the output of those macros is treated specially.

> How is the output
> parsed. 

It simply looks for the strings at the start of the line, e.g. "PASS:" etc.

> Is there a time limit for test cases. 

Yes, there are two, one activity timer which gets reset each time there is
program output, and one overall timeout after which the test is always
killed. The values are target dependent, due to different target speeds.

> Are there Linux and NT
> machines on the network that could be used for peers in network
> tests.

Yes *but* if at all possible such dependencies should be avoided so that
testcases can be run by J. Random User too. Any tests with such
dependencies should be default off. The net stack tests are an example of
this.

> Is the synth target also in the test farm since its a bit
> different from the rest.

Frequently yes, although not always depending on test farm load. We divert
test resources according to demands. But in general we try to run as much
as possible of course.

> How does the test farm decide what to
> test.

It selects a test configuration permutation (a potted eCos configuration)
from a repository, builds it and runs all the tests resulting.

> Is there any way for a CDL writer to influence what options it
> turns on/off when producing configurations for tests? 

Unfortunately not. Our test perms are internal (since they are only usable
by us). However the format is not difficult, e.g. here's one:

#
# A configuration with ROM and RAM filesystems enabled, and a few
# other bits and pieces for fun.

pkgconf -disable-uitron -disable-libm -enable-posix -enable-fileio
-enable-romfs -enable-ramfs

header libc.h {
    disable CYGFUN_LIBC_strtod
    disable CYGSEM_LIBC_STDIO_PRINTF_FLOATING_POINT
    disable CYGSEM_LIBC_STDIO_SCANF_FLOATING_POINT
}

header romfs.h {
    enable CYGTST_ROMFS_BUILD_TESTS
}

header infra.h {
    enable CYGPKG_INFRA_DEBUG
}

header kernel.h {
    enable CYGFUN_KERNEL_ALL_THREADS_STACK_CHECKING
}

If you included such a file in your patch, it would arrive in the test
farm!

> Would there be
> anyway for some of us mere net uses to get access to the test
> farm.

That's something we want to do, but a lot of software and changes (e.g. for
security!) needs to happen first.

Jifl
-- 
Red Hat, Rustat House, Clifton Road, Cambridge, UK. Tel: +44 (1223) 271062
Maybe this world is another planet's Hell -Aldous Huxley || Opinions==mine