This is the mail archive of the
systemtap@sourceware.org
mailing list for the systemtap project.
Re: testsuite and hardcoded timeouts
David Wilder wrote:
I ran into this issue on s390. When a time out occurs if the test
would simply produce a warning message then restarts the timer, allowing
the timeout to be restarted say 4 or 5 times before finally reporting a
failure. Then if something breaks the test will still report a
failure. On slower system the test would still pass. If a system/test
normally passes with one or two restarts of the timer then something
changes and it starts taking 3 or 4 restarts we will know that
investigation is needed.
You might luck out with the caching helping the later attempts skip some of the
phases of the translator and avoid those times on the later runs. However,
restarting 4 or 5 times is probably not going to help that much if the time
required to generate the module is way larger than the time out.
The timeout is there to make sure that forward progress is made on the testing.
We would prefer to have the test fail in a reasonable amount of time than to
have a test hang for an unreasonable amount of time and not get any results at
all. The translator internals are pretty much a black box to the testing
harness, so the timer is used to judge when the the test isn't making forward
progress. Too bad there couldn't be an equivalent to a watchdog for the testing
harness, e.g. if the test is making forward progress, leave the test be.
-Will