Re: Recent simulator patches broke many sims

[Joel Sherrill <joel dot sherrill at oarcorp dot com>]

On 3/24/2013 6:23 PM, Hans-Peter Nilsson wrote:
> > From: Joel Sherrill <joel.sherrill@oarcorp.com>
> > Date: Sun, 24 Mar 2013 15:45:27 +0100
> > This came in after I was done email last night. My test
> > run finished overnight with no horribly bad issues. I have no idea
> > what the make check results should be though and they could be
> > because I simply ran "make check" with no board specified and
> > no gcc for the target installed.
> This would be no news to *you*, but for the record:
It's not a surprise. I fairly regularly run the gcc tests for RTEMS
and they need a baseboard specified.  I knew binutils didn't. And
assumed (incorrectly) that applied to the simulator and gdb
as well.
> You need a board (make check RUNTESTFLAGS=--target_board=$board
> with e.g. board=cris-sim).  All boards are in "recent"
> dejagnu-1.5 IIRC and most in ancient dejagnu-1.4.4.  You need
> installed binutils (e.g. in some temp location added to PATH for
> the duration of the test-run) for each sim configuration as
> mentioned.  I don't run with target gcc; not needed for the
> level of smoke test I'm after and I guess not for this change
> either.
I will add this to my testing script for RTEMS targets. Is there somewhere
to email these results like gcc?
> > There were some unexpected failures but I don't know what "truth"
> > on that is.
> The truth is in the baseline.
Using 7.5.91 for that.  Just got home and the build for that is done.
> >   I have saved logs from the test run and will do so with
> > the run on Mike's patch.
> >
> > Starting another test run with Mike's patch.
> >
> > I can put both sets of logs on an ftp server if someone wants to review
> > them.  Guidance on what is a bad number of "unexpected failures" on
> > a target is appreciated.
> You compare against a baseline you create without your patches;
> comparing old/new build logs should be sufficient.
>
> Having said all that, I think just a successful *build* of each
> sim would be sufficient here, if your grep can find no
> sockser-specific tests in the target sim test-suites.
OK. I will follow up with a test results email.
> brgds, H-P


--
Joel Sherrill, Ph.D.             Director of Research & Development
joel.sherrill@OARcorp.com        On-Line Applications Research
Ask me about RTEMS: a free RTOS  Huntsville AL 35805
Support Available                (256) 722-9985