revamped gdb_mbuild.sh

[Andrew Cagney <ac131313 at redhat dot com>]

Richard,

Finally got to trying the gdb_mbuild.sh script and, in the process, 
integrated some, er, `new features' from my old local script:

The changes are:


- restartable

Instead of starting over from scratch, it tries to continue on from 
where the last run left off (after being CNTRL-Ced or failing).  To 
force a complete rebuild of each target, use the option:

`-f' (force) will force a new run


- stop on failure

The script, by default, aborts on the first failed build.  The 
assumption is that the problem will be fixed and then the script 
re-started.  This can be overriden with the option:

`-k' (keep going) will force the script to try and build everything 
possible (the flag is passed to make as well).


- parallel

Rather than having separate configure and build phases (waiting on all 
the configures to finish before starting the builds), this script 
directly starts the parallel sub-shells.  Each sub-shell then tries to 
do a full configure, build and run for its target.  The number of jobs 
is specified with the option:

`-j <maxjobs>' controls the number of parallel jobs

(the implementation isn't perfect mind, it misses a second job finishing 
before a first).

- testing

In addition to checking the configure and build, the script also tries 
to run the built GDB to the point where it checks that '(gdb) maint 
print architecture' succeeds.

(NB: The h8500hms target currently barfs when run, I think I might 
propose that it be booted out of the build list).


- selecting targets

A sub-set of targets can be selected using the option:

`-e <targexp>' Select a subset of the targets using the (grep) regular 
expression <targexp>.  Multiple expressions can be specified.


- the output was changed vis:
mn10300-elf ,-Werror ...
ns32k-netbsd ,-Werror ...
... configuring mn10300-elf
... configuring ns32k-netbsd
... compiling ns32k-netbsd
... compiling mn10300-elf
... running ns32k-netbsd
... ns32k-netbsd built
... running mn10300-elf
... mn10300-elf built

Comments (for instance on the choice of options),

Andrew

#!/bin/sh

#  Multi-build script for testing compilation of all maintained configs of GDB.
#  Copyright (C) 2002  Free Software Foundation, Inc.
#  Contributed by Richard Earnshaw  (rearnsha@arm.com)

#  This program is free software; you can redistribute it and/or modify
#  it under the terms of the GNU General Public License as published by
#  the Free Software Foundation; either version 2 of the License, or
#  (at your option) any later version.

#  This program is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#  GNU General Public License for more details.

#  You should have received a copy of the GNU General Public License
#  along with this program; if not, write to the Free Software
#  Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA

usage() {
    cat <<EOF
Usage: gdb_mbuild.sh [ <options> ... ] <srcdir> <builddir>
 Options:
  -j <maxjobs>  Run <maxjobs> in parallel.  On a single cpu machine,
                2 is recommended.
  -k            Keep going.  Do not stop after the first build fails.
  -e <regexp>   Regular expression for selecting the targets to build.
  -f            Force rebuild.  Even rebuild previously built directories.
 Environment variables examined (with default if not defined):
  MAKE (make)"
 Note: Everything in <builddir>/gdb-allcross will be blown away.
EOF
    exit 1;
}

### COMMAND LINE OPTIONS

maxjobs=1
keepgoing=false
force=false
targexp=""
while test $# -gt 0
do
    case "$1" in
    -j )
	# Number of parallel make jobs (you probably want about 2 jobs
	# per cpu for maximum throughput)
	shift
	test $# -ge 1 || usage
	maxjobs=$1
	;;
    -k )
	# Should we soldier on after the first build fails?
	keepgoing=true
	;;
    -e )
	# A regular expression for selecting targets
	shift
	test $# -ge 1 || usage
	targexp="${targexp} -e ${1}"
	;;
    -f )
	# Force a rebuild
	force=true ; shift ;;
    -* ) usage ;;
    *) break ;;
    esac
    shift
done


### COMMAND LINE PARAMETERS

if test $# -ne 2
then
    usage
fi

# Convert these to absolute directory paths.

# Where the sources live
srcdir=`cd $1 && /bin/pwd` || exit 1

# Where the builds occur
builddir=`cd $2 && /bin/pwd` || exit 1


### ENVIRONMENT PARAMETERS

# Version of make to use
make=${MAKE:-make}
MAKE=${make}
export MAKE


# Where to look for the list of targets to test
maintainers=${srcdir}/gdb/MAINTAINERS
if [ ! -r ${maintainers} ]
then
    echo Maintainers file ${maintainers} not found
    exit 1
fi

# Get the list of targets and the build options
alltarg=`cat ${maintainers} | tr -s '[\t]' '[ ]' | sed -n '
/^[ ]*[-a-z0-9\.]*[ ]*[(]*--target=.*/ !d
s/^.*--target=//
s/).*$//
h
:loop
  g
  /^[^ ]*,/ !b end
  s/,[^ ]*//
  p
  g
  s/^[^,]*,//
  h
b loop
:end
p
' | if test "${targexp}" = ""
then
    grep -v -e broken -e OBSOLETE
else
    grep ${targexp}
fi`


# Run several jobs in parallel.
set --


# Usage: fail <message> <test-that-should-succeed>.  Should the build
# fail?  If the test is true, and we don't want to keep going, print
# the message and shoot everything in sight and abort the build.

fail ()
{
    msg="$1" ; shift
    if test "$@"
    then
	echo "${target}: ${msg}"
	if ${keepgoing}
	then
	    exit 0
	else
	    kill $$
	    exit 1
	fi
    fi
}


# Warn the user of what is comming, print the list of targets
echo "$alltarg"
echo ""


# For each target, configure and build it.
echo "$alltarg" | while true
do

    # Keep forking jobs until we've exceed our job capacity

    while test $# -lt ${maxjobs} && read target gdbopts simopts
    do

	(
	    trap "exit 1"  1 2 15
	    dir=${builddir}/${target}

	    # Should a scratch rebuild be forced?
	    if ${force}
	    then
		rm -rf ${dir}
	    fi
	    
	    # Don't bother re-building a built target (indicated by
	    # ${dir} being a file).
	    if test -f ${dir}
	    then
		#echo "${target}: already built"
		exit 0
	    fi

	    echo ${target} ${gdbopts} ${simopts} ...

	    # The config options
	    __target="--target=${target}"
	    __enable_gdb_warnings=`test -z "${gdbopts}" \
		|| echo "--enable-gdb-warnings=${gdbopts}"`
	    __enable_sim_warnings=`test -z "${simopts}" \
		|| echo "--enable-sim-warnings=${simopts}"`

	    # Did the previous configure attempt fail?  If it did
	    # restart from scratch.
	    if test ! -r ${dir}/gdb/Makefile
	    then
		rm -rf ${dir}
		mkdir -p ${dir}
	    fi

	    # From now on, we're in this target's build directory
	    cd ${dir} || exit 1

	    # Configure, if not already.
	    if test ! -r gdb/Makefile
	    then
		echo "... configuring ${target}"
		trap 'echo Removing partially configured ${target} directory ...; rm -rf ${dir}; exit 1' 1 2 15
		${srcdir}/configure \
		    ${__target} \
		    ${__enable_gdb_warnings} \
		    ${__enable_sim_warnings} \
		> Config.log 2>&1
		trap "exit 1"  1 2 15
	    fi
	    fail "configure failed" ! -r gdb/Makefile

	    # Build, if not built.
	    if test ! -x gdb/gdb -a ! -x gdb/gdb.exe
	    then
		echo "... compiling ${target}"
		${make} all-gdb > Build.log 2>&1
	    fi
	    fail "compile failed" ! -x gdb/gdb -a ! -x gdb/gdb.exe
	
	    # Check that the built GDB can at least print it's architecture.
	    echo "... running ${target}"
	    rm -f core gdb.core ${dir}/gdb/x
	    cat <<EOF > x
maint print architecture
quit
EOF
	    ./gdb/gdb -batch -nx -x x > Arch.log 2>&1 < /dev/null
	    fail "gdb dumped core" -r core -o -r gdb.core
	    fail "gdb printed no output" ! -s Arch.log
	    grep -e internal-error Arch.log && fail "gdb panic" 1

	    # Replace the build directory with a file as semaphore
	    # that stops a rebuild. (should the logs be saved?)
	    cd ${builddir}
	    rm -f ${target}.tmp
	    mv ${target}/Arch.log ${target}.tmp
	    rm -rf ${target}
	    mv ${target}.tmp ${target}

	    # Success!
	    echo "... ${target} built"

	) &

	# Append this to the list of things to wait for.  Grr, there
	# is a race condition as the cntrl-c could come in before the
	# trap is updated.
	trap "echo Killing jobs $@ $!; kill $@ $!; wait ; exit 1" 1 2 15
	set -- "$@" $!

    done

    # If we've got zero jobs, we're finished
    if test $# -eq 0
    then
	break
    fi

    # Reap the first running process, then starting again
    wait $1 ; shift

done

exit 0