Re: GAS

[Nick Clifton <nickc at redhat dot com>]

Hi Grigory,

> Similar issue was reported against 'ld' 1.5 years ago. Simple change of
> default hash size from 4K to 64K gave 2x link time improvement.
> http://lists.gnu.org/archive/html/bug-gnu-utils/2003-08/msg00153.html
>
> Thus the patch may look like this:
>
> $ diff gas/hash.c.org gas/hash.c
>
> 38c38
> < #define DEFAULT_SIZE (4051)
> ---
> > #define DEFAULT_SIZE (65537)

We might as well do this properly.  Please could you try the attached 
patch which as well as changing the default hash size adds two new 
switches: --hash-size=<NUMBER> and --reduce-memory-overheads which 
behave in much the same way as their linker namesakes.

If the patch works for you and there are no objections then I will apply 
it next week.

Cheers
  Nick

gas/ChangeLog

2005-04-06  Nick Clifton  <nickc@redhat.com>

	* hash.c (DEFAULT_SIZE): Delete.  Replace with:
	(gas_hash_table_size): New static variable.
	(set_gas_hash_table_size): New function:  Records a requested size
	for the hash tables.
	(get_gas_hash_table_size): New function: Return a prime number
	near the requested size of the hash table.
	(hash_new): Use get_gas_hash_table_size.
	* hash.h: Add a prototype for set_gas_hash_table_size.
	* as.c (show_usage): Add description of new switches: --hash-size
	and --reduce-memory-overheads.
	(option_values): Add OPTION_HASH_TABLE_SIZE and
	OPTION_REDUCE_MEMORY_OVERHEADS.
	(std_longpopts): Add entries for the new options.
	(parse_args): Handle the new options.
	* Makefile.am: Add a dependency of as.c on hash.h.
	* Makefile.in: Regenerate.
	* doc/as.texinfo: Document the new switches.
	* NEWS: Mention the new switches.

Index: gas/Makefile.am
===================================================================
RCS file: /cvs/src/src/gas/Makefile.am,v
retrieving revision 1.104
diff -c -3 -p -r1.104 Makefile.am
*** gas/Makefile.am	21 Mar 2005 15:35:30 -0000	1.104
--- gas/Makefile.am	6 Apr 2005 11:06:00 -0000
*************** BMKDEP = #DO NOT PUT ANYTHING BETWEEN TH
*** 2436,2442 ****
  #MKDEP    DO NOT PUT ANYTHING BETWEEN THIS LINE AND THE MATCHING WARNING BELOW.
  app.o: app.c $(INCDIR)/symcat.h
  as.o: as.c $(INCDIR)/symcat.h subsegs.h $(INCDIR)/obstack.h \
!   output-file.h sb.h macro.h dwarf2dbg.h dw2gencfi.h \
    $(INCDIR)/elf/dwarf2.h $(BFDVER_H)
  atof-generic.o: atof-generic.c $(INCDIR)/symcat.h $(INCDIR)/safe-ctype.h
  bignum-copy.o: bignum-copy.c $(INCDIR)/symcat.h
--- 2436,2442 ----
  #MKDEP    DO NOT PUT ANYTHING BETWEEN THIS LINE AND THE MATCHING WARNING BELOW.
  app.o: app.c $(INCDIR)/symcat.h
  as.o: as.c $(INCDIR)/symcat.h subsegs.h $(INCDIR)/obstack.h \
!   output-file.h sb.h macro.h dwarf2dbg.h dw2gencfi.h hash.h \
    $(INCDIR)/elf/dwarf2.h $(BFDVER_H)
  atof-generic.o: atof-generic.c $(INCDIR)/symcat.h $(INCDIR)/safe-ctype.h
  bignum-copy.o: bignum-copy.c $(INCDIR)/symcat.h

Index: gas/NEWS
===================================================================
RCS file: /cvs/src/src/gas/NEWS,v
retrieving revision 1.74
diff -c -3 -p -r1.74 NEWS
*** gas/NEWS	1 Apr 2005 14:17:16 -0000	1.74
--- gas/NEWS	6 Apr 2005 11:06:01 -0000
***************
*** 1,5 ****
--- 1,9 ----
  -*- text -*-
  
+ * Added --reduce-memory-overheads switch to reduce the size of the hash
+   tables used, at the expense of longer assembly times, and
+   --hash-size=<NUMBER> to set the size of the hash tables used by gas.
+ 
  * Support the .f_floating, .d_floating, .g_floating and .h_floating directives
    for the VAX target in order to be more compatible with the VAX MACRO
    assembler.

Index: gas/as.c
===================================================================
RCS file: /cvs/src/src/gas/as.c,v
retrieving revision 1.58
diff -c -3 -p -r1.58 as.c
*** gas/as.c	3 Mar 2005 01:29:52 -0000	1.58
--- gas/as.c	6 Apr 2005 11:06:02 -0000
***************
*** 42,47 ****
--- 42,48 ----
  #include "macro.h"
  #include "dwarf2dbg.h"
  #include "dw2gencfi.h"
+ #include "hash.h"
  
  #ifdef BFD_ASSEMBLER
  #include "bfdver.h"
*************** Options:\n\
*** 292,297 ****
--- 293,300 ----
    fprintf (stream, _("\
    --gdwarf-2              generate DWARF2 debugging information\n"));
    fprintf (stream, _("\
+   --hash-size=<value>     set the hash table size close to <value>\n"));
+   fprintf (stream, _("\
    --help                  show this message and exit\n"));
    fprintf (stream, _("\
    --target-help           show target specific options\n"));
*************** Options:\n\
*** 314,319 ****
--- 317,326 ----
    fprintf (stream, _("\
    -R                      fold data section into text section\n"));
    fprintf (stream, _("\
+   --reduce-memory-overheads \n\
+                           prefer smaller memory use at the cost of longer\n\
+                           assembly times\n"));
+   fprintf (stream, _("\
    --statistics            print various measured statistics from execution\n"));
    fprintf (stream, _("\
    --strip-local-absolute  strip local absolute symbols\n"));
*************** parse_args (int * pargc, char *** pargv)
*** 426,431 ****
--- 433,440 ----
        OPTION_EXECSTACK,
        OPTION_NOEXECSTACK,
        OPTION_ALTERNATE,
+       OPTION_HASH_TABLE_SIZE,
+       OPTION_REDUCE_MEMORY_OVERHEADS,
        OPTION_WARN_FATAL
      /* When you add options here, check that they do
         not collide with OPTION_MD_BASE.  See as.h.  */
*************** parse_args (int * pargc, char *** pargv)
*** 457,462 ****
--- 466,472 ----
      ,{"gen-debug", no_argument, NULL, 'g'}
      ,{"gstabs", no_argument, NULL, OPTION_GSTABS}
      ,{"gstabs+", no_argument, NULL, OPTION_GSTABS_PLUS}
+     ,{"hash-size", required_argument, NULL, OPTION_HASH_TABLE_SIZE}
      ,{"help", no_argument, NULL, OPTION_HELP}
      /* New option for extending instruction set (see also -t above).
         The "-t file" or "--itbl file" option extends the basic set of
*************** parse_args (int * pargc, char *** pargv)
*** 478,483 ****
--- 488,494 ----
      ,{"mri", no_argument, NULL, 'M'}
      ,{"nocpp", no_argument, NULL, OPTION_NOCPP}
      ,{"no-warn", no_argument, NULL, 'W'}
+     ,{"reduce-memory-overheads", no_argument, NULL, OPTION_REDUCE_MEMORY_OVERHEADS}
      ,{"statistics", no_argument, NULL, OPTION_STATISTICS}
      ,{"strip-local-absolute", no_argument, NULL, OPTION_STRIP_LOCAL_ABSOLUTE}
      ,{"version", no_argument, NULL, OPTION_VERSION}
*************** the GNU General Public License.  This pr
*** 865,870 ****
--- 876,899 ----
  	case 'X':
  	  /* -X means treat warnings as errors.  */
  	  break;
+ 
+ 	case OPTION_REDUCE_MEMORY_OVERHEADS:
+ 	  /* The only change we make at the moment is to reduce
+ 	     the size of the hash tables that we use.  */
+ 	  set_gas_hash_table_size (4051);
+ 	  break;
+ 
+ 	case OPTION_HASH_TABLE_SIZE:
+ 	  {
+ 	    bfd_size_type new_size;
+ 
+             new_size = strtoul (optarg, NULL, 0);
+             if (new_size)
+               set_gas_hash_table_size (new_size);
+             else
+               as_fatal (_("--hash-size needs a numeric argument"));
+ 	    break;
+ 	  }
  	}
      }
  
Index: gas/hash.c
===================================================================
RCS file: /cvs/src/src/gas/hash.c,v
retrieving revision 1.13
diff -c -3 -p -r1.13 hash.c
*** gas/hash.c	3 Mar 2005 01:29:53 -0000	1.13
--- gas/hash.c	6 Apr 2005 11:06:02 -0000
***************
***************
*** 33,42 ****
  #include "safe-ctype.h"
  #include "obstack.h"
  
- /* The default number of entries to use when creating a hash table.  */
- 
- #define DEFAULT_SIZE (4051)
- 
  /* An entry in a hash table.  */
  
  struct hash_entry {
--- 33,38 ----
*************** struct hash_control {
*** 72,77 ****
--- 68,107 ----
  #endif /* HASH_STATISTICS */
  };
  
+ /* The default number of entries to use when creating a hash table.
+    Note this value can be reduced to 4051 by using the command line
+    switch --reduce-memory-overheads, or set to other values by using
+    the --hash-size=<NUMBER> switch.  */
+ 
+ static unsigned int gas_hash_table_size = 65537;
+ 
+ void
+ set_gas_hash_table_size (unsigned int size)
+ {
+   gas_hash_table_size = size;
+ }
+ 
+ /* FIXME: This function should be amalgmated with bfd/hash.c:bfd_hash_set_default_size().  */
+ static unsigned int
+ get_gas_hash_table_size (void)
+ {
+   /* Extend this prime list if you want more granularity of hash table size.  */
+   static const unsigned int hash_size_primes[] =
+     {
+       1021, 4051, 8599, 16699, 65537
+     };
+   unsigned int index;
+ 
+   /* Work out the best prime number near the hash_size.
+      FIXME: This could be a more sophisticated algorithm,
+      but is it really worth implementing it ?   */
+   for (index = 0; index < ARRAY_SIZE (hash_size_primes) - 1; ++index)
+     if (gas_hash_table_size <= hash_size_primes[index])
+       break;
+ 
+   return hash_size_primes[index];
+ }
+ 
  /* Create a hash table.  This return a control block.  */
  
  struct hash_control *
*************** hash_new (void)
*** 81,92 ****
    struct hash_control *ret;
    unsigned int alloc;
  
!   size = DEFAULT_SIZE;
  
!   ret = (struct hash_control *) xmalloc (sizeof *ret);
    obstack_begin (&ret->memory, chunksize);
    alloc = size * sizeof (struct hash_entry *);
!   ret->table = (struct hash_entry **) obstack_alloc (&ret->memory, alloc);
    memset (ret->table, 0, alloc);
    ret->size = size;
  
--- 111,122 ----
    struct hash_control *ret;
    unsigned int alloc;
  
!   size = get_gas_hash_table_size ();
  
!   ret = xmalloc (sizeof *ret);
    obstack_begin (&ret->memory, chunksize);
    alloc = size * sizeof (struct hash_entry *);
!   ret->table = obstack_alloc (&ret->memory, alloc);
    memset (ret->table, 0, alloc);
    ret->size = size;
  
Index: gas/hash.h
===================================================================
RCS file: /cvs/src/src/gas/hash.h,v
retrieving revision 1.6
diff -c -3 -p -r1.6 hash.h
*** gas/hash.h	3 Mar 2005 01:29:53 -0000	1.6
--- gas/hash.h	6 Apr 2005 11:06:02 -0000
***************
*** 24,29 ****
--- 24,33 ----
  
  struct hash_control;
  
+ /* Set the size of the hash table used.  */
+ 
+ void set_gas_hash_table_size (unsigned int);
+ 
  /* Create a hash table.  This return a control block.  */
  
  extern struct hash_control *hash_new (void);

Index: gas/doc/as.texinfo
===================================================================
RCS file: /cvs/src/src/gas/doc/as.texinfo,v
retrieving revision 1.118
diff -c -3 -p -r1.118 as.texinfo
*** gas/doc/as.texinfo	28 Mar 2005 22:34:17 -0000	1.118
--- gas/doc/as.texinfo	6 Apr 2005 11:06:04 -0000
*************** gcc(1), ld(1), and the Info entries for 
*** 227,240 ****
  @smallexample
  @c man begin SYNOPSIS
  @value{AS} [@b{-a}[@b{cdhlns}][=@var{file}]] [@b{--alternate}] [@b{-D}]
!  [@b{--defsym} @var{sym}=@var{val}] [@b{-f}] [@b{-g}] [@b{--gstabs}] [@b{--gstabs+}]
!  [@b{--gdwarf-2}] [@b{--help}] [@b{-I} @var{dir}] [@b{-J}] [@b{-K}] [@b{-L}]
!  [@b{--listing-lhs-width}=@var{NUM}] [@b{--listing-lhs-width2}=@var{NUM}]
!  [@b{--listing-rhs-width}=@var{NUM}] [@b{--listing-cont-lines}=@var{NUM}]
!  [@b{--keep-locals}] [@b{-o} @var{objfile}] [@b{-R}] [@b{--statistics}] [@b{-v}]
!  [@b{-version}] [@b{--version}] [@b{-W}] [@b{--warn}] [@b{--fatal-warnings}] 
!  [@b{-w}] [@b{-x}] [@b{-Z}] [@b{--target-help}] [@var{target-options}] 
!  [@b{--}|@var{files} @dots{}]
  @c
  @c Target dependent options are listed below.  Keep the list sorted.
  @c Add an empty line for separation. 
--- 227,241 ----
  @smallexample
  @c man begin SYNOPSIS
  @value{AS} [@b{-a}[@b{cdhlns}][=@var{file}]] [@b{--alternate}] [@b{-D}]
!  [@b{--defsym} @var{sym}=@var{val}] [@b{-f}] [@b{-g}] [@b{--gstabs}]
!  [@b{--gstabs+}] [@b{--gdwarf-2}] [@b{--help}] [@b{-I} @var{dir}] [@b{-J}]
!  [@b{-K}] [@b{-L}] [@b{--listing-lhs-width}=@var{NUM}]
!  [@b{--listing-lhs-width2}=@var{NUM}] [@b{--listing-rhs-width}=@var{NUM}]
!  [@b{--listing-cont-lines}=@var{NUM}] [@b{--keep-locals}] [@b{-o}
!  @var{objfile}] [@b{-R}] [@b{--reduce-memory-overheads}] [@b{--statistics}]
!  [@b{-v}] [@b{-version}] [@b{--version}] [@b{-W}] [@b{--warn}]
!  [@b{--fatal-warnings}] [@b{-w}] [@b{-x}] [@b{-Z}] [@b{--target-help}]
!  [@var{target-options}] [@b{--}|@var{files} @dots{}]
  @c
  @c Target dependent options are listed below.  Keep the list sorted.
  @c Add an empty line for separation. 
*************** Name the object-file output from @comman
*** 554,559 ****
--- 555,572 ----
  @item -R
  Fold the data section into the text section.
  
+ @kindex --hash-size=@var{number}
+ Set the default size of GAS's hash tables to a prime number close to
+ @var{number}.  Increasing this value can reduce the length of time it takes the
+ assembler to perform its tasks, at the expense of increasing the assembler's
+ memory requirements.  Similarly reducing this value can reduce the memory
+ requirements at the expense of speed.
+ 
+ @item --reduce-memory-overheads
+ This option reduces GAS's memory requirements, at the expense of making the
+ assembly processes slower.  Currently this switch is a synonym for
+ @samp{--hash-size=4051}, but in the future it may have other effects as well.
+ 
  @item --statistics
  Print the maximum space (in bytes) and total time (in seconds) used by
  assembly.