RFC: [PATCH] PR ld/18028: 32-bit ld runs out of memory when linking 32-bit clang with debug info

["H.J. Lu" <hongjiu dot lu at intel dot com>]

When link_info.keep_memory is TRUE, linker caches the relocation
information and symbol tables of input files in memory.  On 32-bit
hosts, linker runs out of 32-bit virtual memory on input files with many
relocations.  This patch defaults link_info.keep_memory to TRUE only if
pointer size > 4 and adds a --keep-memory linker option. FIXME: linker
may run out of physical memory on 64-bit hosts with 4GB ram.

Any comments?

Thanks.


H.J.
---
	PR ld/18028
	* ld.texinfo: Document --keep-memory.
	* ldlex.h (option_values): Add OPTION_KEEP_MEMORY.
	* ldmain.c (main): Set link_info.keep_memory to TRUE if pointer
	size > 4.
	* lexsup.c (ld_options): Add OPTION_KEEP_MEMORY.
	(parse_args): Handle OPTION_KEEP_MEMORY.
---
 ld/ld.texinfo | 18 ++++++++++++------
 ld/ldlex.h    |  1 +
 ld/ldmain.c   |  3 ++-
 ld/lexsup.c   |  5 +++++
 4 files changed, 20 insertions(+), 7 deletions(-)

diff --git a/ld/ld.texinfo b/ld/ld.texinfo
index 314d3f3..cba39d6 100644
--- a/ld/ld.texinfo
+++ b/ld/ld.texinfo
@@ -1479,13 +1479,19 @@ Print a link map to the file @var{mapfile}.  See the description of the
 @option{-M} option, above.
 
 @cindex memory usage
+@kindex --keep-memory
 @kindex --no-keep-memory
-@item --no-keep-memory
-@command{ld} normally optimizes for speed over memory usage by caching the
-symbol tables of input files in memory.  This option tells @command{ld} to
-instead optimize for memory usage, by rereading the symbol tables as
-necessary.  This may be required if @command{ld} runs out of memory space
-while linking a large executable.
+@item --keep-memory
+@itemx --no-keep-memory
+Optimize for speed over memory usage by caching the relocation information
+and symbol tables of input files in memory.  @samp{--no-keep-memory}
+tells @command{ld} to instead optimize for memory usage, by rereading
+the relocation information and symbol tables as necessary.
+@samp{--no-keep-memory} option may be required if @command{ld} runs out
+of memory space while linking a large executable.
+
+@samp{--keep-memory} is the default for 64-bit hosts.
+@samp{--no-keep-memory} is the default for 32-bit hosts.
 
 @kindex --no-undefined
 @kindex -z defs
diff --git a/ld/ldlex.h b/ld/ldlex.h
index be7f653..073bfc3 100644
--- a/ld/ldlex.h
+++ b/ld/ldlex.h
@@ -41,6 +41,7 @@ enum option_values
   OPTION_NO_EXPORT_DYNAMIC,
   OPTION_HELP,
   OPTION_IGNORE,
+  OPTION_KEEP_MEMORY,
   OPTION_MAP,
   OPTION_NO_DEMANGLE,
   OPTION_NO_KEEP_MEMORY,
diff --git a/ld/ldmain.c b/ld/ldmain.c
index 4b41288..ea6bae3 100644
--- a/ld/ldmain.c
+++ b/ld/ldmain.c
@@ -274,7 +274,8 @@ main (int argc, char **argv)
   demangling = getenv ("COLLECT_NO_DEMANGLE") == NULL;
 
   link_info.allow_undefined_version = TRUE;
-  link_info.keep_memory = TRUE;
+  /* Don't turn on keep_memory by default for 32-bit hosts.  */
+  link_info.keep_memory = sizeof (void *) > 4;
   link_info.combreloc = TRUE;
   link_info.strip_discarded = TRUE;
   link_info.emit_hash = TRUE;
diff --git a/ld/lexsup.c b/ld/lexsup.c
index aa6c3cd..6046aca 100644
--- a/ld/lexsup.c
+++ b/ld/lexsup.c
@@ -334,6 +334,8 @@ static const struct ld_option ld_options[] =
     '\0', NULL, N_("Print option help"), TWO_DASHES },
   { {"init", required_argument, NULL, OPTION_INIT},
     '\0', N_("SYMBOL"), N_("Call SYMBOL at load-time"), ONE_DASH },
+  { {"keep-memory", no_argument, NULL, OPTION_KEEP_MEMORY},
+    '\0', NULL, N_("Cache relocation information in memory"), TWO_DASHES },
   { {"Map", required_argument, NULL, OPTION_MAP},
     '\0', N_("FILE"), N_("Write a map file"), ONE_DASH },
   { {"no-define-common", no_argument, NULL, OPTION_NO_DEFINE_COMMON},
@@ -826,6 +828,9 @@ parse_args (unsigned argc, char **argv)
 	  help ();
 	  xexit (0);
 	  break;
+	case OPTION_KEEP_MEMORY:
+	  link_info.keep_memory = TRUE;
+	  break;
 	case 'L':
 	  ldfile_add_library_path (optarg, TRUE);
 	  break;
-- 
1.9.3