[PATCH] localedef: Add --localedir and --archive-file options (Bug 19130)

["Carlos O'Donell" <carlos at redhat dot com>]

At present localdef accepts the following non-POSIX options:

--list-archive [FILE]

and

--prefix=PATH

The first allows you to list the contents of the default binary
locale archive via `--list-archive` or any other archive
via `--list-archive FILE` where FILE is an absolute or relative
path to the archive.

This patch does three things, and I have consciously not split
them up into three patches because they logically belong together.

(1) Enhance `--list-archive`, `--add-to-archive`, and 
    `--delete-from-archive` to honour `--prefix=PATH`.

At present the use of --prefix does not adjust the path
interpreted by --list-archive FILE . This is a mistake. We should
always honour --prefix in all of our output operations. The patch makes
--list-archive FILE honour prefix. For example if you ran
`--prefix /mnt/chroot --list-archive /usr/lib/locale/locale-archive-v2`
the final path to the locale archive file would be:
`/mnt/chroot/usr/lib/locale/locale-archive-v2`.

Similarly --add-to-archive and --delete-from-archive honour
--prefix when computing the archive to use.

e.g.
localedef --prefix="/home/carlos/tmp" --add-to-archive ./en_US.UTF-8
cannot create temporary file: /home/carlos/tmp/usr/lib64/locale/locale-archive.3vmNHw: No such file or directory

(2) Add a `--localedir=PATH` option per BZ #19130.

At present all operations are carried out against a hard-coded
LOCALEDIR e.g. $(localedir) or /usr/lib/locale. This patch adds
a --localedir=PATH option which allows you to change the value
of LOCALEDIR used in localedef. For example if you ran
`--localedir=/usr/lib64/locale --list-archive` the final path
to the locale archive file would be:
`/usr/lib64/locale/locale-archive`.

e.g.
./locale/localedef --prefix="/home/carlos/tmp" --localedir="/alt" --add-to-archive ./en_US.UTF-8
cannot create temporary file: /home/carlos/tmp/alt/locale-archive.VCqI8z: No such file or directory

(3) Add a `--archive-file=FILE`/`-a FILE` option.

At present the binary locale archive name is always `locale-archive`,
but it would be easier for distribution maintainers and developers
if localedef could be made to scan an arbitrary file. This patch
makes `--archive-file=FILE` adjust the default locale archive file
name to be FILE. For example if you ran:
`--prefix /mnt/croot --archive-file=locale-archive-v2` the final path
to the locale archive file would be:
`/mnt/chroot/usr/lib/locale/locale-archive-v2`.

With (1), (2) and (3), you have full control over the locale archive
selected by localedef:

[prefix][localedir][archive file]
|        |          |
|        |          |-> --archive-file
|        |
|        |-> --localedir
|
|-> --prefix

Sensible defaults are selected for all values based on the configuration
of glibc e.g. prefix is "", localedir is "$(localedir)" and archive file
is "locale-archive".

(a) Use of I18NPATH, LOCPATH, and change in behaviour of ENOENT for
    locale-archive.

The selection of the character maps is achieved by use of the I18NPATH
env var, and LOCPATH env var for the location of source locale data.
These environment variables do not use --prefix. Both of these variables
will fallback to the default system directory during failure to find the
required files, while the above settings for prefix/localedir/archive-file
will create the requested binary locale archive file if not found. Note
that this is a change in behaviour, previously if the non-default binary
locale archive file was not found we would issue an error (only creating
the default locale archive was supported). Now we treat all locale-archive
files the same way. It might be argued that a "--create" mode should be
used and errors issued if the file doesn't exist in all cases, but this
breaks backwards compatibility and should be avoided. The only thing we
break with this patch is using `--list-archive foo` to test if `foo`
is present and a locale-archive file, which now returns nothing, but
doesn't fail (like we did for the default locale archive).

(b) Future enhancements.

Future enhancements include more options to control locale source
path (LOCPATH) and character map locations (I18NPATH) since falling back
to the default system directories is not always desirable in a cross
or sysroot'd environment.

Tested on x86_64 to view locale archive files in non-standard paths and
to generate locale archive files in non-standard paths.

Using --help gives these addtional options:
~~~
...
      --localedir=PATH       Optional path to locale directory e.g.
                             /usr/lib64/locale
...
  -a, --archive-file=FILE    locale-archive file to use instead of default
...
~~~

OK to checkin?

2015-10-14  Carlos O'Donell  <carlos@redhat.com>

	[BZ #19130]
	* locale/programs/localedef.c (output_localedir): New variable.
	(archive_file): New variable.
	(OPT_LOCALEDIR): New macro.
	(OPT_NO_ARCHIVE, OPT_ADD_TO_ARCHIVE, OPT_REPLACE,
	OPT_DELETE_FROM_ARCHIVE, OPT_LIST_ARCHIVE): Adjust down by one.
	(options): Add --localedir option. Add --archive-file/-a option.
	(main): --list-archive FILE has precedence over -a FILE.
	Pass specified archive file (-a FILE) to callees (prefix still
	applies).
	(parse_opt): Add OPT_LOCALEDIR and store to output_localedir.
	Add 'a' and store to archive_file.
	(construct_output_path): Use output_localedir if non-NULL.
	(normalize_codeset): Use xmalloc.
	(construct_prefix_path): New function.
	* locale/programs/localedef.h: add_locales_to_archive, and
	delete_locales_from_archive accept pointer to struct locarhandle.
	* locale/programs/locarchive.c (add_locales_to_archive): Use struct
	locarhandle pointer passed by caller.
	(delete_locales_from_archive): Likewise.
	(open_archive): Always create the archive if not in readonly mode.
	(ARCHIVE_NAME): Remove.
	* locale/programs/localedef.h (LOCALE_ARCHIVE): Define.
	(ARCHIVE_NAME): Define.
	* locale/programs/locfile.c (write_all_categories): Likewise.
	* locale/programs/locfile.h: write_all_categories accepts pointer to
	struct locarhandle.

diff --git a/locale/programs/localedef.c b/locale/programs/localedef.c
index 06fca12..44f15e4 100644
--- a/locale/programs/localedef.c
+++ b/locale/programs/localedef.c
@@ -65,6 +65,9 @@ static int force_output;
 /* Prefix for output files.  */
 const char *output_prefix;
 
+/* Locale directory to use for output.  */
+const char *output_localedir;
+
 /* Name of the character map file.  */
 static const char *charmap_file;
 
@@ -74,6 +77,9 @@ static const char *input_file;
 /* Name of the repertoire map file.  */
 const char *repertoire_global;
 
+/* Location of the locale-archive file.  */
+const char *archive_file;
+
 /* Name of the locale.alias file.  */
 const char *alias_file;
 
@@ -107,11 +113,12 @@ void (*argp_program_version_hook) (FILE *, struct argp_state *) = print_version;
 #define OPT_QUIET 302
 #define OPT_OLDSTYLE 303
 #define OPT_PREFIX 304
-#define OPT_NO_ARCHIVE 305
-#define OPT_ADD_TO_ARCHIVE 306
-#define OPT_REPLACE 307
-#define OPT_DELETE_FROM_ARCHIVE 308
-#define OPT_LIST_ARCHIVE 309
+#define OPT_LOCALEDIR 305
+#define OPT_NO_ARCHIVE 306
+#define OPT_ADD_TO_ARCHIVE 307
+#define OPT_REPLACE 308
+#define OPT_DELETE_FROM_ARCHIVE 309
+#define OPT_LIST_ARCHIVE 310
 #define OPT_LITTLE_ENDIAN 400
 #define OPT_BIG_ENDIAN 401
 
@@ -131,11 +138,16 @@ static const struct argp_option options[] =
     N_("Create output even if warning messages were issued") },
   { "old-style", OPT_OLDSTYLE, NULL, 0, N_("Create old-style tables") },
   { "prefix", OPT_PREFIX, N_("PATH"), 0, N_("Optional output file prefix") },
+  { "localedir", OPT_LOCALEDIR, N_("PATH"), 0,
+    N_("Optional path to locale directory e.g. /usr/lib64/locale") },
   { "posix", OPT_POSIX, NULL, 0, N_("Strictly conform to POSIX") },
   { "quiet", OPT_QUIET, NULL, 0,
     N_("Suppress warnings and information messages") },
   { "verbose", 'v', NULL, 0, N_("Print more messages") },
+
   { NULL, 0, NULL, 0, N_("Archive control:") },
+  { "archive-file", 'a', N_("FILE"), 0,
+    N_("locale-archive file to use instead of default")},
   { "no-archive", OPT_NO_ARCHIVE, NULL, 0,
     N_("Don't add new data to archive") },
   { "add-to-archive", OPT_ADD_TO_ARCHIVE, NULL, 0,
@@ -178,6 +190,9 @@ static struct argp argp =
 /* Prototypes for local functions.  */
 static void error_print (void);
 static const char *construct_output_path (char *path);
+static const char *construct_prefix_path (const char *prefix,
+					  const char *localedir,
+					  const char *path);
 static const char *normalize_codeset (const char *codeset, size_t name_len);
 
 
@@ -189,6 +204,8 @@ main (int argc, char *argv[])
   struct charmap_t *charmap;
   struct localedef_t global;
   int remaining;
+  struct locarhandle ah;
+  const char *prefix_archive_file = NULL;
 
   /* Set initial values for global variables.  */
   copy_list = NULL;
@@ -207,14 +224,34 @@ main (int argc, char *argv[])
   argp_err_exit_status = 4;
   argp_parse (&argp, argc, argv, 0, &remaining, NULL);
 
+  /* Use the user-specified archive file or NULL (default).  */
+  ah.fname = archive_file;
+  if (output_prefix != NULL || output_localedir != NULL || archive_file != NULL)
+    {
+      prefix_archive_file = construct_prefix_path (output_prefix ?: "",
+						   output_localedir ?: LOCALEDIR,
+						   archive_file ?: LOCALE_ARCHIVE);
+      ah.fname = prefix_archive_file;
+    }
+
   /* Handle a few special cases.  */
   if (list_archive)
-    show_archive_content (remaining > 1 ? argv[remaining] : NULL, verbose);
+    {
+      /* If using --list-archive FILE and -a FILE at the same time the
+	 archive file specified after --list-archive takes precedence.
+	 The use of -a only changes the default locale archive file.  */
+      if (remaining > 1 && remaining < argc)
+	ah.fname = construct_prefix_path (output_prefix,
+					  NULL,
+					  argv[remaining]);
+      show_archive_content (ah.fname, verbose);
+    }
   if (add_to_archive)
-    return add_locales_to_archive (argc - remaining, &argv[remaining],
+    return add_locales_to_archive (&ah, argc - remaining, &argv[remaining],
 				   replace_archive);
   if (delete_from_archive)
-    return delete_locales_from_archive (argc - remaining, &argv[remaining]);
+    return delete_locales_from_archive (&ah, argc - remaining,
+					&argv[remaining]);
 
   /* POSIX.2 requires to be verbose about missing characters in the
      character map.  */
@@ -286,9 +323,13 @@ cannot open locale definition file `%s'"), runp->name));
     {
       if (cannot_write_why != 0)
 	WITH_CUR_LOCALE (error (4, cannot_write_why, _("\
-cannot write output files to `%s'"), output_path ? : argv[remaining]));
+cannot write output files to `%s' (LOCALEDIR is `%s%s')"),
+				output_path ?: argv[remaining],
+				output_prefix ?: "",
+				output_localedir ?: LOCALEDIR));
       else
-	write_all_categories (locales, charmap, argv[remaining], output_path);
+	write_all_categories (&ah, locales, charmap,
+			      argv[remaining], output_path);
     }
   else
     WITH_CUR_LOCALE (error (4, 0, _("\
@@ -317,6 +358,9 @@ parse_opt (int key, char *arg, struct argp_state *state)
     case OPT_PREFIX:
       output_prefix = arg;
       break;
+    case OPT_LOCALEDIR:
+      output_localedir = arg;
+      break;
     case OPT_NO_ARCHIVE:
       no_archive = true;
       break;
@@ -338,6 +382,9 @@ parse_opt (int key, char *arg, struct argp_state *state)
     case OPT_BIG_ENDIAN:
       set_big_endian (true);
       break;
+    case 'a':
+      archive_file = arg;
+      break;
     case 'c':
       force_output = 1;
       break;
@@ -415,6 +462,26 @@ error_print (void)
 {
 }
 
+/* Construct an output path given a prefix directory, an absolute locale
+   directory and a path.  The output path is a constant and can be used
+   by any caller and must not be freed.  */
+static const char *
+construct_prefix_path (const char *prefix,
+		       const char *localedir,
+		       const char *path)
+{
+  ssize_t n;
+  char *result;
+  n = asprintf (&result, "%s%s%s%s%c",
+		prefix ?: "",
+		localedir ?: "",
+		localedir ? "/" : "",
+		path, '\0');
+  if (n < 0)
+    WITH_CUR_LOCALE (error (6, errno, _("cannot allocate path")));
+  return result;
+}
+
 
 /* The parameter to localedef describes the output path.  If it does
    contain a '/' character it is a relative path.  Otherwise it names the
@@ -458,10 +525,12 @@ construct_output_path (char *path)
       ssize_t n;
       if (normal == NULL)
 	n = asprintf (&result, "%s%s/%s%c",
-		      output_prefix ?: "", LOCALEDIR, path, '\0');
+		      output_prefix ?: "",
+		      output_localedir ?: LOCALEDIR, path, '\0');
       else
 	n = asprintf (&result, "%s%s/%.*s%s%s%c",
-		      output_prefix ?: "", LOCALEDIR,
+		      output_prefix ?: "",
+		      output_localedir ?: LOCALEDIR,
 		      (int) (startp - path), path, normal, endp, '\0');
 
       if (n < 0)
@@ -523,7 +592,7 @@ normalize_codeset (codeset, name_len)
 	  only_digit = 0;
       }
 
-  retval = (char *) malloc ((only_digit ? 3 : 0) + len + 1);
+  retval = (char *) xmalloc ((only_digit ? 3 : 0) + len + 1);
 
   if (retval != NULL)
     {
diff --git a/locale/programs/localedef.h b/locale/programs/localedef.h
index 7e19ff0..3baf991 100644
--- a/locale/programs/localedef.h
+++ b/locale/programs/localedef.h
@@ -135,6 +135,8 @@ extern const char *alias_file;
       setlocale (LC_CTYPE, cur_locale_);			\
   } while (0)
 
+#define LOCALE_ARCHIVE "locale-archive"
+#define ARCHIVE_NAME LOCALEDIR "/"LOCALE_ARCHIVE
 
 /* Mark given locale as to be read.  */
 extern struct localedef_t *add_to_readlist (int locale, const char *name,
@@ -165,10 +167,13 @@ extern int add_locale_to_archive (struct locarhandle *ah, const char *name,
 				  locale_data_t data, bool replace);
 
 /* Add content of named directories to locale archive.  */
-extern int add_locales_to_archive (size_t nlist, char *list[], bool replace);
+extern int add_locales_to_archive (struct locarhandle *ah,
+				   size_t nlist,
+				   char *list[], bool replace);
 
 /* Removed named locales from archive.  */
-extern int delete_locales_from_archive (size_t nlist, char *list[]);
+extern int delete_locales_from_archive (struct locarhandle *ah,
+					size_t nlist, char *list[]);
 
 /* List content of locale archive. If FNAME is non-null use that as
    the locale archive to list, otherwise the default.  */
diff --git a/locale/programs/locarchive.c b/locale/programs/locarchive.c
index 49f7f1b..0ede8ea 100644
--- a/locale/programs/locarchive.c
+++ b/locale/programs/locarchive.c
@@ -57,8 +57,6 @@
 
 extern const char *output_prefix;
 
-#define ARCHIVE_NAME LOCALEDIR "/locale-archive"
-
 static const char *locnames[] =
   {
 #define DEFINE_CATEGORY(category, category_name, items, a) \
@@ -581,11 +579,10 @@ open_archive (struct locarhandle *ah, bool readonly)
       fd = open64 (archivefname, readonly ? O_RDONLY : O_RDWR);
       if (fd == -1)
 	{
-	  /* Maybe the file does not yet exist? If we are opening
-	     the default locale archive we ignore the failure and
-	     list an empty archive, otherwise we print an error
-	     and exit.  */
-	  if (errno == ENOENT && archivefname == default_fname)
+	  /* Maybe the file does not yet exist?
+	     Create it if we are not in readonly mode.
+	     If we are in readonly mode pass back an empty structure.  */
+	  if (errno == ENOENT)
 	    {
 	      if (readonly)
 		{
@@ -1328,19 +1325,19 @@ add_locale_to_archive (ah, name, data, replace)
 }
 
 
+/* Add locales to the archive file specified in AH->FNAME or
+   the default if NULL.  */
 int
-add_locales_to_archive (nlist, list, replace)
-     size_t nlist;
-     char *list[];
-     bool replace;
+add_locales_to_archive (struct locarhandle *ah,
+			size_t nlist,
+			char *list[],
+			bool replace)
 {
-  struct locarhandle ah;
   int result = 0;
 
   /* Open the archive.  This call never returns if we cannot
      successfully open the archive.  */
-  ah.fname = NULL;
-  open_archive (&ah, false);
+  open_archive (ah, false);
 
   while (nlist-- > 0)
     {
@@ -1514,7 +1511,7 @@ add_locales_to_archive (nlist, list, replace)
 	  continue;
 	}
 
-      result |= add_locale_to_archive (&ah, basename (fname), data, replace);
+      result |= add_locale_to_archive (ah, basename (fname), data, replace);
 
       for (cnt = 0; cnt < __LC_LAST; ++cnt)
 	if (cnt != LC_ALL)
@@ -1522,28 +1519,28 @@ add_locales_to_archive (nlist, list, replace)
     }
 
   /* We are done.  */
-  close_archive (&ah);
+  close_archive (ah);
 
   return result;
 }
 
 
+/* Delete the listed locales from archive AH->FNAME or the default
+   archive if NULL.  */
 int
-delete_locales_from_archive (nlist, list)
-     size_t nlist;
-     char *list[];
+delete_locales_from_archive (struct locarhandle *ah,
+			     size_t nlist,
+			     char *list[])
 {
-  struct locarhandle ah;
   struct locarhead *head;
   struct namehashent *namehashtab;
 
   /* Open the archive.  This call never returns if we cannot
      successfully open the archive.  */
-  ah.fname = NULL;
-  open_archive (&ah, false);
+  open_archive (ah, false);
 
-  head = ah.addr;
-  namehashtab = (struct namehashent *) ((char *) ah.addr
+  head = ah->addr;
+  namehashtab = (struct namehashent *) ((char *) ah->addr
 					+ GET (head->namehash_offset));
 
   while (nlist-- > 0)
@@ -1565,7 +1562,7 @@ delete_locales_from_archive (nlist, list)
 	{
 	  if (GET (namehashtab[idx].hashval) == hval
 	      && (strcmp (locname,
-			  ((char *) ah.addr
+			  ((char *) ah->addr
 			   + GET (namehashtab[idx].name_offset)))
 		  == 0))
 	    {
@@ -1584,7 +1581,7 @@ delete_locales_from_archive (nlist, list)
 	error (0, 0, _("locale \"%s\" not in archive"), locname);
     }
 
-  close_archive (&ah);
+  close_archive (ah);
 
   return 0;
 }
diff --git a/locale/programs/locfile.c b/locale/programs/locfile.c
index 33da52e..51ecfcf 100644
--- a/locale/programs/locfile.c
+++ b/locale/programs/locfile.c
@@ -326,7 +326,7 @@ static void (*const write_funcs[]) (struct localedef_t *,
 
 
 void
-write_all_categories (struct localedef_t *definitions,
+write_all_categories (struct locarhandle *ah, struct localedef_t *definitions,
 		      const struct charmap_t *charmap, const char *locname,
 		      const char *output_path)
 {
@@ -338,19 +338,15 @@ write_all_categories (struct localedef_t *definitions,
 
   if (! no_archive)
     {
-      /* The data has to be added to the archive.  Do this now.  */
-      struct locarhandle ah;
-
       /* Open the archive.  This call never returns if we cannot
 	 successfully open the archive.  */
-      ah.fname = NULL;
-      open_archive (&ah, false);
+      open_archive (ah, false);
 
-      if (add_locale_to_archive (&ah, locname, to_archive, true) != 0)
+      if (add_locale_to_archive (ah, locname, to_archive, true) != 0)
 	error (EXIT_FAILURE, errno, _("cannot add to locale archive"));
 
       /* We are done.  */
-      close_archive (&ah);
+      close_archive (ah);
     }
 }
 
diff --git a/locale/programs/locfile.h b/locale/programs/locfile.h
index 6fc441b..8dabb6d 100644
--- a/locale/programs/locfile.h
+++ b/locale/programs/locfile.h
@@ -64,7 +64,8 @@ extern void check_all_categories (struct localedef_t *definitions,
 				  const struct charmap_t *charmap);
 
 /* Write out all locale categories.  */
-extern void write_all_categories (struct localedef_t *definitions,
+extern void write_all_categories (struct locarhandle *ah,
+				  struct localedef_t *definitions,
 				  const struct charmap_t *charmap,
 				  const char *locname,
 				  const char *output_path);
---

Cheers,
Carlos.