Re: Is it OK to write ASCII strings directly into locale source files?

[Mike FABIAN <mfabian at redhat dot com>]

Carlos O'Donell <carlos@redhat.com> wrote:

> On 07/24/2017 09:09 AM, Mike FABIAN wrote:
>> 
>> Currently the locale source files use a lot of code points even for
>> strings which are pure ASCII. For example localedata/locales/de_DE
>> contains:
>> 
>> %	"%a %d %b %Y %T %Z"
>> d_t_fmt
>> "<U0025><U0061><U0020><U0025><U0064><U0020><U0025><U0062><U0020><U0025><U0059><U0020><U0025><U0054><U0020><U0025><U005A>"
>> 
>> Would it be OK to write this as
>> 
>> d_t_fmt "%a %d %b %Y %T %Z"
>> 
>> ??
>> 
>> This would make the files much more readable.
>> 
>> Stuff that is mostly ASCII can probably be written like this:
>> 
>> % https://oc.wikipedia.org/wiki/Fran%C3%A7a França
>> country_name "Fran<U00E7>a"
>> 
>> which is already more readable then writing it all in <U00??> code points.
>> 
>> It would be even nicer to write it completely in UTF-8, i.e.:
>> 
>> country_name "França"
>> 
>> but I am not sure whether this is allowed in the locale source files.
>> 
>> But at least for everything which is ASCII, it might be OK already to
>> write the characters directly.
>> 
>> Is writing ASCII there allowed or not??
>  
> It's not ASCII though is it? Since '<' and '>' have to be reserved
> to support parsing of UTF-8 code points, so it's "almost ASCII."
>
> I'm ok using 'almost' ASCII characters as their 1-byte UTF-8 form
> instead of the verbose code-points, but we need to document exactly
> which characters are allowed. I believe the answer is everything
> except '<>'.
>
> I'm not entirely ready to allow all UTF-8, since that descends into
> the much more complex discussion around NFC, NFKC, NFD, NFKD etc. and
> which form should be used. Then there are discussions around uniqueness
> of decomposition and exactly what did the source author want.
>
> So let us start slowly and agree with 'ASCII - [<>]' where < denotes
> the start of a code point and > the end of the code point.

Yes, that sounds like a very reasonable first step!

Is it OK to use that already *now*?

Or is any change necessary to make that work?

I tried

country_name "Fran<U00E7>a"

and it seems to work:

bash-4.4# LC_ALL=oc_FR.UTF-8 locale -k country_name
country_name="França"

So maybe it is possible to use that right now without having to change
anything in the code parsing the locale source files.

-- 
Mike FABIAN <mfabian@redhat.com>