Re: [PATCH][BZ 18934] hu_HU: Fix multiple sorting bugs.

[Luis Javier Merino <ninjalj at gmail dot com>]

I've had a further look at Egmont's patch. It does the following:

- It reverts b008d4c (the "fix" for BZ#13547, which broke collation in
other ways). Reverting this brings collation more in line with ICU.
- It defines DIACRIT_FORWARD. This brings collation more in line with ICU.
- It fixes BZ#18587, defining collating symbols <MIN-MIN> and
<CAP-CAP>. Before, collation went cs (<MIN>) < cS (<MIN-CAP>) < CS
(<CAP>) < Cs (<CAP-MIN>). After, it goes cs (<MIN-MIN>) < cS
(<MIN-CAP>) < Cs (<CAP-MIN>) < CS (<CAP-CAP>). This brings collation a
little more in line with ICU.
- It introduces <SINGLE_OR_COMPOUND> and <COMPOUND> collating symbols,
and assigns secondary weights to digraphs/trigraphs and contracted
digraphs/trigraphs using them. <SINGLE_OR_COMPOUND> is ordered before
<COMPOUND>, which makes short forms collate belong long forms. b008d4c
already made short forms collate before long forms, by ordering
<c_or_cs> and the like before <cs> and the like. ICU doesn't collate
long forms before short forms until level 3. Perl collates them
stably, i.e. just as they appear in the input. In any case, ordering
<COMPOUND> before <SINGLE_OR_COMPOUND> would give ICU's ordering,
which I'm not at all sure it's better. Applying Egmont's patch doesn't
divert from ICU further than b008d4c did, and fixes other things.

I've noticed another difference with respect to ICU:

- When a word appears both with and without hyphen (pingpong and
ping-pong), they collate differently. This probably applies to all
glibc locales. ICU probably changes ordering when selecting a
different algorithm for variable weighings: Perl gives glibc ordering
(hyphenated word before non-hyphenated word) for "Shifted" and
"Non-Ignorable", the opposite ordering for "Shift-Trimmed" and
"Blanked".

So, to recap the other differences to ICU:

- ICU sorts long forms before short forms at L3. Perl collates as per
the input ordering. This can be changed in Egmont's patch by
reordering <COMPOUND> before <SINGLE_OR_COMPOUND>, but I'm not sure
that's better.
- ICU doesn't recognize some mixed case combinations as
digraphs/trigraphs, e.g. cS is treated as
<c><s>;<BAS><BAS>;<MIN><CAP>, not as <cs>;<BAS>;<MIN-CAP>. Perl and
glibc recognize them. Looking at some historical files in CLDR repo,
AIX and MS behaved as ICU, Sun JDK and IBM JDK behaved as glibc. I
haven't looked at the full CLDR repo. The following may be
interesting: http://unicode.org/cldr/trac/ticket/889 and
http://unicode.org/cldr/trac/changeset/1450/trunk/common/collation/hu.xml
, recognition of those digraphs is still marked as unconfirmed draft
in the latest version of the file.