[Bug locale/18927] Different strings should never collate as equal

["egmont at gmail dot com" <sourceware-bugzilla at sourceware dot org>]

https://sourceware.org/bugzilla/show_bug.cgi?id=18927

--- Comment #4 from Egmont Koblinger <egmont at gmail dot com> ---
Going further with the example that strcoll("ssz", "szsz") == 0 (as it used to
be in hu_HU):

It's possible that "sort" prints multiple occurrences of these strings in mixed
order. E.g. it might legally output this:

szsz
ssz
szsz
ssz
ssz
szsz
(etc.)

based on random, or stableness. In turn, when this is piped to "uniq", it might
make some sense to print the first entry only (although it's still random
whether that'll be "ssz" or "szsz"). Piping it to "LC_ALL=C uniq" instead
wouldn't make much more sense either, its output (even the number of lines)
would also be random.

Trying to think of the big picture rather than the details, in my opinion this
unexpected behavior should ideally be stopped at the very core, that is, in the
locale implementation of strcoll() and strxfrm(), so that "sort" cannot produce
the output shown above. Then there's no subsequent problem with "uniq"
whatsoever.

Otherwise, the only reliable way to print each line exactly once would be
"LC_ALL=C sort | LC_ALL=C uniq" (or "LC_ALL=C sort -u" for short), and if you
wanted to have the output sorted according to your current locale, you'd have
to issue "LC_ALL=C sort | LC_ALL=C uniq | sort" (or "LC_ALL=C sort -u | sort"
for short).

Currently if you issue the obvious locale-dependent "sort | uniq" or "sort -u",
you could never be sure whether the underlying locale's definition might cause
unexpected (i.e. faulty - as seen by the user) behavior.

-- 
You are receiving this mail because:
You are on the CC list for the bug.