Consensus: Locale format may change from release to release?

["Carlos O'Donell" <carlos at redhat dot com>]

On 10/11/2015 10:23 PM, Allan McRae wrote:
> On 10/10/15 10:42, Carlos O'Donell wrote:
>> I've backported the fix for bug 18589 to 2.22, which is
>> the only public release with the defect that causes strcoll_l
>> to incorrectly sort digraphs.
>>
>> Those of you with distributions based on 2.22 may wish to rebase
>> to get these changes. They will cause problems with sorting and
>> are relatively serious.
>>
> 
> 
> Fun story...
> 
> [2015-10-12 09:47] [ALPM] upgraded glibc (2.22-3 -> 2.22-4)
> [2015-10-12 09:47] [ALPM-SCRIPTLET] bash: loadlocale.c:130:
> _nl_intern_locale_data: Assertion `cnt < (sizeof
> (_nl_value_type_LC_COLLATE) / sizeof (_nl_value_type_LC_COLLATE[0]))'
> failed.

You have loaded a locale onto your system without recompiling it for
the running glibc. This is a failure in your upgrade process. Though
we don't make this easy for user installed locales (which are static).

In general no new processes should be started during the glibc upgrade
until after all libraries have been replaced as a single atomic unit
and all dependent data upgraded as  single atomic unit.

We don't do this in Fedora. There is a race window where the libraries
are updated one by one (as the writes hit disk), and then the locale
archive is processed entirely and 100% replaced (along with the C.utf8
binary locale). During this race window applications may fail to start
with library dependency issues and locale data format issues. Distributions
should fix this, but they haven't.
 
> And that was given every time I tried running anything.  It is fun to
> watch your system slowly die while trying to figure a way to remove that
> update from the repos!

This is a failure in your upgrade process, it's a failure in the expectations
that we as a project set with our users, and a failure in our own code for
locale handling.

I suggest we make a "Best Practices for upgrading glibc" document that
starts like this:

(1) No new processes should be started during the glibc upgrade
    until after all libraries have been replaced as a single atomic
    unit and all dependent data upgraded as single atomic unit.

(2) Locales should be recompiled for the glibc that is about to run.
    Any old compiled locales cannot be considered safe to load with
    the new glibc. There is no guarantee, even between point releases
    that the binary format remains compatible. This includes recompiling
    the locale-archive and any other binary locales.

(3) The project assures that invalid locales will be detected to the best
    ability of the project, specifically when the project itself changes
    the binary on-disk format and such old locales which can't be loaded
    will be ignored or if specifically requested, errors returned, and the
    fallback locales used.

> The only changes between our 2.22-3 and 2.22-4 builds were these patches
> you committed.  It seems they introduce incompatibilities to locale
> files generated with builds from earlier in the 2.22 release branch?  I
> was using a relatively benign locale (en_AU.UTF8).

They do introduce a binary change in the binary compiled locales, and
as a distribution provider you always have to recompile locales for any
glibc upgrade even point releases.

If we want to change this policy, which has been true since binary locales
were started, we should discuss this in a new thread.

The bug fixed, bug 18589, turns out to be a serious issues with sorting
languages that digraphs. We could have left the binary format the same
and still fixed the bug, but I did not consider that distributions were
not recompiling the locales for the upgraded glibc.

In Fedora we recompile every locale after every glibc build in order to
ensure the right and working locales are installed after every upgrade.
We also use %_install_lang to allow users to minimize the set of locales
that get installed.

Lastly, to treat our users better we should fail gracefully. I'll backport
Ludovic's patches to 2.22 when we get them sorted out.

Cheers,
Carlos.