Re: Glibc stable release process (Glibc 2.26.1)

["Yann E. MORIN" <yann dot morin dot 1998 at free dot fr>]

Tulio, Paul, Arjan, All,

Thanks for the fast replies. :-)

[Replying with Romain, here, as we wrote the query together]

I have also interspersed Tulio's, Paul's, and Arjan's replies...

On 2017-09-29 18:38 -0300, Tulio Magno Quites Machado Filho spake thusly:
> Romain Naour <romain.naour@gmail.com> writes:
> > However, the point is not about choosing what commit to cherry-pick.
> > From the downstream perspective, a commit on the maintenance branch is always
> > worth it. If it were not needed, it would not have been committed to the
> > maintenance branch in the first place.
> >
> > Gabriel F. T. Gomes suggested the use of a git clone from a given hash instead
> > but, as explained by Yann E. Morin, using "2.26.1" is much more descriptive than
> > using a random hash from the stable branch.
> 
> It isn't random. [snip]

Of course we know what a sha1 in a git tree means, thank you. ;-)

What Romain and I were trying to say was that we would have to
arbitrarily choose the commit on the tree. That choice would look
random.

Let's say that we decide to use today's last commit on the branch, which
is d37c951fde57e8acb320a9a7d437ba50a1fc3c8a. That's all good. But then
tomorrow, new commits get pushed to the maintenance branch, and now the
head of the same branch is 548cc83c38a91852b1e44045ead3d20ccd5db4cf
(it is now fdf58ebc60ce0eb459fd616241b52872b3571ac1; writing this mail
took much longer than I anticipated ;-] ).

So in three days time, our "choice" would look no better as if we had
randomly choosen a commit on the tree. There is nothing that makes that
one stand out more than the ones before or after. Except that at some
point in time, it was the latest. But that is useless past that very
moment.

The problem with using a sha1 is two fold. First, it is not
human-readable. While I can pretty easily wrap my head around what
2.26.1 means, d37c951fde57e8acb320a9a7d437ba50a1fc3c8a is defintiely
opaque for me.

Second, because it is opaque, any commit on a branch is semantically
equivalent to another. So, there is nothing that says "this is pretty
good, you can use it".

The alternative, that only guarantees the former, but does not help on
the latter, is to cary the now-42 patches from the branch locally.

This is ugly, because it is very easily prone to bitrot, and like
explained above, would be missing later patches, so the set of patch
would look no less randomly chosen that chosing a sha1 would be.

> That information may not be important for everyone, but it is valuable for
> some users.

And I can assure you that we do find this information very important.

> It also helps downstream to follow the "upstream first" ideology. ;-)

Here, it is just about consuming a release. Contribution is another
important topic, but that's not the point here.

Release tag or git sha1, is not really what would prevent a downstream
from carying local patches. But that's a tangent...

> > A new dot-release is a strong indication that the most critical fixes have made
> > their way to the maintenance branch, and that it has been officially sanctioned
> > by upstream (you! ;-)), which is a very important status for downstreams.
> 
> This behavior varies according to the community.  On glibc, a patch is only
> backported to stable branches if it's already reviewed and accepted on master
> and if there is consensus they're suitable to a stable branch [1].

I think that what we wrote is basically that: patches on the maintenance
branch have a reason to be there, so we as a downstream do not have to
cherry-pick those. We want to use all of them, because you already did a
good job at cherry-picking them from master! :-)



On 2017-09-29 17:26 -0700, Paul Eggert spake thusly:
> Romain Naour <romain.naour@gmail.com> writes:
> > Maybe a middle ground is that a tag is pushed to the repository, just to
> > identify a specific sha1 as "this is 2.26.1". This is relatively lightweight (a
> > signed tag) and gives downstream users that need it a clear identification of a
> > blessed version.
> That "blessing" would be problematic. How can we be expected to "bless" any
> particular commit unless we've checked it? And that checking will take some
> work, work that will slow us down on other things.

Yet, every commit that went on the branch has already been blessed, by
the very process that allowed it to be cherry-picked from master.

So, tagging the branch can happen almost at any time: it only contains
bug fixes that have already been thoroughly reviewed, as Tulio explained.

> Instead of blessing, how about using the output of something like the
> following shell command to generate the version number?
> 
> git describe --match 'glibc-*' --abbrev=7 --dirty
[--SNIP--]
> glibc-2.26-415-g4d3693e

This is indeed a nice representation. But it is not stable with time,
because 7 is not enough to uniquely identify a commit. Even it it were
today, it may no longer be the case in the future, killing
reproducibility.

But the choice of what commit to use is no less arbitrary than the
alternatives, above.

> [snip] A similar (but shorter) version-numbering scheme is
> used by GNU Coreutils and other GNU packages when between releases, and Gnulib
> has a script build-aux/git-version-gen that can generate it.

binutils and gcc, as well as the Linux kernel (for headers), which are
all other components of a toolchain, all do provide dot-releases. ;-)



On 2017-09-29 18:38 -0300, Tulio Magno Quites Machado Filho spake thusly:
> You can track these discussions in the libc-stable mailing list. [2]
> 
> It doesn't affect only glibc 2.26.  If there are older releases downstream
> not following a glibc stable branch which aren't cherry-picking backported
> patches I strongly advise to start doing so in order to get bugs fixes and
> security fixes because the last glibc point release was 2.14.1 in 2011.

That is unrelated. The point is not about cherry-picking individual patches.

Consider that not all downstreams have the manpower of a Redhat, a
Debian, a Fedora or a Ubuntu (to name a few). By providing regular
dot-releases, you also help those downstreams to update.



On 2017-09-29 17:38 -0700, Arjan van de Ven spake thusly:
> as someone who does glibc for a distro... I strongly prefer tarbal
> releases.

May we know what distribution you are refering to?

> taking a patch or two is fine, but doing this over and over again makes
> it less obvious what exact stack is running etc.

Exactly.

Cherry-picking individual patches from the stable branch also risk
missing critical fixes, or introducing regressions, as a fix may rely on
a previous one, and all sort of hard-to-see interactions between fixes.



On 2017-09-29 18:38 -0300, Tulio Magno Quites Machado Filho spake thusly:
> I'm not opposing to a point release, but considering how the glibc community
> has been working, I don't think it helps to make it more stable, reviewed or
> officially sanctioned.
> 
> [1] https://sourceware.org/glibc/wiki/Release/#General_policy

So, lemme quote this:

    Each branch has so-called interested parties, usually glibc maintainers
    in distributions where the particular branch is being used; tagging
    revisions on the release branches should result of consensus between the
    maintainer and interested parties - one workable model is that the
    maintainer suggests that he wants to release and other people check if
    they are happy with the set of patches included and the timing is
    fine-tuned; if a release is important for one of the parties (e.g.
    distribution nearing a release), they can suggest a release of new
    revision as well if it is meaningful. 

Is there a way to register the Buildroot project as an interested party?

As an interested party, we Buildroot would like to suggest that a new
revision be tagged, as it is meaningful to us:

  - at least two major regressions on two major architectures have been
    fixed, namely BZ-21930 and BZ-22146,

  - we are nearing the end of our development cycle (freeze end of
    October) and would benefit from a human-readable version tag.

(Just tell me what rules to play by, and I'll happily play by those
rules! ;-] )

Yes, we are asking you to put in a bit more of efforts to help us, that
is true. But we still believe that pushing a tag is not that much. ;-)

Anyway, we do appreciate that this is your decision to take.

Thank you for reading so far! ;-)

Regards,
Yann E. MORIN, Romain Naour.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'