Re: glibc release branch procedures

[Petr Baudis <pasky at suse dot cz>]

  Hi,

  I added glibc maintainers of several major Linux distributions to Cc
list to gather their opinions on what kind of patches should be included
in the glibc-2.10-branch if they were to use that instead of the current
practice (the only release and variously large patchball of backported
fixes). I included Debian, they just switched to eglibc, but its 2.10
branch is likely to track glibc-2.10 in the future (is it, jsm?).

  I'm full-quoting the mail.  Summary of previous discussion: Me and
Roland agree that (up to exceptions) only cherry-picked commits from
master will appear on the branch, they will be only bugfixes and API/ABI
will never be changed.  What we agree about less is the degree of
strictness on what patches to include.

  (I could also plug in that I triggered this by starting an effort to
maintain an unofficial glibc-2.10 branch at
http://repo.or.cz/w/glibc/pb-stable.git.)

On Fri, May 22, 2009 at 09:25:52PM -0700, Roland McGrath wrote:
> Like I said, that old message was just a starting point for discussion.
> Reading it alone lacks the context of the even earlier discussion about
> this (sorry, pointers not handy).  That gist of that context is that
> community release branches are only really useful at all if they are in
> fact a convenient means of collaboration for distro glibc maintainers.
> All my intents about release management are based on that premise.
> 
> The proposals about strict rules for bugzilla and so forth presupposed
> also actually working out and implementing procedures for bugzilla
> workflow that keep it all convenient and well-tracked for everyone,
> which never happened either.  That is another discussion I hope we might
> be able to jump-start again soon and tie it into the release management
> ideas as we hash it all out.  But certainly any new branch conventions
> relating to bugzilla don't make sense until we work out what to do with
> bugzilla to make it not such a mess.
> 
> The core idea about a conservative filter on what goes into a release
> branch is to make sure it stays something that is a common baseline for
> update releases acceptable to all distro maintainers and to "official"
> GNU maintainers.  To keep GNU maintainers happy, it needs to be all
> cherry-picks (or manual backports when needed) from mainline.  To keep
> distro maintainers happy, it needs to be patches that they would include
> in their glibc update packages anyway.
> 
> So the "gate-keeping" role of a branch's release manager is to
> ascertain a true consensus among different distro maintainers each
> actively working on a distro stream based on this branch, and then
> apply a "smell test" from a "pure" community/GNU perspective.
> (Every distro will always carry some patches, and there may well be
> a patch that every distro agrees on but that we won't ever think is
> appropriate for a GNU release.)

  Note that this does not have to be absolute consensus, though - a
distribution can also include a patch to back out a change (like when we
re-disabled gethostbyname4() for glibc-2.9). Not that this should happen
regularly.

> That consensus comes from each maintainer's judgment of "tested and
> right enough for this maintenance stream".  The character of that
> judgment will vary for different maintenance streams and over
> different periods of the lifetime of a release stream.  That's as it
> should be.  The "tested in production" rules in that old message of
> mine was just a vague attempt to codify what we know every serious
> distro maintainer will be doing anyway (i.e. having tested your new
> stable distro builds with the patches before submitting them).
> 
> The official GNU bug-fix releases (if they were ever to happen) would be
> far less frequent than new distro stable builds being distributed for
> testing.  The latter often happens on a daily basis.

But that also means that we should be conservative about when do we tag
a bug-fix release, but we can be liberal on what we _commit_. If we
commit liberally, _then_ let distribution packagers to test that and tag
if it stays stable for reasonable period of time, the workflow is much
simpler and the overhead is a lot less than what you outline below.
The point being, we can revert patches on the stable branch too.

> The workflow I envision is something like this:
> 
> 1. distro maintainer chooses a patch to include in a distro glibc update.
>    (This step might implicitly include fixing the bug, submitting the fix
>    for mainline, etc.)  First puts it in a distro branch, private topic
>    branch then merged into distro branch, or just package-build patch,
>    builds new distro package, tests it, decides it is good.  (For any
>    nontrivial fix, the "it is good" decision presumably follows some
>    users/testers using that build.)  For open-collaboration distros,
>    doing that inherently includes "publishing" the build, somewhere at
>    least known to the distro's developer/tester community.
> 
> 2. distro maintainer puts patch on a "could be merged" branch
>    (i.e. one that forks from release/x/master).  This could be in a
>    private user/branch or distro/branch in the community repo, or in
>    an advertised personal/distro repo.  Then posts patch, citation,
>    pull-request, whatever, for community consideration.  Other
>    distro maintainers and commenters will post assent or dissent.
> 
>    When multiple distro maintainers either use sourceware or
>    advertise their own well-known repo, then you can affirm one of
>    another distro maintainer's "could be merged" branches by merging
>    it into yours.  Maintainers might have a single "pretty sure
>    about merging" branch they always use, or they might use multiple
>    topic branches for separate fixes.
> 
>    One way to do the distro-private workflow is to use a topic
>    branch forked from the shared release/x/master to begin with, and
>    merge that into your distro/x/master branch from which you make
>    distro builds.  Then after it's verified working and you propose
>    it for the community release branch, that original topic branch
>    can eventually get merged into release/x/master so your next
>    merge onto distro/x/master is automagic.  Of course all that is
>    up to each maintainer.
> 
> 3. At some point the release branch manager comes along and gleans
>    that there is consensus for some set of patches, and applies the
>    smell test.  If it feels kosher, he merges the proposed branch
>    into release/x/master.
> 
>    In the first days of a new branch, this might be happening very
>    often.  But later in a cycle, it may be natural for the branch
>    manager to take stock every week or three and merge in several
>    consensus items.  If distro branch maintainers are pulling in
>    each other's topic branches as they agree on things, then each
>    actual central release branch merge is more pro forma than part
>    of how maintainers manage changes.  It only really matters when
>    it aids someone's convenience and for step #4...
> 
> 4. At some point, somebody looks up and says, "I think I feel a point
>    release coming on, how about y'all?"  The branch manager responds
>    as if poked with a stick if he has been slow on his #3 iterations.
> 
>    When the consensus of maintainers is that release/x/master now just
>    about has the good stuff, the whole good stuff, and nothing but the
>    good stuff, or at least enough to settle the stomach for a while,
>    and thus it might deserve to be enshrined on ftp.gnu.org, the
>    branch manager does the final (and only) non-backport commits like
>    version.h and NEWS updates.  When maintainers tell the branch
>    manager he hasn't just botched that, he makes a signed tag, cranks
>    out tarballs and signs them (and the rest is history).
> 
> Naturally some of the time #2 will be short-circuited to just
> somebody emailing a patch or citing a mainline commit informally.
> A branch manager being proactive might decide some mainline commits
> are "obviously right and obviously necessary" and short-circuit #3
> to do the cherry-pick onto release/x/master himself based on wise
> and conservative discretion that all the distro maintainers will
> have agreed with the selection.  And I have no idea if any
> maintainers will really want to use many mergeable topic branches
> and pull each other's as a common way of working (I just think
> that's what I'd want to do).

I personally feel this workflow looks very nice, but is really
overdesigned. I guess in terms of this proposal, I would skip step #1
and short-circuit step #2. :-)

My idea of using the stable branch is to just keep tracking its HEAD in
the distribution at all times regularly up to the "non-critical bugfixes
freeze" point. If I feel that some bugfix is missing, I'd rather _by
default_ add it to the stable branch right away rather than include it
locally, go through some submission process, and have the overhead of
all that.

Your process seems optimized for the case where large portion of the
patches might be controversial for many parties. But I think this will
be rather an exception than a rule and the process should require as
little effort as possible for the common case of obvious bugfixes. We
aren't that large community not to be able to decide contentious patches
case-by-case basis for a start, I think.

I'm interested about how other maintainers feel.

> This is a loose illustration of what I think could go nicely for
> everybody.  The only hard and fast requirement of what procedures
> we actually set (or even sometimes follow ;-) from me is that in my
> discretion as a GNU maintainer and as a steward of the overall
> glibc community I feel it's something I can participate in and
> endorse as producing proper GNU C Library releases that I will be
> proud to have print my name.  I don't want to be one of the most
> active participants in the long run (it's been 20 years already).
> So I'm all for establishing whatever the modes of collaboration are
> that best make that happen over time with the people who will do it.

-- 
				Petr "Pasky" Baudis
The lyf so short, the craft so long to lerne. -- Chaucer