mirror synchronization woes

[Pierre Demartines <pierre at demartines dot com>]

Hello,

I am questioning whether it is possible to make a local mirror that is
consistent, i.e. that doesn't contain a mixup of versions due to
non-atomic rsync (mirrors trying to sync from what I imagine is a
master, or between each other, and my server trying to sync from one
of the mirrors).  At the moment, my local copy seems corrupted
(setup-x86.exe croaks on signature error and refuses to go ahead), and
I haven't found another mirror to rsync from that would repair the
situation. Several rsyncs from different mirrors lead to the same
situation: an apparently corrupted local repo.


Details
-------
Being behind a (slowish) firewall, I set up a local cygwin mirror on a
linux server.
I let an rsync-based script on crontab eat the slow connection, once
daily, and most of the time rsync does its thing with minimal impact
and all is well.

However, I've bumped today into what seems to be a mismatch of
signatures and versions.  I've tried several mirrors as the source for
my local copy, but I've been unable to find one that let's the client
cygwin installer work.
Instead, when running setup-x86.exe (the latest one, always downloaded
by the same script as the one running rsync), I get a "Mirror error:
Setup.ini signature (...) setup.bz2.sig from (...) failed to verify".

Thinking a bit more about it, I wonder what mechanisms, if any, are in
place to provide consistency and atomicity of the process?  Is there a
special time of the day when the "master copy" is frozen, the
signatures and md5sum are calculated, and a new version is released
for the mirrors to pick up?
Also, if the mirrors are simply updated via rsync, how do I make sure
that I am not myself rsyncing when they are in the midst of it?

I was curious about timestamps and md5sums on various mirrors (I
looked only in the US ones --I put the one-liner to do that as
post-scriptum). And indeed, I have found that different mirrors have
different signature files, and with various timestamps.

For example, as of 20140910-001532UTC, I find the following timestamps
for various setup.bz2.sig:
- 20140908-231037UTC mirrors-usa.go-parts.com_mirrors_cygwin/x86/setup.bz2.sig
- 20140909-191037UTC mirrors.syringanetworks.net_cygwin/x86/setup.bz2.sig
- 20140909-191037UTC mirrors.xmission.com_cygwin/x86/setup.bz2.sig
- 20140909-201044UTC mirror.team-cymru.org_cygwin/x86/setup.bz2.sig
- 20140909-231038UTC rsync.osuosl.org_cygwin/x86/setup.bz2.sig
- 20140910-001035UTC mirrors.kernel.org_sourceware_cygwin/x86/setup.bz2.sig

Worse, their md5sum don't match:

# grep -n -E 'bz2.sig$; */x86/md5.sum | perl -ne 'chomp;s/:/
/g;@a=split;print join("\t",@a[2,0])."\n"'
f88c73ee59cf59f7f32ca4bad26376c1
mirrors.kernel.org_sourceware_cygwin/x86/md5.sum
15a0cca932d75390e01b54a46b2558e2
mirrors.syringanetworks.net_cygwin/x86/md5.sum
f449fdb08944b8563c0ad273069669f1
mirrors-usa.go-parts.com_mirrors_cygwin/x86/md5.sum
15a0cca932d75390e01b54a46b2558e2        mirrors.xmission.com_cygwin/x86/md5.sum
413c8532019a41a1b310482cad0d25bb        mirror.team-cymru.org_cygwin/x86/md5.sum
6616150da358594eaa613032b66d2681        rsync.osuosl.org_cygwin/x86/md5.sum


Any suggestion or explanation?

Thanks!

~Pierre



PS: test all the signatures of the US-based rsync:// servers:

# mkcd ~/tmp/cygwin_signatures

# for i ($(perl -ne 's,/",",g;print "$1\n" while
(s,"rsync://(.*?)",,)' us_mirror_list.html)); do j=$i:gs,/,_; echo $i;
rsync -vaz --exclude 'release/' --progress --timeout 3600 rsync://$i/
$j/; done

# head -20 */x86/md5.sum
==> mirrors.kernel.org_sourceware_cygwin/x86/md5.sum <==
a7f25f8d488a30b68cf1c08f66ca35cb  setup.bz2
f88c73ee59cf59f7f32ca4bad26376c1  setup.bz2.sig
2728b16b23c2c6d697b5ae1fb3a4d936  setup.ini
4b0a55de2687c537124139bd569328ae  setup.ini.sig

==> mirrors.syringanetworks.net_cygwin/x86/md5.sum <==
a7f25f8d488a30b68cf1c08f66ca35cb  setup.bz2
15a0cca932d75390e01b54a46b2558e2  setup.bz2.sig
2728b16b23c2c6d697b5ae1fb3a4d936  setup.ini
d2d94f4e1734e896af4efa44e4c6654f  setup.ini.sig

==> mirrors-usa.go-parts.com_mirrors_cygwin/x86/md5.sum <==
a7f25f8d488a30b68cf1c08f66ca35cb  setup.bz2
f449fdb08944b8563c0ad273069669f1  setup.bz2.sig
2728b16b23c2c6d697b5ae1fb3a4d936  setup.ini
9ff8b9cfb0e29b9960ae918bb68a4dd6  setup.ini.sig

==> mirrors.xmission.com_cygwin/x86/md5.sum <==
a7f25f8d488a30b68cf1c08f66ca35cb  setup.bz2
15a0cca932d75390e01b54a46b2558e2  setup.bz2.sig
2728b16b23c2c6d697b5ae1fb3a4d936  setup.ini
d2d94f4e1734e896af4efa44e4c6654f  setup.ini.sig

==> mirror.team-cymru.org_cygwin/x86/md5.sum <==
a7f25f8d488a30b68cf1c08f66ca35cb  setup.bz2
413c8532019a41a1b310482cad0d25bb  setup.bz2.sig
2728b16b23c2c6d697b5ae1fb3a4d936  setup.ini
212b298441fdf11ee45ef3841dd38b62  setup.ini.sig

==> rsync.osuosl.org_cygwin/x86/md5.sum <==
a7f25f8d488a30b68cf1c08f66ca35cb  setup.bz2
6616150da358594eaa613032b66d2681  setup.bz2.sig
2728b16b23c2c6d697b5ae1fb3a4d936  setup.ini
75dd5f39d40c255951daf0ab5b865333  setup.ini.sig

# md5sum */x86/setup*
a7f25f8d488a30b68cf1c08f66ca35cb
mirrors.kernel.org_sourceware_cygwin/x86/setup.bz2
f88c73ee59cf59f7f32ca4bad26376c1
mirrors.kernel.org_sourceware_cygwin/x86/setup.bz2.sig
2728b16b23c2c6d697b5ae1fb3a4d936
mirrors.kernel.org_sourceware_cygwin/x86/setup.ini
4b0a55de2687c537124139bd569328ae
mirrors.kernel.org_sourceware_cygwin/x86/setup.ini.sig
a7f25f8d488a30b68cf1c08f66ca35cb
mirrors.syringanetworks.net_cygwin/x86/setup.bz2
15a0cca932d75390e01b54a46b2558e2
mirrors.syringanetworks.net_cygwin/x86/setup.bz2.sig
2728b16b23c2c6d697b5ae1fb3a4d936
mirrors.syringanetworks.net_cygwin/x86/setup.ini
d2d94f4e1734e896af4efa44e4c6654f
mirrors.syringanetworks.net_cygwin/x86/setup.ini.sig
a7f25f8d488a30b68cf1c08f66ca35cb
mirrors-usa.go-parts.com_mirrors_cygwin/x86/setup.bz2
f449fdb08944b8563c0ad273069669f1
mirrors-usa.go-parts.com_mirrors_cygwin/x86/setup.bz2.sig
2728b16b23c2c6d697b5ae1fb3a4d936
mirrors-usa.go-parts.com_mirrors_cygwin/x86/setup.ini
9ff8b9cfb0e29b9960ae918bb68a4dd6
mirrors-usa.go-parts.com_mirrors_cygwin/x86/setup.ini.sig
a7f25f8d488a30b68cf1c08f66ca35cb  mirrors.xmission.com_cygwin/x86/setup.bz2
15a0cca932d75390e01b54a46b2558e2  mirrors.xmission.com_cygwin/x86/setup.bz2.sig
2728b16b23c2c6d697b5ae1fb3a4d936  mirrors.xmission.com_cygwin/x86/setup.ini
d2d94f4e1734e896af4efa44e4c6654f  mirrors.xmission.com_cygwin/x86/setup.ini.sig
a7f25f8d488a30b68cf1c08f66ca35cb  mirror.team-cymru.org_cygwin/x86/setup.bz2
413c8532019a41a1b310482cad0d25bb  mirror.team-cymru.org_cygwin/x86/setup.bz2.sig
2728b16b23c2c6d697b5ae1fb3a4d936  mirror.team-cymru.org_cygwin/x86/setup.ini
212b298441fdf11ee45ef3841dd38b62  mirror.team-cymru.org_cygwin/x86/setup.ini.sig
a7f25f8d488a30b68cf1c08f66ca35cb  rsync.osuosl.org_cygwin/x86/setup.bz2
6616150da358594eaa613032b66d2681  rsync.osuosl.org_cygwin/x86/setup.bz2.sig
2728b16b23c2c6d697b5ae1fb3a4d936  rsync.osuosl.org_cygwin/x86/setup.ini
75dd5f39d40c255951daf0ab5b865333  rsync.osuosl.org_cygwin/x86/setup.ini.sig

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple