This is the mail archive of the cygwin@cygwin.com mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Rsync performance increase through buffering (cygwin/ssh solution?)


Below is a description of a buffering speed improvement.
The original message with complete patch for performance can be found here:
http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&newwindow=1&selm=at1i0e%241tj3%241%40FreeBSD.csie.NCTU.edu.tw

This may explain the horrendous performance of rsync
with ssh under cygwin. Perhaps the way that cygwin
emulates these calls. It is especially bad under the
circumstances mentioned here - with a large sync of data
where the files are identical (-I).
The hang fix has been identified elsewhere in cygwin:
Check out these links:
http://www.cygwin.com/ml/cygwin/2002-10/msg00308.html
http://sources.redhat.com/ml/cygwin/2002-09/msg01155.html
patched rsync for hang under cygwin:
http:// www.addr.com/~dmitry/rsync.exe

Jim

craig@atheros.com (Craig Barratt) wrote in message news:<at1i0e$1tj3$1@FreeBSD.csie.NCTU.edu.tw>...
> I've been studying the read and write buffering in rsync and it turns
> out most I/O is done just a couple of bytes at a time. This means there
> are lots of system calls, and also most network traffic comprises lots
> of small packets. The behavior is most extreme when sending/receiving
> file deltas of identical files.
>
> The main case where I/O is buffered is writes from the server (when
> io multiplexing is on). These are usually buffered in 4092 byte
> chunks with a 4 byte header. However, reading of these packets is
> usually unbuffered, and writes from the client are generally not
> buffered. For example: when receiving 1st phase checksums (6 bytes
> per block), 2 reads are done: one of 4 bytes and one of 2 bytes,
> meaning there are 4 system calls (select/read/select/read) per 6
> bytes of checksum data).
>
> One cost of this is some performance, but a significant issue is that
> unbuffered writes generate very short (and very many) ethernet packets,
> which means the overhead is quite large on slow network connections.

... see this link for the rest:
http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&newwindow=1&selm=at1i0e%241tj3%241%40FreeBSD.csie.NCTU.edu.tw




--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Bug reporting: http://cygwin.com/bugs.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]