This is the mail archive of the cygwin mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Antivirus strikes back (probably) (Was: Intermittent failures retrieving process exit codes)


On 2013-11-26 Denis Excoffier wrote:
> On 2013-11-25 Ã 21:58 +02:00, Lasse Collin wrote:
> > If you haven't already tried, please compile both 5.1.2alpha and
> > 5.1.3alpha from source while keeping everything else unchanged, and
> > see if the bug really only occurs with 5.1.3alpha.
> Already done. I did some strace-ing, and since iâm not so fluent with
> the result, iâll send it there in a while (when iâm back on cygwin)
> if someone is interested. But the bug (contrary to what i said
> before) also _sometimes_ occurs with 5.1.2alpha or 5.0.5 and this
> makes me think now that:
> 
> a) my antivirus-anti-intrusion-whatever-software (that i canât remove
> of course) creates some kind of "background noise" where a certain
> percentage of such âtar xf âuse-compress-programâ commands will
> always fail
> 
> b) nevertheless, xz-5.1.3alpha (with its new file I/O code etc.)
> triggers some untypical configuration inside the antivirus that
> increases drastically the percentage, making the failure almost
> certain for some files.
> 
> It is not extraordinary that i cannot observe the failure on XP since
> i do not have this particular antivirus on XP.

OK, so the new I/O code in xz probably isn't the problem even if it may
affect how easily the actual problem gets triggered.

[...]
> When it fails (usually or pathologically), the last file of the
> archive gets truncated (see above), and _this_ is strange from an
> antivirus behaviour. After all, perhaps some flush() or similar is
> missing inside 5.1.3alpha.

xz uses write() which uses a file descriptor argument, so there is
nothing to flush separately. xz just has to write() everything.

When used with tar, xz writes to standard output (FILENO_STDOUT) which
with tar is a pipe. When xz finishes, it closes its end (the writer end)
of the pipe.

With xz 5.1.3alpha, O_NONBLOCK flag is set for FILENO_STDIN and
FILENO_STDOUT if the flag wasn't already set. If xz set the flag, it
will unset it before closing the file descriptor. The setting and
unsetting can be seen in the trace you sent and it seems to work
correctly. I don't have a guess if these fcntl() calls might cause the
difference between 5.1.3alpha and other versions, but it doesn't sound
too important since the bug occurs in some form with all versions.

From the trace file it seems that the last write() from xz gets lost.
xz first makes 173 writes of 8192 bytes and then one 6144-byte write,
totalling 1,423,360 bytes. tar gets 1,417,216 from xz, that is, 6144
bytes too little.

Since things go wrong with old xz versions that don't use non-blocking
I/O, I would expect you to see similar issues with other compressors
too. Maybe it would be worth testing with gzip and bzip2 in the same
way you did with xz 5.0.5.

-- 
Lasse Collin  |  IRC: Larhzu @ IRCnet & Freenode

--
Problem reports:       http://cygwin.com/problems.html
FAQ:                   http://cygwin.com/faq/
Documentation:         http://cygwin.com/docs.html
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]