Re: [PATCH] Unbuffer stdout and stderr on windows

[Pedro Alves <palves at redhat dot com>]

On 08/16/2013 03:44 PM, Eli Zaretskii wrote:
>> Date: Fri, 16 Aug 2013 15:13:49 +0100
>> From: Pedro Alves <palves@redhat.com>
>> CC: gdb-patches@sourceware.org, brobecker@adacore.com, yao@codesourcery.com
>>
>>> Right now, the amount of such "broken"
>>> outputs is usually very small, and the fact that GDB flushes its
>>> stdout probably makes it negligibly small.  If we cause more of these
>>> instances to happen, the probability of a failure in a given session
>>> will go up.
>>
>> Not sure what you mean by causing more instances to happen?
> 
> I mean by that having more smaller chunks of text come out of GDB's
> end of the pipe.

Ack.  I can't say I really agree with this rationale.  The problem
exists, and hiding it under the rug won't make it go away.  Actually
forcing it to happen more frequently should make it easier to
iron out all the kinks.  IIUC, this is just a layer that
splits input into whole lines before passing it up to higher
levels, thus emulating line buffering.  I'm failing to see why
that is heuristic, rather than something that works 100% of the
time, given that that sounds just like fgets.

> 
>> It seems to me the mixing is a direct consequence of buffering.
>> With buffering of stdout and stderr, the runtime will push chunks
>> of lines to both stdout and stderr when the internal buffers are full,
>> with no regard to line endings.  IOW, if gdb does:
>>
>>  "line1\nline2\n" -> stdout
>>  "LINE3\nLINE4\n" -> stderr
>>
>> then depending on when is the internal stdout and stderr
>> buffers filled, the runtime may well flush it like:
>>
>> "line1\nliNE3\nLINE4\nn", and then "e2\n"
> 
> Yes, but since we flush stdout, the above shouldn't happen.

Clearly something like that must be happening, otherwise
the testsuite wouldn't have problems.  It just sounds like we
don't actually flush stdout _or_ stderr in all the places
we need to, although the existence of many flushes in the code
show that that's the intent of the existing solution to this problem.

I found this email from Joel at:

http://sourceware.org/ml/gdb-patches/2009-06/msg00447.html

Saying:

 "... the piece I quoted only
 unbuffers stdout and stderr, so that output sent on both file handles
 do not get printed out of order (in other words, if we print on
 stderr first, and then stdout, we want the output to be in that
 order - with buffering, we observed that stderr output was printed
 after stdout output, even if the actual call to printf was in a
 different order)."

So that actually supports my theory, and, my earlier suggestion
that if fully unbuffered is undesirable, then instead make sure to
always flush stdout before switching output to stderr, and vice
versa, so the order is preserved.  That should fix the issue, and,
avoid the potential (though I'm not sure how real) drop of
performance with going fully unbuffered, and, potentially fixes
the issue for everyone, testsuite and frontends included.
The question, IMO, if it the potential performance hit of going
fully unbuffered is worth it, but I still think that if yes,
the performance issue is real, then the fix should be a generic
one, rather then a testsuite hack.  I don't imagine such a fix to
be a big patch.  It could e.g., go somewhere around fputs_unfiltered
or some such.

>>> Maybe we should dig deeper in the original problem as well, because I
>>> still have only a very vague notion of why would GDB, which is a
>>> single-threaded program, cause mixing when it flushes stdout
>>> regularly.  What am I missing?
>>
>> I think so too.  A paste of the mixed output would be good
>> to start.
> 
> Agreed.
> 

-- 
Pedro Alves