Re: [PATCH] libio: Always use _IO_BUFSIZE for stream buffers [BZ #4099]

[Rich Felker <dalias at libc dot org>]

On Fri, Apr 01, 2016 at 09:15:21PM -0400, Carlos O'Donell wrote:
> On 04/01/2016 02:19 PM, Rich Felker wrote:
> > On Fri, Mar 18, 2016 at 03:52:58PM -0700, Roland McGrath wrote:
> >>> I can do some benchmarking, but I don't expect any compelling results.
> >>
> >> Whatever the results, they would not IMHO be relevant here.
> >>
> >> POSIX specifies that st_blksize is the "preferred I/O block size for this
> >> object".  It's the kernel's responsibility to give userland good advice
> >> through this channel.  If there are common buggy kernels that give bad
> >> advice, that is a reason to apply upper and lower limits to the advice from
> >> the kernel.  But the expectation should be that the kernel gets fixed to
> >> give good advice, and the optimal thing to do with a good kernel is to
> >> follow its advice.  
> >>
> >> Since the recommended use of st_blksize in this way is a standard user
> >> feature and not just what stdio's implementation happens to do, there is an
> >> argument to be made that the limiting of the value should be done in the
> >> *stat functions reported st_blksize values rather than in stdio's use of
> >> them.  (I'm ambivalent about this point.)
> > 
> > Regardless of st_blksize being "the preferred size", it's not suitable
> > for stdio, at least not for read purposes, because sparse/random
> > access reads are a valid application usage for stdio. Reading an
> > unboundedly large "optimal" block size, only to use one byte and throw
> > the rest away, is unacceptably pessimistic behavior and is the whole
> > point behind bug 4099.
> > 
> > If you insist on keeping unboundedly large buffers honoring
> > st_blksize, one option would be to only use the full buffer for
> > writing, and limit it to 4k or 8k for reading. But I think it's best
> > to just ignore st_blksize and use a reasonable buffer size all the
> > time.
> 
> I think Roland has a good point though, if the kernel is going to buffer
> for you using filesystem-based knowledge, then why doesn't it just report
> an st_blksize that's small, say 8192 bytes, given the implementation?
> What purpose does it serve to set st_blksize to 2MB?

Oh, I totally agree that the kernel is being stupid here. But it's
also stupid for glibc to honor values that obviously do not make sense
for the usage case at hand (reading when you can't know whether you'll
throw most of the result away).

Rich