Re: [PATCH] vfprintf stack overflow [BZ #16617]

[Rich Felker <dalias at libc dot org>]

On Fri, Dec 05, 2014 at 10:42:30PM +0000, Joseph Myers wrote:
> On Fri, 5 Dec 2014, Paul Eggert wrote:
> 
> > On 12/05/2014 12:26 PM, Rich Felker wrote:
> > > If N is the size of an actual allocated object, 2*N should not be able
> > > to overflow. If it can, it means you already have a situation where an
> > > object is so large that legal pointer subtractions overflow ptrdiff_t
> > No, because the array elements are of type struct printf_spec, which is
> > several bytes in size.  So even if the number of bytes in the array exceeds
> > PTRDIFF_MAX by a factor of (say) eight, subtracting addresses of array
> > elements won't overflow.
> 
> Such subtraction of pointers differing by more than SIZE_MAX / 2 bytes 
> does not actually work in GCC (it does a subtraction, which overflows, 
> then a division - and that approach is fine for all normal arguments).
> 
> I think malloc should in principle disallow allocations of more than 
> SIZE_MAX / 2 bytes, but right now it doesn't, and I suspect a change would 
> break compatibility for various existing applications that expect to do 
> > 2GB allocations on 32-bit systems.

Yes, malloc (and mmap and shmget/shmat) should disallow allocations
larger than SIZE_MAX/2 for exactly this reason. I doubt it would break
things fixing it. Such a large malloc is unlikely to work on 32-bit
systems anyway, given than 1GB is taken by the kernel and the
placement of the main program, stack,dynamic linker, and shared libs
already fragments the address space a bit. You might get lucky if it
happens very early in program lifetime in a program without lots of
libs, but I think relying on >2GB malloc on a 32-bit system is already
fragile.

Rich