This is the mail archive of the libc-alpha@sourceware.org mailing list for the glibc project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

vfprintf typing problem


When executing stdio-common/bug22.c (which makes sure we fail properly
for huge format widths which end up overflowing INT_MAX), it fails
differently on 32-bit vs 64-bit.

And actually it's dependent upon a whole bunch of things, and most of
the time the test doesn't actually exercise the bug in question at
all.

The problem lies in two things:

1) The amount of address space available.

2) The type of the variable used to track the field width.

First of all, on 64-bit, the malloc() to hold the work buffer will
fail because 'width' is an int and will have the sign bit set, this
gets extended to 64-bits in the malloc() call which results in trying
to allocate a HUGE malloc buffer which fails.

But even on 32-bit, the ability to allocate a 2GB region of memory is
not all that guarenteed.  So the malloc() call can fail for this
reason too.

If the stars align and we get past the above issues, this test just
crunches in a loop in xsputn and padn, writing 2GB of padding out to
/dev/null, just so we can get past the first huge field to the second
one to trigger the field width overflow past INT_MAX.

Anyways, the main thing I want to address is the type used to store
the field width so that it doesn't get improperly extended for the
malloc() call.

Ideally it should be size_t or similar, but this would mean adjusting
all of the code to fetch the width from the format specifier or the
varargs, all of which work with int pointers.

Another option is to keep the type as 'int', but abort with an error
earlier, say as soon as the sign bit is set in the width.

Maybe the latter is the best option.

Comments?


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]