This is the mail archive of the
cygwin
mailing list for the Cygwin project.
Re: 1.7.0 CVS mmap failure
On Jan 9 01:04, Christopher Layne wrote:
> On Sun, Jan 07, 2007 at 11:58:44AM +0100, Corinna Vinschen wrote:
> > Lots of comments throughout the file...
>
> Unfortunately the code-path is less than clear to follow. This may be
> a matter of opinion but it's fairly complex and looks to have history in
> it.
Sure enough :)
> > The strace is rather useless without the (hopefully very short) source
> > code of the (hopefully very small) testcase.
>
> Okay, I understand where you're coming from. Where I'm coming from is that it
> is difficult to generate a test case that actually demonstrates the issue
> outside of the scope of my application. Suffice to say, I do see the errors in
> strace output - and I can see MapViewNT returning an error in the double-map
> -so-we-can-have-4k-pseudopages version.
I'm under the impression you're getting something upside-down. Windows
has 4K system pages, but an allocation granularity of 64K. For a long
time I stubbornly tried to stick with 4K pagesize for POSIX applications,
but the code got more and more complicated because there was always yet
another situation which just didn't work correctly due to the 64K
allocation granularity.
So I decided to give up and change the POSIX-side pagesize to 64K
which really simplified mmap. However, there's this one problem
left: While you can only allocate usefully in 64K steps, the size of
the memory area allocated for a file is only 4K aligned, thus leaving
((roundup(filesize,64K) - roundup(filesize,4K)) / 4K) pages in the last
64K chunk of a file mapping unmapped. The code you quote below is
adding a 4K aligned anonymous mapping attached to the file mapping so
that the POSIX application can access the whole 64K page.
> Do you have a 4GB machine running w/
> PAE enabled in which to even test a test case if I could come up with one which
> consistently works?
No.
> Also, VirtualProtect fails on unmap everytime, a test case
> for this is a matter of any old map/unmap it seems.
I see what happens. I applied a patch for that.
> 1247 if (orig_len)
> 1248 {
> 1249 /* If the requested length is bigger than the file size, the
> 1250 remainder is created as anonymous mapping. Actually two
> 1251 mappings are created, first the reminder from the file end to
> 1252 the next 64K boundary as accessible pages with the same
> 1253 protection as the file's pages, then as much pages as necessary
> 1254 to accomodate the requested length, but as reserved pages which
> 1255 raise a SIGBUS when trying to access them. AT_ROUND_TO_PAGE
> 1256 and page protection on shared pages is only supported by 32 bit NT,
> 1257 so don't even try on 9x and in WOW64. This is accomplished by not
> 1258 setting orig_len on 9x and in WOW64 above. */
> 1259 orig_len = roundup2 (orig_len, pagesize);
> 1260 len = roundup2 (len, getsystempagesize ());
> ^^^^^^^^^^^^^^^^^
> Why? This also violates the "to the next 64k boundary as accessible pages" part.
> The pagesize is 64k, the granularity for addressing is 4k. Don't see why 4k has
> to be mixed in here by calling getsystempagesize() or am I totally missing
> something here?
Yes. The pagesize is 4K in Windows, the allocation granularity is 64K.
orig_len is what should have been allocated considering 64K POSIX pages,
len is what the system actually allocated using 4K Windows pages.
(orig_len - len) is the size of the yet to be allocated anonymous
mapping, which consists of up to two parts. First, the filler which
is normally accessible to the application, called valid_page_len here:
> 1264 orig_len -= len;
> 1265 size_t valid_page_len = orig_len % pagesize;
which is the above mentioned
((roundup(filesize,64K) - roundup(filesize,4K)) / 4K)
Second, the remainder which is handled by a SIGBUS area:
> 1266 size_t sigbus_page_len = orig_len - valid_page_len;
which is (roundup(requested_allocation_size,64K) - (roundup(filesize,64K)))
> " The system shall always zero-fill any partial page at the end of
> an object. Further, the system shall never write out any
> modified portions of the last page of an object which are beyond
> its end. References within the address range starting at pa
> and continuing for len bytes to whole pages following the end of an
> object shall result in delivery of a SIGBUS signal.
> "
Yes, exactly. The "partial page at the end of an object" is what is
named "valid_page_len" above plus the tiny reminder within the last 4K
page of the Windows file mapping.
> Which I would expect to be: |0 <len> 65536| on Windows.
I don't understand this sentence here. What is expected to be 64K?
Anyway, I though the comments in the source should have made this
all clear already.
I don't see anything unusual happen in your small testcase. Looks
good to me. Why do you expect to be unable to read pages beyond the
end of the mapping? Per se you don't know anything about the memory
layout. Any other part of the application including the standard
libs and the OS can have pages allocated at this point.
> $ ./mmt mmt.c w
> psz = 65536
> st.st_size == 1309
> 4093 4094 4095 4096 Segmentation fault (core dumped)
> (what? not 64k?)
>
> Yet the latter works for me, whereas the former results in mmap()
> failures for files smaller than the page size.
The latter works for you because it's wrong. Since it uses 64K, mmap
thinks it doesn't have to add filler pages. So adding the filler pages
can't go wrong. So you can access your file, but you get the SEGV
exaclty after the last 4K page of the file, not at the end of the
expected 64K page.
Corinna
--
Corinna Vinschen Please, send mails regarding Cygwin to
Cygwin Project Co-Leader cygwin AT cygwin DOT com
Red Hat
--
Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple
Problem reports: http://cygwin.com/problems.html
Documentation: http://cygwin.com/docs.html
FAQ: http://cygwin.com/faq/