[Bug libc/17523] New: open() and openat() ignore 'mode' with O_TMPFILE

["e at nanocritical dot com" <sourceware-bugzilla at sourceware dot org>]

https://sourceware.org/bugzilla/show_bug.cgi?id=17523

            Bug ID: 17523
           Summary: open() and openat() ignore 'mode' with O_TMPFILE
           Product: glibc
           Version: 2.21
            Status: NEW
          Severity: normal
          Priority: P2
         Component: libc
          Assignee: unassigned at sourceware dot org
          Reporter: e at nanocritical dot com
                CC: drepper.fsp at gmail dot com

Created attachment 7863
  --> https://sourceware.org/bugzilla/attachment.cgi?id=7863&action=edit
Test case

As the attached test case demonstrates (tested on glibc 2.20, Linux 3.13.5),

  open("/tmp", O_TMPFILE | O_RDWR, 0600) works
  openat(AT_FDCWD, "/tmp", O_TMPFILE | O_RDWR, 0600) does NOT work
  syscall(SYS_openat, AT_FDCWD, "/tmp", O_TMPFILE | O_RDWR, 0600) works

strace shows that the call to openat() fails with -1, EACCES

Note that if mode is 0,
  open("/tmp", O_TMPFILE | O_RDWR, 0) also fails with -1, EACCESS

Looking at the glibc source, it turns out that both open() and openat() are
incorrect: they load the 'mode' argument lazily, using va_arg() only if
O_CREAT is found in 'oflag'. This is wrong, 'mode' can also be used if
O_TMPFILE is in 'oflag'.

  if (oflag & O_CREAT)
    {
      va_list arg;
      va_start (arg, oflag);
      mode = va_arg (arg, int);
      va_end (arg);
    }

It is likely that no one has seen the problem until now because open()
gets away with it: indeed, at least on amd64, 'mode' is the 3rd argument to
open() and is in RDX. When open() gets to the SYSCALL instruction, even if
it skips va_arg(), 'mode' is still in RDX, where the syscall calling
convention expects it.

But openat() is not so lucky: 'mode' is the 4th argument, which is RCX. When
openat() get to SYSCALL, 'mode' is still in RCX, BUT the syscall calling
convention on amd64 requires that the 4th argument be in R10, which is 0 in
the test case.

Patch will follow.

Now, it is not clear to me why the kernel refuses to create an unamed
temporary file with mode 0, but that's a different bug.

-- 
You are receiving this mail because:
You are on the CC list for the bug.