Bug 12201

Summary: Fix setrlimit/getrlimit 32-bit platforms for limits > 2^32-1
Product: glibc Reporter: Michael Kerrisk <mtk.manpages>
Component: libcAssignee: Ulrich Drepper <drepper.fsp>
Status: RESOLVED FIXED    
Severity: normal Flags: fweimer: security-
Priority: P2    
Version: 2.12   
Target Milestone: ---   
Host: Target:
Build: Last reconfirmed:

Description Michael Kerrisk 2010-11-07 15:01:50 UTC
The current glibc wrappers for getrlimit/setrlimit do some work to paper over this kernel bug:
https://bugzilla.kernel.org/show_bug.cgi?id=5042
With the kernel 2.6.36 addition of the prlimit(2) syscall, the wrappers could be written to use prlimit() and thus avoid the problem resulting from the broken kernel syscalls.
Comment 1 Ulrich Drepper 2010-11-08 20:16:22 UTC
What exactly are you talking about?  We are using ugetrlimit for some time.  At the same time setrlimit was changed.  These syscalls now match exactly the required userlevel semantic.
Comment 2 Michael Kerrisk 2010-11-11 05:33:45 UTC
(In reply to comment #1)
> What exactly are you talking about?

It was fairly carefully explained in the kernel.org bug report...

> We are using ugetrlimit for some time.  At
> the same time setrlimit was changed.  These syscalls now match exactly the
> required userlevel semantic.

No, they do not. Look at the following.

$ cat rlimit_large.c 
/* rlimit_large.c
*/
#define _FILE_OFFSET_BITS 64
#include <sys/resource.h>
#include <sys/types.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <errno.h>

#define errMsg(msg)             { perror(msg); }

#define errExit(msg)            { perror(msg); exit(EXIT_FAILURE); }

static void
printRlimit(const char *msg, int resource)
{
    struct rlimit rlim;

    if (getrlimit(resource, &rlim) == -1) errExit("getrlimit");

    printf("%s soft=", msg);
    if (rlim.rlim_cur == RLIM_INFINITY)
        printf("infinite");
    else if (rlim.rlim_cur == RLIM_SAVED_CUR)
        printf("unrepresentable");
    else
        printf("%lld", (long long) rlim.rlim_cur);

    printf("; hard=");
    if (rlim.rlim_max == RLIM_INFINITY)
        printf("infinite\n");
    else if (rlim.rlim_max == RLIM_SAVED_MAX)
        printf("unrepresentable");
    else
        printf("%lld\n", (long long) rlim.rlim_max);
} /* printRlimit */

int
main(int argc, char *argv[])
{
    struct rlimit rl;
    printf("sizeof(rl.rlim_cur) = %ld\n", (long) sizeof(rl.rlim_cur));

    /* Show the value of the glibc constant, just for interest */

    printf("RLIM_INFINITY = %llx\n", (unsigned long long) RLIM_INFINITY);

    if (getrlimit(RLIMIT_FSIZE, &rl) == -1) errExit("setrlimit");
    printRlimit("Initial RLIMIT_FSIZE limits          : ", RLIMIT_FSIZE);

#define VAL1 10000
    rl.rlim_cur = VAL1;
    printf("About to set rlim_cur to %lld\n", (long long) rl.rlim_cur);
    if (setrlimit(RLIMIT_FSIZE, &rl) == -1) errExit("setrlimit");
    if (getrlimit(RLIMIT_FSIZE, &rl) == -1) errExit("setrlimit");
    printRlimit("RLIMIT_FSIZE limits after setrlimit(): ", RLIMIT_FSIZE);

#define VAL2 4999222333LL
    rl.rlim_cur = VAL2;
    printf("About to set rlim_cur to %lld\n", (long long) rl.rlim_cur);
    if (setrlimit(RLIMIT_FSIZE, &rl) == -1) errExit("setrlimit");
    if (getrlimit(RLIMIT_FSIZE, &rl) == -1) errExit("setrlimit");
    printRlimit("RLIMIT_FSIZE limits after setrlimit(): ", RLIMIT_FSIZE);

    exit(EXIT_SUCCESS);
} /* main */

$ cc rlimit_large.c 
$ ./a.out
sizeof(rl.rlim_cur) = 8
RLIM_INFINITY = ffffffffffffffff
Initial RLIMIT_FSIZE limits          :  soft=infinite; hard=infinite
About to set rlim_cur to 10000
RLIMIT_FSIZE limits after setrlimit():  soft=10000; hard=infinite
About to set rlim_cur to 4999222333
RLIMIT_FSIZE limits after setrlimit():  soft=infinite; hard=infinite


The last line of output indicates the problem: a limit > (2^32)-1 got turned into RLIM_INFINITY, even though it is representable in 8 bytes. It's a kernel problem, but glibc's wrappers could now workaround that kernel problem by using prlimit().
Comment 3 Ulrich Drepper 2010-11-11 09:09:37 UTC
If you would only learn to express yourself correctly and concisely you could save yourself work.

Why on earth would you say setrlimit has problems if you really mean setrlimit64?  That's not at all the same code.
Comment 4 Ulrich Drepper 2010-12-26 01:13:06 UTC
I checked in a patch.
Comment 5 Jackie Rosen 2014-02-16 18:24:37 UTC Comment hidden (spam)