This is the mail archive of the
libc-help@sourceware.org
mailing list for the glibc project.
Persistent malloc
- From: Adrian May <adrian dot alexander dot may at gmail dot com>
- To: libc-help at sourceware dot org
- Date: Tue, 7 Nov 2017 19:56:30 +0000
- Subject: Persistent malloc
- Authentication-results: sourceware.org; auth=none
Hi All,
I just managed to make a persistent malloc by replacing MORECORE. My init
routine opens a file and maps it to a huge space at a fixed address, then I
just ftruncate the file when somebody asks my MORECORE for more:
void * myMoreCore(ptrdiff_t s) {
˲ void * ret = heap + HEADSIZE + heapsize; //HEADSIZE is some up-front
stuff in the file
˲ printf("Asked for %x when got %x\n", s, heapsize);
˲ if (heapsize<s) {
˲ ˲ size_t newsize = ((s-1)/HEAPSTEP+1)*HEAPSTEP; //That's 4096
˲ ˲ ftruncate(fd, HEADSIZE+newsize);
˲ ˲ heapsize = newsize;
˲ ˲ printf("Resized heap file\n");
˲ }
˲ return ret; //Yes it should be the old end+1
}
void openHeap() {
˲ __morecore = myMoreCore; //Hook it up
˲ struct stat st;
˲ int exists=(stat(MAPFILE, &st)==0); //We'll zero the head if the file is
new
˲ if (exists) heapsize=st.st_size-HEADSIZE;
˲ fd = open(MAPFILE, O_RDWR | O_CREAT, S_IRWXU);
˲ if (fd == -1) { printf("Can't open heap file %s\n", MAPFILE); die(); }
˲ if (!exists) {
˲ ˲ heapsize=0;
˲ ˲ ftruncate(fd, HEADSIZE+heapsize);
˲ }
˲ heap = mmap(0x300000000000, HEADSIZE+HEAPMAX, PROT_READ|PROT_WRITE,
MAP_SHARED, fd, 0);
˲ if (heap ==(void*) -1) { printf("Failed to map heap file %s\n", MAPFILE);
die(); }
˲ if (!exists) memset(heap, 0, HEADSIZE);
˲ printf("Opened heap file at %p\n", heap);
}
It worked fine until I tried it from multiple threads, at which point I
fell foul of the arena-per-thread thing.
It's not obvious to me from the sources how I could switch that off. I saw
tunables called "arena test" and "arena max" but I'm not sure what they do.
In _libc__malloc the per-thread fetching of the arena looks hard wired as a
macro. I was hoping not to have to recompile libc incidentally.
Then again, what I really want is a pmalloc and pfree function so I can
specifically request persistent allocation but default to the existing
volatile per-thread stuff.
Could I just declare an empty malloc_state and pass it to int_malloc from
pmalloc? Trouble is, in sysmalloc (dunno if that's relevant or not) I see
(av != &main_arena) choosing whether or not to call MORECORE at all.
Is there any way to unravel these hard coded bits? Could I arrange for only
pmalloc to use the main arena while the main thread has its own per-thread
arena? Or is there some different pre-existing concept for how this kind of
thing should be done?
TIA, Adrian.