This is the mail archive of the
libc-alpha@sourceware.org
mailing list for the glibc project.
Re: Question about madvise(DONTNEED) in glibc malloc
- From: OndÅej BÃlka <neleai at seznam dot cz>
- To: KOSAKI Motohiro <kosaki dot motohiro at gmail dot com>
- Cc: Siddhesh Poyarekar <siddhesh dot poyarekar at gmail dot com>, libc-alpha <libc-alpha at sourceware dot org>
- Date: Mon, 15 Apr 2013 11:51:42 +0200
- Subject: Re: Question about madvise(DONTNEED) in glibc malloc
- References: <516ADB3C dot 9040805 at gmail dot com> <CAAHN_R2FK4Fj4u1hHJJ17fr2X_PJxDs+6h2azWbUzbZth2HdfQ at mail dot gmail dot com> <516B535A dot 4080104 at gmail dot com> <CAAHN_R2Y7+eiBAqo1mX6pbos8yN9=dQwcjz6eiLhZy8vTfUwRw at mail dot gmail dot com> <516B8C73 dot 9030308 at gmail dot com>
On Sun, Apr 14, 2013 at 10:13:23PM -0700, KOSAKI Motohiro wrote:
> >>> MADV_DONTNEED
> >>> Do not expect access in the near future. (For the time
> >>> being, the application is finished with the given range, so
> >>> the kernel can free resources associated with it.) Subse-
> >>> quent accesses of pages in this range will succeed, but
> >>> will result either in re-loading of the memory contents
> >>> from the underlying mapped file (see mmap(2)) or zero-fill-
> >>> on-demand pages for mappings without an underlying file.
> >>
> >> Current implemantation cleary does as document. It's not a bug and we don't plan to
> >> change never.
> >
> > I didn't imply that it's any kind of bug in the kernel either.
> >
> > Why can't we extend the TRIM_THRESHOLD to non-main arenas? We
> > sacrifice the extra address space we could have given back to the
> > kernel by not getting rid of a heap when we could, but that's a much
> > better compromise IMO.
>
> Yeah, I haven't consider it. but it can, I think.
>
>
> > I agree with you when you say that the pages we just gave back with
> > MADV_DONTNEED are the ones most likely to be needed when we decide to
> > extend the heap next and in that context, it is a suboptimal use of
> > MADV_DONTNEED. However, the idea here is to release resources
> > periodically to avoid monopolizing it. Maybe we could always only
> > trim half of TRIM_THRESHOLD and keep the other half as a cache. Given
> > the semantics of dynamic thresholds, we would effectively be keeping a
> > cache for the next expected allocation size - the TRIM_THRESHOLD is
> > dynamically set to twice of MMAP_THRESHOLD on freeing of a mapped
> > location.
>
> That's definitely better than current. but I'm incline to solve the issue
> completely by introducing new kernel feature.
>
What having at kernel keep list of pages that are currently unused. On memory
shortage kernel reclaims them. When minor page fault occurs we do not
need to zero page but remap one of unused.