Date: Fri, 12 Oct 2012 09:25:34 -0700 From: Marcel Moolenaar <marcel@xcllnt.net> To: "freebsd-arch@freebsd.org Arch" <freebsd-arch@freebsd.org> Cc: Tim LaBerge <tlaberge@juniper.net>, Alan Cox <alc@rice.edu> Subject: Behavior of madvise(MADV_FREE) Message-ID: <9FEBC10C-C453-41BE-8829-34E830585E90@xcllnt.net>
next in thread | raw e-mail | index | archive | help
All,
Juniper has been intrigued for a while about the beahviour
of madvise(MADV_FREE). Let me give a bit of context before
asking questions:
1. We have an important daemon that needs lots of memory.
It uses sbrk()/brk() to extend its address space and
uses madvise(MADV_FREE) to inform the kernel when a
chunk of memory is effectively unused (the chunk is
first zeroed).
2. Most of the time memory usage of the daemon is pretty
stable, but under certain conditions it can spike,
after which it drops back to a new stability point
(either higher or lower).
3. Obviously the daemon is not the only component in the
system, so whatever it doesn't need we very likely
need somewhere else -- badly in some cases, so we do
like immediate recycling then.
Now on to the questions:
1. madvise(MADV_FREE) marks the pages as clean and moves
them to the inactive queue. Why isn't the reference
state cleared on either the page or the TLB?
2. Why aren't the pages moved to the cache queue in the
first place?
3. What would be the impact or consequence of changing
the behaviour of madvise(MADV_FREE) to mark the page
as clean and unreferenced and have the page moved to
the cache queue (or free queue even)?
Ad 1:
When the system is under memory pressure, the pageout
daemon scans the inactive queue in order to try to move
pages to the cache or free queue. With the MADV_FREE'd
pages still having PG_REFERENCE or the underlying TLBs
still having the access flag set, these pages actually
get bumped to the active queue.
Ad 2:
MADV_DONTNEED is there to signal that the pages contain
valid data, but that the page is not needed right now.
Using this, pages get moved to the inactive queue. That
makes sense. But MADV_FREE signals that there's no valid
data anymore and that the page may be demand zeroed on
next reference. The page is not inactive. It's free. If
the paged was zeroed before calling MADV_FREE, the page
really caches contents that that can be recreated later
(the demand zero).
Thanks,
--
Marcel Moolenaar
marcel@xcllnt.net
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9FEBC10C-C453-41BE-8829-34E830585E90>