Date: Thu, 23 Sep 1999 09:54:37 -0700 (PDT) From: Matthew Dillon <dillon@apollo.backplane.com> To: Kevin Day <toasty@dragondata.com> Cc: toasty@dragondata.com (Kevin Day), dcs@newsguy.com (Daniel C. Sobral), hackers@FreeBSD.ORG Subject: Re: Idea: disposable memory Message-ID: <199909231654.JAA28326@apollo.backplane.com> References: <199909231433.JAA61714@celery.dragondata.com>
next in thread | previous in thread | raw e-mail | index | archive | help
:I'm now playing with compressed data streams. The decompression is slow, so
:I'd like to cache the *decompressed* version of these files. I end up
:allocating large amounts of ram in one process to cache the decompressed
:data. This is a disavantage over the above scenario, since now the system
:swaps out my decompressed data when more ram is needed elsewhere. Swapping
:out then swapping back in my decompressed data is about 4x slower than just
:re-reading my compressed stream and decompressing it again.
:
:Why don't I just allocate a predefined amount of memory and use that for a
:cache all the time? Most of the time we have about 20MB free on our system.
:Sometimes we end up with about 2MB free though, and what's happening now is
:that I start paging out data that I could recreate in less time than the
:page-in/page-out takes.
Hmm. Well, you can check whether the memory has been swapped out with
mincore(), and then MADV_FREE it to get rid of it (MADV_FREE'ing something
that has been swapped out frees the swap and turns it back into zero-fill).
That doesn't get rid of the swapout bandwidth, though.
I think, ultimately, you need to manage the memory used for your cache
manually. That means using mlock() and munlock() to lock your cache into
memory. For example, choose a cache size that you believe the system
can support without going bonkers, like 5MB. mmap() 5MB of ram and
mlock() it into memory. From that point on until you munlock() it or
exit, the memory will not be swapped out.
If the purpose of the box is to maintain the flow of video, then the cache
is a critical resource and should be treated as such.
-Matt
Matthew Dillon
<dillon@backplane.com>
:Kevin
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199909231654.JAA28326>