Date: Wed, 03 Oct 2001 22:13:09 -0700 From: Peter Wemm <peter@wemm.org> To: Matt Dillon <dillon@FreeBSD.org> Cc: cvs-committers@FreeBSD.org, cvs-all@FreeBSD.org Subject: Re: cvs commit: src/sys/kern vfs_cache.c vfs_subr.c src/sys/sys vnode.h Message-ID: <20011004051309.BFAF23808@overcee.netplex.com.au> In-Reply-To: <200110010433.f914XZd11860@freefall.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Matt Dillon wrote:
> dillon 2001/09/30 21:33:35 PDT
> I have also determined through testing that the real problem with numvnodes
> getting too large is due to the VM Page cache preventing the vnode from
> being reclaimed. The directory stuff made only a tiny dent relative
> to Poul's original code, enough so that some tests succeeded. But tests
> with several million small files show that the bigger problem is the VM
> Page cache. This will have to be addressed by a future commit.
I ran into this problem several times in the last few days trying to copy
a 20G file tree from a raid to a disk and back again. I ended up "solving"
the problem by having a script that did this:
cd /home/cd/freebsd-iso/
while 1
tar cvf /dev/null .
sleep 300
end
This read about 2.7GB of freebsd cdrom .iso's and flused the VM page cache.
This caused about 260,000 vnodes to be freed shortly thereafter instead of
running out of ffsnodes. The machine had 3G of ram at the time and it
locked up every time I tried to copy the archive. It was a RELENG_4 box
without vmiodirenable (but turning it on made no difference).
I believe this is the same problem that we're seeing at yahoo.
On this particular box, while it has been idle for a while:
197126 desiredvnodes
214634 numvnodes
5158 freevnodes
peter@thunder[10:02pm]~-102> vmstat -m | grep ' FFS'
FFS node214626 53657K 53657K102400K 3268258 0 0 256
ie: it has used over 50% of its M_FFSNODE limit.
This is probably over-simplistic, but can we more aggressively reclaim
older vnodes (and consequently freeing their pages) and recycle them when
we're over the desiredvnodes limit? Having to blow away the enire VM page
cache in order to make the vnodes recyclable is a bit silly. And even then
they are never "freed", they just go into the free list but keep their
associated ffs inode etc.
Cheers,
-Peter
--
Peter Wemm - peter@FreeBSD.org; peter@yahoo-inc.com; peter@netplex.com.au
"All of this is for nothing if we don't go to the stars" - JMS/B5
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe cvs-all" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20011004051309.BFAF23808>