Date: Thu, 12 Apr 2007 13:23:18 +0900 From: Hidetoshi Shimokawa <simokawa@FreeBSD.ORG> To: pjd@freebsd.org Cc: freebsd-current@freebsd.org Subject: ZFS vnode starvation with large kmem_size Message-ID: <868xcyusp5.wl%simokawa@FreeBSD.ORG>
next in thread | raw e-mail | index | archive | help
Hi Pawel, I observed vnode starvation with large kmem_size in loader.conf. It seems that even if zfs eats vnodes up to maxvnodes, vnodes cannot be reclaimed and vnlru kthread will give up. As far as I understand, the problem comes from difference of name cache mechanism. I attached a patch for a workaround. How to repeat: arch: amd64 or i386 physical memory: 512MB vm.kmem_size=671088640 ( In this setting: kern.maxvnodes: 35077 vfs.zfs.dnlc.ncsize: 33095 ) Create zfs filesystem and extract ports.tar.gz(many files) on it. After a while you can see that the system is not responsive because of vnode starvation. Workarounds: 1. reduce vfs.zfs.dnlc.ncsize or 2. apply the following patch Index: dnlc.c =================================================================== RCS file: /home/ncvs/src/sys/contrib/opensolaris/uts/common/fs/dnlc.c,v retrieving revision 1.1 diff -u -r1.1 dnlc.c --- dnlc.c 6 Apr 2007 01:09:02 -0000 1.1 +++ dnlc.c 12 Apr 2007 04:18:34 -0000 @@ -703,6 +703,9 @@ dnlc_get(uchar_t namlen) { ncache_t *ncp; + extern int desiredvnodes; + extern int numvnodes; + static uint_t wantvnodes_reduce_percent = 10; if (dnlc_nentries > dnlc_max_nentries) { dnlc_max_nentries_cnt++; /* keep a statistic */ @@ -714,7 +717,10 @@ } ncp->namlen = namlen; atomic_add_32(&dnlc_nentries, 1); - dnlc_reduce_cache(NULL); + if (numvnodes > desiredvnodes) + dnlc_reduce_cache((void *)&wantvnodes_reduce_percent); + else + dnlc_reduce_cache(NULL); return (ncp); } I know this kmem_size must be too large but I think zfs should cooperate with existent vnode management. Except this problem, zfs works very well on my machines. Thank you for your great work! /\ Hidetoshi Shimokawa \/ simokawa@FreeBSD.ORG
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?868xcyusp5.wl%simokawa>