Date: Mon, 21 Apr 2008 15:27:53 +0400 From: Igor Sysoev <is@rambler-co.ru> To: gnn@freebsd.org Cc: Robert Watson <rwatson@freebsd.org>, net@freebsd.org Subject: Re: zonelimit issues... Message-ID: <20080421112753.GD44915@rambler-co.ru> In-Reply-To: <m2mynn4o83.wl%gnn@neville-neil.com> References: <m2hcdztsx2.wl%gnn@neville-neil.com> <20080420102827.U67663@fledge.watson.org> <m2od834pmv.wl%gnn@neville-neil.com> <m2mynn4o83.wl%gnn@neville-neil.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Apr 21, 2008 at 05:16:28PM +0900, gnn@freebsd.org wrote: > At Mon, 21 Apr 2008 16:46:00 +0900, > gnn@freebsd.org wrote: > > > > At Sun, 20 Apr 2008 10:32:25 +0100 (BST), > > rwatson wrote: > > > > > > > > > On Fri, 18 Apr 2008, gnn@freebsd.org wrote: > > > > > > > I am wondering why this patch was never committed? > > > > > > > > http://people.freebsd.org/~delphij/misc/patch-zonelimit-workaround > > > > > > > > It does seem to address an issue I'm seeing where processes get into the > > > > zonelimit state through the use of mbufs (a high speed UDP packet receiver) > > > > but even after network pressure is reduced/removed the process never gets > > > > out of that state again. Applying the patch fixed the issue, but I'd like > > > > to have some discussion as to the general merits of the approach. > > > > > > > > Unfortunately the test that currently causes this is tied very tightly to > > > > code at work that I can't share, but I will hopefully be improving mctest to > > > > try to exhibit this behavior. > > > > > > When you take all load off the system, do mbufs and clusters get properly > > > freed back to UMA (as visible in netstat -m)? If not, continuing to bump up > > > against the zonelimit would suggest an mbuf/cluster leak, in which case we > > > need to track that bug. > > > > > > > This is unclear as the process that creates the issue opens 50 UDP > > multicast sockets with very large socket buffers. I am investigating > > this aspect some more. > > > > OK, yes, the clusters etc. go back to normal when the incoming > pressure is released. I do not believe we have a cluster/mbuf leak. There is no cluster/mbuf leak. The problem that FreeBSD has small KVA space: only 2G even on amd64 32G machines. So with vm.kmem_size=1G # 64M KVA kern.maxbcache=64M # 4M KVA kern.ipc.maxpipekva=4M I can use something like this: # 256M KVA/KVM kern.ipc.nmbjumbop=64000 # 216M KVA/KVM kern.ipc.nmbclusters=98304 # 162M KVA/KVM kern.ipc.maxsockets=163840 # 8M KVA/KVM net.inet.tcp.maxtcptw=163840 # 24M KVA/KVM kern.maxfiles=204800 -- Igor Sysoev http://sysoev.ru/en/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080421112753.GD44915>