From owner-freebsd-stable@FreeBSD.ORG Fri Feb 26 15:07:15 2010 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 801A4106564A for ; Fri, 26 Feb 2010 15:07:15 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.16.84]) by mx1.freebsd.org (Postfix) with ESMTP id 3150E8FC1E for ; Fri, 26 Feb 2010 15:07:14 +0000 (UTC) Received: from pampa.cs.huji.ac.il ([132.65.80.32]) by kabab.cs.huji.ac.il with esmtp id 1Nl1mb-0002Mx-M9; Fri, 26 Feb 2010 17:07:13 +0200 X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.2 To: Gerrit =?ISO-8859-1?Q?K=FChn?= In-reply-to: <20100226141754.86ae5a3f.gerrit@pmp.uni-hannover.de> References: <4B86F384.3010308@digiware.nl> <2a41acea1002251459v40e8c6ddxd0437decbada4594@mail.gmail.com> <4B8795B1.4020006@digiware.nl> <20100226120339.GB17798@icarus.home.lan> <20100226133138.d47dd080.gerrit@pmp.uni-hannover.de> <20100226134429.041ea6f2.gerrit@pmp.uni-hannover.de> <20100226141754.86ae5a3f.gerrit@pmp.uni-hannover.de> Comments: In-reply-to Gerrit =?ISO-8859-1?Q?K=FChn?= message dated "Fri, 26 Feb 2010 14:17:54 +0100." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 26 Feb 2010 17:07:13 +0200 From: Daniel Braniss Message-ID: Cc: stable@freebsd.org, Willem Jan Withagen , Jack Vogel , Jeremy Chadwick Subject: Re: em0 freezes on ZFS server X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Feb 2010 15:07:15 -0000 > On Fri, 26 Feb 2010 15:04:37 +0200 Daniel Braniss > wrote about Re: em0 freezes on ZFS server : > > DB> > At least in my case I found out what is eating the buffers: nfsd > DB> > does! The buffers stop increasing as soon as I stop nfsd. However, > DB> > they start increasing as soon as I start nfsd again. > DB> > Are there any ideas how to fix this? Downgrading back to 7-stable is > DB> > not really an easy task as far as I know, and I need the server to > DB> > run without having to reboot it once for twice a day... > > DB> I want to add some spices to this stew: :-) > > You're welcome. :-) > > DB> Some few day later it hung, and it's now hanging every few days. > DB> Most of the hangs are because there is no network, but the NIC is bce > DB> not em! I doubled kern.ipc.nmbclusters and lets see what happens ... > > Do you have nfsd running and serving clients? If so, we should maybe > change the topic to something like "possible nfs mbuf leakage"... > it's only purpose in life is a nfs server. but I wouldn't exclude zfs from the equation yet. I have othere nfs servers, not doing zfs and I don't see this. > DB> 23066/6634/29700 mbufs in use (current/cache/total) > > My server is at 22k now, and the buffer number is still increasing every > few seconds... > Can you monitor your mbuf usage and report if it grows? > I am, and in the last 2hs. it grew by about 300, it does oscilate, i.e. it grows some, then it goes down, but it seems that the low always increases. when I have enough data i'll plot it. Cheers, danny