Date: Sun, 28 Feb 2010 14:11:42 +0200 From: Daniel Braniss <danny@cs.huji.ac.il> To: Jeremy Chadwick <freebsd@jdc.parodius.com> Cc: stable@freebsd.org, freebsd-fs@freebsd.org, Willem Jan Withagen <wjw@digiware.nl>, Gerrit =?iso-8859-1?Q?K=FChn?= <gerrit@pmp.uni-hannover.de>, Eirik =?iso-8859-1?Q?=D8verby?= <ltning@anduin.net>, rwatson@freebsd.org Subject: Re: mbuf leakage with nfs/zfs? Message-ID: <E1Nlhzr-0004ci-3W@kabab.cs.huji.ac.il> In-Reply-To: <20100227220310.GA65110@icarus.home.lan> References: <20100226174021.8feadad9.gerrit@pmp.uni-hannover.de> <E1Nl6VA-000557-D9@kabab.cs.huji.ac.il> <20100226224320.8c4259bf.gerrit@pmp.uni-hannover.de> <4B884757.9040001@digiware.nl> <20100227080220.ac6a2e4d.gerrit@pmp.uni-hannover.de> <4B892918.4080701@digiware.nl> <20100227202105.f31cbef7.gerrit@pmp.uni-hannover.de> <20100227193819.GA60576@icarus.home.lan> <BD8AC9F6-DF96-41F9-8E92-48A4E5606DC7@anduin.net> <4B89943C.70704@digiware.nl> <20100227220310.GA65110@icarus.home.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Sat, Feb 27, 2010 at 10:53:00PM +0100, Willem Jan Withagen wrote: > > On 27-2-2010 21:32, Eirik =D8verby wrote: > > >I've had a discussion with some folks on this for a while. I can eas= il=3Dy > > >reproduce this situation by mounting a FreeBSD ZFS filesystem via > > >NFS-UDP from an OpenBSD machine. Telling the OpenBSD machine to use = TC=3DP > > >instead of UDP makes the problem go away. > > > > > >Other FreeBSD systems mounting the same share, either using UDP or T= CP=3D, > > >does not cause the problem to show up. > > > > > >A patch was suggested by Rick Macklem, but that did not solve the is= su=3De: > > >http://lists.freebsd.org/pipermail/freebsd-current/2009-December/014= 18=3D1.html> >=20 > > I concur. > > Everything in my network is now on TCP, and there is no mbuf leakage.= > > I just don't get over the 5500 mark, no matter what I throw at it. > >=20 > > I do feel that TCP is not as well performing on a local net with Linu= x, > > hence the choice for UDP. But TCP is workable as next best. >=20 > I'm pulling in Robert Watson, who has some familiarity with the UDP > stack/code in FreeBSD. I'm not sure he'll be a sufficient source of > knowledge for this specific issue since it appears (?) to be specific t= o > NFS; Rick Macklem would be a better choice, but as reported, he's MIA. >=20 > Robert, are you aware of any changes or implementation issues which > might cause excessive (read: leaking) mbuf use under UDP-based NFS? Do= > you know of a way folks could determine the source of the leak, either > via DDB or while the system is live? I have been runing some tests in a controlled environment. server and client are both 64bit Xeon/X5550 =40 2.67GHz with 16Gb of mem= ory FreeBSD/SMP: 2 package(s) x 4 core(s) x 2 SMT threads the client is runing latest 8.0 stable the load is created by runing 'make -j32 buildworld' and sleeping 150 sec= . in between runs, this is the straight line you will see in the graphs. Both the src and obj directories are NFS mounted from the server, regular= UFS. when server is running 7.2-stable no leakage is seen. see ftp://ftp.cs.huji.ac.il/users/danny/freebsd/mbufs/=7Btcp,udp=7D-7.2.= ps when server is runing 8.0-stable see ftp://ftp.cs.huji.ac.il/users/danny/freebsd/mbufs/=7Btcp,udp=7D-8.0.= ps you can see that udp is leaking=21 cheers, danny ps: I think the subject should be changed again, removing zfs ...
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?E1Nlhzr-0004ci-3W>