From owner-freebsd-questions@FreeBSD.ORG Wed Aug 2 17:53:37 2006 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6314F16A4E5 for ; Wed, 2 Aug 2006 17:53:37 +0000 (UTC) (envelope-from drumslayer2@yahoo.com) Received: from web34503.mail.mud.yahoo.com (web34503.mail.mud.yahoo.com [66.163.178.169]) by mx1.FreeBSD.org (Postfix) with SMTP id 079A743D78 for ; Wed, 2 Aug 2006 17:53:32 +0000 (GMT) (envelope-from drumslayer2@yahoo.com) Received: (qmail 703 invoked by uid 60001); 2 Aug 2006 17:53:32 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Message-ID:Received:Date:From:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding; b=vBO0hrj4JS6LtfuDbnwJ/yPL6BqeoGhCCubpeGLvr3XGadFXrVRqBHAR5hly1wrYannoyHW68S+xIQ7Y5Js63bPHpQb+QMZzfPrLX1JWw5vp+OC8Bk/tj1fEEoIPgxpL9NRXBrzIdXz4HYrBHCJJvn/LTQDQE7KSUYMAT9KO1CM= ; Message-ID: <20060802175332.701.qmail@web34503.mail.mud.yahoo.com> Received: from [67.112.21.27] by web34503.mail.mud.yahoo.com via HTTP; Wed, 02 Aug 2006 10:53:32 PDT Date: Wed, 2 Aug 2006 10:53:32 -0700 (PDT) From: "N. Harrington" To: Freminlins In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cc: freebsd-questions@freebsd.org Subject: Re: Need Advice for Tuning NFS to place nice with a Netapp X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Aug 2006 17:53:37 -0000 --- Freminlins wrote: > Nicole, > > On 02/08/06, N. Harrington > wrote: > > > > Hi > > I have several web servers that are attached to a > > Netapp (network appliance) unit via NFS-3. A few > > servers are 5.5 and a few are 6.1 for comparison > > testing. All seem to have lousy performance. > > > We have a similar setup and it runs smoothly. Cool! Can you share with me what sort of settings you use on your boxes? sysctl/kerneltunes/mount options? It has taken me a over a month to even get to speak to someone high enough up he food chain at Netapp to not say "FreeBSD - that's a version of Linux right?" > Can you define "lousy performance" ? The web server replies (using either Apache and Lighthttpd) seem to max out at about 17mb/s. Response time for the web server will rise gradually, then suddenly become 10-20seconds for a reply. Much like a backup on a highway. They claim that the netapp unit is spending too much time dealing with file information IOPS than actual transfer of files. However even on a non in-use server, if I make a request for a file, that "heavy file access" seems normal. IE: GtAttr Lookup Rdlink Read Write Rename Access Rddir 0 0 0 0 0 0 0 0 248 160 0 4 0 0 236 0 0 0 0 0 0 0 0 0 > Can you give more details on your network? Are you > using Gig ethernet? And > over what medium? Yes, 4X GigE from the filer via a Vif and trunking on the switch. A nice 10Gb ready HP unit. I have asked if using the Vif and trunking could have any effects but been assured it should not. It does mean I cannot use jumbo frames. But since web pages and images are small, I don't think there would be any benefit. > Can you also try just copying a 100MB file from the > filer to one of the web > servers and record the time? 9907187 bytes/sec for a 16M file. It will transfer in nanotime. So, I believe that eliminates network performance as an issue. > Are you running nfsiod? Yes, I show 4 instances running. > When > > going through the issues with Netapp, the reasons > > given were that we have too many GettAdr/Lookup > > requests compared to actual reads. So all the NFS > IOPS > > are being used up by these requests. As soon as > the > > webservers get busy, requests pile up. > > > > I have tried everything I can think of. The web > > servers are even mounted read only with no help. > > > > My current mount options are: > > filer:/vol/fvol31 /home/13/13 nfs > > ro,noatime,-r=32768,-T,-b,-R0,-i,-D2,-L 0 0 > > > Mounting noatime for web servers is a good idea > but... your "noatime" option > has no effect on NFS mounts (check out the mount man > page). You need "vol > options no_atime_update" on the NetApp. Hmm. Drat. We have some web servers that do nothing but send out data, but some that are used for uploading and file manipulation. I will have to make sure that global of an option will not effect what they do. > Any advice for sysctl tunes or anything else would > be > > much appreciatted! > > > > Thanks > > > > Nicole > > > > One last thing - are you female?! In a UNIX > newsgroup?! Yup :) Oh, and yes, I do play the drums :) > Frem. > Thanks for your assistance!! Nicole The Large Print Giveth And The Small Print Taketh Away -- Anon __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com