From owner-freebsd-performance@FreeBSD.ORG  Fri Feb 18 00:36:08 2005
Return-Path: <owner-freebsd-performance@FreeBSD.ORG>
Delivered-To: freebsd-performance@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id DAE7F16A4CE
	for <freebsd-performance@freebsd.org>;
	Fri, 18 Feb 2005 00:36:08 +0000 (GMT)
Received: from corp.globat.com (corp.globat.com [216.193.201.6])
	by mx1.FreeBSD.org (Postfix) with ESMTP id A161743D4C
	for <freebsd-performance@freebsd.org>;
	Fri, 18 Feb 2005 00:36:08 +0000 (GMT)
	(envelope-from drice@globat.com)
Received: from globat.com (globat [66.159.202.156])
	by corp.globat.com (8.12.11/8.12.9) with ESMTP id j1I0a7ee007743
	for <freebsd-performance@freebsd.org>;
	Thu, 17 Feb 2005 16:36:07 -0800 (PST)
	(envelope-from drice@globat.com)
From: David Rice <drice@globat.com>
Organization: Globat
To: freebsd-performance@freebsd.org
Date: Thu, 17 Feb 2005 16:36:10 -0800
User-Agent: KMail/1.5.4
MIME-Version: 1.0
Content-Type: text/plain;
  charset="us-ascii"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Message-Id: <200502171636.10361.drice@globat.com>
Subject: High traffic NFS performance and availability problems
X-BeenThere: freebsd-performance@freebsd.org
X-Mailman-Version: 2.1.1
Precedence: list
List-Id: Performance/tuning <freebsd-performance.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-performance>,
	<mailto:freebsd-performance-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-performance>
List-Post: <mailto:freebsd-performance@freebsd.org>
List-Help: <mailto:freebsd-performance-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-performance>,
	<mailto:freebsd-performance-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 18 Feb 2005 00:36:09 -0000

We are a web hosting company that runs  exclusivly on FreeBSD. We are having 
storage availability and performance problems. All of our storage is exported
via NFS to the client machines Any suggestions or advice will be greatly 
appreciated. We are willing to pay someone on a consulting basis to help us 
solve these problems. Please email me off list if you are a consultant. 

On the file server side we have:

Dell PowerEdge 1750's
Dell Perc4 RAID controller
Dell Power Vault 220 storage shelf.
(12) 146GB SCSI drives on one SCSI Bus with a hot spare (~1.3TB file system)
GigaBit ethernet to the NFS client machines
1GB RAM
(2) 2.4 GHZ Xenon Processors
FreeBSD 5.2.1 or FreBSD 5.3

On the client side we have: 

Dell PowerEdge 1750's
1GB RAM
(2) 2.4 GHZ Xenon Processors
GigaBit ethernet
36GB SCSI root disk
FreeBSD 4.9 to FreeBSD 4.11

On the network side we have:
Gigabit ethernet
Foundry BigIron and NetIron swithes
Cisco 6509 with Gigabit switch blades 


Typicly we have 7 client boxes mounting storage from a single file server. 
Each client box servers 1000 web sites and associate email. We have
done the basic NFS tuning (ie: Read write size optimization and kernel tuning)


The problems we are having is as follows.

1. Slow perfomance during peek traffic periods
2. Client boxes have high load averages and sometimes crashes due to slow NFS 
performance.
3. File servers that randomly crash with "Fatal trap 12: page fault while in 
kernel mode"
4. With soft updates enabled during FSCK the fileserver will freeze with all 
NFS processs in the "snaplck" state. We disabled soft updates because of 
this.

I can provide and other details about our configuration if needed. 

David Rice
drice@globat.com

Thank You.