From owner-freebsd-questions@FreeBSD.ORG Wed Feb 8 20:09:09 2012 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 544DA106566C for ; Wed, 8 Feb 2012 20:09:09 +0000 (UTC) (envelope-from dan@dan.emsphone.com) Received: from email2.allantgroup.com (email2.emsphone.com [199.67.51.116]) by mx1.freebsd.org (Postfix) with ESMTP id 00B3D8FC0A for ; Wed, 8 Feb 2012 20:09:08 +0000 (UTC) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by email2.allantgroup.com (8.14.4/8.14.4) with ESMTP id q18K95NW042327 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 8 Feb 2012 14:09:05 -0600 (CST) (envelope-from dan@dan.emsphone.com) Received: from dan.emsphone.com (smmsp@localhost [127.0.0.1]) by dan.emsphone.com (8.14.5/8.14.5) with ESMTP id q18K95PA017619 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 8 Feb 2012 14:09:05 -0600 (CST) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.14.5/8.14.5/Submit) id q18K93SA017570; Wed, 8 Feb 2012 14:09:03 -0600 (CST) (envelope-from dan) Date: Wed, 8 Feb 2012 14:09:03 -0600 From: Dan Nelson To: Tim Daneliuk Message-ID: <20120208200903.GI5775@dan.emsphone.com> References: <4F2AD107.40703@tundraware.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4F2AD107.40703@tundraware.com> X-OS: FreeBSD 8.2-STABLE User-Agent: Mutt/1.5.21 (2010-09-15) X-Virus-Scanned: clamav-milter 0.97.2 at email2.allantgroup.com X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.6 (email2.allantgroup.com [199.67.51.78]); Wed, 08 Feb 2012 14:09:05 -0600 (CST) X-Scanned-By: MIMEDefang 2.68 on 199.67.51.78 Cc: FreeBSD Mailing List Subject: Re: Asymmetric NFS Performance X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Feb 2012 20:09:09 -0000 In the last episode (Feb 02), Tim Daneliuk said: > Server: FBSD 8.2-STABLE / MTU set to 15000 > Client: Linux Mint 12 / MTU set to 8192 > NFS Mount Options: rw,soft,intr > Problem: > > Throughput copying from Server to Client is about 2x that when copying a > file from client to server. The client does have a SSD whereas the server > has conventional SATA drives but ... This problem is evident with either > 100- or 1000- speed ethernet so I don't think it is a drive thing since > you'd expect to saturate 100-BASE with either type of drive. > > Things I've Tried So Far: > > - Increasing the MTUs - This helped speed things up, but the up/down > ratio stayed about the same. > > - Fiddling with rsize and wsize on the client - No real difference If "iostat -zx 1" on the server shows the disks at 100% busy, you're probably getting hit by the fact that NFS has to commit writes to stable storage before acking the client, so writes over NFS can be many times slower than local write speed. Setting the vfs.nfsrv.async sysctl to 1 will speed things up, but if the server reboots while a client is writing, you will probably end up with missing data even though the client thought everything was written. If you are serving ZFS filesystems, stick an SSD in the server and point the ZFS intent log at it: "zpool add mypool log da3". 8GB of ZIL is more than enough, but it needs to be fast, so no sticking a $10 thumb drive in and expecting any improvement :) -- Dan Nelson dnelson@allantgroup.com