From owner-freebsd-fs@FreeBSD.ORG Wed Dec 14 07:45:07 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5D03A1065670 for ; Wed, 14 Dec 2011 07:45:07 +0000 (UTC) (envelope-from peter.maloney@brockmann-consult.de) Received: from mo-p05-ob6.rzone.de (mo-p05-ob6.rzone.de [IPv6:2a01:238:20a:202:53f5::1]) by mx1.freebsd.org (Postfix) with ESMTP id BE1898FC0C for ; Wed, 14 Dec 2011 07:45:06 +0000 (UTC) X-RZG-AUTH: :LWIKdA2leu0bPbLmhzXgqn0MTG6qiKEwQRWfNxSw4HzYIwjsnvdDt2QV8d370WOpHNvplOs= X-RZG-CLASS-ID: mo05 Received: from [192.168.179.42] (hmbg-4d06f666.pool.mediaWays.net [77.6.246.102]) by smtp.strato.de (klopstock mo7) (RZmta 26.15 DYNA|AUTH) with (DHE-RSA-AES128-SHA encrypted) ESMTPA id 604ae0nBE6OsBo for ; Wed, 14 Dec 2011 08:44:54 +0100 (MET) Message-ID: <4EE853F4.7060003@brockmann-consult.de> Date: Wed, 14 Dec 2011 08:44:52 +0100 From: Peter Maloney User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:8.0) Gecko/20111105 Thunderbird/8.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <961254586.174058.1323823086880.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <961254586.174058.1323823086880.JavaMail.root@erie.cs.uoguelph.ca> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Subject: Re: FreeBSD 9.0 and NFS async -off topic about ZFS ZIL devices X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Dec 2011 07:45:07 -0000 Am 14.12.2011 01:38, schrieb Rick Macklem: > Johan Hendriks wrote: >> Rick Macklem schreef: >>> Johan Hendriks wrote: >>>> Hello all. >>>> >>>> I used to use async on my 8.x nfs servers! >>>> On the FreeBSD 9.0 server i can not do it through the old 8.x >>>> sysctl. >>>> >>>> Is there an other way to set async on FreeBSD 9.x >>>> >>> You have two choices: >>> 1 - Apply this patch to your NFS server's kernel sources and then >>> set >>> vfs.nfsd.async=1 >>> http://people.freebsd.org/~rmacklem/async.patch >>> >>> 2 - switch to using the old server by setting >>> oldnfs_server_enable="YES" >>> in your /etc/rc.conf and then setting the sysctl. >>> >>> I'll assume that you realize that doing this violates the NFS RFCs >>> because >>> it runs your server in a way where there is a risk of data loss >>> (that the >>> client won't know to re-write) when the server crashes. >>> >>> rick >>>> regards, >>>> Johan Hendriks >>>> _______________________________________________ >>>> freebsd-fs@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>> To unsubscribe, send any mail to >>>> "freebsd-fs-unsubscribe@freebsd.org" >> Yes i do know the risk. >> >> The thing is we want a dataset shared to a ESXi client using NFS. >> I use NFS for my normal usage, (sharing ports tree and so on.) but now >> we want to use it to share a ZFS dataset for a ESXi client. >> We use iscsi now, but this way we miss some zfs goodies. like >> snapshots(not a zvol) and most important, we can reach the files >> directly. >> >> But with a virtual machine shared over NFS i get horrible performance. >> If i copy a file to whatever virtual machine from a windows client >> shared with iscsi , i get arround 80Mb per second (in the windows copy >> window) almost at a steady pace. we are really pleased with that. !! >> If i copy a virtual machine to the NFS share, fire it up, and do a >> file >> copy, it never gets higher than 50 Mb and it sometimes drop to 1 Mb >> then >> goes to 20 back to 10 and so on. >> Also the machines feels sluggish in performance. >> >> Are there other less dangerous things i can try to boost performance? >> > I don't use ZFS, but others have reported using a dedicated SSD that has > good write performance for the ZIL log in order to get better write > performance for ZFS. Indeed, I can get 65 MB/s using a consumer SSD with normal sync nfs clients. But once you use ESXi's NFS client, it will drop to somewhere between 5 and 9 MB/s. Mine currently goes around 7 MB/s. I also tried adding a ramdisk as my ZIL, and then it still only goes 80 MB/s, even though obviously the ramdisk should be able to handle random writes better than anything, and should go GB/s if there was no other bottleneck, not only 80 MB/s (over 10Gbps network, which is tested at about 6 Gbps with 1500 MTU). Anything else has to go through RAM at some point anyway. Unfortunatlely, I didn't test a normal sync nfs client with the ramdisk... I'll put that on my mental todo list. I also tried a UFS zvol, whidh was not using the log, but only went somewhere between 40 and 60 MB/s. Here is a very nice thread with some scores with various SSDs using a normal NFS client. http://forums.freebsd.org/showthread.php?t=23566 Here is my post in the same thread comparing the linux NFS client with sync option, to the ESXi client (writing to a virtual disk filesystem with dd): http://forums.freebsd.org/showpost.php?p=157154&postcount=55 >> regards, >> Johan Hendriks > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"