Date: Fri, 2 Sep 2011 21:36:10 -0400 (EDT) From: Rick Macklem <rmacklem@uoguelph.ca> To: David Brodbeck <brodbd@uw.edu> Cc: freebsd-fs@freebsd.org Subject: Re: ZFSv28+NFSv4 poor file creation performance, "sync=disabled" has no effect Message-ID: <14220705.747900.1315013770239.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <CAHHaOuY=BEMrhYuzXtD5AtXG7niLXEO1yhO5P4EimcsLuTrLXw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
David Brodbeck wrote: > I originally posted this on FreeBSD-questions, but it was suggested > that I > bring it here. > I'm testing FreeBSD 9.0-BETA with an eye toward eventually using > FreeBSD 9.0 to replace some existing OpenSolaris 2008.11 > installations. I've found NFS file creation performance (as measured > by Bonnie++) is equally slow for both with default settings. However, > on OpenSolaris I disable the ZIL to improve file creation performance. I know nothing about ZFS, so all I can do is pass along what others have said in previous posts. (I'd suggest you look through the freebsd-fs@ archives.) One post explained how disabling the ZIL can result in up to 5seconds worth of changes being lost if/when the server crashes. (A lot can change on a file system in 5sec. The NFS protocol assumes all fs changes related to a file creation are done before the server replies to the RPC. As such, disabling the ZIL does violate the protocol specs and means you are living dangerously.) > This tuning parameter was removed from FreeBSD 9.0; its replacement > is supposed to be the per-filesystem flag "sync", but setting this > flag seems to have no effect. > > I did recompile the FreeBSD kernel without debugging features before > doing the tests, so I don't think this is a case of debugging code > slowing things down. > > Here's the relevant data; these are all from bonnie++'s "sequential > create" benchmark. The NFS client was RedHat Enterprise Linux 5.6. > > OpenSolaris 2008.11, default settings: 58/second > OpenSolaris 2008.11, with "zil_disable=1": 1258/second > > FreeBSD 9.0-BETA, default settings: 107/second > FreeBSD 9.0-BETA, with "sync=disabled": 106/second > > > So it appears the "sync" ZFS parameter has no effect in FreeBSD. Has > anyone else seen this? Is there a way to improve NFS file creation > performance now that zil_disable has been removed? > Some have reported good results from putting the ZIL on a dedicated device. Some use an SSD, but there are write bandwidth issues related to this. If I understood them correctly, you need to make sure that the SSD is designed to provide good write performance and should be configured to a much larger size than what the ZIL would use. Again, there were good threads discussing this on freebsd-fs@. I have never used either ZFS nor an SSD, so the above is just paraphrasing my understanding from these threads and could be way off base. (I do know that NFS clients expect the changes related to file creation to be stored on non-volatile storage before replying to the creation RPC.) > -- > David Brodbeck > System Administrator, Linguistics > University of Washington > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?14220705.747900.1315013770239.JavaMail.root>