Date: Thu, 28 Apr 2011 12:56:01 -0700 From: Jeremy Chadwick <freebsd@jdc.parodius.com> To: Daniel O'Connor <doconnor@gsoft.com.au> Cc: freebsd-stable List <freebsd-stable@freebsd.org> Subject: Re: ZFS vs OSX Time Machine Message-ID: <20110428195601.GA31807@icarus.home.lan> In-Reply-To: <537A8F4F-A302-40F9-92DF-403388D99B4B@gsoft.com.au> References: <537A8F4F-A302-40F9-92DF-403388D99B4B@gsoft.com.au>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Apr 28, 2011 at 11:33:22PM +0930, Daniel O'Connor wrote: > Does anyone else use ZFS to store TM backups? > > I find that whenever my laptop (over wifi!) starts a TM the ZFS machine it's backing up to grinds to a halt.. Other systems streaming stuff over NFS from it also tend to stall.. > > I presume that TM is doing something which causes ZFS some issues but I'm not sure how to find out what the real problem is let alone how to fix it.. > > I am running FreeBSD midget.dons.net.au 8.2-PRERELEASE FreeBSD 8.2-PRERELEASE #8 r217094M: Sat Jan 8 11:15:07 CST 2011 darius@midget.dons.net.au:/usr/obj/usr/src/sys/MIDGET amd64 > > It is a 5 disk RAIDZ1 with 1.29Tb free using WD10EADS drives. > > I don't see any SMART errors or ZFS warnings. > > I have the following ZFS related tunables > > vfs.zfs.arc_max="3072M" > vfs.zfs.prefetch_disable="1" > vfs.zfs.txg.timeout=5 > vfs.zfs.cache_flush_disable=1 Are the last two actually *working* in /boot/loader.conf? Can you verify by looking at them via sysctl? AFAIK they shouldn't work, since they lack double-quotes around the values. Parsing errors are supposed to throw you back to the loader prompt. See loader.conf(5) for the syntax. I'm also not sure why you're setting cache_flush_disable at all. > Any help appreciated, thanks :) Others seem to be battling stating that "NFS doesn't work for TM", but that isn't what you're complaining about. You're complaining that FreeBSD with ZFS + NFS performs extremely poorly when trying to do backups from an OS X client using TM (writing to the NFS mount). I have absolutely no experience with TM or OS X, so if it's actually a client-level problem (which I'm doubting) I can't help you there. Just sort of a ramble here at different things... It would be useful to provide ZFS ARC sysctl data from the FreeBSD system where you're seeing performance issues. "sysctl -a kstat.zfs.misc.arcstats" should suffice. You should also try executing "zpool iostat -v 1" during the TM backup to see if there's a particular device which is behaving poorly. There have been reports of ZFS pools behaving poorly when a single device within the pool has slow I/O (e.g. 5 hard disks, one of which has internal issues, resulting in the entire pool performing horribly). You should let this run for probably 60-120 seconds to get an idea. Given your parameters above (assuming vfs.zfs.txg.timeout IS in fact 5!), you should see "bursts" of writes every 5 seconds. I know that there are some things on ZFS that perform badly overall. Anything that involves excessive/large numbers of files (not file sizes, but actual files themselves) seems to perform not-so-great with ZFS. For example, Maildir on ZFS = piss-poor performance. There are ways to work around this issue (if I remember correctly, by adding a dedicated "log" device to your ZFS pool, but be aware your log devices need to be reliable (if you have a single log device and it fails the entire pool is damaged, if I remember right)), but I don't consider it feasible. So if TM is creating tons of files on the NFS mount (backed by ZFS), then I imagine the performance isn't so great. Could you please provide the following sysctl values? Thanks. kern.maxvnodes kern.minvnodes vfs.freevnodes vfs.numvnodes If the FreeBSD machine has a wireless card in it, if at all possible could you try ruling that out by hooking up wired Ethernet instead? It's probably not the cause, but worth trying anyway. If you have a home router or something doing 802.11, don't bother with this idea. Next, you COULD try using Samba/CIFS on the FreeBSD box to see if you can narrow the issue down to bad NFS performance. Please see this post of mine about tuning Samba on FreeBSD (backed by ZFS) to get extremely good performance. Many people responded and said their performance drastically improved (you can see the thread yourself). The trick is AIO. You can ignore the part about setting vm.kmem_size in loader.conf; that advice is now old/deprecated (does not pertain to you given the date of your kernel), and vfs.zfs.txg.write_limit_override is something you shouldn't mess with unless absolutely needed to leave it default: http://lists.freebsd.org/pipermail/freebsd-stable/2011-February/061642.html Finally, when was the last time this FreeBSD machine was rebooted? Some people have seen horrible performance that goes away after a reboot. There's some speculation that memory fragmentation has something to do with it. I simply don't know. I'm not telling you to reboot the box (please don't; it would be more useful if it could be kept up in case folks want to do analysis of it). -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB |
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20110428195601.GA31807>