Date: Thu, 01 Dec 2011 14:03:05 +0100 From: Peter Maloney <peter.maloney@brockmann-consult.de> To: freebsd-fs@freebsd.org Subject: Re: ZFS dedup and replication Message-ID: <4ED77B09.1090709@brockmann-consult.de> In-Reply-To: <CALfReycy29VdegrmDrBJ7U3Mjt7%2BOxUvN7hxOKHOqSX4jD5_kg@mail.gmail.com> References: <CAEUA181wUZC-KjVwcm=tTY0DoBLzrNAuBF3aFimSbLB=xht0jw@mail.gmail.com> <CALfReycy29VdegrmDrBJ7U3Mjt7%2BOxUvN7hxOKHOqSX4jD5_kg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 12/01/2011 11:20 AM, krad wrote: > On 28 November 2011 23:01, Techie <techchavez@gmail.com> wrote: > >> Hi all, >> >> Is there any plans to implement sharing of the ZFS DDT Dedup table or >> to make ZFS aware of the destination duplicate blocks on a remote >> system? >> >> >From how I understand it, the zfs send/recv stream does not know about >> the duplicated blocks on the receiving side when using zfs send -D -i >> to sendonly incremental changes. >> >> So take for example I have an application that I backup each night to >> a ZFS file system. I want to replicate this every night to my remote >> site. Each night that I back up I create a tar file on the ZFS data >> file system. When I go to send an incremental stream it sends the >> entire tar file to the destination even though over 90% of those >> blocks already exist at the destination.. Is there any plans to make >> ZFS aware of what exists already at the destination site to eliminate >> the need to send duplicate blocks over the wire? zfs send -D I believe >> only eliminates the duplicate blocks within the stream. >> >> Perhaps I am wrong.. >> >> >> Thanks >> Jimmy >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> > > Why tar up the stuff? Just do a zfs snap and then you bypass the whole > issue? I was thinking the same thing when I read his message. I don't understand it either. On my system with 12 TiB used up, what I do in a script is basically: -generate a snap name -make a recursive snapshot -ssh to the remote server and compare snapshots (find the latest common snapshot, to find an incremental reference point) -if a usable reference point exists, start the incremental send like this (which wipes all changes on the remote system without confirmation): zfs send -R -I ${destLastSnap} ${srcLastSnap} | ssh ${destHost} zfs recv -d -F -v ${destPool} -and if no usable reference point existed, then do a full send, non-incremental: zfs send -R ${srcLastSnap} | ssh ${destHost} zfs recv -F -v ${destDataSet} The part about finding the reference snapshot is the most complicated part of my script, and missing from anything else I found online when I was looking for a good solution. For example this script: http://blogs.sun.com/clive/resource/zfs_repl.ksh found on this page: http://blogs.oracle.com/clive/entry/replication_using_zfs was found to be quite terrible, and would fail completely when there was a new dataset, or a snapshot missing for some reason. So I suggest you look at that one, but write your own. The only time my script failed is when there was a zfs bug; the same one seen here: http://serverfault.com/questions/66414/cannot-destroy-zfs-snapshot-dataset-already-exists so I just deleted the clone manually and it worked again. I thought gzip could save a small amount of time, eg. I compared speed of "zfs send .... | ssh zfs recv ..." to "zfs send ... | gzip -c | ssh 'gunzip -c | zfs recv...'" and found not much or no difference. But I have no idea why you would use tar. And just to confirm, I have the same problems with dedup causing severe bottlenecks on many things, especially zfs recv and scrub, even though I have 48 GB of memory installed and 44 available to ZFS. But I find incremental sends to be very efficient, taking much less than a minute (depending on how much data was changed) when it runs every hour. And unless your bandwidth is slow and precious, I recommend sending more than daily, because it is very fast if done often enough. I send hourly because I didn't have time to work on some scripts to clean up the old snapshots. Otherwise I would do it every 15 min or maybe 15 seconds ;) > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" -- -------------------------------------------- Peter Maloney Brockmann Consult Max-Planck-Str. 2 21502 Geesthacht Germany Tel: +49 4152 889 300 Fax: +49 4152 889 333 E-mail: peter.maloney@brockmann-consult.de Internet: http://www.brockmann-consult.de --------------------------------------------
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4ED77B09.1090709>