Date: Thu, 13 Mar 2014 17:55:26 +0100 From: "Christopher J. Ruwe" <cjr@cruwe.de> To: <freebsd-questions@freebsd.org> Subject: cannot witness rsync delta-algorithm Message-ID: <20140313175526.1ea13c4f@dijkstra-old.hb22.cruwe.de>
next in thread | raw e-mail | index | archive | help
Having read the rsync delta transfer paper included with the rsync documentation (Andrew Tridgell, Paul Mackerras: The Rsync Algorithm), I wanted to witness this in practice.=20 (rsync should use rolling MD5 checksums on source and target file chunks to determine if the data is present on remote target without having to transfer the data.) I do not succeed. I have setup a file 'here' using dd and /dev/zero: [cjr@dijkstra:~/rsync-exp]$ dd if=3D/dev/zero of=3Dhere bs=3D1M count=3D100 100+0 records in 100+0 records out 104857600 bytes transferred in 0.101729 secs (1030755108 bytes/sec) I used 100 MB to ensure that changing delta-window-size would not be an issue. I then used rsync to copy 'here' to 'there'. Afterwards, I changed the last MB of here: [cjr@dijkstra:~/rsync-exp]$ rsync -vh --inplace here there sent 104.88M bytes received 35 bytes 69.92M bytes/sec total size is 104.86M speedup is 1.00 [cjr@dijkstra-old:cb-krb/rsync-exp]$ diff here there =20 here and there obviously have equal content. Then, I changed the last 1 MB of here: [cjr@dijkstra:~/rsync-exp]$ dd if=3D/dev/random of=3Dhere bs=3D1M count=3D1 seek=3D99 1+0 records in 1+0 records out 1048576 bytes transferred in 0.026790 secs (39140716 bytes/sec) 'here' and 'there' differ: [cjr@dijkstra:~/rsync-exp]$ diff here there Files here and there differ =46rom what I understand from rsync workings, I would expect another rsync from 'here' to 'there' only to copy some MBs due to possibly different window sizes, but significantly less than 100MB. Yet, it does not. Why? [cjr@dijkstra:~/rsync-exp]$ rsync -vhc --inplace here there sent 104.88M bytes received 35 bytes 69.92M bytes/sec total size is 104.86M speedup is 1.00 [cjr@dijkstra:~/rsync-exp]$ diff here there =20 'here' has been copied as a whole to 'there'. The '-W' flag has not been set! My rsync has been compiled thus: [cjr@dijkstra:~]$ pkg info rsync rsync-3.1.0_1 Name : rsync Version : 3.1.0_1 Installed on : Mon Mar 10 21:23:56 CET 2014 Origin : net/rsync Architecture : freebsd:10:x86:64 Prefix : /usr/local Categories : net ipv6 Licenses : GPLv3 Maintainer : ehaupt@FreeBSD.org WWW : http://rsync.samba.org/ Comment : Network file distribution/synchronization utility Options : ACL : off ATIMES : off DOCS : on FLAGS : off ICONV : off POPT_PORT : off RENAMED : off RENFL : on SSH : on TIMELIMIT : off ZLIB_BASE : on [...] Can anybody explain this behaviour? Thanks and cheers, --=20 Christopher TZ: GMT + 1h GnuPG/GPG: 0xE8DE2C14 =20 FreeBSD 10.0-STABLE #0 r262291+cf0fa48(stable/10): Sat Feb 22 02:01:26 CET 2014 cjr@dijkstra.hb22.cruwe.de:/usr/obj/usr/src/sys/GENERIC=20 =20 Punctuation matters: "Lets eat Grandma or Lets eat, Grandma" - Punctuation saves lives. "A panda eats shoots and leaves" or "A panda eats, shoots, and leaves" - Punctuation teaches proper biology. "With sufficient thrust, pigs fly just fine. However, this is not necessarily a good idea. It is hard to be sure where they are going to land, and it could be dangerous sitting under them as they fly overhead." (RFC 1925)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140313175526.1ea13c4f>