Date: Tue, 24 Jun 2003 21:56:35 -0700 From: "Sean J. Schluntz" <schluntz@workofstone.com> To: Tom Samplonius <tom@sdf.com> Cc: "freebsd-isp@freebsd.org" <freebsd-isp@freebsd.org> Subject: Re: What would be the best way to copy lots of files from oneserver to another. Message-ID: <657FE40E-A6C9-11D7-B680-000A95672454@workofstone.com> In-Reply-To: <Pine.BSF.4.05.10306211305250.16428-100000@misery.sdf.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Saturday, June 21, 2003, at 01:08 PM, Tom Samplonius wrote: > > On Sat, 21 Jun 2003, Simon wrote: > >> I recently used rsync to copy ~35gb worth of data, 1,170,168 inodes, >> without any problem over 100mbps interface. Took 3 or so hours. >> You do need lots of RAM, if you swap, you're screwed. RAM >> usage would depend on number of files you need to copy at once. >> >> -Simon > > Yes, rsync uses large amounts of memory when the fileset is large. > Plus, depending on your CPU, you might want to avoid using ssh > encryption > too. > > Usually if it is a LAN to LAN copy, cpdup over NFS will be WAY faster > than rsync, and use little memory. I've used cpdup for moving mail to > a > new server. I synced before the cutover, shutdown the mail server, > synced > one more, and then brought up the mail server on the system. Very > tidy. > even faster than that: cd sourcedir ; find . -print | cpio -o | \ ssh user@host "cd destdir ; cpio -pdmuv" Drop the v to keep from cluttering your screen. You might need to add c to both ends for 'compatibility' if it's different arch or vers of cpio, but I have better luck with it. I have yet to find anything faster over long links, use rsh for local and it's even faster. -Sean
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?657FE40E-A6C9-11D7-B680-000A95672454>