Date: Wed, 25 Jun 2003 11:19:52 +0100 From: Jose Celestino <japc@co.sapo.pt> To: freebsd-isp@freebsd.org Subject: Re: What would be the best way to copy lots of files from one server to another. Message-ID: <20030625101952.GB973@co.sapo.pt> In-Reply-To: <200306181340.06730.etienne@unix.za.org> References: <200306181340.06730.etienne@unix.za.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Words by Etienne Ledoux [Fri, Jun 20, 2003 at 02:21:01PM +0200]: > Greetings, > > I need to copy lots of directories/files from one server to another. Approx > 45Gb . These files are mailboxes stored in maildir format. What would be the > best/safest/quickest way to do this ? > > e. > I've done this about 3 months ago when migrating our mailserver from the old (almost pre-historic) machine it was on to a new one. We had about 55Gb of data, all in maildir format, lots of small files. I left an rsync running over the night, syncing the /maildirs/ from the old, yet still production machine at that time, to the new one. I don't know how much it took for this initial sync, but I guess it took long, I/O on the old machine was awful and there's wasn't that much ram also. Next day, sometime before the switchover, I did another rsync, it took some (about 5) minutes. I then stopped everything except qmail (I stopped tcpserver of course) on the old machine, ALRMed qmail-send to force the immediate delivery of all messages... well, I only really cared about the local ones and indeed some remote messages remained to be delivered but that could be taken care afterwards. There was about 500 messages in queue at that time, about 22 of them where local ones. After no local deliveries pending I stopped qmail and did a last rsync (it took about a couple of minutes) and switched IPs on the machines. On the old one I routed all mail deliveries to the new one (so that I could catch the bounces of any failing remote messages) and started qmail so that it could stay trying to deliver the remotes until queuelifetime. All in all there was about 5 minutes downtime (there was really more as I took the chance to upgrade courier-imap and missed something on the init file that completelly fscked the authentications, but that was only imap). http://rsync.samba.org/ If you want sheer speed and do not want to mess with screwdrivers you should use the nc aproach: on the destination side: nc -l -p 6660 | tar xf - on the origin side: time tar cvf - * | nc DESTINATION_SERVER 6660 > > _______________________________________________ > freebsd-isp@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-isp > To unsubscribe, send any mail to "freebsd-isp-unsubscribe@freebsd.org" > -- Jose Celestino | http://xpto.org/~japc/files/japc-pgpkey.asc ---------------------------------------------------------------- "Quis Custodiet Ipsos Custodes" - "Who will guard the guards?" -- Juvenal's Satires, circa 120 AD
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20030625101952.GB973>