From owner-freebsd-isp@FreeBSD.ORG Sat Jun 21 14:52:44 2003 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7355937B401 for ; Sat, 21 Jun 2003 14:52:44 -0700 (PDT) Received: from misery.sdf.com (misery.sdf.com [207.200.153.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id F379443F3F for ; Sat, 21 Jun 2003 14:52:42 -0700 (PDT) (envelope-from tom@sdf.com) Received: from tom (helo=localhost) by misery.sdf.com with local-esmtp (Exim 2.12 #1) id 19TofD-0004QT-00; Sat, 21 Jun 2003 13:08:43 -0700 Date: Sat, 21 Jun 2003 13:08:29 -0700 (PDT) From: Tom Samplonius To: Simon In-Reply-To: <20030621182925.0A54143F3F@mx1.FreeBSD.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: "freebsd-isp@freebsd.org" Subject: Re: What would be the best way to copy lots of files from oneserver to another. X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Jun 2003 21:52:44 -0000 On Sat, 21 Jun 2003, Simon wrote: > I recently used rsync to copy ~35gb worth of data, 1,170,168 inodes, > without any problem over 100mbps interface. Took 3 or so hours. > You do need lots of RAM, if you swap, you're screwed. RAM > usage would depend on number of files you need to copy at once. > > -Simon Yes, rsync uses large amounts of memory when the fileset is large. Plus, depending on your CPU, you might want to avoid using ssh encryption too. Usually if it is a LAN to LAN copy, cpdup over NFS will be WAY faster than rsync, and use little memory. I've used cpdup for moving mail to a new server. I synced before the cutover, shutdown the mail server, synced one more, and then brought up the mail server on the system. Very tidy. Tom