From owner-freebsd-questions@FreeBSD.ORG  Fri Jul 28 16:12:31 2006
Return-Path: <owner-freebsd-questions@FreeBSD.ORG>
X-Original-To: freebsd-questions@freebsd.org
Delivered-To: freebsd-questions@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 0539916A4DA
	for <freebsd-questions@freebsd.org>;
	Fri, 28 Jul 2006 16:12:31 +0000 (UTC) (envelope-from user@dhp.com)
Received: from shell.dhp.com (shell.dhp.com [199.245.105.1])
	by mx1.FreeBSD.org (Postfix) with ESMTP id AD63E43D76
	for <freebsd-questions@freebsd.org>;
	Fri, 28 Jul 2006 16:12:30 +0000 (GMT) (envelope-from user@dhp.com)
Received: by shell.dhp.com (Postfix, from userid 896)
	id 6072931337; Fri, 28 Jul 2006 12:12:29 -0400 (EDT)
Date: Fri, 28 Jul 2006 12:12:29 -0400 (EDT)
From: Ensel Sharon <user@dhp.com>
To: freebsd-questions@freebsd.org
Message-ID: <Pine.LNX.4.21.0607281209110.12027-100000@shell.dhp.com>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Subject: long, large rsync slows down over time ?
X-BeenThere: freebsd-questions@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: User questions <freebsd-questions.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-questions>, 
	<mailto:freebsd-questions-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-questions>
List-Post: <mailto:freebsd-questions@freebsd.org>
List-Help: <mailto:freebsd-questions-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-questions>, 
	<mailto:freebsd-questions-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 28 Jul 2006 16:12:31 -0000


I am moving a large number of files (about 40 GB, comprising almost 3.5
million files) from one machine to another using rsync.

I am noticing that the longer this rsync process runs, the slower it seems
to go, and the more CPU time it is taking on the destination (FreeBSD
6.1).

Is there a reason for this ?  Is there some aspect of rsync (or
FreeBSD) that just starts to bog down after transferring 1-2 million
files?

When the job started, load on the destination was between .2 and .8, and
now, four days later, it is up to 4.0+  ...  and as I say, although I have
not measured, throughput does seem to be slowing down.

Any thoughts / comments ?