From owner-freebsd-stable@FreeBSD.ORG Fri Feb 1 17:27:27 2013 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 0C59E2D9 for ; Fri, 1 Feb 2013 17:27:27 +0000 (UTC) (envelope-from jcm@visi.com) Received: from g2host.com (mailback4.g2host.com [208.42.184.244]) by mx1.freebsd.org (Postfix) with ESMTP id C39F72D1 for ; Fri, 1 Feb 2013 17:27:26 +0000 (UTC) Received: from [208.42.90.57] (account jcm@visi.com) by mailback4.g2host.com (CommuniGate Pro WEBUSER 5.3.11) with HTTP id 11636808 for freebsd-stable@freebsd.org; Fri, 01 Feb 2013 11:27:17 -0600 From: "John Mehr" Subject: Re: svn - but smaller? To: X-Mailer: CommuniGate Pro WebUser v5.3.11 Date: Fri, 01 Feb 2013 11:27:17 -0600 Message-ID: In-Reply-To: <20130130124752.GB10308@snail.casa> References: <20130123144050.GG51786@e-Gitt.NET> <20130125092723.GC79995@e-Gitt.NET> <1359320641-6493504.60501067.fr0RL3aYw027137@rs149.luxsci.com> <1359478142-8003652.41205547.fr0TGmWxX027565@rs149.luxsci.com> <20130130124752.GB10308@snail.casa> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Feb 2013 17:27:27 -0000 On Wed, 30 Jan 2013 13:47:52 +0100 Arrigo Marchiori wrote: > On Tue, Jan 29, 2013 at 11:48:31AM -0500, Isaac (.ike) > I tried the attached script to download > http://svn.freebsd.org/base/stable/9/ In 27 minutes it >downloaded 67 > megabytes, corresponding to 42 KByte/sec, through an >ADSL link capable > of roughly 400 KByte/sec. Then I stopped it. > > IMHO this should demonstrate that the problem is not >only having a > SVN workalike, but a fast and efficient one. > -- > rigo > > http://rigo.altervista.org Hello all, I've got a "working" solution now. I put working in quotes because I'm seeing similar dismal performance stats to your script (I also have ~ 400 KB/s DSL connection). The problem appears to be with the overhead of sending and receiving all of the ~ 100 byte requests to both get the MD5 signatures and to get the files themselves (large files transfer *very* quickly and I'm not using poll() to check if there's data to be read on the socket). I've got two bugs left to squash and I'll post the source code for everyone to review. I know you can issue more that one command at a time and my next hurdle will be working on speeding things up by combining requests for all files in each folder instead of issuing one request for each file.