From owner-freebsd-questions@FreeBSD.ORG Fri Nov 4 16:23:00 2005 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 04DBC16A41F for ; Fri, 4 Nov 2005 16:23:00 +0000 (GMT) (envelope-from cswiger@mac.com) Received: from pi.codefab.com (pi.codefab.com [199.103.21.227]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9BC7D43D45 for ; Fri, 4 Nov 2005 16:22:59 +0000 (GMT) (envelope-from cswiger@mac.com) Received: from localhost (localhost [127.0.0.1]) by pi.codefab.com (Postfix) with ESMTP id B75165FA5; Fri, 4 Nov 2005 11:22:56 -0500 (EST) Received: from pi.codefab.com ([127.0.0.1]) by localhost (pi.codefab.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 75189-10; Fri, 4 Nov 2005 11:22:56 -0500 (EST) Received: from [192.168.1.3] (pool-68-161-122-227.ny325.east.verizon.net [68.161.122.227]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pi.codefab.com (Postfix) with ESMTP id F19815CF8; Fri, 4 Nov 2005 11:22:55 -0500 (EST) Message-ID: <436B8ADF.4000703@mac.com> Date: Fri, 04 Nov 2005 11:22:55 -0500 From: Chuck Swiger Organization: The Courts of Chaos User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.12) Gecko/20050915 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Kirk Strauser References: <200511040956.19087.kirk@strauser.com> In-Reply-To: <200511040956.19087.kirk@strauser.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at codefab.com Cc: freebsd-questions@freebsd.org Subject: Re: Fast diff command for large files? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Nov 2005 16:23:00 -0000 Kirk Strauser wrote: > I need to routinely find the diffs between two multigigabyte text files > (exporting a set of FoxPro tables to a PostgreSQL database without doing a > complete dump/reload each time, in case you were wondering). GNU diff from > the base system and from ports chokes. The textproc/2bsd-diff works OK, > but is glacially slow. Multigigabyte? Find another approach to solving the problem, a text-base diff is going to require excessive resources and time. A 64-bit platform with 2 GB of RAM & 3GB of swap requires ~1000 seconds to diff ~400MB. -- -Chuck