From owner-freebsd-questions@FreeBSD.ORG Mon Nov 7 04:19:57 2005 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7CD6A16A41F for ; Mon, 7 Nov 2005 04:19:57 +0000 (GMT) (envelope-from on@cs.ait.ac.th) Received: from mail.cs.ait.ac.th (mail.cs.ait.ac.th [192.41.170.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id CA9AF43D46 for ; Mon, 7 Nov 2005 04:19:54 +0000 (GMT) (envelope-from on@cs.ait.ac.th) Received: from banyan.cs.ait.ac.th (banyan.cs.ait.ac.th [192.41.170.5]) by mail.cs.ait.ac.th (8.12.11/8.12.11) with ESMTP id jA74Jlhp008155 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 7 Nov 2005 11:19:47 +0700 (ICT) Received: (from on@localhost) by banyan.cs.ait.ac.th (8.13.1/8.12.11) id jA74JjS7068019; Mon, 7 Nov 2005 11:19:45 +0700 (ICT) Date: Mon, 7 Nov 2005 11:19:45 +0700 (ICT) Message-Id: <200511070419.jA74JjS7068019@banyan.cs.ait.ac.th> From: Olivier Nicole To: kirk@strauser.com In-reply-to: <200511041129.17912.kirk@strauser.com> (message from Kirk Strauser on Fri, 4 Nov 2005 11:29:12 -0600) References: <200511040956.19087.kirk@strauser.com> <436B8ADF.4000703@mac.com> <200511041129.17912.kirk@strauser.com> X-Virus-Scanned: on CSIM by amavisd-milter (http://www.amavis.org/) Cc: freebsd-questions@freebsd.org Subject: Re: Fast diff command for large files? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Nov 2005 04:19:57 -0000 > We do the mirroring by running a program that dumps the FoxPro > tables out as tab-delimited files. Thus far, we'd been using > PostgreSQL's "copy from" command to read those files into the > database. In reality, though, a very, very small percentage of > rows in those tables actually change. So, I wrote a program > that takes the output of diff and converts it into a series of I think the problem could be considered another way. if you have access to the "legacy/FoxPro" application, it should be modifed to add a timestamp to each reccord modification. Then you could only dump those reccords that were modified since the last change. That seems to me the only long term viable solution that could sizeup nicely with your set of data. Not to mention that once implemented it would be much much faster. Olivier