From owner-freebsd-ports@FreeBSD.ORG Fri Jul 6 17:07:54 2007 Return-Path: X-Original-To: ports@freebsd.org Delivered-To: freebsd-ports@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id D084A16A400 for ; Fri, 6 Jul 2007 17:07:54 +0000 (UTC) (envelope-from michel@lpthe.jussieu.fr) Received: from shiva.jussieu.fr (shiva.jussieu.fr [134.157.0.129]) by mx1.freebsd.org (Postfix) with ESMTP id 6798113C45B for ; Fri, 6 Jul 2007 17:07:53 +0000 (UTC) (envelope-from michel@lpthe.jussieu.fr) Received: from parthe.lpthe.jussieu.fr (parthe.lpthe.jussieu.fr [134.157.10.1]) by shiva.jussieu.fr (8.13.8/jtpda-5.4) with ESMTP id l66H7qkR044485 ; Fri, 6 Jul 2007 19:07:52 +0200 (CEST) X-Ids: 165 Received: by parthe.lpthe.jussieu.fr (Postfix, from userid 10096) id C8ECDBF6D3; Fri, 6 Jul 2007 19:07:51 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.1.9 (2007-02-13) on parthe.lpthe.jussieu.fr X-Spam-Level: X-Spam-Status: No, score=-3.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.9 Received: from niobe.lpthe.jussieu.fr (niobe.lpthe.jussieu.fr [134.157.10.41]) by parthe.lpthe.jussieu.fr (Postfix) with ESMTP id 1C9CABF60E; Fri, 6 Jul 2007 19:07:51 +0200 (CEST) Received: by niobe.lpthe.jussieu.fr (Postfix, from userid 2005) id 1393180; Fri, 6 Jul 2007 19:07:51 +0200 (CEST) Date: Fri, 6 Jul 2007 19:07:51 +0200 From: Michel Talon To: ports@freebsd.org, hackers@freebsd.org Message-ID: <20070706170750.GA99504@lpthe.jussieu.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.2i X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (shiva.jussieu.fr [134.157.0.165]); Fri, 06 Jul 2007 19:07:52 +0200 (CEST) X-Virus-Scanned: ClamAV 0.88.7/3607/Fri Jul 6 01:51:19 2007 on shiva.jussieu.fr X-Virus-Status: Clean X-j-chkmail-Score: MSGID : 468E76E8.002 on shiva.jussieu.fr : j-chkmail score : X : 0/50 0 0.507 -> 1 X-Miltered: at shiva.jussieu.fr with ID 468E76E8.002 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! Cc: Subject: Re: Finding slowdowns in pkg_install X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Jul 2007 17:07:54 -0000 Tim Kientzle said: > One approach I prototyped sometime back was to use > libarchive in pkg_add as follows: > * Open the archive > * Read +CONTENTS directly into memory (it's > guaranteed to always be first in the archive) I can only concur with that. In my program http://www.lpthe.jussieu.fr/~talon/check_pkg.py i discovered that memory mapping +CONTENTS and then working in memory before rewriting it was around 5 times faster than reading line by line and parsing each line. See function fix_pkg_database(). By the way i am writing a new +CONTENTS fileand then renaming it, which avoids leaving a mess if something goes astray like portupgrade does. -- Michel TALON