From owner-freebsd-hackers@FreeBSD.ORG Sun Jun 17 16:27:34 2007 Return-Path: X-Original-To: hackers@freebsd.org Delivered-To: freebsd-hackers@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AA62B16A468; Sun, 17 Jun 2007 16:27:34 +0000 (UTC) (envelope-from youshi10@u.washington.edu) Received: from mxout7.cac.washington.edu (mxout7.cac.washington.edu [140.142.32.178]) by mx1.freebsd.org (Postfix) with ESMTP id 8557F13C4AD; Sun, 17 Jun 2007 16:27:34 +0000 (UTC) (envelope-from youshi10@u.washington.edu) Received: from smtp.washington.edu (smtp.washington.edu [140.142.33.9] (may be forged)) by mxout7.cac.washington.edu (8.13.7+UW06.06/8.13.7+UW07.05) with ESMTP id l5HGRX8B004838 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Sun, 17 Jun 2007 09:27:33 -0700 X-Auth-Received: from [192.168.10.45] (c-24-10-12-194.hsd1.ca.comcast.net [24.10.12.194]) (authenticated authid=youshi10) by smtp.washington.edu (8.13.7+UW06.06/8.13.7+UW07.03) with ESMTP id l5HGRW9T006430 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sun, 17 Jun 2007 09:27:33 -0700 Message-ID: <467560F4.9050007@u.washington.edu> Date: Sun, 17 Jun 2007 09:27:32 -0700 From: Garrett Cooper User-Agent: Thunderbird 2.0.0.4 (Windows/20070604) MIME-Version: 1.0 To: Andrew Turner References: <46703EE9.1030804@freebsd.org> <4674B268.4030502@u.washington.edu> <4674BE32.300@freebsd.org> <20070617203640.334524fc@hermies.int.fubar.geek.nz> In-Reply-To: <20070617203640.334524fc@hermies.int.fubar.geek.nz> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-PMX-Version: 5.3.1.294258, Antispam-Engine: 2.5.1.298604, Antispam-Data: 2007.6.17.91435 X-Uwash-Spam: Gauge=IIIIIII, Probability=7%, Report='__CP_URI_IN_BODY 0, __CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __HAS_MSGID 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __SANE_MSGID 0, __USER_AGENT 0' Cc: hackers@freebsd.org, Tim Kientzle Subject: Re: Using shell commands versus C equivalents X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Jun 2007 16:27:34 -0000 Andrew Turner wrote: > On Sat, 16 Jun 2007 21:53:06 -0700 > Tim Kientzle wrote: > > >>> Also, were the bottlenecks seen in pkg_delete and pkg_add, or >>> does it appear to be distributed across the board? >>> >> The biggest time sink in pkg_add is writing each file to a temp >> dir then copying it to its final location. There are a couple >> of strategies for avoiding this (by writing the files directly >> to their final location), but it basically requires rewriting >> pkg_add from scratch. I prototyped this a long time ago and >> found about a 3x speedup. (Parts of that prototype eventually >> became libarchive.) >> > I've also seen a 3x speedup by using my reimplementation of pkg_add > using my package management library, libpkg (http://libpkg.berlios.de). > It is not production ready yet as if it fails partway through an > installation it won't clean up and installed files. > > > >> I haven't looked closely at pkg_delete, but I doubt there's >> much that can be done to speed it up; once you've examined the >> dependency information to determine what can be deleted, >> actually removing the files is a pretty straightforward >> operation. >> > I ran a quick test on installing and removing a single package with > both the cvs and my own version of pkg_delete. I got a small but > significant speed improvement with my implementation. The difference > was too small to be noticeable by a human though (from 0.11s to 0.07s). > > Andrew > > Your source looks very nice, but there are a few comments: 1. How do you read BDB stuff without including the BDB headers/libs? 2. I can't go and graft your libs, or do something similar with the current source because I don't want to break production code (pkg_install) of this importance and muck up a lot of user's systems irrevocably. I'll be looking at your source though for good ideas and see if I can integrate them into the existing pkg_install set though, and give you credit where necessary. Thanks, -Garrett