From owner-freebsd-hackers@FreeBSD.ORG Sun Jun 17 06:47:53 2007 Return-Path: X-Original-To: hackers@freebsd.org Delivered-To: freebsd-hackers@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7BD6616A468; Sun, 17 Jun 2007 06:47:53 +0000 (UTC) (envelope-from youshi10@u.washington.edu) Received: from mxout3.cac.washington.edu (mxout3.cac.washington.edu [140.142.32.166]) by mx1.freebsd.org (Postfix) with ESMTP id 5958013C45D; Sun, 17 Jun 2007 06:47:53 +0000 (UTC) (envelope-from youshi10@u.washington.edu) Received: from smtp.washington.edu (smtp.washington.edu [140.142.32.141] (may be forged)) by mxout3.cac.washington.edu (8.13.7+UW06.06/8.13.7+UW07.05) with ESMTP id l5H6lqr0010808 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Sat, 16 Jun 2007 23:47:52 -0700 X-Auth-Received: from [192.168.10.45] (c-24-10-12-194.hsd1.ca.comcast.net [24.10.12.194]) (authenticated authid=youshi10) by smtp.washington.edu (8.13.7+UW06.06/8.13.7+UW07.03) with ESMTP id l5H6lq9D029341 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Sat, 16 Jun 2007 23:47:52 -0700 Message-ID: <4674D917.7000502@u.washington.edu> Date: Sat, 16 Jun 2007 23:47:51 -0700 From: Garrett Cooper User-Agent: Thunderbird 2.0.0.4 (Windows/20070604) MIME-Version: 1.0 To: Tim Kientzle References: <46703EE9.1030804@freebsd.org> <4674B268.4030502@u.washington.edu> <4674BE32.300@freebsd.org> In-Reply-To: <4674BE32.300@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-PMX-Version: 5.3.1.294258, Antispam-Engine: 2.5.1.298604, Antispam-Data: 2007.6.16.232633 X-Uwash-Spam: Gauge=IIIIIII, Probability=7%, Report='__CT 0, __CTE 0, __CT_TEXT_PLAIN 0, __HAS_MSGID 0, __MIME_TEXT_ONLY 0, __MIME_VERSION 0, __SANE_MSGID 0, __USER_AGENT 0' Cc: hackers@freebsd.org Subject: Re: Using shell commands versus C equivalents X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Jun 2007 06:47:53 -0000 Tim Kientzle wrote: >> Also, were the bottlenecks seen in pkg_delete and pkg_add, or does >> it appear to be distributed across the board? > > The biggest time sink in pkg_add is writing each file to a temp > dir then copying it to its final location. There are a couple > of strategies for avoiding this (by writing the files directly > to their final location), but it basically requires rewriting > pkg_add from scratch. I prototyped this a long time ago and > found about a 3x speedup. (Parts of that prototype eventually > became libarchive.) > > I haven't looked closely at pkg_delete, but I doubt there's > much that can be done to speed it up; once you've examined the > dependency information to determine what can be deleted, > actually removing the files is a pretty straightforward > operation. > > The two operations that people focus on performance issues have > been index rebuilding (which requires inspecting every port in > /usr/ports) and update (which requires inspecting every > installed port). The modular Xorg is especially going to stress > updates, since it greatly increases the number of ports on the > average system. > > One useful tool: "truss" can include timing information that > can give a lot of insight into where a program is really > spending time. > > Tim Kientzle > Hmmm.. not sure if you're referring to the temp creation of files in the playpen portion of the set of programs, or something else, but as I see it the playpen idea is a good one because it's like the Gentoo Linux version of a sandbox, and in case something goes wrong during an install or the user backs out, that's the way to go when dealing with a partially created / installed package. Maybe the strategy behind the playpen should be revised though.. Another question for everyone who's experiencing really slow pkg_add times though -- is it maybe because of slice boundaries that need to be crossed going from the work dir to the installation slice in moving files, perhaps? I'll definitely look into strace'ing (not really a big fan of truss(1) yet) the operation though, just to see how fast or slow stuff is. -Garrett