From owner-freebsd-ports@FreeBSD.ORG  Fri Jul  6 16:01:54 2007
Return-Path: <owner-freebsd-ports@FreeBSD.ORG>
X-Original-To: ports@freebsd.org
Delivered-To: freebsd-ports@FreeBSD.ORG
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id AD2BE16A400
	for <ports@freebsd.org>; Fri,  6 Jul 2007 16:01:54 +0000 (UTC)
	(envelope-from kientzle@freebsd.org)
Received: from kientzle.com (h-66-166-149-50.snvacaid.covad.net
	[66.166.149.50])
	by mx1.freebsd.org (Postfix) with ESMTP id 4FDF913C457
	for <ports@freebsd.org>; Fri,  6 Jul 2007 16:01:54 +0000 (UTC)
	(envelope-from kientzle@freebsd.org)
Received: from [10.0.0.222] (p54.kientzle.com [66.166.149.54])
	by kientzle.com (8.12.9/8.12.9) with ESMTP id l66FY1H7038895;
	Fri, 6 Jul 2007 08:34:01 -0700 (PDT)
	(envelope-from kientzle@freebsd.org)
Message-ID: <468E60E9.80507@freebsd.org>
Date: Fri, 06 Jul 2007 08:34:01 -0700
From: Tim Kientzle <kientzle@freebsd.org>
User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20060422
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: Garrett Cooper <youshi10@u.washington.edu>
References: <468C96C0.1040603@u.washington.edu>
	<468C9718.1050108@u.washington.edu>
In-Reply-To: <468C9718.1050108@u.washington.edu>
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
Cc: ports@freebsd.org, hackers@freebsd.org
Subject: Re: Finding slowdowns in pkg_install (continuations of
	previous	threads)
X-BeenThere: freebsd-ports@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Porting software to FreeBSD <freebsd-ports.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-ports>,
	<mailto:freebsd-ports-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-ports>
List-Post: <mailto:freebsd-ports@freebsd.org>
List-Help: <mailto:freebsd-ports-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-ports>,
	<mailto:freebsd-ports-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 06 Jul 2007 16:01:54 -0000

>>    I'm currently running a gamut of tests (500 tests, per package -- 
>> 128 total on my server), and outputting all data to CSV files to 
>> interpret later, using another Perl script to interpret calculated 
>> averages and standard deviations.

Excellent!  Much-needed work.

>>    Using basic printf(2)'s with clock_gettime(2) I have determined 
>> that the majority of the issues are disk-bound (as Tom Kientzle put 
>> it).

Next question:  What are those disk operations and are any
of them redundant?

>> The scope of my problem is not to analyze tar,...

I've spent the last three years+ doing exactly that.
Make sure you're using the newest bsdtar/libarchive,
which has some very noticable performance improvements.

>> but I've 
>> discovered that a lot of time is spent in reading and interpreting the 
>> +CONTENTS and related files (most notably in parsing commands to be 
>> honest).

Oh?  That's interesting.  Is data being re-parsed (in which case
some structural changes to parse it once and store the results
may help)?  Or is the parser just slow?

>>    Will post more conclusive results tomorrow once all of my results 
>> are available.

I don't follow ports@ so didn't see these "conclusive results"
of yours.  I'm very interested, though.

Tim Kientzle