From owner-freebsd-performance@FreeBSD.ORG  Wed Jan 23 20:48:05 2008
Return-Path: <owner-freebsd-performance@FreeBSD.ORG>
Delivered-To: performance@FreeBSD.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 5AB5116A420;
	Wed, 23 Jan 2008 20:48:05 +0000 (UTC)
	(envelope-from kris@FreeBSD.org)
Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28])
	by mx1.freebsd.org (Postfix) with ESMTP id 0A12A13C44B;
	Wed, 23 Jan 2008 20:48:03 +0000 (UTC)
	(envelope-from kris@FreeBSD.org)
Message-ID: <4797A802.8060509@FreeBSD.org>
Date: Wed, 23 Jan 2008 21:48:02 +0100
From: Kris Kennaway <kris@FreeBSD.org>
User-Agent: Thunderbird 2.0.0.9 (Macintosh/20071031)
MIME-Version: 1.0
To: Robert Watson <rwatson@FreeBSD.org>
References: <4796C717.9000507@cederstrand.dk>	<20080123193400.N63024@fledge.watson.org>	<4797A245.7080202@cederstrand.dk>
	<20080123202433.E63024@fledge.watson.org>
In-Reply-To: <20080123202433.E63024@fledge.watson.org>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: performance@FreeBSD.org, Erik Cederstrand <erik@cederstrand.dk>
Subject: Re: Performance Tracker project update
X-BeenThere: freebsd-performance@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Performance/tuning <freebsd-performance.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-performance>,
	<mailto:freebsd-performance-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-performance>
List-Post: <mailto:freebsd-performance@freebsd.org>
List-Help: <mailto:freebsd-performance-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-performance>,
	<mailto:freebsd-performance-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 23 Jan 2008 20:48:05 -0000

Robert Watson wrote:

>> I think it's best if participating machines supply data regularly for 
>> an extended period of time. Single or infrequent data points for a 
>> specific configuration don't make much sense. We need to compare 
>> apples to apples.
> 
> Yes -- I was mostly thinking about backdating in order to play "catchup" 
> when a new benchmark is introduced.

One thing I am looking at is how to best create a library of world 
tarballs that can be used to populate a nfsroot (or hybrid of periodic 
tarballs + binary diffs to save space).  Then you could provide your 
benchmark in a standardized format (start/end/cleanup scripts, etc) and 
tell a machine "go and run this benchmark on every daily snapshot for 
the last year and give me the numbers".

Kris