From owner-freebsd-performance@FreeBSD.ORG Mon Jan 31 15:25:19 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B96C316A4CE for ; Mon, 31 Jan 2005 15:25:19 +0000 (GMT) Received: from cyrus.watson.org (cyrus.watson.org [204.156.12.53]) by mx1.FreeBSD.org (Postfix) with ESMTP id 673DD43D31 for ; Mon, 31 Jan 2005 15:25:19 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by cyrus.watson.org (Postfix) with SMTP id B72A746B0D; Mon, 31 Jan 2005 10:25:18 -0500 (EST) Date: Mon, 31 Jan 2005 15:24:39 +0000 (GMT) From: Robert Watson X-Sender: robert@fledge.watson.org To: "Jim C. Nasby" In-Reply-To: <20050130230527.GR64304@decibel.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-performance@freebsd.org Subject: Re: Automated performance testing X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Jan 2005 15:25:19 -0000 On Sun, 30 Jan 2005, Jim C. Nasby wrote: > With all the discussion of performance testing between 4.11, 5.3, and > Linux, would it be useful to make performance testing part of the > automated testing that already occurs (via tinderbox, iirc). Doing so > might make it easier to detect performance impacting changes, as well as > making performance testing easier in general. Yes, it would be quite valuable. I've been hoping to set up something like this for a while, but have never found the opportunity. I have been tracking the long term behavior of MySQL performance as part of the netperf work, but because testing is fairly hardware and time consuming, the polling intervals are uneven, and not quite close enough to nail down culprits. I'd really like to see a small and fairly well-defined set of tests run every couple of days so we can show long term graphs, and catch regressions quickly. Unfortunately, this is a bit harder than tinder-boxing, because it involves swapping out whole system configurations, recovering from the inevitable failure modes, etc, which proves to be the usual sticking point in implementing this. However, I'd love to see someone work on it :-). Robert N M Watson