Date: Mon, 31 Jan 2005 22:59:58 -0600 From: "Jim C. Nasby" <decibel@decibel.org> To: Robert Watson <rwatson@FreeBSD.org> Cc: freebsd-performance@freebsd.org Subject: Re: Automated performance testing Message-ID: <20050201045958.GD32356@decibel.org> In-Reply-To: <Pine.NEB.3.96L.1050131152227.35704J-100000@fledge.watson.org> References: <20050130230527.GR64304@decibel.org> <Pine.NEB.3.96L.1050131152227.35704J-100000@fledge.watson.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Jan 31, 2005 at 03:24:39PM +0000, Robert Watson wrote: > > On Sun, 30 Jan 2005, Jim C. Nasby wrote: > > > With all the discussion of performance testing between 4.11, 5.3, and > > Linux, would it be useful to make performance testing part of the > > automated testing that already occurs (via tinderbox, iirc). Doing so > > might make it easier to detect performance impacting changes, as well as > > making performance testing easier in general. > > Yes, it would be quite valuable. I've been hoping to set up something > like this for a while, but have never found the opportunity. I have been > tracking the long term behavior of MySQL performance as part of the > netperf work, but because testing is fairly hardware and time consuming, > the polling intervals are uneven, and not quite close enough to nail down > culprits. I'd really like to see a small and fairly well-defined set of > tests run every couple of days so we can show long term graphs, and catch > regressions quickly. Unfortunately, this is a bit harder than > tinder-boxing, because it involves swapping out whole system > configurations, recovering from the inevitable failure modes, etc, which > proves to be the usual sticking point in implementing this. However, I'd > love to see someone work on it :-). FWIW, I'd suggest something less complicated than a database for performance testing. For starters, there's no way to isolate what part of the OS (if any) is responsible for a performance change. Databases also continually improve their own performance, so it's very much a moving standard. -- Jim C. Nasby, Database Consultant decibel@decibel.org Give your computer some brain candy! www.distributed.net Team #1828 Windows: "Where do you want to go today?" Linux: "Where do you want to go tomorrow?" FreeBSD: "Are you guys coming, or what?"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20050201045958.GD32356>