From owner-freebsd-performance@FreeBSD.ORG Tue May 10 19:11:15 2005 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7140C16A4CE for ; Tue, 10 May 2005 19:11:15 +0000 (GMT) Received: from smtp815.mail.sc5.yahoo.com (smtp815.mail.sc5.yahoo.com [66.163.170.1]) by mx1.FreeBSD.org (Postfix) with SMTP id 337CF43D48 for ; Tue, 10 May 2005 19:11:15 +0000 (GMT) (envelope-from noackjr@alumni.rice.edu) Received: from unknown (HELO optimator.noacks.org) (noacks@swbell.net@70.240.205.64 with login) by smtp815.mail.sc5.yahoo.com with SMTP; 10 May 2005 18:35:50 -0000 Received: from localhost (localhost [127.0.0.1]) by optimator.noacks.org (Postfix) with ESMTP id 128266125; Tue, 10 May 2005 13:35:49 -0500 (CDT) Received: from optimator.noacks.org ([127.0.0.1]) by localhost (optimator.noacks.org [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 41547-04; Tue, 10 May 2005 13:35:47 -0500 (CDT) Received: from [127.0.0.1] (optimator [192.168.1.11]) by optimator.noacks.org (Postfix) with ESMTP id 37CF960F3; Tue, 10 May 2005 13:35:47 -0500 (CDT) Message-ID: <4280FEFC.4080107@alumni.rice.edu> Date: Tue, 10 May 2005 13:35:40 -0500 From: Jonathan Noack User-Agent: Mozilla Thunderbird 1.0.2 (Windows/20050317) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Bakul Shah References: <200505101518.j4AFImSv071163@gate.bitblocks.com> In-Reply-To: <200505101518.j4AFImSv071163@gate.bitblocks.com> X-Enigmail-Version: 0.91.0.0 OpenPGP: id=991D8195; url=http://www.noacks.org/cert/noackjr.asc Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig27BDEB018E649B3C3993C4E7" X-Virus-Scanned: amavisd-new at noacks.org cc: performance@freebsd.org Subject: Re: Regression testing (was Re: Performance issue) X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: noackjr@alumni.rice.edu List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 May 2005 19:11:15 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig27BDEB018E649B3C3993C4E7 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 5/10/2005 10:18 AM, Bakul Shah wrote: > This thread makes me wonder if there is value in runing > performance tests on a regular basis. This would give an > early warning of any peformance loss and can be a useful > forensic tool (one can pinpoint when some performance curve > changed discontinuously even though at the time of change it > may be too small to be noticed). Over a period of time > one can gain a view of how the performance evolves. > > This would not be a single metric but a set of low and high > level measures: such as syscall overhead, interrupt overhead, > specific h/w devices, disk and fs performance for various > filesystems and file sizes, networking data and pkt > throughput, routing performance, VM, other subsystems, effect > of SMP, various threading libraries, scaling with number of > users/programs/cpus/memory, typical applications under normal > and stressed loads, compile time for the system and kernel > etc. etc. etc. > > The setup would allow for easy addition of new benchmarks > (the only way anything like this can be bootstrapped). Of > course, one would need to record disk/processor/memory speed > and capacities + kernel config options, system build tools > and their options to interpret the results as best as > possible. For the results to be useful the setup has to > remain as stable as possible for a long time. > > [While I am dreaming...] A follow on project would be to > create visualization tools -- mainly graphing and comparing > graphs. It would be neat if one can click on a performance > graph to zoom in or see commits made during some selected > period. > > Such a detailed look, combined with profiling can help people > focus on specific hotspots & feel good about any improvements > they are making. This can be a great way to rope in new > people;-) Sounds great! When do you begin? ;-) This has been proposed before and has been (to my knowledge) universally accepted as a Good Idea. If you have the interest and time to devote to it, I would urge you to work on it. The benefit to the community would be huge. -- Jonathan Noack | noackjr@alumni.rice.edu | OpenPGP: 0x991D8195 --------------enig27BDEB018E649B3C3993C4E7 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (MingW32) iD8DBQFCgP8BUFz01pkdgZURAjrUAKCgrZwGxaFIRgtN8QH4nJ9FH9w3DgCeMg0K t+VXCdCckzeo4fUshP3FP+U= =i/ts -----END PGP SIGNATURE----- --------------enig27BDEB018E649B3C3993C4E7--