Date: Tue, 10 May 2005 13:35:40 -0500 From: Jonathan Noack <noackjr@alumni.rice.edu> To: Bakul Shah <bakul@BitBlocks.com> Cc: performance@freebsd.org Subject: Re: Regression testing (was Re: Performance issue) Message-ID: <4280FEFC.4080107@alumni.rice.edu> In-Reply-To: <200505101518.j4AFImSv071163@gate.bitblocks.com> References: <200505101518.j4AFImSv071163@gate.bitblocks.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig27BDEB018E649B3C3993C4E7 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 5/10/2005 10:18 AM, Bakul Shah wrote: > This thread makes me wonder if there is value in runing > performance tests on a regular basis. This would give an > early warning of any peformance loss and can be a useful > forensic tool (one can pinpoint when some performance curve > changed discontinuously even though at the time of change it > may be too small to be noticed). Over a period of time > one can gain a view of how the performance evolves. > > This would not be a single metric but a set of low and high > level measures: such as syscall overhead, interrupt overhead, > specific h/w devices, disk and fs performance for various > filesystems and file sizes, networking data and pkt > throughput, routing performance, VM, other subsystems, effect > of SMP, various threading libraries, scaling with number of > users/programs/cpus/memory, typical applications under normal > and stressed loads, compile time for the system and kernel > etc. etc. etc. > > The setup would allow for easy addition of new benchmarks > (the only way anything like this can be bootstrapped). Of > course, one would need to record disk/processor/memory speed > and capacities + kernel config options, system build tools > and their options to interpret the results as best as > possible. For the results to be useful the setup has to > remain as stable as possible for a long time. > > [While I am dreaming...] A follow on project would be to > create visualization tools -- mainly graphing and comparing > graphs. It would be neat if one can click on a performance > graph to zoom in or see commits made during some selected > period. > > Such a detailed look, combined with profiling can help people > focus on specific hotspots & feel good about any improvements > they are making. This can be a great way to rope in new > people;-) Sounds great! When do you begin? ;-) This has been proposed before and has been (to my knowledge) universally accepted as a Good Idea. If you have the interest and time to devote to it, I would urge you to work on it. The benefit to the community would be huge. -- Jonathan Noack | noackjr@alumni.rice.edu | OpenPGP: 0x991D8195 --------------enig27BDEB018E649B3C3993C4E7 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (MingW32) iD8DBQFCgP8BUFz01pkdgZURAjrUAKCgrZwGxaFIRgtN8QH4nJ9FH9w3DgCeMg0K t+VXCdCckzeo4fUshP3FP+U= =i/ts -----END PGP SIGNATURE----- --------------enig27BDEB018E649B3C3993C4E7--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4280FEFC.4080107>