Date: Sat, 10 Nov 2007 15:18:23 -0500 From: "Josh Carroll" <josh.carroll@gmail.com> To: "Kris Kennaway" <kris@freebsd.org> Cc: Jeff Roberson <jroberson@chesapeake.net>, freebsd-performance@freebsd.org Subject: Re: ULE vs. 4BSD in RELENG_7 Message-ID: <8cb6106e0711101218l532fbd27xb21a03f71cb85975@mail.gmail.com> In-Reply-To: <47361134.3050301@FreeBSD.org> References: <8cb6106e0710230902x4edf2c8eu2d912d5de1f5d4a2@mail.gmail.com> <8cb6106e0711041243s37ef92e3i9c5c79827f547dbb@mail.gmail.com> <20071104151722.M544@10.0.0.1> <8cb6106e0711050141g674a7733h53f7e6a20e75ad07@mail.gmail.com> <20071106171059.M544@10.0.0.1> <8cb6106e0711062136r18ca8e67sbdb4d4372e10e086@mail.gmail.com> <20071109145827.W639@10.0.0.1> <8cb6106e0711091855x1305ee2bt6d57696df7c7564@mail.gmail.com> <8cb6106e0711101158o16887999rf7f7302ad44d0de8@mail.gmail.com> <47361134.3050301@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
> BTW, it doesn't make much sense to be measuring to millisecond precision > a quantity which has variation that is unknown but probably much larger > :) When trying to make comparisons to identify performance changes, a > careful statistical approach is necessary. > > Try the /usr/src/tools/tools/ministat utility for a simple and effective > way to compare these kinds of noisy measurements and extract reliable > comparisons. Thanks Kris! I have been taking an average of 3 values, but this looks extremely useful. I'll go ahead and build the necessary input files for ministat for stock 4BSD and stock ULE from RELENG_7, so I can generate something like the ministat README for future benchmark runs. I'll also increase my number of runs to 5 to provide a better statistical set of data. Appreciate the pointer, I wasn't aware of ministat. :) Thanks! Josh
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?8cb6106e0711101218l532fbd27xb21a03f71cb85975>