From owner-freebsd-current@FreeBSD.ORG Thu Aug 19 13:35:26 2010 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F131E1065693; Thu, 19 Aug 2010 13:35:26 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id BF2F48FC24; Thu, 19 Aug 2010 13:35:26 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 5D31046B32; Thu, 19 Aug 2010 09:35:26 -0400 (EDT) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 6015D8A04F; Thu, 19 Aug 2010 09:35:25 -0400 (EDT) From: John Baldwin To: freebsd-current@freebsd.org Date: Thu, 19 Aug 2010 09:22:53 -0400 User-Agent: KMail/1.13.5 (FreeBSD/7.3-CBSD-20100217; KDE/4.4.5; amd64; ; ) References: <4C673F02.8000805@FreeBSD.org> <4C6C4CBF.1040200@andric.com> <4C6C56A1.9050405@andric.com> In-Reply-To: <4C6C56A1.9050405@andric.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201008190922.54056.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Thu, 19 Aug 2010 09:35:25 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Dimitry Andric , Alan Cox , current@freebsd.org, Peter Jeremy Subject: Re: Official request: Please make GNU grep the default X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 19 Aug 2010 13:35:27 -0000 On Wednesday, August 18, 2010 5:54:41 pm Dimitry Andric wrote: > On 2010-08-18 23:12, Dimitry Andric wrote: > >> And one trial is not statistically valid - especially given the small > >> differences. How about multiple multiple trials with ministat. > > > > The result were averages of three trials > > Actually, since I kept using Doug's original grep-time-trial.sh, each of > the three 'trials' consisted of running grep 100 times, and the listed > time was the total elapsed time for those 100 runs. So I assume that > will reasonably average out the differences between each individual run? You need the distribution, not just the averages so you can detect outliers and determine the standard deviation and confidence intervals. You could use ministat on a file that contained all 100 runtimes perhaps. I would use at least 10 trials though, 3 is a bit small. -- John Baldwin