From owner-freebsd-current@FreeBSD.ORG Sun May 7 20:27:42 2006 Return-Path: X-Original-To: current@freebsd.org Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 67ACD16A463; Sun, 7 May 2006 20:27:42 +0000 (UTC) (envelope-from hadara@bsd.ee) Received: from mx2.starman.ee (smtp-out4.starman.ee [85.253.0.6]) by mx1.FreeBSD.org (Postfix) with ESMTP id D843643D6B; Sun, 7 May 2006 20:27:33 +0000 (GMT) (envelope-from hadara@bsd.ee) Received: from depression.softematic.com (depression.softematic.com [62.65.205.81]) by mx2.starman.ee (Postfix) with ESMTP id 178A932C255; Sun, 7 May 2006 23:27:31 +0300 (EEST) From: Sven Petai To: Kris Kennaway Date: Sun, 7 May 2006 23:27:22 +0300 User-Agent: KMail/1.9.1 References: <20060506150622.C17611@fledge.watson.org> <200605072200.42529.hadara@bsd.ee> <20060507191641.GA1851@xor.obsecurity.org> In-Reply-To: <20060507191641.GA1851@xor.obsecurity.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200605072327.23901.hadara@bsd.ee> X-Virus-Scanned: by Amavisd-New at mx2.starman.ee Cc: rwatson@freebsd.org, performance@freebsd.org, current@freebsd.org Subject: Re: Fine-grained locking for POSIX local sockets (UNIX domain sockets) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 May 2006 20:27:43 -0000 On Sunday 07 May 2006 22:16, you wrote: > On Sun, May 07, 2006 at 10:00:41PM +0300, Sven Petai wrote: > > The results in my mail were mean values over 2 runs, > > only once did I see really huge (more than 10%) differences between > > several subsequent runs with same settings, this case was clearly > > mentioned in the results. > > FYI, 2 is not really enough, you should do at least 10 repetitions of > each test to reduce variance (which can be a lot, despite what you > saw!) and so that differences between them can be accurately > estimated. Ministat is really helpful for this. > I'm well aware that 2 is not enough for quality measurements and I certainly would have liked to do more repetitions, but I was running against a clock - this machine might be shipped out to client any time and I wanted to test several combinations of OS [fbsd 6, fbsd current, current + watsons patch, linux] with different threading library and scheduler combinations at different thread counts and nice values. This creates nice combinatorial explosion. Even a single run on one OS ver + one of the schedulers with 2 repetitions of each test takes about 2 hours, so I had to make some compromises. But i believe the trends are clear enough from these results and while I certainly can't say anything about <~5% performance changes what we see are consistent trends and 20%+ performance differences.