From owner-freebsd-performance@FreeBSD.ORG  Sun May  7 20:27:42 2006
Return-Path: <owner-freebsd-performance@FreeBSD.ORG>
X-Original-To: performance@freebsd.org
Delivered-To: freebsd-performance@FreeBSD.ORG
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 67ACD16A463;
	Sun,  7 May 2006 20:27:42 +0000 (UTC) (envelope-from hadara@bsd.ee)
Received: from mx2.starman.ee (smtp-out4.starman.ee [85.253.0.6])
	by mx1.FreeBSD.org (Postfix) with ESMTP id D843643D6B;
	Sun,  7 May 2006 20:27:33 +0000 (GMT) (envelope-from hadara@bsd.ee)
Received: from depression.softematic.com (depression.softematic.com
	[62.65.205.81])
	by mx2.starman.ee (Postfix) with ESMTP id 178A932C255;
	Sun,  7 May 2006 23:27:31 +0300 (EEST)
From: Sven Petai <hadara@bsd.ee>
To: Kris Kennaway <kris@obsecurity.org>
Date: Sun, 7 May 2006 23:27:22 +0300
User-Agent: KMail/1.9.1
References: <20060506150622.C17611@fledge.watson.org>
	<200605072200.42529.hadara@bsd.ee>
	<20060507191641.GA1851@xor.obsecurity.org>
In-Reply-To: <20060507191641.GA1851@xor.obsecurity.org>
MIME-Version: 1.0
Content-Type: text/plain;
  charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Message-Id: <200605072327.23901.hadara@bsd.ee>
X-Virus-Scanned: by Amavisd-New at mx2.starman.ee
Cc: rwatson@freebsd.org, performance@freebsd.org, current@freebsd.org
Subject: Re: Fine-grained locking for POSIX local sockets (UNIX domain
	sockets)
X-BeenThere: freebsd-performance@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Performance/tuning <freebsd-performance.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-performance>,
	<mailto:freebsd-performance-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-performance>
List-Post: <mailto:freebsd-performance@freebsd.org>
List-Help: <mailto:freebsd-performance-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-performance>,
	<mailto:freebsd-performance-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 07 May 2006 20:27:43 -0000

On Sunday 07 May 2006 22:16, you wrote:
> On Sun, May 07, 2006 at 10:00:41PM +0300, Sven Petai wrote:
> > The results in my mail were mean values over 2 runs,
> > only once did I see really huge (more than 10%) differences between
> > several subsequent runs with same settings, this case was clearly
> > mentioned in the results.
>
> FYI, 2 is not really enough, you should do at least 10 repetitions of
> each test to reduce variance (which can be a lot, despite what you
> saw!) and so that differences between them can be accurately
> estimated.  Ministat is really helpful for this.
>

I'm well aware that 2 is not enough for quality measurements and
I certainly would have liked to do more repetitions, but I was running against 
a clock - this machine might be shipped out to client any
time and I wanted to test several combinations of OS [fbsd 6, fbsd current, 
current + watsons patch, linux] with different threading library and 
scheduler combinations at different thread counts and nice values. 
This creates nice combinatorial explosion.

Even a single run on one OS ver + one of the schedulers with 2 repetitions of 
each test takes about 2 hours, so I had to make some compromises.

But i believe the trends are clear enough from these results and while I 
certainly can't say anything about <~5% performance changes what we see are
consistent trends and 20%+ performance differences.