From owner-freebsd-current@FreeBSD.ORG Tue Mar 4 10:03:52 2008 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CDCD41065672; Tue, 4 Mar 2008 10:03:52 +0000 (UTC) (envelope-from kris@FreeBSD.org) Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id E68D18FC25; Tue, 4 Mar 2008 10:03:51 +0000 (UTC) (envelope-from kris@FreeBSD.org) Message-ID: <47CD1E88.7000608@FreeBSD.org> Date: Tue, 04 Mar 2008 11:03:52 +0100 From: Kris Kennaway User-Agent: Thunderbird 2.0.0.12 (Macintosh/20080213) MIME-Version: 1.0 To: Jeff Roberson References: <200803020821.m228L0Yw042389@repoman.freebsd.org> <20080301222513.Y920@desktop> <18380.58229.379738.408078@hergotha.csail.mit.edu> <20080303214214.G920@desktop> In-Reply-To: <20080303214214.G920@desktop> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Garrett Wollman , current@freebsd.org Subject: Re: cvs commit: src/sys/kern sched_ule.c X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Mar 2008 10:03:52 -0000 Jeff Roberson wrote: > On Tue, 4 Mar 2008, Garrett Wollman wrote: > >> <> said: >> >>> Kris has done some excellent benchmarking as usual. Here you can see >>> the >>> improvement in postgres depending on various scheduler debug settings: >> >>> http://people.freebsd.org/~kris/scaling/pgsql-16cpu.png >> >> Can you comment on the area under the knee in the 8-cpu topologies? I >> seems surprising that 16 cores performs worse than 8 cores in this >> regime. > > Depending on the flags you can see different scaling properties of > different cpu selection algorithms. That's what the userret=x, > tryself=y parameters are changing. Certain parameters can cause less > concurrency which works better when workloads are heavily contended. > > See the light blue line, tryself=0, userret=0. This scales up more > poorly because there is less concurrency when there is no lock > contention but behaves better when there is contention because we're > less likely to distribute load that would preempt a lock holder. > > The default settings scale the best when there is little or no > contention. That's userret=1, tryself=1. There are other parameters > that are important but these were the ones we were most recently > experimenting with. This drops off harshly when there is significant > contention because most of the threads end up blocked against the same > lock and may be preempted then rely on priority propagation to kick in. > > The default settings should encourage further refinements to subsystem > locking to yield the best performance. I didnt run the 8-core configuration with the ULE topology patch, so part of the reason why it has a kink at 5 threads is probably due to poor scheduling. This system is very sensitive to scheduling decisions, as you can see from the previous CVS curve. I think there is also something else going on at high loads (>15) on this test, so it should be viewed as a WIP. Specifically, contention doesnt seem to be high enough to account for a 30% performance drop, and I see similar drops on other 8-core tests where contention is eliminated. What you should focus on is the large difference between the green curve showing previous CVS performance, with the brown curve showing current default performance. Kris