From owner-freebsd-hackers@FreeBSD.ORG Sat Dec 31 16:25:18 2005 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2CBD916A41F; Sat, 31 Dec 2005 16:25:18 +0000 (GMT) (envelope-from cracauer@schlepper.zs64.net) Received: from schlepper.zs64.net (schlepper.zs64.net [212.12.50.230]) by mx1.FreeBSD.org (Postfix) with ESMTP id B306143D5F; Sat, 31 Dec 2005 16:25:08 +0000 (GMT) (envelope-from cracauer@schlepper.zs64.net) Received: from schlepper.zs64.net (schlepper [212.12.50.230]) by schlepper.zs64.net (8.13.3/8.12.9) with ESMTP id jBVGP7fG062707; Sat, 31 Dec 2005 17:25:07 +0100 (CET) (envelope-from cracauer@schlepper.zs64.net) Received: (from cracauer@localhost) by schlepper.zs64.net (8.13.3/8.12.9/Submit) id jBVGOt4k062700; Sat, 31 Dec 2005 11:24:55 -0500 (EST) (envelope-from cracauer) Date: Sat, 31 Dec 2005 11:24:55 -0500 From: Martin Cracauer To: Robert Watson Message-ID: <20051231112455.A62454@cons.org> References: <43A26FFB.9080405@samsco.org> <20051216104022.A20877@cons.org> <20051217063409.GB19094@silverwraith.com> <20051231070750.T5785@fledge.watson.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20051231070750.T5785@fledge.watson.org>; from rwatson@freebsd.org on Sat, Dec 31, 2005 at 07:12:23AM +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: My wish list for 6.1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Dec 2005 16:25:18 -0000 Robert Watson wrote on Sat, Dec 31, 2005 at 07:12:23AM +0000: > > On Fri, 16 Dec 2005, Avleen Vig wrote: > > > On Fri, Dec 16, 2005 at 10:40:22AM -0500, Martin Cracauer wrote: > >>> 2. SMP kernels for install. Right now we only install a UP kernel, for > >>> performance reasons. We should be able to package both a UP and SMP > >>> kernel into the release bits, and have sysinstall install both. It > >>> should also select the correct one for the target system and make that > >>> the default on boot. > >> > >> If people are concerned about performance, I benchmarked a 6-beta kernel > >> SMP versus UP on a socket 939 Opteron. > > > > If those results are accurate, there's no real reason not to just use an SMP > > kernel on default install? > > This is an old thread that I'm just catching up on, but I figured I'd chime in > anyway: you have to be really careful benchmarking across CPU types and > configurations, as the performance characteristics of important insturctions > differ a lot across hardware variations. For example, the performance of > atomic operations, used to synchronize between CPUs, varies significantly by > CP, bus configuration, etc. On modern opteron hardware, the performance of > inter-CPU synchronization instructions is blindingly fast. On modern Xeon P4 > hardware, it is incredibly slow. Well, my runs included P4s and P4-based Xeons, and hyperthreading, too. The core of the problem here is that while my parallel benchmarks are partly system-call exercising, I use apache over localhost and zero-spaced files to get the disk and network out of the equitation. I think I have a solid framework in place to run parallel benchmarks and see the tradeoffs involved, but I need to fill it with activity that exercises what we want to see. Still, I bet that my measurements are good enough to label the SMP kernel "defaultable" for FreeBSD installations, from a performance standpoint. After all, I *do* test parallel activity, CPU-intensive and systemcall-intensive and various mixes thereof. Remember that those people who do a lot of parallel activity and hence would suffer from the additional locks in the SMP kernel are very likely to have a SMP system, dual-cores or at least hyperthreading in first place. On the other hand, people who use very low-end hardware to do demanding tasks are very likely to build their own kernel anyway. > Software optimized for the Opteron will > often perform much slower on Xeon P4 hardware as a result. P3 hardware tends > to behave a lot more like Opteron in terms of speed of insturctions relating > to disabling interrupts, where on P4 Xeon they are proprtionally much slower. > The critical section optimizations made by John Baldwin, and the movement to > critical sections in UMA and kernel malloc that I made, made a big performance > difference on Xeon P4 hardware, but relatively little difference on > Opteron. One thing I noticed is that anything P4-based is very sensitive to spinlocks being placed on the same cache line as the data it protects. Putting a lock into a struct without cache-line crossing padding means doom for the P4-based/netburst CPUs (I'm sure it's not a good thing for Opterons either but they don't seem to mind that much). Martin -- %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Martin Cracauer http://www.cons.org/cracauer/ FreeBSD - where you want to go, today. http://www.freebsd.org/