From owner-freebsd-net@FreeBSD.ORG Mon Jan 14 20:50:31 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id A5F3F58E for ; Mon, 14 Jan 2013 20:50:31 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id 80D6B996 for ; Mon, 14 Jan 2013 20:50:31 +0000 (UTC) Received: from pakbsde14.localnet (unknown [38.105.238.108]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id CE716B95E; Mon, 14 Jan 2013 15:50:30 -0500 (EST) From: John Baldwin To: freebsd-net@freebsd.org Subject: Re: To SMP or not to SMP Date: Mon, 14 Jan 2013 15:07:50 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p22; KDE/4.5.5; amd64; ; ) References: <330287752.17.1358057713463.JavaMail.root@daemoninthecloset.org> In-Reply-To: <330287752.17.1358057713463.JavaMail.root@daemoninthecloset.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201301141507.50250.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 14 Jan 2013 15:50:30 -0500 (EST) Cc: Peter Jeremy , Bryan Venteicher X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Jan 2013 20:50:31 -0000 On Sunday, January 13, 2013 1:15:13 am Bryan Venteicher wrote: > > ----- Original Message ----- > > From: "John Baldwin" > > To: freebsd-net@freebsd.org > > Cc: "Barney Cordoba" , "Peter Jeremy" > > Sent: Friday, January 11, 2013 9:39:17 AM > > Subject: Re: To SMP or not to SMP > > > > On Thursday, January 10, 2013 02:36:59 PM Peter Jeremy wrote: > > > On 2013-Jan-07 18:25:58 -0800, Barney Cordoba > > > > > wrote: > > > >I have a situation where I have to run 9.1 on an old single core > > > >box. Does anyone have a handle on whether it's better to build a > > > >non > > > >SMP kernel or to just use a standard SMP build with just the one > > > >core? > > > > > > Another input for this decision is kern/173322. Currently on x86, > > > atomic operations within kernel modules are implemented using calls > > > to code in the kernel, which do or don't use lock prefixes > > > depending > > > on whethur the kernel was built as SMP. My proposed change changes > > > kernel modules to inline atomic operations but always include lock > > > prefixes (effectively reverting r49999). I'm appreciate anyone who > > > feels like testing the impact of this change. > > > > Presumably a locked atomic op is cheaper than a function call then? > > The > > current setup assumes the opposite. > > > > I think we should actually do this for atomics in modules on x86: > > > > 1) If a module is built standalone, it should do whichever is cheaper: > > a function call or always use "LOCK". > > > > 2) If a module is built as part of the kernel build, it should use inlined > > atomics that match what the kernel does. Thus, modules built with a > > non-SMP kernel would use inlined atomic ops that do not use LOCK. We > > have a way to detect this now (some HAVE_FOO #define added in the past > > few years) that we didn't back when this bit of atomic.h was > > written. > > > > It would be nice to have the LOCK variants available even on UP > kernels in non-hackish way. For VirtIO, we need to handle an guest > UP kernel running on an SMP host. Whether this is an #define that > forces the SMP atomics to be inlined, or if they're exposed with > an _smp suffix. > > VirtIO currently uses mb() to enforce ordering. I have a patch > to change to use atomic(9), but can only do so when VirtIO is > included in the an SMP kernel (among other constraints - must > have 16-bit atomic operations too). > > (FreeBSD's VirtIO is x86 only for now - but that will be changing > soon; I haven't looked if other arch's atomic(9) behave differently > for UP/SMP.) Only x86 does this weirdness. The simplest workaround might be to require guest kernels to be compiled with SMP for now. -- John Baldwin