From owner-freebsd-net@FreeBSD.ORG Wed Jan 9 21:13:12 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 04071883 for ; Wed, 9 Jan 2013 21:13:12 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id D7064E4F for ; Wed, 9 Jan 2013 21:13:11 +0000 (UTC) Received: from pakbsde14.localnet (unknown [38.105.238.108]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 3A836B943; Wed, 9 Jan 2013 16:13:11 -0500 (EST) From: John Baldwin To: freebsd-net@freebsd.org Subject: Re: To SMP or not to SMP Date: Wed, 9 Jan 2013 15:54:43 -0500 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p22; KDE/4.5.5; amd64; ; ) References: <1357743631.24911.YahooMailClassic@web121605.mail.ne1.yahoo.com> In-Reply-To: <1357743631.24911.YahooMailClassic@web121605.mail.ne1.yahoo.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201301091554.43618.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 09 Jan 2013 16:13:11 -0500 (EST) Cc: Barney Cordoba , erichsfreebsdlist@alogt.com, jack.vogel@gmail.com, atkin901@gmail.com, sthaug@nethelp.no X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jan 2013 21:13:12 -0000 On Wednesday, January 09, 2013 10:00:31 am Barney Cordoba wrote: > > --- On Wed, 1/9/13, sthaug@nethelp.no wrote: > > > From: sthaug@nethelp.no > > Subject: Re: To SMP or not to SMP > > To: erichsfreebsdlist@alogt.com > > Cc: barney_cordoba@yahoo.com, freebsd-net@freebsd.org, jack.vogel@gmail.com, atkin901@gmail.com > > Date: Wednesday, January 9, 2013, 9:32 AM > > > > 4BSD runs pretty well with > > an SMP kernel. I can test ULE and compare > > > > easily. A no SMP kernel is problematic as the igb > > driver doesn't seem > > > > to work and my onboard NICs are, sadly, igb. > > > > > > > this is bad luck. I know of the kernels as I have had > > SMP and single > > > CPU machines since 4.x times. > > > > I have had igb working with both SMP and non-SMP kernel for > > at least a > > year or two, 8.x-STABLE. No specific problems. > > > > Steinar Haug, Nethelp consulting, sthaug@nethelp.no > > > > Maybe a problem with "legacy interrupts" on more modern processors? > I'm using an E5520 and while the NIC inits ok, it just doesnt seem to > gen interrupts. I can't spend much time debugging it.... > > I notice that HAMMER kernels use MSI/X interrupts whether SMP is enabled > or not, while i386 kernels seem to require APIC. Is there some physical > reason for this? MSI always requires APIC. (MSI interrupts on x86 have to be delivered to a local APIC, no way to send them to 8259As.) You can build an i386 kernel with device apic but without 'options SMP' which is akin to leaving SMP out of an amd64 kernel. Removing SMP on x86 changes the following things: - Spin mutexes just disable interrupts on the local CPU and don't use any atomic operations at all. All other lock types work the same. - atomic operations don't use the "lock" prefix so are cheaper. However, the atomic op used for locks (cmpxchg) has an implicit "lock" prefix, so this isn't but so much of a gain. -- John Baldwin