From owner-freebsd-net@FreeBSD.ORG Thu Apr 24 16:46:44 2008 Return-Path: Delivered-To: net@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 428561065670; Thu, 24 Apr 2008 16:46:44 +0000 (UTC) (envelope-from tom.hurst@clara.net) Received: from spork.qfe3.net (spork.qfe3.net [212.13.207.101]) by mx1.freebsd.org (Postfix) with ESMTP id 16CF28FC19; Thu, 24 Apr 2008 16:46:43 +0000 (UTC) (envelope-from tom.hurst@clara.net) Received: from [81.104.123.28] (helo=voi.aagh.net) by spork.qfe3.net with esmtp (Exim 4.66 (FreeBSD)) (envelope-from ) id 1Jp4Je-000CgU-RR; Thu, 24 Apr 2008 17:28:58 +0100 Received: from freaky by voi.aagh.net with local (Exim 4.69 (FreeBSD)) (envelope-from ) id 1Jp4Je-000DEx-L8; Thu, 24 Apr 2008 17:28:58 +0100 Date: Thu, 24 Apr 2008 17:28:58 +0100 From: Thomas Hurst To: Jeremy Chadwick Message-ID: <20080424162858.GA46157@voi.aagh.net> Mail-Followup-To: Jeremy Chadwick , "Arno J. Klaassen" , Clayton Milos , Kris Kennaway , stable@FreeBSD.ORG, net@FreeBSD.ORG References: <20080421094718.GY25623@hub.freebsd.org> <20080421154333.GA96237@eos.sc1.parodius.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20080421154333.GA96237@eos.sc1.parodius.com> Organization: Not much. User-Agent: Mutt/1.5.17 (2007-11-01) Sender: Thomas Hurst Cc: Clayton Milos , Kris Kennaway , stable@FreeBSD.ORG, net@FreeBSD.ORG Subject: Re: nfs-server silent data corruption X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Apr 2008 16:46:44 -0000 * Jeremy Chadwick (koitsu@freebsd.org) wrote: > > I added it directly to the 2nd CPU (diagram on page 9 of > > http://www.tyan.com/manuals/m_s2895_101.pdf) and the problem > > seems to be the interaction between nfe0 and powerd .... : > > That board is the weirdest thing I've seen in years. K8WE's a very popular workstation board. I've been using one for years. > Two separate CPUs using a single (shared) memory controller, Er, no. Where are you getting that? 4 DIMMs are connected per CPU, though it's hardly strange to only have one, just cheap and nasty. > two separate (and different!) nVidia chipsets, a SMSC I/O controller > probably used for serial and parallel I/O, Er, so? Sun X4x00 M2's do exactly the same; they run a 2200 off one CPU and a 2050 off another (via an AMD 8132 no less). !M2's did much the same with a pair of AMD 8131's. They use SMSC IO controllers too: http://www.sun.com/servers/entry/x4100/arch-wp.pdf http://www.sun.com/servers/netra/x4200/wp.pdf We've used dozens of these systems in production in various configurations for years wuthout a problem. > two separate nVidia NICs with Marvell PHYs (yet somehow you can bridge > the two NICs and PHYs?), They're not seperate, they hang off the same chip according to the linked document. They are nve nonsense, though, not worth using imo. > two separate PCI-e busses (each associated with a separate nVidia > chipset), two separate PCI-X busses... the list continues. Again, nothing surprising. Each CPU gets its own bus via its own HT link. Back in the day when the K8WE was first released, this was the only way to get a pair of 16x PCIe slots. > I know you don't need opinions at this point, but what a behemoth. I > can't imagine that thing running reliably. The only stability problems I've experience have been the occasional lockup using PowerNow since migrating from dual single core to dual dual core. -- Thomas 'Freaky' Hurst http://hur.st/