From owner-freebsd-questions@FreeBSD.ORG Thu Jul 28 16:31:44 2005 Return-Path: X-Original-To: questions@freebsd.org Delivered-To: freebsd-questions@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C609E16A41F for ; Thu, 28 Jul 2005 16:31:44 +0000 (GMT) (envelope-from dpk@dpk.net) Received: from shared10.hosting.flyingcroc.net (shared10.hosting.flyingcroc.net [207.246.149.144]) by mx1.FreeBSD.org (Postfix) with ESMTP id 69C8343D53 for ; Thu, 28 Jul 2005 16:31:44 +0000 (GMT) (envelope-from dpk@dpk.net) Received: from shared10.hosting.flyingcroc.net (localhost [127.0.0.1]) by shared10.hosting.flyingcroc.net (8.12.9p2/8.12.10) with ESMTP id j6SGVhrR083356 for ; Thu, 28 Jul 2005 09:31:43 -0700 (PDT) Received: from localhost (dpk@localhost) by shared10.hosting.flyingcroc.net (8.12.9p2/8.12.10/Submit) with ESMTP id j6SGVhS4083351 for ; Thu, 28 Jul 2005 09:31:43 -0700 (PDT) X-Authentication-Warning: shared10.hosting.flyingcroc.net: dpk owned process doing -bs Date: Thu, 28 Jul 2005 09:31:43 -0700 (PDT) From: dpk X-X-Sender: dpk@shared10.hosting.flyingcroc.net To: questions@freebsd.org In-Reply-To: <20050727185853.D23753@shared10.hosting.flyingcroc.net> Message-ID: <20050728093106.E79761@shared10.hosting.flyingcroc.net> References: <20050727185853.D23753@shared10.hosting.flyingcroc.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: Subject: Re: FreeBSD 5.4+SMP, severe network degredation X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jul 2005 16:31:44 -0000 By the way, I also compared GENERIC performance against GENERIC w/ "options SMP" added, and had the same results. On Wed, 27 Jul 2005, dpk wrote: > We just received several SuperMicro servers, 3.0Ghz Xeon x 2, 4GB RAM. > They're using the em driver and the ports are set to 1000Mbit (we also > tried 100Mbit/full duplex on the card and on the switch). They're running > FreeBSD 5.4. > > I ran a steady ping on a couple of them while they were running "GENERIC", > and then rebooted them with a kernel built with the "PAE" kernel included > with the installation, with "option SMP" added. > > The PAE-SMP-GENERIC kernel was built after cvsup'ing with "tag=RELENG_5_4" > and the uname reports "5.4-RELEASE-p5". > > Here are the ping results: > > GENERIC: > > 117 packets transmitted, 117 packets received, 0% packet loss > round-trip min/avg/max/stddev = 0.451/0.554/0.856/0.059 ms > > PAE-SMP-GENERIC: > > 102 packets transmitted, 102 packets received, 0% packet loss > round-trip min/avg/max/stddev = 0.569/4.262/7.944/2.065 ms > > Fetching a 637MB ISO from a local server, also on 100/FDX: > > GENERIC: > > /dev/null 100% of 637 MB 10 MBps 00m00s > > real 0m58.071s > user 0m1.954s > sys 0m6.278s > > PAE-SMP-GENERIC: > > /dev/null 100% of 637 MB 5764 kBps 00m00s > > real 1m53.324s > user 0m1.478s > sys 0m5.624s > > Running GENERIC, systat shows about 7000 interrupts/second, and around 600 > interrupts/second using PAE-SMP-GENERIC, while fetch was running. > > I've checked the errata and hardware notes, as well as gnats, and was not > able to find anything that explains or matches this behavior. We've run > SMP servers for years, using 4.5-4.11, but we've never seen the network > performance cut in half (or pings go up 10x). > > Removing "option SMP" makes the problem go away, but at a very significant > performance cost obviously. > > Could it be something from -p5? Is this explained/examined in a PR I've > missed, and if so can I add some information? >