From owner-freebsd-current@FreeBSD.ORG Thu Oct 14 19:57:39 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5703C16A4E0 for ; Thu, 14 Oct 2004 19:57:39 +0000 (GMT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id CD27643D5F for ; Thu, 14 Oct 2004 19:57:38 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.13.1/8.13.1) with ESMTP id i9EJturu024010; Thu, 14 Oct 2004 15:55:56 -0400 (EDT) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)i9EJttFc024007; Thu, 14 Oct 2004 15:55:56 -0400 (EDT) (envelope-from robert@fledge.watson.org) Date: Thu, 14 Oct 2004 15:55:55 -0400 (EDT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Ian FREISLICH In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Andrey Chernov cc: Robert Huff cc: current@FreeBSD.ORG cc: Bruce M Simpson cc: Kris Kennaway Subject: Re: network slowness/freez-up since update 10/11 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Oct 2004 19:57:39 -0000 On Thu, 14 Oct 2004, Ian FREISLICH wrote: > Andrey Chernov wrote: > > > You mean, until rwatson changed the default to debug.mpsafenet=1? :-) > > > > Your guess is precisely right! :-) > > > > (IMHO making such commit without testing major drivers such as if_de was > > wrong step) > > I always thought the spin on debug.mpsafenet=1 with if_de was YYMV. > There were many calls for the maintainers of the driver to fix it, but > zero response IIRC. Maybe making it on by default was a little hasty, > but anyone that follows -CURRENT like they should if they run it weuld > have been aware of this and set debug.mpsafenet=0 in their loader.conf > when they saw that commit. (Kind comments on handling of mpsafenet work ommitted in quote, but much appreciated). I was chatting wit Max Laier this evening, and he suggested that he was worried that the ALTQ changes might actually be the problem. He has created a small patch to back those changes out, as well as a change to tweak the behavior. You can find the patches here: http://people.freebsd.org/~mlaier/if_de.c.backout.diff http://people.freebsd.org/~mlaier/if_de.c.drvlen.diff I looked at the queueing pieces yesterday but didn't see any obvious problems with them. I think it's worth trying each of these patches to see if one of them has the desired effect, however. The problem appears to lie somehow in the hand-off between the network stack and driver, as that's the primary difference between the debug.mpsafenet={0,1} cases. FYI, here are some things we've tried looking at so far: - We thought there might be a race in the handling of IFF_OACTIVE and its use in if_handoff(), since IFF_OACTIVE is used differently in if_de that most drivers. However, removing the IFF_OACTIVE test in iff_handoff() did not resolve the problem in John's configuration. - We were concerned there was a race in the task queue handoff used to schedule the interface start routine asynchronusly from the queue insert. We instrumented the task queue code with timing and didn't find anything abnormal (i.e., no waits long enough to explain the observed delays). So it seems likely to be one of the two following sorts of things: - A problem in the if_de driver, perhaps due to less Giant on the rest of the stack, that causes it to improperly move data in and out of the interface queues, or monitor for entires in the queue, resulting in delays. - A race introduced by Giant removal wherein the if_de driver behaves incorrectly if a packet is found in the ifq by the interrupt handler if the tulip_start function has not yet been run for that packet. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Principal Research Scientist, McAfee Research