From owner-freebsd-net@FreeBSD.ORG Tue Jul 12 15:30:19 2005 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8609E16A41C for ; Tue, 12 Jul 2005 15:30:19 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [204.156.12.53]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3778043D45 for ; Tue, 12 Jul 2005 15:30:19 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by cyrus.watson.org (Postfix) with ESMTP id B9FFE46B9B; Tue, 12 Jul 2005 11:30:18 -0400 (EDT) Date: Tue, 12 Jul 2005 16:30:18 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Ed Maste In-Reply-To: <20050712150224.GA38249@sandvine.com> Message-ID: <20050712162332.Q79478@fledge.watson.org> References: <42CEF0EB.4000107@borderware.com> <42D006DB.8080108@errno.com> <20050712150224.GA38249@sandvine.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Sam Leffler , ming fu , freebsd-net@freebsd.org Subject: Re: what to replace splnet in FreeBSD 5.x? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Jul 2005 15:30:19 -0000 On Tue, 12 Jul 2005, Ed Maste wrote: > On Sat, Jul 09, 2005 at 10:18:19AM -0700, Sam Leffler wrote: >> spl's lock execution threads. 5.x and later systems mostly lock data >> structures using mtx's (there are a very few exceptions). Thus there >> isn't necessarily a direct replacement, you usually need to rethink your >> locking/synchronization strategy. > > This brings up the issue of the remaining splnet()s in 5.x and -CURRENT. > Grepping for "= splnet" in net/ and netinet/ shows more than 50 now > no-op splnet()s left in the stack. > > We've run into corruption in the multicast address lists (in_multihead) > on 5.x, and it turns out in_addmulti still has splnet() "protecting" the > list. > > I'm not sure how many of the splnet()s are actually false positives > (i.e. no longer relevant, locked in another way, etc.) but they're > probably all good indicators of places that locking still needs to be > revisited. In many cases, the splnet's have been left in as references indicators of earlier synchronization requirements and strategies. In some places, they are signs of code still running with Giant over it (i.e., KAME IPSEC, I4B). There are a number of areas of weakness in the current locking work, and this includes: - Several areas of the network stack that still require Giant to operate correctly. Examples are KAME IPSEC (not FAST_IPSEC), some interactions between the tty and network code, such as SLIP, and portions of the ATM stack, and some of the edge case hardware drivers (i.e., older ISA ethernet cards). When these components are present, some or all of the network stack will run with Giant over it. - Several areas where inadequate synchronization is present. Typically they are associated with hard to exploit races, such as unicast address configuration, and therefore generally don't result in instability (and in most cases, we've actually done significant stability testing to make sure they don't). Almost always, these races are around administratively modified data structures. - Several cases where undesirable synchronization is present. I.e., more overhead than we'd like, don't match well with the data structures and data management strategies, or don't interact well with the layering in the network stack. There is active work in all of these areas to remedy the problems. Some are substantially better off in 6.x than 5.x; others will require additional work. I'm concerned about the multicast address list problems you've been experiencing, but haven't yet had a chance to investigate. If you could provide a code fragment that exercises this problem, that would probably get me started a lot more quickly. Thankms, Robert N M Watson