From owner-freebsd-arch@FreeBSD.ORG Fri Jun 16 09:57:00 2006 Return-Path: X-Original-To: arch@FreeBSD.org Delivered-To: freebsd-arch@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8D87616A47A for ; Fri, 16 Jun 2006 09:57:00 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id CBFBC43D48 for ; Fri, 16 Jun 2006 09:56:59 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id 8878A46BD4; Fri, 16 Jun 2006 05:56:59 -0400 (EDT) Date: Fri, 16 Jun 2006 10:56:59 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Doug White In-Reply-To: <20060615175853.Y52950@carver.gumbysoft.com> Message-ID: <20060616102458.N742@fledge.watson.org> References: <20060611141632.Y26634@fledge.watson.org> <20060615175853.Y52950@carver.gumbysoft.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: arch@FreeBSD.org Subject: Re: MFC of socket/protocol reference improvements X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jun 2006 09:57:00 -0000 On Thu, 15 Jun 2006, Doug White wrote: > On Sun, 11 Jun 2006, Robert Watson wrote: > >> (1) Normally, RELENG_* has significant constraints on changes to the kernel >> APIs used by loadable modules -- especially for device drivers. In the >> past, we've not made a lot of changes to the protocol switch interface, >> and historically it hasn't been a run-time extensible interface. Andre >> has recent made changes to allow IP protocols to be loaded at runtime, >> such as IP divert, and these will be affected, however. Do we consider >> modules programmed against these interfaces to be "breakable" -- i.e., >> the >> require a recompile and or changes in the RELENG_6 branch? > > From a policy standpoint, breaking modules in a -STABLE branch is forbidden > since it causes pain for 3rd party developers. Exceptions can be made for > changes that provide more benefit than damage caused by breaking the ABI. > > Usually the question revolves around, "Does anyone actually distribute > modules that use that interface?" :-) So far I've identified one third party infiniband stack used in a product that implements the protosw API, and would need changes. However, the scope and nature of the product mean that this wouldn't be a significant issue for them. I guess the question is whether and how many other protocol modules exist out there. My leaning is to say very few, but perhaps those people simply don't talk to me/us. It would be good to know if there are any other significant protocol stacks being distributed in binary form. The source changes are relatively minor to update a protocol, but do need to be done. I have some more changes in the works that follow on the heels of this change that require a slightly larger set of protocol changes -- all in the management of socket setup and teardown (attach, abort, close, detach). The larger overhaul portion of my changes is within protocols, and those changes largely rely on the protocol switch changes. These changes do a couple of things: (1) Close a number of known races and eliminate a number of known panics. For example, a panic in tcp_ctloutput() has been repeatedly reported, and is fixed by this because so_pcb can no longer suddenly become NULL during a call to setsockopt(). (2) Reduce lock contention on the tcbinfo lock by avoiding calling it in the socket send path in a significant number of situations -- specifically, normal, off-the-self send and receive on TCP. (3) Lay the groundwork for future changes to break down tcbinfo and otherwise optimize TCP locking. We can do (1) through some workarounds I've been looking at in a few cases, and I believe specifically in the setsockopt() case. (2) and (3) are basically impossible without these changes, so if we want to get them into RELENG_6 as opposed to just RELENG_7, merging this set of changes is necessary. I hope to post an initial draft patch in a few days -- I'm currently setting up a Perforce branch and merging the changes to that branch, but there are about 65 commits involved. Robert N M Watson Computer Laboratory University of Cambridge