From owner-freebsd-arch@FreeBSD.ORG Thu Jan 10 23:24:55 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9F19D16A469 for ; Thu, 10 Jan 2008 23:24:55 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.177]) by mx1.freebsd.org (Postfix) with ESMTP id 85ACE13C4E1 for ; Thu, 10 Jan 2008 23:24:55 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: by wa-out-1112.google.com with SMTP id k17so1474325waf.3 for ; Thu, 10 Jan 2008 15:24:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=HWsKAumDoNRzjvygoNoi2AGET8UDaW8Ctaof99Ixyac=; b=XLwM5CYP2Lv2Kr8jD5tIJ3tkqc24HYA+jqUSWKuCiVHAPOsafWBg6FOtwgFN/bL/+hZTcHSBV1kZML9VRjPcVHgpHcpVXM/6Ut6ibYMLmGWPjEDVgeS/urkV7i4yUkVwZE0sCc+4CXoObW9kAJ/TlvB+qLISUsl6v52B/B1DuA0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=P1EH2uYnyGfoHwtptPKf5cDi+/aPieBkCPhHbh4VVLYjwpTNWrcPaeIiPmQw4sW7/qFknw8H5P5CDk34B0aQwEThxqTz0/StsqQi4oiiIJ1Js3hvRwDrc12hHtL221FPYJAoog5ZuYiSz8jeybe1UwBYKOijfkpJXKCYZ/94rTA= Received: by 10.114.61.1 with SMTP id j1mr2882494waa.62.1200005792257; Thu, 10 Jan 2008 14:56:32 -0800 (PST) Received: by 10.114.255.11 with HTTP; Thu, 10 Jan 2008 14:56:32 -0800 (PST) Message-ID: Date: Thu, 10 Jan 2008 14:56:32 -0800 From: "Kip Macy" To: gnn@freebsd.org In-Reply-To: <7ive61uwfm.wl%gnn@neville-neil.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20080106124517.G105@fledge.watson.org> <7ive61uwfm.wl%gnn@neville-neil.com> Cc: arch@freebsd.org, Robert Watson , kmacy@freebsd.org, net@freebsd.org Subject: Re: Network device driver KPI/ABI and TOE X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jan 2008 23:24:55 -0000 On Jan 10, 2008 2:50 PM, wrote: > At Sun, 6 Jan 2008 13:47:24 +0000 (GMT), > rwatson wrote: > > > > > > There's also the opportunity to think about whether it's possible to > > harden things in such a ways as to not give up our flexibility to > > keep maintaining and improving TCP (and other related subsystems), > > yet improving the quality of life for a third party TOE driver > > maintainer. For example, might we provide accessor routines for > > certain data structures, or attempt to structure things to hide more > > of TCP locking from a TOE implementation? Should we suggest that > > non-native TOE implementations rely less on our TCP code and provide > > there own where the hardware doesn't provide a complete > > implementation, in order to avoid building dependency on things that > > we know will change? > > > > Given the intimacy that I just perused in the code, basically the > driver knows a lot about internal TCP data structures, I think we need > to think about a kernel KPI just for these things. I'm not very happy > that there are things like cxgb_tcp_ctlinput() although I do know that > cleaning that kind of thing up and making a better KPI will be hard. Although you are correct in the need for a more thought out KPI, that is actually not a good example. Although the way it is currently implemented is not multi-TOE friendly tcp_ctlinput is the correct way to extend socket options. A better example is the way it needs to know the specifics of not only the tcpcb, but the inpcb, and parts of the socket as well. By extension it needs to understand the subtleties of inpcb and pcbinfo locking. This is, needless to say, quite fragile. -Kip