From owner-freebsd-current@FreeBSD.ORG Thu Aug 16 20:06:46 2007 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1424A16A421 for ; Thu, 16 Aug 2007 20:06:46 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.freebsd.org (Postfix) with ESMTP id C5D6813C467 for ; Thu, 16 Aug 2007 20:06:45 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.14.0/8.14.0) with ESMTP id l7GK6jUE003843 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 16 Aug 2007 16:06:45 -0400 (EDT) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.12.9p2/8.12.9/Submit) id l7GK6H4Y001942; Thu, 16 Aug 2007 16:06:17 -0400 (EDT) (envelope-from gallatin) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <18116.44624.144286.621286@grasshopper.cs.duke.edu> Date: Thu, 16 Aug 2007 16:06:17 -0400 (EDT) To: "Kip Macy" In-Reply-To: References: <18116.43755.107638.103132@grasshopper.cs.duke.edu> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Cc: freebsd-current@freebsd.org Subject: Re: IPSEC disables TSO X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Aug 2007 20:06:46 -0000 Kip Macy writes: > On 8/16/07, Kip Macy wrote: > > On 8/16/07, Andrew Gallatin wrote: > > > > > > When tracking down an mxge 10GbE performance issue, I noticed that > > > simply compiling IPSEC into the kernel totally destroys 10GbE transmit > > > performance because it will silently disable TSO. > > > > > > The problem seems to be at least that the test on line 463 of > > > tcp_output.c (tp->t_inpcb->inp_sp == NULL) will always be > > > false. If I comment that line out, TSO works as normal > > > (though I have no idea what would happen on an IPSEC connection). > > > > > > Is there some way to make this not happen? I believe that the mxge > > > user simply wanted to run some handful of applications through ipsec > > > tunnels (perhaps even on a different NIC entirely). > > > > IPSEC encrypts the TCP header - how is the card going to do TSO? > > > I thought about this after sending and realized that the appropriate > response is that IPSEC needs to be careful to disable TSO when its in > use for a connection. What you're seeing was most likely done as an > expedient. Yes, exactly, there needs to be a smarter test that can distingiush if IPSEC is actually in use on a connection or not; I should have been more clear about this. The problem is that I have zero knowledge about IPSEC, so I have no idea how to do this. I'm worried that people will compile IPSEC into the kernel to run an encrypted tunnel (or the TCP MD5 signature stuff for BGP), and then be rather surprised that their their "normal" TCP performance stinks. Drew