From owner-freebsd-current@FreeBSD.ORG Sat Nov 17 06:12:55 2007 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B836A16A418 for ; Sat, 17 Nov 2007 06:12:55 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: from ro-out-1112.google.com (ro-out-1112.google.com [72.14.202.178]) by mx1.freebsd.org (Postfix) with ESMTP id 5EB6B13C448 for ; Sat, 17 Nov 2007 06:12:54 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: by ro-out-1112.google.com with SMTP id k5so813453rog for ; Fri, 16 Nov 2007 22:12:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=+8L2S/jrMACKSwfruLum5feAZgR9/K8OzEK1bmN57U4=; b=Y5fjClXijXNVooCEOYpzr6BWoEyvJY2Hcpjtilx2F5KASU25CuOpo50zATcyeQFsCvkQ95k8H1494KJTMrb3hSK0pLJZ+XNjQOQZRujHTe0J+PCRZx+HRQ2kt9C+zGGUFq8eArjOlbqUD/JHWTPlNCCpeXN0gRHwlpFKZ6Ky2uw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=acyd1EcZYv03UozQJEzXoBB//HB1T1q4r0motoYeynyxU84UwCWE0u53wWwilxDXvJbObPiSevuIsB0SGh3tiPyzKfHTB3IEB0DoWIhMYUsWcVBSrzzvWPz/DbyEnFS39e7oaMBgtVaCD+5c8tqAKF14oJZAAua8FIY3YsUvZjg= Received: by 10.114.201.1 with SMTP id y1mr158224waf.1195279963914; Fri, 16 Nov 2007 22:12:43 -0800 (PST) Received: by 10.114.13.15 with HTTP; Fri, 16 Nov 2007 22:12:43 -0800 (PST) Message-ID: Date: Fri, 16 Nov 2007 22:12:43 -0800 From: "Kip Macy" To: "Mike Andrews" In-Reply-To: <20071117003504.R31357@mindcrime.int.bit0.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20071117003504.R31357@mindcrime.int.bit0.com> Cc: freebsd-current@freebsd.org Subject: Re: bizarre em + TSO + MSS issue in RELENG_7 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Nov 2007 06:12:55 -0000 On Nov 16, 2007 9:42 PM, Mike Andrews wrote: > Has anyone run into problems with MSS not being respected when using TSO, > specifically on em cards? > > I've run into a weird 3-way interaction with TSO, users with small MTU's > (like 576), and a Cisco firewall that by default drops packets larger than > the MSS for that session. Most users can connect to our web servers > behind this firewall just fine, but a very few couldn't since we moved > from RELENG_6 to RELENG_7 about two weeks ago. One of them could connect > again after removing a Linksys box at his end that was clamping the MSS at > 536. Some of the others were on dialup which tends to use 576 MTU / 536 > MSS. The firewall was dropping a lot of *outbound* traffic to these users > because the MSS was exceeded -- ~1000 byte packets for a 536 MSS > connection. All of those users' problems disappeared after disabling TSO > on our end. Pretty weird. :) > > I'm quite happy running without TSO and I really don't expect this to get > looked at before 7.0-RELEASE -- I just thought the info that MSS appears > (to me anyway) to not always be respected might be useful for those > looking at TSO-related throughput issues. In our case throughput isn't > the issue, it's certain connections stalling out -- it actually behaves > sort of like (but isn't) a PMTUD issue. > > I've got a lot more detail for whoever wants it (Jack Vogel, probably) > including tcpdumps and firewall logs but I thought I'd put it out here > first in case it wasn't em-specific rather than clutter the list right > away, or if it's a known issue that's already fixed in HEAD or something. > This is with em driver version 6.5.3, 7.0-BETA2 from Nov 2. Hrrm. Either TCP isn't setting tso_segsz correctly, or em isn't honoring tso_segsz. Can you add some instrumentation to the TSO code in em to see if tso_segsz is being set to the MSS? -Kip