From owner-freebsd-net@FreeBSD.ORG Mon Oct 21 01:44:08 2013 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 0C991E2D for ; Mon, 21 Oct 2013 01:44:08 +0000 (UTC) (envelope-from bounces+73574-866e-freebsd-net=freebsd.org@sendgrid.me) Received: from o3.shared.sendgrid.net (o3.shared.sendgrid.net [208.117.48.85]) by mx1.freebsd.org (Postfix) with SMTP id BEF342022 for ; Mon, 21 Oct 2013 01:44:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sendgrid.info; h=from:mime-version:to:subject:references:in-reply-to:content-type:content-transfer-encoding; s=smtpapi; bh=m45Vb5BjwCn9XcF5dnBPlyVZTBM=; b=PeSWdjAcWJzgBmtQST iWLEfm56CS5miH7VV1i+GJzGz8Zu8eOBtRj0EtFseBmt4yaZzB+bohvFodd+q3+l Q/d8ISCAFcBJdflppBKpIc10YZtr2JUu7tt8kdoTpnlvG9keYYSatPTEsoIdOdcp 9X4B2+s+gbYUIUoALJkltG8gw= Received: by mf21 with SMTP id mf21.9550.526486E67 Mon, 21 Oct 2013 01:44:06 +0000 (GMT) Received: from mail.tarsnap.com (unknown [10.60.208.13]) by mi17 (SG) with ESMTP id 141d8aef453.185.2457f70 for ; Mon, 21 Oct 2013 01:44:06 +0000 (UTC) Received: (qmail 85133 invoked from network); 21 Oct 2013 01:44:05 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by ec2-107-20-205-189.compute-1.amazonaws.com with ESMTP; 21 Oct 2013 01:44:05 -0000 Received: (qmail 75352 invoked from network); 21 Oct 2013 01:42:54 -0000 Received: from unknown (HELO clamshell.daemonology.net) (127.0.0.1) by clamshell.daemonology.net with SMTP; 21 Oct 2013 01:42:54 -0000 Message-ID: <5264869E.4000308@freebsd.org> Date: Sun, 20 Oct 2013 18:42:54 -0700 From: Colin Percival User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:24.0) Gecko/20100101 Thunderbird/24.0 MIME-Version: 1.0 To: Julian Elischer , freebsd-net@freebsd.org Subject: Re: LRO causing stretch ACK violations interacts badly with delayed ACKing References: <52605EC9.6090406@freebsd.org> <526478D0.1000601@freebsd.org> In-Reply-To: <526478D0.1000601@freebsd.org> X-Enigmail-Version: 1.5.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-SG-EID: ISGKkmHlRE12gnWy0TjFyGQSFR1WnpjQdICm3qu6YxH0p0B8WINVqHgZLJ6EVWriWAfOAcdgKav5Hsrd2jvz7v3UcsmVho1qt3IYgqW2c47ipqOfZ/gaNgswyDsrqUHxrNWLt5DMFEOfBjgkp8Nvogg6Ymxs3bWxiL9WS33rG1M= X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Oct 2013 01:44:08 -0000 On 10/20/13 17:44, Julian Elischer wrote: > On 10/18/13 6:03 AM, Colin Percival wrote: >> I know {TSO, LRO, ACKing policy} has been discussed here recently, and I don't >> want to rehash everything, but I'm seeing some very bad misbehaviour with LRO >> and delayed ACKing turned on. >> >> Running 'fetch -o /dev/null https://www.amazon.com/' on an EC2 instance running > [...] > is this just for -current? Good question. Turns out that it isn't -- on 9.2 I see a 95.5 ms delayed ACK: > 00:00:00.000000 IP 10.142.129.245.59172 > 176.32.98.166.443: Flags [S], seq 3310207763, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 292712 ecr 0], length 0 > 00:00:00.001031 IP 176.32.98.166.443 > 10.142.129.245.59172: Flags [S.], seq 3504196464, ack 3310207764, win 8190, options [mss 1460,nop,wscale 6], length 0 > 00:00:00.001139 IP 10.142.129.245.59172 > 176.32.98.166.443: Flags [.], ack 1, win 1026, length 0 > 00:00:00.002269 IP 176.32.98.166.443 > 10.142.129.245.59172: Flags [.], ack 1, win 127, length 0 > 00:00:00.002938 IP 10.142.129.245.59172 > 176.32.98.166.443: Flags [P.], seq 1:140, ack 1, win 1026, length 139 > 00:00:00.003815 IP 176.32.98.166.443 > 10.142.129.245.59172: Flags [.], seq 1:4097, ack 140, win 108, length 4096 > 00:00:00.099328 IP 10.142.129.245.59172 > 176.32.98.166.443: Flags [.], ack 4097, win 1026, length 0 but not on 9.1... although that might just be that LRO isn't happening: > 00:00:00.000000 IP 10.148.177.92.48728 > 176.32.98.166.443: Flags [S], seq 2729946716, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 64564 ecr 0], length 0 > 00:00:00.000722 IP 176.32.98.166.443 > 10.148.177.92.48728: Flags [S.], seq 595247561, ack 2729946717, win 8190, options [mss 1460,nop,wscale 6], length 0 > 00:00:00.000820 IP 10.148.177.92.48728 > 176.32.98.166.443: Flags [.], ack 1, win 1026, length 0 > 00:00:00.001998 IP 176.32.98.166.443 > 10.148.177.92.48728: Flags [.], ack 1, win 127, length 0 > 00:00:00.002716 IP 10.148.177.92.48728 > 176.32.98.166.443: Flags [P.], seq 1:140, ack 1, win 1026, length 139 > 00:00:00.003527 IP 176.32.98.166.443 > 10.148.177.92.48728: Flags [.], ack 140, win 108, length 0 > 00:00:00.003834 IP 176.32.98.166.443 > 10.148.177.92.48728: Flags [.], seq 1:1461, ack 140, win 108, length 1460 > 00:00:00.003850 IP 176.32.98.166.443 > 10.148.177.92.48728: Flags [.], seq 1461:2921, ack 140, win 108, length 1460 > 00:00:00.003870 IP 10.148.177.92.48728 > 176.32.98.166.443: Flags [.], ack 2921, win 981, length 0 > 00:00:00.003888 IP 176.32.98.166.443 > 10.148.177.92.48728: Flags [P.], seq 2921:4097, ack 140, win 108, length 1176 > 00:00:00.003973 IP 10.148.177.92.48728 > 176.32.98.166.443: Flags [.], ack 4097, win 1026, length 0 I can't find any changes in netfront.c or tcp_lro.c to explain why 9.1 and 9.2 are behaving differently -- anyone have any ideas? -- Colin Percival Security Officer Emeritus, FreeBSD | The power to serve Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid