From owner-freebsd-net@FreeBSD.ORG  Sat Jan  4 22:36:48 2014
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id 2F005DBE
 for <freebsd-net@freebsd.org>; Sat,  4 Jan 2014 22:36:48 +0000 (UTC)
Received: from hergotha.csail.mit.edu
 (wollman-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:ccb::2])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id DDA5A1738
 for <freebsd-net@freebsd.org>; Sat,  4 Jan 2014 22:36:47 +0000 (UTC)
Received: from hergotha.csail.mit.edu (localhost [127.0.0.1])
 by hergotha.csail.mit.edu (8.14.7/8.14.7) with ESMTP id s04Majdj026167;
 Sat, 4 Jan 2014 17:36:45 -0500 (EST)
 (envelope-from wollman@hergotha.csail.mit.edu)
Received: (from wollman@localhost)
 by hergotha.csail.mit.edu (8.14.7/8.14.4/Submit) id s04MaiBn026166;
 Sat, 4 Jan 2014 17:36:44 -0500 (EST) (envelope-from wollman)
Date: Sat, 4 Jan 2014 17:36:44 -0500 (EST)
From: Garrett Wollman <wollman@hergotha.csail.mit.edu>
Message-Id: <201401042236.s04MaiBn026166@hergotha.csail.mit.edu>
To: peter@wemm.org
Subject: Re: Long-haul problems - connections stuck in slow start
X-Newsgroups: mit.lcs.mail.freebsd-net
In-Reply-To: <CAGE5yCpK7N1BwzGCXmx=8M4Q4cu9iUKo4Za-92gRwnkR=HvAiA@mail.gmail.com>
References: <52C85537.7080307@wemm.org>	<52C85EED.801@wemm.org>
Organization: none
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3
 (hergotha.csail.mit.edu [127.0.0.1]); Sat, 04 Jan 2014 17:36:45 -0500 (EST)
X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED
 autolearn=disabled version=3.3.2
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on
 hergotha.csail.mit.edu
Cc: freebsd-net@freebsd.org
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 04 Jan 2014 22:36:48 -0000

In article
<CAGE5yCpK7N1BwzGCXmx=8M4Q4cu9iUKo4Za-92gRwnkR=HvAiA@mail.gmail.com>,
Peter Wemm writes:

>I'm looking at the duplicate acks in this trace from each end..  any
>clues where they're coming from?

My original suggestion was going to be that they're probably a result
of your packets getting reordered because of that path instability you
pointed out.  10 ms differential delay is much higher than the
interarrival time should be, so you likely have a burst of packets
headed down the slow path, followed by a burst of packets down the
fast path, and the later packets arrive at the destination first,
causing a burst of dupacks until the slow packets catch up.

But that doesn't seem to be born out by your traces.  Do you have TCP
RX offload on this hardware?  Could it be sending the dupacks without
looping them back to BPF?  (I haven't looked at our TCP offload code
so I have no idea if that's even possible.)

-GAWollman