From owner-freebsd-net@FreeBSD.ORG  Mon Mar  3 15:05:04 2014
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@FreeBSD.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id 97CDFFD7
 for <freebsd-net@FreeBSD.org>; Mon,  3 Mar 2014 15:05:04 +0000 (UTC)
Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140])
 by mx1.freebsd.org (Postfix) with ESMTP id 3D748214
 for <freebsd-net@FreeBSD.org>; Mon,  3 Mar 2014 15:05:02 +0000 (UTC)
Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua
 [212.40.38.100])
 by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA10890
 for <freebsd-net@FreeBSD.org>; Mon, 03 Mar 2014 17:05:00 +0200 (EET)
 (envelope-from avg@FreeBSD.org)
Received: from localhost ([127.0.0.1])
 by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD))
 id 1WKUQN-000O3p-1v
 for freebsd-net@FreeBSD.org; Mon, 03 Mar 2014 17:04:59 +0200
Message-ID: <531499E2.2080109@FreeBSD.org>
Date: Mon, 03 Mar 2014 17:04:02 +0200
From: Andriy Gapon <avg@FreeBSD.org>
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64;
 rv:24.0) Gecko/20100101 Thunderbird/24.2.0
MIME-Version: 1.0
To: freebsd-net <freebsd-net@FreeBSD.org>
Subject: delayed_ack and Nagle's algorithm interaction
X-Enigmail-Version: 1.6
Content-Type: text/plain; charset=X-VIET-VPS
Content-Transfer-Encoding: 7bit
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.17
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 03 Mar 2014 15:05:04 -0000


Guys,

my knowledge of TCP and our TCP/IP stack is quite basic.
It seems that we are running into a problem described here or something with
very similar symptoms:
http://www.stuartcheshire.org/papers/NagleDelayedAck/
At least, if I either disable delayed_ack or use TCP_NODELAY option, then the
problem goes away.  Otherwise, we see quite significant extra latencies.
Maybe an important detail: the problem is observed for communication between a
local host client and a server in a jail.  So, all traffic is local (via
loopback interface).

I have a pcap file that captures the problem.
Ideas, suggestion, etc are welcome.
Thanks!
-- 
Andriy Gapon