From owner-freebsd-net@FreeBSD.ORG  Wed Feb 15 09:12:42 2006
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
X-Original-To: net@freebsd.org
Delivered-To: freebsd-net@FreeBSD.ORG
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 65F1616A420
	for <net@freebsd.org>; Wed, 15 Feb 2006 09:12:42 +0000 (GMT)
	(envelope-from raglon@packetfront.com)
Received: from mail.packetfront.com (mail.packetfront.com [212.247.6.198])
	by mx1.FreeBSD.org (Postfix) with ESMTP id E7C5D43D45
	for <net@freebsd.org>; Wed, 15 Feb 2006 09:12:41 +0000 (GMT)
	(envelope-from raglon@packetfront.com)
Received: from localhost (localhost [127.0.0.1])
	by mail.packetfront.com (Postfix) with ESMTP id 52B2EA3436;
	Wed, 15 Feb 2006 10:12:42 +0100 (CET)
Received: from mail.packetfront.com ([127.0.0.1])
	by localhost (mail.packetfront.com [127.0.0.1]) (amavisd-new,
	port 10024)
	with ESMTP id 00961-03; Wed, 15 Feb 2006 10:12:42 +0100 (CET)
Received: from [192.168.1.137] (unknown [192.168.1.137])
	by mail.packetfront.com (Postfix) with ESMTP id 0DEC1A3425;
	Wed, 15 Feb 2006 10:12:42 +0100 (CET)
Message-ID: <43F2F03D.90603@packetfront.com>
Date: Wed, 15 Feb 2006 10:11:25 +0100
From: Ragnar Lonn <raglon@packetfront.com>
User-Agent: Mozilla Thunderbird 0.8 (Windows/20040913)
X-Accept-Language: en-us, en
MIME-Version: 1.0
To: Ivan Voras <ivoras@fer.hr>
References: <43F0CE40.5040800@fer.hr> <43F1A2D1.7040402@packetfront.com>
	<43F24E7E.4060503@fer.hr>
In-Reply-To: <43F24E7E.4060503@fer.hr>
Content-Type: text/plain; charset=ISO-8859-2; format=flowed
Content-Transfer-Encoding: 7bit
X-Virus-Scanned: by amavisd-new-20030616-p10 (Debian) at packetfront.com
Cc: net@freebsd.org
Subject: Re: TCP Performance advice needed [long!]
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 15 Feb 2006 09:12:42 -0000

Ivan Voras wrote:

> I understand this bottleneck, and know (at least in theory :) ) how it 
> could be solved, but my problems are not directly related to that:
>
> - For small (but consistent in size) packet sizes, I get randomly 
> varying round-trip times, and much lower packets-per-second ratio then 
> with big packets (consistent in size) with the exact same lock-step 
> protocol. Packet generation and processing are not CPU intensive.
>
> - When using big packets (actually, when switching back and forth from 
> small packets to big packets), the PPS performance starts low and 
> climbs to "normal" levels, and I'd like to avoid this. This is a local 
> network with 0 errors.


My guess would be that it's an effect of how TCP works. The slow start, 
for instance (but
I saw you had tried setting TCP_NODELAY). Maybe you should experiement 
with UDP
as the transport protocol instead. Of course, it requires more work if 
you need
reliable and in-order delivery. But if you want low roundtrip times, use 
UDP. There isn't
an online action game today that uses TCP, because it's not suited for 
applications which
need fast response times.

Cheers,

  /Ragnar