From owner-freebsd-net@FreeBSD.ORG  Wed May 11 23:12:17 2011
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id F13FD106564A
	for <net@freebsd.org>; Wed, 11 May 2011 23:12:17 +0000 (UTC)
	(envelope-from oberman@es.net)
Received: from mailgw.es.net (mail1.es.net [IPv6:2001:400:201:1::2])
	by mx1.freebsd.org (Postfix) with ESMTP id E056D8FC15
	for <net@freebsd.org>; Wed, 11 May 2011 23:12:17 +0000 (UTC)
Received: from ptavv.es.net (ptavv.es.net [IPv6:2001:400:910::29])
	by mailgw.es.net (8.14.3/8.14.3) with ESMTP id p4BNCHJK024405
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT)
	for <net@freebsd.org>; Wed, 11 May 2011 16:12:17 -0700
Received: from ptavv.es.net (localhost [127.0.0.1])
	by ptavv.es.net (Tachyon Server) with ESMTP id 737751CC2C
	for <net@freebsd.org>; Wed, 11 May 2011 16:12:17 -0700 (PDT)
To: net@freebsd.org
Date: Wed, 11 May 2011 16:12:17 -0700
From: "Kevin Oberman" <oberman@es.net>
Message-Id: <20110511231217.737751CC2C@ptavv.es.net>
Cc: 
Subject: Performance crash when setting TOS bits
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 11 May 2011 23:12:18 -0000

Under FreeBSD 8.2 we have found that something goes haywire when TOS
bits are set. When the TOS is 0 (default), we can get 9 Gbps performance
with one of our test systems running perfSONAR
(http://www.perfsonar.net). When we set the TOS bits to 32, the
performance drops to about 4Mbps. This does not happen with FreeBSD 7.3.

We ran a tcptrace on the data and found very large numbers of
retransmits and duplicate ACKs. I would attach the traces, but I know
the mail list would remove them.

Has anyone else seen this? Any idea what might be happening?
-- 
R. Kevin Oberman, Network Engineer
Energy Sciences Network (ESnet)
Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab)
E-mail: oberman@es.net			Phone: +1 510 486-8634
Key fingerprint:059B 2DDF 031C 9BA3 14A4  EADA 927D EBB3 987B 3751