From owner-freebsd-net@FreeBSD.ORG Thu Feb 18 19:08:18 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8F8C6106568F for ; Thu, 18 Feb 2010 19:08:18 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 3E7D58FC12 for ; Thu, 18 Feb 2010 19:08:17 +0000 (UTC) Received: by vws14 with SMTP id 14so861589vws.13 for ; Thu, 18 Feb 2010 11:08:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=lS5MQijfVUfBFSPQZXezzG32M7KRauwCbX/JWRGzAMM=; b=etyC51W5AdzGac3KAD3DCp+cv6elihPCVXBQLMkoo20WhvVohHk/LfpXm7A+MRe1N+ zb5trGCEvDhYA27oXCfp4cz4A8rTwTAv8XSOf0felxX8Xe/y0wvRWh83LBw4YGoB2s/O /ICQSNWRAmDR+2vW/BPazMRrenEjdmN0znWqo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=bLFqTaD8fQ6qCTJnqSiAjd2fgUxgQJekNTLCsZMPmW51Vb3KreLuCsU70FD1tLKK61 W/bvuPr65inVKdCY4tyafDZXXpTTQZUdXmhUFZZv1Th/yEsnvqFPtG82sMzMbCZQi/qX XNcdeFXHuATolyw/lREDP4m339VMM/lkqbaH8= Received: by 10.220.126.167 with SMTP id c39mr3352717vcs.131.1266520097294; Thu, 18 Feb 2010 11:08:17 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id 23sm29659360vws.6.2010.02.18.11.08.15 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 18 Feb 2010 11:08:16 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Thu, 18 Feb 2010 11:07:36 -0800 From: Pyun YongHyeon Date: Thu, 18 Feb 2010 11:07:36 -0800 To: Patrick Mahan Message-ID: <20100218190736.GA11675@michelle.cdnetworks.com> References: <4B7D5DA0.1020500@mahan.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B7D5DA0.1020500@mahan.org> User-Agent: Mutt/1.4.2.3i Cc: freebsd-net@freebsd.org Subject: Re: Issues with em(4) device under FreeBSD 8.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Feb 2010 19:08:18 -0000 On Thu, Feb 18, 2010 at 07:32:48AM -0800, Patrick Mahan wrote: > All, > > I have seen a few mentions on the mailing lists in regard to issues > with em(4) and FreeBSD 8.0 with regard to throughput. > > We are also seeing similar issues on HP Proliant systems with > this HP GE interfaces. Previously we were running FreeBSD 6.2 and > iperf was showing ~900 Mbits/sec between two directly connected > systems. After the upgrade, iperf only shows around ~350 Mbits/sec. > > This seems only to be happening on the HP's. When we upgrade another > x86 box (privately built) we are seeing ~900 Mbits/sec even to > one of the HP systems. > > I haven't seen anything yet to account for this behavior. Has anyone > else seen similar issues? > I know there is a possible Tx checksum offloading issue but testing with iperf may not hit the case. One of the big change made in 8.x is switching to buf_ring which will take advantage of multi TX queues. So the only guess I have is TCP segment reordering caused by em(4) and buf_ring interface. Can you capture the traffic on receiver side and check whether you see out-of-ordered TCP segment delivery? If the theory is right you may see the lots of out-of-ordered segments(this could be easily checked on receiver side with netstat -s after clearing the stats) and it will be more frequently seen when TCP window scaling is used. Personally I had trouble to reproduce it on my environments. It may depend on specific workloads. > Thanks, > > Patrick