From owner-freebsd-stable@freebsd.org  Mon Aug 17 09:41:55 2015
Return-Path: <owner-freebsd-stable@freebsd.org>
Delivered-To: freebsd-stable@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id 05B179BBBDF;
 Mon, 17 Aug 2015 09:41:55 +0000 (UTC) (envelope-from slw@zxy.spb.ru)
Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id B7E051C15;
 Mon, 17 Aug 2015 09:41:54 +0000 (UTC) (envelope-from slw@zxy.spb.ru)
Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD))
 (envelope-from <slw@zxy.spb.ru>)
 id 1ZRGvJ-000HwX-Ji; Mon, 17 Aug 2015 12:41:45 +0300
Date: Mon, 17 Aug 2015 12:41:45 +0300
From: Slawa Olhovchenkov <slw@zxy.spb.ru>
To: Daniel Braniss <danny@cs.huji.ac.il>
Cc: FreeBSD stable <freebsd-stable@freebsd.org>,
 FreeBSD Net <freebsd-net@freebsd.org>
Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance
Message-ID: <20150817094145.GB3158@zxy.spb.ru>
References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il>
MIME-Version: 1.0
Content-Type: text/plain; charset=koi8-r
Content-Disposition: inline
In-Reply-To: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il>
User-Agent: Mutt/1.5.23 (2014-03-12)
X-SA-Exim-Connect-IP: <locally generated>
X-SA-Exim-Mail-From: slw@zxy.spb.ru
X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false
X-BeenThere: freebsd-stable@freebsd.org
X-Mailman-Version: 2.1.20
Precedence: list
List-Id: Production branch of FreeBSD source code <freebsd-stable.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-stable>, 
 <mailto:freebsd-stable-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-stable/>
List-Post: <mailto:freebsd-stable@freebsd.org>
List-Help: <mailto:freebsd-stable-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-stable>,
 <mailto:freebsd-stable-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Aug 2015 09:41:55 -0000

On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote:

> hi,
> 	I have a host (Dell R730) with both cards, connected to an HP8200 switch at 10Gb.
> 	when writing to the same storage (netapp) this is what I get:
> 		ix0:		~130MGB/s
> 		mlxen0	~330MGB/s
> 	this is via nfs/tcpv3
> 
> 	I can get similar (bad) performance with the mellanox if I increase the file size
> 	to 512MGB.

Look like mellanox have internal beffer for caching and do ACK acclerating.

> 	so at face value, it seems the mlxen does a better use of resources than the intel.
> 	Any ideas how to improve ix/intel's performance?

Are you sure about netapp performance?