From owner-freebsd-net@freebsd.org  Mon Aug 17 10:35:14 2015
Return-Path: <owner-freebsd-net@freebsd.org>
Delivered-To: freebsd-net@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B52C9BBA94;
 Mon, 17 Aug 2015 10:35:14 +0000 (UTC)
 (envelope-from danny@cs.huji.ac.il)
Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id C77E11812;
 Mon, 17 Aug 2015 10:35:13 +0000 (UTC)
 (envelope-from danny@cs.huji.ac.il)
Received: from mbpro-w.cs.huji.ac.il ([132.65.80.91])
 by kabab.cs.huji.ac.il with esmtp
 id 1ZRHkw-000Pr5-D5; Mon, 17 Aug 2015 13:35:06 +0300
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2102\))
Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance
From: Daniel Braniss <danny@cs.huji.ac.il>
In-Reply-To: <20150817094145.GB3158@zxy.spb.ru>
Date: Mon, 17 Aug 2015 13:35:06 +0300
Cc: FreeBSD stable <freebsd-stable@freebsd.org>,
 FreeBSD Net <freebsd-net@freebsd.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <197995E2-0C11-43A2-AB30-FBB0FB8CE2C5@cs.huji.ac.il>
References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il>
 <20150817094145.GB3158@zxy.spb.ru>
To: Slawa Olhovchenkov <slw@zxy.spb.ru>
X-Mailer: Apple Mail (2.2102)
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.20
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Aug 2015 10:35:14 -0000


> On Aug 17, 2015, at 12:41 PM, Slawa Olhovchenkov <slw@zxy.spb.ru> =
wrote:
>=20
> On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote:
>=20
>> hi,
>> 	I have a host (Dell R730) with both cards, connected to an =
HP8200 switch at 10Gb.
>> 	when writing to the same storage (netapp) this is what I get:
>> 		ix0:		~130MGB/s
>> 		mlxen0	~330MGB/s
>> 	this is via nfs/tcpv3
>>=20
>> 	I can get similar (bad) performance with the mellanox if I =
increase the file size
>> 	to 512MGB.
>=20
> Look like mellanox have internal beffer for caching and do ACK =
acclerating.
what ever they are doing, it=E2=80=99s impressive :-)

>=20
>> 	so at face value, it seems the mlxen does a better use of =
resources than the intel.
>> 	Any ideas how to improve ix/intel's performance?
>=20
> Are you sure about netapp performance?

yes, and why should it act differently if the request is coming from the =
same host? in any case
the numbers are quiet consistent since I have measured it from several =
hosts, and at different times.

danny