From owner-freebsd-stable@freebsd.org  Mon Aug 17 10:51:51 2015
Return-Path: <owner-freebsd-stable@freebsd.org>
Delivered-To: freebsd-stable@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id C21519BBF07;
 Mon, 17 Aug 2015 10:51:51 +0000 (UTC)
 (envelope-from danny@cs.huji.ac.il)
Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.116.210])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 7591613CA;
 Mon, 17 Aug 2015 10:51:50 +0000 (UTC)
 (envelope-from danny@cs.huji.ac.il)
Received: from mbpro-w.cs.huji.ac.il ([132.65.80.91])
 by kabab.cs.huji.ac.il with esmtp
 id 1ZRI13-00005x-IM; Mon, 17 Aug 2015 13:51:45 +0300
Mime-Version: 1.0 (Mac OS X Mail 8.2 \(2102\))
Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance
From: Daniel Braniss <danny@cs.huji.ac.il>
In-Reply-To: <CAB2_NwBOLcL4EVjFN6=BvBC_YN=gmfZMweVbmb5ZPCsK4Hnx1g@mail.gmail.com>
Date: Mon, 17 Aug 2015 13:51:45 +0300
Cc: Slawa Olhovchenkov <slw@zxy.spb.ru>, FreeBSD Net <freebsd-net@freebsd.org>,
 FreeBSD stable <freebsd-stable@freebsd.org>
Message-Id: <17871443-E105-4434-80B1-6939306A865F@cs.huji.ac.il>
References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il>
 <20150817094145.GB3158@zxy.spb.ru>
 <CAB2_NwBOLcL4EVjFN6=BvBC_YN=gmfZMweVbmb5ZPCsK4Hnx1g@mail.gmail.com>
To: Christopher Forgeron <csforgeron@gmail.com>
X-Mailer: Apple Mail (2.2102)
Content-Type: text/plain;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Content-Filtered-By: Mailman/MimeDel 2.1.20
X-BeenThere: freebsd-stable@freebsd.org
X-Mailman-Version: 2.1.20
Precedence: list
List-Id: Production branch of FreeBSD source code <freebsd-stable.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-stable>, 
 <mailto:freebsd-stable-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-stable/>
List-Post: <mailto:freebsd-stable@freebsd.org>
List-Help: <mailto:freebsd-stable-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-stable>,
 <mailto:freebsd-stable-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Aug 2015 10:51:52 -0000


> On Aug 17, 2015, at 1:41 PM, Christopher Forgeron =
<csforgeron@gmail.com> wrote:
>=20
> FYI, I can regularly hit 9.3 Gib/s with my Intel X520-DA2's and =
FreeBSD 10.1. Before 10.1 it was less.
>=20

this is NOT iperf/3 where i do get close to wire speed,
it=E2=80=99s NFS writes, i.e., almost real work :-)

> I used to tweak the card settings, but now it's just stock. You may =
want to check your settings, the Mellanox may just have better defaults =
for your switch.=20
>=20
> On Mon, Aug 17, 2015 at 6:41 AM, Slawa Olhovchenkov <slw@zxy.spb.ru =
<mailto:slw@zxy.spb.ru>> wrote:
> On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote:
>=20
> > hi,
> >       I have a host (Dell R730) with both cards, connected to an =
HP8200 switch at 10Gb.
> >       when writing to the same storage (netapp) this is what I get:
> >               ix0:            ~130MGB/s
> >               mlxen0  ~330MGB/s
> >       this is via nfs/tcpv3
> >
> >       I can get similar (bad) performance with the mellanox if I =
increase the file size
> >       to 512MGB.
>=20
> Look like mellanox have internal beffer for caching and do ACK =
acclerating.
>=20
> >       so at face value, it seems the mlxen does a better use of =
resources than the intel.
> >       Any ideas how to improve ix/intel's performance?
>=20
> Are you sure about netapp performance?
> _______________________________________________
> freebsd-net@freebsd.org <mailto:freebsd-net@freebsd.org> mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-net =
<https://lists.freebsd.org/mailman/listinfo/freebsd-net>
> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org =
<mailto:freebsd-net-unsubscribe@freebsd.org>"
>=20