From owner-freebsd-net@freebsd.org  Mon Aug 17 12:28:38 2015
Return-Path: <owner-freebsd-net@freebsd.org>
Delivered-To: freebsd-net@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id 621A49B998E;
 Mon, 17 Aug 2015 12:28:38 +0000 (UTC) (envelope-from slw@zxy.spb.ru)
Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 1FA7116AE;
 Mon, 17 Aug 2015 12:28:38 +0000 (UTC) (envelope-from slw@zxy.spb.ru)
Received: from slw by zxy.spb.ru with local (Exim 4.84 (FreeBSD))
 (envelope-from <slw@zxy.spb.ru>)
 id 1ZRJWm-000Kyn-7S; Mon, 17 Aug 2015 15:28:36 +0300
Date: Mon, 17 Aug 2015 15:28:36 +0300
From: Slawa Olhovchenkov <slw@zxy.spb.ru>
To: Daniel Braniss <danny@cs.huji.ac.il>
Cc: FreeBSD stable <freebsd-stable@freebsd.org>,
 FreeBSD Net <freebsd-net@freebsd.org>
Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance
Message-ID: <20150817122836.GC3158@zxy.spb.ru>
References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il>
MIME-Version: 1.0
Content-Type: text/plain; charset=koi8-r
Content-Disposition: inline
In-Reply-To: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il>
User-Agent: Mutt/1.5.23 (2014-03-12)
X-SA-Exim-Connect-IP: <locally generated>
X-SA-Exim-Mail-From: slw@zxy.spb.ru
X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.20
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 17 Aug 2015 12:28:38 -0000

On Mon, Aug 17, 2015 at 10:27:41AM +0300, Daniel Braniss wrote:

> hi,
> 	I have a host (Dell R730) with both cards, connected to an HP8200 switch at 10Gb.
> 	when writing to the same storage (netapp) this is what I get:
> 		ix0:		~130MGB/s
> 		mlxen0	~330MGB/s
> 	this is via nfs/tcpv3
> 
> 	I can get similar (bad) performance with the mellanox if I increase the file size
> 	to 512MGB.
> 	so at face value, it seems the mlxen does a better use of resources than the intel.
> 	Any ideas how to improve ix/intel's performance?

Any way, please show

OS version
/var/run/dmesg.boot
What's tuning perfomed (loader.conf, sysctl.conf)?
top -PHS in both cases
ifconfig -a in both cases
netstat -rn in both cases
I am don't know netapp -- what is hardware configuration (disks and
etc) and software tuning (MTU?).