From owner-freebsd-net@freebsd.org  Sat Jan 23 06:12:08 2016
Return-Path: <owner-freebsd-net@freebsd.org>
Delivered-To: freebsd-net@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id F40E3A8E450
 for <freebsd-net@mailman.ysv.freebsd.org>;
 Sat, 23 Jan 2016 06:12:07 +0000 (UTC)
 (envelope-from cenzatti@hush.com)
Received: from smtp5.hushmail.com (smtp5a.hushmail.com [65.39.178.235])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client CN "smtp.hushmail.com", Issuer "smtp.hushmail.com" (not verified))
 by mx1.freebsd.org (Postfix) with ESMTPS id DA26B100A
 for <freebsd-net@freebsd.org>; Sat, 23 Jan 2016 06:12:07 +0000 (UTC)
 (envelope-from cenzatti@hush.com)
Received: from smtp5.hushmail.com (smtp5a.hushmail.com [65.39.178.235])
 by smtp5.hushmail.com (Postfix) with SMTP id 4A26D201CB
 for <freebsd-net@freebsd.org>; Sat, 23 Jan 2016 05:34:28 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=hush.com;
 h=date:to:subject:from; s=hush;
 bh=YPxBlJZu+2/3mLZJKHuUnB5l2gQwW5jsMGq2wtGU/6g=;
 b=YvU21+qyRfd27fzpwcF/j6uDh+e9tu8nlHsc7ZToljXVQrXuFGeTfdxjS44GohDORmc7VhaFvRafUsXAbSd8nllDTXJg2BZsVSk+Lqquv5fyW7kEAPEj1KSkCFlLj/uqtwpvXBYflwB4z3l5aExV8YFbQCobWp0n/TIJlO5riCJUQqMDiJ8dYJZ+pMo3jdm2YpXpu7nxg/rBfF2bGA5hwRqHHxDKPDc48tKOo4l0TvN9aliqhlKxXaaYBV3HTiHt2RP/vKNnIaJbMV+vuDT7iy94x5gE53NzBbwFm1OdzqjrvojsK3Fe19QItOc1uuspkQww0juCH9e2IL/vsPJ2Eg==
Received: from smtp.hushmail.com (w5.hushmail.com [65.39.178.80])
 by smtp5.hushmail.com (Postfix) with ESMTP
 for <freebsd-net@freebsd.org>; Sat, 23 Jan 2016 05:34:28 +0000 (UTC)
Received: by smtp.hushmail.com (Postfix, from userid 99)
 id 2091EA0121; Sat, 23 Jan 2016 05:34:28 +0000 (UTC)
MIME-Version: 1.0
Date: Sat, 23 Jan 2016 03:34:27 -0200
To: freebsd-net@freebsd.org
Subject: Chelsio T520-SO-CR low performance (netmap tested) for RX
From: "Marcus Cenzatti" <cenzatti@hush.com>
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="UTF-8"
Message-Id: <20160123053428.2091EA0121@smtp.hushmail.com>
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.20
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 23 Jan 2016 06:12:08 -0000

hello,

I am testing a chelsio t520-so-cr connected to a Intel card with ix(4) driver, I can get the ncxl0 interface to transmit at 14Mpps to another chelsio or to a Intel card. However I can only get 800Kpps-1Mpps for RX tests from both chelsio or Intel.

I have test with both FreeBSD 11 and FreeBSD 10.3-PRERELEASE.

I tested it untuned first and later I have applied tuning recommendations I found on BSDRP[1] website. Results still ranging from 800Kpps to 1Mpps for RX.

Tests are done w/ with pkt-gen in netmap mode on ncxl interface with both IP address and MAC address source/dest.

I have tested ix-ix and I can confirm 14Mpps for both RX and TX directions. I have tested with two different chelsio T520 and both have the very same results.

What particular loader/sysctl or ifconfig options I should investigate?

I also tested disabling txcsum, rxcsum and TSO. Results are different but still on the much lower mentioned 800K-1M pps rate.

thank you

[1]http://bsdrp.net/documentation/examples/forwarding_performance_lab_of_a_hp_proliant_dl360p_gen8_with_10-gigabit_with_10-gigabit_chelsio_t540-cr