From owner-freebsd-net@freebsd.org  Fri Aug  5 08:25:48 2016
Return-Path: <owner-freebsd-net@freebsd.org>
Delivered-To: freebsd-net@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id 318FFBAE062
 for <freebsd-net@mailman.ysv.freebsd.org>;
 Fri,  5 Aug 2016 08:25:48 +0000 (UTC) (envelope-from hps@selasky.org)
Received: from mail.turbocat.net (mail.turbocat.net
 [IPv6:2a01:4f8:d16:4514::2])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id ED7AC18ED
 for <freebsd-net@freebsd.org>; Fri,  5 Aug 2016 08:25:47 +0000 (UTC)
 (envelope-from hps@selasky.org)
Received: from laptop015.home.selasky.org (unknown [62.141.129.119])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (No client certificate requested)
 by mail.turbocat.net (Postfix) with ESMTPSA id EBEFB1FE022;
 Fri,  5 Aug 2016 10:25:45 +0200 (CEST)
Subject: Re: Unstable local network throughput
To: Ben RUBSON <ben.rubson@gmail.com>, freebsd-net <freebsd-net@freebsd.org>
References: <3C0D892F-2BE8-4650-B9FC-93C8EE0443E1@gmail.com>
 <bed13ae3-0b8f-b1af-7418-7bf1b9fc74bc@selasky.org>
 <3B164B7B-CBFB-4518-B57D-A96EABB71647@gmail.com>
 <5D6DF8EA-D9AA-4617-8561-2D7E22A738C3@gmail.com>
 <06E414D5-9CDA-46D1-A26F-0B07E76FDB34@gmail.com>
 <0b14bf39-ed71-b9fb-1998-bd9676466df6@selasky.org>
 <E5BE8DAC-AB6A-491E-A901-4E513367278B@gmail.com>
 <CAFMmRNz8WryZVVR-_OvB7Ad3tR1NqPpXpv_QEPkoffxdFzdUQw@mail.gmail.com>
 <A5742F79-C2C2-4040-A369-D8CFE6B48D33@gmail.com>
From: Hans Petter Selasky <hps@selasky.org>
Message-ID: <bc872304-aac7-2b21-2f83-7aea3cc82386@selasky.org>
Date: Fri, 5 Aug 2016 10:30:04 +0200
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101
 Thunderbird/45.0
MIME-Version: 1.0
In-Reply-To: <A5742F79-C2C2-4040-A369-D8CFE6B48D33@gmail.com>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 7bit
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net/>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-net>,
 <mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 05 Aug 2016 08:25:48 -0000

On 08/04/16 23:49, Ben RUBSON wrote:
>>
>> On 04 Aug 2016, at 20:15, Ryan Stone <rysto32@gmail.com> wrote:
>>
>> On Thu, Aug 4, 2016 at 11:33 AM, Ben RUBSON <ben.rubson@gmail.com> wrote:
>> But even without RSS, I should be able to go up to 2x40Gbps, don't you think so ?
>> Nobody already did this ?
>>
>> Try this patch
>> (...)
>
> I also just tested the NODEBUG kernel but it did not help.

Hi,

When running these tests, do you see any CPUs fully utilized?

Did you check the RX/TX pauseframes settings and the mlx4 sysctl 
statistics counters, if there is packet loss?

--HPS