From owner-freebsd-net@freebsd.org Thu Jul 21 20:55:55 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 14D3FBA03E6 for ; Thu, 21 Jul 2016 20:55:55 +0000 (UTC) (envelope-from chris@dunbar.net) Received: from 002.las.mailroute.net (002.las.mailroute.net [199.89.4.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.mailroute.net", Issuer "AlphaSSL CA - SHA256 - G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E5B39198A for ; Thu, 21 Jul 2016 20:55:54 +0000 (UTC) (envelope-from chris@dunbar.net) Received: from localhost (localhost.localdomain [127.0.0.1]) by 002.las.mailroute.net (Postfix) with ESMTP id 3rwQx52q5kz1y8m for ; Thu, 21 Jul 2016 20:53:45 +0000 (UTC) X-Virus-Scanned: by MailRoute Received: from 002.las.mailroute.net ([199.89.4.5]) by localhost (002.las.mailroute.net. [127.0.0.1]) (mroute_mailscanner, port 10026) with LMTP id CKPEg6XBX1hp for ; Thu, 21 Jul 2016 20:53:43 +0000 (UTC) Received: from zimbra.earthside.net (unknown [50.251.189.243]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by 002.las.mailroute.net (Postfix) with ESMTPS id 3rwQx26XZSz1y88 for ; Thu, 21 Jul 2016 20:53:42 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by zimbra.earthside.net (Postfix) with ESMTP id B401D50267B for ; Thu, 21 Jul 2016 16:53:41 -0400 (EDT) Received: from zimbra.earthside.net ([127.0.0.1]) by localhost (zimbra.earthside.net [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id fZvdDxXumRwq for ; Thu, 21 Jul 2016 16:53:40 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by zimbra.earthside.net (Postfix) with ESMTP id 7721750267D for ; Thu, 21 Jul 2016 16:53:40 -0400 (EDT) Received: from zimbra.earthside.net ([127.0.0.1]) by localhost (zimbra.earthside.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id FuTTrZZmA0JE for ; Thu, 21 Jul 2016 16:53:40 -0400 (EDT) Received: from zimbra.earthside.net (zimbra.earthside.net [10.11.12.148]) by zimbra.earthside.net (Postfix) with ESMTP id 531BB50267B for ; Thu, 21 Jul 2016 16:53:40 -0400 (EDT) Date: Thu, 21 Jul 2016 16:53:40 -0400 (EDT) From: Chris Dunbar To: freebsd-net Message-ID: <1441424852.712842.1469134420198.JavaMail.zimbra@dunbar.net> In-Reply-To: References: <1244557023.708807.1469061382192.JavaMail.zimbra@dunbar.net> Subject: Re: Slow performance with Intel X540-T2 10Gb NIC MIME-Version: 1.0 X-Originating-IP: [10.11.12.148] X-Mailer: Zimbra 8.6.0_GA_1153 (ZimbraWebClient - GC51 (Mac)/8.6.0_GA_1153) Thread-Topic: Slow performance with Intel X540-T2 10Gb NIC Thread-Index: bkXeUbMihgtikNh5aEfKledchPk+cA== Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Jul 2016 20:55:55 -0000 Eric, et al: I haven't tried netperf yet, but I do have some new information to share. I have two systems that I am using for testing: the new server and an older (not too old) desktop PC. I installed CentOS on the new server again because I know it can achieve >9 GB/s with the X540. I replaced Windows on the desktop PC with FreeBSD 10.3 (it also has an X540) and ran iperf3 again. I was able to achieve >9 GB/s so I know the problem isn't the X540 and I know the problem isn't anything with the default installation of FreeBSD 10.3. So, what in the world might be nutty in my BIOS settings (or elsewhere) that would cause the new server + FreeBSD 10.3 + X540 to equal slow performance? Regards, Chris From: "Eric Joyner" To: "chris" , "freebsd-net" Sent: Thursday, July 21, 2016 1:27:10 PM Subject: Re: Slow performance with Intel X540-T2 10Gb NIC (Replying-all this time) Did you try to set these settings that ESnet recommends? https://fasterdata.es.net/host-tuning/freebsd/ We don't use iperf3 here at Intel (we use netperf instead), so I'm not sure I can be much help diagnosing what's wrong. On Wed, Jul 20, 2016 at 5:39 PM Chris Dunbar < chris@dunbar.net > wrote: Hello, I am new to FreeBSD and recently built a file server out of new components running FreeBSD 10.3. I installed an Intel X540-T2 10 Gb NIC and am experiencing what I consider to be slow transfer speeds. I am using iperf3 to measure the speed and test the results of modifications. So far nothing I have done has made a noticeable difference. If I run iperf3 -s on the FreeBSD server, I see transfer speeds of approximately 1.6 Gb/s. If I run iperf3 in client mode, the speed improves to ~2.75 Gb/s. However, if I replace FreeBSD with CentOS 7 on the same hardware, I see iperf3 speeds surpassing 8 GB/s. The other end of my iperf3 test is a Windows 10 box that also has an Intel X540-T2 installed. I did notice that FreeBSD 10.3 (and 11.0 alpha 6 for that matter) includes a slightly older Intel driver (v3.1.13-k). I managed to build a custom kernel that removed the Intel PRO/10GbE PCIE NIC drivers. That allowed me to manually load the latest 3.1.14 driver downloaded from Intel's web site. Unfortunately that did not produce any improvements. I am working my way through man tuning() and some other articles on network performance. So far nothing I tweak makes a noticeable difference. I'm increasingly skeptical that I am going to find a setting or two that more than doubles the speed I am currently experiencing. I am open to any and all suggestions at this point. Thank you! Chris _______________________________________________ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to " freebsd-net-unsubscribe@freebsd.org "