From owner-freebsd-net@freebsd.org Fri Jul 22 12:34:32 2016 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B7958BA1C81 for ; Fri, 22 Jul 2016 12:34:32 +0000 (UTC) (envelope-from sodynet1@gmail.com) Received: from mail-lf0-x22b.google.com (mail-lf0-x22b.google.com [IPv6:2a00:1450:4010:c07::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2402816EE for ; Fri, 22 Jul 2016 12:34:32 +0000 (UTC) (envelope-from sodynet1@gmail.com) Received: by mail-lf0-x22b.google.com with SMTP id b199so84700238lfe.0 for ; Fri, 22 Jul 2016 05:34:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc; bh=x62ruNJw2Jtxrghu6NhWGEf/0ObSOCuiZZa9LFfITcI=; b=tiaU3N9KhIxCdwEI4BGrxeDvL5OzthIyk3Q5WRUhBqu4NZVoEkDXtQEvM8RBKVIITI 1ZJDKHAzSaj2WfBQDIaGGt9I9oAHhgOtCBj0nbebJYp0W8BRZj4NXl6FWZxKDoMm1/Qs ekjwg5fRUVz/GhI8nkbkFZQViXEOQIiLZrBFlVeDQ8VxX9T1Msz181dnEqCasTlscLrK spRGao/ntJ9Fmngds6Z7uE7mLMPFOHklyxWv26gCxrSfbUMDsJp4daUXtv9qa0J04e44 M9yT/i17pwMV4H4JPyWkVJ/AKRM9iZTaB/EFt/HAQPuMBT9yzONFqMSD7AZlt9Z7Ei+t PaVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc; bh=x62ruNJw2Jtxrghu6NhWGEf/0ObSOCuiZZa9LFfITcI=; b=N4Pc71IlL/Sq36OPKliZgo6lTZdctPQU4jP9u0uplwwzkjh3kd9eogaCBJtvlIJySi /5WmL6oWnGBRvYeksXVnUdmmj3/Rj66xcpZV+iH20WSfH0/l5EGQZVE9dVI/DHnWbVm9 u76K0q+Z7gY55nAdEK7hh5JrMPjN69FXgwrN512rsVdJTTRShYh7KPLJfNQY2+gyPIjZ NPByLCfWvaAmeyoKoRMTgc1JdeerPygWu2v8A/v8itNEIhl3jwzeMB3gnks2V491DK2R sUH6u42Ln8ip9h/b4FypqvXEHeA6U1jPFsTh1A43QaZdMnR0wlco7GKCy0tekEsAVb+w Q73w== X-Gm-Message-State: AEkoouv2fkSYRGw7YOY5izkY1OUJa/d7IaZQlVG0Z+5v3DvybClV6Mf8uVvmtTOA7JLsvlhRF8CA3m7tSSyJPg== MIME-Version: 1.0 X-Received: by 10.46.1.26 with SMTP id 26mr2540436ljb.26.1469190870170; Fri, 22 Jul 2016 05:34:30 -0700 (PDT) Received: by 10.25.196.17 with HTTP; Fri, 22 Jul 2016 05:34:30 -0700 (PDT) Received: by 10.25.196.17 with HTTP; Fri, 22 Jul 2016 05:34:30 -0700 (PDT) In-Reply-To: <183608784.713013.1469136611853.JavaMail.zimbra@dunbar.net> References: <1244557023.708807.1469061382192.JavaMail.zimbra@dunbar.net> <1441424852.712842.1469134420198.JavaMail.zimbra@dunbar.net> <183608784.713013.1469136611853.JavaMail.zimbra@dunbar.net> Date: Fri, 22 Jul 2016 15:34:30 +0300 Message-ID: Subject: Re: Slow performance with Intel X540-T2 10Gb NIC From: Sami Halabi To: Chris Dunbar Cc: freebsd-net Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Jul 2016 12:34:32 -0000 hi, would you share what was wrong in the windows side and how you solved it? Sami =D7=91=D7=AA=D7=90=D7=A8=D7=99=D7=9A 22 =D7=91=D7=99=D7=95=D7=9C=D7=99 2016= 12:33 AM,=E2=80=8F "Chris Dunbar" =D7=9B=D7=AA=D7=91: > Hello again, > > I have good news and bad news: > > The bad news first: I am an idiot and I have wasted some of your time for > which I apologize. > > The good news: Testing now between two FreeBSD 10.3 systems, I am > achieving blistering speeds with iperf3. I apparently fell into the trap = of > assuming the new thing (FreeBSD is new to me) was broken. Now I see that = I > was assuming Windows was working fine and focusing all my attention on > FreeBSD. Looking back over everything I have done to troubleshoot this > situation I must conclude that the performance issue was on the Windows > side and not the FreeBSD side. I am less concerned about that because my > ultimate goal is to install my three X540s into one FreeBSD server and tw= o > VMware ESXi hosts. I am now fairly confident performance will be great. > > Many thanks for your collective attention and the suggestions I received > from Eric and others. > > Regards, > Chris > > ----- Original Message ----- > From: "chris" > To: "freebsd-net" > Sent: Thursday, July 21, 2016 4:53:40 PM > Subject: Re: Slow performance with Intel X540-T2 10Gb NIC > > Eric, et al: > > I haven't tried netperf yet, but I do have some new information to share. > I have two systems that I am using for testing: the new server and an old= er > (not too old) desktop PC. I installed CentOS on the new server again > because I know it can achieve >9 GB/s with the X540. I replaced Windows o= n > the desktop PC with FreeBSD 10.3 (it also has an X540) and ran iperf3 > again. I was able to achieve >9 GB/s so I know the problem isn't the X540 > and I know the problem isn't anything with the default installation of > FreeBSD 10.3. So, what in the world might be nutty in my BIOS settings (o= r > elsewhere) that would cause the new server + FreeBSD 10.3 + X540 to equal > slow performance? > > Regards, > Chris > > > From: "Eric Joyner" > To: "chris" , "freebsd-net" > Sent: Thursday, July 21, 2016 1:27:10 PM > Subject: Re: Slow performance with Intel X540-T2 10Gb NIC > > (Replying-all this time) > > Did you try to set these settings that ESnet recommends? > https://fasterdata.es.net/host-tuning/freebsd/ > > We don't use iperf3 here at Intel (we use netperf instead), so I'm not > sure I can be much help diagnosing what's wrong. > > On Wed, Jul 20, 2016 at 5:39 PM Chris Dunbar < chris@dunbar.net > wrote: > > > Hello, > > I am new to FreeBSD and recently built a file server out of new component= s > running FreeBSD 10.3. I installed an Intel X540-T2 10 Gb NIC and am > experiencing what I consider to be slow transfer speeds. I am using iperf= 3 > to measure the speed and test the results of modifications. So far nothin= g > I have done has made a noticeable difference. If I run iperf3 -s on the > FreeBSD server, I see transfer speeds of approximately 1.6 Gb/s. If I run > iperf3 in client mode, the speed improves to ~2.75 Gb/s. However, if I > replace FreeBSD with CentOS 7 on the same hardware, I see iperf3 speeds > surpassing 8 GB/s. The other end of my iperf3 test is a Windows 10 box th= at > also has an Intel X540-T2 installed. > > I did notice that FreeBSD 10.3 (and 11.0 alpha 6 for that matter) include= s > a slightly older Intel driver (v3.1.13-k). I managed to build a custom > kernel that removed the Intel PRO/10GbE PCIE NIC drivers. That allowed me > to manually load the latest 3.1.14 driver downloaded from Intel's web sit= e. > Unfortunately that did not produce any improvements. I am working my way > through man tuning() and some other articles on network performance. So f= ar > nothing I tweak makes a noticeable difference. I'm increasingly skeptical > that I am going to find a setting or two that more than doubles the speed= I > am currently experiencing. > > I am open to any and all suggestions at this point. > > Thank you! > Chris > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to " freebsd-net-unsubscribe@freebsd.org " > > > > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >