From owner-freebsd-net@freebsd.org Thu Dec 10 18:40:03 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 979539D63C4 for ; Thu, 10 Dec 2015 18:40:03 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-io0-x22c.google.com (mail-io0-x22c.google.com [IPv6:2607:f8b0:4001:c06::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 63A911242 for ; Thu, 10 Dec 2015 18:40:03 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: by iofh3 with SMTP id h3so102244588iof.3 for ; Thu, 10 Dec 2015 10:40:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=8STRfGfC47mO1xoUZTji8u0F7Ou0suCTPrQYtkR76RY=; b=zpFk2iTgf4Q05vWs/hXCKrgD8OicgBgjsQo3nKkOdTi1jnRXxHoXOIf8CSD3+K9fAm MaZrTpbyLXDkIjiFGMfLrTh8THrUkgHUl0rUGkHXsErshaWQ70N1TcYYicx8hczlBHlv wGMJw5H8MQg8CQ1h1FYGsq16FLzj2Hq+5Oc6m8JfJ2u6EgQ9Ruf6ezFL+9+yYod1ovFu n8WtTMHeVm9664Gs/vI4uixOQMI/UVI8+mMKL6Cpr82yKo+WU52f8MgdJA+C3u0DtwHf 92sTo8vbHK3oTIgGHE49mDfJgBc+CIQDiaaW1BrxRJeqHYsAzzrItSwu1nGdwMOME9/d LRQA== MIME-Version: 1.0 X-Received: by 10.107.11.147 with SMTP id 19mr12394735iol.165.1449772802868; Thu, 10 Dec 2015 10:40:02 -0800 (PST) Received: by 10.36.121.202 with HTTP; Thu, 10 Dec 2015 10:40:02 -0800 (PST) In-Reply-To: References: <5aae0ee63c44627223d5d179f1901d00@pyret.net> <0E4C2D93-FBAF-48CB-A704-499ABFC892B9@netapp.com> <2A35EA60C3C77D438915767F458D6568807F2A8A@ORSMSX111.amr.corp.intel.com> <99E53825-99F8-4E82-A710-6BC07B123F77@netapp.com> <2A35EA60C3C77D438915767F458D6568807F2D52@ORSMSX111.amr.corp.intel.com> Date: Thu, 10 Dec 2015 10:40:02 -0800 Message-ID: Subject: Re: ixl 40G bad performance? From: Adrian Chadd To: Denis Pearson Cc: "Eggert, Lars" , "Pieper, Jeffrey E" , Kevin Oberman , Daniel Engberg , "freebsd-net@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Dec 2015 18:40:03 -0000 On 10 December 2015 at 10:29, Denis Pearson wrote: > On Thu, Dec 10, 2015 at 2:18 PM, Eggert, Lars wrote: > >> On 2015-10-26, at 18:40, Eggert, Lars wrote: >> > On 2015-10-26, at 17:08, Pieper, Jeffrey E >> wrote: >> >> As a caveat, this was using default netperf message sizes. >> > >> > I get the same ~3 Gb/s with the default netperf sizes and driver 1.4.5. >> >> Now there is version 1.4.8 on the Intel website, but it doesn't change >> things for me. >> > > I had the opportunity to see similar numbers and behavior while using XL710 > 1.4.3 as of FreeBSD r291085 while in DPDK poll mode, but driver 1.2.8 as of > r292035 was providing expected numbers. While removing rxcsum/txcsum did > not provide differences, fully removing RSS + disabling rx/cxsum support > provided better numbers. Can someone debug this a bit more? (My kit with ixl NICs in it is still not up and available. :( ) Device RSS, even without kernel RSS enabled, shouldn't cause a massive performance drop. If it is then something else odd is going on. Do you have a diff where you removed things? -adrian > However now with driver 1.4.8 and the same set of hardware setup, except > for a different transceiver, I can get 36Gbps/24Mpps with no further > tweaks, so if you can replace your transceiver, shall be a different test > as a starting point.