From owner-freebsd-net@freebsd.org Wed Oct 21 14:14:51 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8B51FA1B6ED for ; Wed, 21 Oct 2015 14:14:51 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 65B80ECA for ; Wed, 21 Oct 2015 14:14:51 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 62AD8A1B6EB; Wed, 21 Oct 2015 14:14:51 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 48676A1B6EA for ; Wed, 21 Oct 2015 14:14:51 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-yk0-x22e.google.com (mail-yk0-x22e.google.com [IPv6:2607:f8b0:4002:c07::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 034FDEC8; Wed, 21 Oct 2015 14:14:51 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: by ykba4 with SMTP id a4so44203191ykb.3; Wed, 21 Oct 2015 07:14:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=Coe9FL2Gmi6JbUaHArwbKu5XaK7fQDSvS2y6jrQEBn4=; b=PNLeZStpgRHlJtQdPOy4icRc0DVXOs8tq2vmXyT2exF2Hhafp2n5FWUZguAvlkSX15 BzBBIm7zJ8Wgv6vzemOT6BH9lqSH1QBXBJ231ObLrY4jL77YYjt3nIxhIWMy4W64kNdJ FZ7t7EdwZxqmVfRAn5/CVBJM4nweb1NyAa0tkB+kWSo/Mln63zTIKKcCCRz99SV8IOge 8xg11aiZD5xfQUhyZG1p7S7QT+Hl2SqtEdgBWZr1/LqfGpL7kPufjieRLPn7TbK1vX76 uqcZoXb6fb+HbljWZLfJS2cecBNa/KJDg98PrWLAQwhWOWtTRw2kczXMk47SH1oIzIqr plsg== MIME-Version: 1.0 X-Received: by 10.13.204.65 with SMTP id o62mr1299419ywd.303.1445436889942; Wed, 21 Oct 2015 07:14:49 -0700 (PDT) Received: by 10.37.36.204 with HTTP; Wed, 21 Oct 2015 07:14:49 -0700 (PDT) In-Reply-To: <12938D48-44B9-456A-AD7C-630D7E09BB35@netapp.com> References: <79830D9D-94E6-47A9-92B9-D63DF5432272@netapp.com> <20151020190541.B15983@sola.nimnet.asn.au> <6CD6754D-FC0E-4B24-AAEC-7C9D68284141@netapp.com> <20151020232218.G1833@besplex.bde.org> <12938D48-44B9-456A-AD7C-630D7E09BB35@netapp.com> Date: Wed, 21 Oct 2015 07:14:49 -0700 Message-ID: Subject: Re: ixl 40G bad performance? From: Jack Vogel To: "Eggert, Lars" Cc: Bruce Evans , Ian Smith , Kevin Oberman , "net@freebsd.org" , "jfv@FreeBSD.org" , "ricera10@gmail.com" , Luigi Rizzo , Giuseppe Lettieri Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2015 14:14:51 -0000 The 40G hardware is absolutely dependent on firmware, if you have a mismatch for instance, it can totally bork things. So, I would work with your Intel rep and be sure you have the correct version for your specific hardware. Good luck, Jack On Wed, Oct 21, 2015 at 5:25 AM, Eggert, Lars wrote: > Hi Bruce, > > thanks for the very detailed analysis of the ixl sysctls! > > On 2015-10-20, at 16:51, Bruce Evans wrote: > > > > Lowering (improving) latency always lowers (unimproves) throughput by > > increasing load. > > That, I also understand. But even when I back off the itr values to > something more reasonable, throughput still remains low. > > With all the tweaking I have tried, I have yet to top 3 Gb/s with ixl > cards, whereas they do ~13 Gb/s on Linux straight out of the box. > > Lars >