From owner-freebsd-net@freebsd.org Sat Oct 24 08:32:57 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AC680A1DE26 for ; Sat, 24 Oct 2015 08:32:57 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 84570C2 for ; Sat, 24 Oct 2015 08:32:57 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 7EC6FA1DE25; Sat, 24 Oct 2015 08:32:57 +0000 (UTC) Delivered-To: net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D707A1DE24 for ; Sat, 24 Oct 2015 08:32:57 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-yk0-x241.google.com (mail-yk0-x241.google.com [IPv6:2607:f8b0:4002:c07::241]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2CECEBF; Sat, 24 Oct 2015 08:32:57 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: by ykfw201 with SMTP id w201so11597735ykf.2; Sat, 24 Oct 2015 01:32:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=0p/Zagey6vp/YtiELlFcAeNaBUTKrel+VxW3wCq+v+A=; b=oVrdjnLh2OXdl2rNAVndCnQx9oF5QyPxUUB+NrCESsAd6eabAYIvggWECi8Un+cKLy 8QQYVICptrX4He1M6pqhNCcWwmgHrQdbJ6QKHmE2sUiw11o/11YNlaVUqAXeoVtdo67u FLY3b6I4L0BCCVZwZRUJx54UX2onm9g9SR9108BvdtrC/fEWIWlTD3D9psry9m5cADzG MgVR/Or/AWHU8ncBszNRLkLG2TO3IPZ3OWX+u/jj3j8AVXJlfyPX7P8ZIlTO37MjJxUj idkDJbfddoAapjDH7O3eG0SOWFCrI69TfdTHk4HFCLOhaxxZIt5IQ4gSeMoBFoXgue/V cgmA== MIME-Version: 1.0 X-Received: by 10.129.125.212 with SMTP id y203mr18438392ywc.182.1445675576374; Sat, 24 Oct 2015 01:32:56 -0700 (PDT) Received: by 10.37.36.204 with HTTP; Sat, 24 Oct 2015 01:32:56 -0700 (PDT) In-Reply-To: <0CFE1957-8FEB-4000-9E39-1F6A8983EFA6@netapp.com> References: <79830D9D-94E6-47A9-92B9-D63DF5432272@netapp.com> <20151020190541.B15983@sola.nimnet.asn.au> <6CD6754D-FC0E-4B24-AAEC-7C9D68284141@netapp.com> <20151020232218.G1833@besplex.bde.org> <20151023162337.L1149@besplex.bde.org> <0CFE1957-8FEB-4000-9E39-1F6A8983EFA6@netapp.com> Date: Sat, 24 Oct 2015 01:32:56 -0700 Message-ID: Subject: Re: ixl 40G bad performance? From: Jack Vogel To: "Eggert, Lars" Cc: Eric Joyner , Bruce Evans , Ian Smith , Kevin Oberman , "net@freebsd.org" , "jfv@FreeBSD.org" , Luigi Rizzo , Giuseppe Lettieri Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Oct 2015 08:32:57 -0000 13 on a 40G interface?? I don't think that's very good for Linux either, is this a 4x10 adapter? Maybe elaborating on the details of the hardware, you sure you don't have a bad PCI slot somewhere that might be throttling everything? Cheers, Jack On Sat, Oct 24, 2015 at 12:43 AM, Eggert, Lars wrote: > On 2015-10-23, at 23:36, Eric Joyner wrote: > > I see that the sysctl does clobber the global value, but have you tried > lowering the interval / raising the rate? You could try something like > 10usecs, and see if that helps. We'll do some more investigation here -- > 3Gb/s on a 40Gb/s using default settings is terrible, and we shouldn't let > that be happening. > > > I played with different settings, but I've never been able to get more > than 4Gb/s, whereas under Linux 4.2 without any special settings I get 13. > > See my other email on TSO/LRO not looking to be effective; that would > certainly explain it. Plausible? Anything to try here? > > Lars > >