From owner-freebsd-stable@FreeBSD.ORG Fri Oct 4 02:56:34 2013 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 781978F for ; Fri, 4 Oct 2013 02:56:34 +0000 (UTC) (envelope-from adrian.chadd@gmail.com) Received: from mail-wi0-x22e.google.com (mail-wi0-x22e.google.com [IPv6:2a00:1450:400c:c05::22e]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id F3FFD2F2B for ; Fri, 4 Oct 2013 02:56:33 +0000 (UTC) Received: by mail-wi0-f174.google.com with SMTP id hj3so961800wib.13 for ; Thu, 03 Oct 2013 19:56:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=2Mfy/oEBHg1WOALDypfR5G4/7r4UFuys8UIvQM9A5D8=; b=1IAXvtjBpKoQN+cWozy+4WsdKneuAElXP3uygvKdvuuJFvrMFFZuO2NQo1UWamVA07 OE7EsS5spFQf2B9VMKs8PYR+ItzpPtjgu9onGbcybEsZ0FwLsX47PGf/b3iTB4npea3p 6UlF7c5eBFe6+sJksB1tPg5Z7zsTEQCamBWRP97b4sZiTQUDQfjNcnrKjbLp8cIm2kil VTStebBA2PfNxKeieObXfJ0EKtmQ9LhvL8qDWJvRL2pGDbdsVXLieQGJ1ja0z6np+lg+ ph3IfgVjfTUqTho31WNmyTdx2U1beZxz14E+1fYCQjMCqCl8klrkVvVayOo0RahAez7v eRRw== MIME-Version: 1.0 X-Received: by 10.180.37.164 with SMTP id z4mr5300879wij.30.1380855391096; Thu, 03 Oct 2013 19:56:31 -0700 (PDT) Sender: adrian.chadd@gmail.com Received: by 10.216.73.133 with HTTP; Thu, 3 Oct 2013 19:56:31 -0700 (PDT) In-Reply-To: References: Date: Thu, 3 Oct 2013 19:56:31 -0700 X-Google-Sender-Auth: 3bM8mqfV0NOj4Z8f-ZHjaupYLbQ Message-ID: Subject: Re: ixgbe/ix sysctl missing in FreeBSD 9.2 From: Adrian Chadd To: Rumen Telbizov Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-stable@freebsd.org" , Jack Vogel X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Oct 2013 02:56:34 -0000 There's an LACP knob that ignores doing the LACP hash if the mbuf has a flowid. Find that knob and set it to 0. Then, retry. -a On 3 October 2013 15:24, Rumen Telbizov wrote: > Jack, > > Thanks for the hints. > > I checked that the card is inserted into PCIe v3.0 x16 + I get no > complaints from the driver. > I checked for interrupt throttling messages in the logs - found nothing. I > also set hw.intr_storm_threshold: 10000 > then I saw hw.igb.max_interrupt_rate: 8000 and figured that I should have > hw.ix or hw.ixgbe which seems > to have existed in the past but I don't have it.Also tried this > dev.ix.0.enable_aim: 0 + dev.ix.1.enable_aim: 0 > > I think I have enough mbufs: > kern.ipc.nmbclusters=262144 > kern.ipc.nmbjumbop=262144 > > I also tried doubling those. > > So after trying all those it still seems like I can't get past the > 4-5Gbit/s barrier so I decided to ask. > Now I start wondering if it has anything to do with the lacp/switch or even > with that patch from Oleg which helped me overcome the problem with the > flapping interface. > > If anything else comes to mind and there should be another knob to turn - > feel free to suggest it. > > Thanks, > Rumen Telbizov > > > On Thu, Oct 3, 2013 at 3:15 PM, Jack Vogel wrote: > > > Make sure you are in slots with enough lanes/bandwidth, the driver should > > complain when loaded > > if you aren't though. Make sure you aren't getting interrupts throttled, > > and that you have sufficient > > mbuf resources so you aren't waiting on those, that can make a BIG > > difference. > > > > Jack > > > > > > > > On Thu, Oct 3, 2013 at 2:37 PM, Rumen Telbizov > wrote: > > > >> Thanks Jack, > >> > >> So I am running a couple of FreeBSD machines with those and they are in > >> LACP pair plus vlans on top of them. > >> When I run iperf between the two I get about about 1.7Gbit/s with 1 > >> thread and I max out around 4.5-5Gbp/s with 6-8 multiple parallel > threads > >> (tried both UDP and TCP). > >> > >> What kind of tuning would you recommend that I try? > >> > >> Thanks, > >> Rumen Telbizov > >> > >> > >> On Thu, Oct 3, 2013 at 1:35 PM, Jack Vogel wrote: > >> > >>> sysctl dev.ix.0 etc... > >>> > >>> Jack > >>> > >>> > >>> > >>> On Thu, Oct 3, 2013 at 12:35 PM, Rumen Telbizov >wrote: > >>> > >>>> Hello everyone, > >>>> > >>>> I am trying to tweak some of the sysctl tunables for the ix (ixgbe) > >>>> driver > >>>> in FreeBSD 9.2 since I am experiencing less than ideal performance and > >>>> it > >>>> seems like I can't find any: > >>>> > >>>> # sysctl -a | grep -i ixgbe > >>>> device ixgbe > >>>> > >>>> I am running 9.2-RC4. > >>>> > >>>> Any input appreciated. > >>>> > >>>> Thanks, > >>>> -- > >>>> Rumen Telbizov > >>>> Unix Systems Administrator > >>>> _______________________________________________ > >>>> freebsd-stable@freebsd.org mailing list > >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-stable > >>>> To unsubscribe, send any mail to " > >>>> freebsd-stable-unsubscribe@freebsd.org" > >>>> > >>> > >>> > >> > >> > >> -- > >> Rumen Telbizov > >> Unix Systems Administrator > >> > > > > > > > -- > Rumen Telbizov > Unix Systems Administrator > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >