From owner-freebsd-stable@FreeBSD.ORG Thu Oct 3 22:24:55 2013 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id DD81DADC for ; Thu, 3 Oct 2013 22:24:54 +0000 (UTC) (envelope-from telbizov@gmail.com) Received: from mail-ie0-x22d.google.com (mail-ie0-x22d.google.com [IPv6:2607:f8b0:4001:c03::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id AA86923C0 for ; Thu, 3 Oct 2013 22:24:54 +0000 (UTC) Received: by mail-ie0-f173.google.com with SMTP id ar20so7073892iec.4 for ; Thu, 03 Oct 2013 15:24:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=cXekGzNHIh4yfF9D9M56kE6tnc56LIOLu4uHCaXbMzQ=; b=EvEsOj06O8jNiFcTkJnQKUHeuPGIFZ1pu5+5f08eLplWGkzIvREmA/QqpDLn4ZmUe6 B33nHsaCjwWUQxUMt/Ls9bQmwwKKi5L6NHurnC30k8i06UIlCfHgRPfTz9l7oAG1aiA1 O8Ds9vfo5qRnSXZO33RQDlXFLdGsQ7P8lTvE4z6a6AZ5VeZECO1TYnZ47k/8lnwOkddU Wn+MO+dyN7g6ZYxLPbSfw30lqZsrsQBwS4bMclxTu+svA47Yus+uVBXQ5iFiYBH4Hxql /Nby+R+DWjantWsT/1UXS3ssy+Is7nbIBeIVv4SjnpCaPhNSm9EjnVRpMUYoqcTXb/OU zk1w== MIME-Version: 1.0 X-Received: by 10.43.132.137 with SMTP id hu9mr22572icc.72.1380839094014; Thu, 03 Oct 2013 15:24:54 -0700 (PDT) Received: by 10.50.196.136 with HTTP; Thu, 3 Oct 2013 15:24:53 -0700 (PDT) In-Reply-To: References: Date: Thu, 3 Oct 2013 15:24:53 -0700 Message-ID: Subject: Re: ixgbe/ix sysctl missing in FreeBSD 9.2 From: Rumen Telbizov To: Jack Vogel Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: "freebsd-stable@freebsd.org" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Oct 2013 22:24:55 -0000 Jack, Thanks for the hints. I checked that the card is inserted into PCIe v3.0 x16 + I get no complaints from the driver. I checked for interrupt throttling messages in the logs - found nothing. I also set hw.intr_storm_threshold: 10000 then I saw hw.igb.max_interrupt_rate: 8000 and figured that I should have hw.ix or hw.ixgbe which seems to have existed in the past but I don't have it.Also tried this dev.ix.0.enable_aim: 0 + dev.ix.1.enable_aim: 0 I think I have enough mbufs: kern.ipc.nmbclusters=262144 kern.ipc.nmbjumbop=262144 I also tried doubling those. So after trying all those it still seems like I can't get past the 4-5Gbit/s barrier so I decided to ask. Now I start wondering if it has anything to do with the lacp/switch or even with that patch from Oleg which helped me overcome the problem with the flapping interface. If anything else comes to mind and there should be another knob to turn - feel free to suggest it. Thanks, Rumen Telbizov On Thu, Oct 3, 2013 at 3:15 PM, Jack Vogel wrote: > Make sure you are in slots with enough lanes/bandwidth, the driver should > complain when loaded > if you aren't though. Make sure you aren't getting interrupts throttled, > and that you have sufficient > mbuf resources so you aren't waiting on those, that can make a BIG > difference. > > Jack > > > > On Thu, Oct 3, 2013 at 2:37 PM, Rumen Telbizov wrote: > >> Thanks Jack, >> >> So I am running a couple of FreeBSD machines with those and they are in >> LACP pair plus vlans on top of them. >> When I run iperf between the two I get about about 1.7Gbit/s with 1 >> thread and I max out around 4.5-5Gbp/s with 6-8 multiple parallel threads >> (tried both UDP and TCP). >> >> What kind of tuning would you recommend that I try? >> >> Thanks, >> Rumen Telbizov >> >> >> On Thu, Oct 3, 2013 at 1:35 PM, Jack Vogel wrote: >> >>> sysctl dev.ix.0 etc... >>> >>> Jack >>> >>> >>> >>> On Thu, Oct 3, 2013 at 12:35 PM, Rumen Telbizov wrote: >>> >>>> Hello everyone, >>>> >>>> I am trying to tweak some of the sysctl tunables for the ix (ixgbe) >>>> driver >>>> in FreeBSD 9.2 since I am experiencing less than ideal performance and >>>> it >>>> seems like I can't find any: >>>> >>>> # sysctl -a | grep -i ixgbe >>>> device ixgbe >>>> >>>> I am running 9.2-RC4. >>>> >>>> Any input appreciated. >>>> >>>> Thanks, >>>> -- >>>> Rumen Telbizov >>>> Unix Systems Administrator >>>> _______________________________________________ >>>> freebsd-stable@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >>>> To unsubscribe, send any mail to " >>>> freebsd-stable-unsubscribe@freebsd.org" >>>> >>> >>> >> >> >> -- >> Rumen Telbizov >> Unix Systems Administrator >> > > -- Rumen Telbizov Unix Systems Administrator