Date: Thu, 3 Oct 2013 15:41:27 -0700 From: Rumen Telbizov <telbizov@gmail.com> To: Jack Vogel <jfvogel@gmail.com> Cc: "freebsd-stable@freebsd.org" <freebsd-stable@freebsd.org> Subject: Re: ixgbe/ix sysctl missing in FreeBSD 9.2 Message-ID: <CAENR%2B_WsU%2BjJfCy_WKp2nueHa5cifhF6iBxvTaVvBGKnANesAg@mail.gmail.com> In-Reply-To: <CAFOYbc=AAtFknM2YRHT5-jLLoEve5SPy3pf%2BbgO5qTRu69wGAw@mail.gmail.com> References: <CAENR%2B_WxhoB30fwfB8wXCsgrF8YMKH5_F99SKRtXYmNH=U40Sg@mail.gmail.com> <CAFOYbcnfp5rvHT70MPLoW==kCeE_bMi979wcbbyWgQWKAdawgw@mail.gmail.com> <CAENR%2B_Xcav2HMGLKCE7pxuMrLQCg3=SM4e=EC0bunh1QGZh1vw@mail.gmail.com> <CAFOYbcn1DQueHOJEP5RkYQuBjT0drxKFvLSw4c3y2=h=CtpTiw@mail.gmail.com> <CAENR%2B_WQdcjZE71mOw_Jr5HX1HBwdYCmf4qF1EbkCuvrJcerXw@mail.gmail.com> <CAFOYbc=AAtFknM2YRHT5-jLLoEve5SPy3pf%2BbgO5qTRu69wGAw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Thanks Jack, The switch/lacp is where I was shifting my focus to. Unfortunately it's a remote site and it'll take some time before I can take it out of the picture so I figured I might ask you first ;) I'll report back here when I have more information. Thanks for your feedback. Highly appreciated. Cheers, Rumen Telbizov On Thu, Oct 3, 2013 at 3:38 PM, Jack Vogel <jfvogel@gmail.com> wrote: > Set the storm threshold to 0, that way you disable the check and don't > have to wonder > if its a problem :) > > Given everything you've told me I'd suspect the switch... easy enough to > test, remove it > from the equation, run two systems back-to-back and see how that effects > the numbers. > > Jack > > > > On Thu, Oct 3, 2013 at 3:24 PM, Rumen Telbizov <telbizov@gmail.com> wrote: > >> Jack, >> >> Thanks for the hints. >> >> I checked that the card is inserted into PCIe v3.0 x16 + I get no >> complaints from the driver. >> I checked for interrupt throttling messages in the logs - found nothing. I >> also set hw.intr_storm_threshold: 10000 >> then I saw hw.igb.max_interrupt_rate: 8000 and figured that I should have >> hw.ix or hw.ixgbe which seems >> to have existed in the past but I don't have it.Also tried this >> dev.ix.0.enable_aim: 0 + dev.ix.1.enable_aim: 0 >> >> I think I have enough mbufs: >> kern.ipc.nmbclusters=262144 >> kern.ipc.nmbjumbop=262144 >> >> I also tried doubling those. >> >> So after trying all those it still seems like I can't get past the >> 4-5Gbit/s barrier so I decided to ask. >> Now I start wondering if it has anything to do with the lacp/switch or >> even >> with that patch from Oleg which helped me overcome the problem with the >> flapping interface. >> >> If anything else comes to mind and there should be another knob to turn - >> feel free to suggest it. >> >> Thanks, >> Rumen Telbizov >> >> >> On Thu, Oct 3, 2013 at 3:15 PM, Jack Vogel <jfvogel@gmail.com> wrote: >> >> > Make sure you are in slots with enough lanes/bandwidth, the driver >> should >> > complain when loaded >> > if you aren't though. Make sure you aren't getting interrupts throttled, >> > and that you have sufficient >> > mbuf resources so you aren't waiting on those, that can make a BIG >> > difference. >> > >> > Jack >> > >> > >> > >> > On Thu, Oct 3, 2013 at 2:37 PM, Rumen Telbizov <telbizov@gmail.com> >> wrote: >> > >> >> Thanks Jack, >> >> >> >> So I am running a couple of FreeBSD machines with those and they are in >> >> LACP pair plus vlans on top of them. >> >> When I run iperf between the two I get about about 1.7Gbit/s with 1 >> >> thread and I max out around 4.5-5Gbp/s with 6-8 multiple parallel >> threads >> >> (tried both UDP and TCP). >> >> >> >> What kind of tuning would you recommend that I try? >> >> >> >> Thanks, >> >> Rumen Telbizov >> >> >> >> >> >> On Thu, Oct 3, 2013 at 1:35 PM, Jack Vogel <jfvogel@gmail.com> wrote: >> >> >> >>> sysctl dev.ix.0 etc... >> >>> >> >>> Jack >> >>> >> >>> >> >>> >> >>> On Thu, Oct 3, 2013 at 12:35 PM, Rumen Telbizov <telbizov@gmail.com >> >wrote: >> >>> >> >>>> Hello everyone, >> >>>> >> >>>> I am trying to tweak some of the sysctl tunables for the ix (ixgbe) >> >>>> driver >> >>>> in FreeBSD 9.2 since I am experiencing less than ideal performance >> and >> >>>> it >> >>>> seems like I can't find any: >> >>>> >> >>>> # sysctl -a | grep -i ixgbe >> >>>> device ixgbe >> >>>> >> >>>> I am running 9.2-RC4. >> >>>> >> >>>> Any input appreciated. >> >>>> >> >>>> Thanks, >> >>>> -- >> >>>> Rumen Telbizov >> >>>> Unix Systems Administrator <http://telbizov.com>; >> >>>> _______________________________________________ >> >>>> freebsd-stable@freebsd.org mailing list >> >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >> >>>> To unsubscribe, send any mail to " >> >>>> freebsd-stable-unsubscribe@freebsd.org" >> >>>> >> >>> >> >>> >> >> >> >> >> >> -- >> >> Rumen Telbizov >> >> Unix Systems Administrator <http://telbizov.com>; >> >> >> >> > >> > >> >> >> -- >> Rumen Telbizov >> Unix Systems Administrator <http://telbizov.com>; >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >> > > -- Rumen Telbizov Unix Systems Administrator <http://telbizov.com>;
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAENR%2B_WsU%2BjJfCy_WKp2nueHa5cifhF6iBxvTaVvBGKnANesAg>