From owner-freebsd-pf@FreeBSD.ORG Fri Nov 5 23:55:45 2010 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F2FDC106566B for ; Fri, 5 Nov 2010 23:55:44 +0000 (UTC) (envelope-from RCharlet@adaranet.com) Received: from barracuda.adaranet.com (smtp.adaranet.com [72.5.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id CCD6B8FC13 for ; Fri, 5 Nov 2010 23:55:44 +0000 (UTC) X-ASG-Debug-ID: 1289001343-0d0f4ab60001-7tp3by Received: from SJ-EXCH-1.adaranet.com ([10.10.1.29]) by barracuda.adaranet.com with ESMTP id 2gDChWntC6nuoTEt; Fri, 05 Nov 2010 16:55:43 -0700 (PDT) X-Barracuda-Envelope-From: RCharlet@adaranet.com Received: from SJ-EXCH-1.adaranet.com ([fe80::7042:d8c2:5973:c523]) by SJ-EXCH-1.adaranet.com ([fe80::7042:d8c2:5973:c523%14]) with mapi; Fri, 5 Nov 2010 16:55:44 -0700 From: "Ricky Charlet" X-Barracuda-BBL-IP: fe80::7042:d8c2:5973:c523 X-Barracuda-RBL-IP: fe80::7042:d8c2:5973:c523 To: "Ricky Charlet" , =?iso-8859-1?Q?'Ermal_Lu=E7i'?= Date: Fri, 5 Nov 2010 16:55:43 -0700 X-ASG-Orig-Subj: RE: Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000 Thread-Topic: Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000 Thread-Index: Act9DYVywabqTZ91Qwelyiop4y9BUAAG+FwQAAbMrAA= Message-ID: <32AB5C9615CC494997D9ABB1DB12783C024C6FC08F@SJ-EXCH-1.adaranet.com> References: <32AB5C9615CC494997D9ABB1DB12783C024C6FC071@SJ-EXCH-1.adaranet.com> <32AB5C9615CC494997D9ABB1DB12783C024C6FC07B@SJ-EXCH-1.adaranet.com> <32AB5C9615CC494997D9ABB1DB12783C024C6FC085@SJ-EXCH-1.adaranet.com> In-Reply-To: <32AB5C9615CC494997D9ABB1DB12783C024C6FC085@SJ-EXCH-1.adaranet.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Barracuda-Connect: UNKNOWN[10.10.1.29] X-Barracuda-Start-Time: 1289001343 X-Barracuda-URL: http://172.16.10.203:8000/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at adaranet.com Cc: "altq@csl.sony.co.jp" , "freebsd-pf@freebsd.org" Subject: RE: Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Nov 2010 23:55:45 -0000 OK, so what is the theory of operation behind this (empirically effective) = solution? I'll post my guess, please correct me. The slowness in the system is waiting on the timer to pop. The timer did no= t need to be set any quicker, just the granularity of how quickly the timer= s are run needed to be updated. > -----Original Message----- > From: owner-freebsd-pf@freebsd.org [mailto:owner-freebsd- > pf@freebsd.org] On Behalf Of Ricky Charlet > Sent: Friday, November 05, 2010 1:43 PM > To: 'Ermal Lu=E7i' > Cc: altq@csl.sony.co.jp; freebsd-pf@freebsd.org > Subject: RE: Questioning altq (cbq) performance above 4Mb on gif or > above 40 Mb on e1000 > > Yes.. this works very well. > > the first thing I tried: > I had a kern.hz of 1000 and changed to 10000 (through loader.conf + > reboot) > I had a queue depth of 50 and changed to 900 (through `qlimit` > parameter in pf.conf) > > > And now I am hitting my configured rate limits beyond my goal of 45Mb > (T3 speed) even into 100 Mb range. I'm going to continue to experiment > around with hz and q-depth and pick some eventual values for my > implementation. > > But thanks, Emeral. This seems to be exactly the parameters to play > around with to increase altq performance into the 50~100 Mb range. > > --- > Ricky Charlet > Adara Networks > USA 408-433-4942 > > > > > > > > > -----Original Message----- > > From: ermal.luci@gmail.com [mailto:ermal.luci@gmail.com] On Behalf Of > > Ermal Lu=E7i > > Sent: Friday, November 05, 2010 10:18 AM > > To: Ricky Charlet > > Cc: freebsd-pf@freebsd.org; altq@csl.sony.co.jp > > Subject: Re: Questioning altq (cbq) performance above 4Mb on gif or > > above 40 Mb on e1000 > > > > On Fri, Nov 5, 2010 at 1:33 AM, Ricky Charlet > > > wrote: > > > Has anyone out there run altq with cbq with bandwidth limits set > > around 40 ~ 50 Mb and seen it work well (actual through put allowed > to > > come near that speed)? > > > > > > Thanks > > > --- > > > Ricky Charlet > > > Adara Networks > > > USA 408-433-4942 > > > > > > > I can suggest to try with higher hz and increase queue lengths in > altq. > > By default they are 50 which is quite low in that regard. > > > > > > > >> -----Original Message----- > > >> From: owner-freebsd-pf@freebsd.org [mailto:owner-freebsd- > > >> pf@freebsd.org] On Behalf Of Ricky Charlet > > >> Sent: Wednesday, November 03, 2010 9:47 PM > > >> To: 'freebsd-pf@freebsd.org'; 'altq@csl.sony.co.jp' > > >> Subject: Questioning altq (cbq) performance above 4Mb on gif or > > above > > >> 40 Mb on e1000 > > >> > > >> Howdy, > > >> > > >> (FreeBsd 8.0REL plus a modified e1000 driver borrowing the 8.1 > altq > > >> capabilities, amd64, 8 core) > > >> > > >> I have been measuring altq performance on my system. I set > a > > >> cbq bandwith limit for a queue, match traffic to the queue with an > > >> appropriate rule and pump data. > > >> > > >> Without altq enabled, my traffic gen tool (nuttcp) can > pump > > >> 800Mb across my gig links. > > >> > > >> With altq(cbq) setting bandwith limits (no borrow), up to > > about > > >> 40Mb on em0 altq keeps up. But from 40Mb bandwith limit and up, > altq > > >> seems to slow down my traffic much slower than the bandwidth > limits. > > >> > > >> Same effect is observed on gif interfaces (where my real > > >> interest lies) but much worse. With altq(cbq) setting bandwith > > limits > > >> (no borrow), up to about 4Mb on gif0 altq keeps up. But from 4Mb > > >> bandwidth limit and up, altq seems to slow down my traffic much > > lower > > >> than the bandwidth limits. > > >> > > >> I have a large file of sample data in excel formant which > > I'd > > >> like to share. Is there an appropriate way to share it with this > > list? > > >> > > >> I know I'm not cpu bound (97% idle while nuttcp pushes > 800Mb > > >> traffic and altq configured with a bandwidth limit of 50Mb > actually > > >> passes 30Mb.) > > >> > > >> I sort of suspect I'm mutex bound but have no idea how to > > test > > >> that. > > >> > > >> > > >> > > >> (1) this config allows about 4Mb tcp traffic to pass: > > >> ----------cut------------------ > > >> altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 } > > >> queue queue1 bandwidth 1Mb priority 7 cbq (default) > > >> queue queue2 bandwidth 50Mb priority 1 cbq > > >> pass out quick on gif10 inet proto tcp no state queue queue2 > > >> pass out quick on gif10 proto icmp queue queue2 > > >> ----------paste---------------- > > >> > > >> (2) this config allows about 40 Mb tcp traffic to pass > > >> ----------cut------------------ > > >> altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 } > > >> queue queue1 bandwidth 1Mb priority 7 cbq (default) > > >> queue queue2 bandwidth 50Mb priority 1 cbq > > >> pass out quick on em0 inet proto tcp no state queue queue2 > > >> pass out quick on em0 proto icmp queue queue2 > > >> ----------paste---------------- > > >> > > >> Yes, I send appropriate traffic in each case to be > 'caught' > > by > > >> the tcp pass out rule wether it be found on a gif or an em. > > >> > > >> > > >> My goal is to get 'expected' behavior (a bandwidth limit > of > > 40 > > >> Mb allows 40Mb to pass) at T3 link speed (45Mb). Is this a > > reasonable > > >> expectation? Any ideas about config jigs or tests to run? > > >> > > >> > > >> > > >> --- > > >> Ricky Charlet > > >> Adara Networks > > >> USA 408-433-4942 > > >> > > >> > > >> _______________________________________________ > > >> freebsd-pf@freebsd.org mailing list > > >> http://lists.freebsd.org/mailman/listinfo/freebsd-pf > > >> To unsubscribe, send any mail to "freebsd-pf- > > unsubscribe@freebsd.org" > > > _______________________________________________ > > > freebsd-pf@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-pf > > > To unsubscribe, send any mail to "freebsd-pf- > unsubscribe@freebsd.org" > > > > > > > > > > > -- > > Ermal > _______________________________________________ > freebsd-pf@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-pf > To unsubscribe, send any mail to "freebsd-pf-unsubscribe@freebsd.org"