From owner-freebsd-pf@FreeBSD.ORG Mon Nov 1 11:07:02 2010 Return-Path: Delivered-To: freebsd-pf@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C7E3110656C0 for ; Mon, 1 Nov 2010 11:07:02 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id B4EEF8FC27 for ; Mon, 1 Nov 2010 11:07:02 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id oA1B72X8019250 for ; Mon, 1 Nov 2010 11:07:02 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id oA1B72sK019248 for freebsd-pf@FreeBSD.org; Mon, 1 Nov 2010 11:07:02 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 1 Nov 2010 11:07:02 GMT Message-Id: <201011011107.oA1B72sK019248@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-pf@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-pf@FreeBSD.org X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Nov 2010 11:07:02 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/148290 pf [pf] "sticky-address" option of Packet Filter (PF) blo o kern/148260 pf [pf] [patch] pf rdr incompatible with dummynet o kern/147789 pf [pf] Firewall PF no longer drops connections by sendin o kern/146832 pf [pf] "(self)" not always matching all local IPv6 addre o kern/143543 pf [pf] [panic] PF route-to causes kernel panic o bin/143504 pf [patch] outgoing states are not killed by authpf(8) o conf/142961 pf [pf] No way to adjust pidfile in pflogd o conf/142817 pf [patch] etc/rc.d/pf: silence pfctl o kern/141905 pf [pf] [panic] pf kernel panic on 7.2-RELEASE with empty o kern/140697 pf [pf] pf behaviour changes - must be documented o kern/137982 pf [pf] when pf can hit state limits, random IP failures o kern/136781 pf [pf] Packets appear to drop with pf scrub and if_bridg o kern/135948 pf [pf] [gre] pf not natting gre protocol o kern/135162 pf [pfsync] pfsync(4) not usable with GENERIC kernel o kern/134996 pf [pf] Anchor tables not included when pfctl(8) is run w o kern/133732 pf [pf] max-src-conn issue o kern/132769 pf [pf] [lor] 2 LOR's with pf task mtx / ifnet and rtent f kern/132176 pf [pf] pf stalls connection when using route-to [regress o conf/130381 pf [rc.d] [pf] [ip6] ipv6 not fully configured when pf st o kern/129861 pf [pf] [patch] Argument names reversed in pf_table.c:_co o kern/127920 pf [pf] ipv6 and synproxy don't play well together o conf/127814 pf [pf] The flush in pf_reload in /etc/rc.d/pf does not w o kern/127439 pf [pf] deadlock in pf f kern/127345 pf [pf] Problem with PF on FreeBSD7.0 [regression] o kern/127121 pf [pf] [patch] pf incorrect log priority o kern/127042 pf [pf] [patch] pf recursion panic if interface group is o kern/125467 pf [pf] pf keep state bug while handling sessions between s kern/124933 pf [pf] [ip6] pf does not support (drops) IPv6 fragmented o kern/124364 pf [pf] [panic] Kernel panic with pf + bridge o kern/122773 pf [pf] pf doesn't log uid or pid when configured to o kern/122014 pf [pf] [panic] FreeBSD 6.2 panic in pf o kern/120281 pf [pf] [request] lost returning packets to PF for a rdr o kern/120057 pf [pf] [patch] Allow proper settings of ALTQ_HFSC. The c o bin/118355 pf [pf] [patch] pfctl(8) help message options order false o kern/114567 pf [pf] [lor] pf_ioctl.c + if.c o kern/114095 pf [carp] carp+pf delay with high state limit o kern/111220 pf [pf] repeatable hangs while manipulating pf tables s conf/110838 pf [pf] tagged parameter on nat not working on FreeBSD 5. o kern/103283 pf pfsync fails to sucessfully transfer some sessions o kern/103281 pf pfsync reports bulk update failures o kern/93825 pf [pf] pf reply-to doesn't work o sparc/93530 pf [pf] Incorrect checksums when using pf's route-to on s o kern/92949 pf [pf] PF + ALTQ problems with latency o bin/86635 pf [patch] pfctl(8): allow new page character (^L) in pf. o kern/82271 pf [pf] cbq scheduler cause bad latency 45 problems total. From owner-freebsd-pf@FreeBSD.ORG Tue Nov 2 05:29:25 2010 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3D2C91065670 for ; Tue, 2 Nov 2010 05:29:25 +0000 (UTC) (envelope-from libin3389@126.com) Received: from m14-16.126.com (m14-16.126.com [220.181.14.16]) by mx1.freebsd.org (Postfix) with ESMTP id 2D3658FC12 for ; Tue, 2 Nov 2010 05:29:20 +0000 (UTC) Received: from libin3389 ( [220.250.12.250] ) by ajax-webmail-wmsvr101 (Coremail) ; Tue, 2 Nov 2010 12:57:34 +0800 (CST) Date: Tue, 2 Nov 2010 12:57:34 +0800 (CST) From: =?gbk?B?wO6x8w==?= To: freebsd-pf@freebsd.org Message-ID: <1479430.487b.12c0af285a8.Coremail.libin3389@126.com> MIME-Version: 1.0 X-Originating-IP: [220.250.12.250] X-Priority: 3 X-Mailer: Coremail Webmail Server Version SP_ntes V3.5 build 101029(12229.3533.3507) Copyright (c) 2002-2010 www.mailtech.cn 126com X-CM-CTRLDATA: 4DuVS2Zvb3Rlcl9odG09MzcyOjgx X-CM-TRANSID: ZcqowLBLqwc+ms9MZWYIAA--.462W X-CM-SenderInfo: polex0ittymqqrswhudrp/1tbiDwB9HEzBq1-vugAAsC X-Coremail-Antispam: 1U5529EdanIXcx71UUUUU7vcSsGvfC2KfnxnUU== X-Mailman-Approved-At: Tue, 02 Nov 2010 11:07:54 +0000 Content-Type: text/plain; charset=gbk Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: freebsd + pf packet loss problem X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Nov 2010 05:29:25 -0000 hello! I am a Chinese, my name is Lee I have a freebsd problem I used to do the firewall freebsd + pf Found in synflood attack, normal packets 1000 packets received only 971 Here are some of my system parameters and configuration, hoping to help solve the problem Very grateful Waiting for your letter From owner-freebsd-pf@FreeBSD.ORG Thu Nov 4 06:02:11 2010 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A9D4A106564A for ; Thu, 4 Nov 2010 06:02:11 +0000 (UTC) (envelope-from RCharlet@adaranet.com) Received: from barracuda.adaranet.com (smtp.adaranet.com [72.5.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 8B0EA8FC14 for ; Thu, 4 Nov 2010 06:02:11 +0000 (UTC) X-ASG-Debug-ID: 1288846019-0d0f33950001-7tp3by Received: from SJ-EXCH-1.adaranet.com ([10.10.1.29]) by barracuda.adaranet.com with ESMTP id OfCmXse58uLh9UIL; Wed, 03 Nov 2010 21:46:59 -0700 (PDT) X-Barracuda-Envelope-From: RCharlet@adaranet.com Received: from SJ-EXCH-1.adaranet.com ([fe80::7042:d8c2:5973:c523]) by SJ-EXCH-1.adaranet.com ([fe80::7042:d8c2:5973:c523%14]) with mapi; Wed, 3 Nov 2010 21:46:59 -0700 From: "Ricky Charlet" X-Barracuda-BBL-IP: fe80::7042:d8c2:5973:c523 X-Barracuda-RBL-IP: fe80::7042:d8c2:5973:c523 To: "'freebsd-pf@freebsd.org'" , "'altq@csl.sony.co.jp'" Date: Wed, 3 Nov 2010 21:46:58 -0700 X-ASG-Orig-Subj: Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000 Thread-Topic: Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000 Thread-Index: Act721AUxtVLhm2sSfimBYxth/NxEg== Message-ID: <32AB5C9615CC494997D9ABB1DB12783C024C6FC071@SJ-EXCH-1.adaranet.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Barracuda-Connect: UNKNOWN[10.10.1.29] X-Barracuda-Start-Time: 1288846019 X-Barracuda-URL: http://172.16.10.203:8000/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at adaranet.com Cc: Subject: Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Nov 2010 06:02:11 -0000 Howdy, (FreeBsd 8.0REL plus a modified e1000 driver borrowing the 8.1 altq capabil= ities, amd64, 8 core) I have been measuring altq performance on my system. I set a cbq ba= ndwith limit for a queue, match traffic to the queue with an appropriate ru= le and pump data. Without altq enabled, my traffic gen tool (nuttcp) can pump 800Mb a= cross my gig links. With altq(cbq) setting bandwith limits (no borrow), up to about 40M= b on em0 altq keeps up. But from 40Mb bandwith limit and up, altq seems to = slow down my traffic much slower than the bandwidth limits. Same effect is observed on gif interfaces (where my real interest l= ies) but much worse. With altq(cbq) setting bandwith limits (no borrow), up= to about 4Mb on gif0 altq keeps up. But from 4Mb bandwidth limit and up, a= ltq seems to slow down my traffic much lower than the bandwidth limits. I have a large file of sample data in excel formant which I'd like = to share. Is there an appropriate way to share it with this list? I know I'm not cpu bound (97% idle while nuttcp pushes 800Mb traffi= c and altq configured with a bandwidth limit of 50Mb actually passes 30Mb.) I sort of suspect I'm mutex bound but have no idea how to test that= . (1) this config allows about 4Mb tcp traffic to pass: ----------cut------------------ altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 } queue queue1 bandwidth 1Mb priority 7 cbq (default) queue queue2 bandwidth 50Mb priority 1 cbq pass out quick on gif10 inet proto tcp no state queue queue2 pass out quick on gif10 proto icmp queue queue2 ----------paste---------------- (2) this config allows about 40 Mb tcp traffic to pass ----------cut------------------ altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 } queue queue1 bandwidth 1Mb priority 7 cbq (default) queue queue2 bandwidth 50Mb priority 1 cbq pass out quick on em0 inet proto tcp no state queue queue2 pass out quick on em0 proto icmp queue queue2 ----------paste---------------- Yes, I send appropriate traffic in each case to be 'caught' by the = tcp pass out rule wether it be found on a gif or an em. My goal is to get 'expected' behavior (a bandwidth limit of 40 Mb a= llows 40Mb to pass) at T3 link speed (45Mb). Is this a reasonable expectati= on? Any ideas about config jigs or tests to run? --- Ricky Charlet Adara Networks USA 408-433-4942 From owner-freebsd-pf@FreeBSD.ORG Fri Nov 5 00:33:26 2010 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 25D9A1065698 for ; Fri, 5 Nov 2010 00:33:26 +0000 (UTC) (envelope-from RCharlet@adaranet.com) Received: from barracuda.adaranet.com (smtp.adaranet.com [72.5.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 0321D8FC16 for ; Fri, 5 Nov 2010 00:33:25 +0000 (UTC) X-ASG-Debug-ID: 1288917205-0d0f3ec90001-7tp3by Received: from SJ-EXCH-1.adaranet.com ([10.10.1.29]) by barracuda.adaranet.com with ESMTP id S1U644etm3RauqD1; Thu, 04 Nov 2010 17:33:25 -0700 (PDT) X-Barracuda-Envelope-From: RCharlet@adaranet.com Received: from SJ-EXCH-1.adaranet.com ([fe80::7042:d8c2:5973:c523]) by SJ-EXCH-1.adaranet.com ([fe80::7042:d8c2:5973:c523%14]) with mapi; Thu, 4 Nov 2010 17:33:25 -0700 From: "Ricky Charlet" X-Barracuda-BBL-IP: fe80::7042:d8c2:5973:c523 X-Barracuda-RBL-IP: fe80::7042:d8c2:5973:c523 To: "Ricky Charlet" , "'freebsd-pf@freebsd.org'" , "'altq@csl.sony.co.jp'" Date: Thu, 4 Nov 2010 17:33:24 -0700 X-ASG-Orig-Subj: RE: Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000 Thread-Topic: Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000 Thread-Index: Act721AUxtVLhm2sSfimBYxth/NxEgApYedA Message-ID: <32AB5C9615CC494997D9ABB1DB12783C024C6FC07B@SJ-EXCH-1.adaranet.com> References: <32AB5C9615CC494997D9ABB1DB12783C024C6FC071@SJ-EXCH-1.adaranet.com> In-Reply-To: <32AB5C9615CC494997D9ABB1DB12783C024C6FC071@SJ-EXCH-1.adaranet.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Barracuda-Connect: UNKNOWN[10.10.1.29] X-Barracuda-Start-Time: 1288917205 X-Barracuda-URL: http://172.16.10.203:8000/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at adaranet.com Cc: Subject: RE: Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Nov 2010 00:33:26 -0000 Has anyone out there run altq with cbq with bandwidth limits set around 40 = ~ 50 Mb and seen it work well (actual through put allowed to come near that= speed)? Thanks --- Ricky Charlet Adara Networks USA 408-433-4942 > -----Original Message----- > From: owner-freebsd-pf@freebsd.org [mailto:owner-freebsd- > pf@freebsd.org] On Behalf Of Ricky Charlet > Sent: Wednesday, November 03, 2010 9:47 PM > To: 'freebsd-pf@freebsd.org'; 'altq@csl.sony.co.jp' > Subject: Questioning altq (cbq) performance above 4Mb on gif or above > 40 Mb on e1000 > > Howdy, > > (FreeBsd 8.0REL plus a modified e1000 driver borrowing the 8.1 altq > capabilities, amd64, 8 core) > > I have been measuring altq performance on my system. I set a > cbq bandwith limit for a queue, match traffic to the queue with an > appropriate rule and pump data. > > Without altq enabled, my traffic gen tool (nuttcp) can pump > 800Mb across my gig links. > > With altq(cbq) setting bandwith limits (no borrow), up to about > 40Mb on em0 altq keeps up. But from 40Mb bandwith limit and up, altq > seems to slow down my traffic much slower than the bandwidth limits. > > Same effect is observed on gif interfaces (where my real > interest lies) but much worse. With altq(cbq) setting bandwith limits > (no borrow), up to about 4Mb on gif0 altq keeps up. But from 4Mb > bandwidth limit and up, altq seems to slow down my traffic much lower > than the bandwidth limits. > > I have a large file of sample data in excel formant which I'd > like to share. Is there an appropriate way to share it with this list? > > I know I'm not cpu bound (97% idle while nuttcp pushes 800Mb > traffic and altq configured with a bandwidth limit of 50Mb actually > passes 30Mb.) > > I sort of suspect I'm mutex bound but have no idea how to test > that. > > > > (1) this config allows about 4Mb tcp traffic to pass: > ----------cut------------------ > altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 } > queue queue1 bandwidth 1Mb priority 7 cbq (default) > queue queue2 bandwidth 50Mb priority 1 cbq > pass out quick on gif10 inet proto tcp no state queue queue2 > pass out quick on gif10 proto icmp queue queue2 > ----------paste---------------- > > (2) this config allows about 40 Mb tcp traffic to pass > ----------cut------------------ > altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 } > queue queue1 bandwidth 1Mb priority 7 cbq (default) > queue queue2 bandwidth 50Mb priority 1 cbq > pass out quick on em0 inet proto tcp no state queue queue2 > pass out quick on em0 proto icmp queue queue2 > ----------paste---------------- > > Yes, I send appropriate traffic in each case to be 'caught' by > the tcp pass out rule wether it be found on a gif or an em. > > > My goal is to get 'expected' behavior (a bandwidth limit of 40 > Mb allows 40Mb to pass) at T3 link speed (45Mb). Is this a reasonable > expectation? Any ideas about config jigs or tests to run? > > > > --- > Ricky Charlet > Adara Networks > USA 408-433-4942 > > > _______________________________________________ > freebsd-pf@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-pf > To unsubscribe, send any mail to "freebsd-pf-unsubscribe@freebsd.org" From owner-freebsd-pf@FreeBSD.ORG Fri Nov 5 17:18:52 2010 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 65D0B106564A for ; Fri, 5 Nov 2010 17:18:52 +0000 (UTC) (envelope-from ermal.luci@gmail.com) Received: from mail-bw0-f54.google.com (mail-bw0-f54.google.com [209.85.214.54]) by mx1.freebsd.org (Postfix) with ESMTP id E003D8FC14 for ; Fri, 5 Nov 2010 17:18:51 +0000 (UTC) Received: by bwz3 with SMTP id 3so2841938bwz.13 for ; Fri, 05 Nov 2010 10:18:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:sender:received :in-reply-to:references:from:date:x-google-sender-auth:message-id :subject:to:cc:content-type:content-transfer-encoding; bh=AgqmPydeCS9uhGtPiQxpYVERjzv3TObUXfcYQKuOGpk=; b=aG79V91TkM0VxeiZkSWUFqwXym3/5VRIea2PAtVTcbmYAPPs2rcVRJ4at29a/cCGVu zyGLz32MA0DJL3Y3Rcrd5G8KchsZFPb0bSYQ2I2iWAopUvgaZ30kFfZEZKeruTjOMMwA h9nByinQ5q0UWfhMTTnLiOH7JTzMAQC/UMlPc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type :content-transfer-encoding; b=D4LS5q4R4IfIz+/E0mWAg78ciI9E4SOhF+5bfTi0/k/Rre/gR5PIkZqe/JsOZtNhlt YfG5rMTASHAtr9CUWobzHv4/7Cmjxj+Jt/+NFPTdpJKKVNEw0zq8yuUUZ5r2DgHWDO0e F2OpcsCE/vhoMaQHHWQBn+iEpSu5mSJh6IGNA= Received: by 10.204.120.136 with SMTP id d8mr2011280bkr.152.1288977530602; Fri, 05 Nov 2010 10:18:50 -0700 (PDT) MIME-Version: 1.0 Sender: ermal.luci@gmail.com Received: by 10.204.35.68 with HTTP; Fri, 5 Nov 2010 10:18:27 -0700 (PDT) In-Reply-To: <32AB5C9615CC494997D9ABB1DB12783C024C6FC07B@SJ-EXCH-1.adaranet.com> References: <32AB5C9615CC494997D9ABB1DB12783C024C6FC071@SJ-EXCH-1.adaranet.com> <32AB5C9615CC494997D9ABB1DB12783C024C6FC07B@SJ-EXCH-1.adaranet.com> From: =?ISO-8859-1?Q?Ermal_Lu=E7i?= Date: Fri, 5 Nov 2010 18:18:27 +0100 X-Google-Sender-Auth: NlRRsCD7EPRMOqqFuqLF0dMp2g8 Message-ID: To: Ricky Charlet Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: "altq@csl.sony.co.jp" , "freebsd-pf@freebsd.org" Subject: Re: Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Nov 2010 17:18:52 -0000 On Fri, Nov 5, 2010 at 1:33 AM, Ricky Charlet wrot= e: > Has anyone out there run altq with cbq with bandwidth limits set around 4= 0 ~ 50 Mb and seen it work well (actual through put allowed to come near th= at speed)? > > Thanks > --- > Ricky Charlet > Adara Networks > USA 408-433-4942 > I can suggest to try with higher hz and increase queue lengths in altq. By default they are 50 which is quite low in that regard. > >> -----Original Message----- >> From: owner-freebsd-pf@freebsd.org [mailto:owner-freebsd- >> pf@freebsd.org] On Behalf Of Ricky Charlet >> Sent: Wednesday, November 03, 2010 9:47 PM >> To: 'freebsd-pf@freebsd.org'; 'altq@csl.sony.co.jp' >> Subject: Questioning altq (cbq) performance above 4Mb on gif or above >> 40 Mb on e1000 >> >> Howdy, >> >> (FreeBsd 8.0REL plus a modified e1000 driver borrowing the 8.1 altq >> capabilities, amd64, 8 core) >> >> =A0 =A0 =A0 =A0 I have been measuring altq performance on my system. I s= et a >> cbq bandwith limit for a queue, match traffic to the queue with an >> appropriate rule and pump data. >> >> =A0 =A0 =A0 =A0 Without altq enabled, my traffic gen tool (nuttcp) can p= ump >> 800Mb across my gig links. >> >> =A0 =A0 =A0 =A0 With altq(cbq) setting bandwith limits (no borrow), up t= o about >> 40Mb on em0 altq keeps up. But from 40Mb bandwith limit and up, altq >> seems to slow down my traffic much slower than the bandwidth limits. >> >> =A0 =A0 =A0 =A0 Same effect is observed on gif interfaces (where my real >> interest lies) but much worse. With altq(cbq) setting bandwith limits >> (no borrow), up to about 4Mb on gif0 altq keeps up. But from 4Mb >> bandwidth limit and up, altq seems to slow down my traffic much lower >> than the bandwidth limits. >> >> =A0 =A0 =A0 =A0 I have a large file of sample data in excel formant whic= h I'd >> like to share. Is there an appropriate way to share it with this list? >> >> =A0 =A0 =A0 =A0 I know I'm not cpu bound (97% idle while nuttcp pushes 8= 00Mb >> traffic and altq configured with a bandwidth limit of 50Mb actually >> passes 30Mb.) >> >> =A0 =A0 =A0 =A0 I sort of suspect I'm mutex bound but have no idea how t= o test >> that. >> >> >> >> =A0(1) this config allows about 4Mb tcp traffic to pass: >> ----------cut------------------ >> altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 } >> queue queue1 =A0bandwidth 1Mb priority 7 cbq =A0(default) >> queue queue2 bandwidth 50Mb priority 1 cbq >> pass out quick on gif10 inet proto tcp no state queue queue2 >> pass out quick on gif10 proto icmp queue queue2 >> ----------paste---------------- >> >> (2) this config allows about 40 Mb tcp traffic to pass >> ----------cut------------------ >> altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 } >> queue queue1 =A0bandwidth 1Mb priority 7 cbq =A0(default) >> queue queue2 bandwidth 50Mb priority 1 cbq >> pass out quick on em0 inet proto tcp no state queue queue2 >> pass out quick on em0 proto icmp queue queue2 >> ----------paste---------------- >> >> =A0 =A0 =A0 =A0 Yes, I send appropriate traffic in each case to be 'caug= ht' by >> the tcp pass out rule wether it be found on a gif or an em. >> >> >> =A0 =A0 =A0 =A0 My goal is to get 'expected' behavior (a bandwidth limit= of 40 >> Mb allows 40Mb to pass) at T3 link speed (45Mb). Is this a reasonable >> expectation? Any ideas about config jigs or tests to run? >> >> >> >> --- >> Ricky Charlet >> Adara Networks >> USA 408-433-4942 >> >> >> _______________________________________________ >> freebsd-pf@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-pf >> To unsubscribe, send any mail to "freebsd-pf-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-pf@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-pf > To unsubscribe, send any mail to "freebsd-pf-unsubscribe@freebsd.org" > --=20 Ermal From owner-freebsd-pf@FreeBSD.ORG Fri Nov 5 19:32:11 2010 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 68ABD1065674 for ; Fri, 5 Nov 2010 19:32:11 +0000 (UTC) (envelope-from RCharlet@adaranet.com) Received: from barracuda.adaranet.com (smtp.adaranet.com [72.5.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 43A8F8FC0A for ; Fri, 5 Nov 2010 19:32:11 +0000 (UTC) X-ASG-Debug-ID: 1288983302-0d0f47c90001-7tp3by Received: from SJ-EXCH-1.adaranet.com ([10.10.1.29]) by barracuda.adaranet.com with ESMTP id CArGnVGpMPasAyhW; Fri, 05 Nov 2010 11:55:02 -0700 (PDT) X-Barracuda-Envelope-From: RCharlet@adaranet.com Received: from SJ-EXCH-1.adaranet.com ([fe80::7042:d8c2:5973:c523]) by SJ-EXCH-1.adaranet.com ([fe80::7042:d8c2:5973:c523%14]) with mapi; Fri, 5 Nov 2010 11:55:02 -0700 From: "Ricky Charlet" X-Barracuda-BBL-IP: fe80::7042:d8c2:5973:c523 X-Barracuda-RBL-IP: fe80::7042:d8c2:5973:c523 To: =?iso-8859-1?Q?'Ermal_Lu=E7i'?= Date: Fri, 5 Nov 2010 11:55:01 -0700 X-ASG-Orig-Subj: RE: Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000 Thread-Topic: Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000 Thread-Index: Act9DYVywabqTZ91Qwelyiop4y9BUAACLhKw Message-ID: <32AB5C9615CC494997D9ABB1DB12783C024C6FC081@SJ-EXCH-1.adaranet.com> References: <32AB5C9615CC494997D9ABB1DB12783C024C6FC071@SJ-EXCH-1.adaranet.com> <32AB5C9615CC494997D9ABB1DB12783C024C6FC07B@SJ-EXCH-1.adaranet.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Barracuda-Connect: UNKNOWN[10.10.1.29] X-Barracuda-Start-Time: 1288983302 X-Barracuda-URL: http://172.16.10.203:8000/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at adaranet.com Cc: "altq@csl.sony.co.jp" , "freebsd-pf@freebsd.org" Subject: RE: Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Nov 2010 19:32:11 -0000 Thanks for the reply Ermal, Based on your suggestion, I tried raising the qlimit, but it had no effect. I have a GigE physical link. I set a child queue to catch my traffic and se= t a bandwidth limit of 1000Mb on the root and 45 Mb on the child. Through t= his config, I can pump and measure about 10Mb of traffic. I tried qlimits (= on both root and child) of 100 and 800. Neither had any effect. I also obse= rved with `pfctl -vvsq` that my queue depths tended to be in the 10s. I nev= er saw a queue depth sample above 5 and by far, it was most often 0. `systa= t -v` shows a 99%idle cpu (an 8 core amd64). Note: I am testing with tcp traffic, which throttles itself to the observed= path speed. But I still expect to come near to my configured bandwidth lim= it. I've been trying to read the altq code and grok it. On the dequeue side, I'= m not sure how the token bucket regulator interacts with cbq_dequeue. Can a= nyone point me to an architecture or design guide document/ presentation ab= out this? Hopefully, --- Ricky Charlet Adara Networks USA 408-433-4942 > -----Original Message----- > From: ermal.luci@gmail.com [mailto:ermal.luci@gmail.com] On Behalf Of > Ermal Lu=E7i > Sent: Friday, November 05, 2010 10:18 AM > To: Ricky Charlet > Cc: freebsd-pf@freebsd.org; altq@csl.sony.co.jp > Subject: Re: Questioning altq (cbq) performance above 4Mb on gif or > above 40 Mb on e1000 > > On Fri, Nov 5, 2010 at 1:33 AM, Ricky Charlet > wrote: > > Has anyone out there run altq with cbq with bandwidth limits set > around 40 ~ 50 Mb and seen it work well (actual through put allowed to > come near that speed)? > > > > Thanks > > --- > > Ricky Charlet > > Adara Networks > > USA 408-433-4942 > > > > I can suggest to try with higher hz and increase queue lengths in altq. > By default they are 50 which is quite low in that regard. > > > > >> -----Original Message----- > >> From: owner-freebsd-pf@freebsd.org [mailto:owner-freebsd- > >> pf@freebsd.org] On Behalf Of Ricky Charlet > >> Sent: Wednesday, November 03, 2010 9:47 PM > >> To: 'freebsd-pf@freebsd.org'; 'altq@csl.sony.co.jp' > >> Subject: Questioning altq (cbq) performance above 4Mb on gif or > above > >> 40 Mb on e1000 > >> > >> Howdy, > >> > >> (FreeBsd 8.0REL plus a modified e1000 driver borrowing the 8.1 altq > >> capabilities, amd64, 8 core) > >> > >> I have been measuring altq performance on my system. I set a > >> cbq bandwith limit for a queue, match traffic to the queue with an > >> appropriate rule and pump data. > >> > >> Without altq enabled, my traffic gen tool (nuttcp) can pump > >> 800Mb across my gig links. > >> > >> With altq(cbq) setting bandwith limits (no borrow), up to > about > >> 40Mb on em0 altq keeps up. But from 40Mb bandwith limit and up, altq > >> seems to slow down my traffic much slower than the bandwidth limits. > >> > >> Same effect is observed on gif interfaces (where my real > >> interest lies) but much worse. With altq(cbq) setting bandwith > limits > >> (no borrow), up to about 4Mb on gif0 altq keeps up. But from 4Mb > >> bandwidth limit and up, altq seems to slow down my traffic much > lower > >> than the bandwidth limits. > >> > >> I have a large file of sample data in excel formant which > I'd > >> like to share. Is there an appropriate way to share it with this > list? > >> > >> I know I'm not cpu bound (97% idle while nuttcp pushes 800Mb > >> traffic and altq configured with a bandwidth limit of 50Mb actually > >> passes 30Mb.) > >> > >> I sort of suspect I'm mutex bound but have no idea how to > test > >> that. > >> > >> > >> > >> (1) this config allows about 4Mb tcp traffic to pass: > >> ----------cut------------------ > >> altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 } > >> queue queue1 bandwidth 1Mb priority 7 cbq (default) > >> queue queue2 bandwidth 50Mb priority 1 cbq > >> pass out quick on gif10 inet proto tcp no state queue queue2 > >> pass out quick on gif10 proto icmp queue queue2 > >> ----------paste---------------- > >> > >> (2) this config allows about 40 Mb tcp traffic to pass > >> ----------cut------------------ > >> altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 } > >> queue queue1 bandwidth 1Mb priority 7 cbq (default) > >> queue queue2 bandwidth 50Mb priority 1 cbq > >> pass out quick on em0 inet proto tcp no state queue queue2 > >> pass out quick on em0 proto icmp queue queue2 > >> ----------paste---------------- > >> > >> Yes, I send appropriate traffic in each case to be 'caught' > by > >> the tcp pass out rule wether it be found on a gif or an em. > >> > >> > >> My goal is to get 'expected' behavior (a bandwidth limit of > 40 > >> Mb allows 40Mb to pass) at T3 link speed (45Mb). Is this a > reasonable > >> expectation? Any ideas about config jigs or tests to run? > >> > >> > >> > >> --- > >> Ricky Charlet > >> Adara Networks > >> USA 408-433-4942 > >> > >> > >> _______________________________________________ > >> freebsd-pf@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-pf > >> To unsubscribe, send any mail to "freebsd-pf- > unsubscribe@freebsd.org" > > _______________________________________________ > > freebsd-pf@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-pf > > To unsubscribe, send any mail to "freebsd-pf-unsubscribe@freebsd.org" > > > > > > -- > Ermal From owner-freebsd-pf@FreeBSD.ORG Fri Nov 5 20:43:11 2010 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 206691065672 for ; Fri, 5 Nov 2010 20:43:11 +0000 (UTC) (envelope-from RCharlet@adaranet.com) Received: from barracuda.adaranet.com (smtp.adaranet.com [72.5.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id EFD148FC16 for ; Fri, 5 Nov 2010 20:43:10 +0000 (UTC) X-ASG-Debug-ID: 1288989790-0d0f48f50001-7tp3by Received: from SJ-EXCH-1.adaranet.com ([10.10.1.29]) by barracuda.adaranet.com with ESMTP id zlrms3YsdstgH00u; Fri, 05 Nov 2010 13:43:10 -0700 (PDT) X-Barracuda-Envelope-From: RCharlet@adaranet.com Received: from SJ-EXCH-1.adaranet.com ([fe80::7042:d8c2:5973:c523]) by SJ-EXCH-1.adaranet.com ([fe80::7042:d8c2:5973:c523%14]) with mapi; Fri, 5 Nov 2010 13:43:10 -0700 From: "Ricky Charlet" X-Barracuda-BBL-IP: fe80::7042:d8c2:5973:c523 X-Barracuda-RBL-IP: fe80::7042:d8c2:5973:c523 To: =?iso-8859-1?Q?'Ermal_Lu=E7i'?= Date: Fri, 5 Nov 2010 13:43:09 -0700 X-ASG-Orig-Subj: RE: Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000 Thread-Topic: Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000 Thread-Index: Act9DYVywabqTZ91Qwelyiop4y9BUAAG+FwQ Message-ID: <32AB5C9615CC494997D9ABB1DB12783C024C6FC085@SJ-EXCH-1.adaranet.com> References: <32AB5C9615CC494997D9ABB1DB12783C024C6FC071@SJ-EXCH-1.adaranet.com> <32AB5C9615CC494997D9ABB1DB12783C024C6FC07B@SJ-EXCH-1.adaranet.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Barracuda-Connect: UNKNOWN[10.10.1.29] X-Barracuda-Start-Time: 1288989790 X-Barracuda-URL: http://172.16.10.203:8000/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at adaranet.com Cc: "altq@csl.sony.co.jp" , "freebsd-pf@freebsd.org" Subject: RE: Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Nov 2010 20:43:11 -0000 Yes.. this works very well. the first thing I tried: I had a kern.hz of 1000 and changed to 10000 (through loader.conf + reboot) I had a queue depth of 50 and changed to 900 (through `qlimit` parameter in= pf.conf) And now I am hitting my configured rate limits beyond my goal of 45Mb (T3 s= peed) even into 100 Mb range. I'm going to continue to experiment around wi= th hz and q-depth and pick some eventual values for my implementation. But thanks, Emeral. This seems to be exactly the parameters to play around = with to increase altq performance into the 50~100 Mb range. --- Ricky Charlet Adara Networks USA 408-433-4942 > -----Original Message----- > From: ermal.luci@gmail.com [mailto:ermal.luci@gmail.com] On Behalf Of > Ermal Lu=E7i > Sent: Friday, November 05, 2010 10:18 AM > To: Ricky Charlet > Cc: freebsd-pf@freebsd.org; altq@csl.sony.co.jp > Subject: Re: Questioning altq (cbq) performance above 4Mb on gif or > above 40 Mb on e1000 > > On Fri, Nov 5, 2010 at 1:33 AM, Ricky Charlet > wrote: > > Has anyone out there run altq with cbq with bandwidth limits set > around 40 ~ 50 Mb and seen it work well (actual through put allowed to > come near that speed)? > > > > Thanks > > --- > > Ricky Charlet > > Adara Networks > > USA 408-433-4942 > > > > I can suggest to try with higher hz and increase queue lengths in altq. > By default they are 50 which is quite low in that regard. > > > > >> -----Original Message----- > >> From: owner-freebsd-pf@freebsd.org [mailto:owner-freebsd- > >> pf@freebsd.org] On Behalf Of Ricky Charlet > >> Sent: Wednesday, November 03, 2010 9:47 PM > >> To: 'freebsd-pf@freebsd.org'; 'altq@csl.sony.co.jp' > >> Subject: Questioning altq (cbq) performance above 4Mb on gif or > above > >> 40 Mb on e1000 > >> > >> Howdy, > >> > >> (FreeBsd 8.0REL plus a modified e1000 driver borrowing the 8.1 altq > >> capabilities, amd64, 8 core) > >> > >> I have been measuring altq performance on my system. I set a > >> cbq bandwith limit for a queue, match traffic to the queue with an > >> appropriate rule and pump data. > >> > >> Without altq enabled, my traffic gen tool (nuttcp) can pump > >> 800Mb across my gig links. > >> > >> With altq(cbq) setting bandwith limits (no borrow), up to > about > >> 40Mb on em0 altq keeps up. But from 40Mb bandwith limit and up, altq > >> seems to slow down my traffic much slower than the bandwidth limits. > >> > >> Same effect is observed on gif interfaces (where my real > >> interest lies) but much worse. With altq(cbq) setting bandwith > limits > >> (no borrow), up to about 4Mb on gif0 altq keeps up. But from 4Mb > >> bandwidth limit and up, altq seems to slow down my traffic much > lower > >> than the bandwidth limits. > >> > >> I have a large file of sample data in excel formant which > I'd > >> like to share. Is there an appropriate way to share it with this > list? > >> > >> I know I'm not cpu bound (97% idle while nuttcp pushes 800Mb > >> traffic and altq configured with a bandwidth limit of 50Mb actually > >> passes 30Mb.) > >> > >> I sort of suspect I'm mutex bound but have no idea how to > test > >> that. > >> > >> > >> > >> (1) this config allows about 4Mb tcp traffic to pass: > >> ----------cut------------------ > >> altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 } > >> queue queue1 bandwidth 1Mb priority 7 cbq (default) > >> queue queue2 bandwidth 50Mb priority 1 cbq > >> pass out quick on gif10 inet proto tcp no state queue queue2 > >> pass out quick on gif10 proto icmp queue queue2 > >> ----------paste---------------- > >> > >> (2) this config allows about 40 Mb tcp traffic to pass > >> ----------cut------------------ > >> altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 } > >> queue queue1 bandwidth 1Mb priority 7 cbq (default) > >> queue queue2 bandwidth 50Mb priority 1 cbq > >> pass out quick on em0 inet proto tcp no state queue queue2 > >> pass out quick on em0 proto icmp queue queue2 > >> ----------paste---------------- > >> > >> Yes, I send appropriate traffic in each case to be 'caught' > by > >> the tcp pass out rule wether it be found on a gif or an em. > >> > >> > >> My goal is to get 'expected' behavior (a bandwidth limit of > 40 > >> Mb allows 40Mb to pass) at T3 link speed (45Mb). Is this a > reasonable > >> expectation? Any ideas about config jigs or tests to run? > >> > >> > >> > >> --- > >> Ricky Charlet > >> Adara Networks > >> USA 408-433-4942 > >> > >> > >> _______________________________________________ > >> freebsd-pf@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-pf > >> To unsubscribe, send any mail to "freebsd-pf- > unsubscribe@freebsd.org" > > _______________________________________________ > > freebsd-pf@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-pf > > To unsubscribe, send any mail to "freebsd-pf-unsubscribe@freebsd.org" > > > > > > -- > Ermal From owner-freebsd-pf@FreeBSD.ORG Fri Nov 5 23:55:45 2010 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F2FDC106566B for ; Fri, 5 Nov 2010 23:55:44 +0000 (UTC) (envelope-from RCharlet@adaranet.com) Received: from barracuda.adaranet.com (smtp.adaranet.com [72.5.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id CCD6B8FC13 for ; Fri, 5 Nov 2010 23:55:44 +0000 (UTC) X-ASG-Debug-ID: 1289001343-0d0f4ab60001-7tp3by Received: from SJ-EXCH-1.adaranet.com ([10.10.1.29]) by barracuda.adaranet.com with ESMTP id 2gDChWntC6nuoTEt; Fri, 05 Nov 2010 16:55:43 -0700 (PDT) X-Barracuda-Envelope-From: RCharlet@adaranet.com Received: from SJ-EXCH-1.adaranet.com ([fe80::7042:d8c2:5973:c523]) by SJ-EXCH-1.adaranet.com ([fe80::7042:d8c2:5973:c523%14]) with mapi; Fri, 5 Nov 2010 16:55:44 -0700 From: "Ricky Charlet" X-Barracuda-BBL-IP: fe80::7042:d8c2:5973:c523 X-Barracuda-RBL-IP: fe80::7042:d8c2:5973:c523 To: "Ricky Charlet" , =?iso-8859-1?Q?'Ermal_Lu=E7i'?= Date: Fri, 5 Nov 2010 16:55:43 -0700 X-ASG-Orig-Subj: RE: Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000 Thread-Topic: Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000 Thread-Index: Act9DYVywabqTZ91Qwelyiop4y9BUAAG+FwQAAbMrAA= Message-ID: <32AB5C9615CC494997D9ABB1DB12783C024C6FC08F@SJ-EXCH-1.adaranet.com> References: <32AB5C9615CC494997D9ABB1DB12783C024C6FC071@SJ-EXCH-1.adaranet.com> <32AB5C9615CC494997D9ABB1DB12783C024C6FC07B@SJ-EXCH-1.adaranet.com> <32AB5C9615CC494997D9ABB1DB12783C024C6FC085@SJ-EXCH-1.adaranet.com> In-Reply-To: <32AB5C9615CC494997D9ABB1DB12783C024C6FC085@SJ-EXCH-1.adaranet.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Barracuda-Connect: UNKNOWN[10.10.1.29] X-Barracuda-Start-Time: 1289001343 X-Barracuda-URL: http://172.16.10.203:8000/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at adaranet.com Cc: "altq@csl.sony.co.jp" , "freebsd-pf@freebsd.org" Subject: RE: Questioning altq (cbq) performance above 4Mb on gif or above 40 Mb on e1000 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Nov 2010 23:55:45 -0000 OK, so what is the theory of operation behind this (empirically effective) = solution? I'll post my guess, please correct me. The slowness in the system is waiting on the timer to pop. The timer did no= t need to be set any quicker, just the granularity of how quickly the timer= s are run needed to be updated. > -----Original Message----- > From: owner-freebsd-pf@freebsd.org [mailto:owner-freebsd- > pf@freebsd.org] On Behalf Of Ricky Charlet > Sent: Friday, November 05, 2010 1:43 PM > To: 'Ermal Lu=E7i' > Cc: altq@csl.sony.co.jp; freebsd-pf@freebsd.org > Subject: RE: Questioning altq (cbq) performance above 4Mb on gif or > above 40 Mb on e1000 > > Yes.. this works very well. > > the first thing I tried: > I had a kern.hz of 1000 and changed to 10000 (through loader.conf + > reboot) > I had a queue depth of 50 and changed to 900 (through `qlimit` > parameter in pf.conf) > > > And now I am hitting my configured rate limits beyond my goal of 45Mb > (T3 speed) even into 100 Mb range. I'm going to continue to experiment > around with hz and q-depth and pick some eventual values for my > implementation. > > But thanks, Emeral. This seems to be exactly the parameters to play > around with to increase altq performance into the 50~100 Mb range. > > --- > Ricky Charlet > Adara Networks > USA 408-433-4942 > > > > > > > > > -----Original Message----- > > From: ermal.luci@gmail.com [mailto:ermal.luci@gmail.com] On Behalf Of > > Ermal Lu=E7i > > Sent: Friday, November 05, 2010 10:18 AM > > To: Ricky Charlet > > Cc: freebsd-pf@freebsd.org; altq@csl.sony.co.jp > > Subject: Re: Questioning altq (cbq) performance above 4Mb on gif or > > above 40 Mb on e1000 > > > > On Fri, Nov 5, 2010 at 1:33 AM, Ricky Charlet > > > wrote: > > > Has anyone out there run altq with cbq with bandwidth limits set > > around 40 ~ 50 Mb and seen it work well (actual through put allowed > to > > come near that speed)? > > > > > > Thanks > > > --- > > > Ricky Charlet > > > Adara Networks > > > USA 408-433-4942 > > > > > > > I can suggest to try with higher hz and increase queue lengths in > altq. > > By default they are 50 which is quite low in that regard. > > > > > > > >> -----Original Message----- > > >> From: owner-freebsd-pf@freebsd.org [mailto:owner-freebsd- > > >> pf@freebsd.org] On Behalf Of Ricky Charlet > > >> Sent: Wednesday, November 03, 2010 9:47 PM > > >> To: 'freebsd-pf@freebsd.org'; 'altq@csl.sony.co.jp' > > >> Subject: Questioning altq (cbq) performance above 4Mb on gif or > > above > > >> 40 Mb on e1000 > > >> > > >> Howdy, > > >> > > >> (FreeBsd 8.0REL plus a modified e1000 driver borrowing the 8.1 > altq > > >> capabilities, amd64, 8 core) > > >> > > >> I have been measuring altq performance on my system. I set > a > > >> cbq bandwith limit for a queue, match traffic to the queue with an > > >> appropriate rule and pump data. > > >> > > >> Without altq enabled, my traffic gen tool (nuttcp) can > pump > > >> 800Mb across my gig links. > > >> > > >> With altq(cbq) setting bandwith limits (no borrow), up to > > about > > >> 40Mb on em0 altq keeps up. But from 40Mb bandwith limit and up, > altq > > >> seems to slow down my traffic much slower than the bandwidth > limits. > > >> > > >> Same effect is observed on gif interfaces (where my real > > >> interest lies) but much worse. With altq(cbq) setting bandwith > > limits > > >> (no borrow), up to about 4Mb on gif0 altq keeps up. But from 4Mb > > >> bandwidth limit and up, altq seems to slow down my traffic much > > lower > > >> than the bandwidth limits. > > >> > > >> I have a large file of sample data in excel formant which > > I'd > > >> like to share. Is there an appropriate way to share it with this > > list? > > >> > > >> I know I'm not cpu bound (97% idle while nuttcp pushes > 800Mb > > >> traffic and altq configured with a bandwidth limit of 50Mb > actually > > >> passes 30Mb.) > > >> > > >> I sort of suspect I'm mutex bound but have no idea how to > > test > > >> that. > > >> > > >> > > >> > > >> (1) this config allows about 4Mb tcp traffic to pass: > > >> ----------cut------------------ > > >> altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 } > > >> queue queue1 bandwidth 1Mb priority 7 cbq (default) > > >> queue queue2 bandwidth 50Mb priority 1 cbq > > >> pass out quick on gif10 inet proto tcp no state queue queue2 > > >> pass out quick on gif10 proto icmp queue queue2 > > >> ----------paste---------------- > > >> > > >> (2) this config allows about 40 Mb tcp traffic to pass > > >> ----------cut------------------ > > >> altq on em0 cbq bandwidth 100Mb queue { queue1, queue2 } > > >> queue queue1 bandwidth 1Mb priority 7 cbq (default) > > >> queue queue2 bandwidth 50Mb priority 1 cbq > > >> pass out quick on em0 inet proto tcp no state queue queue2 > > >> pass out quick on em0 proto icmp queue queue2 > > >> ----------paste---------------- > > >> > > >> Yes, I send appropriate traffic in each case to be > 'caught' > > by > > >> the tcp pass out rule wether it be found on a gif or an em. > > >> > > >> > > >> My goal is to get 'expected' behavior (a bandwidth limit > of > > 40 > > >> Mb allows 40Mb to pass) at T3 link speed (45Mb). Is this a > > reasonable > > >> expectation? Any ideas about config jigs or tests to run? > > >> > > >> > > >> > > >> --- > > >> Ricky Charlet > > >> Adara Networks > > >> USA 408-433-4942 > > >> > > >> > > >> _______________________________________________ > > >> freebsd-pf@freebsd.org mailing list > > >> http://lists.freebsd.org/mailman/listinfo/freebsd-pf > > >> To unsubscribe, send any mail to "freebsd-pf- > > unsubscribe@freebsd.org" > > > _______________________________________________ > > > freebsd-pf@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-pf > > > To unsubscribe, send any mail to "freebsd-pf- > unsubscribe@freebsd.org" > > > > > > > > > > > -- > > Ermal > _______________________________________________ > freebsd-pf@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-pf > To unsubscribe, send any mail to "freebsd-pf-unsubscribe@freebsd.org" From owner-freebsd-pf@FreeBSD.ORG Sat Nov 6 00:12:11 2010 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5570B106566C for ; Sat, 6 Nov 2010 00:12:11 +0000 (UTC) (envelope-from RCharlet@adaranet.com) Received: from barracuda.adaranet.com (smtp.adaranet.com [72.5.229.2]) by mx1.freebsd.org (Postfix) with ESMTP id 3AFDE8FC0A for ; Sat, 6 Nov 2010 00:12:11 +0000 (UTC) X-ASG-Debug-ID: 1289001169-0d0f4ab00001-7tp3by Received: from SJ-EXCH-1.adaranet.com ([10.10.1.29]) by barracuda.adaranet.com with ESMTP id e0DznsQYTsUpfH4E for ; Fri, 05 Nov 2010 16:52:49 -0700 (PDT) X-Barracuda-Envelope-From: RCharlet@adaranet.com Received: from SJ-EXCH-1.adaranet.com ([fe80::7042:d8c2:5973:c523]) by SJ-EXCH-1.adaranet.com ([fe80::7042:d8c2:5973:c523%14]) with mapi; Fri, 5 Nov 2010 16:52:49 -0700 From: "Ricky Charlet" X-Barracuda-BBL-IP: fe80::7042:d8c2:5973:c523 X-Barracuda-RBL-IP: fe80::7042:d8c2:5973:c523 To: "freebsd-pf@freebsd.org" Date: Fri, 5 Nov 2010 16:52:49 -0700 X-ASG-Orig-Subj: shy about borrowing Thread-Topic: shy about borrowing Thread-Index: Act9RI151XMR2iIiRhSfmoeL8oiibw== Message-ID: <32AB5C9615CC494997D9ABB1DB12783C024C6FC08E@SJ-EXCH-1.adaranet.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Barracuda-Connect: UNKNOWN[10.10.1.29] X-Barracuda-Start-Time: 1289001169 X-Barracuda-URL: http://172.16.10.203:8000/cgi-mod/mark.cgi X-Virus-Scanned: by bsmtpd at adaranet.com Subject: shy about borrowing X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 06 Nov 2010 00:12:11 -0000 Howdy, altq on gif10 cbq qlimit 300 bandwidth 45Mb queue { queue1, queue2 } queue queue1 qlimit 300 bandwidth 1Mb priority 7 cbq (default) queue queue2 qlimit 300 bandwidth 10Mb priority 1 cbq(borrow red) pass out quick on gif10 inet proto tcp no state queue queue2 If I run traffic through my queue2, it runs about 12Mb and `pfctl -vvsq` do= es show some borrows. But even though it is borrowing some, why does it not= borrow up to my 45Mb root queue limit? --- Ricky Charlet Adara Networks USA 408-433-4942