From owner-freebsd-ipfw@FreeBSD.ORG Mon Aug 9 00:53:55 2004 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9979E16A4CE for ; Mon, 9 Aug 2004 00:53:55 +0000 (GMT) Received: from ns1.itga.com.au (ns1.itga.com.au [202.53.40.214]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7690243D1D for ; Mon, 9 Aug 2004 00:53:54 +0000 (GMT) (envelope-from gnb@itga.com.au) Received: from lightning.itga.com.au (lightning.itga.com.au [192.168.71.20]) by ns1.itga.com.au (8.12.9/8.12.9) with ESMTP id i790rmR5020128; Mon, 9 Aug 2004 10:53:48 +1000 (EST) (envelope-from gnb@itga.com.au) Received: from lightning.itga.com.au (localhost [127.0.0.1]) by lightning.itga.com.au (8.9.3/8.9.3) with ESMTP id KAA24990; Mon, 9 Aug 2004 10:53:47 +1000 (EST) Message-Id: <200408090053.KAA24990@lightning.itga.com.au> X-Mailer: exmh version 2.4 05/15/2001 with nmh-1.0.4 From: Gregory Bond To: Forrest Aldrich In-reply-to: Your message of Fri, 06 Aug 2004 18:31:51 -0400. Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 09 Aug 2004 10:53:46 +1000 Sender: gnb@itga.com.au cc: freebsd-ipfw@freebsd.org Subject: Re: Blocking SMTP traffic based upon RBL.... X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Aug 2004 00:53:55 -0000 The IPFW rules are really not the place for this. Your MTA will have built-in support for RBLs which is a much more appropriate place. For sendmail, see the "FEATURE(`dnsbl'...)", well, feature. From owner-freebsd-ipfw@FreeBSD.ORG Mon Aug 9 11:02:18 2004 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AB2A016A4CE for ; Mon, 9 Aug 2004 11:02:18 +0000 (GMT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id A187243D3F for ; Mon, 9 Aug 2004 11:02:18 +0000 (GMT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.11/8.12.11) with ESMTP id i79B2I95078195 for ; Mon, 9 Aug 2004 11:02:18 GMT (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.12.11/8.12.11/Submit) id i79B2H6Z078189 for ipfw@freebsd.org; Mon, 9 Aug 2004 11:02:17 GMT (envelope-from owner-bugmaster@freebsd.org) Date: Mon, 9 Aug 2004 11:02:17 GMT Message-Id: <200408091102.i79B2H6Z078189@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: ipfw@FreeBSD.org Subject: Current problem reports assigned to you X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Aug 2004 11:02:18 -0000 Current FreeBSD problem reports Critical problems Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2002/12/27] kern/46557 ipfw ipfw pipe show fails with lots of queues o [2003/04/22] kern/51274 ipfw ipfw2 create dynamic rules with parent nu f [2003/04/24] kern/51341 ipfw ipfw rule 'deny icmp from any to any icmp o [2003/12/11] i386/60154 ipfw ipfw core (crash) o [2004/03/03] kern/63724 ipfw IPFW2 Queues dont t work o [2004/03/14] kern/64240 ipfw IPFW tee terminates rule processing 6 problems total. Non-critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- a [2001/04/13] kern/26534 ipfw Add an option to ipfw to log gid/uid of w o [2002/12/07] kern/46080 ipfw [PATCH] logamount in ipfw2 does not defau o [2002/12/10] kern/46159 ipfw ipfw dynamic rules lifetime feature o [2002/12/27] kern/46564 ipfw IPFilter and IPFW processing order is not o [2003/02/11] kern/48172 ipfw ipfw does not log size and flags o [2003/03/10] kern/49086 ipfw [patch] Make ipfw2 log to different syslo o [2003/03/12] bin/49959 ipfw ipfw tee port rule skips parsing next rul o [2003/04/09] bin/50749 ipfw ipfw2 incorrectly parses ports and port r o [2003/08/26] kern/55984 ipfw [patch] time based firewalling support fo o [2003/12/30] kern/60719 ipfw ipfw: Headerless fragments generate cryp o [2004/01/12] kern/61259 ipfw [patch] make "ipfw tee" work as intended o [2004/03/09] kern/63961 ipfw ipfw2 uid matching doesn't work correctly 12 problems total. From owner-freebsd-ipfw@FreeBSD.ORG Thu Aug 12 16:59:11 2004 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 90BF116A4CE for ; Thu, 12 Aug 2004 16:59:11 +0000 (GMT) Received: from mx1.mail.ru (mx1.mail.ru [194.67.23.121]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5656F43D53 for ; Thu, 12 Aug 2004 16:59:11 +0000 (GMT) (envelope-from avalon_temp@mail.ru) Received: from [80.237.26.193] (port=5586 helo=[127.0.0.1]) by mx1.mail.ru with esmtp id 1BvIv0-0008OF-00 for freebsd-ipfw@FreeBSD.org; Thu, 12 Aug 2004 20:59:10 +0400 Date: Thu, 12 Aug 2004 19:52:41 +0300 From: none X-Mailer: The Bat! (v2.10.01) X-Priority: 3 (Normal) Message-ID: <1806191849.20040812195241@none.no> To: freebsd-ipfw@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Spam: Not detected Subject: X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: none List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Aug 2004 16:59:11 -0000 From owner-freebsd-ipfw@FreeBSD.ORG Thu Aug 12 19:30:25 2004 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E957616A4CE for ; Thu, 12 Aug 2004 19:30:25 +0000 (GMT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9048D43D48 for ; Thu, 12 Aug 2004 19:30:25 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.11/8.12.11) with ESMTP id i7CJUPXt059590 for ; Thu, 12 Aug 2004 19:30:25 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.12.11/8.12.11/Submit) id i7CJUPYE059582; Thu, 12 Aug 2004 19:30:25 GMT (envelope-from gnats) Date: Thu, 12 Aug 2004 19:30:25 GMT Message-Id: <200408121930.i7CJUPYE059582@freefall.freebsd.org> To: ipfw@FreeBSD.org From: Pawel Malachowski Subject: Re: kern/46557: ipfw pipe show fails with lots of queues X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Pawel Malachowski List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Aug 2004 19:30:26 -0000 The following reply was made to PR kern/46557; it has been noted by GNATS. From: Pawel Malachowski To: freebsd-gnats-submit@FreeBSD.org, eugen@grosbein.pp.ru Cc: freebsd-bugs@freebsd.org Subject: Re: kern/46557: ipfw pipe show fails with lots of queues Date: Thu, 12 Aug 2004 16:10:20 +0200 Hello, I've just hit this problem on my 4.10-STABLE dummynet shaper. Very bad, since `ipfw pipe show N' + net.inet.ip.dummynet.expire=0 is quite useful for grabbing some per-user statistics. :/ Here are more details what is going on. I hope someone will look at this and explain why malloc() can fail here and what can be done to prevent this. % ipfw pipe show ipfw: getsockopt(IP_DUMMYNET_GET): No buffer space available This command invokes getsockopt() trying to fetch all pipes data from kernel to userland. Kernel part of this is being done in dummynet_get(), which tries to allocate buf big (previously computed size) enough for all data: buf = malloc(size, M_TEMP, M_NOWAIT); if (buf == 0) { splx(s); return ENOBUFS ; } This malloc() call fails sometimes on loaded system (10k-70k of dynamic pipes + make buildworld;)) causing `ipfw pipe show' command failure. I've registered temporary MALLOC_DEFINE M_YOYO and changed this malloc/free from M_TEMP to M_YOYO, ;) so I can easily track this in `vmstat -m' output: 1. Quite early (no problems): Type InUse MemUse HighUse Limit Requests Limit Limit Size(s) IpFw/IpAcct 113 14K 14K 42107K 113 0 0 64,128,256 yoyo 0 0K 120K 42107K 1 0 0 128K dummynet 1612 311K 313K 42107K 139366 0 0 16,128,256,4K Memory Totals: In Use Free Requests 3921K 32K 173824 2. After some time malloc() in dummynet_get() fails and: Type InUse MemUse HighUse Limit Requests Limit Limit Size(s) IpFw/IpAcct 113 14K 14K 42107K 113 0 0 64,128,256 yoyo 0 0K 2828K 42107K 12913 0 0 128K,512K dummynet 21432 2790K 3354K 42107K 9804136 0 0 16,128,256,512,1K,4K Memory Totals: In Use Free Requests 9932K 3029K 14216860 In /sbin/ipfw2, list(): (do_cmd(ocmd, data, (uintptr_t)&nbytes) < 0) nbytes were 2188288 // In loop, I +1024 numbytes instead of *2+200, ignore this In ip_dummynet.c, dummynet_get(): buf = malloc(size, M_YOYO, M_NOWAIT); // note: M_TEMP => M_YOYO size was 2390416 and malloc() failed. -- Paweł Małachowski From owner-freebsd-ipfw@FreeBSD.ORG Fri Aug 13 02:32:43 2004 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 93F3F16A4CE for ; Fri, 13 Aug 2004 02:32:43 +0000 (GMT) Received: from ctb-mesg6.saix.net (ctb-mesg6.saix.net [196.25.240.78]) by mx1.FreeBSD.org (Postfix) with ESMTP id DC82343D2F for ; Fri, 13 Aug 2004 02:32:42 +0000 (GMT) (envelope-from savage@savage.za.org) Received: from netsphere.varynet.co.za (wblv-237-185.telkomadsl.co.za [165.165.237.185]) by ctb-mesg6.saix.net (Postfix) with ESMTP id AB6742C124 for ; Fri, 13 Aug 2004 04:32:38 +0200 (SAST) Received: from 192-168-0-251.ops.varynet.co.za ([192.168.0.251] helo=netphobia) by netsphere.varynet.co.za with smtp (Exim 4.34 (FreeBSD)) id 1BvRru-000Mru-sS for freebsd-ipfw@freebsd.org; Fri, 13 Aug 2004 04:32:36 +0200 Message-ID: <000b01c480dd$d4531940$fb00a8c0@savage.za.org> From: "Chris Knipe" To: Date: Fri, 13 Aug 2004 04:32:51 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1437 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 X-Spam-Score: 0.1 (/) X-Spam-Report: 0.1/5.5 Subject: ipfw & skipto.... confused a bit... X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Chris Knipe List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Aug 2004 02:32:43 -0000 Hi all, I'm a tad confused with skiptos. I want to use them, because I am automating setup procedures of rather large firewall tables via perl / mysql. From the 65535 available rules, blocks have been reserved for certain type of functions during the firewalling process. As such, I basically use all the available numbers. My last automated block is from 65450 to 65500 :/ Let's have a look quickly at a small block so that I can have a example of what I am referring to.... ####################################################################### #### Transparent Services ### ####################################################################### ${fwcmd} add 16000 allow tcp from ${LANIP} to any 25 out via tun1 skipto 16010 ${fwcmd} add 16001 allow tcp from ${LANIP} to any 80 out via tun0 skipto 16010 ${fwcmd} add 16002 allow tcp from ${LANIP} to any 80 out via tun1 skipto 16010 ${fwcmd} add 16003 fwd ${LANIP},3128 tcp from any to any dst-port 80 ${fwcmd} add 16004 fwd ${LANIP},25 tcp from any to any 25 ${fwcmd} add 16005 deny tcp from any to any 25 out via tun0 ${fwcmd} add 16006 deny tcp from any to any 80 out via tun0 ${fwcmd} add 16007 deny tcp from any to any 80 out via tun1 Right. That block, is for transparent proxy and smtp services. Except for the skipto's, the rules are fine and work. My next automated block will start at rule 16010. What will be the correct way to write those skipto lines????? Basically, I want to allow tcp/25 & tcp/80 from ${LANIP} via tun0 & tun1. If the rule has matched the traffic, I want the traffic passed, and the next rule processed to be 16010. Basically, I want to tell ipfw to allow / deny / count / queue the packets, and go to rule 16010. And yes, I've tried 'ipfw add skipto x count ip from any to any', and no, it does not work... :/ This is just a example as well. I use skipto's in the above configuration for allow, deny, count, and queue rules (From what I can see, it will be the deny, count, and queue rules that will be troublesome with skiptos).... So it's quite broad, and well yeah... Any help will be appreciated. Another question. If I use pipes and queues to configure bandwidth usage... What is the general way to accommodate transparent services and pipes (basically, so that the transparent service won't bypass a queue that should be enforced)? Should the bandwidth be shaped before, or after the transparent service? The same can be asked for divert sockets (Divert before or after queues). If anyone care to explain, I'd appreciate it allot. The divert I am referring to, is also not for NAT purposes - should anyone wonder.... -- me From owner-freebsd-ipfw@FreeBSD.ORG Fri Aug 13 07:03:21 2004 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 74BD516A4CE for ; Fri, 13 Aug 2004 07:03:21 +0000 (GMT) Received: from hetzner.co.za (lfw.hetzner.co.za [196.7.18.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id AD9AF43D1F for ; Fri, 13 Aug 2004 07:03:20 +0000 (GMT) (envelope-from ianf@hetzner.co.za) Received: from localhost ([127.0.0.1]) by hetzner.co.za with esmtp (Exim 3.36 #1) id 1BvW54-0002Wf-00; Fri, 13 Aug 2004 09:02:26 +0200 To: Chris Knipe From: Ian FREISLICH In-Reply-To: Message from "Chris Knipe" <000b01c480dd$d4531940$fb00a8c0@savage.za.org> Date: Fri, 13 Aug 2004 09:02:26 +0200 Sender: ianf@hetzner.co.za Message-Id: cc: freebsd-ipfw@freebsd.org Subject: Re: ipfw & skipto.... confused a bit... X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Aug 2004 07:03:21 -0000 "Chris Knipe" wrote: > Hi all, > > I'm a tad confused with skiptos. I want to use them, because I am > automating setup procedures of rather large firewall tables via perl / > mysql. From the 65535 available rules, blocks have been reserved for > certain type of functions during the firewalling process. As such, I > basically use all the available numbers. My last automated block is from > 65450 to 65500 :/ > > Let's have a look quickly at a small block so that I can have a example of > what I am referring to.... > > ####################################################################### > #### Transparent Services ### > ####################################################################### > ${fwcmd} add 16000 allow tcp from ${LANIP} to any 25 out via tun1 skipto > 16010 I thought that you had to use skipto as the action, not the rule body: ${fwcmd} add 16000 skipto 16010 tcp from ${LANIP} to any 25 out via tun1 Ian -- Ian Freislich From owner-freebsd-ipfw@FreeBSD.ORG Fri Aug 13 07:08:07 2004 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E61E516A4CE for ; Fri, 13 Aug 2004 07:08:07 +0000 (GMT) Received: from ctb-mesg1.saix.net (ctb-mesg1.saix.net [196.25.240.73]) by mx1.FreeBSD.org (Postfix) with ESMTP id 81E4343D45 for ; Fri, 13 Aug 2004 07:08:07 +0000 (GMT) (envelope-from savage@savage.za.org) Received: from netsphere.varynet.co.za (wblv-237-185.telkomadsl.co.za [165.165.237.185]) by ctb-mesg1.saix.net (Postfix) with ESMTP id F1526673B for ; Fri, 13 Aug 2004 09:08:02 +0200 (SAST) Received: from 192-168-0-251.ops.varynet.co.za ([192.168.0.251] helo=netphobia) by netsphere.varynet.co.za with smtp (Exim 4.34 (FreeBSD)) id 1BvWAS-000NVk-sg for freebsd-ipfw@freebsd.org; Fri, 13 Aug 2004 09:08:02 +0200 Message-ID: <000901c48104$4f0b4f10$fb00a8c0@savage.za.org> From: "Chris Knipe" To: References: Date: Fri, 13 Aug 2004 09:08:17 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1437 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 X-Spam-Score: 0.1 (/) X-Spam-Report: 0.1/5.5 Subject: Re: ipfw & skipto.... confused a bit... X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Chris Knipe List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Aug 2004 07:08:08 -0000 ----- Original Message ----- From: "Ian FREISLICH" To: "Chris Knipe" Cc: Sent: Friday, August 13, 2004 9:02 AM Subject: Re: ipfw & skipto.... confused a bit... > "Chris Knipe" wrote: > > Hi all, > > > > I'm a tad confused with skiptos. I want to use them, because I am > > automating setup procedures of rather large firewall tables via perl / > > mysql. From the 65535 available rules, blocks have been reserved for > > certain type of functions during the firewalling process. As such, I > > basically use all the available numbers. My last automated block is from > > 65450 to 65500 :/ > > > > Let's have a look quickly at a small block so that I can have a example of > > what I am referring to.... > > > > ####################################################################### > > #### Transparent Services ### > > ####################################################################### > > ${fwcmd} add 16000 allow tcp from ${LANIP} to any 25 out via tun1 skipto > > 16010 > > I thought that you had to use skipto as the action, not the rule body: > > ${fwcmd} add 16000 skipto 16010 tcp from ${LANIP} to any 25 out via tun1 Yes. That is correct. However, that will only skip of the rule matches vs. a simple allow statement. How do you match a skipto if you are not allowing traffic, but queueing / denying / forwarding it?? -- Chris. From owner-freebsd-ipfw@FreeBSD.ORG Fri Aug 13 07:39:51 2004 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C230D16A4CE for ; Fri, 13 Aug 2004 07:39:51 +0000 (GMT) Received: from hetzner.co.za (lfw.hetzner.co.za [196.7.18.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3190943D67 for ; Fri, 13 Aug 2004 07:39:51 +0000 (GMT) (envelope-from ianf@hetzner.co.za) Received: from localhost ([127.0.0.1]) by hetzner.co.za with esmtp (Exim 3.36 #1) id 1BvWef-0002eB-00; Fri, 13 Aug 2004 09:39:13 +0200 To: Chris Knipe From: Ian FREISLICH In-Reply-To: Message from "Chris Knipe" <000901c48104$4f0b4f10$fb00a8c0@savage.za.org> Date: Fri, 13 Aug 2004 09:39:13 +0200 Sender: ianf@hetzner.co.za Message-Id: cc: freebsd-ipfw@freebsd.org Subject: Re: ipfw & skipto.... confused a bit... X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Aug 2004 07:39:51 -0000 "Chris Knipe" wrote: > > > ####################################################################### > > > #### Transparent Services ### > > > ####################################################################### > > > ${fwcmd} add 16000 allow tcp from ${LANIP} to any 25 out via tun1 skipto > > > 16010 > > > > I thought that you had to use skipto as the action, not the rule body: > > > > ${fwcmd} add 16000 skipto 16010 tcp from ${LANIP} to any 25 out via tun1 > > Yes. That is correct. However, that will only skip of the rule matches vs. > a simple allow statement. How do you match a skipto if you are not allowing > traffic, but queueing / denying / forwarding it?? I think you're thinking about the skipto rule in the wrong way. It's more of a conditional branch. Here's how I use the skipto with dummynet and NAT: (net.inet.ip.fw.one_pass=0) # Outgoing queues pipe 1 config bw 256Kbits/s queue 10 queue 11 config pipe 1 weight 9 queue 10 queue 12 config pipe 1 weight 1 queue 10 queue 13 config pipe 1 weight 2 queue 10 queue 14 config pipe 1 weight 3 queue 10 queue 15 config pipe 1 weight 1 queue 10 # Incoming queues pipe 2 config bw 256Kbits/s queue 10 queue 21 config pipe 2 weight 9 queue 10 queue 22 config pipe 2 weight 1 queue 10 queue 23 config pipe 2 weight 2 queue 10 queue 24 config pipe 2 weight 3 queue 10 queue 25 config pipe 2 weight 1 queue 10 00600 skipto 700 ip from any to $DMZ out via rl1 00610 queue 11 tcp from any to $OUR_HOSTS dst-port 80,443 out via rl1 00610 skipto 700 tcp from any to $OUR_HOSTS dst-port 80,443 out via rl1 00620 queue 12 { tcp or udp } from any to any dst-port 53 out via rl1 00620 skipto 700 { tcp or udp } from any to any dst-port 53 out via rl1 00630 queue 13 { tcp or udp } from any to any dst-port 143 out via rl1 00630 skipto 700 { tcp or udp } from any to any dst-port 143 out via rl1 00640 queue 14 tcp from any to any dst-port 22 out via rl1 00640 skipto 700 tcp from any to any dst-port 22 out via rl1 00650 queue 15 ip from any to any out via rl1 00700 divert 8668 ip from any to any via rl1 00705 skipto 800 ip from $DMZ to any in via rl1 00710 queue 21 tcp from $OUR_HOSTS 80,443 to any in via rl1 00710 skipto 800 tcp from $OUR_HOSTS 80,443 to any in via rl1 00720 queue 22 { tcp or udp } from any 53 to any in via rl1 00720 skipto 800 { tcp or udp } from any 53 to any in via rl1 00730 queue 23 { tcp or udp } from any 143 to any in via rl1 00730 skipto 800 { tcp or udp } from any 143 to any in via rl1 00740 queue 24 tcp from any 22 to any in via rl1 00740 skipto 800 tcp from any 22 to any in via rl1 00750 queue 25 ip from any to any in via rl1 00850 allow tcp from me to 196.7.162.29 dst-port 9000 Ian -- Ian Freislich From owner-freebsd-ipfw@FreeBSD.ORG Fri Aug 13 07:42:07 2004 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C11D916A4CE for ; Fri, 13 Aug 2004 07:42:07 +0000 (GMT) Received: from ctb-mesg1.saix.net (ctb-mesg1.saix.net [196.25.240.73]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3480F43D2D for ; Fri, 13 Aug 2004 07:42:05 +0000 (GMT) (envelope-from savage@savage.za.org) Received: from netsphere.varynet.co.za (wblv-237-185.telkomadsl.co.za [165.165.237.185]) by ctb-mesg1.saix.net (Postfix) with ESMTP id 93B026540 for ; Fri, 13 Aug 2004 09:41:47 +0200 (SAST) Received: from 192-168-0-251.ops.varynet.co.za ([192.168.0.251] helo=netphobia) by netsphere.varynet.co.za with smtp (Exim 4.34 (FreeBSD)) id 1BvWh8-000NbY-pu for freebsd-ipfw@freebsd.org; Fri, 13 Aug 2004 09:41:47 +0200 Message-ID: <000e01c48109$063bfd20$fb00a8c0@savage.za.org> From: "Chris Knipe" To: References: Date: Fri, 13 Aug 2004 09:42:03 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1437 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 X-Spam-Score: 0.1 (/) X-Spam-Report: 0.1/5.5 Subject: Re: ipfw & skipto.... confused a bit... X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Chris Knipe List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Aug 2004 07:42:07 -0000 > 00710 queue 21 tcp from $OUR_HOSTS 80,443 to any in via rl1 > 00710 skipto 800 tcp from $OUR_HOSTS 80,443 to any in via rl1 *AHA*! Thanks Ian. I never thought about it like this.... -- Chris. From owner-freebsd-ipfw@FreeBSD.ORG Fri Aug 13 08:20:24 2004 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 47EB416A4CE for ; Fri, 13 Aug 2004 08:20:24 +0000 (GMT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 37BDE43D5C for ; Fri, 13 Aug 2004 08:20:24 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.11/8.12.11) with ESMTP id i7D8KOgh062945 for ; Fri, 13 Aug 2004 08:20:24 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.12.11/8.12.11/Submit) id i7D8KN21062944; Fri, 13 Aug 2004 08:20:23 GMT (envelope-from gnats) Date: Fri, 13 Aug 2004 08:20:23 GMT Message-Id: <200408130820.i7D8KN21062944@freefall.freebsd.org> To: ipfw@FreeBSD.org From: Pawel Malachowski Subject: Re: kern/46557: ipfw pipe show fails with lots of queues X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Pawel Malachowski List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Aug 2004 08:20:24 -0000 The following reply was made to PR kern/46557; it has been noted by GNATS. From: Pawel Malachowski To: Pawel Malachowski Cc: freebsd-gnats-submit@FreeBSD.org Subject: Re: kern/46557: ipfw pipe show fails with lots of queues Date: Fri, 13 Aug 2004 10:13:35 +0200 On Thu, Aug 12, 2004 at 07:30:25PM +0000, Pawel Malachowski wrote: > Here are more details what is going on. I hope someone will look at this and > explain why malloc() can fail here and what can be done to prevent this. Well, I will explain myself. First, a patch: *** ip_dummynet.c.orig Fri Aug 13 09:51:54 2004 --- ip_dummynet.c Fri Aug 13 09:52:23 2004 *************** *** 1833,1839 **** for (set = all_flow_sets ; set ; set = set->next ) size += sizeof ( *set ) + set->rq_elements * sizeof(struct dn_flow_queue); ! buf = malloc(size, M_TEMP, M_NOWAIT); if (buf == 0) { splx(s); return ENOBUFS ; --- 1833,1839 ---- for (set = all_flow_sets ; set ; set = set->next ) size += sizeof ( *set ) + set->rq_elements * sizeof(struct dn_flow_queue); ! buf = malloc(size, M_TEMP, M_WAITOK); if (buf == 0) { splx(s); return ENOBUFS ; Since buf can be very big (on my system, up to 3MB), malloc typically consumes only 128k-512k buckets (as checked in vmstat -m). On heavy loaded system with huge number of pipes, there is a risk that first-fit strategy won't find memory because of fragmentation. We will change M_NOWAIT to M_WAITOK flag so malloc() will always succeed; dummynet_get() is called only when superuser runs `ipfw pipe show', so WAITOK is not a problem for us. -- Paweł Małachowski From owner-freebsd-ipfw@FreeBSD.ORG Fri Aug 13 10:08:47 2004 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E35C616A4CE for ; Fri, 13 Aug 2004 10:08:46 +0000 (GMT) Received: from shellma.zin.lublin.pl (shellma.zin.lublin.pl [212.182.126.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id 542CA43D39 for ; Fri, 13 Aug 2004 10:08:46 +0000 (GMT) (envelope-from pawmal-posting@freebsd.lublin.pl) Received: by shellma.zin.lublin.pl (Postfix, from userid 1018) id 044463474C2; Fri, 13 Aug 2004 12:06:19 +0200 (CEST) Date: Fri, 13 Aug 2004 12:06:18 +0200 From: Pawel Malachowski To: Chris Knipe Message-ID: <20040813100618.GE96469@shellma.zin.lublin.pl> References: <000e01c48109$063bfd20$fb00a8c0@savage.za.org> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-2 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <000e01c48109$063bfd20$fb00a8c0@savage.za.org> User-Agent: Mutt/1.4.2i cc: freebsd-ipfw@freebsd.org Subject: Re: ipfw & skipto.... confused a bit... X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Aug 2004 10:08:47 -0000 Almost ~64k rules ruleset is weird. Consider using IPFW2 Lookup Tables for aggregating different types of clients and skiptos for separating traffic on per-direction and per-interface basis. This can greatly increase readability and reduce ruleset size. RECV_NIC1=1000 OMIT=1500 XMIT_NIC1=2000 RECV_NIC2=3000 XMIT_NIC2=4000 ... REST=10000 ipfw table 1 add client1/32 ipfw table 1 add client2/24 ... ipfw table X add clientN/M ipfw add skipto ${RECV_NIC1} ip from any to any in recv ${NIC1} ipfw add skipto ${XMIT_NIC1} ip from any to any out xmit ${NIC1} ipfw add skipto ${RECV_NIC2} ip from any to any in recv ${NIC2} ipfw add skipto ${XMIT_NIC2} ip from any to any out xmit ${NIC2} ... ipfw add skipto ${REST} ip from any to any // All other traffic pass somewhere ipfw add ${RECV_NIC1} count ip from any to any // jump here for RECV_NIC1 ipfw add pipe XXX ip from any to table(X) ipfw add skipto $OMIT ip from any to table(X) ipfw add pipe YYY ip from any to table(Y) ipfw add skipto $OMIT ip from any to table(Y) ... ipfw add ${OMIT} count ip from any to any // Jump here after applying pipes ... ipfw add skipto ${END} ip from any to any // We are done for this interface and direction . . . ipfw add ${REST} count ip from any to any // All other traffic pass here ... ipfw add skipto ${END} ip from any to any // We are done for this interface and direction Very often we apply the same rules on each interface, so we can put these rules in sh(1) function and call a function in shell script that creates ruleset for us, for example: lower_p2p () { # Lower weight of P2P # ${1} = ${PIPE_xxx_xxx_P2P} ${FW} add set ${SET_SHAPE_P2P} queue ${1}10 \ tcp from any ${TCPP2PPORT} to any // Lower weight of incoming P2P traffic ${FW} add set ${SET_SHAPE_P2P} skipto ${HOP} \ tcp from any ${TCPP2PPORT} to any // Jump over not-P2P rule ${FW} add set ${SET_SHAPE_P2P} queue ${1}10 \ tcp from any to any ${TCPP2PPORT} // Lower weight of incoming P2P traffic ${FW} add set ${SET_SHAPE_P2P} skipto ${HOP} \ tcp from any to any ${TCPP2PPORT} // Jump over not-P2P rule ${FW} add set ${SET_SHAPE_P2P} queue ${1}90 \ ip from any to any // Prefer NOT P2P traffic } And call lower_p2p() function like this: ###################################################################### # ISP1 in/down # HOP=${ISP1d} hop Jump here for ISP1 incoming traffic # firewall rules here typical_firewall # Hook for SETs allowing skipping of shaper (local and global). skipshaperhook ${SET_SKIPSHAPE_ISP1} perclient_shaper_in ${SET_SHAPE_ISP1_PERCLIENTLIMIT} ${PIPE_ISP1_DOWN_PERCLIENT} hop Jump here after perclient_in queueing and before P2P-limits lower_p2p ${PIPE_ISP1_DOWN_P2P} <-------------- hop Jump here after P2P-limits before prefere-servers prefer_servers_in longjump # # This is end of ruleset for ISP1 incoming. In this file it looks short but # we have here: # . Typical firewall (filter MS Windows ports) # . Rules (skipto with set) allowing disabling traffic shaping with `ipfw set ${SKIPSHAPINGFORISP1} enable' # . Shaping on per-client basis (different cliens have their own lookup tables and different bw) # . Limiting P2P (we can turn this on/off with `ipfw set ${SET_SHAPE_P2P} enable/disable' # . Rules that increase weight of our hosting servers and reduce workstations # . Jump to the end. # ###################################################################### regards, -- Paweł Małachowski From owner-freebsd-ipfw@FreeBSD.ORG Fri Aug 13 10:47:02 2004 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9B77416A4CE for ; Fri, 13 Aug 2004 10:47:02 +0000 (GMT) Received: from ctb-mesg3.saix.net (ctb-mesg3.saix.net [196.25.240.75]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1EAA043D45 for ; Fri, 13 Aug 2004 10:47:02 +0000 (GMT) (envelope-from savage@savage.za.org) Received: from netsphere.varynet.co.za (wblv-237-185.telkomadsl.co.za [165.165.237.185]) by ctb-mesg3.saix.net (Postfix) with ESMTP id 2AD9139F7 for ; Fri, 13 Aug 2004 12:46:57 +0200 (SAST) Received: from 192-168-0-251.ops.varynet.co.za ([192.168.0.251] helo=netphobia) by netsphere.varynet.co.za with smtp (Exim 4.34 (FreeBSD)) id 1BvZaJ-000NzW-qm for freebsd-ipfw@freebsd.org; Fri, 13 Aug 2004 12:46:56 +0200 Message-ID: <006d01c48122$e41885a0$fb00a8c0@savage.za.org> From: "Chris Knipe" To: References: <000e01c48109$063bfd20$fb00a8c0@savage.za.org> <20040813100618.GE96469@shellma.zin.lublin.pl> Date: Fri, 13 Aug 2004 12:47:12 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1437 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 X-Spam-Score: 0.1 (/) X-Spam-Report: 0.1/5.5 Subject: Re: ipfw & skipto.... confused a bit... X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Chris Knipe List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Aug 2004 10:47:02 -0000 ----- Original Message ----- From: "Pawel Malachowski" To: "Chris Knipe" Cc: Sent: Friday, August 13, 2004 12:06 PM Subject: Re: ipfw & skipto.... confused a bit... > Almost ~64k rules ruleset is weird. It's mainly allot of rules due to per IP and per Port (as well as combinations) used for traffic accounting... So most of it is ipfw count.... The number of rules will therefore also directly depend on the number of hosts on the network, as well as the actual configuration. We're kinda working on a hardware based Layer 7 firewall (using divert sockets) to kill P2P. Obviously, FreeBSD is my desired choice of OS. Traffic accounting and stats is a crucial part of the system. I mean, we must give end-users some nice fancy graphs to look at now, don't we? ;) And yes, I was not quite accurate on my numbers. After closer inspection, I saw that my rule blocks jump from 20000 to 60000 so allot is skipped. 10000-20000 is mainly reserved for accounting, and then 60000 for queues. I have moved this down to lower levels now to make the tables smaller. Thanks for all the replies... It's much appreciated -- Chris.