From owner-freebsd-ipfw@FreeBSD.ORG Sun Jul 20 19:34:31 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7674337B401 for ; Sun, 20 Jul 2003 19:34:31 -0700 (PDT) Received: from ms-smtp-01.tampabay.rr.com (ms-smtp-01.tampabay.rr.com [65.32.1.43]) by mx1.FreeBSD.org (Postfix) with ESMTP id AF22F43FAF for ; Sun, 20 Jul 2003 19:34:30 -0700 (PDT) (envelope-from ipfw@preston.ath.cx) Received: from preston.ath.cx (200.140.8.67.cfl.rr.com [67.8.140.200]) h6L2YTqM006035 for ; Sun, 20 Jul 2003 22:34:29 -0400 (EDT) Received: from kimberly (fabben.kimberly.vasd [10.0.6.30]) by preston.ath.cx (8.12.8p1/8.12.8) with SMTP id h6L2XfEq001996 for ; Sun, 20 Jul 2003 22:33:41 -0400 (EDT) (envelope-from ipfw@preston.ath.cx) Message-ID: <000a01c34f30$8c9d4f10$6401a8c0@kimberly> From: "Preston Connors" To: Date: Sun, 20 Jul 2003 22:34:02 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Subject: ipfw causing mass amounts of delay when piping a large amount of ips. X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2003 02:34:31 -0000 I am trying to implement bandwidth limiting on a large network at a college apartment complex. There are a possible of 700 residents using our Internet connection at one time, with an average of 300 users always connected. Most of them like to abuse P2P services. Allocating upstream and downstream pipes causes mass amounts of delay. There is not a large increase in latency (+10ms), the connections are just very intemittant. It seems that connections time out or are just very delayed. I can't figure out why the pipes won't work correctly. I've tried different queue sizes and nothing stops the delay. Below is the ipfw rules I use. rl0 is our internet interface (209.114.194.138) - 3Mbits upstream and 3Mbits downstream connection over 10baseT/UTP (full duplex) fxp0 is our LAN interface (10.0.0.0/8) - 100baseTX (full duplex) /sbin/sysctl -w net.inet.ip.fw.one_pass=0 /sbin/natd -interface rl0 /sbin/ipfw -q add 65000 divert natd all from any to any via rl0 /sbin/ipfw -q pipe 1000 config mask src-ip 0xffffffff bw 64kbit/s queue 8Kbytes /sbin/ipfw -q add 1000 pipe 1000 all from 10.0.0.0/8 to any /sbin/ipfw -q pipe 65100 config mask dst-ip 0xffffffff bw 128kbit/s queue 8Kbytes /sbin/ipfw -q add 65100 pipe 65100 all from any to 10.0.0.0/8 And here is an ipfw show: 01000 23115 4636964 pipe 1000 ip from 10.0.0.0/8 to any 65000 34258323 19554484874 divert 8668 ip from any to any via rl0 65100 19221 10286845 pipe 65100 ip from any to 10.0.0.0/8 65535 72375096 40894477147 allow ip from any to any Thanks, Preston From owner-freebsd-ipfw@FreeBSD.ORG Sun Jul 20 19:40:27 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ECB3737B401 for ; Sun, 20 Jul 2003 19:40:27 -0700 (PDT) Received: from ms-smtp-03.tampabay.rr.com (ms-smtp-03.tampabay.rr.com [65.32.1.41]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2E2C743FBF for ; Sun, 20 Jul 2003 19:40:27 -0700 (PDT) (envelope-from ipfw@preston.ath.cx) Received: from preston.ath.cx (200.140.8.67.cfl.rr.com [67.8.140.200]) h6L2eQ4U002386 for ; Sun, 20 Jul 2003 22:40:26 -0400 (EDT) Received: from kimberly (fabben.kimberly.vasd [10.0.6.30]) by preston.ath.cx (8.12.8p1/8.12.8) with SMTP id h6L2dcEq002027 for ; Sun, 20 Jul 2003 22:39:38 -0400 (EDT) (envelope-from ipfw@preston.ath.cx) Message-ID: <001f01c34f31$6151d2d0$6401a8c0@kimberly> From: "Preston Connors" To: Date: Sun, 20 Jul 2003 22:39:59 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Subject: ipfw causing mass amounts of delay when piping a large amount of ips. X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2003 02:40:28 -0000 I am trying to implement bandwidth limiting on a large network at a college apartment complex. There are a possible of 700 residents using our Internet connection at one time, with an average of 300 users always connected. Most of them like to abuse P2P services. Allocating upstream and downstream pipes causes mass amounts of delay. There is not a large increase in latency (+10ms), the connections are just very intemittant. It seems that connections time out or are just very delayed. I can't figure out why the pipes won't work correctly. I've tried different queue sizes and nothing stops the delay. Below is the ipfw rules I use. rl0 is our internet interface (209.114.194.138) - 3Mbits upstream and 3Mbits downstream connection over 10baseT/UTP (full duplex) fxp0 is our LAN interface (10.0.0.0/8) - 100baseTX (full duplex) /sbin/sysctl -w net.inet.ip.fw.one_pass=0 /sbin/natd -interface rl0 /sbin/ipfw -q add 65000 divert natd all from any to any via rl0 /sbin/ipfw -q pipe 1000 config mask src-ip 0xffffffff bw 64kbit/s queue 8Kbytes /sbin/ipfw -q add 1000 pipe 1000 all from 10.0.0.0/8 to any /sbin/ipfw -q pipe 65100 config mask dst-ip 0xffffffff bw 128kbit/s queue 8Kbytes /sbin/ipfw -q add 65100 pipe 65100 all from any to 10.0.0.0/8 And here is an ipfw show: 01000 23115 4636964 pipe 1000 ip from 10.0.0.0/8 to any 65000 34258323 19554484874 divert 8668 ip from any to any via rl0 65100 19221 10286845 pipe 65100 ip from any to 10.0.0.0/8 65535 72375096 40894477147 allow ip from any to any Thanks, Preston From owner-freebsd-ipfw@FreeBSD.ORG Sun Jul 20 22:38:17 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 034BF37B401 for ; Sun, 20 Jul 2003 22:38:17 -0700 (PDT) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6332443F3F for ; Sun, 20 Jul 2003 22:38:16 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.8p1/8.12.3) with ESMTP id h6L5cGkN017027; Sun, 20 Jul 2003 22:38:16 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.8p1/8.12.3/Submit) id h6L5cGsT017026; Sun, 20 Jul 2003 22:38:16 -0700 (PDT) (envelope-from rizzo) Date: Sun, 20 Jul 2003 22:38:16 -0700 From: Luigi Rizzo To: Preston Connors Message-ID: <20030720223816.A16984@xorpc.icir.org> References: <001f01c34f31$6151d2d0$6401a8c0@kimberly> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <001f01c34f31$6151d2d0$6401a8c0@kimberly>; from ipfw@preston.ath.cx on Sun, Jul 20, 2003 at 10:39:59PM -0400 cc: freebsd-ipfw@freebsd.org Subject: Re: ipfw causing mass amounts of delay when piping a large amount of ips. X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2003 05:38:17 -0000 i believe you are not matching the right traffic. Out of 34 million diverted packets only 20k go to pipes. Additionally, the delay numbers you mention are a bit weird, even a single 1500 byte packets on a 128kbit/s link consumes 100ms so your 8kbytes queues should show a delay up to .5-1s cheers luigi On Sun, Jul 20, 2003 at 10:39:59PM -0400, Preston Connors wrote: > I am trying to implement bandwidth limiting on a large network at a college > apartment complex. There are a possible of 700 residents using our Internet > connection at one time, with an average of 300 users always connected. Most > of them like to abuse P2P services. Allocating upstream and downstream pipes > causes mass amounts of delay. There is not a large increase in latency > (+10ms), the connections are just very intemittant. It seems that > connections time out or are just very delayed. I can't figure out why the > pipes won't work correctly. I've tried different queue sizes and nothing > stops the delay. Below is the ipfw rules I use. > > rl0 is our internet interface (209.114.194.138) - 3Mbits upstream and 3Mbits > downstream connection over 10baseT/UTP (full duplex) > fxp0 is our LAN interface (10.0.0.0/8) - 100baseTX (full duplex) > > /sbin/sysctl -w net.inet.ip.fw.one_pass=0 > > /sbin/natd -interface rl0 > /sbin/ipfw -q add 65000 divert natd all from any to any via rl0 > > /sbin/ipfw -q pipe 1000 config mask src-ip 0xffffffff bw 64kbit/s queue > 8Kbytes > /sbin/ipfw -q add 1000 pipe 1000 all from 10.0.0.0/8 to any > > /sbin/ipfw -q pipe 65100 config mask dst-ip 0xffffffff bw 128kbit/s queue > 8Kbytes > /sbin/ipfw -q add 65100 pipe 65100 all from any to 10.0.0.0/8 > > And here is an ipfw show: > > 01000 23115 4636964 pipe 1000 ip from 10.0.0.0/8 to any > 65000 34258323 19554484874 divert 8668 ip from any to any via rl0 > 65100 19221 10286845 pipe 65100 ip from any to 10.0.0.0/8 > 65535 72375096 40894477147 allow ip from any to any > > Thanks, > > Preston > > _______________________________________________ > freebsd-ipfw@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw > To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@freebsd.org" From owner-freebsd-ipfw@FreeBSD.ORG Sun Jul 20 23:53:03 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3F4B337B401 for ; Sun, 20 Jul 2003 23:53:03 -0700 (PDT) Received: from s1.routec.net (s1.routec.net [212.1.95.130]) by mx1.FreeBSD.org (Postfix) with ESMTP id BAA3643F3F for ; Sun, 20 Jul 2003 23:53:01 -0700 (PDT) (envelope-from admin@routec.net) Received: from h3n172-31-0.rlan (h3n172-31-0.rlan [172.31.0.3]) by s1.routec.net (Postfix) with ESMTP id BBC4B55DD4B for ; Mon, 21 Jul 2003 09:55:19 +0300 (EEST) Date: Mon, 21 Jul 2003 09:54:57 +0300 From: Ruslan Sulemanov X-Mailer: The Bat! (v1.63 Beta/10) Organization: ISP Routec X-Priority: 3 (Normal) Message-ID: <1003954045.20030721095457@routec.net> To: freebsd-ipfw@freebsd.org MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7bit Subject: rules in ipfw X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: admin List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2003 06:53:03 -0000 Hello all. What mean this rules: add 10 deny udp from any to 255.255.255.255 in recv fxp0 and add 20 deny udp from any to 10.1.255.255 in recv fxp0 ? -- Best regards, Ruslan mailto:admin@routec.net From owner-freebsd-ipfw@FreeBSD.ORG Mon Jul 21 03:42:09 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 66FCB37B401 for ; Mon, 21 Jul 2003 03:42:09 -0700 (PDT) Received: from desktop-guardian.com (desktopguardian.plus.com [81.174.191.17]) by mx1.FreeBSD.org (Postfix) with SMTP id A5EB643F85 for ; Mon, 21 Jul 2003 03:42:07 -0700 (PDT) (envelope-from simong@desktop-guardian.com) Received: (qmail 30273 invoked by uid 1006); 21 Jul 2003 10:43:04 -0000 Received: from simong@desktop-guardian.com by dtg25 by uid 82 with qmail-scanner-1.16 (clamscan: 0.54. spamassassin: 2.55. Clear:. Processed in 2.001997 secs); 21 Jul 2003 10:43:04 -0000 Received: from unknown (HELO dtg17) (192.168.0.17) by 192.168.0.25 with SMTP; 21 Jul 2003 10:43:00 -0000 Message-ID: <012201c34f74$7cc769b0$1100a8c0@dtg17> From: "Simon Gray" To: "admin" , References: <1003954045.20030721095457@routec.net> Date: Mon, 21 Jul 2003 11:40:22 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Subject: Re: rules in ipfw X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Simon Gray List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2003 10:42:09 -0000 > add 10 deny udp from any to 255.255.255.255 in recv fxp0 > > and > > add 20 deny udp from any to 10.1.255.255 in recv fxp0 ? looks like its denying udp broadcasts From owner-freebsd-ipfw@FreeBSD.ORG Mon Jul 21 07:40:17 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2DE3B37B401 for ; Mon, 21 Jul 2003 07:40:17 -0700 (PDT) Received: from octo.sytes.net (h24-86-191-15.ed.shawcable.net [24.86.191.15]) by mx1.FreeBSD.org (Postfix) with ESMTP id E4C5F43F85 for ; Mon, 21 Jul 2003 07:40:10 -0700 (PDT) (envelope-from otacon@octo.sytes.net) Received: from octo.sytes.net (localhost [127.0.0.1]) by octo.sytes.net (8.12.9/8.12.9) with ESMTP id h6LEe2RL076971 for ; Mon, 21 Jul 2003 08:40:04 -0600 (MDT) (envelope-from otacon@octo.sytes.net) Received: by octo.sytes.net (8.12.9/8.12.9/Submit) id h6LEe1kh076960 for freebsd-ipfw@FreeBSD.ORG; Mon, 21 Jul 2003 08:40:01 -0600 (MDT) From: Patrick C To: freebsd-ipfw@FreeBSD.ORG Date: Mon, 21 Jul 2003 08:40:01 -0600 User-Agent: KMail/1.5.2 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200307210840.01042.patrick@filespanker.com> Subject: Not specifying bandwidth for Dummynet X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: patrick@filespanker.com List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2003 14:40:17 -0000 I'm on a cable connection, and the bandwidth is varying at different times of the day. Because of the dynamic speed of this connection, is specifying the bandwidth limits required when creating a dummynet pipe? For example... could I just create the pipe with "ipfw pipe 10 config queue 25k/s" ? From owner-freebsd-ipfw@FreeBSD.ORG Mon Jul 21 09:16:42 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 79B2E37B401 for ; Mon, 21 Jul 2003 09:16:42 -0700 (PDT) Received: from laptop.tenebras.com (laptop.tenebras.com [66.92.188.18]) by mx1.FreeBSD.org (Postfix) with SMTP id D5EC643FBF for ; Mon, 21 Jul 2003 09:16:41 -0700 (PDT) (envelope-from kudzu@tenebras.com) Received: (qmail 12611 invoked from network); 21 Jul 2003 16:16:38 -0000 Received: from sapphire.tenebras.com (HELO tenebras.com) (192.168.188.241) by 0 with SMTP; 21 Jul 2003 16:16:38 -0000 Message-ID: <3F1C11E4.7000309@tenebras.com> Date: Mon, 21 Jul 2003 09:16:36 -0700 From: Michael Sierchio User-Agent: Mozilla/5.0 (X11; U; Linux i386; en-US; rv:1.3.1) Gecko/20030425 X-Accept-Language: en-us, zh-tw, zh-cn, fr, en, de-de MIME-Version: 1.0 To: patrick@filespanker.com References: <200307210840.01042.patrick@filespanker.com> In-Reply-To: <200307210840.01042.patrick@filespanker.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-ipfw@FreeBSD.ORG Subject: Re: Not specifying bandwidth for Dummynet X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2003 16:16:42 -0000 Patrick C wrote: > I'm on a cable connection, and the bandwidth is varying at different times of > the day. Because of the dynamic speed of this connection, is specifying the > bandwidth limits required when creating a dummynet pipe? For example... could > I just create the pipe with "ipfw pipe 10 config queue 25k/s" ? Set the pipe bw to be the maximum you'll ever get. Set the queue length to be something like avg latency * avg bw. You can do more interesting things with queues, but start there. From owner-freebsd-ipfw@FreeBSD.ORG Mon Jul 21 11:01:35 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EC23D37B401 for ; Mon, 21 Jul 2003 11:01:35 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8F9C043FAF for ; Mon, 21 Jul 2003 11:01:35 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h6LI1ZUp058022 for ; Mon, 21 Jul 2003 11:01:35 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h6LI1Ygr058011 for ipfw@freebsd.org; Mon, 21 Jul 2003 11:01:34 -0700 (PDT) Date: Mon, 21 Jul 2003 11:01:34 -0700 (PDT) Message-Id: <200307211801.h6LI1Ygr058011@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: ipfw@FreeBSD.org Subject: Current problem reports assigned to you X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2003 18:01:36 -0000 Current FreeBSD problem reports Critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2003/01/26] kern/47529 ipfw natd/ipfw lose TCP packets for firewalled o [2003/03/23] kern/50216 ipfw kernel panic on 5.0-current when use ipfw 2 problems total. Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2002/12/27] kern/46557 ipfw ipfw pipe show fails with lots of queues o [2003/04/22] kern/51274 ipfw ipfw2 create dynamic rules with parent nu f [2003/04/24] kern/51341 ipfw ipfw rule 'deny icmp from any to any icmp 3 problems total. Non-critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2003/03/12] bin/49959 ipfw ipfw tee port rule skips parsing next rul o [2003/04/09] bin/50749 ipfw ipfw2 incorrectly parses ports and port r a [2001/04/13] kern/26534 ipfw Add an option to ipfw to log gid/uid of w o [2002/12/07] kern/46080 ipfw [PATCH] logamount in ipfw2 does not defau o [2002/12/10] kern/46159 ipfw ipfw dynamic rules lifetime feature o [2002/12/27] kern/46564 ipfw IPFilter and IPFW processing order is not o [2003/02/11] kern/48172 ipfw ipfw does not log size and flags o [2003/03/10] kern/49086 ipfw [patch] Make ipfw2 log to different syslo 8 problems total. From owner-freebsd-ipfw@FreeBSD.ORG Mon Jul 21 12:25:22 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D301637B401 for ; Mon, 21 Jul 2003 12:25:22 -0700 (PDT) Received: from mail.coreps.com (www.coreps.com [207.241.137.100]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0DC7E43F93 for ; Mon, 21 Jul 2003 12:25:22 -0700 (PDT) (envelope-from dhopp@coreps.com) Received: from dennis (dhopp.michix.net [207.241.136.9]) by mail.coreps.com (Postfix) with SMTP id 453BF3F9E for ; Mon, 21 Jul 2003 15:29:23 -0500 (EST) Message-ID: <01ab01c34fbd$d6d01440$0201a8c0@dennis> From: "Dennis B. Hopp" To: Date: Mon, 21 Jul 2003 15:25:26 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Subject: allowing internal machines to traceroute X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2003 19:25:23 -0000 I have setup a freebsd machine to act as a firewall/NAT device. NAT is working fine and the firewall is working but I'm having trouble allowing internal machines to do traceroutes. Pings work fine but traceroutes die at the freebsd machine. My firewall.rules file contains: #stop spoofing add 00010 deny log all from 192.168.1.0/24 to any in via fxp0 # Stop RFC1918 nets on the outside interface add 00020 deny log all from any to 10.0.0.0/8 via fxp0 add 00030 deny log all from any to 172.16.0.0/12 via fxp0 add 00040 deny log all from any to 192.168.0.0/16 via fxp0 add 00100 divert 8668 ip from any to any via fxp0 add 00110 deny log ip from 192.168.1.0/24 to any in recv fxp0 add 00120 deny log ip from 207.241.136.0/24 to any in recv fxp1 #Stop RFC1918 at the outside interface both from being received and being sent: add 00150 deny log ip from 192.168.0.0/16 to any in recv fxp0 add 00150 deny log ip from any to 192.168.0.0/16 out xmit fxp0 add 00150 deny log ip from 172.16.0.0/12 to any in recv fxp0 add 00150 deny log ip from any to 172.16.0.0/12 out xmit fxp0 add 00150 deny log ip from 10.0.0.0/8 to any in recv fxp0 add 00150 deny log ip from any to 10.0.0.0/8 out xmit fxp0 add 00200 check-state add 00201 allow ip from any to any via lo0 add 00202 deny log ip from any to 127.0.0.0/8 add 00203 deny log ip from 127.0.0.0/8 to any add 00215 allow tcp from any to any established add 00216 allow tcp from to any out xmit fxp0 setup add 00217 allow tcp from 192.168.1.0/24 to any in recv fxp1 setup add 00218 allow udp from to any out xmit fxp0 keep-state add 00219 allow udp from 192.168.1.0/24 to any in recv fxp1 keep-state add 00235 allow icmp from 192.168.1.0/24 to any keep-state via fxp1 add 00236 allow icmp from 207.241.136.9 to any keep-state out via fxp0 add 00640 allow tcp from any to any 22 out via fxp0 setup keep-state Any ideas? --Dennis From owner-freebsd-ipfw@FreeBSD.ORG Mon Jul 21 12:34:06 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E354837B407 for ; Mon, 21 Jul 2003 12:34:06 -0700 (PDT) Received: from laptop.tenebras.com (laptop.tenebras.com [66.92.188.18]) by mx1.FreeBSD.org (Postfix) with SMTP id 36F1843FCB for ; Mon, 21 Jul 2003 12:34:05 -0700 (PDT) (envelope-from kudzu@tenebras.com) Received: (qmail 13611 invoked from network); 21 Jul 2003 19:34:04 -0000 Received: from sapphire.tenebras.com (HELO tenebras.com) (192.168.188.241) by 0 with SMTP; 21 Jul 2003 19:34:04 -0000 Message-ID: <3F1C402B.9060908@tenebras.com> Date: Mon, 21 Jul 2003 12:34:03 -0700 From: Michael Sierchio User-Agent: Mozilla/5.0 (X11; U; Linux i386; en-US; rv:1.3.1) Gecko/20030425 X-Accept-Language: en-us, zh-tw, zh-cn, fr, en, de-de MIME-Version: 1.0 To: "Dennis B. Hopp" References: <01ab01c34fbd$d6d01440$0201a8c0@dennis> In-Reply-To: <01ab01c34fbd$d6d01440$0201a8c0@dennis> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-ipfw@freebsd.org Subject: Re: allowing internal machines to traceroute X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2003 19:34:07 -0000 Dennis B. Hopp wrote: > I have setup a freebsd machine to act as a firewall/NAT device. NAT is > working fine and the firewall is working but I'm having trouble allowing > internal machines to do traceroutes. I'm exceeding tired of reading firewall rulesets, so I'll just give you a tutorial on traceroute. Traceroute (by default) sends UDP packets with a TTL of (in succession) 1, 2, ... etc. and generates ICMP error messages in response (TTL exceeded in transit). The assumption is that all the packets take the same route, which might even be true. allow outbound UDP to ports 33434-33599 from your internal hosts allow ICMP type 11 to your internal hosts From owner-freebsd-ipfw@FreeBSD.ORG Mon Jul 21 14:35:49 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 083B537B404 for ; Mon, 21 Jul 2003 14:35:49 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 33D6743FB1 for ; Mon, 21 Jul 2003 14:35:43 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h6LLZhUp003987 for ; Mon, 21 Jul 2003 14:35:43 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h6LLZgFZ003981 for freebsd-ipfw@freebsd.org; Mon, 21 Jul 2003 14:35:42 -0700 (PDT) Date: Mon, 21 Jul 2003 14:35:42 -0700 (PDT) Message-Id: <200307212135.h6LLZgFZ003981@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: freebsd-ipfw@FreeBSD.org Subject: Current problem reports assigned to you X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2003 21:35:49 -0000 Current FreeBSD problem reports Critical problems Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2002/09/02] bin/42318 ipfw NATD redirect limitations 1 problem total. Non-critical problems From owner-freebsd-ipfw@FreeBSD.ORG Mon Jul 21 14:36:02 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C7B5F37B401 for ; Mon, 21 Jul 2003 14:36:02 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3FB6643F85 for ; Mon, 21 Jul 2003 14:36:01 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h6LLa1Up004295 for ; Mon, 21 Jul 2003 14:36:01 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h6LLa08r004289 for ipfw@freebsd.org; Mon, 21 Jul 2003 14:36:00 -0700 (PDT) Date: Mon, 21 Jul 2003 14:36:00 -0700 (PDT) Message-Id: <200307212136.h6LLa08r004289@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: ipfw@FreeBSD.org Subject: Current problem reports assigned to you X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2003 21:36:03 -0000 Current FreeBSD problem reports Critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2003/01/26] kern/47529 ipfw natd/ipfw lose TCP packets for firewalled o [2003/03/23] kern/50216 ipfw kernel panic on 5.0-current when use ipfw 2 problems total. Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2002/12/27] kern/46557 ipfw ipfw pipe show fails with lots of queues o [2003/04/22] kern/51274 ipfw ipfw2 create dynamic rules with parent nu f [2003/04/24] kern/51341 ipfw ipfw rule 'deny icmp from any to any icmp 3 problems total. Non-critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2003/03/12] bin/49959 ipfw ipfw tee port rule skips parsing next rul o [2003/04/09] bin/50749 ipfw ipfw2 incorrectly parses ports and port r a [2001/04/13] kern/26534 ipfw Add an option to ipfw to log gid/uid of w o [2002/12/07] kern/46080 ipfw [PATCH] logamount in ipfw2 does not defau o [2002/12/10] kern/46159 ipfw ipfw dynamic rules lifetime feature o [2002/12/27] kern/46564 ipfw IPFilter and IPFW processing order is not o [2003/02/11] kern/48172 ipfw ipfw does not log size and flags o [2003/03/10] kern/49086 ipfw [patch] Make ipfw2 log to different syslo 8 problems total. From owner-freebsd-ipfw@FreeBSD.ORG Mon Jul 21 14:45:57 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E341737B404 for ; Mon, 21 Jul 2003 14:45:55 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id F2E5143F75 for ; Mon, 21 Jul 2003 14:45:54 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h6LLjsUp008609 for ; Mon, 21 Jul 2003 14:45:54 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h6LLjs0X008603 for freebsd-ipfw@freebsd.org; Mon, 21 Jul 2003 14:45:54 -0700 (PDT) Date: Mon, 21 Jul 2003 14:45:54 -0700 (PDT) Message-Id: <200307212145.h6LLjs0X008603@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: freebsd-ipfw@FreeBSD.org Subject: Current problem reports assigned to you X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2003 21:45:58 -0000 Current FreeBSD problem reports Critical problems Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2002/09/02] bin/42318 ipfw NATD redirect limitations 1 problem total. Non-critical problems From owner-freebsd-ipfw@FreeBSD.ORG Mon Jul 21 14:46:19 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B5F9D37B476 for ; Mon, 21 Jul 2003 14:46:14 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 31C7B43F75 for ; Mon, 21 Jul 2003 14:46:14 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h6LLkEUp009008 for ; Mon, 21 Jul 2003 14:46:14 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h6LLkDjL009002 for ipfw@freebsd.org; Mon, 21 Jul 2003 14:46:13 -0700 (PDT) Date: Mon, 21 Jul 2003 14:46:13 -0700 (PDT) Message-Id: <200307212146.h6LLkDjL009002@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: ipfw@FreeBSD.org Subject: Current problem reports assigned to you X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2003 21:46:19 -0000 Current FreeBSD problem reports Critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2003/01/26] kern/47529 ipfw natd/ipfw lose TCP packets for firewalled o [2003/03/23] kern/50216 ipfw kernel panic on 5.0-current when use ipfw 2 problems total. Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2002/12/27] kern/46557 ipfw ipfw pipe show fails with lots of queues o [2003/04/22] kern/51274 ipfw ipfw2 create dynamic rules with parent nu f [2003/04/24] kern/51341 ipfw ipfw rule 'deny icmp from any to any icmp 3 problems total. Non-critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2003/03/12] bin/49959 ipfw ipfw tee port rule skips parsing next rul o [2003/04/09] bin/50749 ipfw ipfw2 incorrectly parses ports and port r a [2001/04/13] kern/26534 ipfw Add an option to ipfw to log gid/uid of w o [2002/12/07] kern/46080 ipfw [PATCH] logamount in ipfw2 does not defau o [2002/12/10] kern/46159 ipfw ipfw dynamic rules lifetime feature o [2002/12/27] kern/46564 ipfw IPFilter and IPFW processing order is not o [2003/02/11] kern/48172 ipfw ipfw does not log size and flags o [2003/03/10] kern/49086 ipfw [patch] Make ipfw2 log to different syslo 8 problems total. From owner-freebsd-ipfw@FreeBSD.ORG Mon Jul 21 21:35:19 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6E4C137B401 for ; Mon, 21 Jul 2003 21:35:19 -0700 (PDT) Received: from ms-smtp-04.tampabay.rr.com (ms-smtp-04.tampabay.rr.com [65.32.1.35]) by mx1.FreeBSD.org (Postfix) with ESMTP id A1A9243FA3 for ; Mon, 21 Jul 2003 21:35:18 -0700 (PDT) (envelope-from ipfw@preston.ath.cx) Received: from preston.ath.cx (200.140.8.67.cfl.rr.com [67.8.140.200]) h6M4ZHjn012222; Tue, 22 Jul 2003 00:35:17 -0400 (EDT) Received: from kimberly (fabben.kimberly.vasd [10.0.6.30]) by preston.ath.cx (8.12.8p1/8.12.8) with SMTP id h6M4YGMn000445; Tue, 22 Jul 2003 00:34:16 -0400 (EDT) (envelope-from ipfw@preston.ath.cx) Message-ID: <000b01c3500a$966d5af0$6401a8c0@kimberly> From: "Preston Connors" To: "Luigi Rizzo" References: <001f01c34f31$6151d2d0$6401a8c0@kimberly> <20030720223816.A16984@xorpc.icir.org> Date: Tue, 22 Jul 2003 00:34:48 -0400 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 cc: freebsd-ipfw@freebsd.org Subject: Re: ipfw causing mass amounts of delay when piping a large amountof ips. X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jul 2003 04:35:19 -0000 Luigi, > i believe you are not matching the right traffic. Out of > 34 million diverted packets only 20k go to pipes. > I just briefly added the pipe rules, they were added no more than 30 seconds before I did the ipfw show. The 34 million diverted packets has been for a day or so. I just briefly added the pipe rules for testing purposes. > Additionally, the delay numbers you mention are a bit weird, > even a single 1500 byte packets on a 128kbit/s link consumes 100ms > so your 8kbytes queues should show a delay up to .5-1s I have HZ=1000 set in the kernel, could this be why the delay is lower than expected? The pipes still don't seem to work correctly. Do you have any other ideas? Thankyou, Preston ----- Original Message ----- From: "Luigi Rizzo" To: "Preston Connors" Cc: Sent: Monday, July 21, 2003 1:38 AM Subject: Re: ipfw causing mass amounts of delay when piping a large amountof ips. > i believe you are not matching the right traffic. Out of > 34 million diverted packets only 20k go to pipes. > > Additionally, the delay numbers you mention are a bit weird, > even a single 1500 byte packets on a 128kbit/s link consumes 100ms > so your 8kbytes queues should show a delay up to .5-1s > > cheers > luigi > > On Sun, Jul 20, 2003 at 10:39:59PM -0400, Preston Connors wrote: > > I am trying to implement bandwidth limiting on a large network at a college > > apartment complex. There are a possible of 700 residents using our Internet > > connection at one time, with an average of 300 users always connected. Most > > of them like to abuse P2P services. Allocating upstream and downstream pipes > > causes mass amounts of delay. There is not a large increase in latency > > (+10ms), the connections are just very intemittant. It seems that > > connections time out or are just very delayed. I can't figure out why the > > pipes won't work correctly. I've tried different queue sizes and nothing > > stops the delay. Below is the ipfw rules I use. > > > > rl0 is our internet interface (209.114.194.138) - 3Mbits upstream and 3Mbits > > downstream connection over 10baseT/UTP (full duplex) > > fxp0 is our LAN interface (10.0.0.0/8) - 100baseTX (full duplex) > > > > /sbin/sysctl -w net.inet.ip.fw.one_pass=0 > > > > /sbin/natd -interface rl0 > > /sbin/ipfw -q add 65000 divert natd all from any to any via rl0 > > > > /sbin/ipfw -q pipe 1000 config mask src-ip 0xffffffff bw 64kbit/s queue > > 8Kbytes > > /sbin/ipfw -q add 1000 pipe 1000 all from 10.0.0.0/8 to any > > > > /sbin/ipfw -q pipe 65100 config mask dst-ip 0xffffffff bw 128kbit/s queue > > 8Kbytes > > /sbin/ipfw -q add 65100 pipe 65100 all from any to 10.0.0.0/8 > > > > And here is an ipfw show: > > > > 01000 23115 4636964 pipe 1000 ip from 10.0.0.0/8 to any > > 65000 34258323 19554484874 divert 8668 ip from any to any via rl0 > > 65100 19221 10286845 pipe 65100 ip from any to 10.0.0.0/8 > > 65535 72375096 40894477147 allow ip from any to any > > > > Thanks, > > > > Preston > > > > _______________________________________________ > > freebsd-ipfw@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw > > To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-ipfw@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw > To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@freebsd.org" > From owner-freebsd-ipfw@FreeBSD.ORG Mon Jul 21 23:22:33 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9E7F337B401 for ; Mon, 21 Jul 2003 23:22:33 -0700 (PDT) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id E895843FB1 for ; Mon, 21 Jul 2003 23:22:32 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.8p1/8.12.3) with ESMTP id h6M6MWkN022074; Mon, 21 Jul 2003 23:22:32 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.8p1/8.12.3/Submit) id h6M6MW7g022073; Mon, 21 Jul 2003 23:22:32 -0700 (PDT) (envelope-from rizzo) Date: Mon, 21 Jul 2003 23:22:32 -0700 From: Luigi Rizzo To: Preston Connors Message-ID: <20030721232232.B21241@xorpc.icir.org> References: <001f01c34f31$6151d2d0$6401a8c0@kimberly> <20030720223816.A16984@xorpc.icir.org> <000b01c3500a$966d5af0$6401a8c0@kimberly> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <000b01c3500a$966d5af0$6401a8c0@kimberly>; from ipfw@preston.ath.cx on Tue, Jul 22, 2003 at 12:34:48AM -0400 cc: freebsd-ipfw@freebsd.org Subject: Re: ipfw causing mass amounts of delay when piping a large amountof ips. X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jul 2003 06:22:33 -0000 On Tue, Jul 22, 2003 at 12:34:48AM -0400, Preston Connors wrote: > Luigi, > > > i believe you are not matching the right traffic. Out of > > 34 million diverted packets only 20k go to pipes. > > > > I just briefly added the pipe rules, they were added no more than 30 seconds > before I did the ipfw show. The 34 million diverted packets has been for a > day or so. I just briefly added the pipe rules for testing purposes. ok... > > Additionally, the delay numbers you mention are a bit weird, > > even a single 1500 byte packets on a 128kbit/s link consumes 100ms > > so your 8kbytes queues should show a delay up to .5-1s > > I have HZ=1000 set in the kernel, could this be why the delay is lower than > expected? no, it has nothing to do with HZ. How are you measuring the delay ? doing a 'ping' from a machine with no other traffic of course will not show an increase, as the packets will never see a delay, and the 320 bits of the ICMP packet will experience only ~5ms each way which matches your result. if you do an 'ipfw pipe show' then you probably should see massive drops in the pipes, explaining why you are seeing the connection going intermittent. Raising the queue size from your 8KB to something higher (e.g. 10-15 pkts) might help reduce the burstiness. Also put a 'in' option in rule 1000, and 'out' in rule 65100 so you avoid passing traffic through the pipe twice (thus reducing bandwidth and effective buffer size). cheers luigi > The pipes still don't seem to work correctly. Do you have any other ideas? > > Thankyou, > > Preston > > > > > ----- Original Message ----- > From: "Luigi Rizzo" > To: "Preston Connors" > Cc: > Sent: Monday, July 21, 2003 1:38 AM > Subject: Re: ipfw causing mass amounts of delay when piping a large amountof > ips. > > > i believe you are not matching the right traffic. Out of > > 34 million diverted packets only 20k go to pipes. > > > > Additionally, the delay numbers you mention are a bit weird, > > even a single 1500 byte packets on a 128kbit/s link consumes 100ms > > so your 8kbytes queues should show a delay up to .5-1s > > > > cheers > > luigi > > > > On Sun, Jul 20, 2003 at 10:39:59PM -0400, Preston Connors wrote: > > > I am trying to implement bandwidth limiting on a large network at a > college > > > apartment complex. There are a possible of 700 residents using our > Internet > > > connection at one time, with an average of 300 users always connected. > Most > > > of them like to abuse P2P services. Allocating upstream and downstream > pipes > > > causes mass amounts of delay. There is not a large increase in latency > > > (+10ms), the connections are just very intemittant. It seems that > > > connections time out or are just very delayed. I can't figure out why > the > > > pipes won't work correctly. I've tried different queue sizes and nothing > > > stops the delay. Below is the ipfw rules I use. > > > > > > rl0 is our internet interface (209.114.194.138) - 3Mbits upstream and > 3Mbits > > > downstream connection over 10baseT/UTP (full duplex) > > > fxp0 is our LAN interface (10.0.0.0/8) - 100baseTX (full duplex) > > > > > > /sbin/sysctl -w net.inet.ip.fw.one_pass=0 > > > > > > /sbin/natd -interface rl0 > > > /sbin/ipfw -q add 65000 divert natd all from any to any via rl0 > > > > > > /sbin/ipfw -q pipe 1000 config mask src-ip 0xffffffff bw 64kbit/s queue > > > 8Kbytes > > > /sbin/ipfw -q add 1000 pipe 1000 all from 10.0.0.0/8 to any > > > > > > /sbin/ipfw -q pipe 65100 config mask dst-ip 0xffffffff bw 128kbit/s > queue > > > 8Kbytes > > > /sbin/ipfw -q add 65100 pipe 65100 all from any to 10.0.0.0/8 > > > > > > And here is an ipfw show: > > > > > > 01000 23115 4636964 pipe 1000 ip from 10.0.0.0/8 to any > > > 65000 34258323 19554484874 divert 8668 ip from any to any via rl0 > > > 65100 19221 10286845 pipe 65100 ip from any to 10.0.0.0/8 > > > 65535 72375096 40894477147 allow ip from any to any > > > > > > Thanks, > > > > > > Preston > > > > > > _______________________________________________ > > > freebsd-ipfw@freebsd.org mailing list > > > http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw > > > To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@freebsd.org" > > _______________________________________________ > > freebsd-ipfw@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw > > To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@freebsd.org" > > > From owner-freebsd-ipfw@FreeBSD.ORG Tue Jul 22 02:11:55 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9D90C37B401 for ; Tue, 22 Jul 2003 02:11:55 -0700 (PDT) Received: from rdsnet.ro (mail.rdsnet.ro [193.231.236.16]) by mx1.FreeBSD.org (Postfix) with SMTP id C981843FB1 for ; Tue, 22 Jul 2003 02:11:53 -0700 (PDT) (envelope-from itetcu@buh.cameradicommercio.ro) Received: (qmail 1263 invoked from network); 22 Jul 2003 09:11:50 -0000 Received: from unknown (HELO buh.cameradicommercio.ro) (81.196.25.19) by mail.rdsnet.ro with SMTP; 22 Jul 2003 09:11:50 -0000 Received: from buh.cameradicommercio.ro (localhost [127.0.0.1]) h6M9C6D9052093; Tue, 22 Jul 2003 12:12:06 +0300 (EEST) (envelope-from itetcu@buh.cameradicommercio.ro) Received: from localhost (localhost [[UNIX: localhost]]) by buh.cameradicommercio.ro (8.12.9/8.12.9/Submit) id h6M9C5r4052092; Tue, 22 Jul 2003 12:12:05 +0300 (EEST) From: Ion-Mihai Tetcu Organization: Tecnik'93 S.R.L. To: "Dennis B. Hopp" , Date: Tue, 22 Jul 2003 12:12:05 +0300 User-Agent: KMail/1.5.2 References: <01ab01c34fbd$d6d01440$0201a8c0@dennis> In-Reply-To: <01ab01c34fbd$d6d01440$0201a8c0@dennis> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200307221212.05282.itetcu@tecnik93.com> Subject: Re: allowing internal machines to traceroute X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jul 2003 09:11:55 -0000 On Monday 21 July 2003 22:25, Dennis B. Hopp wrote: > I have setup a freebsd machine to act as a firewall/NAT device. NAT > is working fine and the firewall is working but I'm having trouble > allowing internal machines to do traceroutes. # TRACEROUTE - Allow outgoing ${fwcmd} add pass udp from any to any 33434-33523 out via ${oif} # ICMP packets # Allow all ICMP packets on internal interface ${fwcmd} add pass icmp from any to any via ${iif} # Allow outgoing pings ${fwcmd} add pass icmp from any to any icmptypes 8 out via ${oif} ${fwcmd} add pass icmp from any to any icmptypes 0 in via ${oif} From owner-freebsd-ipfw@FreeBSD.ORG Thu Jul 24 13:37:00 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F366037B407; Thu, 24 Jul 2003 13:36:59 -0700 (PDT) Received: from perrin.int.nxad.com (internal.ext.nxad.com [69.1.70.251]) by mx1.FreeBSD.org (Postfix) with ESMTP id 02CDA43FA3; Thu, 24 Jul 2003 13:36:59 -0700 (PDT) (envelope-from sean@nxad.com) Received: by perrin.int.nxad.com (Postfix, from userid 1001) id A12672105A; Thu, 24 Jul 2003 13:36:57 -0700 (PDT) Date: Thu, 24 Jul 2003 13:36:57 -0700 From: Sean Chittenden To: ipfw@FreeBSD.org Message-ID: <20030724203657.GA415@perrin.int.nxad.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="dDRMvlgZJXvWKvBx" Content-Disposition: inline X-PGP-Key: finger seanc@FreeBSD.org X-PGP-Fingerprint: 3849 3760 1AFE 7B17 11A0 83A6 DD99 E31F BC84 B341 X-Web-Homepage: http://sean.chittenden.org/ User-Agent: Mutt/1.5.4i cc: luigi@freebsd.org Subject: Dynamic rules not being matched after divert... X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2003 20:37:00 -0000 --dDRMvlgZJXvWKvBx Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I'm setting up an ipfw2+natd gateway and am pretty convinced there's a bug in the way that ipfw2 promotes dynamic rules to being fully established. Here's the rule set that I'm using: ##### BEGIN SCRIPT fwcmd=3D"/sbin/ipfw" # Inside interface iif=3D"sis0" # Outside interface oif=3D"sis1" $fwcmd -f flush # Basic anti-spoofing $fwcmd add set 1 deny log all from any to 0.0.0.0/8,10.0.0.0/8,169.254.0.0/= 16,192.0.2.0/24,172.16.0.0/12,192.168.0.0/16,224.0.0.0/4,240.0.0.0/4 via ${= oif} # Allow connections from the internal network to be NAT'ed $fwcmd add set 1 divert natd all from any to me in recv ${oif} $fwcmd add set 1 divert natd all from not me to any recv ${iif} out xmit ${= oif} # Finish the last of the basic anti-spoofing now that packets have been NAT= 'ed $fwcmd add set 1 deny log all from 0.0.0.0/8,10.0.0.0/8,169.254.0.0/16,192.= 0.2.0/24,172.16.0.0/12,192.168.0.0/16,224.0.0.0/4,240.0.0.0/4 to any via ${= oif} # Allow all connections that have dynamic rules built for them, # but deny established connections that don't have a dynamic rule. $fwcmd add set 1 check-state $fwcmd add set 1 deny log tcp from any to any established $fwcmd add set 1 deny log ip from any to any in frag $fwcmd add set 1 deny log ip from any to any not verrevpath in # Allow all localhost connections $fwcmd add set 1 allow tcp from me to any out via lo0 setup keep-state $fwcmd add set 1 deny log tcp from me to any out via lo0 $fwcmd add set 1 allow ip from me to any out via lo0 keep-state # Allow all connections from the internal network $fwcmd add set 1 allow tcp from 192.168.1.0/24 to any in via ${iif} setup k= eep-state $fwcmd add set 1 deny log tcp from 192.168.1.0/24 to any $fwcmd add set 1 allow ip from 192.168.1.0/24 to any in via ${iif} keep-st= ate $fwcmd add set 1 allow ip from any to any in via ${iif} keep-state # Allow all connections from my network card that I initiate $fwcmd add set 1 allow tcp from me to any out xmit any setup keep-state $fwcmd add set 1 deny log tcp from me to any $fwcmd add set 1 allow ip from me to any out xmit any keep-state # Enable ICMP to work since it is not a stateful protocol $fwcmd add set 1 allow icmp from any to any icmptypes 0,3,8,11,12,13,14 # This sends a RESET to all ident packets. $fwcmd add set 1 reset log tcp from any to me 113 in recv any # Deny all the rest. $fwcmd add set 1 deny log ip from any to any ##### END SCRIPT Packets can travel across the firewall correctly, however the dynamic rule for TCP connections made from the Intranet to the Internet, aren't correct. For example: laptop% ssh host.example.com 'sleep 30 && exit' & # sudo ipfw -d show [snip] 01800 14 2929 (13s) STATE tcp 64.x.y.z 49678 <-> 69.u.v.w 22 The src address has been translated correctly and I can use SSH, however, because the connection hasn't been promoted to a fully established state, if I stop typing and let the connection idle for more than 20s, the dynamic rule cleans up and I have to re-establish a connection. If I hop onto the gateway directly and issue the same ssh command, things work correctly: fw% ssh host.example.com 'sleep 30 && exit' & # sudo ipfw -d show [snip] 01800 30 4605 (297s) STATE tcp 64.x.y.z 49506 <-> 69.u.v.w 22 =46rom what I can tell, the check-state rule is failing to match rules that have been divert'ed/nat'ed, which seems exceedingly broken given the only thing that the check-state rule needs to do is match the src ip, src port, dst ip, and dst port. In case someone asks, yes, the connection is setup correctly according to tcpdump: # tcpdump -n -i sis1 tcp tcpdump: listening on sis1 13:27:43.955298 64.x.y.z.49680 > 69.u.v.w.22: S 311390851:311390851(0) win = 65535 (DF) 13:27:43.995120 69.u.v.w.22 > 64.x.y.z.49680: S 73338311:73338311(0) ack 31= 1390852 win 32768 (DF) 13:27:43.997839 64.x.y.z.49680 > 69.u.v.w.22: . ack 1 win 65535 (DF) 13:27:44.043933 69.u.v.w.22 > 64.x.y.z.49680: P 1:41(40) ack 1 win 33580 (D= F) 13:27:44.047096 64.x.y.z.49680 > 69.u.v.w.22: P 1:42(41) ack 41 win 65535 (= DF) 13:27:44.106271 69.u.v.w.22 > 64.x.y.z.49680: P 41:577(536) ack 42 win 3358= 0 (DF) 13:27:44.111365 64.x.y.z.49680 > 69.u.v.w.22: P 42:586(544) ack 577 win 651= 64 (DF) 13:27:44.288589 69.u.v.w.22 > 64.x.y.z.49680: . ack 586 win 33580 (DF) 13:27:44.291236 64.x.y.z.49680 > 69.u.v.w.22: P 586:610(24) ack 577 win 655= 35 (DF) 13:27:44.346961 69.u.v.w.22 > 64.x.y.z.49680: P 577:857(280) ack 610 win 33= 580 (DF) 13:27:44.404742 64.x.y.z.49680 > 69.u.v.w.22: P 610:882(272) ack 857 win 65= 535 (DF) 13:27:44.519718 69.u.v.w.22 > 64.x.y.z.49680: P 857:1641(784) ack 882 win 3= 3580 (DF) 13:27:44.606056 64.x.y.z.49680 > 69.u.v.w.22: P 882:898(16) ack 1641 win 65= 535 (DF) 13:27:44.737744 69.u.v.w.22 > 64.x.y.z.49680: . ack 898 win 33580 (DF) [snip] 13:27:45.120273 64.x.y.z.49680 > 69.u.v.w.22: P 2178:2290(112) ack 2329 win= 65535 (DF) 13:27:45.120804 64.x.y.z.49680 > 69.u.v.w.22: P 2290:2354(64) ack 2329 win = 65535 (DF) [tos 0x10] 13:27:45.178022 69.u.v.w.22 > 64.x.y.z.49680: . ack 2354 win 33516 (DF) 13:27:45.181483 69.u.v.w.22 > 64.x.y.z.49680: P 2329:2377(48) ack 2354 win = 33580 (DF) 13:27:45.278207 64.x.y.z.49680 > 69.u.v.w.22: . ack 2377 win 65535 (DF) [to= s 0x10] 13:27:47.651804 64.x.y.z.49680 > 69.u.v.w.22: F 2354:2354(0) ack 2377 win 6= 5535 (DF) [tos 0x10] 13:27:47.691808 69.u.v.w.22 > 64.x.y.z.49680: . ack 2355 win 33580 (DF) 13:27:47.694033 69.u.v.w.22 > 64.x.y.z.49680: F 2377:2377(0) ack 2355 win 3= 3580 (DF) 13:27:47.696684 64.x.y.z.49680 > 69.u.v.w.22: . ack 2378 win 65534 [tos 0x1= 0] 13:27:47.728191 64.x.y.z.49680 > 69.u.v.w.22: . ack 2377 win 0 13:27:47.729124 64.x.y.z.49680 > 69.u.v.w.22: . ack 2378 win 65534 [tos 0x1= 0] 13:27:47.768700 69.u.v.w.22 > 64.x.y.z.49680: R 73340688:73340688(0) win 0 13:27:47.775331 69.u.v.w.22 > 64.x.y.z.49680: R 73340689:73340689(0) win 0 I've stuck 'count log // test' rules in and have verified that everything is correct coming across the wire/kernel, however ipfw2's just not detecting that the connection has been fully established. If anyone would like, please feel free to use the above rules and test them out, they should be quite good and a fair bit more secure than what's suggested currently (the above works for surfing the web/pop3, but is infuriating if you use ssh), though they're not quite working right yet. For a while I thought it had to do with net.inet.ip.fw.one_pass, but toggling that has no bearing on the outcome at all. Comments/suggestions? -sc --=20 Sean Chittenden --dDRMvlgZJXvWKvBx Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Comment: Sean Chittenden iD8DBQE/IENp3ZnjH7yEs0ERAtMuAKCYom3a8c4gUdqEQ3Simeb++BiHKQCeKF2B GslVxmyUL9kIcFVk6WVEYZU= =2I5d -----END PGP SIGNATURE----- --dDRMvlgZJXvWKvBx-- From owner-freebsd-ipfw@FreeBSD.ORG Thu Jul 24 13:50:45 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 22BBE37B401 for ; Thu, 24 Jul 2003 13:50:44 -0700 (PDT) Received: from laptop.tenebras.com (laptop.tenebras.com [66.92.188.18]) by mx1.FreeBSD.org (Postfix) with SMTP id 598BA43FA3 for ; Thu, 24 Jul 2003 13:50:43 -0700 (PDT) (envelope-from kudzu@tenebras.com) Received: (qmail 28551 invoked from network); 24 Jul 2003 20:50:42 -0000 Received: from sapphire.tenebras.com (HELO tenebras.com) (192.168.188.241) by 0 with SMTP; 24 Jul 2003 20:50:42 -0000 Message-ID: <3F2046A1.7070807@tenebras.com> Date: Thu, 24 Jul 2003 13:50:41 -0700 From: Michael Sierchio User-Agent: Mozilla/5.0 (X11; U; Linux i386; en-US; rv:1.3.1) Gecko/20030425 X-Accept-Language: en-us, zh-tw, zh-cn, fr, en, de-de MIME-Version: 1.0 To: Sean Chittenden References: <20030724203657.GA415@perrin.int.nxad.com> In-Reply-To: <20030724203657.GA415@perrin.int.nxad.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: luigi@freebsd.org cc: ipfw@FreeBSD.org Subject: Re: Dynamic rules not being matched after divert... X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2003 20:50:45 -0000 Sean Chittenden wrote: > I'm setting up an ipfw2+natd gateway and am pretty convinced there's a > bug in the way that ipfw2 promotes dynamic rules to being fully > established. I and others have said similar things, but we were simply wrong. The problem is that natd is already a stateful bugger, and when packets match a stateful rule in one direction (after natting, say) they cannot match the rule in the other direction -- addresses won't match. In one case you have the private address, in the other, the public address. This has been discussed before. I'm working on new examples for rc.firewall.... From owner-freebsd-ipfw@FreeBSD.ORG Thu Jul 24 14:08:04 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E66B337B405; Thu, 24 Jul 2003 14:08:04 -0700 (PDT) Received: from perrin.int.nxad.com (internal.ext.nxad.com [69.1.70.251]) by mx1.FreeBSD.org (Postfix) with ESMTP id B04EA43FAF; Thu, 24 Jul 2003 14:08:03 -0700 (PDT) (envelope-from sean@nxad.com) Received: by perrin.int.nxad.com (Postfix, from userid 1001) id 776D62105C; Thu, 24 Jul 2003 14:08:02 -0700 (PDT) Date: Thu, 24 Jul 2003 14:08:02 -0700 From: Sean Chittenden To: Michael Sierchio Message-ID: <20030724210802.GA1904@perrin.int.nxad.com> References: <20030724203657.GA415@perrin.int.nxad.com> <3F2046A1.7070807@tenebras.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3F2046A1.7070807@tenebras.com> X-PGP-Key: finger seanc@FreeBSD.org X-PGP-Fingerprint: 3849 3760 1AFE 7B17 11A0 83A6 DD99 E31F BC84 B341 X-Web-Homepage: http://sean.chittenden.org/ User-Agent: Mutt/1.5.4i cc: luigi@freebsd.org cc: ipfw@FreeBSD.org Subject: Re: Dynamic rules not being matched after divert... X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2003 21:08:05 -0000 > >I'm setting up an ipfw2+natd gateway and am pretty convinced there's a > >bug in the way that ipfw2 promotes dynamic rules to being fully > >established. > > I and others have said similar things, but we were simply wrong. > The problem is that natd is already a stateful bugger, Very aware that natd is stateful, that's how it does its translation mappings. This would be fine with me if divert actually aborted processing of instructions after the divert rule and natd had some way for me to see its state table. Since it doesn't, however, and since the packet would match a state entry in ipfw2 if it were given the chance to, I consider this a bug (and not a feature as I read in the archives). > and when packets match a stateful rule in one direction (after > natting, say) they cannot match the rule in the other direction -- > addresses won't match. In one case you have the private address, in > the other, the public address. That's irrelevant: the packet has been translated when traversing the firewall in either direction. Packets even go so far as to update the TTL count down timer on the existing dynamic rule for the connection. > This has been discussed before. I'm working on new examples for > rc.firewall.... I read 'em: the discussions was half baked, at best and were far from conclusive in terms of a secure rule set using ipfw2+natd. Allowing established connections is not secure. I like that ipfw2 sends tcp keep alives, something I'm almost certain that natd(8) doesn't do. Luigi, ipfw(8) is incorrect, btw: divert port Divert packets that match this rule to the divert(4) socket bound to port port. The search terminates. Packets that have gone through natd(8) are being re-injected into the rule set. From ip_fw2.c ~line 1256: * args->divert_rule (in/out) * Skip up to the first rule past this rule number; * upon return, non-zero port number for divert or tee. This is consistent with what is found in natd(8): After translation by natd, packets re-enter the firewall at the rule number following the rule number that caused the diversion (not the next rule if there are several at the same number). Let me know if it's okay if I commit a doc correction for the above. -sc -- Sean Chittenden From owner-freebsd-ipfw@FreeBSD.ORG Thu Jul 24 15:53:21 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C306637B408; Thu, 24 Jul 2003 15:53:21 -0700 (PDT) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8C19443FBF; Thu, 24 Jul 2003 15:53:17 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.8p1/8.12.3) with ESMTP id h6OMrHkN036143; Thu, 24 Jul 2003 15:53:17 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.8p1/8.12.3/Submit) id h6OMrHNk036142; Thu, 24 Jul 2003 15:53:17 -0700 (PDT) (envelope-from rizzo) Date: Thu, 24 Jul 2003 15:53:17 -0700 From: Luigi Rizzo To: Sean Chittenden Message-ID: <20030724155317.A35706@xorpc.icir.org> References: <20030724203657.GA415@perrin.int.nxad.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <20030724203657.GA415@perrin.int.nxad.com>; from seanc@FreeBSD.org on Thu, Jul 24, 2003 at 01:36:57PM -0700 cc: ipfw@FreeBSD.org Subject: Re: Dynamic rules not being matched after divert... X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2003 22:53:22 -0000 On Thu, Jul 24, 2003 at 01:36:57PM -0700, Sean Chittenden wrote: > I'm setting up an ipfw2+natd gateway and am pretty convinced there's a > bug in the way that ipfw2 promotes dynamic rules to being fully > established. Thanks for the detailed report, but I am afraid to say that ipfw2 works correctly, it's your ruleset which is buggy. What happens is that if you run the session from the laptop, you have the following packets: #1 in sis0 SYN laptop->host.example.com this matches rule 01200, creates a stateful entry and is accepted. The IP layer decides it needs to be forwarded and the firewall is invoked again by ip_output() with packet #1. This matches rule 00300, and is reinjected at 00400 as #2: #2 in-sis0 out sis1 SYN fw->host.example.com this matches rule 01800, creates a stateful entry and is accepted. The SYN/ACK coming back is #3: #3 in sis1 SYN-ACK host.example.com->fw matches rule 200, gets diverted and reinjected at 300 as #4: #4 in sis1 SYN-ACK host.example.com->laptop which matches dynamic rule 01200 and is accepted. >From now on, forward packets (laptop->host.example.com) will match rule 00500 (dynamic rule 01200) in the way in, go to ip_output(), match rule 00300, get translated into (fw->host.example.com), and match rule 00500 (dynamic rule 01800). Return packets (host.example.com->fw) match rule 00200, get translated into (host.example.com->laptop) and match rule 00500 (dynamic rule 01200) on the way in, and then again rule 00500 (dynamic rule 01200) on the way out. Note that RETURN PACKETS NEVER MATCH DYNAMIC RULE 01800! The counts you see only refer to forward packets. The situation is different for locally generated packets because they are not translated, so return packets will match. In order to fix your ruleset I would suggest to put the keep-state into a 'skipto' action so that even after a check-state you will have a chance for further processing of the packet. But it is extremely tricky to get it right. cheers luigi ----- (your rules numbered, for reference) ---- # Basic anti-spoofing 00100 set 1 deny log all from any to 0.0.0.0/8,... via ${oif} # Allow connections from the internal network to be NAT'ed 00200 set 1 divert natd all from any to me in recv ${oif} 00300 set 1 divert natd all from not me to any recv ${iif} out xmit ${oif} # Finish the last of the basic anti-spoofing now that packets have been NAT'ed 00400 set 1 deny log all from 0.0.0.0/8,... to any via ${oif} # Allow all connections that have dynamic rules built for them, # but deny established connections that don't have a dynamic rule. 00500 set 1 check-state 00600 set 1 deny log tcp from any to any established 00700 set 1 deny log ip from any to any in frag 00800 set 1 deny log ip from any to any not verrevpath in # Allow all localhost connections 00900 set 1 allow tcp from me to any out via lo0 setup keep-state 01000 set 1 deny log tcp from me to any out via lo0 01100 set 1 allow ip from me to any out via lo0 keep-state # Allow all connections from the internal network 01200 set 1 allow tcp from 192.168.1.0/24 to any in via ${iif} setup keep-state 01300 set 1 deny log tcp from 192.168.1.0/24 to any 01400 set 1 allow ip from 192.168.1.0/24 to any in via ${iif} keep-state 01500 set 1 allow ip from any to any in via ${iif} keep-state # Allow all connections from my network card that I initiate 01600 set 1 allow tcp from me to any out xmit any setup keep-state 01700 set 1 deny log tcp from me to any 01800 set 1 allow ip from me to any out xmit any keep-state # Enable ICMP to work since it is not a stateful protocol 01900 set 1 allow icmp from any to any icmptypes 0,3,8,11,12,13,14 # This sends a RESET to all ident packets. 02000 set 1 reset log tcp from any to me 113 in recv any # Deny all the rest. 02100 set 1 deny log ip from any to any ##### END SCRIPT > Here's the rule set that I'm using: > > ##### BEGIN SCRIPT > fwcmd="/sbin/ipfw" > > # Inside interface > iif="sis0" > > # Outside interface > oif="sis1" > > $fwcmd -f flush > > # Basic anti-spoofing > $fwcmd add set 1 deny log all from any to 0.0.0.0/8,10.0.0.0/8,169.254.0.0/16,192.0.2.0/24,172.16.0.0/12,192.168.0.0/16,224.0.0.0/4,240.0.0.0/4 via ${oif} > > # Allow connections from the internal network to be NAT'ed > $fwcmd add set 1 divert natd all from any to me in recv ${oif} > $fwcmd add set 1 divert natd all from not me to any recv ${iif} out xmit ${oif} > > # Finish the last of the basic anti-spoofing now that packets have been NAT'ed > $fwcmd add set 1 deny log all from 0.0.0.0/8,10.0.0.0/8,169.254.0.0/16,192.0.2.0/24,172.16.0.0/12,192.168.0.0/16,224.0.0.0/4,240.0.0.0/4 to any via ${oif} > > # Allow all connections that have dynamic rules built for them, > # but deny established connections that don't have a dynamic rule. > $fwcmd add set 1 check-state > $fwcmd add set 1 deny log tcp from any to any established > $fwcmd add set 1 deny log ip from any to any in frag > $fwcmd add set 1 deny log ip from any to any not verrevpath in > > # Allow all localhost connections > $fwcmd add set 1 allow tcp from me to any out via lo0 setup keep-state > $fwcmd add set 1 deny log tcp from me to any out via lo0 > $fwcmd add set 1 allow ip from me to any out via lo0 keep-state > > # Allow all connections from the internal network > $fwcmd add set 1 allow tcp from 192.168.1.0/24 to any in via ${iif} setup keep-state > $fwcmd add set 1 deny log tcp from 192.168.1.0/24 to any > $fwcmd add set 1 allow ip from 192.168.1.0/24 to any in via ${iif} keep-state > $fwcmd add set 1 allow ip from any to any in via ${iif} keep-state > > # Allow all connections from my network card that I initiate > $fwcmd add set 1 allow tcp from me to any out xmit any setup keep-state > $fwcmd add set 1 deny log tcp from me to any > $fwcmd add set 1 allow ip from me to any out xmit any keep-state > > # Enable ICMP to work since it is not a stateful protocol > $fwcmd add set 1 allow icmp from any to any icmptypes 0,3,8,11,12,13,14 > > # This sends a RESET to all ident packets. > $fwcmd add set 1 reset log tcp from any to me 113 in recv any > > # Deny all the rest. > $fwcmd add set 1 deny log ip from any to any > ##### END SCRIPT > > Packets can travel across the firewall correctly, however the dynamic > rule for TCP connections made from the Intranet to the Internet, > aren't correct. For example: > > laptop% ssh host.example.com 'sleep 30 && exit' & > # sudo ipfw -d show > [snip] > 01800 14 2929 (13s) STATE tcp 64.x.y.z 49678 <-> 69.u.v.w 22 > > The src address has been translated correctly and I can use SSH, > however, because the connection hasn't been promoted to a fully > established state, if I stop typing and let the connection idle for > more than 20s, the dynamic rule cleans up and I have to re-establish a > connection. If I hop onto the gateway directly and issue the same ssh > command, things work correctly: > > fw% ssh host.example.com 'sleep 30 && exit' & > # sudo ipfw -d show > [snip] > 01800 30 4605 (297s) STATE tcp 64.x.y.z 49506 <-> 69.u.v.w 22 > > From what I can tell, the check-state rule is failing to match rules > that have been divert'ed/nat'ed, which seems exceedingly broken given > the only thing that the check-state rule needs to do is match the src > ip, src port, dst ip, and dst port. In case someone asks, yes, the > connection is setup correctly according to tcpdump: > > # tcpdump -n -i sis1 tcp > tcpdump: listening on sis1 > 13:27:43.955298 64.x.y.z.49680 > 69.u.v.w.22: S 311390851:311390851(0) win 65535 (DF) > 13:27:43.995120 69.u.v.w.22 > 64.x.y.z.49680: S 73338311:73338311(0) ack 311390852 win 32768 (DF) > 13:27:43.997839 64.x.y.z.49680 > 69.u.v.w.22: . ack 1 win 65535 (DF) > 13:27:44.043933 69.u.v.w.22 > 64.x.y.z.49680: P 1:41(40) ack 1 win 33580 (DF) > 13:27:44.047096 64.x.y.z.49680 > 69.u.v.w.22: P 1:42(41) ack 41 win 65535 (DF) > 13:27:44.106271 69.u.v.w.22 > 64.x.y.z.49680: P 41:577(536) ack 42 win 33580 (DF) > 13:27:44.111365 64.x.y.z.49680 > 69.u.v.w.22: P 42:586(544) ack 577 win 65164 (DF) > 13:27:44.288589 69.u.v.w.22 > 64.x.y.z.49680: . ack 586 win 33580 (DF) > 13:27:44.291236 64.x.y.z.49680 > 69.u.v.w.22: P 586:610(24) ack 577 win 65535 (DF) > 13:27:44.346961 69.u.v.w.22 > 64.x.y.z.49680: P 577:857(280) ack 610 win 33580 (DF) > 13:27:44.404742 64.x.y.z.49680 > 69.u.v.w.22: P 610:882(272) ack 857 win 65535 (DF) > 13:27:44.519718 69.u.v.w.22 > 64.x.y.z.49680: P 857:1641(784) ack 882 win 33580 (DF) > 13:27:44.606056 64.x.y.z.49680 > 69.u.v.w.22: P 882:898(16) ack 1641 win 65535 (DF) > 13:27:44.737744 69.u.v.w.22 > 64.x.y.z.49680: . ack 898 win 33580 (DF) > [snip] > 13:27:45.120273 64.x.y.z.49680 > 69.u.v.w.22: P 2178:2290(112) ack 2329 win 65535 (DF) > 13:27:45.120804 64.x.y.z.49680 > 69.u.v.w.22: P 2290:2354(64) ack 2329 win 65535 (DF) [tos 0x10] > 13:27:45.178022 69.u.v.w.22 > 64.x.y.z.49680: . ack 2354 win 33516 (DF) > 13:27:45.181483 69.u.v.w.22 > 64.x.y.z.49680: P 2329:2377(48) ack 2354 win 33580 (DF) > 13:27:45.278207 64.x.y.z.49680 > 69.u.v.w.22: . ack 2377 win 65535 (DF) [tos 0x10] > 13:27:47.651804 64.x.y.z.49680 > 69.u.v.w.22: F 2354:2354(0) ack 2377 win 65535 (DF) [tos 0x10] > 13:27:47.691808 69.u.v.w.22 > 64.x.y.z.49680: . ack 2355 win 33580 (DF) > 13:27:47.694033 69.u.v.w.22 > 64.x.y.z.49680: F 2377:2377(0) ack 2355 win 33580 (DF) > 13:27:47.696684 64.x.y.z.49680 > 69.u.v.w.22: . ack 2378 win 65534 [tos 0x10] > 13:27:47.728191 64.x.y.z.49680 > 69.u.v.w.22: . ack 2377 win 0 > 13:27:47.729124 64.x.y.z.49680 > 69.u.v.w.22: . ack 2378 win 65534 [tos 0x10] > 13:27:47.768700 69.u.v.w.22 > 64.x.y.z.49680: R 73340688:73340688(0) win 0 > 13:27:47.775331 69.u.v.w.22 > 64.x.y.z.49680: R 73340689:73340689(0) win 0 > > > I've stuck 'count log // test' rules in and have verified that > everything is correct coming across the wire/kernel, however ipfw2's > just not detecting that the connection has been fully established. If > anyone would like, please feel free to use the above rules and test > them out, they should be quite good and a fair bit more secure than > what's suggested currently (the above works for surfing the web/pop3, > but is infuriating if you use ssh), though they're not quite working > right yet. > > For a while I thought it had to do with net.inet.ip.fw.one_pass, but > toggling that has no bearing on the outcome at all. > Comments/suggestions? -sc > > -- > Sean Chittenden From owner-freebsd-ipfw@FreeBSD.ORG Fri Jul 25 06:09:09 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5F18637B401 for ; Fri, 25 Jul 2003 06:09:09 -0700 (PDT) Received: from mail012.syd.optusnet.com.au (mail012.syd.optusnet.com.au [210.49.20.170]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8897F43F85 for ; Fri, 25 Jul 2003 06:09:07 -0700 (PDT) (envelope-from markhannon@optusnet.com.au) Received: from doorway.homeip.net (c211-28-121-120.sunsh3.vic.optusnet.com.au [211.28.121.120])h6PD95o26713 for ; Fri, 25 Jul 2003 23:09:06 +1000 Received: from optusnet.com.au (tbird.home.lan [192.168.1.5]) by doorway.homeip.net (8.12.9/8.12.9) with ESMTP id h6PD9686010564 for ; Fri, 25 Jul 2003 23:09:06 +1000 (EST) (envelope-from markhannon@optusnet.com.au) Message-ID: <3F212BF7.4060602@optusnet.com.au> Date: Fri, 25 Jul 2003 23:09:11 +1000 From: Mark Hannon User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.4) Gecko/20030719 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-ipfw@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: using dummynet to simulate modem, dsl, etc X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2003 13:09:09 -0000 Hello, I am trying to use dummynet to simulate different network links and demonstrate application level performance for a variety of scenarios. The first test is three ftp file transfers over a simulated 56k modem as per below: iifconfig lo0 mtu 1500 fconfig lo0 alias 127.0.0.100 ipfw -q add 1 pipe 1 ip from any to 127.0.0.100 ipfw -q add 2 pipe 2 ip from 127.0.0.100 to any ipfw -q pipe 1 config bw 33kbit/s queue 0 delay 40ms ipfw -q pipe 2 config bw 56kbit/s queue 0 delay 112ms The results seem strange ... larger file sizes seem to have a slower transfer rate, where I would have thought that steady-state transfers would have been reached for the larger files. Are these results logical? Have I done something silly with the pipe configs? Receiving test_large (1937201 bytes): 100% (ETA 00:00) 1937201 bytes transferred in 551.1 seconds (3.43 kBps) Receiving test_medium (248476 bytes): 100% (ETA 00:00) 248476 bytes transferred in 66.7 seconds (3.64 kBps) Receiving test_small (40960 bytes): 100% 40960 bytes transferred in 7.3 seconds (5.52 kBps) Regards/mark From owner-freebsd-ipfw@FreeBSD.ORG Fri Jul 25 07:57:50 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8DAD137B401 for ; Fri, 25 Jul 2003 07:57:49 -0700 (PDT) Received: from mta10.srv.hcvlny.cv.net (mta10.srv.hcvlny.cv.net [167.206.5.45]) by mx1.FreeBSD.org (Postfix) with ESMTP id C423343FA3 for ; Fri, 25 Jul 2003 07:57:48 -0700 (PDT) (envelope-from agapon@cv-nj.com) Received: from asv10.srv.hcvlny.cv.net (asv10.srv.hcvlny.cv.net [167.206.5.38]) by mta10.srv.hcvlny.cv.net (iPlanet Messaging Server 5.2 HotFix 1.16 (built May 14 2003)) ipfw@FreeBSD.org; Fri, 25 Jul 2003 10:57:48 -0400 (EDT) Received: from edge.foundation.invalidasv10.srv.hcvlny.cv.net (8.12.9/8.12.9) with ESMTP id h6PEvhbc001934 for ; Fri, 25 Jul 2003 10:57:44 -0400 (EDT) Received: from cv-nj.com (localhost.foundation.invalid [127.0.0.1]) h6PEvhh6022362 for ; Fri, 25 Jul 2003 10:57:43 -0400 (EDT envelope-from agapon@cv-nj.com) Date: Fri, 25 Jul 2003 10:57:43 -0400 From: Andriy Gapon To: ipfw@FreeBSD.org Message-id: <3F214567.9060308@cv-nj.com> MIME-version: 1.0 Content-type: text/plain; charset=us-ascii; format=flowed Content-transfer-encoding: 7BIT X-Accept-Language: en, uk, ru User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.4) Gecko/20030701 Subject: Re: Dynamic rules not being matched after divert... X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2003 14:57:50 -0000 Sean, it's understandable why you are tempted to call the interaction between ipfw stateful rules and natd a bug, but you are wrong. Yes, in both cases the packets are matched against the dynamic rules after address transaltion, but that's exactly the problem - the outgoing packets already have an external src address, but the incoming packest already have an internal dst address - obviously they won't match. Advice from Michael Sierchio is pretty reasonable, and I am not sure why you would want to see internal state table of natd - if you want to account traffic or take a look at the established connections, then there are the specialized tools for that e.g. trafshow, trafd. However, if you have reasons to not fully trust natd, and you don't mind performance overhead of having both dynamic ipfw rules and natd, then there is a solution as well - it is to use skipto dynamic rules. In the case you haven't found it while searching for the previous discussions on this topic, both "trusted natd" and "non trusted natd" ideas are explained a little bit more here: http://docs.freebsd.org/cgi/getmsg.cgi?fetch=11483+0+archive/2002/freebsd-ipfw/20021027.freebsd-ipfw I can provide the examples of the working sets of the rules upon request. I do not promise however to generalize them from my specific setup or to find a time to give an advice on your specific setup. P.S. about the ipfw page - it is correct (but a bit confusing for a novice), the search does terminate. It's just that natd (and probably all other reasonable daemons that use divert) *reinserts* a packet after the same rule. But it isn't required to reinsert at that place, nor it is required to reinsert a packet at all. divert(4). -- Andriy Gapon From owner-freebsd-ipfw@FreeBSD.ORG Fri Jul 25 10:38:39 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 21ACF37B401 for ; Fri, 25 Jul 2003 10:38:39 -0700 (PDT) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id B6FE24400D for ; Fri, 25 Jul 2003 10:38:18 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.8p1/8.12.3) with ESMTP id h6PHcHkN054604; Fri, 25 Jul 2003 10:38:17 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.8p1/8.12.3/Submit) id h6PHcEHn054603; Fri, 25 Jul 2003 10:38:14 -0700 (PDT) (envelope-from rizzo) Date: Fri, 25 Jul 2003 10:38:14 -0700 From: Luigi Rizzo To: Mark Hannon Message-ID: <20030725103814.A54554@xorpc.icir.org> References: <3F212BF7.4060602@optusnet.com.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <3F212BF7.4060602@optusnet.com.au>; from markhannon@optusnet.com.au on Fri, Jul 25, 2003 at 11:09:11PM +1000 cc: freebsd-ipfw@freebsd.org Subject: Re: using dummynet to simulate modem, dsl, etc X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2003 17:38:39 -0000 at the very least you are passing packets through each pipe twice (once in ip_output(), once in ipo_input()), which coupled with the delays causes a lot of interference in the queues. I would expect you to get around 3.5KBytes/s in the steady state, but with a packet transmission time of 200ms and the 600ms RTT of your setting, the first few rounds might achieve somewhere between that and twice the bandwidth because of less interference. This is exactly what you are seeing. Try this: ipfw -q add 1 pipe 1 ip from any to 127.0.0.100 in ipfw -q add 2 pipe 2 ip from 127.0.0.100 to any in ipfw -q pipe 1 config bw 33kbit/s queue 0 delay 40ms ipfw -q pipe 2 config bw 56kbit/s queue 0 delay 112ms to have a more realistic setting (300ms RTT is still a bit on the high side i would say.) cheers luigi On Fri, Jul 25, 2003 at 11:09:11PM +1000, Mark Hannon wrote: > Hello, > > I am trying to use dummynet to simulate different network links and > demonstrate > application level performance for a variety of scenarios. The first > test is three > ftp file transfers over a simulated 56k modem as per below: > > iifconfig lo0 mtu 1500 > fconfig lo0 alias 127.0.0.100 > ipfw -q add 1 pipe 1 ip from any to 127.0.0.100 > ipfw -q add 2 pipe 2 ip from 127.0.0.100 to any > ipfw -q pipe 1 config bw 33kbit/s queue 0 delay 40ms > ipfw -q pipe 2 config bw 56kbit/s queue 0 delay 112ms > > > The results seem strange ... larger file sizes seem to have a slower > transfer rate, > where I would have thought that steady-state transfers would have been > reached > for the larger files. Are these results logical? Have I done something > silly with the > pipe configs? > > Receiving test_large (1937201 bytes): 100% (ETA 00:00) > 1937201 bytes transferred in 551.1 seconds (3.43 kBps) > Receiving test_medium (248476 bytes): 100% (ETA 00:00) > 248476 bytes transferred in 66.7 seconds (3.64 kBps) > Receiving test_small (40960 bytes): 100% > 40960 bytes transferred in 7.3 seconds (5.52 kBps) > > > Regards/mark > > > > > > > _______________________________________________ > freebsd-ipfw@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw > To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@freebsd.org" From owner-freebsd-ipfw@FreeBSD.ORG Fri Jul 25 11:48:23 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C998437B401 for ; Fri, 25 Jul 2003 11:48:23 -0700 (PDT) Received: from gradlab.ucsd.edu (gradlab.ucsd.edu [132.239.55.107]) by mx1.FreeBSD.org (Postfix) with ESMTP id 412D343F3F for ; Fri, 25 Jul 2003 11:48:23 -0700 (PDT) (envelope-from ycheng@cs.ucsd.edu) Received: (from ycheng@localhost) by gradlab.ucsd.edu (8.11.6/8.11.6) id h6PImMD03479; Fri, 25 Jul 2003 11:48:22 -0700 (PDT) Date: Fri, 25 Jul 2003 11:48:22 -0700 From: Yuchung Cheng To: freebsd-ipfw@freebsd.org Message-ID: <20030725114822.A2287@cs.ucsd.edu> Mail-Followup-To: freebsd-ipfw@freebsd.org References: <3F212BF7.4060602@optusnet.com.au> <20030725103814.A54554@xorpc.icir.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <20030725103814.A54554@xorpc.icir.org>; from rizzo@icir.org on Fri, Jul 25, 2003 at 10:38:14AM -0700 Subject: Re: using dummynet to simulate modem, dsl, etc X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2003 18:48:24 -0000 On 07-25-2003, Luigi Rizzo wrote: > I would expect you to get around 3.5KBytes/s in the steady state, > but with a packet transmission time of 200ms and the 600ms RTT of > your setting, the first few rounds might achieve somewhere between > that and twice the bandwidth because of less interference. > > This is exactly what you are seeing. > > Try this: > ipfw -q add 1 pipe 1 ip from any to 127.0.0.100 in > ipfw -q add 2 pipe 2 ip from 127.0.0.100 to any in > ipfw -q pipe 1 config bw 33kbit/s queue 0 delay 40ms > ipfw -q pipe 2 config bw 56kbit/s queue 0 delay 112ms > why set queue to 0? aren't modem banks usually 4K bytes? the reverse link has 3 times delay (c->s), is this for simulating ack spacing and busy server to client pipe? I am doing similar experiments. Anybody can share parameters for ADSL, cable modems too? Searching the web does not give any good information. Yu-chung From owner-freebsd-ipfw@FreeBSD.ORG Fri Jul 25 16:39:17 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AFDCB37B401 for ; Fri, 25 Jul 2003 16:39:17 -0700 (PDT) Received: from anchor-post-39.mail.demon.net (anchor-post-39.mail.demon.net [194.217.242.80]) by mx1.FreeBSD.org (Postfix) with ESMTP id EC26443F85 for ; Fri, 25 Jul 2003 16:39:16 -0700 (PDT) (envelope-from darren@dazdaz.org) Received: from no-dns-yet.demon.co.uk ([62.49.203.115] helo=localhost) by anchor-post-39.mail.demon.net with esmtp (Exim 3.36 #2) id 19gC9b-00015z-0U for freebsd-ipfw@freebsd.org; Sat, 26 Jul 2003 00:39:15 +0100 Date: Sat, 26 Jul 2003 00:39:10 +0100 From: Darren X-Mailer: The Bat! (v1.61) X-Priority: 3 (Normal) Message-ID: <13347545536.20030726003910@dazdaz.org> To: freebsd-ipfw@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Subject: no keep-state and and unpredictable ssh connections X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Darren List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2003 23:39:18 -0000 Hello freebsd-ipfw, I'm not using keep-state and yet ssh'ing into my FreeBSD 4.8-STABLE box does'nt happen every attempt, more like 1 attempt in every 15! Incoming ssh connection work fine when firewall is disabled. My ips obviously changed. This is my rc.firewall script. Greatly appreciate some guidance, i've read some docs, but am missing something. #!/bin/sh fwcmd="/sbin/ipfw" myip="11.11.203.114" bcast="11.11.203.119" network="11.11.203.112/29" dns_server="158.152.1.43" # Run this if you want to run it multiple times # echo y | sudo ipfw flush zero resetlog # Reset all rules in case script run multiple times echo y | ${fwcmd} flush zero resetlog ${fwcmd} add allow log all from any to any via lo0 # Allow ourself ${fwcmd} add allow log tcp from ${myip} to $myip{} in recv xl0 # Allow our netblock ${fwcmd} add allow log tcp from ${mynetwork} to any in recv xl0 # Allow broadcasts ${fwcmd} add allow log tcp from ${myip} to ${bcast} in recv xl0 # Allow INCOMING ssh and HTTP from anywhere on the internet ${fwcmd} add allow log tcp from 0.0.0.0 to ${myip} 22,80 in recv xl0 # Allow DNS client lookups ${fwcmd} add allow udp from ${myip} to ${dns_server} 53 in recv xl0 ${fwcmd} add allow udp from ${dns_server} 53 to ${my_ip} in recv xl0 ################################ # Block RFC 1918 networks ################################ ${fwcmd} add deny all from 0.0.0.0/7 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 1.0.0.0/8 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 2.0.0.0/8 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 5.0.0.0/8 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 10.0.0.0/8 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 23.0.0.0/8 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 27.0.0.0/8 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 31.0.0.0/8 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 67.0.0.0/8 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 68.0.0.0/6 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 72.0.0.0/5 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 80.0.0.0/4 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 96.0.0.0/3 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 127.0.0.0/8 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 128.0.0.0/16 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 128.66.0.0/16 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 169.254.0.0/16 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 172.16.0.0/12 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 191.255.0.0/16 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 192.0.0.0/16 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 192.168.0.0/16 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 197.0.0.0/8 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 201.0.0.0/8 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 204.152.64.0/23 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 224.0.0.0/3 to 0.0.0.0 in recv xl0 ${fwcmd} add deny all from 240.0.0.0/8 to 0.0.0.0 in recv xl0 # disable icmp ${fwcmd} add deny log icmp from any to any in recv xl0 ${fwcmd} add deny log all from any to any recv xl0 # End of rc.firewall -- Best regards, Darren mailto:darren@dazdaz.org From owner-freebsd-ipfw@FreeBSD.ORG Fri Jul 25 20:47:55 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F1AFF37B401 for ; Fri, 25 Jul 2003 20:47:54 -0700 (PDT) Received: from laptop.tenebras.com (laptop.tenebras.com [66.92.188.18]) by mx1.FreeBSD.org (Postfix) with SMTP id 5341C43FA3 for ; Fri, 25 Jul 2003 20:47:52 -0700 (PDT) (envelope-from kudzu@tenebras.com) Received: (qmail 34400 invoked from network); 26 Jul 2003 03:47:48 -0000 Received: from sapphire.tenebras.com (HELO tenebras.com) (192.168.188.241) by 0 with SMTP; 26 Jul 2003 03:47:48 -0000 Message-ID: <3F21F9E4.9060408@tenebras.com> Date: Fri, 25 Jul 2003 20:47:48 -0700 From: Michael Sierchio User-Agent: Mozilla/5.0 (X11; U; Linux i386; en-US; rv:1.3.1) Gecko/20030425 X-Accept-Language: en-us, zh-tw, zh-cn, fr, en, de-de MIME-Version: 1.0 To: Darren References: <13347545536.20030726003910@dazdaz.org> In-Reply-To: <13347545536.20030726003910@dazdaz.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-ipfw@freebsd.org Subject: Re: no keep-state and and unpredictable ssh connections X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Jul 2003 03:47:55 -0000 Darren wrote: > Hello freebsd-ipfw, > > I'm not using keep-state and yet ssh'ing into my FreeBSD 4.8-STABLE > box does'nt happen every attempt, more like 1 attempt in every 15! > Incoming ssh connection work fine when firewall is disabled. > > My ips obviously changed. This is my rc.firewall script. > > Greatly appreciate some guidance, i've read some docs, but am missing > something. Is this a firewall-router, or are you trying to protect the box itself? (In other words, is $myip an address on this box?) The ruleset could use some refactoring -- that's the polite word -- but the direction depends on the answer to my question above. > #!/bin/sh > > fwcmd="/sbin/ipfw" > myip="11.11.203.114" Uh, Darren, some burly guys with shaved heads and no necks are going to be knocking on your door any minute now if you use that address. They were humorless before 9/11, think of how much fun they are now. From owner-freebsd-ipfw@FreeBSD.ORG Sat Jul 26 01:13:50 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7330737B401 for ; Sat, 26 Jul 2003 01:13:50 -0700 (PDT) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id EF0AB43F3F for ; Sat, 26 Jul 2003 01:13:49 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.8p1/8.12.3) with ESMTP id h6Q8DnkN060257; Sat, 26 Jul 2003 01:13:49 -0700 (PDT) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.8p1/8.12.3/Submit) id h6Q8DnPm060256; Sat, 26 Jul 2003 01:13:49 -0700 (PDT) (envelope-from rizzo) Date: Sat, 26 Jul 2003 01:13:49 -0700 From: Luigi Rizzo To: freebsd-ipfw@freebsd.org Message-ID: <20030726011349.A60140@xorpc.icir.org> References: <3F212BF7.4060602@optusnet.com.au> <20030725103814.A54554@xorpc.icir.org> <20030725114822.A2287@cs.ucsd.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <20030725114822.A2287@cs.ucsd.edu>; from ycheng@cs.ucsd.edu on Fri, Jul 25, 2003 at 11:48:22AM -0700 Subject: Re: using dummynet to simulate modem, dsl, etc X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Jul 2003 08:13:50 -0000 On Fri, Jul 25, 2003 at 11:48:22AM -0700, Yuchung Cheng wrote: > On 07-25-2003, Luigi Rizzo wrote: > > I would expect you to get around 3.5KBytes/s in the steady state, > > but with a packet transmission time of 200ms and the 600ms RTT of > > your setting, the first few rounds might achieve somewhere between > > that and twice the bandwidth because of less interference. > > > > This is exactly what you are seeing. > > > > Try this: > > ipfw -q add 1 pipe 1 ip from any to 127.0.0.100 in > > ipfw -q add 2 pipe 2 ip from 127.0.0.100 to any in > > ipfw -q pipe 1 config bw 33kbit/s queue 0 delay 40ms > > ipfw -q pipe 2 config bw 56kbit/s queue 0 delay 112ms > > > why set queue to 0? aren't modem banks usually 4K bytes? the reverse link > has 3 times delay (c->s), is this for simulating ack spacing and busy > server to client pipe? i just copied those parameters from the original posters. I have no idea on what system he had in mind. In terms of delay, i typically used to see some 120-150mss RTT for short pings (including everything). Just the transmission time is in the order of 5-10ms at each step, and there are many instances of it (serial->modem; modem->modem; modem->serial on the remote side; and another 3 on the way back) so i think this eats almost half ot the total delay. In terms of buffering, again i have no idea, but at least on the client side you have as much as the local OS gives you. cheers luigi > I am doing similar experiments. Anybody can share parameters for ADSL, cable > modems too? Searching the web does not give any good information. > > Yu-chung > _______________________________________________ > freebsd-ipfw@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw > To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@freebsd.org" From owner-freebsd-ipfw@FreeBSD.ORG Sat Jul 26 03:53:59 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D709337B401 for ; Sat, 26 Jul 2003 03:53:59 -0700 (PDT) Received: from anchor-post-35.mail.demon.net (anchor-post-35.mail.demon.net [194.217.242.85]) by mx1.FreeBSD.org (Postfix) with ESMTP id BE11D43FA3 for ; Sat, 26 Jul 2003 03:53:58 -0700 (PDT) (envelope-from darren@dazdaz.org) Received: from no-dns-yet.demon.co.uk ([62.49.203.115] helo=localhost) by anchor-post-35.mail.demon.net with esmtp (Exim 3.36 #2) id 19gMgX-0002sm-0Z; Sat, 26 Jul 2003 11:53:57 +0100 Date: Sat, 26 Jul 2003 11:53:50 +0100 From: Darren X-Mailer: The Bat! (v1.61) X-Priority: 3 (Normal) Message-ID: <2384322.20030726115350@dazdaz.org> To: Michael Sierchio In-Reply-To: <3F21F9E4.9060408@tenebras.com> References: <13347545536.20030726003910@dazdaz.org> <3F21F9E4.9060408@tenebras.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit cc: freebsd-ipfw@freebsd.org Subject: Re[2]: no keep-state and and unpredictable ssh connections X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Darren List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Jul 2003 10:54:00 -0000 Hello Michael, Saturday, July 26, 2003, 4:47:48 AM, you wrote: MS> Darren wrote: >> Hello freebsd-ipfw, >> >> I'm not using keep-state and yet ssh'ing into my FreeBSD 4.8-STABLE >> box does'nt happen every attempt, more like 1 attempt in every 15! >> Incoming ssh connection work fine when firewall is disabled. >> >> My ips obviously changed. This is my rc.firewall script. >> >> Greatly appreciate some guidance, i've read some docs, but am missing >> something. MS> Is this a firewall-router, or are you trying to protect the box itself? MS> (In other words, is $myip an address on this box?) There is no firewall-router in-between. $myip is an address on the box itself. MS> The ruleset could use some refactoring -- that's the polite word -- but MS> the direction depends on the answer to my question above. Fine. What would you change or refactor and why? If it should be ripped apart, can you please explain which bits and why? >> #!/bin/sh >> >> fwcmd="/sbin/ipfw" >> myip="11.11.203.114" MS> Uh, Darren, some burly guys with shaved heads and no necks are MS> going to be knocking on your door any minute now if you use that MS> address. MS> They were humorless before 9/11, think of how much fun they are now. Greatly appreciate your concern, however as I pointed out above, I changed the IP address for just this reason :-) -- Best regards, Darren mailto:darren@dazdaz.org From owner-freebsd-ipfw@FreeBSD.ORG Sat Jul 26 16:15:39 2003 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8774D37B401 for ; Sat, 26 Jul 2003 16:15:39 -0700 (PDT) Received: from out001.verizon.net (out001pub.verizon.net [206.46.170.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id 92F5C43FA3 for ; Sat, 26 Jul 2003 16:15:38 -0700 (PDT) (envelope-from cswiger@mac.com) Received: from mac.com ([141.149.47.46]) by out001.verizon.net (InterMail vM.5.01.05.33 201-253-122-126-133-20030313) with ESMTP id <20030726231537.MONZ12592.out001.verizon.net@mac.com> for ; Sat, 26 Jul 2003 18:15:37 -0500 Message-ID: <3F230B9A.5000703@mac.com> Date: Sat, 26 Jul 2003 19:15:38 -0400 From: Chuck Swiger Organization: The Courts of Chaos User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-ipfw@freebsd.org References: <13347545536.20030726003910@dazdaz.org> <3F21F9E4.9060408@tenebras.com> <2384322.20030726115350@dazdaz.org> In-Reply-To: <2384322.20030726115350@dazdaz.org> X-Enigmail-Version: 0.76.1.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Authentication-Info: Submitted using SMTP AUTH at out001.verizon.net from [141.149.47.46] at Sat, 26 Jul 2003 18:15:37 -0500 Subject: Re: no keep-state and and unpredictable ssh connections X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Jul 2003 23:15:39 -0000 Darren wrote: [ ... ] > MS> Uh, Darren, some burly guys with shaved heads and no necks are > MS> going to be knocking on your door any minute now if you use that > MS> address. > > MS> They were humorless before 9/11, think of how much fun they are now. > > Greatly appreciate your concern, however as I pointed out above, I changed the IP > address for just this reason :-) For those who are curious about what that IP points to: 2-sec% whois 11.11.203.114 OrgName: DoD Network Information Center OrgID: DNIC Address: 7990 Science Applications Ct Address: M/S CV 50 City: Vienna StateProv: VA PostalCode: 22183-7000 Country: US NetRange: 11.0.0.0 - 11.255.255.255 CIDR: 11.0.0.0/8 NetName: DODIIS NetHandle: NET-11-0-0-0-1 Parent: NetType: Direct Allocation Comment: DoD Intel Information Systems Comment: Defense Intelligence Agency Comment: Washington, DC 20301 US RegDate: 1984-01-19 Updated: 1998-09-26 :-) -- -Chuck