From owner-freebsd-ipfw@FreeBSD.ORG Tue Jan 6 22:09:41 2004 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CC16F16A4CE for ; Tue, 6 Jan 2004 22:09:41 -0800 (PST) Received: from xorpc.icir.org (xorpc.icir.org [192.150.187.68]) by mx1.FreeBSD.org (Postfix) with ESMTP id 16D9B43D5E for ; Tue, 6 Jan 2004 22:09:32 -0800 (PST) (envelope-from rizzo@icir.org) Received: from xorpc.icir.org (localhost [127.0.0.1]) by xorpc.icir.org (8.12.9p1/8.12.8) with ESMTP id i0769VAF035161; Tue, 6 Jan 2004 22:09:31 -0800 (PST) (envelope-from rizzo@xorpc.icir.org) Received: (from rizzo@localhost) by xorpc.icir.org (8.12.9p1/8.12.3/Submit) id i0769P0D035160; Tue, 6 Jan 2004 22:09:25 -0800 (PST) (envelope-from rizzo) Date: Tue, 6 Jan 2004 22:09:25 -0800 From: "'Luigi Rizzo'" To: Artis Caune Message-ID: <20040106220925.A35110@xorpc.icir.org> References: <20031106033919.A65661@xorpc.icir.org> <200401061552.i06Fqn1h007092@wyvern.icir.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <200401061552.i06Fqn1h007092@wyvern.icir.org>; from ac-lists@latnet.lv on Tue, Jan 06, 2004 at 05:51:29PM +0200 cc: freebsd-ipfw@freebsd.org Subject: Re: loading lot of rules takes very long time X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 07 Jan 2004 06:09:42 -0000 good to hear you found a viable solution for you :) cheers luigi On Tue, Jan 06, 2004 at 05:51:29PM +0200, Artis Caune wrote: > Happy new year! > > Back to old topic... > > I found solution to my problem ! > > we really need 10000++ rules. ;) > > I wrote my own traffic shaper: kld module. > It attach to ip filter 'fr_checkp' function pointer. > It use binary tree to match IP packets, and > it always match in 32 steps. > It then use CISCO CAR (token bucket) algorithm to > shape (rate limit) traffic. > It can use CISCO 'compounded debt' algorithm > to simulate RED ;) > It don't use queues, it don't add delays. > It shape very close to configured rate. > Grr, and it works in bridge mode. > > I can load 30000 rules on PII 300 box in 1-2 seconds. > Yeh, that's seconds, not hours ;) > > config file is very simple: > > interface em0 in dst-ip > > pipe test 64KB > > 10.0.0.0/24 pipe test > > 192.168.0.1/32 pipe test > > All works great!!! > > Totaly about 1500 lines of kernel/userland code. > > > > > cheers > Artis > > -----Original Message----- > From: owner-freebsd-ipfw@freebsd.org [mailto:owner-freebsd-ipfw@freebsd.org] > On Behalf Of Luigi Rizzo > Sent: ceturtdiena, 2003. gada 6. novembri 13:39 > To: Artis Caune > Cc: freebsd-ipfw@freebsd.org > Subject: Re: loading lot of rules takes very long time > > most likely, because you are not using "-n", the printing > code will use the nameserver to try and resolve addresses, and > if halfway through you are limiting/blocking access to the > nameserver you incur in timeouts. > > To tell the truth i suspect you have a quite poorly designed > ruleset if you are adding individual rules and pipes for each > client. Almost surely you should make use of masks in pipes, > and address sets in rules, to reduce the size of your ruleset > to something manageable and efficient. > > cheers > luigi > > > On Thu, Nov 06, 2003 at 01:04:31PM +0200, Artis Caune wrote: > > Hello, > > > > We have about 10000-20000 pipes for > > different subnets, and it takes very long > > time to load them - about 10-15min. > > > > 92.8% interrupt, 0.0% idle > > > > strange that things slow down when count > > reaches 2000-2500 rules. > > > > is there something we can do to speed things up? > > > > rules are added like: > > ipfw -q add 1 pipe 1 src-ip 1.1.1.1 out via em0 > > ipfw pipe 1 config bw 30Kbytes/s queue 10 > > ... > > soo 'ipfw' is invoked '2 x client_count' !!! > > > > maybe ipfw need feature like: > > ipfw -f /etc/rc.firewall > > > > > > > > # FreeBSD-4.9, IPFW2, > > # HZ=2000, DEVICE_POLLING, > > # 1G RAM, 2.4xeon on Intel server board > > > > > > > > > > > > ..... > > Artis > > > > > > _______________________________________________ > > freebsd-ipfw@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw > > To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-ipfw@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw > To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@freebsd.org" >