From owner-freebsd-ipfw@FreeBSD.ORG Tue Jan 6 07:52:55 2004 Return-Path: Delivered-To: freebsd-ipfw@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F263F16A4CE for ; Tue, 6 Jan 2004 07:52:54 -0800 (PST) Received: from mail.latnet.lv (mail.latnet.lv [159.148.108.13]) by mx1.FreeBSD.org (Postfix) with SMTP id EF5E743D41 for ; Tue, 6 Jan 2004 07:52:46 -0800 (PST) (envelope-from ac-lists@latnet.lv) Received: (qmail 25671 invoked by uid 64014); 6 Jan 2004 15:52:45 -0000 Received: from ac-lists@latnet.lv by mail by uid 64011 with qmail-scanner-1.16 (clamscan: 0.54. Clear:. Processed in 0.139005 secs); 06 Jan 2004 15:52:45 -0000 Received: from unknown (HELO artis) (159.148.107.1) by mail.latnet.lv with SMTP; 6 Jan 2004 15:52:45 -0000 From: "Artis Caune" To: "'Luigi Rizzo'" Date: Tue, 6 Jan 2004 17:51:29 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook, Build 11.0.5510 In-Reply-To: <20031106033919.A65661@xorpc.icir.org> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Thread-Index: AcOkWsQR5BVxSiSNRvmpLbAFblGCpQwEWSzQ X-Qmail-Scanner-Message-ID: <107340436552625664@mail> Message-Id: <20040106155246.EF5E743D41@mx1.FreeBSD.org> cc: freebsd-ipfw@freebsd.org Subject: RE: loading lot of rules takes very long time X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Jan 2004 15:52:55 -0000 Happy new year! Back to old topic... I found solution to my problem ! we really need 10000++ rules. ;) I wrote my own traffic shaper: kld module. It attach to ip filter 'fr_checkp' function pointer. It use binary tree to match IP packets, and it always match in 32 steps. It then use CISCO CAR (token bucket) algorithm to shape (rate limit) traffic. It can use CISCO 'compounded debt' algorithm to simulate RED ;) It don't use queues, it don't add delays. It shape very close to configured rate. Grr, and it works in bridge mode. I can load 30000 rules on PII 300 box in 1-2 seconds. Yeh, that's seconds, not hours ;) config file is very simple: > interface em0 in dst-ip > pipe test 64KB > 10.0.0.0/24 pipe test > 192.168.0.1/32 pipe test All works great!!! Totaly about 1500 lines of kernel/userland code. cheers Artis -----Original Message----- From: owner-freebsd-ipfw@freebsd.org [mailto:owner-freebsd-ipfw@freebsd.org] On Behalf Of Luigi Rizzo Sent: ceturtdiena, 2003. gada 6. novembri 13:39 To: Artis Caune Cc: freebsd-ipfw@freebsd.org Subject: Re: loading lot of rules takes very long time most likely, because you are not using "-n", the printing code will use the nameserver to try and resolve addresses, and if halfway through you are limiting/blocking access to the nameserver you incur in timeouts. To tell the truth i suspect you have a quite poorly designed ruleset if you are adding individual rules and pipes for each client. Almost surely you should make use of masks in pipes, and address sets in rules, to reduce the size of your ruleset to something manageable and efficient. cheers luigi On Thu, Nov 06, 2003 at 01:04:31PM +0200, Artis Caune wrote: > Hello, > > We have about 10000-20000 pipes for > different subnets, and it takes very long > time to load them - about 10-15min. > > 92.8% interrupt, 0.0% idle > > strange that things slow down when count > reaches 2000-2500 rules. > > is there something we can do to speed things up? > > rules are added like: > ipfw -q add 1 pipe 1 src-ip 1.1.1.1 out via em0 > ipfw pipe 1 config bw 30Kbytes/s queue 10 > ... > soo 'ipfw' is invoked '2 x client_count' !!! > > maybe ipfw need feature like: > ipfw -f /etc/rc.firewall > > > > # FreeBSD-4.9, IPFW2, > # HZ=2000, DEVICE_POLLING, > # 1G RAM, 2.4xeon on Intel server board > > > > > > ..... > Artis > > > _______________________________________________ > freebsd-ipfw@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw > To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@freebsd.org" _______________________________________________ freebsd-ipfw@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-ipfw To unsubscribe, send any mail to "freebsd-ipfw-unsubscribe@freebsd.org"