From owner-freebsd-pf@FreeBSD.ORG Mon Oct 24 16:34:07 2005 Return-Path: X-Original-To: freebsd-pf@freebsd.org Delivered-To: freebsd-pf@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 47D3416A41F for ; Mon, 24 Oct 2005 16:34:07 +0000 (GMT) (envelope-from gallasch@free.de) Received: from gedankenkraft.free.de (gedankenkraft.free.de [193.28.225.6]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9B35043D49 for ; Mon, 24 Oct 2005 16:34:06 +0000 (GMT) (envelope-from gallasch@free.de) Received: (qmail 78593 invoked by uid 1005); 24 Oct 2005 16:34:05 -0000 Received: from gallasch@free.de by gedankenkraft.free.de by uid 82 with qmail-scanner-1.20 (spamassassin: 2.63. Clear:RC:1(193.28.225.190):SA:0(-104.9/4.0):. Processed in 1.005373 secs); 24 Oct 2005 16:34:05 -0000 X-Spam-Status: No Received: from orwell.free.de (HELO [193.28.225.190]) (gallasch@[193.28.225.190]) (envelope-sender ) by gedankenkraft.free.de (qmail-ldap-1.03) with RC4-SHA encrypted SMTP for ; 24 Oct 2005 16:34:04 -0000 Mime-Version: 1.0 (Apple Message framework v734) Content-Transfer-Encoding: 7bit Message-Id: <6BDA08CF-3930-4F37-BB47-EAC722391D41@free.de> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed To: freebsd-pf@freebsd.org From: Kai Gallasch Date: Mon, 24 Oct 2005 18:34:03 +0200 X-Mailer: Apple Mail (2.734) Subject: FreeBSD 6.0RC1 - pf and big tables, pfspamd X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Oct 2005 16:34:07 -0000 Hi list. Following setup: - FreeBSD 6.0RC1 + pf - /usr/ports/mail/spamd + recommended pf.conf for spamd - several huge rbl zonefiles in rbldnsd format - pf.conf table persist no rdr on { lo0, lo1 } from any to any rdr inet proto tcp from to any port smtp -> 192.168.0.100 port 8025 When I startup my spamd installation I am loading the zonefiles into the table through method "file" from disk It all works as expected, but when I load some of my bigger rbl zonefiles through command "spamd-setup" the application uses up huge amounts of memory and finally stops with error "malloc failed" - too bad. (and this after about an hour runtime, cough!) Probably spamd never was planned to get along with millions of entries in a table.. If I try to squeeze in the IPs manually through pfctl I get the error shorty# pfctl -t spamd -Tr -f spammers.txt pfctl: Cannot allocate memory. spammers.txt is about 30M in size and contains about 2 million entries Has someone found a workaround for using (and handling) up to 10 million IPs inside a pf table? :-) without using high end hardware (I currently use for testing pentium3, 1Ghz, 512M main memmory) pf: Is there a possibility to abuse pf in the following fashion? rdr inet proto tcp from a.b.c.d/32 [if dnsquery d.c.b.a.list.dsbl.org == 127.0.0.2] to any port smtp -> 192.168.0.100 port 8025 For example /usr/ports/dns/rbldnsd can handle such huge amounts of rbl data and even reloads take only a few seconds (with > 100M rbl files!!) If a firewall rule would be possible to do local RBL queries one could have the best of both worlds - use - as in my case rblndsd for keeping the rbldata and the pf for a flexible response to incoming spam.. Any idea? -- "Whenever bicycles are broken, or menaced by international communism, Bicycle Repair Man is ready!"