From owner-freebsd-pf@FreeBSD.ORG Fri Jan 23 18:04:24 2009 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8144D106564A for ; Fri, 23 Jan 2009 18:04:24 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.187]) by mx1.freebsd.org (Postfix) with ESMTP id 111068FC17 for ; Fri, 23 Jan 2009 18:04:24 +0000 (UTC) (envelope-from max@love2party.net) Received: from vampire.homelinux.org (dslb-088-066-034-230.pools.arcor-ip.net [88.66.34.230]) by mrelayeu.kundenserver.de (node=mrelayeu2) with ESMTP (Nemesis) id 0MKwtQ-1LQQOF0EUN-0004tH; Fri, 23 Jan 2009 19:04:23 +0100 Received: (qmail 29760 invoked from network); 23 Jan 2009 18:04:22 -0000 Received: from fbsd8.laiers.local (192.168.4.200) by router.laiers.local with SMTP; 23 Jan 2009 18:04:22 -0000 From: Max Laier Organization: FreeBSD To: freebsd-pf@freebsd.org Date: Fri, 23 Jan 2009 19:04:22 +0100 User-Agent: KMail/1.10.4 (FreeBSD/8.0-CURRENT; KDE/4.1.4; i386; ; ) References: <17838240D9A5544AAA5FF95F8D520316056585C1@ad-exh01.adhost.lan> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200901231904.22558.max@love2party.net> X-Provags-ID: V01U2FsdGVkX1+MAURQE8ncjaZ3mYNlZujZwbY5AIVCBM3XfpK +MT+qxYmNRxm7flV/69lpTnA0O/q+Jhc2fh0qYkQVDdep6Jl4/ YE44uy1Fq9NgRt+5AoToQ== Cc: Subject: Re: Issues with PF and 7.1 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Jan 2009 18:04:24 -0000 On Friday 23 January 2009 18:21:32 Scott Ullrich wrote: > On Thu, Jan 22, 2009 at 2:32 PM, Michael K. Smith - Adhost > > wrote: > > Hello All: > > > > We are having memory issues with PF and 7.1p2 that we didn't experience > > with 6.3. Here's what happens. > > > > # pfctl -f /usr/local/etc/pf.conf > > /usr/local/etc/pf.conf:135: cannot define table smtpd_reject_policyd: > > Cannot allocate memory /usr/local/etc/pf.conf:139: cannot define table > > smtpd_reject_spam: Cannot allocate memory pfctl: Syntax error in config > > file: pf rules not loaded > > # pfctl -t smtpd_reject_policyd -T flush > > 94390 addresses deleted. > > # pfctl -t smtpd_reject_spam -T flush > > 62464 addresses deleted. > > # pfctl -f /usr/local/etc/pf.conf > > > > So, after I flush the tables it loads. Sometimes, however, we get a > > global out of memory error " DIOCADDRULE: Cannot allocate memory " > > > > Here are my entries from pf.conf for various limits. Everything else is > > defaults. > > > > set limit tables 500 > > set limit table-entries 250000 > > set limit { states 1000000, src-nodes 300000, frags 100000 } > > set optimization normal > > set skip on lo0 > > set state-policy if-bound > > set timeout interval 300 > > set timeout src.track 1200 > > > > Finally, the box is using EM interfaces with VLAN's and has 4 Gig of > > physical RAM. There are two PF boxes in Active/Failover and the errors > > show up on both, although they seem to show up more often on the Backup > > device, which seems odd. > > > > Any help would be greatly appreciated. > > My first response would have been to set set limit table-entries but > you already did that. > > Next thing I would check is a shot in the dark, but worth trying.. > > What does sysctl vm.kmem_size_max show? Try increasing that size a > bit in loader.conf and see if that helps. Seconded. My guess is that the system flushes buffers when you first load the tables due to memory pressure, so when you load the tables a second time there is more space available. This, however, suggest that you are pretty thin stretched regarding kvm and should really increase it. I'd shoot for at least 512M which I believe is the maximum in 7.1 with the stock kernel. It seems that there is work in progress to increase that limit for amd64 in releng_7, however. Increasing this is worthwhile in any case, as I have a hard time imagining what else you'd be doing with those 4G on the firewalls (unless you are running heavy webcaches on them, too). -- /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News