From owner-freebsd-stable@FreeBSD.ORG Fri Oct 3 11:38:28 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 51A7C10656A0 for ; Fri, 3 Oct 2008 11:38:28 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from QMTA10.westchester.pa.mail.comcast.net (qmta10.westchester.pa.mail.comcast.net [76.96.62.17]) by mx1.freebsd.org (Postfix) with ESMTP id E457B8FC2E for ; Fri, 3 Oct 2008 11:38:27 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from OMTA04.westchester.pa.mail.comcast.net ([76.96.62.35]) by QMTA10.westchester.pa.mail.comcast.net with comcast id NBU41a0010ldTLk5ABeT23; Fri, 03 Oct 2008 11:38:27 +0000 Received: from koitsu.dyndns.org ([69.181.141.110]) by OMTA04.westchester.pa.mail.comcast.net with comcast id NBeQ1a00K2P6wsM3QBeQw2; Fri, 03 Oct 2008 11:38:27 +0000 X-Authority-Analysis: v=1.0 c=1 a=CHt_mEGTzOcA:10 a=OJujf1_Ut9MA:10 a=QycZ5dHgAAAA:8 a=JOr_kaPOtu08zEbprysA:9 a=8El2cu6u8I-SJ3X6EeIA:7 a=mhKmPw4oBOWUK70ZEuP3FqN_iTEA:4 a=EoioJ0NPDVgA:10 a=LY0hPdMaydYA:10 Received: by icarus.home.lan (Postfix, from userid 1000) id 3630FC941A; Fri, 3 Oct 2008 04:38:24 -0700 (PDT) Date: Fri, 3 Oct 2008 04:38:24 -0700 From: Jeremy Chadwick To: freebsd-pf@freebsd.org Message-ID: <20081003113824.GA27757@icarus.home.lan> References: <48E535D3.8000805@cran.org.uk> <20081003111703.GA27385@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20081003111703.GA27385@icarus.home.lan> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: Bruce Cran , freebsd-stable@freebsd.org Subject: Re: pf rules not being loaded during boot on 7.1-PRERELEASE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Oct 2008 11:38:28 -0000 On Fri, Oct 03, 2008 at 04:17:03AM -0700, Jeremy Chadwick wrote: > On Thu, Oct 02, 2008 at 09:57:55PM +0100, Bruce Cran wrote: > > I recently upgraded my i386 router from 7.0 to 7.1-PRERELEASE. I > > rebooted it today but despite pf_enable="YES" being in /etc/rc.conf no > > rules got loaded during boot, despite pf itself having been enabled: > > > > router# pfctl -s rules > > router# pfctl -e -f /etc/pf.conf > > pfctl: pf already enabled > > [connection is closed due to new rules being loaded] > > router# pfctl -s rules > > scrub in all fragment reassemble > > [... lots of rules listed] > > > > Has anyone else seen this problem, or have I just missed something > > that's changed between 7.0 and 7.1 in the way pf works? > > I was seeing something similar on my own box which I just upgraded from > a 150-day-old RELENG_6 to present RELENG_6. pfctl -s rules output no > rules. pfctl -s info showed packet counters, but no interface stats > (due to the rules not being loaded, e.g. no loginterface). > > kldstat showed pflog.ko and pf.ko loaded. > > If I did /etc/rc.d/pf start, the rules would loaded, and everything > starts working as expected. > > I rebooted the box and saw the following on serial console, which I'm > pretty sure is what's responsible for the breakage: > > Enabling pf. > Oct 3 04:14:51 pflogd[374]: [priv]: msg PRIV_OPEN_LOG received > cannot determine interface bandwidth for bge0, specify an absolute > bandwidth > altq not defined on bge0 > altq not defined on bge0 > /conf/ME/pf.conf:52: errors in queue definition > altq not defined on bge0 > /conf/ME/pf.conf:53: errors in queue definition > altq not defined on bge0 > /conf/ME/pf.conf:54: errors in queue definition > pfctl: Syntax error in config file: pf rules not loaded > pf enabled Cross-posting to freebsd-pf (I'm sorry for doing this, but it needs attention from both -pf and -stable). I've figured out what the problem is. This is not good, and is guaranteed to bite other people. I'd like to believe this is an rc-related problem, but I'm not sure how to fix it. The problem in my case: The physical interfaces were brought online, but were still technically offline (the switch and NIC PHY were taking some time to negotiate speed and duplex). Boot messages: bge0: link state changed to DOWN bge1: link state changed to DOWN lo0: flags=8049 mtu 16384 inet 127.0.0.1 netmask 0xff000000 bge0: flags=8843 mtu 1500 options=1b inet XXXXXXXXXXX netmask 0xffffff80 broadcast XXXXXXXXXXXXX ether 00:30:48:81:fc:8a media: Ethernet autoselect (none) status: no carrier bge1: flags=8843 mtu 1500 options=1b inet XXXXXXXXX netmask 0xffffff00 broadcast XXXXXXXXXXX ether 00:30:48:81:fc:8b media: Ethernet autoselect (none) status: no carrier Note that the interfaces are UP, not DOWN. Then the very next thing seen on the console: Starting pflog. pflog0: promiscuous mode enabled Enabling pf. Oct 3 04:14:51 pflogd[374]: [priv]: msg PRIV_OPEN_LOG received cannot determine interface bandwidth for bge0, specify an absolute bandwidth altq not defined on bge0 altq not defined on bge0 /conf/ME/pf.conf:52: errors in queue definition altq not defined on bge0 /conf/ME/pf.conf:53: errors in queue definition altq not defined on bge0 /conf/ME/pf.conf:54: errors in queue definition pfctl: Syntax error in config file: pf rules not loaded pf enabled The error message about "interface bandwidth" is the key here. My ALTQ rules use "bandwidth ", not a static amount in bits: altq on $ext_if cbq bandwidth 100% queue { std, blah, blah2 } queue std bandwidth 95% cbq(default borrow) queue blah bandwidth 384Kb queue blah2 bandwidth 384Kb Since the PHY hadn't negotiated speed, pf was unable to determine what the percentage really mapped to bandwidth/bit-wise. If at all possible, pf should wait for the interfaces to come up fully (that includes autonegotiation being completed; do we have framework for this?) before starting. I changed my rules to use a static speeds (100% --> 100Mb, and 95% --> 95Mb), which appear to work, but after the 2nd reboot the speed/duplex had been negotiated by the time pf had started, so I don't know if it truly fixed anything. I don't know what pf will do if you say "100Mb" for an interface which has no link/speed defined yet. It may behave the same way as shown above; I don't know. This needs some thought and definitely a solution. Again, note that I'm using RELENG_6, but I've a feeling this might bite RELENG_7 too. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |