From owner-freebsd-pf@FreeBSD.ORG Wed Mar 26 11:47:10 2008 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B9A09106564A for ; Wed, 26 Mar 2008 11:47:10 +0000 (UTC) (envelope-from jdc@parodius.com) Received: from mx01.sc1.parodius.com (mx01.sc1.parodius.com [72.20.106.3]) by mx1.freebsd.org (Postfix) with ESMTP id A133A8FC2D for ; Wed, 26 Mar 2008 11:47:10 +0000 (UTC) (envelope-from jdc@parodius.com) Received: by mx01.sc1.parodius.com (Postfix, from userid 1000) id 7F07E1CC060; Wed, 26 Mar 2008 04:47:10 -0700 (PDT) Date: Wed, 26 Mar 2008 04:47:10 -0700 From: Jeremy Chadwick To: Greg Hennessy Message-ID: <20080326114710.GA81567@eos.sc1.parodius.com> References: <9DE6EC5B5CF8C84281AE3D7454376A0D6D0290@cetus.dawnsign.com> <20080326025316.GA68607@eos.sc1.parodius.com> <47EA12CA.90305@nviz.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <47EA12CA.90305@nviz.net> User-Agent: Mutt/1.5.17 (2007-11-01) Cc: freebsd-pf@freebsd.org Subject: Re: Bacula File/Storage Connection Woes using PF X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2008 11:47:10 -0000 On Wed, Mar 26, 2008 at 09:09:30AM +0000, Greg Hennessy wrote: > Jeremy Chadwick wrote: >> There's been too many cases I've experienced where using "keep state" >> blindly results in state-mismatch increasing at a very fast rate. When >> I implemented this mentality on our production servers, our users >> started pointing out that scp's between machines would randomly get >> severed mis-stream, same with ssh sessions where large TCP windows were >> used (such as doing 'dmesg' over and over): >> >> http://lists.freebsd.org/pipermail/freebsd-pf/2008-January/004050.html > > Which (taking a rough guess) looking at your rule set in the above has very > little to do with 'keep state' and a lot to do with 'modulate state'. IIRC > there is a filed bug which displays all of the aforementioned symptoms when > modulate state meets selective acknowledgement (SACK). I'm sure Max has the > gory detail, it may even be fixed. Which is interesting for two reasons: 1) I never got an explanation in aforementioned thread, thus I had no knowledge of this bug until now -- thank you for telling me! 2) The documentation for FreeBSD pf is minimal. The best we've got is pf.conf(5), /etc/pf.conf (which tells you to read pf.conf(5)), and /usr/share/examples/pf/* which doesn't give you much in the ways of education either. This forces users to turn to the official OpenBSD pf documentation, which strongly advocates the use of "modulate state" with TCP packets: http://www.openbsd.org/faq/pf/filter.html#state With these facts at hand, surely you can see how someone would end up in this situation? And I am not the only one: Doug Sampson's mail is further proof of this. Now that you have educated me in regards to said bug, I will have to go back and try using "keep state" instead. Again, thank you! >> The "use keep state on everything!" attitude seems to stem from people >> reading the OpenBSD pf.conf documentation, which states that as of >> OpenBSD 4.1, "keep state" is implicit on every rule (meaning it's done >> whether you say "keep state" or not). FreeBSD's pf isn't like this. >> > You miss out the most important bit of the new PF 4.1 state keeping > defaults, 'flags S/SA'. I assume "pf 4.1" means "OpenBSD 4.1's pf", or is there some internal pf versioning number? This brings up another situation: there's no version number of pf in FreeBSD that I can find. The OpenBSD docs continually say "as of OpenBSD x.y". This confuses people, who when using pf under FreeBSD, have no knowledge of what version of pf we're using. What version is in RELENG_6? 7? CURRENT? I didn't know until a few minutes ago -- because I went to cvsweb and had to look up the CVS commit messages myself: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/contrib/pf/net/pf.c Now that I know, I can make appropriate adjustments. But requiring users to look at CVS commit messages is a bit unrealistic, don't you think? Maybe I should submit a PR asking that the version of pf pulled into FreeBSD be kept in the pf(4), pf.conf(5), and pfctl(8) manpages? What do you suggest? > Our cousins over the road in the OpenBSD neighbourhood have done this > precisely because of the issues caused in prior versions of PF by using > stateless rules and/or establishing TCP state on anything other than the 3 > way handshake. Yep, aware of that -- except that users have no idea as to whether the implicit "keep state" on every rule applies to FreeBSD or not, or if it's "safe" or not, because OpenBSD != FreeBSD. They read the OpenBSD docs and go "errr... so what version is FreeBSD using?" >> It gets more confusing when you consider the fact that even though UDP >> and ICMP are stateless protocols, pf can keep track of their state too, >> though I don't know if FreeBSD pf supports that (OpenBSD pf does). >> > This is not a flame, but if you really do not know that, you really should > not be publicly advocating a position on the basis of incomplete > information. And "incomplete information" is induced by what I described in my above paragraphs. Please don't be so harsh under the circumstances -- I was going off of all the available information I had at the time. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |