From owner-freebsd-pf@FreeBSD.ORG Tue Apr 3 23:08:55 2007 Return-Path: X-Original-To: freebsd-pf@freebsd.org Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 2962216A405 for ; Tue, 3 Apr 2007 23:08:55 +0000 (UTC) (envelope-from drew@mykitchentable.net) Received: from qsmtp4.mc.surewest.net (qsmtp.mc.surewest.net [66.60.130.145]) by mx1.freebsd.org (Postfix) with SMTP id 0C7E013C484 for ; Tue, 3 Apr 2007 23:08:54 +0000 (UTC) (envelope-from drew@mykitchentable.net) Received: (qmail 2986 invoked from network); 3 Apr 2007 16:08:54 -0700 Received: by simscan 1.1.0 ppid: 2957, pid: 2958, t: 3.2054s scanners: regex: 1.1.0 attach: 1.1.0 clamav: 0.84/m:42/d:2665 spam: 3.0.3 Received: from unknown (HELO blacklamb.mykitchentable.net) (66.205.146.210) by qsmtp4 with SMTP; 3 Apr 2007 16:08:51 -0700 Received: from [192.168.25.6] (unknown [192.168.25.6]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by blacklamb.mykitchentable.net (Postfix) with ESMTP id DA3F0164964; Tue, 3 Apr 2007 16:08:50 -0700 (PDT) Message-ID: <4612DE86.2000706@mykitchentable.net> Date: Tue, 03 Apr 2007 16:08:54 -0700 From: Drew Tomlinson User-Agent: Thunderbird 1.5.0.10 (Windows/20070221) MIME-Version: 1.0 To: Max Laier References: <46117263.3060203@mykitchentable.net> <200704031812.00089.max@love2party.net> In-Reply-To: <200704031812.00089.max@love2party.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Checker-Version: SpamAssassin 3.0.3 (2005-04-27) on qsmtp4.surewest.net X-Spam-Level: X-Spam-Status: No, score=-0.2 required=5.0 tests=AWL,BAYES_00, RCVD_IN_SORBS_DUL autolearn=no version=3.0.3 Cc: freebsd-pf@freebsd.org Subject: Re: Bacula and pf X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Apr 2007 23:08:55 -0000 On 4/3/2007 9:11 AM Max Laier wrote: > On Monday 02 April 2007 23:15, Drew Tomlinson wrote: > >> I run Bacula v1.38 on my home network. Ever since I moved from ipfw2 >> to pf, backups fail intermittently on my router due to "broken network >> pipes" usually after somewhere around 10 MB - 12 MB has been >> transfered. Thus small incremental backups are successful but larger >> full backups are not. I do not have this problem when I disable pf on >> the router, nor do I have problems when completing backups with other >> machines on my internal network. My setup looks like this: >> >> bacula director --------- router (client) >> 192.168.1.4 (fxp0) 192.168.1.2 (dc0) >> >> Communication takes place on ports 9102 and 9103. I captured this >> output from pflog0 after starting a backup: >> >> blacksheep# tcpdump -netttti pflog0 "( host blacksheep or blacklamb ) >> and ( port 9102 or port 9103 )" >> tcpdump: WARNING: pflog0: no IPv4 address assigned >> tcpdump: verbose output suppressed, use -v or -vv for full protocol >> decode listening on pflog0, link-type PFLOG (OpenBSD pflog file), >> capture size 96 bytes >> 2007-04-02 13:57:21.021122 rule 7/0(match): pass in on dc0: >> 192.168.1.4.52295 > 192.168.1.2.9102: S 2822997678:2822997678(0) win >> 65535 >> 2007-04-02 13:57:23.532037 rule 13/0(match): pass out on dc0: >> 192.168.1.2.64955 > 192.168.1.4.9103: S 2265048451:2265048451(0) win >> 65535 >> 2007-04-02 13:57:23.532323 rule 7/0(match): pass in on dc0: >> 192.168.1.4.9103 > 192.168.1.2.64955: S 3452777266:3452777266(0) ack >> 2265048452 win 65535 >> >> And the rules are: >> >> @7 pass in log on dc0 inet proto tcp from 192.168.1.0/24 to any >> modulate state queue(std_out, ack_out) >> > > This rule should have "flags S/SA" on it. > In my attempts to get ALTQ queuing working, I have found that adding flags here breaks it. However I am sure I am not approaching queuing correctly. I posted a bit about the problem here: http://www.freebsd.org/cgi/getmsg.cgi?fetch=4242+9504+/usr/local/www/db/text/2007/freebsd-pf/20070225.freebsd-pf After getting no response (which made me think my approach was way off), I attempted to redo my rule set and asked for help here: http://www.freebsd.org/cgi/getmsg.cgi?fetch=87780+93096+/usr/local/www/db/text/2007/freebsd-pf/20070401.freebsd-pf This post received one response regarding "keep state" and flags as well. I think I understand the concept about stateful inspections but I do not understand how to get queuing to work only on packets sent from my router to machines over the Internet. Seems that when I make "keep state" rules on inbound connections, the return traffic matches the state rules and thus never gets queued. I would LOVE to understand this better and would really appreciate any links to suggested reading. >> @13 pass out log on dc0 inet all >> >> Any ideas why Bacula would have such a problem? Other things to check? >> > > Can you turn on pf debugging via "pfctl -xm" and watch the console while > doing the backup? Also monitor "pfctl -si" for increasing counters - > esp. state-mismatch. > OK, I tried this and it's obvious to me that my pf configuration is not correct. I see tons of messages such as these: Apr 3 15:49:42 blacksheep kernel: pf_map_addr: selected address 66.205.146.210 Apr 3 15:49:46 blacksheep kernel: pf: BAD state: TCP 140.105.134.102:54934 140.105.134.102:54934 192.168.1.4:25 [lo=836336158 high=836336204 win=33304 modulator=0] [lo=1850627322 high=1850660626 win=46 modulator=0] 4:4 PA seq=836336158 ack=1850627322 len=185 ackskew=0 pkts=4:5 dir=in,fwd Apr 3 15:49:46 blacksheep kernel: pf: State failure on: 1 | However in searching the logs for messages containing the IP address of the router (192.168.1.2) while running a full backup that errored out after just 2.2 MB of data transfer, I found these entries: Apr 3 15:50:19 blacksheep kernel: pf: BAD state: TCP 192.168.1.2:50083 192.168.1.2:50083 192.168.1.4:9103 [lo=1243881036 high=1243914340 win=33304 modulator=0] [lo=3549637128 high=3549637922 win=33304 modulator=0] 4:4 A seq=3549637128 ack=1243881036 len=1448 ackskew=0 pkts=1081:1727 dir=out,rev Apr 3 15:50:19 blacksheep kernel: pf: State failure on: 1 | Apr 3 15:50:19 blacksheep kernel: pf: BAD state: TCP 192.168.1.2:50083 192.168.1.2:50083 192.168.1.4:9103 [lo=1243881036 high=1243914340 win=33304 modulator=0] [lo=3549638576 high=3549639370 win=33304 modulator=0] 4:4 A seq=3549638576 ack=1243881036 len=1448 ackskew=0 pkts=1082:1728 dir=out,rev I didn't monitor "pfctl -si" as you suggested. Obviously the counters would be increasing dramatically. So apparently state failure is my problem, likely caused by my misunderstanding of how to create a proper pf ruleset to achieve my goals. I've been through OpenBSD's pf FAQ numerous times. I've read Peter Hansteen's tutorial many times. However I still can't seem to get it through my thick head how to write a proper ruleset to get queuing to work the way I want. Thanks for any suggestions, Drew -- Be a Great Magician! Visit The Alchemist's Warehouse http://www.alchemistswarehouse.com