From owner-freebsd-pf@FreeBSD.ORG Fri Jan 8 22:51:00 2010 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E18041065670 for ; Fri, 8 Jan 2010 22:51:00 +0000 (UTC) (envelope-from j65nko@gmail.com) Received: from mail-ew0-f226.google.com (mail-ew0-f226.google.com [209.85.219.226]) by mx1.freebsd.org (Postfix) with ESMTP id 68BDB8FC08 for ; Fri, 8 Jan 2010 22:51:00 +0000 (UTC) Received: by ewy26 with SMTP id 26so18365018ewy.3 for ; Fri, 08 Jan 2010 14:50:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=jpp7C9h76D0VzraVrJ9RmoVjpY5d6vGA7ix2wwLwgGQ=; b=GE5z8yJXm65ZkVW7pOmNV6WOLCu5zPcaUo5wBcfCahm8uxyGiy4UnX92v/854ZSuie x354xSehzj+6m2/H80v3yeDx/F2syOqSg7Zki/oxEf5xyUQygzduUPGww6Evc+At5wsZ lRNzmGkFc4CqAx0hCjINtzStdPLwNTDFOS004= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=N5T86v9w63dCJjtar+B5qP+PHHuEffxwrdC8ebmbrrkZCECgpvg2SS4kEWMNlB7yIZ XqAh8AJSJ8MDfsOdtpSo2FKMwsTU+JJiDRJc0q37/EeoglmmNzQlhOKFmfZ250SiOBk/ DVBEz7tjcQ4e+b/JjFoHQdcKbWXETBHEa3704= MIME-Version: 1.0 Received: by 10.213.100.145 with SMTP id y17mr1949900ebn.27.1262991054592; Fri, 08 Jan 2010 14:50:54 -0800 (PST) In-Reply-To: References: <7731938b1001060923n5de4b511of07b8c63cff4e011@mail.gmail.com> <2cf1d0681001071216p6b516e9egcf7401f2b38e3c3d@mail.gmail.com> <19861fba1001071237ncc440d5u1ab280d2aaf0c72f@mail.gmail.com> <19861fba1001072018g115a0bccrf9510a38454cc9db@mail.gmail.com> Date: Fri, 8 Jan 2010 23:50:54 +0100 Message-ID: <19861fba1001081450w4d9096bdt83c8024b6e58af55@mail.gmail.com> From: J65nko To: "M. Keith Thompson" Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-pf@freebsd.org Subject: Re: ftp problem X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jan 2010 22:51:01 -0000 On Fri, Jan 8, 2010 at 9:50 PM, M. Keith Thompson wrote: > It looks like it was a tcp windowing problem. The command: "sysctl -w > sysctl net.inet.tcp.rfc1323=0" > fixed the problem. This only fixes a symptom. :) There is something wrong with your ruleset. >>> # Allow ftp >>> pass in quick on $ext_if proto tcp from any to $ext_IP port 21 keep state >>> pass in quick on $ext_if proto tcp from any to $ext_IP port > 49151 keep state >>> pass in quick on $ext_if proto tcp from any port > 10000 to $ext_IP >>> port 20 keep state With this active ftp rule you had before, the HP client could initiate and transfer over an active ftp data channel, while this rule does not allow this at all. There shouldn't be any active ftp datachannel communication possible because port 20 is the source port and not the destination port. I tried the following pf.conf , with a similar ftp-data rule , on a local FreeBSD 7.2 box in my network EXT_IF=fxp0 # net.inet.ip.portrange.hilast: 65535 # net.inet.ip.portrange.hifirst: 49152 PASSIVE='49152:65535' set skip on lo0 # default policy block log all # outgoing DNS requests pass out quick on $EXT_IF inet proto udp from $EXT_IF:network to any port domain keep state pass out quick on $EXT_IF inet proto tcp from $EXT_IF:network to any port domain keep state flags S/SA # incoming SSH pass in quick on $EXT_IF inet proto tcp from $EXT_IF:network to port 2022 keep state flags S/SA # incoming FTP # ftp command channel pass in quick on $EXT_IF inet proto tcp from any to $EXT_IF port ftp keep state flags S/SA # ftp data channel (passive) pass in quick on $EXT_IF inet proto tcp from any to $EXT_IF port $PASSIVE keep state flags S/SA # ftp data channel (active) # wrong direction!!! pass in quick on $EXT_IF inet proto tcp from any to $EXT_IF port ftp-data keep state flags S/SA Here I also erroneously used port 20 as destination port. An active ftp command to upload a file aborts with 200 EPRT command successful. 425 Can't build data connection: Operation not permitted. A pflog0 tcpdump shows the reason: tcpdump -eni pflog0 -s0 -v tcpdump: WARNING: pflog0: no IPv4 address assigned tcpdump: listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size 65535 bytes 22:16:42.551221 rule 0/0(match): block out on fxp0: (tos 0x8, ttl 64, id 2766, offset 0, flags [DF], proto TCP (6), length 60) 192.168.222.244.20 > 192.168.222.20.62316: S, cksum 0xc222 (correct), 488073960:488073960(0) win 65535 So this rule set blocks the first packet of the three-way TCP handshake. 192.168.222.244.20 > 192.168.222.20.62316. or server.20 > client.62316 Your previous ruleset somehow created state or allowed the server to initiate the data channel to the client somewhere else in your rule set. It didn't create state on the initial packet of the TCP handshake, where the TCP window size is negotiated. Just like the Daniel Hartmeier article states, this a caused tcp window scaling issue. Remember that in pf the last matching rule wins. You only can defeat this behaviour by using the 'quick' keyword. After adding the following corrected rule the transfer proceeds without any blockage # correct direction/source port pass out quick on $EXT_IF inet proto tcp from any port ftp-data to any keep state flags S/SA The rules as parsed are now: # pfctl -vvf /etc/pf.conf No ALTQ support in kernel ALTQ related functions disabled Loaded 696 passive OS fingerprints EXT_IF = "fxp0" PASSIVE = "49152:65535" set skip on { lo0 } @0 block drop log all @1 pass out quick on fxp0 inet proto udp from 192.168.222.0/24 to any port = domain keep state @2 pass out quick on fxp0 inet proto tcp from 192.168.222.0/24 to any port = domain flags S/SA keep state @3 pass in quick on fxp0 inet proto tcp from 192.168.222.0/24 to any port = down flags S/SA keep state @4 pass in quick on fxp0 inet proto tcp from any to 192.168.222.244 port = ftp flags S/SA keep state @5 pass in quick on fxp0 inet proto tcp from any to 192.168.222.244 port 49152:65535 flags S/SA keep state @6 pass in quick on fxp0 inet proto tcp from any to 192.168.222.244 port = ftp-data flags S/SA keep state @7 pass out quick on fxp0 inet proto tcp from any port = ftp-data to any flags S/SA keep state Rule 6 is the rule with the wrong direction/port specification. Rule 7 is the corrected one pflog0 doesn't show any errors: tcpdump: listening on pflog0, link-type PFLOG (OpenBSD pflog file), capture size 65535 bytes ^C 0 packets captured 0 packets received by filter 0 packets dropped by kernel And a quick state check still showed some remnants of the connections # pfctl -ss No ALTQ support in kernel ALTQ related functions disabled [snip] all tcp 192.168.222.244:21 <- 192.168.222.20:13497 FIN_WAIT_2:FIN_WAIT_2 all tcp 192.168.222.244:20 -> 192.168.222.20:55246 FIN_WAIT_2:FIN_WAIT_2 all tcp 192.168.222.244:20 -> 192.168.222.20:59354 FIN_WAIT_2:FIN_WAIT_2 The first state is the ftp command channel client:13497 > server:21 The last two are the active data channels: server:20 > client:55246, and server.20 > client.59354 Don't get confused, the last octet of the client IP (192.168.222.20) happens to be the same as the ftp-data port number. ;) A repeat of the transfer shows the following verbose state # pfctl -vvss all tcp 192.168.222.244:20 -> 192.168.222.20:64857 ESTABLISHED:ESTABLISHED [1427346948 + 65536] wscale 3 [200059941 + 37224] wscale 0 age 00:00:02, expires in 24:00:00, 13716:20572 pkts, 713240:30853060 bytes, rule 7 id: 4b47883900000033 creatorid: c576b1b2 By using this command you should be able to figure out which actual rule, here rule 7, creates state for the active ftp data channel. The same when the transfer has finished all tcp 192.168.222.244:20 -> 192.168.222.20:64857 FIN_WAIT_2:FIN_WAIT_2 [1427346950 + 65534] wscale 3 [259693437 + 66608] wscale 0 age 00:00:59, expires in 00:00:38, 41209:61758 pkts, 2142876:92628227 bytes, rule 7 id: 4b47883900000033 creatorid: c576b1b2 Happy rule/state hunting Adriaan all tcp 192.168.222.244:20 -> 192.168.222.20:64857 FIN_WAIT_2:FIN_WAIT_2 [1427346950 + 65534] wscale 3 [259693437 + 66608] wscale 0 age 00:00:59, expires in 00:00:38, 41209:61758 pkts, 2142876:92628227 bytes, rule 7 id: 4b47883900000033 creatorid: c576b1b2