Date: Fri, 8 Jan 2010 17:05:16 -0600 From: "M.Keith.Thompson " <m.keith.thompson@gmail.com> To: J65nko <j65nko@gmail.com> Cc: "freebsd-pf@freebsd.org" <freebsd-pf@freebsd.org> Subject: Re: ftp problem Message-ID: <BC848642-A55F-4EAD-9BA2-4DD04460F177@GMail.com> In-Reply-To: <19861fba1001081450w4d9096bdt83c8024b6e58af55@mail.gmail.com> References: <fc2243911001060809m5417b810vf2ed40c8a969fb5f@mail.gmail.com> <7731938b1001060923n5de4b511of07b8c63cff4e011@mail.gmail.com> <fc2243911001060957n16f906f7m703c696b970e8c3c@mail.gmail.com> <2cf1d0681001071216p6b516e9egcf7401f2b38e3c3d@mail.gmail.com> <19861fba1001071237ncc440d5u1ab280d2aaf0c72f@mail.gmail.com> <fc2243911001071319q4bbbc4a6o6b58031bba59c39b@mail.gmail.com> <19861fba1001072018g115a0bccrf9510a38454cc9db@mail.gmail.com> <fc2243911001080551n251851b5kb56a6c5f216c4a14@mail.gmail.com> <fc2243911001081250n4471d9d8kf5bcf691d2cbd50d@mail.gmail.com> <19861fba1001081450w4d9096bdt83c8024b6e58af55@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Remember the problem is only with one client and files over 98K. Hundreds of other clients transfer multi-megabyte files no problem. On Jan 8, 2010, at 4:50 PM, J65nko <j65nko@gmail.com> wrote: > On Fri, Jan 8, 2010 at 9:50 PM, M. Keith Thompson > <m.keith.thompson@gmail.com> wrote: >> It looks like it was a tcp windowing problem. The command: "sysctl >> -w >> sysctl net.inet.tcp.rfc1323=0" >> fixed the problem. > > This only fixes a symptom. :) There is something wrong with your > ruleset. > >>>> # Allow ftp >>>> pass in quick on $ext_if proto tcp from any to $ext_IP port 21 >>>> keep state >>>> pass in quick on $ext_if proto tcp from any to $ext_IP port > >>>> 49151 keep state >>>> pass in quick on $ext_if proto tcp from any port > 10000 to $ext_IP >>>> port 20 keep state > > With this active ftp rule you had before, the HP client could initiate > and transfer over an active ftp data channel, > while this rule does not allow this at all. > There shouldn't be any active ftp datachannel communication possible > because port 20 is the source port and not the destination port. > > I tried the following pf.conf , with a similar ftp-data rule , on a > local FreeBSD 7.2 box in my network > > EXT_IF=fxp0 > > # net.inet.ip.portrange.hilast: 65535 > # net.inet.ip.portrange.hifirst: 49152 > PASSIVE='49152:65535' > > set skip on lo0 > > # default policy > block log all > > # outgoing DNS requests > > pass out quick on $EXT_IF inet proto udp from $EXT_IF:network to any > port domain keep state > pass out quick on $EXT_IF inet proto tcp from $EXT_IF:network to any > port domain keep state flags S/SA > > # incoming SSH > > pass in quick on $EXT_IF inet proto tcp from $EXT_IF:network to port > 2022 keep state flags S/SA > > # incoming FTP > > # ftp command channel > pass in quick on $EXT_IF inet proto tcp from any to $EXT_IF port ftp > keep state flags S/SA > > # ftp data channel (passive) > pass in quick on $EXT_IF inet proto tcp from any to $EXT_IF port > $PASSIVE keep state flags S/SA > > # ftp data channel (active) > # wrong direction!!! > pass in quick on $EXT_IF inet proto tcp from any to $EXT_IF port > ftp-data keep state flags S/SA > > Here I also erroneously used port 20 as destination port. > > An active ftp command to upload a file aborts with > > 200 EPRT command successful. > 425 Can't build data connection: Operation not permitted. > > A pflog0 tcpdump shows the reason: > > tcpdump -eni pflog0 -s0 -v > tcpdump: WARNING: pflog0: no IPv4 address assigned > tcpdump: listening on pflog0, link-type PFLOG (OpenBSD pflog file), > capture size 65535 bytes > 22:16:42.551221 rule 0/0(match): block out on fxp0: (tos 0x8, ttl 64, > id 2766, offset 0, flags [DF], proto TCP (6), length 60) > 192.168.222.244.20 > 192.168.222.20.62316: S, cksum 0xc222 (correct), > 488073960:488073960(0) win 65535 <mss 1460,nop,wscale > 3,sackOK,timestamp 6278055 0> > > So this rule set blocks the first packet of the three-way TCP > handshake. > > 192.168.222.244.20 > 192.168.222.20.62316. > or > server.20 > client.62316 > > Your previous ruleset somehow created state or allowed the server to > initiate the data channel to the client somewhere else in your rule > set. It didn't create state on the initial packet of the TCP > handshake, where the TCP window size is negotiated. Just like the > Daniel Hartmeier article states, this a caused tcp window scaling > issue. > > Remember that in pf the last matching rule wins. > You only can defeat this behaviour by using the 'quick' keyword. > > After adding the following corrected rule the transfer proceeds > without any blockage > > # correct direction/source port > pass out quick on $EXT_IF inet proto tcp from any port ftp-data to > any keep state flags S/SA > > The rules as parsed are now: > > # pfctl -vvf /etc/pf.conf > No ALTQ support in kernel > ALTQ related functions disabled > Loaded 696 passive OS fingerprints > EXT_IF = "fxp0" > PASSIVE = "49152:65535" > set skip on { lo0 } > @0 block drop log all > @1 pass out quick on fxp0 inet proto udp from 192.168.222.0/24 to any > port = domain keep state > @2 pass out quick on fxp0 inet proto tcp from 192.168.222.0/24 to any > port = domain flags S/SA keep state > @3 pass in quick on fxp0 inet proto tcp from 192.168.222.0/24 to any > port = down flags S/SA keep state > @4 pass in quick on fxp0 inet proto tcp from any to 192.168.222.244 > port = ftp flags S/SA keep state > @5 pass in quick on fxp0 inet proto tcp from any to 192.168.222.244 > port 49152:65535 flags S/SA keep state > @6 pass in quick on fxp0 inet proto tcp from any to 192.168.222.244 > port = ftp-data flags S/SA keep state > @7 pass out quick on fxp0 inet proto tcp from any port = ftp-data to > any flags S/SA keep state > > Rule 6 is the rule with the wrong direction/port specification. > Rule 7 is the corrected one > > pflog0 doesn't show any errors: > tcpdump: listening on pflog0, link-type PFLOG (OpenBSD pflog file), > capture size 65535 bytes > ^C > 0 packets captured > 0 packets received by filter > 0 packets dropped by kernel > > And a quick state check still showed some remnants of the connections > > # pfctl -ss > No ALTQ support in kernel > ALTQ related functions disabled > [snip] > all tcp 192.168.222.244:21 <- 192.168.222.20:13497 > FIN_WAIT_2:FIN_WAIT_2 > all tcp 192.168.222.244:20 -> 192.168.222.20:55246 > FIN_WAIT_2:FIN_WAIT_2 > all tcp 192.168.222.244:20 -> 192.168.222.20:59354 > FIN_WAIT_2:FIN_WAIT_2 > > The first state is the ftp command channel client:13497 > server:21 > The last two are the active data channels: server:20 > client:55246, > and server.20 > client.59354 > > Don't get confused, the last octet of the client IP (192.168.222.20) > happens to be the same as > the ftp-data port number. ;) > > A repeat of the transfer shows the following verbose state > > # pfctl -vvss > > all tcp 192.168.222.244:20 -> 192.168.222.20:64857 > ESTABLISHED:ESTABLISHED > [1427346948 + 65536] wscale 3 [200059941 + 37224] wscale 0 > age 00:00:02, expires in 24:00:00, 13716:20572 pkts, > 713240:30853060 bytes, rule 7 > id: 4b47883900000033 creatorid: c576b1b2 > > By using this command you should be able to figure out which actual > rule, here rule 7, creates state for the active ftp data channel. > > The same when the transfer has finished > > all tcp 192.168.222.244:20 -> 192.168.222.20:64857 > FIN_WAIT_2:FIN_WAIT_2 > [1427346950 + 65534] wscale 3 [259693437 + 66608] wscale 0 > age 00:00:59, expires in 00:00:38, 41209:61758 pkts, > 2142876:92628227 bytes, rule 7 > id: 4b47883900000033 creatorid: c576b1b2 > > Happy rule/state hunting > > Adriaan > > > all tcp 192.168.222.244:20 -> 192.168.222.20:64857 > FIN_WAIT_2:FIN_WAIT_2 > [1427346950 + 65534] wscale 3 [259693437 + 66608] wscale 0 > age 00:00:59, expires in 00:00:38, 41209:61758 pkts, > 2142876:92628227 bytes, rule 7 > id: 4b47883900000033 creatorid: c576b1b2
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?BC848642-A55F-4EAD-9BA2-4DD04460F177>