From owner-freebsd-pf@FreeBSD.ORG Wed Jul 28 20:04:45 2010 Return-Path: Delivered-To: freebsd-pf@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F1AE3106566C for ; Wed, 28 Jul 2010 20:04:44 +0000 (UTC) (envelope-from justin@sk1llz.net) Received: from sed.awknet.com (sed.awknet.com [69.42.208.18]) by mx1.freebsd.org (Postfix) with ESMTP id DE8528FC1A for ; Wed, 28 Jul 2010 20:04:44 +0000 (UTC) Received: from [192.168.1.64] (99-118-214-35.lightspeed.irvnca.sbcglobal.net [99.118.214.35]) by sed.awknet.com (Postfix) with ESMTP id 8F69F10824AC for ; Wed, 28 Jul 2010 20:04:44 +0000 (UTC) Message-ID: <4C508D5A.6000600@sk1llz.net> Date: Wed, 28 Jul 2010 13:04:42 -0700 From: Justin User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.11) Gecko/20100711 Thunderbird/3.0.6 MIME-Version: 1.0 To: freebsd-pf@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: pf synproxy X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Jul 2010 20:04:45 -0000 Logged to files and dumped; Outside: 19:58:09.571810 IP (tos 0x0, ttl 118, id 12726, offset 0, flags [DF], proto TCP (6), length 52) REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [S], cksum 0xb15f (correct), se q 3641874856, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], length 0 19:58:09.571835 IP (tos 0x10, ttl 64, id 6840, offset 0, flags [DF], proto TCP ( 6), length 44) TARGET_HOST.80 > REMOTE_CLIENT.56270: Flags [S.], cksum 0xa352 (correct), s eq 4110180879, ack 3641874857, win 0, options [mss 1460], length 0 19:58:09.583132 IP (tos 0x0, ttl 118, id 12727, offset 0, flags [DF], proto TCP (6), length 40) REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [.], cksum 0xc01e (correct), ac k 4110180880, win 64240, length 0 19:58:09.593498 IP (tos 0x0, ttl 118, id 12728, offset 0, flags [DF], proto TCP (6), length 40) REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [.], cksum 0xc01e (correct), ac k 4110180880, win 64240, length 0 19:58:09.603117 IP (tos 0x0, ttl 118, id 12729, offset 0, flags [DF], proto TCP (6), length 40) REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [.], cksum 0xc01e (correct), ac k 4110180880, win 64240, length 0 19:58:09.617101 IP (tos 0x0, ttl 118, id 12730, offset 0, flags [DF], proto TCP (6), length 40) REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [.], cksum 0xc01e (correct), ac k 4110180880, win 64240, length 0 19:58:09.628812 IP (tos 0x0, ttl 118, id 12731, offset 0, flags [DF], proto TCP (6), length 40) REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [.], cksum 0xc01e (correct), ac k 4110180880, win 64240, length 0 19:58:09.641101 IP (tos 0x0, ttl 118, id 12732, offset 0, flags [DF], proto TCP (6), length 40) REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [.], cksum 0xc01e (correct), ac k 4110180880, win 64240, length 0 19:58:09.652398 IP (tos 0x0, ttl 118, id 12733, offset 0, flags [DF], proto TCP (6), length 40) REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [.], cksum 0xc01e (correct), ac k 4110180880, win 64240, length 0 19:58:09.662468 IP (tos 0x0, ttl 118, id 12734, offset 0, flags [DF], proto TCP (6), length 40) REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [.], cksum 0xc01e (correct), ac k 4110180880, win 64240, length 0 19:58:09.674054 IP (tos 0x0, ttl 118, id 12735, offset 0, flags [DF], proto TCP (6), length 40) Inside: 19:58:09.583144 IP (tos 0x10, ttl 64, id 6841, offset 0, flags [DF], proto TCP ( 6), length 44) REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [S], cksum 0x2a53 (correct), se q 2026752522, win 0, options [mss 1460], length 0 19:58:09.593505 IP (tos 0x10, ttl 64, id 6842, offset 0, flags [DF], proto TCP ( 6), length 44) REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [S], cksum 0x2a53 (correct), se q 2026752522, win 0, options [mss 1460], length 0 19:58:09.603123 IP (tos 0x10, ttl 64, id 6843, offset 0, flags [DF], proto TCP ( 6), length 44) REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [S], cksum 0x2a53 (correct), se q 2026752522, win 0, options [mss 1460], length 0 19:58:09.617107 IP (tos 0x10, ttl 64, id 6844, offset 0, flags [DF], proto TCP ( 6), length 44) REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [S], cksum 0x2a53 (correct), se q 2026752522, win 0, options [mss 1460], length 0 19:58:09.628819 IP (tos 0x10, ttl 64, id 6845, offset 0, flags [DF], proto TCP ( 6), length 44) REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [S], cksum 0x2a53 (correct), se q 2026752522, win 0, options [mss 1460], length 0 19:58:09.641108 IP (tos 0x10, ttl 64, id 6846, offset 0, flags [DF], proto TCP ( 6), length 44) REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [S], cksum 0x2a53 (correct), se q 2026752522, win 0, options [mss 1460], length 0 19:58:09.709553 IP (tos 0x10, ttl 64, id 6852, offset 0, flags [DF], proto TCP ( 6), length 44) REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [S], cksum 0x2a53 (correct), se q 2026752522, win 0, options [mss 1460], length 0 19:58:09.719427 IP (tos 0x10, ttl 64, id 6853, offset 0, flags [DF], proto TCP ( 6), length 44) REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [S], cksum 0x2a53 (correct), se q 2026752522, win 0, options [mss 1460], length 0 19:58:09.729501 IP (tos 0x10, ttl 64, id 6854, offset 0, flags [DF], proto TCP ( 6), length 44) REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [S], cksum 0x2a53 (correct), se q 2026752522, win 0, options [mss 1460], length 0 19:58:09.739096 IP (tos 0x10, ttl 64, id 6855, offset 0, flags [DF], proto TCP ( 6), length 44) REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [S], cksum 0x2a53 (correct), se q 2026752522, win 0, options [mss 1460], length 0 19:58:09.749464 IP (tos 0x10, ttl 64, id 6856, offset 0, flags [DF], proto TCP ( 6), length 44) REMOTE_CLIENT.56270 > TARGET_HOST.80: Flags [S], cksum 0x2a53 (correct), se q 2026752522, win 0, options [mss 1460], length 0 Some people on the OpenBSD list suggested it had something to do with synproxy only working on the interface that has a default gateway - I've tried all variations of em0/1 being the default route out, still no changes. em0 tcp TARGET_HOST:80 <- REMOTE_CLIENT:56273 PROXY:DST [0 + 4143199136] [2614600686 + 3007453693] age 00:00:10, expires in 00:01:50, 0:0 pkts, 0:0 bytes, rule 1 id: 4c4f86b6000012a8 creatorid: e7945cd2 Never gets beyond that. On 7/28/2010 12:06 AM, Daniel Hartmeier wrote: > On Tue, Jul 27, 2010 at 07:24:56PM -0700, Justin wrote: > >> - tcpdumps showing the initial connect attempt (logs below were >> furhter along the process); >> >> external: >> 02:21:25.595977 IP (tos 0x0, ttl 118, id 10020, offset 0, flags [DF], >> proto TCP >> (6), length 52) >> REMOTE_VIEWING_HOST.53782> CLIENT_DESTINATION_IP.80: Flags [S], >> cksum 0x537b (correct), se >> q 2569879850, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], >> length 0 >> 02:21:25.596000 IP (tos 0x10, ttl 64, id 27957, offset 0, flags [DF], >> proto TCP >> (6), length 44) >> CLIENT_DESTINATION_IP.80> REMOTE_VIEWING_HOST.53782: Flags [S.], >> cksum 0x7920 (correct), s >> eq 3819585455, ack 2569879851, win 0, options [mss 1460], length 0 >> 02:21:25.605825 IP (tos 0x0, ttl 118, id 10021, offset 0, flags [DF], >> proto TCP >> (6), length 40) >> REMOTE_VIEWING_HOST.53782> CLIENT_DESTINATION_IP.80: Flags [.], >> cksum 0x95ec (correct), ac >> k 3819585456, win 64240, length 0 >> 02:21:25.615953 IP (tos 0x0, ttl 118, id 10022, offset 0, flags [DF], >> proto TCP >> (6), length 40) > The TCP handshake with the client makes sense. > > The client's SYN offers wscale and sack, pf's SYN+ACK has IP tos 0x10 > and neither wscale nor sack, and win 0. > >> internal: >> 02:22:34.717332 IP (tos 0x0, ttl 118, id 22015, offset 0, flags [DF], >> proto TCP >> (6), length 52) >> REMOTE_VIEWING_HOST.53786> CLIENT_DESTINATION_IP.80: Flags [S], >> cksum 0x84ee (correct), se >> q 1816476827, win 8192, options [mss 1460,nop,wscale 2,nop,nop,sackOK], >> length 0 > This doesn't seem to be captured on the internal interface, at least > it is not the server handshake replayed by pf's synproxy. > Note the (random) client source port 53786 doesn't match 53782 of the > external capture above. > > Instead, it looks like ANOTHER connection captured on the external > interface. The SYN has IP tos 0x0 and offers wscale and sack, so > this CANNOT be a packet generated by pf's synproxy... > > What we want to see is the two TCP handshakes related to the same > client connection, once on the interface towards the client, once > on the interface towards the server. The client source port is the > same. > > Ideally, tcpdump on both interface, writing to two files with > tcpdump -w. Reproduce the issue. Find out what client source port > was used for that connection. Then tcpdump -r and extract all packets > of that port (src or dst) in both files... > > And the pfctl -vvss output should contain the state with that port > as well. > > There is precisely one external and one internal interface, and > pf is the only connection between these two networks, right? > If the client's SYN can somehow reach the server bypassing pf, > the synproxy will not work, there'd be two handshakes to the server > using different sequence numbers, the server would accept the first, > ignore the other, confusion would ensue ;) > > Daniel