From owner-freebsd-pf@freebsd.org Tue Jan 5 23:55:58 2016 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3B16EA639E3 for ; Tue, 5 Jan 2016 23:55:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1F1581E40 for ; Tue, 5 Jan 2016 23:55:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id u05Ntv3x030091 for ; Tue, 5 Jan 2016 23:55:57 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-pf@FreeBSD.org Subject: [Bug 198868] pf brakes tcp checksum if enabled for ue adapter Date: Tue, 05 Jan 2016 23:55:58 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: dewayne@heuristicsystems.com.au X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: kp@freebsd.org X-Bugzilla-Flags: mfc-stable9+ mfc-stable10+ X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Jan 2016 23:55:58 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D198868 dewayne@heuristicsystems.com.au changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |dewayne@heuristicsystems.co | |m.au --- Comment #9 from dewayne@heuristicsystems.com.au --- (In reply to Kristof Provost from comment #8) Hi Kristof, I've just inserted two, recently purchased new, usb interface cards from J5Create into a 10.2Stable box. They are recognised as follows: # Card 1: This is a J5Create JUE125 USB2.0 Ethernet adapter (that I bought = new a few weeks ago) ugen0.4: at usbus0 axe0: on usbus0 miibus0: on axe0 ukphy0: PHY 16 on miibus0 ukphy0: none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-f= low ue0: on axe0 ue0: Ethernet address: 00:05:1b:a4:9f:1c ue0: link state changed to DOWN ue0: link state changed to UP ue0: flags=3D8802 metric 0 mtu 1500 options=3D8000b ether 00:05:1b:a4:9f:1c nd6 options=3D29 media: Ethernet autoselect (100baseTX ) status: active # Card 2: J5Create JUE130 USB3.0 Gigabit Ethernet adapter (as above) ugen0.5: at usbus0 axge0: on usbus0 miibus1: on axge0 rgephy0: PHY 3 on miibus1 rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow ue1: on axge0 ue1: Ethernet address: 00:05:1b:a1:79:76 ue1: link state changed to DOWN ue1: flags=3D8802 metric 0 mtu 1500 options=3D8000b ether 00:05:1b:a1:79:76 nd6 options=3D29 media: Ethernet autoselect (none) status: no carrier The following demonstrates the difference between em1: and ue0 that is axe0: The source box running pf and 10.2Stable (FreeBSD hathor 10.2-STABLE FreeBSD 10.2-STABLE #0 r293123M: Mon Jan 4 17:36:12 AEDT 2016=20=20=20=20 root@hathor:/usr/obj/prod/100201/D/K8/pd3/src/sys/hqdev-amd64-smp-vga=20=20= =20=20=20=20=20=20=20=20 amd64 1002505 1002505) Source IP: 10.0.7.91 (and jail 10.0.7.92) Destination: 10.0.7.6 (Running FreeBSD92 and ipfw) Using a working interface (em1) to a box one hop away and using incorrect s= sh key, I've listed the first six lines from tcpdump for a negotation between= =20 A) Using em1 interface between a base system (10.0.7.91) and a remote node B) Using em1 interface from within a jail (10.0.7.92)=20 C) Using ue0 between base 10.0.7.91=20 D) Using ue0 between jail 10.0.7.92=20 A)=20 # tcpdump -s1518 -vni em1 host 10.0.7.6 and port 22 tcpdump: listening on em1, link-type EN10MB (Ethernet), capture size 1518 b= ytes 09:19:25.943098 IP (tos 0x0, ttl 48, id 26320, offset 0, flags [DF], proto = TCP (6), length 60) 10.0.7.91.52418 > 10.0.7.6.22: Flags [S], cksum 0x228f (incorrect -> 0xef75), seq 2076369872, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 90899540 ecr 0], length 0 09:19:25.943317 IP (tos 0x0, ttl 64, id 45550, offset 0, flags [DF], proto = TCP (6), length 60) 10.0.7.6.22 > 10.0.7.91.52418: Flags [S.], cksum 0xef96 (correct), seq 1751036782, ack 2076369873, win 65535, options [mss 1460,nop,wscale 5,sackO= K,TS val 324653224 ecr 90899540], length 0 09:19:25.943354 IP (tos 0x0, ttl 48, id 48994, offset 0, flags [DF], proto = TCP (6), length 52) 10.0.7.91.52418 > 10.0.7.6.22: Flags [.], cksum 0x2287 (incorrect -> 0x1653), ack 1, win 2058, options [nop,nop,TS val 90899544 ecr 324653224], length 0 09:19:25.943574 IP (tos 0x0, ttl 64, id 45551, offset 0, flags [DF], proto = TCP (6), length 52) 10.0.7.6.22 > 10.0.7.91.52418: Flags [.], cksum 0x0e48 (correct), ack 1, win 4117, options [nop,nop,TS val 324653224 ecr 90899544], length 0 09:19:25.943862 IP (tos 0x0, ttl 48, id 25543, offset 0, flags [DF], proto = TCP (6), length 81) 10.0.7.91.52418 > 10.0.7.6.22: Flags [P.], cksum 0x22a4 (incorrect -> 0x1540), seq 1:30, ack 1, win 2058, options [nop,nop,TS val 90899544 ecr 324653224], length 29 09:19:26.043316 IP (tos 0x0, ttl 64, id 45552, offset 0, flags [DF], proto = TCP (6), length 52) 10.0.7.6.22 > 10.0.7.91.52418: Flags [.], cksum 0x0dc8 (correct), ack 3= 0, win 4116, options [nop,nop,TS val 324653324 ecr 90899544], length 0 And from a jail using the same interface to a jail B) # tcpdump -s1518 -vni em1 host 10.0.7.6 and port 22 tcpdump: listening on em1, link-type EN10MB (Ethernet), capture size 1518 b= ytes 09:24:12.271961 IP (tos 0x0, ttl 48, id 24663, offset 0, flags [DF], proto = TCP (6), length 60) 10.0.7.92.11369 > 10.0.7.6.22: Flags [S], cksum 0x2290 (incorrect -> 0x81ba), seq 3957334854, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 91185872 ecr 0], length 0 09:24:12.272467 IP (tos 0x0, ttl 64, id 45639, offset 0, flags [DF], proto = TCP (6), length 60) 10.0.7.6.22 > 10.0.7.92.11369: Flags [S.], cksum 0x1a60 (correct), seq 2844259951, ack 3957334855, win 65535, options [mss 1460,nop,wscale 5,sackO= K,TS val 3406822466 ecr 91185872], length 0 09:24:12.272488 IP (tos 0x0, ttl 48, id 39982, offset 0, flags [DF], proto = TCP (6), length 52) 10.0.7.92.11369 > 10.0.7.6.22: Flags [.], cksum 0x2288 (incorrect -> 0x4120), ack 1, win 2058, options [nop,nop,TS val 91185872 ecr 3406822466], length 0 09:24:12.272710 IP (tos 0x0, ttl 64, id 45640, offset 0, flags [DF], proto = TCP (6), length 52) 10.0.7.6.22 > 10.0.7.92.11369: Flags [.], cksum 0x3915 (correct), ack 1, win 4117, options [nop,nop,TS val 3406822466 ecr 91185872], length 0 09:24:12.273335 IP (tos 0x0, ttl 48, id 54149, offset 0, flags [DF], proto = TCP (6), length 81) 10.0.7.92.11369 > 10.0.7.6.22: Flags [P.], cksum 0x22a5 (incorrect -> 0x400d), seq 1:30, ack 1, win 2058, options [nop,nop,TS val 91185872 ecr 3406822466], length 29 09:24:12.373101 IP (tos 0x0, ttl 64, id 45641, offset 0, flags [DF], proto = TCP (6), length 52) 10.0.7.6.22 > 10.0.7.92.11369: Flags [.], cksum 0x3894 (correct), ack 3= 0, win 4116, options [nop,nop,TS val 3406822567 ecr 91185872], length 0 Disconnecting the cable from that interface and inserting into ue0, changing the external interface in pf and restarting, ue0 has this C)=20 # tcpdump -s1518 -vni ue0 host 10.0.7.6 and port 22 tcpdump: listening on ue0, link-type EN10MB (Ethernet), capture size 1518 b= ytes 09:33:13.585464 IP (tos 0x0, ttl 48, id 641, offset 0, flags [DF], proto TCP (6), length 60, bad cksum 0 (->25db)!) 10.0.7.91.29122 > 10.0.7.6.22: Flags [S], cksum 0x228f (incorrect -> 0xdaf0), seq 1199433362, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 91727184 ecr 0], length 0 09:33:13.587153 IP (tos 0x0, ttl 64, id 45967, offset 0, flags [DF], proto = TCP (6), length 60) 10.0.7.6.22 > 10.0.7.91.29122: Flags [S.], cksum 0xe549 (correct), seq 3082007530, ack 1199433363, win 65535, options [mss 1460,nop,wscale 5,sackO= K,TS val 4112225500 ecr 91727184], length 0 09:33:13.587175 IP (tos 0x0, ttl 48, id 5810, offset 0, flags [DF], proto T= CP (6), length 52, bad cksum 0 (->11b2)!) 10.0.7.91.29122 > 10.0.7.6.22: Flags [.], cksum 0x2287 (incorrect -> 0x0c06), ack 1, win 2058, options [nop,nop,TS val 91727188 ecr 4112225500], length 0 09:33:13.587504 IP (tos 0x0, ttl 48, id 39472, offset 0, flags [DF], proto = TCP (6), length 81, bad cksum 0 (->8e16)!) 10.0.7.91.29122 > 10.0.7.6.22: Flags [P.], cksum 0x22a4 (incorrect -> 0x0af3), seq 1:30, ack 1, win 2058, options [nop,nop,TS val 91727188 ecr 4112225500], length 29 09:33:13.588820 IP (tos 0x0, ttl 64, id 45968, offset 0, flags [DF], proto = TCP (6), length 52) 10.0.7.6.22 > 10.0.7.91.29122: Flags [.], cksum 0x03f9 (correct), ack 1, win 4117, options [nop,nop,TS val 4112225502 ecr 91727188], length 0 09:33:13.688305 IP (tos 0x0, ttl 64, id 45969, offset 0, flags [DF], proto = TCP (6), length 52) 10.0.7.6.22 > 10.0.7.91.29122: Flags [.], cksum 0x0379 (correct), ack 3= 0, win 4116, options [nop,nop,TS val 4112225602 ecr 91727188], length 0 And from a jail D)=20 # tcpdump -s1518 -vni ue0 host 10.0.7.6 and port 22 tcpdump: listening on ue0, link-type EN10MB (Ethernet), capture size 1518 b= ytes 09:34:25.701515 IP (tos 0x0, ttl 48, id 7850, offset 0, flags [DF], proto T= CP (6), length 60, bad cksum 0 (->9b1)!) 10.0.7.92.58725 > 10.0.7.6.22: Flags [S], cksum 0x2290 (incorrect -> 0x9b31), seq 2136629531, win 65535, options [mss 1460,nop,wscale 6,sackOK,TS val 91799300 ecr 0], length 0 09:34:25.703204 IP (tos 0x0, ttl 64, id 46014, offset 0, flags [DF], proto = TCP (6), length 60) 10.0.7.6.22 > 10.0.7.92.58725: Flags [S.], cksum 0x47ef (correct), seq 2091274476, ack 2136629532, win 65535, options [mss 1460,nop,wscale 5,sackO= K,TS val 269251987 ecr 91799300], length 0 09:34:25.703224 IP (tos 0x0, ttl 48, id 4000, offset 0, flags [DF], proto T= CP (6), length 52, bad cksum 0 (->18c3)!) 10.0.7.92.58725 > 10.0.7.6.22: Flags [.], cksum 0x2288 (incorrect -> 0x6eab), ack 1, win 2058, options [nop,nop,TS val 91799304 ecr 269251987], length 0 09:34:25.703539 IP (tos 0x0, ttl 48, id 41112, offset 0, flags [DF], proto = TCP (6), length 81, bad cksum 0 (->87ad)!) 10.0.7.92.58725 > 10.0.7.6.22: Flags [P.], cksum 0x22a5 (incorrect -> 0x6d98), seq 1:30, ack 1, win 2058, options [nop,nop,TS val 91799304 ecr 269251987], length 29 09:34:25.704871 IP (tos 0x0, ttl 64, id 46015, offset 0, flags [DF], proto = TCP (6), length 52) 10.0.7.6.22 > 10.0.7.92.58725: Flags [.], cksum 0x669e (correct), ack 1, win 4117, options [nop,nop,TS val 269251989 ecr 91799304], length 0 09:34:25.804852 IP (tos 0x0, ttl 64, id 46016, offset 0, flags [DF], proto = TCP (6), length 52) 10.0.7.6.22 > 10.0.7.92.58725: Flags [.], cksum 0x661e (correct), ack 3= 0, win 4116, options [nop,nop,TS val 269252089 ecr 91799304], length 0 These dumps are from a failed ssh connection between two nodes so we can see the handshaking experience. I've compressed the content (attached) and pro= vide an abridged content for the PR. I hope that this provides some clarification that this doesn't apply to old interfaces, nor complex vm setups ;) Happy to assist further, as I was about to change our firewall from ipfw to= pf but we do use ue (devices)... Regards, Dewayne. --=20 You are receiving this mail because: You are on the CC list for the bug.=