From owner-freebsd-net@freebsd.org Tue Dec 1 11:22:42 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0997BA3C035 for ; Tue, 1 Dec 2015 11:22:42 +0000 (UTC) (envelope-from elof2@sentor.se) Received: from smtp-out.sentor.se (smtp-out.sentor.se [176.124.225.2]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BB47C1DBC for ; Tue, 1 Dec 2015 11:22:41 +0000 (UTC) (envelope-from elof2@sentor.se) Received: from localhost (localhost [127.0.0.1]) by farmermaggot.shire.sentor.se (Postfix) with ESMTP id 87738B61D233; Tue, 1 Dec 2015 12:22:38 +0100 (CET) Date: Tue, 1 Dec 2015 12:22:38 +0100 (CET) From: "elof2@sentor.se" To: "Alexander V. Chernikov" cc: freebsd-net Subject: Re: ngrep/ixgbe bpf bug In-Reply-To: <944961448953115@web11h.yandex.ru> Message-ID: References: <944961448953115@web11h.yandex.ru> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=koi8-r; format=flowed Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Dec 2015 11:22:42 -0000 Yes, 100% of the traffic is vlan-tagged, but I get the same results with: ngrep -d ix1 "q" vlan no matches If I invert the test to show all packets that do not contain "foobar", it still matches 0 packets: ngrep -d ix1 -v "foobar" ix1: no IPv4 address assigned: Can't assign requested address interface: ix1 don't match: foobar ^Cexit 263567 received, 0 dropped So 263567 vlan-tagged packets are received, 0 dropped. There are "q":s in there, and I promise that they don't all contain "foobar". :) If I add -R, to not drop privileges, I get the same result. 'tcpdump -i ix1 -lnes0' show packets just as it should: 12:14:02.113842 28:c0:da:db:c7:40 > 00:e0:20:11:0a:95, ethertype 802.1Q (0x8100), length 139: vlan 123, p 0, ethertype IPv4, 10.123.123.123.993 > 10.321.321.321.50904: Flags [P.], seq 2632602368:2632602437, ack 2214392113, win 8316, options [nop,nop,TS val 3824950099 ecr 179534445], length 69 ...and so on... Everything looks just as expected. #ifconfig ix1 ix1: flags=488c3 metric 0 mtu 1500 options=8403bb ether 0c:c4:7a:58:e2:3d nd6 options=29 media: Ethernet autoselect (10Gbase-T ) status: active Could ngrep fail because of monitor mode? #ifconfig ix1 -monitor #ifconfig ix1 ix1: flags=88c3 metric 0 mtu 1500 options=8403bb ether 0c:c4:7a:58:e2:3d nd6 options=29 media: Ethernet autoselect (10Gbase-T ) status: active #ngrep -d ix1 -v "foobar" ix1: no IPv4 address assigned: Can't assign requested address interface: ix1 don't match: foobar ^Cexit 157082 received, 0 dropped Nope, same-same. Zero BPF matches. #netstat -B Pid Netif Flags Recv Drop Match Sblen Hblen Command 12293 ix1 p--s--- 157082 0 0 0 0 ngrep ^^^^^^ /Elof On Tue, 1 Dec 2015, Alexander V. Chernikov wrote: > Do you have vlans on top of ixgbe? > And actually I wonder what does tcpdump show for the same expression. > ( and tcpdump -i ixX -lnes0 might provide good traces on what is going on) > > 30.11.2015, 19:09, "elof2@sentor.se" : >> No one has a theory? >> >> /Elof >> >> On Thu, 5 Nov 2015, elof2@sentor.se wrote: >> >>> šHi all! >>> >>> šWhy don't ngrep work on ix interfaces? >>> >>> šIt shows nice values if I sniff on other interfaces, e.g. igb0 and >>> šbridge0: >>> >>> š# ngrep -d igb0 "q" ip >>> šI see some matching packets. Everything looks good. >>> >>> š# netstat -B >>> ššPid Netif Flags Recv Drop Match Sblen Hblen Command >>> š1800 igb0 p--s--- 135 0 129 380 0 ngrep >>> šThe BPF stats show Recv and Match values. Good. >>> >>> š# ngrep -d bridge0 "q" ip >>> ššI see some matching packets. Good. >>> >>> š# netstat -B >>> ššPid Netif Flags Recv Drop Match Sblen Hblen Command >>> š1901 bridge0 p--s--- 661897 0 659170 425606 0 ngrep >>> šAgain, the BPF stats show Recv and Match values. Good. >>> >>> šHowever, if I sniff on an ix interface: >>> >>> š# ngrep -d ix1 "q" ip >>> šI get no matching packets! >>> >>> š# netstat -B >>> ššPid Netif Flags Recv Drop Match Sblen Hblen Command >>> š1816 ix1 p--s--- 45340 0 0 0 0 ngrep >>> šššššššššššššššššššššššššššššššššššššššššššššššš^^^ >>> š...and the BPF stats always show zero Matches. >>> >>> šBug in the ixgbe driver or in ngrep? >>> >>> š/Elof >>> >>> š_______________________________________________ >>> šfreebsd-net@freebsd.org mailing list >>> šhttps://lists.freebsd.org/mailman/listinfo/freebsd-net >>> šTo unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >> >> _______________________________________________ >> freebsd-net@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-net >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" From owner-freebsd-net@freebsd.org Tue Dec 1 15:05:41 2015 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A93CEA2C34E for ; Tue, 1 Dec 2015 15:05:41 +0000 (UTC) (envelope-from feld@FreeBSD.org) Received: from out4-smtp.messagingengine.com (out4-smtp.messagingengine.com [66.111.4.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7DE371858 for ; Tue, 1 Dec 2015 15:05:41 +0000 (UTC) (envelope-from feld@FreeBSD.org) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.nyi.internal (Postfix) with ESMTP id 01DDE20EE4 for ; Tue, 1 Dec 2015 10:05:33 -0500 (EST) Received: from web3 ([10.202.2.213]) by compute3.internal (MEProxy); Tue, 01 Dec 2015 10:05:34 -0500 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-sasl-enc:x-sasl-enc; s=smtpout; bh=h6Km/b5eBPbP9pM 7oWEY80aCbf8=; b=erm4IwEGUa/MDL97e4qYHQfV4bnCl+VHV1LYNriQWJi/RHt aRHsQN95VM3i98AEQsO/NmN6v9YRpPQeUxOdD+Rk98wuKF/crHDYPhbeq5TtsDZf qACB0Tc6GcSRE406LLWQnbgh6vt6PVIHFkajAVlwQtUlbYOTZubgyxNX1Cn8= Received: by web3.nyi.internal (Postfix, from userid 99) id B156810BE42; Tue, 1 Dec 2015 10:05:33 -0500 (EST) Message-Id: <1448982333.1269981.454734633.11BA4DB2@webmail.messagingengine.com> X-Sasl-Enc: /aLTHO70yUjVu3j9jc57/mB3bySOz+ZeqX6SA9T1yJ8n 1448982333 From: Mark Felder To: wishmaster , freebsd-net@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Mailer: MessagingEngine.com Webmail Interface - ajax-b94e6169 In-Reply-To: <1448956697.854911427.15is5btc@frv34.fwdcdn.com> References: <1448920706.962818.454005905.61CF9154@webmail.messagingengine.com> <1448956697.854911427.15is5btc@frv34.fwdcdn.com> Subject: Re: IPFW blocked my IPv6 NTP traffic Date: Tue, 01 Dec 2015 09:05:33 -0600 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 Dec 2015 15:05:41 -0000 On Tue, Dec 1, 2015, at 02:02, wishmaster wrote: > > Hi, Mark. > > > > I'm hoping someone can explain what happened here and this isn't a bug, > > but if it is a bug I'll gladly open a PR. > > > > I noticed in my ipfw logs that I was getting a log of "DENY" entries for > > an NTP server > > > > Nov 30 13:35:16 gw kernel: ipfw: 4540 Deny UDP > > [2604:a880:800:10::bc:c004]:123 [2001:470:1f11:1e8::2]:58285 in via gif0 > > > > Strange... I looked at ntpq output and sure enough I was trying to > > communicate with that server. But why was it getting blocked? I don't > > have a rule to allow IPv4 input from source port 123. I expected IPFW to > > handle this for me. I know UDP is stateless, but firewalls are usually > > able to "keep state" for UDP. I looked at my v4 rules which and I have > > keep-state on there: > > > > # Allow all outgoing, skip to NAT > > ###################################### > > $cmd 01300 skipto 5000 tcp from any to any out via $pif $ks > > $cmd 01310 skipto 5000 udp from any to any out via $pif $ks > > $cmd 01320 skipto 5000 icmp from any to any out via $pif > > ###################################### > > > > I noticed my outbound IPv6 didn't have $ks for udp, so I added it. > > However, that had no effect. The solution was to add an incoming rule: > > > > $cmd 03755 allow udp from any to any src-port 123 in via $pif6 $ks > > > > This seems wrong. Thoughts? > > > > What is your 5000 rule? > $cmd 05000 nat 1 ip4 from any to any out via $pif > In general on public interface you should: > $cmd 12345 allow log all from any to me 123 $ks > > And for outgoing traffic just: > $cmd 1234 allow log all from me to any $ks > > This works for me. > Sure, I understand the risks and the workaround here but it just seems odd that I have to do this for ipv6 when ipv4 has worked fine. With UDP not really having "state" I understand why that traffic was blocked because the NTP server was trying to send data to my port 58285. I did a tcpdump and restarted ntpd to see what I could find other instances of this in the ipv4 traffic: root@gw:/usr/home/feld # tcpdump -i re0 -nttt dst port 123 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on re0, link-type EN10MB (Ethernet), capture size 262144 bytes 00:00:00.000000 IP 68.117.126.78.123 > 129.6.15.29.123: NTPv4, Client, length 48 00:00:00.105172 IP 129.6.15.29.123 > 68.117.126.78.123: NTPv4, Server, length 48 00:00:01.894636 IP 68.117.126.78.123 > 209.114.111.1.123: NTPv4, Client, length 48 00:00:00.000068 IP 68.117.126.78.123 > 129.6.15.29.123: NTPv4, Client, length 48 00:00:00.065549 IP 209.114.111.1.123 > 68.117.126.78.123: NTPv4, Server, length 48 00:00:00.045821 IP 129.6.15.29.123 > 68.117.126.78.123: NTPv4, Server, length 48 00:00:01.888500 IP 68.117.126.78.123 > 209.114.111.1.123: NTPv4, Client, length 48 00:00:00.000066 IP 68.117.126.78.123 > 129.6.15.29.123: NTPv4, Client, length 48 00:00:00.064383 IP 209.114.111.1.123 > 68.117.126.78.123: NTPv4, Server, length 48 00:00:00.042029 IP 129.6.15.29.123 > 68.117.126.78.123: NTPv4, Server, length 48 00:00:01.893509 IP 68.117.126.78.123 > 209.114.111.1.123: NTPv4, Client, length 48 00:00:00.000061 IP 68.117.126.78.123 > 129.6.15.29.123: NTPv4, Client, length 48 00:00:00.064418 IP 209.114.111.1.123 > 68.117.126.78.123: NTPv4, Server, length 48 00:00:00.044275 IP 129.6.15.29.123 > 68.117.126.78.123: NTPv4, Server, length 48 00:00:01.891510 IP 68.117.126.78.123 > 209.114.111.1.123: NTPv4, Client, length 48 00:00:00.000068 IP 68.117.126.78.123 > 129.6.15.29.123: NTPv4, Client, length 48 00:00:00.063342 IP 209.114.111.1.123 > 68.117.126.78.123: NTPv4, Server, length 48 00:00:00.044807 IP 129.6.15.29.123 > 68.117.126.78.123: NTPv4, Server, length 48 00:00:01.891549 IP 68.117.126.78.123 > 209.114.111.1.123: NTPv4, Client, length 48 00:00:00.000067 IP 68.117.126.78.123 > 129.6.15.29.123: NTPv4, Client, length 48 00:00:00.065415 IP 209.114.111.1.123 > 68.117.126.78.123: NTPv4, Server, length 48 00:00:00.038239 IP 68.117.126.78.16205 > 69.164.201.165.123: NTPv4, Client, length 48 00:00:01.896362 IP 68.117.126.78.123 > 209.114.111.1.123: NTPv4, Client, length 48 00:00:00.064590 IP 209.114.111.1.123 > 68.117.126.78.123: NTPv4, Server, length 48 00:00:06.057206 IP 68.117.126.78.16205 > 4.53.160.75.123: NTPv4, Client, length 48 00:00:08.003668 IP 68.117.126.78.16205 > 72.20.40.62.123: NTPv4, Client, length 48 00:00:08.051264 IP 68.117.126.78.16205 > 104.131.53.252.123: NTPv4, Client, length 48 Notice how almost all of them are port 123 on both sides, but a few of them are not. Why? The RFC says that NTP is supposed to be using port 123 as both the source and destination port, but I clearly have something happening on port 16205. Is something screwy with ntpd in CURRENT? -- Mark Felder ports-secteam member feld@FreeBSD.org