From owner-freebsd-hackers@FreeBSD.ORG Sat Jul 12 22:48:56 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 93BC982D; Sat, 12 Jul 2014 22:48:56 +0000 (UTC) Received: from mail-vc0-x22b.google.com (mail-vc0-x22b.google.com [IPv6:2607:f8b0:400c:c03::22b]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 41FCA2B30; Sat, 12 Jul 2014 22:48:56 +0000 (UTC) Received: by mail-vc0-f171.google.com with SMTP id id10so4791146vcb.30 for ; Sat, 12 Jul 2014 15:48:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=ubFymp2Oh0irJm3Wno878g2gTg8Gj80Q0lAXNkXG3HU=; b=mZQHP9Ie33Ux5RxjPPKGF671+rRVV1vME1exIvRvQYiUHm16O0ptuEsDsFqJlEAqj4 9jXyoHcghaXCwqQXZBjMeaUV1+1F7KZ7FehRTQH+bBRoGDg79l30gEpq1yCVgdN9Ry6N x2tXLPpJYKumoZx8pnm8yO4WS9sxTjaf0AsCZHieMSqmxIMHQU5qnTxdhqkC88fwV7EG Tn+b8X54K/gTGxv+KGhEca37+/1HsxJGLN+MWN/k/uZTEY+0QQ/cG913JqYmUrakz0CV tBOfgvCnxH3nZqmanB4QArxuLRSf+gmeoxNb0HsqnpgatesADPKMPAVvYWwW75reax9A bSHA== MIME-Version: 1.0 X-Received: by 10.220.251.134 with SMTP id ms6mr7561772vcb.10.1405205335150; Sat, 12 Jul 2014 15:48:55 -0700 (PDT) Received: by 10.220.249.132 with HTTP; Sat, 12 Jul 2014 15:48:55 -0700 (PDT) Date: Sat, 12 Jul 2014 18:48:55 -0400 Message-ID: Subject: ngX connected hosts not receiving replies from non-kernel IP services. From: Zaphod Beeblebrox To: FreeBSD Hackers , FreeBSD Net Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Jul 2014 22:48:56 -0000 I have a network of computers at home. The gateway/firewall is FreeBSD 9.2 running mpd5. The host requesting the service is FreeBSD 9.2. The misbehaving host is FreeBSD 10.0p6 running mpd5. So the details: ssh is listening (output of netstat -an) tcp4 0 0 *.22 *.* LISTEN tcp6 0 0 *.22 *.* LISTEN named is listening (installed from bind99 port) tcp4 0 0 xx.yy.30.99.53 *.* LISTEN udp4 0 0 xx.yy.30.99.53 *.* mpd 5 on the server is up: [2:35:335]root@owl:~> ifconfig ng29 ng29: flags=88d1 metric 0 mtu 1436 inet xx.yy.31.6 --> xx.yy.16.50 netmask 0xffffffff inet6 fe80::219:b9ff:fef9:b9e7%ng29 prefixlen 64 scopeid 0x23 nd6 options=21 ping works: [1:71:137]root@virtual:/vr2/backup/nozfs/ox/local-etc> ping xx.yy.16.3 PING xx.yy.16.3 (xx.yy.16.3): 56 data bytes 64 bytes from xx.yy.16.3: icmp_seq=0 ttl=63 time=7.439 ms 64 bytes from xx.yy.16.3: icmp_seq=1 ttl=63 time=6.756 ms now tcpdumping from the FreeBSD 10.0p6 server host while I ssh: [2:29:329]root@owl:~> tcpdump -nvi ng29 host xx.yy.16.3 tcpdump: listening on ng29, link-type NULL (BSD loopback), capture size 65535 bytes capability mode sandbox enabled 18:14:36.276578 IP (tos 0x0, ttl 63, id 3249, offset 0, flags [none], proto TCP (6), length 60) xx.yy.20.52.39218 > xx.yy.16.3.22: Flags [S], cksum 0x4aa1 (correct), seq 3433340283, win 65535, options [mss 1396,nop,wscale 6,sackOK,TS val 435369805 ecr 0], length 0 18:14:39.290104 IP (tos 0x0, ttl 63, id 4999, offset 0, flags [none], proto TCP (6), length 60) xx.yy.20.52.39218 > xx.yy.16.3.22: Flags [S], cksum 0x3ee9 (correct), seq 3433340283, win 65535, options [mss 1396,nop,wscale 6,sackOK,TS val 435372805 ecr 0], length 0 18:14:42.502893 IP (tos 0x0, ttl 63, id 6832, offset 0, flags [none], proto TCP (6), length 60) xx.yy.20.52.39218 > xx.yy.16.3.22: Flags [S], cksum 0x3269 (correct), seq 3433340283, win 65535, options [mss 1396,nop,wscale 6,sackOK,TS val 435376005 ecr 0], length 0 Similarly tcpdumping from the server while running "dig google.ca @xx.yy.30.99" [2:37:337]root@owl:~> tcpdump -nvi ng29 host xx.yy.30.99 tcpdump: listening on ng29, link-type NULL (BSD loopback), capture size 65535 bytes capability mode sandbox enabled 18:36:02.841942 IP (tos 0x0, ttl 63, id 30407, offset 0, flags [none], proto UDP (17), length 66) xx.yy.20.52.27400 > xx.yy.30.99.53: 40608+ [1au] A? google.ca. (38) 18:36:07.838721 IP (tos 0x0, ttl 63, id 33612, offset 0, flags [none], proto UDP (17), length 66) xx.yy.20.52.27400 > xx.yy.30.99.53: 40608+ [1au] A? google.ca. (38) Frustratingly, ssh and bind work just fine from hosts that are on the lan with the server. It's like some portion of the packet routing machinery is broken with ngX. Before y'all ask, too, ip.forwarding is 1. The ng-connected hosts can use the rest of the internet ... just not non-kernel services on the host that breaks up their l2tp.