From owner-freebsd-net@FreeBSD.ORG Thu Dec 22 02:25:10 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 092F2106566C for ; Thu, 22 Dec 2011 02:25:10 +0000 (UTC) (envelope-from freebsd-net@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 608FB8FC0A for ; Thu, 22 Dec 2011 02:25:09 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1RdY6f-00043Z-RR for freebsd-net@freebsd.org; Thu, 22 Dec 2011 03:10:05 +0100 Received: from l.saper.info ([91.121.203.103]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 22 Dec 2011 03:10:05 +0100 Received: from saper by l.saper.info with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 22 Dec 2011 03:10:05 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-net@freebsd.org From: Marcin Cieslak Date: Thu, 22 Dec 2011 01:59:32 +0000 (UTC) Organization: http://saper.info Lines: 251 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: l.saper.info X-Face: "MPx|KfVwz7Gg!ayb)rH,hKiCBJXvLY7t+%r1s0Uiw; (%xWn-C-H38.2Oa4JL|4Cx}a"V ~a pL4%i"s20r0%z0yZew?2><1ZfOFF27cPqcAKp?wG+-c&%BgXeJVm[lylYKH?j User-Agent: slrn/0.9.9p1 (FreeBSD) Subject: IPv6 not responding on some aliases (recent 8-stable) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Dec 2011 02:25:10 -0000 Hello, I upgraded my Nov 2010 8.x-something machine to Dec 4th and later Dec 19th userland and kernel: FreeBSD x.saper.info 8.2-STABLE FreeBSD 8.2-STABLE #0: Mon Dec 19 22:13:54 UTC 2011 root@x.saper.info:/usr/obj/usr/src/sys/IPSEC amd64 Machine has 6 IPv6 addresses configured (out of provider-supplied /64 range). rtsol is used to get link-local default gateway, but addresses are static. What happens: After boot, SOME IPv6 addresses do not respond to anything (ICMPv6 ping, netcat...), for example: 2001:abcd:f:abcd::1000 does not work 2001:abcd:f:abcd::1001 works 2001:abcd:f:abcd::1002 works 2001:abcd:f:abcd::1003 does not work 2001:abcd:f:abcd::1004 works 2001:abcd:f:abcd::1005 does not work after a reboot it changes a bit, for example :1000 starts working There is a jail runnng on IPv4/IPv6: export jail_myjail_ip="eee.ff.gg.227,2001:abcd:f:abcd::1005" Turning the jail off does not make any difference. Turning off services listening on :1003 does not make any difference (tested with rebooting) The problem exhibited previously with 30% chance to connect to port 22 on :1000 (with ICMPv6 fully working, only port 22 affected). but now having cleaned up configuration I come to this result now: no IPv6 connectivity on some, but not all IPv6 addresses. Going out from the "not working" IPv6 addresses also fails: $ ping6 -S 2001:abcd:f:abcd::1005 www.freebsd.org PING6(56=40+8+8 bytes) 2001:abcd:f:abcd::1005 --> 2001:4f8:fff6::22 ^C --- red.freebsd.org ping6 statistics --- 3 packets transmitted, 0 packets received, 100.0% packet loss $ ping6 -S 2001:abcd:f:abcd::1000 www.freebsd.org PING6(56=40+8+8 bytes) 2001:abcd:f:abcd::1000 --> 2001:4f8:fff6::22 16 bytes from 2001:4f8:fff6::22, icmp_seq=0 hlim=54 time=163.839 ms 16 bytes from 2001:4f8:fff6::22, icmp_seq=1 hlim=54 time=163.789 ms ^C --- red.freebsd.org ping6 statistics --- 2 packets transmitted, 2 packets received, 0.0% packet loss round-trip min/avg/max/std-dev = 163.789/163.814/163.839/0.025 ms What's wrong? Must be something obvious... //Marcin FreeBSD x.saper.info 8.2-STABLE FreeBSD 8.2-STABLE #0: Mon Dec 19 22:13:54 UTC 2011 root@x.saper.info:/usr/obj/usr/src/sys/IPSEC amd64 My configs: kernel: include GENERIC ident IPSEC options IPSEC options IPSEC_NAT_T options IPSEC_FILTERTUNNEL device crypto (IPsec is compiled in, isn't used right now) /etc/rc.conf.local: ifconfig_sis0="inet aa.bbb.ccc.103 netmask 255.255.255.0 broadcast aa.bbb.ccc.255" defaultrouter="aa.bbb.ccc.254" ifconfig_sis0_alias0="inet eee.ff.gg.227 netmask 0xffffffff" ifconfig_sis0_alias1="inet eee.ff.gg.228 netmask 0xffffffff" ifconfig_sis0_alias2="inet eee.ff.gg.229 netmask 0xffffffff" ipv6_ifconfig_sis0="2001:abcd:f:abcd::1000/64" ipv6_ifconfig_sis0_alias0="2001:abcd:f:abcd::1001/64" ipv6_ifconfig_sis0_alias1="2001:abcd:f:abcd::1002/64" ipv6_ifconfig_sis0_alias2="2001:abcd:f:abcd::1003/64" ipv6_ifconfig_sis0_alias3="2001:abcd:f:abcd::1004/64" ipv6_ifconfig_sis0_alias4="2001:abcd:f:abcd::1005/64" ipv6_default_interface="sis0" /etc/rc.conf: # This file now contains just the overrides from /etc/defaults/rc.conf. # Please make all changes to this file, not to /etc/defaults/rc.conf. sshd_enable="YES" ntpdate_enable="YES" ntpdate_hosts="213.186.33.99" fsck_y_enable="YES" named_enable="YES" ipv6_enable="YES" ipv6_ipv4mapping="YES" sendmail_enable="YES" inetd_enable="YES" kerberos5_server_enable="YES" kerberos5_server_flags="--detach --addresses='eee.ff.gg.229' --addresses='2001:41d0:1:d467::1003' --ports='88/tcp 88/udp'" milterdkim_enable="YES" tor_enable="YES" freeswitch_enable="YES" firewall_enable="YES" firewall_type="open" dummynet_enable="YES" #firewall_type="/etc/l.firewall" mysql_enable="YES" rbldnsd_enable="YES" rbldnsd_flags="-r /usr/local/etc/rbldnsd -b eee.ff.gg.229 blacklist.saper.info:ip4set:blacklist" php_fpm_enable="YES" nginx_enable="YES" ezjail_enable="YES" spawn_fcgi_enable="YES" spawn_fcgi_app="/usr/local/sbin/hgwebdir.fcgi" spawn_fcgi_bindport=9002 dovecot_enable="YES" openfire_enable="YES" openfire_javargs="-Xmx256M -Djava.net.preferIPv6Stack=true" /etc/sysctl.conf: #security.bsd.see_other_uids=0 net.inet6.ip6.accept_rtadv=1 ifconfig sis0: sis0: flags=8843 metric 0 mtu 1500 options=82008 ether 00:1c:c0:de:ad:bf inet aa.bbb.ccc.103 netmask 0xffffff00 broadcast aa.bbb.ccc.255 inet6 fe80::21c:c0ff:fede:adbf%sis0 prefixlen 64 scopeid 0x5 inet eee.ff.gg.227 netmask 0xffffffff broadcast eee.ff.gg.227 inet eee.ff.gg.228 netmask 0xffffffff broadcast eee.ff.gg.228 inet eee.ff.gg.229 netmask 0xffffffff broadcast eee.ff.gg.229 inet6 2001:abcd:f:abcd::1000 prefixlen 64 inet6 2001:abcd:f:abcd::1001 prefixlen 64 inet6 2001:abcd:f:abcd::1002 prefixlen 64 inet6 2001:abcd:f:abcd::1003 prefixlen 64 inet6 2001:abcd:f:abcd::1004 prefixlen 64 inet6 2001:abcd:f:abcd::1005 prefixlen 64 nd6 options=8003 media: Ethernet autoselect (100baseTX ) status: active netstat -rnf inet6: Routing tables Internet6: Destination Gateway Flags Refs Use Mtu Netif Expire ::/96 ::1 UGRS 0 0 16384 lo0 => default fe80::5:73ff:fea0:0%sis0 UG 0 2691 1500 sis0 ::1 ::1 UH 0 19 16384 lo0 ::ffff:0.0.0.0/96 ::1 UGRS 0 0 16384 lo0 2001:41d0:1:d400::/56 link#5 U 0 0 1500 sis0 2001:abcd:f:abcd::/64 link#5 U 0 0 1500 sis0 2001:abcd:f:abcd::1000 link#5 UHS 0 0 16384 lo0 2001:abcd:f:abcd::1001 link#5 UHS 0 0 16384 lo0 2001:abcd:f:abcd::1002 link#5 UHS 0 18 16384 lo0 2001:abcd:f:abcd::1003 link#5 UHS 0 205 16384 lo0 2001:abcd:f:abcd::1004 link#5 UHS 0 0 16384 lo0 2001:abcd:f:abcd::1005 link#5 UHS 0 0 16384 lo0 fe80::/10 ::1 UGRS 0 0 16384 lo0 fe80::%sis0/64 link#5 U 0 103 1500 sis0 fe80::21c:c0ff:fede:adbf%sis0 link#5 UHS 0 0 16384 lo0 fe80::%lo0/64 link#7 U 0 0 16384 lo0 fe80::1%lo0 link#7 UHS 0 0 16384 lo0 ff01::%sis0/32 fe80::21c:c0ff:fede:adbf%sis0 U 0 0 1500 sis0 ff01::%lo0/32 ::1 U 0 0 16384 lo0 ff02::/16 fe80::21c:c0ff:fede:adbf%sis0 US 0 0 1500 sis0 ff02::%sis0/32 fe80::21c:c0ff:fede:adbf%sis0 U 0 0 1500 sis0 ff02::%lo0/32 ::1 U 0 0 16384 lo0 netstat -anWf inet6 Active Internet connections (including servers) Proto Recv-Q Send-Q Local Address Foreign Address (state) tcp6 0 0 2001:abcd:f:abcd::1000.26339 2001:1418:13:1::25.6667 ESTABLISHED tcp6 0 0 2001:abcd:f:abcd::1000.12832 2001:610:1908:8010::10.6667 ESTABLISHED tcp6 0 0 2001:abcd:f:abcd::1003.5223 2001:abcd:f:abcd::1003.58883 ESTABLISHED tcp6 0 0 2001:abcd:f:abcd::1003.58883 2001:abcd:f:abcd::1003.5223 ESTABLISHED tcp6 0 0 2001:abcd:f:abcd::1003.5269 2a01:4f8:130:3381::2.47825 ESTABLISHED tcp6 0 0 2001:abcd:f:abcd::1000.22 2a01:aaa:eee::1.10927 ESTABLISHED tcp6 0 0 2001:abcd:f:abcd::1000.22 2a01:aaa:eee::1.11145 ESTABLISHED tcp6 0 0 2001:abcd:f:abcd::1003.5080 *.* LISTEN tcp46 0 0 *.* *.* CLOSED tcp46 0 0 *.7443 *.* LISTEN tcp46 0 0 *.7070 *.* LISTEN tcp46 0 0 *.5223 *.* LISTEN tcp46 0 0 *.5222 *.* LISTEN tcp46 0 0 *.9091 *.* LISTEN tcp46 0 0 *.9090 *.* LISTEN tcp6 0 0 *.113 *.* LISTEN tcp6 0 0 *.21 *.* LISTEN tcp46 0 0 *.25 *.* LISTEN tcp6 0 0 2001:abcd:f:abcd::1005.22 *.* LISTEN tcp6 0 0 2001:abcd:f:abcd::1005.80 *.* LISTEN tcp46 0 0 *.5269 *.* LISTEN tcp46 0 0 *.5229 *.* LISTEN tcp46 0 0 *.7777 *.* LISTEN tcp46 0 0 *.3306 *.* LISTEN tcp6 0 0 2001:abcd:f:abcd::1000.22 *.* LISTEN tcp6 0 0 2001:abcd:f:abcd::1003.80 *.* LISTEN tcp6 0 0 2001:abcd:f:abcd::1003.88 *.* LISTEN tcp6 0 0 ::1.953 *.* LISTEN tcp6 0 0 ::1.53 *.* LISTEN tcp6 0 0 2001:abcd:f:abcd::1000.53 *.* LISTEN udp6 0 0 2001:abcd:f:abcd::1003.5080 *.* udp6 0 0 *.59041 *.* udp6 0 0 2001:abcd:f:abcd::1005.514 *.* udp6 0 0 2001:abcd:f:abcd::1003.88 *.* udp6 0 0 ::1.53 *.* udp6 0 0 2001:abcd:f:abcd::1000.53 *.* udp6 0 0 *.514 *.* ndp -I: ND default interface = sis0 ndp -an: Neighbor Linklayer Address Netif Expire S Flags fe80::21e:79ff:fe1e:f000%sis0 00:1e:79:1e:f0:00 sis0 23h59m44s S R 2001:abcd:f:abcd::1000 00:1c:c0:de:ad:bf sis0 permanent R 2001:abcd:f:abcd::1001 00:1c:c0:de:ad:bf sis0 permanent R 2001:abcd:f:abcd::1002 00:1c:c0:de:ad:bf sis0 permanent R 2001:abcd:f:abcd::1003 00:1c:c0:de:ad:bf sis0 permanent R fe80::21e:79ff:fe1e:d400%sis0 00:1e:79:1e:d4:00 sis0 25s R R 2001:abcd:f:abcd::1004 00:1c:c0:de:ad:bf sis0 permanent R 2001:abcd:f:abcd::1005 00:1c:c0:de:ad:bf sis0 permanent R fe80::21c:c0ff:fede:adbf%sis0 00:1c:c0:de:ad:bf sis0 permanent R fe80::5:73ff:fea0:0%sis0 00:05:73:a0:00:00 sis0 4s D R ipfw set: 00100 allow ip from any to any via lo0 00200 deny ip from any to 127.0.0.0/8 00300 deny ip from 127.0.0.0/8 to any 00400 deny ip from any to ::1 00500 deny ip from ::1 to any 00600 allow ipv6-icmp from :: to ff02::/16 00700 allow ipv6-icmp from fe80::/10 to fe80::/10 00800 allow ipv6-icmp from fe80::/10 to ff02::/16 00900 allow ipv6-icmp from any to any ip6 icmp6types 1 01000 allow ipv6-icmp from any to any ip6 icmp6types 2,135,136 65000 allow ip from any to any 65535 deny ip from any to any