From owner-freebsd-net@freebsd.org Tue May 16 21:24:28 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CB0AAD70F51 for ; Tue, 16 May 2017 21:24:28 +0000 (UTC) (envelope-from william@gathoye.be) Received: from mail-wm0-x22a.google.com (mail-wm0-x22a.google.com [IPv6:2a00:1450:400c:c09::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 640AD17EA for ; Tue, 16 May 2017 21:24:27 +0000 (UTC) (envelope-from william@gathoye.be) Received: by mail-wm0-x22a.google.com with SMTP id u65so134295789wmu.1 for ; Tue, 16 May 2017 14:24:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gathoye-be.20150623.gappssmtp.com; s=20150623; h=to:from:subject:message-id:disposition-notification-to:date :user-agent:mime-version; bh=dZHEERbC1ZzjSb50LvjcEKC0TmzZAjbNfmJpGS0wY2k=; b=QkojFcxOG4wMqRfJ+jaAOtH63k6Dz3elL7t4Nx42kgWzQEiLGuOq89mlfQ0OvUN3vx F5YwzQhmzvMvLZFDSSsirGaBkFBGlMAVvS96WypbQAPb6LB3WVRuUxHPHd4CsFCWIaXe 3kWww8HWHP49bhF162moKQ3WDw/d+KuDL18PyZEqLS61d+9vp2PHnawT0eYgxHEfd+jH a/lQm36SNlyo1wqcyZhZ35xB8pftyDZ8IZrWb1G6kZXxoXumKBJe5BMXxuMt5buM7IGk GVZztAyMX6gyjEMcCewUwzSIgN409t3ZUVLJ09TVNeyMbMImD9xCzQPI7u6IoE6dMyDm ERIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:message-id :disposition-notification-to:date:user-agent:mime-version; bh=dZHEERbC1ZzjSb50LvjcEKC0TmzZAjbNfmJpGS0wY2k=; b=Wdbsq5mvgzfYJvWsD6NUaPDJD2QTV8lk7x36w3/MerleJvlrjNw0GBEpMgAYtYscHC 8saVhUY0pb6XWsPqjC2b3ORx/DjF8S9ka0oUbcsx7Dv18yK98QGy16bhDgy+t1d04vtw BCksoU4varn/bvcbUane5rRdiD7CE9D1pvLbOOADrcqtKYXJVrpSjRXPzgqrWv0GrU7u v2kU8w0oAORsFzvFKf7gUdK0hKP9/MCdar6RjrOfXl1at3rsPd1t/DzokPKvFrKJGOm2 7IcJNnzgIynaLEUG/4Ey6Vw7x+7ZPwGhlDqGYoHx8nF2RWZ3SUlYbgT+6UB3u1Od1z6U xxRQ== X-Gm-Message-State: AODbwcBgoXh4eMcT0ZZgVo5iW5MK0nVdlKr+2pNnb6Pe1EJ9Z5C8gpzf ub0gMNfM+t63q7b4qqjv5A== X-Received: by 10.80.185.3 with SMTP id m3mr453523ede.41.1494969865189; Tue, 16 May 2017 14:24:25 -0700 (PDT) Received: from [10.0.0.3] ([82.212.185.202]) by smtp.gmail.com with ESMTPSA id l4sm418058edd.69.2017.05.16.14.24.24 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 May 2017 14:24:24 -0700 (PDT) To: freebsd-net@freebsd.org From: William Gathoye Subject: Public IPv6s fail on KVM bridge with "No buffer space available" Message-ID: Date: Tue, 16 May 2017 23:24:17 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.1.0 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="jOrTa7knUgVnwGSSPUALPWr8g9IXFmktt" X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 May 2017 21:24:28 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --jOrTa7knUgVnwGSSPUALPWr8g9IXFmktt Content-Type: multipart/mixed; boundary="J65aFDgSBDOakth6cTK6Gsufn4EHmFSVe"; protected-headers="v1" From: William Gathoye To: freebsd-net@freebsd.org Message-ID: Subject: Public IPv6s fail on KVM bridge with "No buffer space available" --J65aFDgSBDOakth6cTK6Gsufn4EHmFSVe Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable Hello everyone, I've already asked this question on the #networking and #freebsd IRC channels on Freenode but nobody was able to answer my question and forwarded me over here as it seems this issue is kicky to solve. I have a Proxmox hypervisor where are hosted LXC containers and KVM machines going from Debian 9, Arch Linux and Windows. All of them are bridged to the hypervisor and the IPv6 stack is working flawlessly with public IP addresses given by my ISP and routed directly to the VMs. One of the VM I have is an OPNsense firewall appliance (based on FreeBSD 11.0-RELEASE-p8). Contrary to the other LXC/KVM, the latter cannot ping any IPv6 machines outside those accessible directly from the bridge. As soon I try to ping either the gateway of my hypervisor (still in IPv6), or any other far away IPv6 hosts (e.g. google.com), I got the following error message: [...] ping6: sendmsg: No buffer space available [...] - At first, I thought the issue was due to issues with VirtIO drivers (bundled with FreeBSD). So I switched to emulated Intel E1000 NIC, but the problem persists. (I'm back with VirtIO now). - A netstat -m reports the buffers as empty, so the problem doesn't come from here either. - Putting the interface down and up again or rebooting doesn't fix the issue. - I tested with a fresh FreeBSD and OpenBSD install (to avoid the OPNsense overlay), but the problem persists as well. Pinging the VM either from the bridge or from a machine completely outside of the infrastructure doesn't respond, nor connect (i.e. I had started sshd on 2222, but weren't able to connect). The FreeBSD host is configured like this: ifconfig vtnet0 /32 route add -iface vtnet0 route add default ifconfig vtnet0 inet6 prefixlen 64 route add -inet6 -iface vtnet0 route add -inet6 default Please note all my GWs are outside of my IP subnets. After applying these lines, the routes reported by netstat -rn are sensible to me. Nothing wrong. I precise the pf firewall is completely disabled (pfctl -d). I want to make sure this is working flawlessly before enabling yet another level of failures. :) Is there a bug somewhere in the BSD IPv6 stack as Linux is not complaining at all? This sounds weird as I think I'm not the only one in this situation and not the only one having GW outside their IP ranges. Thanks in advance for your time / help. Regards. -- William Gathoye --J65aFDgSBDOakth6cTK6Gsufn4EHmFSVe-- --jOrTa7knUgVnwGSSPUALPWr8g9IXFmktt Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEE8ucX65+2FhkmJe7RDn3lLATXFoMFAlkbbgcACgkQDn3lLATX FoN7AhAAlLrzmVAtLCtCTufzzdF+RJXF5t5l+j9SZhiUbXhzCxq7vcGEmrWQ3oHb 6AGY72VHsuTHd2YWne9bMkANqtmWnbwe0tfqdWWQNqDZWadUpI3d/jPg4csQ6PzN IRFZ1rR5v95my1epSWgux+5GTizaoGtOMI8ET0rORDtPrX1v7rG404tT+i18LmON dSr8ygCj1g2uDmLTC2cd1G0W3BHubk+JIqNBREonzZJ/xTnSXFNzU3knX/Am8irN iES90TDfBcUvLoCKzrsIL4BFtY8TmyNPG6FWra5W7TIQ8HH7iddYtbTov67IPWOn CVSl/uyxhvqeEh/Db3sTX2rFnQhUX5dPIfUjfOqAiPaQS14QXm/wSfMC99/VbUkS zFfeoEjXXuS9/8s3jND7hQZ26xXPC3FlsZgCXhP1Hfhz/1oqX1DznsdlS2dDRE5j faG5ZCGcYJM6JrihtOl2Wlpz0qb6vjqY+b1gcns+iXL8hyfKFG1a39QH84enzK6P A0zaFyuNRZ9AXreGevXy5UXRoOd9H4ww5FioQgJOpgi0Mfj2/NK4DAtAHhkXQRZx lD3clrYlVimQfuqHhyQXArUISvJepw3w0woCJnEMSuu6i9rg9gCLlVxINWkfLK7F IDCMCxZYWd8Ix2aPx7fu1FtbVoRFvx8BGAk+ma/jdBZ4aSgnWBo= =VAFQ -----END PGP SIGNATURE----- --jOrTa7knUgVnwGSSPUALPWr8g9IXFmktt--