From owner-freebsd-virtualization@freebsd.org Tue Nov 15 12:07:24 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 04576C42927 for ; Tue, 15 Nov 2016 12:07:24 +0000 (UTC) (envelope-from rajil.s@gmail.com) Received: from mail-qk0-x22f.google.com (mail-qk0-x22f.google.com [IPv6:2607:f8b0:400d:c09::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B10779B6 for ; Tue, 15 Nov 2016 12:07:23 +0000 (UTC) (envelope-from rajil.s@gmail.com) Received: by mail-qk0-x22f.google.com with SMTP id n21so132077495qka.3 for ; Tue, 15 Nov 2016 04:07:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=rIaS8mc1PrmtgXsx8HvHuh4Z8BrZ98zordb9bqWtP00=; b=N9lTUkQ0rHFt0hwBL8WuvKphKOPjrrcMh1YwBRNtqp+WrPWLdMcuPLKt+lb49P+OY6 jQ8X/bm/qyNekd5PpmN42/Gy44nn3XpJ6keHmyG2u9nEL91eJUjNdpIhfcXN4UVZdQmf FLKdkneCAGQ4L+ynpWXpR3r+fmMykGjHkG6mBbmMPBgFp6B/DQ251KDoir4g0r64O2D2 YMAUW7J+4ULWfx3fni2RDLGPIrZQELomaeZCeyMYU7VtGbOujB+neJ8l+6ANnJvifqQ2 Up6lVFt5jrFA3GUnOn1xLjSLmMZsI8/PBpZ2m0J5kRLdTdH9GRGQ8SEPuTn45HqSPBHr as6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=rIaS8mc1PrmtgXsx8HvHuh4Z8BrZ98zordb9bqWtP00=; b=c8gDgmXR+R9ek8yDvBRYLnAQAG/257G1W/6ILg4JGVhI0eVk3sfzNPsRW/N8Q3oiR5 jQbmEJ5rW2EcsyBQkfh73KfGlLr2v1DALBdbU55u8HffrsLXbh4SHeX4b5W3fcA7IxwA naFka82sytis3nkpq1lp3gcpMyMWOsIPIZjduB0mCtjgZ32pSCOgXPrhx5vY+TmKFtoc ELYOTwEjkZZLepxtBPaLXKCzsbcYMzuyLXnEud4ZfHJvNUQLLUdWsB3PFT0l+5Sytuuc p6cbpYbuzg5+xTbqiSPLtnp8jXxyRpq3yx9l2YbiTHTsEd2LRiY5Y2AH7COK8n/A5WmC gZvg== X-Gm-Message-State: ABUngvfMbuJEmovoMlunnPN08ovwk/Et11sEymlFjyKn/hUM7zBhGDyoqvoo0OxGIyunLMOc0efdd3ItBnNjIQ== X-Received: by 10.55.108.193 with SMTP id h184mr21205517qkc.155.1479211642912; Tue, 15 Nov 2016 04:07:22 -0800 (PST) MIME-Version: 1.0 Received: by 10.12.149.109 with HTTP; Tue, 15 Nov 2016 04:07:22 -0800 (PST) Received: by 10.12.149.109 with HTTP; Tue, 15 Nov 2016 04:07:22 -0800 (PST) In-Reply-To: <4cf1a528d9a4e85d5aa351ca2ff417a9@mail.jonathanprice.org> References: <4cf1a528d9a4e85d5aa351ca2ff417a9@mail.jonathanprice.org> From: Rajil Saraswat Date: Tue, 15 Nov 2016 06:07:22 -0600 Message-ID: Subject: Re: bhyve windows guest unable to ping default gateway or external networks To: freebsd@jonathanprice.org Cc: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Nov 2016 12:07:24 -0000 On Nov 15, 2016 5:58 AM, wrote: > > Hello, > > I am trying to get a windows server VM running under bhyve. So far I can use VNC to install Windows, but I've been unable to get the networking working. I will provide details below, and include a summary up here. On the guest, I have installed the latest version of winvirtio, and I have configured static IP addressing. The guest can ping itself, but not the host machine or anything external. It is worth noting that I am also currently running a CentOS guest, and this can connect to both the host and the outside world with no issues. > > Host: FreeBSD 11.0-RELEASE > Guest: Tried both Windows Server 2016 Essentials and Windows Server 2012 Standard > > bhyve command: > bhyve -c 1 -H -w -m 3G -s 0,hostbridge -s 3,ahci-cd,/iohyve/ISO// -s '4,ahci-hd,/dev/zvol/zroot/iohyve/win12/disk0,sectorsize=512' -s 10,virtio-net,tap2 -s '11,fbuf,tcp=0.0.0.0:5902,w=800,h=600,wait' -s 12,xhci,tablet -s 31,lpc -l com1,/dev/nmdm3A -l bootrom,/iohyve/Firmware/BHYVE_UEFI.fd/BHYVE_UEFI.fd ioh-win12 > > ifconfig output: > em0: flags=8843 metric 0 mtu 1500 > options=4219b > ether e0:69:95:a2:f1:d9 > inet 46.105.xxx.xxx netmask 0xffffff00 broadcast 46.105.124.255 > inet6 fe80::e269:95ff:xxxx:f1d9%em0 prefixlen 64 scopeid 0x1 > inet6 2001:41d0:2:xxxx::1 prefixlen 56 > nd6 options=21 > media: Ethernet autoselect (100baseTX ) > status: active > lo0: flags=8049 metric 0 mtu 16384 > options=600003 > inet6 ::1 prefixlen 128 > inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 > inet 127.0.0.1 netmask 0xff000000 > nd6 options=21 > groups: lo > bridge0: flags=8843 metric 0 mtu 1500 > ether 02:cf:0c:8f:13:00 > inet6 fd00::ffff prefixlen 112 > inet 10.0.0.254 netmask 0xffffff00 broadcast 10.0.0.255 > nd6 options=1 > groups: bridge > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > member: tap2 flags=143 > ifmaxaddr 0 port 7 priority 128 path cost 2000000 > member: tap1 flags=143 > ifmaxaddr 0 port 5 priority 128 path cost 2000000 > member: tap0 flags=143 > ifmaxaddr 0 port 4 priority 128 path cost 2000000 > pflog0: flags=0<> metric 0 mtu 33160 > groups: pflog > tap0: flags=8943 metric 0 mtu 1500 > description: iohyve-centos > options=80000 > ether 00:bd:5d:d3:e8:00 > nd6 options=21 > media: Ethernet autoselect > status: active > groups: tap > Opened by PID 96206 > tap1: flags=8902 metric 0 mtu 1500 > description: iohyve-win16es > options=80000 > ether 00:bd:61:ab:eb:01 > nd6 options=21 > media: Ethernet autoselect > status: no carrier > groups: tap > tap2: flags=8943 metric 0 mtu 1500 > description: iohyve-win12 > options=80000 > ether 00:bd:c9:1a:59:02 > nd6 options=21 > media: Ethernet autoselect > status: active > groups: tap > Opened by PID 1911 > > It is using NAT with PF to reach the internet. It should not rely on NAT to be able to ping the host machine, but I will post the NAT configuration below anyway: > > /etc/pf.conf: > # Automatically NAT any VMs > nat on em0 inet from bridge0:network to any -> (em0) > > > (On the Guest) > ============== > > C:\Users\Administrator>ipconfig > > Ethernet adapter Ethernet: > > IPv4 Address: 10.0.0.3 > Subnet Mask: 255.255.255.0 > Default Gateway: 10.0.0.254 > > C:\Users\Administrator>ping 10.0.0.3 > > Pinging 10.0.0.3 with 32 bytes of data: > Reply from 10.0.0.3: bytes=32 time<1ms TTL=128 > Reply from 10.0.0.3: bytes=32 time<1ms TTL=128 > Reply from 10.0.0.3: bytes=32 time<1ms TTL=128 > Reply from 10.0.0.3: bytes=32 time<1ms TTL=128 > > C:\Users\Administrator>ping 10.0.0.254 > > Pinging 10.0.0.254 with 32 bytes of data: > Reply from 10.0.0.3: Destination host unreachable. > Reply from 10.0.0.3: Destination host unreachable. > Reply from 10.0.0.3: Destination host unreachable. > Reply from 10.0.0.3: Destination host unreachable. > > C:\Users\Administrator>arp -a > > Interface: 10.0.0.3 --- 0xc > Internet Address Physical Address Type > 10.0.0.255 ff-ff-ff-ff-ff-ff static > 224.0.0.22 01-00-5e-00-00-16 static > 224.0.0.252 01-00-5e-00-00-fc static > > (PF log testing) > ================ > > I was curious to see if PF was seeing packets on the bridge interface, so I added the following to /etc/pf.conf: > pass log (all) all > > I then ran the following command, which had no results when I tried a test ping: > sudo tcpdump -n -e -ttt -i pflog0 host 10.0.0.3 > > (I also watched live traffic on 10.0.0.254 and saw nothing from this IP) > > > Summary > ======== > > I appear to be having some network connectivity issues that either lies in bhyve, virtio-net or the winvirtio drivers. If anybody has any suggestions for things I could try I'd love to hear it. > > Thanks, > > Jonathan. > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " freebsd-virtualization-unsubscribe@freebsd.org" I had to disable offloading on the host NIC to get the network working inside bhyve session. This seems to be a FreeBSD bug in virtio stack. ifconfig_igb0="-rxcsum -txcsum -vlanmtu -vlanhwtso -tso4 -tso6 -vlanhwtso -vlanhwcsum up" -Rajil