From owner-freebsd-virtualization@freebsd.org Tue Nov 15 21:51:09 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 12512C43848 for ; Tue, 15 Nov 2016 21:51:09 +0000 (UTC) (envelope-from dmarquess@gmail.com) Received: from mail-it0-x229.google.com (mail-it0-x229.google.com [IPv6:2607:f8b0:4001:c0b::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CFA36887 for ; Tue, 15 Nov 2016 21:51:08 +0000 (UTC) (envelope-from dmarquess@gmail.com) Received: by mail-it0-x229.google.com with SMTP id c20so176442430itb.0 for ; Tue, 15 Nov 2016 13:51:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=HrS2by6lJoh3zlrBMg7S1iFlinkG6tYpaZkAI+1JUvc=; b=v/NZCCfQdME8BCur2dtF9CTeZef46bpZAyOVRCCf+ZpN9c2tdnV3QdRWooRkZGMlRI NQ2j5QnhpfAuc/ohaVWrPAoOkQdqIuqId5kExjP0kF7eo/r0TVe22V1gUc/QEcge55lT t8VmxncdJtWk0/gq4sJi+s6DOXuSQjIsG9meTiWRbEEQB7+7qN3nKbp3Hg0WNyFmXzAB YGNuzLoFchUpp4oPIQUlkxqibrqhRfO5zUT9LdbiUzScWqTE82in/cfHJ65EQpaPU7WF 24YvycCGXt1KzRId7Jw0Ie9faM36x1r45UCZY0VCrbhTu460fD1KZMUqQqVP1B1Bs88I vzCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=HrS2by6lJoh3zlrBMg7S1iFlinkG6tYpaZkAI+1JUvc=; b=KsqFCbFo3ueexo8oFNWuE8XLXYlBn6lck4KcUMok6pXxf4NbSM+3VyPlFeqxbMy9Qe UPhXiWTI1u3JJdoV0kDE6ZMKp4RApGyx88J2V4ZpJcqkh/gYjJQka8WL8opSpoiSPP88 QZTi60CEf/C4Hbo840EjdBJgpr6wIUAQBbDLjUpdAtaTxL14tWjKYXl0nG8cMMCi711w +8Th3aRZ+ywk7PwMc5kDxNHJFKDwnrpiQ7fC4cUpjW8HkiSkALpU+cDATx8BIDMGgNnG yrpUSmSSiajJwPsRzRq1IrcmRJ3NW80HZwreemX3h/xr986O3VvdLLLqRAycmOTG971F YY1w== X-Gm-Message-State: ABUngvd/IjcHrtzxYUE5iGi5gW5eeXE0yrKcHUqOGCulE9xnJw0b2hXXplGRZpVNqdSmvxPfWlbfmIeNQE6GbA== X-Received: by 10.107.190.6 with SMTP id o6mr107467iof.95.1479246668063; Tue, 15 Nov 2016 13:51:08 -0800 (PST) MIME-Version: 1.0 Received: by 10.107.137.14 with HTTP; Tue, 15 Nov 2016 13:51:07 -0800 (PST) In-Reply-To: References: <4cf1a528d9a4e85d5aa351ca2ff417a9@mail.jonathanprice.org> From: Dustin Marquess Date: Tue, 15 Nov 2016 15:51:07 -0600 Message-ID: Subject: Re: bhyve windows guest unable to ping default gateway or external networks To: Rajil Saraswat Cc: freebsd@jonathanprice.org, freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Nov 2016 21:51:09 -0000 If it's a bug, I don't believe it's in the virtio stack. I'm using Chelsio NICs on my bhyve servers, and I don't have to disable anything for my guests to work, I even have TOE enabled. -Dustin On Tue, Nov 15, 2016 at 6:07 AM, Rajil Saraswat wrote: > On Nov 15, 2016 5:58 AM, wrote: >> >> Hello, >> >> I am trying to get a windows server VM running under bhyve. So far I can > use VNC to install Windows, but I've been unable to get the networking > working. I will provide details below, and include a summary up here. On > the guest, I have installed the latest version of winvirtio, and I have > configured static IP addressing. The guest can ping itself, but not the > host machine or anything external. It is worth noting that I am also > currently running a CentOS guest, and this can connect to both the host and > the outside world with no issues. >> >> Host: FreeBSD 11.0-RELEASE >> Guest: Tried both Windows Server 2016 Essentials and Windows Server 2012 > Standard >> >> bhyve command: >> bhyve -c 1 -H -w -m 3G -s 0,hostbridge -s 3,ahci-cd,/iohyve/ISO// -s > '4,ahci-hd,/dev/zvol/zroot/iohyve/win12/disk0,sectorsize=512' -s > 10,virtio-net,tap2 -s '11,fbuf,tcp=0.0.0.0:5902,w=800,h=600,wait' -s > 12,xhci,tablet -s 31,lpc -l com1,/dev/nmdm3A -l > bootrom,/iohyve/Firmware/BHYVE_UEFI.fd/BHYVE_UEFI.fd ioh-win12 >> >> ifconfig output: >> em0: flags=8843 metric 0 mtu 1500 >> > options=4219b >> ether e0:69:95:a2:f1:d9 >> inet 46.105.xxx.xxx netmask 0xffffff00 broadcast 46.105.124.255 >> inet6 fe80::e269:95ff:xxxx:f1d9%em0 prefixlen 64 scopeid 0x1 >> inet6 2001:41d0:2:xxxx::1 prefixlen 56 >> nd6 options=21 >> media: Ethernet autoselect (100baseTX ) >> status: active >> lo0: flags=8049 metric 0 mtu 16384 >> options=600003 >> inet6 ::1 prefixlen 128 >> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2 >> inet 127.0.0.1 netmask 0xff000000 >> nd6 options=21 >> groups: lo >> bridge0: flags=8843 metric 0 mtu > 1500 >> ether 02:cf:0c:8f:13:00 >> inet6 fd00::ffff prefixlen 112 >> inet 10.0.0.254 netmask 0xffffff00 broadcast 10.0.0.255 >> nd6 options=1 >> groups: bridge >> id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 >> maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 >> root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 >> member: tap2 flags=143 >> ifmaxaddr 0 port 7 priority 128 path cost 2000000 >> member: tap1 flags=143 >> ifmaxaddr 0 port 5 priority 128 path cost 2000000 >> member: tap0 flags=143 >> ifmaxaddr 0 port 4 priority 128 path cost 2000000 >> pflog0: flags=0<> metric 0 mtu 33160 >> groups: pflog >> tap0: flags=8943 metric 0 > mtu 1500 >> description: iohyve-centos >> options=80000 >> ether 00:bd:5d:d3:e8:00 >> nd6 options=21 >> media: Ethernet autoselect >> status: active >> groups: tap >> Opened by PID 96206 >> tap1: flags=8902 metric 0 mtu 1500 >> description: iohyve-win16es >> options=80000 >> ether 00:bd:61:ab:eb:01 >> nd6 options=21 >> media: Ethernet autoselect >> status: no carrier >> groups: tap >> tap2: flags=8943 metric 0 > mtu 1500 >> description: iohyve-win12 >> options=80000 >> ether 00:bd:c9:1a:59:02 >> nd6 options=21 >> media: Ethernet autoselect >> status: active >> groups: tap >> Opened by PID 1911 >> >> It is using NAT with PF to reach the internet. It should not rely on NAT > to be able to ping the host machine, but I will post the NAT configuration > below anyway: >> >> /etc/pf.conf: >> # Automatically NAT any VMs >> nat on em0 inet from bridge0:network to any -> (em0) >> >> >> (On the Guest) >> ============== >> >> C:\Users\Administrator>ipconfig >> >> Ethernet adapter Ethernet: >> >> IPv4 Address: 10.0.0.3 >> Subnet Mask: 255.255.255.0 >> Default Gateway: 10.0.0.254 >> >> C:\Users\Administrator>ping 10.0.0.3 >> >> Pinging 10.0.0.3 with 32 bytes of data: >> Reply from 10.0.0.3: bytes=32 time<1ms TTL=128 >> Reply from 10.0.0.3: bytes=32 time<1ms TTL=128 >> Reply from 10.0.0.3: bytes=32 time<1ms TTL=128 >> Reply from 10.0.0.3: bytes=32 time<1ms TTL=128 >> >> C:\Users\Administrator>ping 10.0.0.254 >> >> Pinging 10.0.0.254 with 32 bytes of data: >> Reply from 10.0.0.3: Destination host unreachable. >> Reply from 10.0.0.3: Destination host unreachable. >> Reply from 10.0.0.3: Destination host unreachable. >> Reply from 10.0.0.3: Destination host unreachable. >> >> C:\Users\Administrator>arp -a >> >> Interface: 10.0.0.3 --- 0xc >> Internet Address Physical Address Type >> 10.0.0.255 ff-ff-ff-ff-ff-ff static >> 224.0.0.22 01-00-5e-00-00-16 static >> 224.0.0.252 01-00-5e-00-00-fc static >> >> (PF log testing) >> ================ >> >> I was curious to see if PF was seeing packets on the bridge interface, so > I added the following to /etc/pf.conf: >> pass log (all) all >> >> I then ran the following command, which had no results when I tried a > test ping: >> sudo tcpdump -n -e -ttt -i pflog0 host 10.0.0.3 >> >> (I also watched live traffic on 10.0.0.254 and saw nothing from this IP) >> >> >> Summary >> ======== >> >> I appear to be having some network connectivity issues that either lies > in bhyve, virtio-net or the winvirtio drivers. If anybody has any > suggestions for things I could try I'd love to hear it. >> >> Thanks, >> >> Jonathan. >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > > I had to disable offloading on the host NIC to get the network working > inside bhyve session. This seems to be a FreeBSD bug in virtio stack. > > ifconfig_igb0="-rxcsum -txcsum -vlanmtu -vlanhwtso -tso4 -tso6 > -vlanhwtso -vlanhwcsum up" > > -Rajil > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org"