From owner-freebsd-virtualization@freebsd.org Sat Jun 4 14:35:24 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F39B7B6A4F7 for ; Sat, 4 Jun 2016 14:35:24 +0000 (UTC) (envelope-from rowley.evan@gmail.com) Received: from mail-pf0-x232.google.com (mail-pf0-x232.google.com [IPv6:2607:f8b0:400e:c00::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CA9CF1CD5 for ; Sat, 4 Jun 2016 14:35:24 +0000 (UTC) (envelope-from rowley.evan@gmail.com) Received: by mail-pf0-x232.google.com with SMTP id g64so53057635pfb.2 for ; Sat, 04 Jun 2016 07:35:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=yGRrbuWKpnjggR+BN0gHareKAmvcGCnSOFhMlcXqCIU=; b=KJCLCk61uG/Fx5a7JajCnDahfGYd09Kdpzmi62KpGHLmpkfQR1q8HmMistNDdsxURM QpRsxK+9qTiVfpt9gQEUdJq6dmhnw6rn8gK9k4GYdOoqVFdZ7k9OONkLMeDXLd+//w+P OiubyOxlmlSoGXUA/DHLP7uA85ITOEASjH6dbWpA+Q5kjt1JzdKaBrJkA60Yoharlbhv 6uDYFWrVcqebnbNF8WmMqNs6fv5zQTIFqxPm0mhEgchWhJHMG1t7qCxSohEEF2zD4V1C 2oX/Oujya0bvTcLNh4kkAiVsMiD/7fnCIE9ljH9DioexvPiAa5p+iZbSUsx30Dq1wEWF QkDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=yGRrbuWKpnjggR+BN0gHareKAmvcGCnSOFhMlcXqCIU=; b=gutXxIcoChdIjjd1EBQGWgzhet9mnTxJiBaTMe1hUxwsoUvWK8hDj3WxZqt5wrdspH K1CyNyhXSFhjGU9PdZc37biCz3g8FQOfPwUg1z6GYREUcMvJLZ/HTM9wl5Zs/V7YsuAV 5abpcPf8BJ0MXlKqruXNhwi/vsUquO08fC0MxNDnYDYalVhSX+VSbwn2Zm7wKM2U3XKB xfxMAORmLnsaWGH8U2iP3HvUY3xNsEZMq4LLGHJgEe2h1CcNAPu8HPjxKgNVGcWg2vNp BnhSKQw7Q1K/U9+iCFP+IzTyMNFzFCsGTPnL+Qys8IvLF8ZS7j/ujg5BNqseFn/eW0K3 lcIA== X-Gm-Message-State: ALyK8tJ4WHOctxgOKquqRfc6VKdOQHBBl1HieWCfOed6ZKT+n8XtcfgFMs+A/FDYLjVgmJ4DoXrVSonqunSfPg== X-Received: by 10.98.52.65 with SMTP id b62mr4102779pfa.40.1465050924076; Sat, 04 Jun 2016 07:35:24 -0700 (PDT) MIME-Version: 1.0 Received: by 10.66.75.225 with HTTP; Sat, 4 Jun 2016 07:35:23 -0700 (PDT) From: Evan Rowley Date: Sat, 4 Jun 2016 10:35:23 -0400 Message-ID: Subject: a bhyve and Windows Server 2012R2 story To: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Jun 2016 14:35:25 -0000 I've booted my first Windows Server 2012R2 VM in bhyve. It took about 1 day's setup but I finally have it working. My notes below detail a couple pitfalls and things which I did differently than described in online references I used. The further I progressed in this, the more I realized that the directions I started from are based on directions from other places on the net. Started here: https://jameslodge.com/freebsd-bhyve-hypervisor-running-windows-server-2012-r2-standard/ Discovered this: https://people.freebsd.org/~grehan/bhyve_uefi/windows_install.txt And this: https://people.freebsd.org/~grehan/bhyve_uefi/windows_iso_repack.txt Finally, saw this: https://wiki.freebsd.org/bhyve/Windows Anyways, my machine is TRUEOS 11.0-CURRENTMAY2016 [root@true11] ~# uname -a FreeBSD true11 11.0-CURRENTMAY2016 FreeBSD 11.0-CURRENTMAY2016 #19 5bab0d2(master): Fri May 6 17:56:25 UTC 2016 root@devastator:/usr/obj/usr/src/sys/GENERIC amd64 CPU is AMD 6366 HE Motherboard is SuperMicro H8DGU-F In the guide, the p7zip-9.38.1_2 package is used. Here is the latest version that I installed: [root@true11] /mnt/vm0/ds0/iso# pkg info | grep 7z p7zip-15.14 File archiver with high compression ratio Unfortunately, it seems this version of 7z was not able to extract the UDF ISO correctly. Also, the version on this TrueOS11 machine required a different syntax to specify the output folder, compared to the syntax shown in the link above. Many error messages have been omitted, but here is basically what happened: [root@true11] /mnt/vm0/ds0/iso# //usr/local/bin/7z x msdn/en_windows_server_2012_r2_with_update_x64_dvd_6052708.iso -o{win2012iso} .... ERROR: Data Error : support/logging/setupcletw.dll ERROR: Data Error : support/logging/setupetw.dll ERROR: Data Error : support/logging/setupugcetw.dll ERROR: Data Error : support/logging/sysprepetw.dll ERROR: Data Error : support/logging/windeployetw.dll ERROR: Data Error : support/logging/winsetupetw.dll ERROR: Data Error : support/samples/headlessunattend.xml Sub items Errors: 2075 Archives with Errors: 1 Open Errors: 1 Sub items Errors: 2075 ^ this is the output For each of the files that "ERROR: Data Error" was shown, their z7 extracted contents were 0 bytes in size. Here is an example of the last file's word count: [root@true11] /mnt/vm0/ds0/iso# wc win2012iso/support/samples/headlessunattend.xml 0 0 0 win2012iso/support/samples/headlessunattend.xml I decided to try extraction on Windows using graphical 7zip. The version my computer had installed was 15.06. It worked. Here is the word count for the same file, computed in Cygwin: ejr@win7box1 ~ $ wc /cygdrive/c/Users/ejr/Downloads/Win2012R2/support/samples/headlessunattend.xml 108 301 6475 /cygdrive/c/Users/ejr/Downloads/Win2012R2/support/samples/headlessunattend.xml I also added NetKVM drivers from this version virtio-win-0.1.102.iso, not virtio-win-0.1.96.iso as was specified in the guide. After editing the set files on my Windows machine according to the guide, I transferred the files back to my TrueOS 11 machine using smbclient. The mkisofs command to create the new ISO worked flawlessly. I decided to use the 2016 version of the UEFI loader: https://people.freebsd.org/~grehan/bhyve_uefi/BHYVE_UEFI_20160526.fd Here is the command I used to boot the VM: bhyve \ -c 1 \ -s 0,hostbridge \ -s 3,ahci-hd,/dev/zvol/vm0/ds0/test/w2012R2-0/disk0 \ -s 4,ahci-cd,/mnt/vm0/ds0/iso/Win2012R2.iso \ -s 10,virtio-net,tap0 \ -s 31,lpc -l com1,/dev/nmdm0A \ -l bootrom,/mnt/vm0/ds0/test/w2012R2-0/BHYVE_UEFI_20160526.fd \ -m 2G -H \ -w w2012R2-0 My VM had a problem. It did boot and the SAC channels did start - but somewhere after WIMApplyImage message was displayed in the SACSetupAct - the VM would hang and top would show the bhyve process at 99% WCPU. I destroyed the VM after it hanged like this for 1 hour. It took some guessing, but eventually I recalled reading somewhere - possibly on a KVM/QEMU/Xen wiki page - that IOMMU is needed for more than 1 core. I suspected that this machine's IOMMU was disabled because I've set the SuperMicro BIOS to Optimized Defaults a couple times before and noticed that IOMMU would get turned off. I couldn't recall if I had re-enabled it. Rather than reboot the machine and set IOMMU to Enabled, I decided to re-launch the bhyve VM with only 1 core. This way I was able to compete the installation. Later, enabling IOMMU and relaunching the VM with 2 cores worked. I had difficulty at first with the network configuration. TrueOS has ipfw enabled and I suspect it was originally blocking my RDP traffic. Also, I did not have any successful network connections before I set and loaded what's described in 21.7.7 Persistent Configuration: https://www.freebsd.org/doc/handbook/virtualization-host-bhyve.html At first, I attempted to RDP to an IP I had assigned from the SAC channel using the i <#> command. This never worked and I am not sure why. Eventually, I created a CMD channel, then used ipconfig to determine that Windows had leased a different IP via DHCP. Using RDP to reach the DHCP IP worked! Oddly, I did not see the static IP I had set in SAC updated with the DHCP one - but I checked later, and it was updated. I'm new to SAC and maybe this is just one of it's quirks. Not a FreeBSD problem in any case, but perhaps a stumbling block others on freebsd-virtualization may encounter. -- - EJR