From owner-freebsd-virtualization@freebsd.org Tue Oct 10 01:37:55 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ACF46E42694 for ; Tue, 10 Oct 2017 01:37:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9B43F640A7 for ; Tue, 10 Oct 2017 01:37:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v9A1btvx087237 for ; Tue, 10 Oct 2017 01:37:55 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 222704] `sudo pkg update` segfaults in Vagrant Date: Tue, 10 Oct 2017 01:37:55 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Oct 2017 01:37:55 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222704 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-virtualization@Free | |BSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Wed Oct 11 19:37:18 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0258AE350BE; Wed, 11 Oct 2017 19:37:18 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9444B74741; Wed, 11 Oct 2017 19:37:17 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (mh0.gentlemail.de [78.138.80.135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id v9BJbExb023061; Wed, 11 Oct 2017 21:37:14 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id B13F3268; Wed, 11 Oct 2017 21:37:13 +0200 (CEST) Message-ID: <59DE72E9.1050006@omnilan.de> Date: Wed, 11 Oct 2017 21:37:13 +0200 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: freebsd-stable@freebsd.org, FreeBSD virtualization Subject: bhyve ppt usage can cause severe RAM corruption [Was: Re: panic: Memory modified after free in zio_create, passthru in use] References: <59369A15.2010901@omnilan.de> <593D1D5C.907@omnilan.de> In-Reply-To: <593D1D5C.907@omnilan.de> Content-Type: multipart/mixed; boundary="------------090102070804030709070902" X-Greylist: ACL 129 matched, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [78.138.80.130]); Wed, 11 Oct 2017 21:37:14 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: 78.138.80.135; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Oct 2017 19:37:18 -0000 This is a multi-part message in MIME format. --------------090102070804030709070902 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Bezglich Harry Schmalzbauer's Nachricht vom 11.06.2017 12:37 (localtime): > Bezglich Harry Schmalzbauer's Nachricht vom 06.06.2017 14:03 (localtime): >> Hello, >> >> suddenly, I'm getting this error: >> /lib/libc.so.7: Undefined symbol "xdr_accepted_reply" >> >> Very mysterious: It showed up on a running system, which worked >> flawlessly for some hours. And that host has root-fs (/) mounted >> readonly from a memorydisk. So to my understanding, it's completely >> impossible that /lib/libc.so.7 is corrupted since last boot. >> >> I'm completely out of ideas what could cause this strange error during >> "normal" operation. >> >> Normal operation in this case is serving as a bhyve test machine. >> I first noticed that error after one guest - with passthru device >> attached - was shut down. >> >> My suspicion is some undiscovered passthru interference... Since I >> noticed one other _very_ strange passthru-effect: >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=215740 > Hello, > > this time I caught a panic with a debuging kernel under 11.1-BETA1, > which again occured after shuting down a VM which had ppt in use: > > Please, can anybody of the xperts add a comment? It turned out that it's a problem with PCIe cards which don't support FLR or cards, which are not PCIe, even if they have FLR capabilitiy. jhb@ helped me to diagnose this. Unfortunately I once forgot to manually bring down the passthrough-nics in question, which resulted in a completely destroyed ZFS pool. That hurted, so I won't rely on manual intervention before shutting down (I had to recreate the complete (system) pool). Unfortunately my skills don't allow me to help fixing the root cause, so I created a little rc(8) script, which should protect reliably. Please see also https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=222937 Since it's quite small overhead, I'll also attach it here (to be copied to /etc/rc.d). -harry --------------090102070804030709070902 Content-Type: text/plain; name="pciptdetach" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="pciptdetach" #!/bin/sh # # PROVIDE: pciptdetach # REQUIRE: swap # BEFORE: devd # KEYWORD: shutdown . /etc/rc.subr name=pciptdetach rcvar=pciptdetach_enable load_rc_config ${name} : ${pciptdetach_enable:="YES"} start_cmd="true" stop_cmd="${name}" pciptdetach() { sysctl -n hw.hv_vendor | grep -q bhyve || return 0 echo "Disabling passthrough adapters:" pptcandidate=`pciconf -l | grep -v -E \ "^([[:blank:]]|hostb|virtio|isab)[^@]+" | sed -n -E \ 's/^[[:blank:]]*(^[[:alnum:]]+)@([^[:blank:]]+)(:[[:blank:]]).*$/\2/p'` for pcidev in ${pptcandidate}; do drv_class=`pciconf -lv | grep -A 3 "@${pcidev}" | sed -n -E -e \ 's/^[[:blank:]]*class[[:blank:]]+=[[:blank:]]+([^[:blank:]].*)$/\1/p' \ -e 's/^([[:alnum:]]+)@.*$/\1/p' | tr '\n' ' '` # Don't disable mass storage devices, might be busy for shutdown [ X"${drv_class}" = X"${drv_class%mass storage*}" ] || continue # Make sure network adapters don't have active vlan(4) clones. if [ -z "${netstoped}" ] && [ X"${drv_class}" != X"${drv_class%network*}" ] then /etc/rc.d/netif stop >/dev/null 2>&1 && netstoped=y fi # Non-PCIe devices and PCIe devices without FLR support are # known to cause RAM corruption. if ! pciconf -lc ${pcidev} | grep -A 20 PCI-Express | grep -q "[[:blank:]]FLR" then devctl disable ${pcidev} >/dev/null 2>&1 || echo " ${drv_class%% *}:FAILED" fi done } run_rc_command "$1" --------------090102070804030709070902-- From owner-freebsd-virtualization@freebsd.org Wed Oct 11 19:52:37 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DC139E35ACD for ; Wed, 11 Oct 2017 19:52:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C876D75EB5 for ; Wed, 11 Oct 2017 19:52:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v9BJqbNT057178 for ; Wed, 11 Oct 2017 19:52:37 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 222590] hangs when rebooted KVM Guest Date: Wed, 11 Oct 2017 19:52:38 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: pnz.stalker@mail.ru X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Oct 2017 19:52:38 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222590 --- Comment #3 from Anton --- Update qemu resolve this issue --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Wed Oct 11 20:38:35 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 544E6E36AA5 for ; Wed, 11 Oct 2017 20:38:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 42A40776ED for ; Wed, 11 Oct 2017 20:38:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v9BKcZlS063644 for ; Wed, 11 Oct 2017 20:38:35 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 222590] hangs when rebooted KVM Guest Date: Wed, 11 Oct 2017 20:38:35 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: eugen@freebsd.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Oct 2017 20:38:35 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222590 Eugene Grosbein changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |Closed Resolution|--- |FIXED --- Comment #4 from Eugene Grosbein --- Submitter reports the problem's gone with qemu update. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Fri Oct 13 10:19:35 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D51A9E438A6 for ; Fri, 13 Oct 2017 10:19:35 +0000 (UTC) (envelope-from tarkhil@webmail.sub.ru) Received: from mail.sub.ru (mail.sub.ru [88.212.205.2]) by mx1.freebsd.org (Postfix) with SMTP id 1E6C981DC9 for ; Fri, 13 Oct 2017 10:19:34 +0000 (UTC) (envelope-from tarkhil@webmail.sub.ru) Received: (qmail 22354 invoked from network); 13 Oct 2017 13:12:50 +0300 Received: from tarkhil.infotel.ru (tarkhil.infotel.ru [195.170.223.197]) by mail.sub.ru ([88.212.205.2]) with ESMTP via TCP; 31 Dec 1969 23:59:59 -0000 To: freebsd-virtualization@freebsd.org From: =?UTF-8?B?0JDQu9C10LrRgdCw0L3QtNGAINCf0L7QstC+0LvQvtGG0LrQuNC5?= Subject: Bhyve and windows 2012/2016 Message-ID: <90634b92-dea2-0866-39fc-a1c36e1a9800@webmail.sub.ru> Date: Fri, 13 Oct 2017 13:12:50 +0300 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: ru X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Oct 2017 10:19:35 -0000 Hello I'm trying to install Windows 2012 evaluation in bhyve. It looks like neither HDD (ahci-hd) nor CD (ahci-cd) are available. Command is  /usr/sbin/bhyve  -c 2 -m 17179869184 -H -A -w -s 0:0,hostbridge    -s 3:0,ahci-hd,/usr/jails/jails-data/serv1c-data/dsk1.vhd,sectorsize=512 -s 2:0,ahci-cd,/usr/jails/src/iso/windows2012.iso  -s 4:0,e1000,tap6,mac=00:a0:98:cf:ab:82 -s 5:0,virtio-rnd  -s 6,fbuf,tcp=0.0.0.0:5902,w=1024,h=768,wait -s 30,xhci,tablet -s 31,lpc -l com1,stdio -l bootrom,/usr/local/cbsd/upgrade/patch/efi.fd serv1c Only X: drive is available, seems to be some memory image What should I check? What could have I done wrong? Alex From owner-freebsd-virtualization@freebsd.org Fri Oct 13 12:42:44 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A050E472B7 for ; Fri, 13 Oct 2017 12:42:44 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 36F89358A for ; Fri, 13 Oct 2017 12:42:44 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (ezra.dcm1.omnilan.net [78.138.80.135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id v9DCggbf052025; Fri, 13 Oct 2017 14:42:42 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 2358062E; Fri, 13 Oct 2017 14:42:42 +0200 (CEST) Message-ID: <59E0B4C1.6070907@omnilan.de> Date: Fri, 13 Oct 2017 14:42:41 +0200 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: =?UTF-8?B?0JDQu9C10LrRgdCw0L3QtNGAINCf0L7QstC+0LvQvtGG0LrQuNC5?= CC: freebsd-virtualization@freebsd.org Subject: Re: Bhyve and windows 2012/2016 References: <90634b92-dea2-0866-39fc-a1c36e1a9800@webmail.sub.ru> In-Reply-To: <90634b92-dea2-0866-39fc-a1c36e1a9800@webmail.sub.ru> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Greylist: ACL 129 matched, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [78.138.80.130]); Fri, 13 Oct 2017 14:42:42 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: 78.138.80.135; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Oct 2017 12:42:44 -0000 Bezüglich Александр Поволоцкий's Nachricht vom 13.10.2017 12:12 (localtime): > Hello > > I'm trying to install Windows 2012 evaluation in bhyve. It looks like > neither HDD (ahci-hd) nor CD (ahci-cd) are available. > > Command is > > /usr/sbin/bhyve -c 2 -m 17179869184 -H -A -w -s 0:0,hostbridge -s > 3:0,ahci-hd,/usr/jails/jails-data/serv1c-data/dsk1.vhd,sectorsize=512 > -s 2:0,ahci-cd,/usr/jails/src/iso/windows2012.iso -s > 4:0,e1000,tap6,mac=00:a0:98:cf:ab:82 -s 5:0,virtio-rnd -s > 6,fbuf,tcp=0.0.0.0:5902,w=1024,h=768,wait -s 30,xhci,tablet -s 31,lpc > -l com1,stdio -l bootrom,/usr/local/cbsd/upgrade/patch/efi.fd serv1c > > Only X: drive is available, seems to be some memory image > > What should I check? What could have I done wrong? Hmm, nut sure about vhd image struture, but I doubt you can use that as block backend. You can attach the HDD and ODD to the same controller. If you use md(4) as hdd backend, does taht change anything (mdconfig -a -t swap -s 10G and "-s 3:0,ahci,hd:/dev/mdX,cd:usr/jails/src/iso/windows2012.iso")? Or a zvol or sparse file (truncate -s 10g /your/data/path/bhyve-testdisk.file) insetad of md, just to rule out the vhd image file. -harry From owner-freebsd-virtualization@freebsd.org Fri Oct 13 13:23:00 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F12BAE47EBA for ; Fri, 13 Oct 2017 13:23:00 +0000 (UTC) (envelope-from doctor@doctor.nl2k.ab.ca) Received: from doctor.nl2k.ab.ca (doctor.nl2k.ab.ca [204.209.81.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DA4D663EFE for ; Fri, 13 Oct 2017 13:23:00 +0000 (UTC) (envelope-from doctor@doctor.nl2k.ab.ca) Received: from doctor by doctor.nl2k.ab.ca with local (Exim 4.89 (FreeBSD)) (envelope-from ) id 1e2zuj-000GdA-4K; Fri, 13 Oct 2017 07:22:09 -0600 Date: Fri, 13 Oct 2017 07:22:09 -0600 From: The Doctor To: ?????????????????? ???????????????????? Cc: freebsd-virtualization@freebsd.org Subject: Re: Bhyve and windows 2012/2016 Message-ID: <20171013132209.GA54325@doctor.nl2k.ab.ca> References: <90634b92-dea2-0866-39fc-a1c36e1a9800@webmail.sub.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <90634b92-dea2-0866-39fc-a1c36e1a9800@webmail.sub.ru> User-Agent: Mutt/1.9.1 (2017-09-22) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Oct 2017 13:23:01 -0000 On Fri, Oct 13, 2017 at 01:12:50PM +0300, ?????????????????? ???????????????????? wrote: > Hello > > I'm trying to install Windows 2012 evaluation in bhyve. It looks like > neither HDD (ahci-hd) nor CD (ahci-cd) are available. > > Command is > > ??/usr/sbin/bhyve?? -c 2 -m 17179869184 -H -A -w -s 0:0,hostbridge?????? -s > 3:0,ahci-hd,/usr/jails/jails-data/serv1c-data/dsk1.vhd,sectorsize=512 -s > 2:0,ahci-cd,/usr/jails/src/iso/windows2012.iso?? -s > 4:0,e1000,tap6,mac=00:a0:98:cf:ab:82 -s 5:0,virtio-rnd?? -s > 6,fbuf,tcp=0.0.0.0:5902,w=1024,h=768,wait -s 30,xhci,tablet -s 31,lpc -l > com1,stdio -l bootrom,/usr/local/cbsd/upgrade/patch/efi.fd serv1c > > Only X: drive is available, seems to be some memory image > > What should I check? What could have I done wrong? > > Alex > > Did you modify the ISO as you did in Windows 2012? > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" -- Member - Liberal International This is doctor@@nl2k.ab.ca Ici doctor@@nl2k.ab.ca Yahweh, Queen & country!Never Satan President Republic!Beware AntiChrist rising! https://www.empire.kred/ROOTNK?t=94a1f39b Look at Psalms 14 and 53 on Atheism Talk Sense to a fool and he calls you foolish - Euripides From owner-freebsd-virtualization@freebsd.org Fri Oct 13 15:29:57 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 75721E4AE58 for ; Fri, 13 Oct 2017 15:29:57 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2E08C699FB for ; Fri, 13 Oct 2017 15:29:56 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id v9DFTm3B095567; Fri, 13 Oct 2017 08:29:48 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id v9DFTlj1095566; Fri, 13 Oct 2017 08:29:47 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201710131529.v9DFTlj1095566@pdx.rh.CN85.dnsmgr.net> Subject: Re: Bhyve and windows 2012/2016 In-Reply-To: <90634b92-dea2-0866-39fc-a1c36e1a9800@webmail.sub.ru> To: =?UTF-8?Q?=D0=90=D0=BB=D0=B5=D0=BA=D1=81=D0=B0=D0=BD=D0=B4=D1=80_?= =?UTF-8?Q?=D0=9F=D0=BE=D0=B2=D0=BE=D0=BB=D0=BE=D1=86=D0=BA=D0=B8=D0=B9?= Date: Fri, 13 Oct 2017 08:29:47 -0700 (PDT) CC: freebsd-virtualization@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UNKNOWN-8BIT X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Oct 2017 15:29:57 -0000 > Hello > > I'm trying to install Windows 2012 evaluation in bhyve. It looks like > neither HDD (ahci-hd) nor CD (ahci-cd) are available. What version of FreeBSD are you doing this on? uname -a? > Command is > >  /usr/sbin/bhyve  -c 2 -m 17179869184 -H -A -w -s 0:0,hostbridge    -s ^^^^^^^^^^^ read the manual page very carefully, if you do not specify a unit for -m the default is to use it as megabytes, this is requesting a VM of 17G x 10^6, or 17 x 10^15 bytes of memory, you probalby want to say -m 16G > 3:0,ahci-hd,/usr/jails/jails-data/serv1c-data/dsk1.vhd,sectorsize=512 -s ^^^ bhyve does not support .vhd format disks, it supports only raw images, how was this file created? > 2:0,ahci-cd,/usr/jails/src/iso/windows2012.iso  -s > 4:0,e1000,tap6,mac=00:a0:98:cf:ab:82 -s 5:0,virtio-rnd  -s > 6,fbuf,tcp=0.0.0.0:5902,w=1024,h=768,wait -s 30,xhci,tablet -s 31,lpc -l > com1,stdio -l bootrom,/usr/local/cbsd/upgrade/patch/efi.fd serv1c ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ where is this bootrom from? The officially supported firmware should be at /usr/local/share/uefi-firmware/BHYVE_UEFI.fd > > Only X: drive is available, seems to be some memory image I believe your X: drive is the ramdisk image of the windows 2012 installer? I do not see any setup for a fbuf/vnc console, you probalby want a -s 29,fbuf,tcp=0.0.0.0:5900,w=800,h=600,wait line added to above and connect to it once your vm is started. Windows wants a graphical device, and unless you have handcrafted an autoinstall .iso that can run unattended you wont get very far without one. > > What should I check? What could have I done wrong? > > Alex -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Fri Oct 13 15:37:55 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62E28E2515E for ; Fri, 13 Oct 2017 15:37:55 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 24A6169F4A for ; Fri, 13 Oct 2017 15:37:54 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id v9DFbsgs095615; Fri, 13 Oct 2017 08:37:54 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id v9DFbs9s095614; Fri, 13 Oct 2017 08:37:54 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201710131537.v9DFbs9s095614@pdx.rh.CN85.dnsmgr.net> Subject: Re: Bhyve and windows 2012/2016 In-Reply-To: <201710131529.v9DFTlj1095566@pdx.rh.CN85.dnsmgr.net> To: "Rodney W. Grimes" Date: Fri, 13 Oct 2017 08:37:54 -0700 (PDT) CC: =?UTF-8?Q?=D0=90=D0=BB=D0=B5=D0=BA=D1=81=D0=B0=D0=BD=D0=B4=D1=80_?= =?UTF-8?Q?=D0=9F=D0=BE=D0=B2=D0=BE=D0=BB=D0=BE=D1=86=D0=BA=D0=B8=D0=B9?= , freebsd-virtualization@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UNKNOWN-8BIT X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Oct 2017 15:37:55 -0000 > > > Hello > > > > I'm trying to install Windows 2012 evaluation in bhyve. It looks like > > neither HDD (ahci-hd) nor CD (ahci-cd) are available. > > What version of FreeBSD are you doing this on? uname -a? > > > Command is > > > >  /usr/sbin/bhyve  -c 2 -m 17179869184 -H -A -w -s 0:0,hostbridge    -s > ^^^^^^^^^^^ read the manual page very carefully, > if you do not specify a unit for -m the default is to use it as megabytes, this is requesting > a VM of 17G x 10^6, or 17 x 10^15 bytes of memory, you probalby want to say > -m 16G > > > 3:0,ahci-hd,/usr/jails/jails-data/serv1c-data/dsk1.vhd,sectorsize=512 -s > ^^^ > bhyve does not support .vhd format disks, it supports only raw > images, how was this file created? > > > 2:0,ahci-cd,/usr/jails/src/iso/windows2012.iso  -s > > 4:0,e1000,tap6,mac=00:a0:98:cf:ab:82 -s 5:0,virtio-rnd  -s > > 6,fbuf,tcp=0.0.0.0:5902,w=1024,h=768,wait -s 30,xhci,tablet -s 31,lpc -l ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Ooopss.. ignore my comments about fbuf/vnc, I missed it. > > com1,stdio -l bootrom,/usr/local/cbsd/upgrade/patch/efi.fd serv1c > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > where is this bootrom from? The officially supported firmware > should be at /usr/local/share/uefi-firmware/BHYVE_UEFI.fd > > > > > Only X: drive is available, seems to be some memory image > > I believe your X: drive is the ramdisk image of the windows 2012 > installer? > > I do not see any setup for a fbuf/vnc console, you probalby want a > -s 29,fbuf,tcp=0.0.0.0:5900,w=800,h=600,wait > line added to above and connect to it once your vm is started. > Windows wants a graphical device, and unless you have handcrafted > an autoinstall .iso that can run unattended you wont get very > far without one. > > > > > What should I check? What could have I done wrong? > > > > Alex > > -- > Rod Grimes rgrimes@freebsd.org > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Fri Oct 13 15:48:17 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C75A1E25540 for ; Fri, 13 Oct 2017 15:48:17 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A3B086A430 for ; Fri, 13 Oct 2017 15:48:17 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (unknown [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 7B0741356C for ; Fri, 13 Oct 2017 15:48:15 +0000 (UTC) Subject: Re: Bhyve and windows 2012/2016 To: freebsd-virtualization@freebsd.org References: <201710131529.v9DFTlj1095566@pdx.rh.CN85.dnsmgr.net> From: Allan Jude Message-ID: Date: Fri, 13 Oct 2017 11:48:14 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <201710131529.v9DFTlj1095566@pdx.rh.CN85.dnsmgr.net> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="wQ3WvkQftK7FUwotEhPhjt8aXeOXCc5F1" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Oct 2017 15:48:17 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --wQ3WvkQftK7FUwotEhPhjt8aXeOXCc5F1 Content-Type: multipart/mixed; boundary="B48t2hM1QXSmiOlFlc1ndBbp691T0kgA9"; protected-headers="v1" From: Allan Jude To: freebsd-virtualization@freebsd.org Message-ID: Subject: Re: Bhyve and windows 2012/2016 References: <201710131529.v9DFTlj1095566@pdx.rh.CN85.dnsmgr.net> In-Reply-To: <201710131529.v9DFTlj1095566@pdx.rh.CN85.dnsmgr.net> --B48t2hM1QXSmiOlFlc1ndBbp691T0kgA9 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 2017-10-13 11:29, Rodney W. Grimes wrote: >=20 >> Hello >> >> I'm trying to install Windows 2012 evaluation in bhyve. It looks like = >> neither HDD (ahci-hd) nor CD (ahci-cd) are available. >=20 > What version of FreeBSD are you doing this on? uname -a? >=20 >> Command is >> >> =C2=A0/usr/sbin/bhyve=C2=A0 -c 2 -m 17179869184 -H -A -w -s 0:0,hostb= ridge=C2=A0=C2=A0=C2=A0 -s=20 > ^^^^^^^^^^^ read the manual = page very carefully, > if you do not specify a unit for -m the default is to use it as megabyt= es, this is requesting > a VM of 17G x 10^6, or 17 x 10^15 bytes of memory, you probalby want to= say > -m 16G >=20 >> 3:0,ahci-hd,/usr/jails/jails-data/serv1c-data/dsk1.vhd,sectorsize=3D51= 2 -s=20 > ^^^ > bhyve does not support .vhd format disks, it supports only raw > images, how was this file created? You can use qemu-img or VBoxManage to convert the .vhd into a .raw so it will work with bhyve >=20 >> 2:0,ahci-cd,/usr/jails/src/iso/windows2012.iso=C2=A0 -s=20 >> 4:0,e1000,tap6,mac=3D00:a0:98:cf:ab:82 -s 5:0,virtio-rnd=C2=A0 -s=20 >> 6,fbuf,tcp=3D0.0.0.0:5902,w=3D1024,h=3D768,wait -s 30,xhci,tablet -s 3= 1,lpc -l=20 >> com1,stdio -l bootrom,/usr/local/cbsd/upgrade/patch/efi.fd serv1c > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > where is this bootrom from? The officially supported firmware > should be at /usr/local/share/uefi-firmware/BHYVE_UEFI.fd=20 >=20 >> >> Only X: drive is available, seems to be some memory image >=20 > I believe your X: drive is the ramdisk image of the windows 2012 > installer? >=20 > I do not see any setup for a fbuf/vnc console, you probalby want a > -s 29,fbuf,tcp=3D0.0.0.0:5900,w=3D800,h=3D600,wait > line added to above and connect to it once your vm is started. > Windows wants a graphical device, and unless you have handcrafted > an autoinstall .iso that can run unattended you wont get very=20 > far without one. >=20 >> >> What should I check? What could have I done wrong? >> >> Alex >=20 >=20 >=20 > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@fr= eebsd.org" >=20 --=20 Allan Jude --B48t2hM1QXSmiOlFlc1ndBbp691T0kgA9-- --wQ3WvkQftK7FUwotEhPhjt8aXeOXCc5F1 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJZ4OA+AAoJEBmVNT4SmAt+1fcP/2m9aWpW2UYF6BX5tzGA4C+0 L2DwS7P7o7VA7yJBOA1w6lrSCjUvaMvaBUT0TEHw3fWdJQrpDivXDWZgWZdhYsSB PDzXdb+W1Mi+ZTu3S9YB6+DGb9tWpTDBR9PgVtUq84RYcP+xQJtK7gXruVgTWjR2 uFt3zzpFMRfMe0Y5kJFrmMSXDsY67MgEDNZ/p2XL5AKnTEHj1aej+VTUgOJ2Qufu p6GfEox/wlDzcAXpvsQGVjKxbQhR5WLZn1v6qcEWGOx2iSH5mTGocPJ6EdzLuQgC GaH61UrZuDqDE909rZvBNxBVHtJ3p0JqIu+JJ4zj+PHOzc+jdOnzwR0Dc/SBuyrg /blGO3Jn3nXoCpOGjatAqe/6JUpx6HaEiYIzgXPU2lhGOyNHvgUKxNGUIs59Tsgj 5c/sK83HG/8Ksieb3ib3LcUhWkr+NnhQXnuswpH5DOz2hybgA4y/793gOGaYzkiA q6A3DttKefynycvZVmvHT4ONCiL5F8nYky3J1NnoZAO9OqUDF9aEfKdXE4TtUbh8 hY72toXS2GpWdMfrZ+4TstOUV1ij1S4mZHDXh+WbtjSowNqMaSk1pVGOzmUkfBSS SalFk/wXWn9/Yd0nRnyUHDCHg0tV3gcMSRhZw+C15qQg8x6Dlm76qmoJR+kAAwIh RjsmqXCy444Os7KdNsTh =fjmO -----END PGP SIGNATURE----- --wQ3WvkQftK7FUwotEhPhjt8aXeOXCc5F1-- From owner-freebsd-virtualization@freebsd.org Sat Oct 14 00:34:18 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 01B2CE313F2 for ; Sat, 14 Oct 2017 00:34:17 +0000 (UTC) (envelope-from paul.g.webster@googlemail.com) Received: from mail-qt0-x22b.google.com (mail-qt0-x22b.google.com [IPv6:2607:f8b0:400d:c0d::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ACDA57ED8C for ; Sat, 14 Oct 2017 00:34:17 +0000 (UTC) (envelope-from paul.g.webster@googlemail.com) Received: by mail-qt0-x22b.google.com with SMTP id 8so22601657qtv.1 for ; Fri, 13 Oct 2017 17:34:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=yVPsZz2CFdhmEH3aUk/PITL9BmiYWSykZmxVbr6Yh5g=; b=eqT93KtbH2138DW1y8SZ+ghjITmnyumDE/A5EH70WrCoiUtU8DsosmqRrtf//rSXaZ JvkHe7fRRQieKxHsTGeI/BTqHbEdmI6iDm2L4pNWZbcbYm6p5Xu9EjTwr51cXbz6/A3w g8CNAk8ZqNkmEhuvM04jE8krJHIIX6RQvS0u3pNFyAMdkBdWHslNOP8BzMDTTI5ReyXV G3inG9VTgG34tnBB1/GVNByxpHDH8xPMgfwPgx5/WxYjg8VEUiRiEHueWGzjiukhwDBW sAH+LxMhnWroEAUs97nSpwKbFtzrOLMqMGwauV0MfRLrsJBsi/sKvoIqEr3ulF/l0UpT Xjiw== X-Gm-Message-State: AMCzsaVqz7iVE6zWxeYV+kIST1FJIcUeX6OBfk4+nNTLTT7XTWdkwtyT tqtE6czNmzWh0ZXuPIuZqMjrDrieNIZdpXCu1C6QQw== X-Google-Smtp-Source: AOwi7QA8LNjh8+ECXLvzZSh1vCOeXsNWk87QGEppv8mBWrjU8IHi+WerqBaGfR0wrbHwVAGoj+++Ck7C8hlihrII0f4= X-Received: by 10.13.207.198 with SMTP id r189mr2043419ywd.287.1507941256645; Fri, 13 Oct 2017 17:34:16 -0700 (PDT) MIME-Version: 1.0 Received: by 10.37.175.136 with HTTP; Fri, 13 Oct 2017 17:34:16 -0700 (PDT) In-Reply-To: <90634b92-dea2-0866-39fc-a1c36e1a9800@webmail.sub.ru> References: <90634b92-dea2-0866-39fc-a1c36e1a9800@webmail.sub.ru> From: Paul Webster Date: Sat, 14 Oct 2017 01:34:16 +0100 Message-ID: Subject: Re: Bhyve and windows 2012/2016 To: =?UTF-8?B?0JDQu9C10LrRgdCw0L3QtNGAINCf0L7QstC+0LvQvtGG0LrQuNC5?= Cc: freebsd-virtualization@freebsd.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Oct 2017 00:34:18 -0000 Hey there, I got it working in stable using VNC for the actual 'vga installation processes' no need for vbox or anything else; I am using and still use this config: $ cat kenny.tmp.group #!/bin/sh bhyve -A -H -P \ -s 0,hostbridge \ -s 2,virtio-net,tap4 \ -s 4,ahci-cd,/vm/src/2012.R2VL.ESD.ENU.June2016.iso \ -s 5,ahci-hd,/dev/zvol/zdisk/vm/store/kenny \ -s 29,fbuf,tcp=3D172.19.18.1:5900,w=3D1024,h=3D768 \ -s 30,xhci,tablet \ -s 31,lpc \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ -c 8 \ -m 6144M \ kenny bhyvectl --destroy --vm=3Dkenny FreeBSD version and related packages: 11.1-STABLE #0 r322131 uefi-edk2-bhyve-20160704_1 UEFI-EDK2 firmware for bhyve uefi-edk2-bhyve-csm-20160704_1 UEFI-EDK2 firmware for bhyve with CSM On 13 October 2017 at 11:12, =D0=90=D0=BB=D0=B5=D0=BA=D1=81=D0=B0=D0=BD=D0= =B4=D1=80 =D0=9F=D0=BE=D0=B2=D0=BE=D0=BB=D0=BE=D1=86=D0=BA=D0=B8=D0=B9 wrote: > Hello > > I'm trying to install Windows 2012 evaluation in bhyve. It looks like > neither HDD (ahci-hd) nor CD (ahci-cd) are available. > > Command is > > /usr/sbin/bhyve -c 2 -m 17179869184 -H -A -w -s 0:0,hostbridge -s > 3:0,ahci-hd,/usr/jails/jails-data/serv1c-data/dsk1.vhd,sectorsize=3D512 -= s > 2:0,ahci-cd,/usr/jails/src/iso/windows2012.iso -s > 4:0,e1000,tap6,mac=3D00:a0:98:cf:ab:82 -s 5:0,virtio-rnd -s 6,fbuf,tcp= =3D > 0.0.0.0:5902,w=3D1024,h=3D768,wait -s 30,xhci,tablet -s 31,lpc -l com1,st= dio > -l bootrom,/usr/local/cbsd/upgrade/patch/efi.fd serv1c > > Only X: drive is available, seems to be some memory image > > What should I check? What could have I done wrong? > > Alex > > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubs > cribe@freebsd.org" From owner-freebsd-virtualization@freebsd.org Sat Oct 14 00:59:47 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7341CE31AED for ; Sat, 14 Oct 2017 00:59:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 607237F8A8 for ; Sat, 14 Oct 2017 00:59:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v9E0xlpU010685 for ; Sat, 14 Oct 2017 00:59:47 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 222916] [bhyve] Debian guest kernel panics with message "CPU#0 stuck for Xs!" Date: Sat, 14 Oct 2017 00:59:47 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Oct 2017 00:59:47 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222916 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-virtualization@Free | |BSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Sat Oct 14 01:03:30 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3D9EFE31E97 for ; Sat, 14 Oct 2017 01:03:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 24ECC7FDCF for ; Sat, 14 Oct 2017 01:03:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v9E13TdA031881 for ; Sat, 14 Oct 2017 01:03:30 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 222937] [byhve] Severe RAM corruption after PciPassThrough-guest shutdown Date: Sat, 14 Oct 2017 01:03:29 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to keywords Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Oct 2017 01:03:30 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222937 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-bugs@FreeBSD.org |freebsd-virtualization@Free | |BSD.org Keywords| |patch --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Sat Oct 14 01:33:22 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2C3AE3396E for ; Sat, 14 Oct 2017 01:33:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A10338120E for ; Sat, 14 Oct 2017 01:33:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v9E1XMmC004850 for ; Sat, 14 Oct 2017 01:33:22 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 222916] [bhyve] Debian guest kernel panics with message "CPU#0 stuck for Xs!" Date: Sat, 14 Oct 2017 01:33:22 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: grehan@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Oct 2017 01:33:22 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222916 Peter Grehan changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |grehan@FreeBSD.org --- Comment #2 from Peter Grehan --- The "CPU stuck" messages generally occurr when a system is oversubscribed i= .e. the guest has been given more virtual resources than are physically availab= le. In this case, the guest has been assigned the same number of vCPUs as the h= ost (assuming Intel-ARK is correct here: https://ark.intel.com/products/97478/Intel-Xeon-Processor-E3-1275-v6-8M-Cac= he-3_80-GHz) so if there is a time when both the host and guest require all CPUs, there = is only 4 available to service the needs of 8 assumed available (4 host, 4 gue= st). In addition, the guest has been assigned 32G of RAM, and ZFS is being used = on the host. The issue there is that the default ZFS setup on FreeBSD will all= ow use of all RAM minus 1GB for ZFS ARC. This competes with use by bhyve (swap-backed RAM), and can result in excessive swapping by the bhyve proces= s. This can also result in "CPU stuck" messages as vCPUs are halted while awai= ting guest physical memory to be paged in. A recommendation would be to a) restrict ZFS ARC usage to less than that required by the host and bhy= ve guests. In this case, perhaps 16-24G (if the mobo has 64G) ? This can be do= ne using the vfs.zfs.arc_max parameter in /boot/loader.conf b) Only use 2 vCPUs for the guest, or, enable hyperthreading. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Sat Oct 14 01:52:06 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B4122E34979 for ; Sat, 14 Oct 2017 01:52:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A23B681D97 for ; Sat, 14 Oct 2017 01:52:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v9E1q6R9048548 for ; Sat, 14 Oct 2017 01:52:06 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 222916] [bhyve] Debian guest kernel panics with message "CPU#0 stuck for Xs!" Date: Sat, 14 Oct 2017 01:52:06 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: karihre@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Oct 2017 01:52:06 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222916 --- Comment #3 from karihre@gmail.com --- Thank you for the quick reply. Hyperthreading was initially enabled, with t= he guest assigned 4 vCPUs, but disabled as part of the debug process due to re= cent problems with hyper-threading on Intel processors. The memory in the machine was indeed 64GB, and ZFS ARC usage was not being limited like you suggested. I have now set the limit to 20GB and plan to re= boot soon to see if this resolves the issue. I have also (since last boot of the guest) limited the vCPUs to 1 as part of the debug process. The system has = yet to hang, but in general it survived a few days before running into this iss= ue, so the verdict is still in on that mitigation. I will post as soon as I get the next crash, or consider this resolved if everything appears stable for the coming weeks. I'm not sure if you want to mark the bug as closed, or it can remain open until something (or nothing) happens. Thank you, Kari Hreinsson --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Sat Oct 14 02:21:20 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8F73DE3603C for ; Sat, 14 Oct 2017 02:21:20 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7D67782E3E for ; Sat, 14 Oct 2017 02:21:20 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v9E2LKG2011021 for ; Sat, 14 Oct 2017 02:21:20 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 222916] [bhyve] Debian guest kernel panics with message "CPU#0 stuck for Xs!" Date: Sat, 14 Oct 2017 02:21:20 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: grehan@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Oct 2017 02:21:20 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222916 --- Comment #4 from Peter Grehan --- Single vCPU guests perform very well in oversubscribed environments since t= here are no lock-spins against other possibly de-scheduled vCPUs. However, you s= till may hit the ZFS memory competition issue. Note that you can set the ARC-max= on the fly in 11.1 and avoid the reboot, using the vfs.zfs.arc_max sysctl var.= ZFS should slowly drop down to this value if it is above it. >not sure if you want to mark the bug as closed Fine to keep this open until the issue is resolved. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Sat Oct 14 06:15:32 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C6B97E3E428 for ; Sat, 14 Oct 2017 06:15:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B4DCD650C7 for ; Sat, 14 Oct 2017 06:15:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v9E6FWw6003223 for ; Sat, 14 Oct 2017 06:15:32 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 222916] [bhyve] Debian guest kernel panics with message "CPU#0 stuck for Xs!" Date: Sat, 14 Oct 2017 06:15:32 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: karihre@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Oct 2017 06:15:32 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222916 --- Comment #5 from karihre@gmail.com --- Thanks for that tip. I reduced the ARC size with sysctl and confirmed it to= be 20GB with zfs-info. Thinking back to the 4 guest / 4 host cpus. Lets say the collection of gues= ts consume 4 cpus and four tasks on the host consume 4 cpus (totaling a load average of 8), does the host system scheduler not shuffle tasks around like= it would if I were running 8 cpu intensive processes on the host? Or does the interaction between bhyve and the host scheduler somehow result in the virt= ual cpus being set aside for tens of seconds? I guess I'm just trying to understand, I would think one of the main motivations for using a hypervisor is exactly over-subscribing cpu cores as= you may have guests with "bursty" load behavior, so on average your total guests+host load is less than the number of cpus, but surely you can divide= the cpu time in a "fair" manner when the system is overloaded. Memory I would think is a little trickyer, there it makes sense to make sure the host system consumption + guest consumption never exceeds the total host memory. Anyhow, just trying to make sense of this, there doesn't seem to be too much information available online on these topics, or perhaps I'm looking in all= the wrong places. Thank you, Kari --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Sat Oct 14 17:12:44 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BF6DEE27D3E for ; Sat, 14 Oct 2017 17:12:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AD62176038 for ; Sat, 14 Oct 2017 17:12:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v9EHCiTg094280 for ; Sat, 14 Oct 2017 17:12:44 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 222916] [bhyve] Debian guest kernel panics with message "CPU#0 stuck for Xs!" Date: Sat, 14 Oct 2017 17:12:44 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: grehan@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Oct 2017 17:12:44 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222916 --- Comment #6 from Peter Grehan --- >Or does the interaction between bhyve and the host scheduler somehow >result in the virtual cpus being set aside Yes, though: > for tens of seconds? The error message from Linux is a bit misleading. There is a low-priority kernel thread that tries to run every 5 seconds and then sleeps. If it hasn= 't been able to run for an extended amount of time, for example due to high interrupt activity, higher priority threads running, or spinlocks being hel= d, the error message will be displayed. What I believe you are seeing is a classic hypervisor problem, not specifi= c to bhyve, known as "lock-holder preemption" where a vCPU holding a spin-lock is preempted by the host, and other vCPUs that are running then spin attemptin= g to acquire that lock which can't be released. A search will show the large amo= unt of literature on this issue :) Maybe the best reading on this is the ESXi scheduler paper: =20=20=20 http://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/techpaper/= vmware-vsphere-cpu-sched-performance-white-paper.pdf There has been some talk of putting knowledge of vCPUs in the FreeBSD scheduler to allow some form of gang scheduling, but nothing has come of th= at so far. As to your point; it's more than just fairness that the hypervisor schedul= er has to provide - heuristics about guest o/s behaviour are also needed. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Sat Oct 14 23:44:09 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F4162E2FC9C for ; Sat, 14 Oct 2017 23:44:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E1E2C79E for ; Sat, 14 Oct 2017 23:44:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v9ENi8nn070940 for ; Sat, 14 Oct 2017 23:44:08 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 222996] FreeBSD 11.1 on Hyper-V with PCI Express Pass Through Date: Sat, 14 Oct 2017 23:44:08 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.1-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to component Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Oct 2017 23:44:09 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222996 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-standards@FreeBSD.o |freebsd-virtualization@Free |rg |BSD.org Component|standards |bin --=20 You are receiving this mail because: You are the assignee for the bug.=