From owner-freebsd-virtualization@freebsd.org Tue Dec 26 00:12:55 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 028A5E84333 for ; Tue, 26 Dec 2017 00:12:55 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id D3AB079D05 for ; Tue, 26 Dec 2017 00:12:54 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id D2FBBE84330; Tue, 26 Dec 2017 00:12:54 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D258EE8432F; Tue, 26 Dec 2017 00:12:54 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-io0-x230.google.com (mail-io0-x230.google.com [IPv6:2607:f8b0:4001:c06::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A2B3279D03; Tue, 26 Dec 2017 00:12:51 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: by mail-io0-x230.google.com with SMTP id q188so7042874iod.1; Mon, 25 Dec 2017 16:12:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=yYz7/82v+JZ3K8dcoIS12RKIdDqY4l1NYK6OetPJd48=; b=LRsIC8Hp0+lveVWUINcbaeR6Ej0S6e7wVKO/4Y6Ga9uwUWNUvRvEHQTdSt82oL0wy9 FSyaI3fF5y9dm8K8VgoI2PLW/dj2PS2rLUaNwjJ0zgnWsRdOq4MZ15UhwiOftaop6D8Q AEWHr6CGxaPTjvpB4POj439UdtihgXjdH++Eyu7oVR8ncF00IFfNUGycKIsNanP+fM4s KlwOo7iMAkiNb9Z9Au9sTXtPolzTpeAR/HhKjr3zLKoX7BEKgJwv4alP3CTXQUNrv8zn 4buK7VjIsj6cGLCwJNSn7VsPymuzoe6bQuPV22PsxDLPMDeER1l8SoBzEMo9ZP/Nv7PP q0kQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=yYz7/82v+JZ3K8dcoIS12RKIdDqY4l1NYK6OetPJd48=; b=lUWVoRMDD29o+4XmClcNlN+IAsPR2ojqTRRh9kjmAZQuSW8c8z3D0HC9VgS7RusRKM b+M30O4cUr2lXtnUyKA3OwoDGnaoVMsBYGSt1rAKzrHvyN2r/9JganmWnlh6AtI2eCOG bzB20lMJW5c/PSXFxmSHdmmjNCusO/3b2ov78E5mizRFmy0FIA6IxIPoOLpXX8sKk4E+ dbVzAnKAu2FHyuUZURZ12Sp/oN185o5SNUKgW5dtNv6UUr3r0+zlEdyf4Js3aI+Gl2MG eJ4jx0E6mIgcJI/iqAyBx4JoXlUIxbnJmc3CJAFzJP4iUqwc8h63F+5CI8+4M0y/Oa7D hP5A== X-Gm-Message-State: AKGB3mLTtQpmVsWARj7MQLvAvWalNpSHd3PAiKI0TG87g+jCmv7c0hC7 AY3vXOfTciu1+wEDUEvtwBpOgOIG8oVCZiIIIBk= X-Google-Smtp-Source: ACJfBoti/swAPZYdyBKzGHAkYTzPxHXbo90Lf11kQdzk7BT1TMRF5gUsTGRvVNwpmjqiQd16yqkhw3Csc/w529tb45w= X-Received: by 10.107.232.9 with SMTP id f9mr33167601ioh.157.1514247170564; Mon, 25 Dec 2017 16:12:50 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.105.3 with HTTP; Mon, 25 Dec 2017 16:12:50 -0800 (PST) From: Aryeh Friedman Date: Mon, 25 Dec 2017 19:12:50 -0500 Message-ID: Subject: Running dual boot windows inside of bhyve To: "freebsd-virtualization@freebsd.org" , FreeBSD Mailing List , FreeBSD Mailing List Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Dec 2017 00:12:55 -0000 Cross posted to virtualization@, hackers@ and questions. I have a dual boot machine (windows 7 64 bit and fbsd 11.1-RELEASE [amd64]) and want to run the windows partition as a vm in bhyve how would I go about this. Bonus if the process is standalone scriptable so I can add it as a feature of PetiteCloud. -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-virtualization@freebsd.org Wed Dec 27 16:25:16 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A7DAE9F6D3 for ; Wed, 27 Dec 2017 16:25:16 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id E79706BFE0 for ; Wed, 27 Dec 2017 16:25:15 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id E3E99E9F6D0; Wed, 27 Dec 2017 16:25:15 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E161EE9F6CF; Wed, 27 Dec 2017 16:25:15 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-it0-x22c.google.com (mail-it0-x22c.google.com [IPv6:2607:f8b0:4001:c0b::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B661D6BFDF; Wed, 27 Dec 2017 16:25:15 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: by mail-it0-x22c.google.com with SMTP id p139so25644990itb.1; Wed, 27 Dec 2017 08:25:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=GFKo6/7hsbnfTl6MipDjGxvqlfEc64cjVaVn8Sm3jIQ=; b=AHM6enSykesfvFjO3daWAbQ8MJzvv2tUk+kegTBKh8akW9agjtHiYjdXeTu8rFNaVM /ACri89XH7k6ywX9AHH5Yo5/qu7/QHdxG3NoDQ17aNz0J8YoveIP/VrQ8sQ7u4uA8uLi +ACnXLMWdD0tT8DArzuXfPh11D8o+hSo+OkUoevpYqpbvS4ztj5y1c3yK1gderg2gIo+ hNopkyDQVc1r5cS99U9MOv9v0h3dDt9yK5DQAt5/ilwRrOAcioFtd4m8WROpyOAePXCL +Hwo7J4Yh6hkhtyq4U9eIFkxp568FC5bdq+Gfxsz3AlyjKwW2xvj8TBjavtBE9D+ZLdv A19g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=GFKo6/7hsbnfTl6MipDjGxvqlfEc64cjVaVn8Sm3jIQ=; b=k3dx3HF5867JEJaBMBiXi4A+dPDaB/m4OscO+tFDVVExwemOgXIUoagnxV3e10ILFd PkQcvaKOX50lTmxOfE3bqdqzSOPAIKUC/q39d3lGS0wyk8OXt1jWDtDvIFF0UmkqJ8s6 5fGQzZUqV+VqObtgIl48by4KaJAaGJQig2G99xrmSrtWTozf91Hq9OtF1ipoRru7RpAr ht3sKv+nFeUmrcPu+BcHFpTeDZDqJwInitCq3n6qHzgjIZOclAZNnS3G3O6ycCienM8B ikq+8qqKwr2OEpl19Xu8qihhDFYgBATmqkDneT4oBll7B0kzv/00nFQFy+GQT0xTtVUT njIw== X-Gm-Message-State: AKGB3mJDnUgqxF4SQYdK5Iwvs3sN8FZ6lXpaMLtzIkPXt3YZAemaxkqE g1Pex2RT3K4mUf8joeZCMgwyq6s1RibtWTasAwI= X-Google-Smtp-Source: ACJfBot5Ki+gPGcrrhKjQQUz3BadxL76nC2sV7LJsydaW4tLamagDKkc6D5hjqalLdsyccV8R9fyznBhzQgbhGJb7d0= X-Received: by 10.36.129.212 with SMTP id q203mr40128866itd.152.1514391914949; Wed, 27 Dec 2017 08:25:14 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.105.3 with HTTP; Wed, 27 Dec 2017 08:25:14 -0800 (PST) In-Reply-To: References: From: Aryeh Friedman Date: Wed, 27 Dec 2017 11:25:14 -0500 Message-ID: Subject: Re: Running dual boot windows inside of bhyve To: Joe Nosay Cc: "freebsd-virtualization@freebsd.org" , FreeBSD Mailing List , FreeBSD Mailing List Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2017 16:25:16 -0000 On Wed, Dec 27, 2017 at 11:17 AM, Joe Nosay wrote: > I'm curious as to why you would choose a lesser reliable system as the > guest. > Since I have several other VM's also running on it (they are where I do the torture testing of a webapp I maintain as well as browser compatibility [the reason for win 7/IE 11.0 is the client's largest customer uses that and *REFUSES* to upgrade/switch {doctors are even more stubborn then programmers} to a better/newer browser/os). Thus I often need to run two VM's at once on it (one is a copy of FBSD that can repeately be blown up in testing and the other is a copy of the OS we are having browser/client issues with at the moment). > The first thing you need to do is copy the partition to another drive. > Impractical since all bays are used. > On Mon, Dec 25, 2017 at 7:12 PM, Aryeh Friedman > wrote: > >> Cross posted to virtualization@, hackers@ and questions. >> >> I have a dual boot machine (windows 7 64 bit and fbsd 11.1-RELEASE >> [amd64]) >> and want to run the windows partition as a vm in bhyve how would I go >> about >> this. Bonus if the process is standalone scriptable so I can add it as a >> feature of PetiteCloud. >> >> -- >> Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org >> " >> > > -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-virtualization@freebsd.org Wed Dec 27 17:15:16 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 58869EA2019 for ; Wed, 27 Dec 2017 17:15:16 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 321896DA11 for ; Wed, 27 Dec 2017 17:15:16 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 2E735EA2016; Wed, 27 Dec 2017 17:15:16 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2DD12EA2015; Wed, 27 Dec 2017 17:15:16 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lf0-x229.google.com (mail-lf0-x229.google.com [IPv6:2a00:1450:4010:c07::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BCD0D6DA10; Wed, 27 Dec 2017 17:15:15 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lf0-x229.google.com with SMTP id o26so28709034lfc.10; Wed, 27 Dec 2017 09:15:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=zy3Kwhdy74Tl5mXUkiB0n6nYdDWjbRF1ZlxU33xj1Do=; b=u0/tZeSpyxyxlBDWJExJoAJt+pLNRKO3p3cQDE9RtRW7alHYx5MAd64r6whkoFl7Gt kO9mFsNVp1trvfyr7X6a5uundEfRGruwyM8I3qnmZsAS+q7kyAo+T8W+8XTHG2M3YfbA FTZOZKxMKQDnmVfyTmIOUu2fQ6447V3KjFbfW1Y3EzzILpXyeucGQ/7UhCB7/jUcl6pY gxF7h/pL9PEq8xzL6wC9FzZvbgdZSnd9TG1g3uz2TVwvfrqcxClLc1l44uNSHaNiP8gW X0H640P+CX9ub2kMazg2n7G6ydW7N9zYp6xKbKFMyOBs7V8d5OdV+TpIQNb52PmFMjNe WFDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=zy3Kwhdy74Tl5mXUkiB0n6nYdDWjbRF1ZlxU33xj1Do=; b=NNmAUBAfJdwHphYKyociM8l82Jhx1M6fyoLFcrGbbkgGGmS9oX4KUfkTGyjdGY+7Ea sFjPr9zkQnPfG7ZuZ9aQJ9crLlXsKLtIDRioTgJnZxs55wSdzpfxy02cip4HCHOuI1yX PLTF4yfZi3SON8QE+F2UUGkGxPEoJ0skrqVsFWRYcdlyUdlw/oL81Zy0mn24/bJDobz9 hoQ00QAgS1l1Wv5yNPJ+sTOVURZfFxfUl7JCki3QJ2bIuBLvEF+yr2C82XlQrrJwdJgQ QthCAy/9+boMXB/Ucl9jpI1e8rbJhXDqUyHzOcFa7plYTs09QK3FR74O+UXpCuXgMee6 /RjQ== X-Gm-Message-State: AKGB3mIirmkPmtMZP9W4yytKV6LR3NILgkfY14OpUw/G3fRJtQkZXrFw JGzl2A4bWynhquIhOJgDTqsnK+Wa0G9+iVUujSc= X-Google-Smtp-Source: ACJfBotZ68mcEnlPLJXwFgGbtcKJZJBcpDcthxvMSrWlO9V+WW42wQzpPclBWRAZ7Nt9DdmP19VbsxDi108Q3DthZNU= X-Received: by 10.25.151.211 with SMTP id z202mr1769523lfd.51.1514394913713; Wed, 27 Dec 2017 09:15:13 -0800 (PST) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.179.76.19 with HTTP; Wed, 27 Dec 2017 09:15:12 -0800 (PST) In-Reply-To: References: From: Alan Somers Date: Wed, 27 Dec 2017 10:15:12 -0700 X-Google-Sender-Auth: ULpyuG8OLoj6lkiBlf2ao30p5Ps Message-ID: Subject: Re: Running dual boot windows inside of bhyve To: Aryeh Friedman Cc: "freebsd-virtualization@freebsd.org" , FreeBSD Mailing List , FreeBSD Mailing List Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2017 17:15:16 -0000 On Mon, Dec 25, 2017 at 5:12 PM, Aryeh Friedman wrote: > Cross posted to virtualization@, hackers@ and questions. > > I have a dual boot machine (windows 7 64 bit and fbsd 11.1-RELEASE [amd64]) > and want to run the windows partition as a vm in bhyve how would I go about > this. Bonus if the process is standalone scriptable so I can add it as a > feature of PetiteCloud. > > I fear that you may be out of luck. Windows deliberately frustrates this use case by profiling its hardware at installation time and at every boot thereafter. If the hardware changes too much, then Windows demands a new license fee. Moving from physical hardware to a VM would probably trip every one of its alarms. For this reason, I would recommend using a separate instance for your VM. However, if you do use a shared instance, then make sure that your PC is booting in UEFI mode. BHyve can only boot Windows in that mode. If you do that, then BHyve will probably be able to boot it just fine. -Alan From owner-freebsd-virtualization@freebsd.org Wed Dec 27 17:37:28 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1822DEA3315 for ; Wed, 27 Dec 2017 17:37:28 +0000 (UTC) (envelope-from rebecca@bluestop.org) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id F1B546E844 for ; Wed, 27 Dec 2017 17:37:27 +0000 (UTC) (envelope-from rebecca@bluestop.org) Received: by mailman.ysv.freebsd.org (Postfix) id EE0FBEA3311; Wed, 27 Dec 2017 17:37:27 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EB8A4EA3310; Wed, 27 Dec 2017 17:37:27 +0000 (UTC) (envelope-from rebecca@bluestop.org) Received: from muon.bluestop.org (muon.bluestop.org [IPv6:2605:7700:0:8:1:0:4a32:3323]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CB97C6E843; Wed, 27 Dec 2017 17:37:27 +0000 (UTC) (envelope-from rebecca@bluestop.org) Received: from muon.bluestop.org (localhost [127.0.0.1]) by muon.bluestop.org (Postfix) with ESMTP id AE2BC7301; Wed, 27 Dec 2017 17:37:19 +0000 (UTC) Received: from muon.bluestop.org ([127.0.0.1]) by muon.bluestop.org (muon.bluestop.org [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id d2RdyqRhL7nT; Wed, 27 Dec 2017 17:37:19 +0000 (UTC) Received: from [10.0.10.140] (gw.bluestop.org [96.73.9.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by muon.bluestop.org (Postfix) with ESMTPSA; Wed, 27 Dec 2017 17:37:18 +0000 (UTC) Subject: Re: Running dual boot windows inside of bhyve To: Alan Somers , Aryeh Friedman Cc: "freebsd-virtualization@freebsd.org" , FreeBSD Mailing List , FreeBSD Mailing List References: From: Rebecca Cran Message-ID: <7352645c-ecf1-008b-e61a-29b580d1f98c@bluestop.org> Date: Wed, 27 Dec 2017 10:37:18 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2017 17:37:28 -0000 On 12/27/2017 10:15 AM, Alan Somers wrote: > > I fear that you may be out of luck. Windows deliberately frustrates this > use case by profiling its hardware at installation time and at every boot > thereafter. If the hardware changes too much, then Windows demands a new > license fee. Moving from physical hardware to a VM would probably trip > every one of its alarms. Fortunately modern versions at least allow you to move the installation to a different machine without requiring a reinstall. You might be told you need to relicense it, and it will take some time to reconfigure itself to the new hardware it finds itself running on, but it'll work at least. -- Rebecca From owner-freebsd-virtualization@freebsd.org Wed Dec 27 20:44:18 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ECE0DEAC24A for ; Wed, 27 Dec 2017 20:44:18 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id D55D477627 for ; Wed, 27 Dec 2017 20:44:18 +0000 (UTC) (envelope-from hausen@punkt.de) Received: by mailman.ysv.freebsd.org (Postfix) id D1D24EAC247; Wed, 27 Dec 2017 20:44:18 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D12AEEAC246; Wed, 27 Dec 2017 20:44:18 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from kagate.punkt.de (kagate.punkt.de [217.29.33.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4E87377626; Wed, 27 Dec 2017 20:44:17 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from hugo10.ka.punkt.de (hugo10.ka.punkt.de [217.29.44.10]) by gate2.intern.punkt.de with ESMTP id vBRKfLqY080979; Wed, 27 Dec 2017 21:41:21 +0100 (CET) Received: from [217.29.46.120] ([217.29.46.120]) by hugo10.ka.punkt.de (8.14.2/8.14.2) with ESMTP id vBRKfLqq075013; Wed, 27 Dec 2017 21:41:21 +0100 (CET) (envelope-from hausen@punkt.de) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: Running dual boot windows inside of bhyve From: "Patrick M. Hausen" In-Reply-To: Date: Wed, 27 Dec 2017 21:41:20 +0100 Cc: Aryeh Friedman , "freebsd-virtualization@freebsd.org" , FreeBSD Mailing List , FreeBSD Mailing List Content-Transfer-Encoding: quoted-printable Message-Id: <921CAF09-DDCD-43CE-BD45-9F59E7052BBA@punkt.de> References: To: Alan Somers X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2017 20:44:19 -0000 Hi all, > Am 27.12.2017 um 18:15 schrieb Alan Somers > I fear that you may be out of luck. Windows deliberately frustrates = this > use case by profiling its hardware at installation time and at every = boot > thereafter. If the hardware changes too much, then Windows demands a = new > license fee. Moving from physical hardware to a VM would probably = trip > every one of its alarms. Correct. But we are talking Windows 7 here. It will lose it's activation = for sure, but as far as my experience goes you can always reactivate Windows = 7 with the same keycode (unless it was a counterfeit one and is = intentionally blocked). As a last resort MS offers a phone dialog system that has always worked = for me. > For this reason, I would recommend using a > separate instance for your VM. However, if you do use a shared = instance, > then make sure that your PC is booting in UEFI mode. BHyve can only = boot > Windows in that mode. If you do that, then BHyve will probably be = able to > boot it just fine. This is actually the bigger of the two obstacles. If the original = installation is MBR booting, I don't know of a way to convert it to EFI. If you happen to have some genuine not "OEM" Windows 7 install medium *and* your original key, you should be able to do a fresh installation = and complete activation - via phone if necessary. HTH, Patrick --=20 punkt.de GmbH Internet - Dienstleistungen - Beratung Kaiserallee 13a Tel.: 0721 9109-0 Fax: -100 76133 Karlsruhe info@punkt.de http://punkt.de AG Mannheim 108285 Gf: Juergen Egeling From owner-freebsd-virtualization@freebsd.org Wed Dec 27 22:31:15 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D8D74E8206E for ; Wed, 27 Dec 2017 22:31:15 +0000 (UTC) (envelope-from cy.schubert@komquats.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id BA35E7B5E3 for ; Wed, 27 Dec 2017 22:31:15 +0000 (UTC) (envelope-from cy.schubert@komquats.com) Received: by mailman.ysv.freebsd.org (Postfix) id B96DEE8206A; Wed, 27 Dec 2017 22:31:15 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B54C3E82069; Wed, 27 Dec 2017 22:31:15 +0000 (UTC) (envelope-from cy.schubert@komquats.com) Received: from smtp-out-so.shaw.ca (smtp-out-so.shaw.ca [64.59.136.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 698DF7B5E1; Wed, 27 Dec 2017 22:31:14 +0000 (UTC) (envelope-from cy.schubert@komquats.com) Received: from spqr.komquats.com ([96.50.22.10]) by shaw.ca with ESMTPA id UKEAemyLhb3YIUKEBeHT9Y; Wed, 27 Dec 2017 15:31:12 -0700 X-Authority-Analysis: v=2.2 cv=J/va1EvS c=1 sm=1 tr=0 a=jvE2nwUzI0ECrNeyr98KWA==:117 a=jvE2nwUzI0ECrNeyr98KWA==:17 a=ocR9PWop10UA:10 a=YxBL1-UpAAAA:8 a=6I5d2MoRAAAA:8 a=gG4suzkTnGaVT7dcZ6YA:9 a=Lep1XYZ0wGAbpZAW:21 a=q8-Z87JhJnZApf8S:21 a=CjuIK1q_8ugA:10 a=PLpP4ctxackZs43qJlkA:9 a=FjOSnLTqnpTALeEM:21 a=OqxMexbwkKxuhw4z:21 a=a-Dni1MZVWkbyNyE:21 a=_W_S_7VecoQA:10 a=Ia-lj3WSrqcvXOmTRaiG:22 a=IjZwj45LgO3ly-622nXo:22 Received: from [25.81.38.66] (unknown [24.114.38.70]) by spqr.komquats.com (Postfix) with ESMTPSA id 810548A5; Wed, 27 Dec 2017 14:31:09 -0800 (PST) MIME-Version: 1.0 From: Cy Schubert Subject: RE: Running dual boot windows inside of bhyve Date: Wed, 27 Dec 2017 14:31:20 -0800 To: Rebecca Cran , Alan Somers , Aryeh Friedman CC: "freebsd-virtualization@freebsd.org" , FreeBSD Mailing List , FreeBSD Mailing List Message-Id: <20171227223109.810548A5@spqr.komquats.com> X-CMAE-Envelope: MS4wfEYx265y4VVp/44qoZlDlXaMKzXY3PIaH3fPGmPunZoacvX7qlG3D7n2VDBj203/xA5Wq6htcOnzjyzLGppXz+k+heTAtyFUSyQkiaBeQxb9tFDBIoN6 QEBIw6jgkdBMlOBTii+L8b365E11NFEbxytCg3YUqPJsH8YfKh9ZOJ8dNF8QiG3kGsQ7i05HjoEDLKP+g6Dyall30IkN209TWBSL6//xCYj0nM+BBiY1GYZ7 3O2y6VD+ChELUYnYPLU6bKbGNLi/oQTkbprrjutUdTaqp6rt9DguGhXUPjeeo79NuKpNRgCnLnACSeZ8VEWd1A9qHh+7z46sZK/qYjr3wLdLhgy89JdVyOf7 A5tHq3GzcDaSsMjzlMbCEqW8cPmL1w== Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Dec 2017 22:31:15 -0000 I've had no issues when I p2v Windows partitions. In the case of my laptop,= which has 3 FreeBSD partitions and a Windows partition, I've had no licens= ing issues interchangeably booting it under VirtualBox under FreeBSD or on = the bare metal. --- Sent using a tiny phone keyboard. Apologies for any typos and autocorrect. Also, this old phone only supports top post. Apologies. Cy Schubert or The need of the many outweighs the greed of the few. --- -----Original Message----- From: Rebecca Cran Sent: 27/12/2017 09:37 To: Alan Somers; Aryeh Friedman Cc: freebsd-virtualization@freebsd.org; FreeBSD Mailing List; FreeBSD Maili= ng List Subject: Re: Running dual boot windows inside of bhyve On 12/27/2017 10:15 AM, Alan Somers wrote: > > I fear that you may be out of luck. Windows deliberately frustrates this > use case by profiling its hardware at installation time and at every boot > thereafter. If the hardware changes too much, then Windows demands a new > license fee. Moving from physical hardware to a VM would probably trip > every one of its alarms. Fortunately modern versions at least allow you to move the installation=20 to a different machine without requiring a reinstall. You might be told=20 you need to relicense it, and it will take some time to reconfigure=20 itself to the new hardware it finds itself running on, but it'll work at=20 least. --=20 Rebecca _______________________________________________ freebsd-hackers@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" From owner-freebsd-virtualization@freebsd.org Thu Dec 28 01:16:55 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 22B88E8BB14 for ; Thu, 28 Dec 2017 01:16:55 +0000 (UTC) (envelope-from stryqx@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id F1F291659 for ; Thu, 28 Dec 2017 01:16:54 +0000 (UTC) (envelope-from stryqx@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id F102FE8BB11; Thu, 28 Dec 2017 01:16:54 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F061EE8BB10; Thu, 28 Dec 2017 01:16:54 +0000 (UTC) (envelope-from stryqx@gmail.com) Received: from mail-qk0-x229.google.com (mail-qk0-x229.google.com [IPv6:2607:f8b0:400d:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B22F41658; Thu, 28 Dec 2017 01:16:54 +0000 (UTC) (envelope-from stryqx@gmail.com) Received: by mail-qk0-x229.google.com with SMTP id j137so30425300qke.10; Wed, 27 Dec 2017 17:16:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=UZJpyVocOmJX/Vhc47NbM1ZiCN3/gpNz6/INICN1fVY=; b=oaD24BNhtnSNnH/3i3N5XZVBLPXV9GryPL4fBbl4E2mNxP5zQ+QgZYXwtL+LGt7r7K jyauowumqY5+5zN/MO8iwBcXKMYpJvZ/OU5BfbJjpTb9prrvuYYkzMndmsNT8QsSILPc mQDJTh9uDd+/xupK/H7yru/GX+47lubpjtiGoSlO/lwDIgkVhfuS3AOEzbxpJpDgsPWk C+XtOrmJkqD9YMt7Qrv0LE4Ie+5bKH6HKbR2rVkcEegg/N3T5AR7Q7ucp2BWxV0+q7Sh lCzh+pINQNZTtXoTPV11tRAZO4pxBCeIWba0EsB50dhhVcl8vuezVzK+VvnKIPvzC+Eg hY0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=UZJpyVocOmJX/Vhc47NbM1ZiCN3/gpNz6/INICN1fVY=; b=GKknDnHDCJ8IwvPUsSpOZz7+PcAAe6Cvp0LAAkC+oc3xZO4czCgpXbok+vVEgMCHn5 tmAPo/8ZQkqvBYyMwujKtAGiMKqnNIJkNMsna+NGH3qtRExZVr2/RwKFvaA7ndfc4XFG FdBgOpOdFCSc8Glrgp1NACwx5OJdDp1+/j6bCdkiezrd0ikIWyNvShR9dVtrGVive3EZ h8k4KrCqx4LZtIztt5tKWCs3nxPerWDEqOyn2PVIeW0P/fLEK5/XUVrxxAaBMNptyr4F WYVvqlq5d6SMBc7FwzU+F3xWHKyPTfD2q5iMzSCFJViiJa3ZXixcJotf80b92bWKWNVi GL/A== X-Gm-Message-State: AKGB3mJIWKDlaE/JT7UUyuuJqtaNtFRiJTcXblFF70Pua9MHpUO1HfK+ TuPWQ4YBYBHvri+idIhsVzTmqDuo0fhN8oz6eB5QlA== X-Google-Smtp-Source: ACJfBov/NEALlfOVWrhU83+0iLg4eBhXJ0NavKyClMIVyDE3QJ0MqwRC/KV2GAt5YwQOLr53JcWf1N8RdT0NJQen5pg= X-Received: by 10.55.143.131 with SMTP id r125mr11203737qkd.215.1514423813574; Wed, 27 Dec 2017 17:16:53 -0800 (PST) MIME-Version: 1.0 Received: by 10.12.173.136 with HTTP; Wed, 27 Dec 2017 17:16:13 -0800 (PST) In-Reply-To: <20171227223109.810548A5@spqr.komquats.com> References: <20171227223109.810548A5@spqr.komquats.com> From: Chris Knight Date: Thu, 28 Dec 2017 12:16:13 +1100 Message-ID: Subject: Re: Running dual boot windows inside of bhyve To: Cy Schubert Cc: Rebecca Cran , Alan Somers , Aryeh Friedman , "freebsd-virtualization@freebsd.org" , FreeBSD Mailing List , FreeBSD Mailing List Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Dec 2017 01:16:55 -0000 Howdy, On 28 December 2017 at 09:31, Cy Schubert wrote: > I've had no issues when I p2v Windows partitions. In the case of my laptop, which has 3 FreeBSD partitions and a Windows partition, I've had no licensing issues interchangeably booting it under VirtualBox under FreeBSD or on the bare metal. > Key activation != licensing. Just because you can move it to another machine or P2V it without needing to re-activate it doesn't mean you're licensed to use it either in a VM or on a new machine. Microsoft licensing can be quite opaque, so if you're doing this in a corporate environment make sure you get the appropriate advise to ensure you're properly licensed. From owner-freebsd-virtualization@freebsd.org Fri Dec 29 22:28:03 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3407DEB3A62 for ; Fri, 29 Dec 2017 22:28:03 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lf0-f47.google.com (mail-lf0-f47.google.com [209.85.215.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BAD567E8BC for ; Fri, 29 Dec 2017 22:28:02 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lf0-f47.google.com with SMTP id c19so29359907lfg.3 for ; Fri, 29 Dec 2017 14:28:02 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version:content-language:content-transfer-encoding; bh=Cq0ziDJdYuLxyDrmhy0ZUxsSlT772RQ9CvtekIrJdlU=; b=nQh+gMuf7yXUerDYoaxZQ08iQZPgGl6gH3Ch298eC+WKie/dPE0S9pwQ8/zuqCoz9Q u+w6XHH6w2mkojaHQCCSBTrbRiRNXlzJ7MRXBeRHZVuw5uK3PJ4fQq1sZbYYrNcEqycZ AZNs7SQl67t2l6Ffe4itKXDNPk7IpuqubsZh6lxLV0q+tvnhkyTuncxTZaq1HFHK+gI5 klf6AJiTBtMEgyx966oTWEcuIMhdALKo/KwYEzA5YNpF/i/gMvMGoTLOnPHUBK5imA+O BZHhYIRPUBSNvykbs9LEK9YKUKod2q82oFcaeEH+FJuVEzO1N6s/1OlCQJnUoTG2rRnN ucrQ== X-Gm-Message-State: AKGB3mLtO7ALvKpicJj8g3hfEO1Z/wnqNYhCPqZI7z+BUNL+4uVN8G4A NP5o30Vw7qqgD3wr5nPihVs= X-Google-Smtp-Source: ACJfBosnTy0MmUUr0sx8ND4IHMGSro6zrFMiCI5vXwM8VTQJSXWEQQu6L8RBpHGlyxbCz4+4dkzU/Q== X-Received: by 10.46.16.138 with SMTP id 10mr24320983ljq.115.1514586144280; Fri, 29 Dec 2017 14:22:24 -0800 (PST) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id b75sm8053110lfe.47.2017.12.29.14.22.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 29 Dec 2017 14:22:23 -0800 (PST) To: freebsd-virtualization@freebsd.org From: Andriy Gapon Subject: bhyve/amd: interrupt delivered when it shouldn't be? Message-ID: <42c22179-ae42-e4bb-e77d-a1d49fe634ed@FreeBSD.org> Date: Sat, 30 Dec 2017 00:22:21 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Dec 2017 22:28:03 -0000 First, about the setup. It's a FreeBSD/amd64 head guest on a FreeBSD/amd64 head host. The hardware is AMD. The hypervisor is bhyve. Under a certain specific load, that involves a lot of page faults and IPI-s, I see the guest system getting stuck. This is pretty consistent. Typically I find a thread spinning on smp_ipi_mtx. And an owner of the mutex appears to be in mi_switch() -> sched_switch(). The debugging data that I have is somewhat flaky, but it seems that the owner is typically in this code path: smp_targeted_tlb_shootdown -> ipi_send_cpu -> native_lapic_ipi_raw smp_targeted_tlb_shootdown holds smp_ipi_mtx. native_lapic_ipi_raw, in this setup, performs the following manipulations: saveintr = intr_disable(); ... intr_restore(saveintr); The interrupts are already disabled when this function is entered, because smp_ipi_mtx is a spinlock and our spinlock implementation disables interrupts. So, intr_restore() in this case should be a NOP (BTW, it's implemented via popf). But what I see suggests that at this point a Local APIC timer interrupt gets delivered to the thread. And that causes all the mess as the thread holding the spinlock gets preempted. Does this ring a bell to anyone? Is there any suspect code? It seems that we set v_intr_masking bit, so the rFLAGS / eFLAGS should be completely virtualized. So, maybe a hardware issue? Thank you! -- Andriy Gapon From owner-freebsd-virtualization@freebsd.org Fri Dec 29 23:51:12 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AF1F5E81377 for ; Fri, 29 Dec 2017 23:51:12 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 64D5B18E1 for ; Fri, 29 Dec 2017 23:51:12 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id BE62520B1AEF for ; Sat, 30 Dec 2017 09:45:54 +1000 (AEST) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id B9D502820A9 for ; Sat, 30 Dec 2017 09:45:54 +1000 (AEST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 8lYHhMt0burU for ; Sat, 30 Dec 2017 09:45:54 +1000 (AEST) Received: from Peters-MacBook-Pro-2.local (96-82-80-65-static.hfc.comcastbusiness.net [96.82.80.65]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 00BF52809DB; Sat, 30 Dec 2017 09:45:52 +1000 (AEST) Subject: Re: bhyve/amd: interrupt delivered when it shouldn't be? To: Andriy Gapon References: <42c22179-ae42-e4bb-e77d-a1d49fe634ed@FreeBSD.org> From: Peter Grehan Cc: freebsd-virtualization@freebsd.org Message-ID: <450137ba-52dd-8b4c-63d2-3c3ce1909d69@freebsd.org> Date: Fri, 29 Dec 2017 15:45:49 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 In-Reply-To: <42c22179-ae42-e4bb-e77d-a1d49fe634ed@FreeBSD.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-CMAE-Score: 0 X-CMAE-Analysis: v=2.3 cv=KPZ08mNo c=1 sm=1 tr=0 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=mwgbnDbW7alINpy3vhoKyg==:17 a=IkcTkHD0fZMA:10 a=ocR9PWop10UA:10 a=gzpREF0ZZXXHYDOYZD4A:9 a=QEXdDO2ut3YA:10 wl=host:3 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Dec 2017 23:51:12 -0000 Hi Andriy, > The hardware is AMD. Ryzen ? > But what I see suggests that at this point a Local APIC timer interrupt gets > delivered to the thread. And that causes all the mess as the thread holding the > spinlock gets preempted. > > Does this ring a bell to anyone? I have seen something similar to this after about ~20 mins when doing a current -j 16 buildworld in a guest, with the symptom being a spinlock timeout, with one vCPU spinning in smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0x352/frame 0xfffffe02c80098d0 smp_masked_invlpg() at smp_masked_invlpg+0x4c/frame 0xfffffe02c8009900 pmap_invalidate_page() at pmap_invalidate_page+0x191/frame 0xfffffe02c8009950 pmap_ts_referenced() at pmap_ts_referenced+0x7b3/frame 0xfffffe02c8009a00 vm_pageout() at vm_pageout+0xe04/frame 0xfffffe02c8009a70 ... and all the others eventually spinning on that held lock. However, NMIs are still able to get through (the post-panic ddb NMI IPI) so the VM isn't completely locked up - either an interrupt is missed, or a write isn't seen by the vCPU issuing the tlb shootdown. > Is there any suspect code? Not sure yet, but the interrupt-injection path could do with a close inspection. > It seems that we set v_intr_masking bit, so the rFLAGS / eFLAGS should be > completely virtualized. So, maybe a hardware issue? Hard to say. Running with all vCPUs pinned makes the problem go away, but that could just mean the issue is isolated to when vCPUs migrate. later, Peter. From owner-freebsd-virtualization@freebsd.org Sat Dec 30 06:11:38 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A8B5CEAC04E for ; Sat, 30 Dec 2017 06:11:38 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-qk0-x230.google.com (mail-qk0-x230.google.com [IPv6:2607:f8b0:400d:c09::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5BC866FE73 for ; Sat, 30 Dec 2017 06:11:38 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: by mail-qk0-x230.google.com with SMTP id v188so27397209qkh.11 for ; Fri, 29 Dec 2017 22:11:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd-org.20150623.gappssmtp.com; s=20150623; h=date:from:to:subject:message-id:mime-version:content-disposition :user-agent; bh=hB5m2gAwRmW1ixB7R9lgmIHq0lQzcOc/OmldW5o5jXw=; b=Xq3yOWYcMcU4ueVGOCBDXhTAsZ1Zge2pLtkF9wqA7r7QudG2ofqFuxN/e6Abe5m/7M dVnW0kc+BzI7n1W3iVNDKKC+29RQwCUlTRdO+k6XM8FslJZHzhdJLZ2AOm26Gag0LaEr ZGPQawo9E4cSeDNvL326dVDvXgixfKcs2iT2LmFF91+M7N2QfbyrKozqm/UgNf7Kq1V5 cBhcZv1w7MTnG+J0mdUBPuxnbE6ZNSOuH1pnsFiYpk6QSjCsNqkZVUmtREs6qng1ZiE9 O685kbZP9iiE7m5QcnaJGmyOplJc9Y3E+X00kv6uFsvHiNyBl9jG4B9xIahegWEtXKVu CWJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:subject:message-id:mime-version :content-disposition:user-agent; bh=hB5m2gAwRmW1ixB7R9lgmIHq0lQzcOc/OmldW5o5jXw=; b=CaqHs1+UXcxdY8BhypEov+etCS9kss6QiHjWoUHv+AZjD0LJdxkRAhKDuMWkFjeArh M4s1aPtqjyuxxMcUpwMCnrN27pPjWQW2eYzJC9nZRl6r1GfrYsyN3njWZKM2Z4b45FH+ Z5n6gA1JWG1klq/CwLYGR7grOZKTe/117LDcpeubBCtNuBTP7aE3NHn5jcQ1tu20higC 9NFfCVnS8riW2HJihHBSg6OJdrjnWlKRGPQNYl99BoqIVldG36ODHSoMjUD8gtqEj+tZ KSHktzdRYCEbYsbkCi9o0Q/j9Bf1J//CWatSi28QR/jCne2uITGqvVvjIsgD6BEj9fsa eXfQ== X-Gm-Message-State: AKGB3mKS7SbjyukqVveDCUBG+gGzNMVmY1gEnN2TWbrmzIF9UNv7brII wrZAF2Qs03mMpzpvzAHBtZlrOtDpVwVVsILH4QTryvkpbFvDDPKFF3pvfMd+1Har1XOBDg3Z+1z E3E8uVmteWYNIXuCR1bV635Qq/CER9zjsjzRRK0ya4wnuh+TscWX9y8TQwmtkS9am/WfIeI2Uly Fhn9x/CVsoSPBHI2Mc X-Google-Smtp-Source: ACJfBotnREgOfPlha+z+8Ixroqk1vxL5CRJ7lO2fZXVvaAhYf+FKWkNyECLGylE+eTxz2fkasF8cuA== X-Received: by 10.55.221.20 with SMTP id n20mr49207796qki.181.1514614296861; Fri, 29 Dec 2017 22:11:36 -0800 (PST) Received: from mutt-hbsd (xanaduregio.emeraldonion.org. [23.129.64.102]) by smtp.gmail.com with ESMTPSA id f34sm20024411qtb.63.2017.12.29.22.11.31 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 29 Dec 2017 22:11:36 -0800 (PST) Date: Sat, 30 Dec 2017 01:10:53 -0500 From: Shawn Webb To: freebsd-virtualization@freebsd.org Subject: Unable to use renamed tap device Message-ID: <20171230061053.ui4wc4yqw7szsbuw@mutt-hbsd> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="tpupydl7rq2r67qv" Content-Disposition: inline X-Operating-System: FreeBSD mutt-hbsd 12.0-CURRENT FreeBSD 12.0-CURRENT X-PGP-Key: http://pgp.mit.edu/pks/lookup?op=vindex&search=0x6A84658F52456EEE User-Agent: NeoMutt/20171208 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Dec 2017 06:11:38 -0000 --tpupydl7rq2r67qv Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hey All, I'm in the process of reorganizing my bhyve setup on my development laptop. I'd like to have rename the tap devices to match the name of the VM so that it's easier to keep track of. Otherwise, I have to keep a spreadsheet of (tap3 -> win10-vm, tap4 -> fbsd-vm). It appears bhyve doesn't attach renamed tap devices. Here's the steps I used: ifconfig bridge0 create ifconfig tap0 create name fbsd-01 ifconfig bridge0 addm em0 addm fbsd-01 up sh /usr/share/examples/bhyve/vmrun.sh -t fbsd-01 [normal vmrun.sh arguments= here] (In this example, em0 is the physical network device connected to the LAN. I want to share em0 with the host and the guest via the bridge.) The net.link.tap.up_on_open sysctl node is set to 1. Normally, when bhyve starts up (with tap0 instead of fbsd-01), it opens the tap device and UPs it. I'm not seeing that same behavior with a renamed tap interface: $ ifconfig ld-03_01 ld-03_01: flags=3D8903 metric 0 mtu= 1500 options=3D80000 ether 00:bd:df:e9:f6:04 groups: tap media: Ethernet autoselect status: no carrier nd6 options=3D29 So, it seems to me that bhyve doesn't like it when tap devices are renamed. Can anyone shed some light on this? Thanks, --=20 Shawn Webb Cofounder and Security Engineer HardenedBSD Tor-ified Signal: +1 443-546-8752 GPG Key ID: 0x6A84658F52456EEE GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE --tpupydl7rq2r67qv Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEKrq2ve9q9Ia+iT2eaoRlj1JFbu4FAlpHLesACgkQaoRlj1JF bu4VrRAArGaSuhDPhy6J21kHF7cPCJvFQxXvIqGzA1o/GaPx56EI6KpRc4Urm85Q uhpzWWV/c6SmUSgqGwIFmQI3VNlYfkn3yELKVzLT9ohxdsm/CfctFWhGxfzl4166 am0XjNj7f6wYPhqrY7i7KXzWy4BzAJiFuppDurUWIU4J8OslPr6jHLBiUo7I1lUO JVR1x6z68wgETK/3TOJhi3RExPuozCU0Sb//bdZmv5lRiqF+Tk6L3G3Gg57R2Sen y3z4V/pYJGeSFLGNgROXQv7H8p/7IW2p/y+mz2msVT1ZF/JJ+/Bm/+ScQOWDRSPs esiIvHuMDPqGb93yS2t2QX/ZERqfOq3QAtIdJzRd1dHUuVjM3qwYwngkgu9bU1E2 kdMT/yaR9HaTDNU58ueZDQcipOMu1QSzzAfhgzmadFXbsQySQnT1jrMfV5jfeTwY T9BPnine6W1f1NKvl3L2wrChzdyF/4cNdluT7y2si6p1TIeR1HLwTxMkC5ZSgPK0 1i3igLEEX0aq2tR9TkVo0rPWvGqDAEGVCJmwp5Hwu8+7vIvtHpi4/4+vLkahbA4c vJ4S45mMqEPBnQcY26QZ1NH9li/YUqDyHksQ2H22y8LFpeLtPLi0NfyDThIYrsaM k2RmUdp8iOG1e79S/Iqu4O1bsJI2kj82NMod9E5YnaOJl4NNIxY= =BB1k -----END PGP SIGNATURE----- --tpupydl7rq2r67qv-- From owner-freebsd-virtualization@freebsd.org Sat Dec 30 08:11:38 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 722CAEB0A30 for ; Sat, 30 Dec 2017 08:11:38 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lf0-f47.google.com (mail-lf0-f47.google.com [209.85.215.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1919573D10; Sat, 30 Dec 2017 08:11:37 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lf0-f47.google.com with SMTP id h137so5731472lfe.8; Sat, 30 Dec 2017 00:11:37 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=xPHP5YtbnOs7rK8vHdYcCxg6EV9SKd30vMoOfRdl3FI=; b=b2aAaVLAwzRmjpV4Zvxaa9bPvPuf0bCX7PihA8tu8UnW4ALFndocVar7c2xyoKjIo8 tchZ8g2JcpuPUUSy94auKtkJQ/5iKjAGuwnMuEceqKZ6ckjrbo6yWPfebUEDFQ3GTBQl 3j+6dww66AjSRBCfAwwGHCThWT3nz8H29bl882Rq7GQn0zXgN85jrNgOFi992gxgKfzf Cf1RIMNzY+j02ktt6ZSxPaVC80o7yB1KZIaCcj0S1A2yOVyy6D48LGIR7+IreOlPJBbI 1fXuXEmvBiKYTjxZoXhiqM7TtrTBrK50v8L2FS/OLXXHeUrmPXlIbthWbbs9OidrlPDD j/Rg== X-Gm-Message-State: AKGB3mKTFE3tvnJc3Lccjb/bme0dnwTyM/CRzTEgP1Qrr7curkc9e1Hz UbdsP5TPLACNKbR70obZNlICD+/S X-Google-Smtp-Source: ACJfBouL+svxyhV/sxG37fak1jdcnrSjF9wg+6jxvuJwzBX8Xbut21rbmJNXGvomk4/B8Z5Fk7O5NQ== X-Received: by 10.46.88.4 with SMTP id m4mr21328935ljb.15.1514621495108; Sat, 30 Dec 2017 00:11:35 -0800 (PST) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id 1sm7532098ljt.35.2017.12.30.00.11.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 30 Dec 2017 00:11:33 -0800 (PST) Subject: Re: bhyve/amd: interrupt delivered when it shouldn't be? To: Peter Grehan Cc: freebsd-virtualization@freebsd.org References: <42c22179-ae42-e4bb-e77d-a1d49fe634ed@FreeBSD.org> <450137ba-52dd-8b4c-63d2-3c3ce1909d69@freebsd.org> From: Andriy Gapon Message-ID: Date: Sat, 30 Dec 2017 10:11:32 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.5.2 MIME-Version: 1.0 In-Reply-To: <450137ba-52dd-8b4c-63d2-3c3ce1909d69@freebsd.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Dec 2017 08:11:38 -0000 On 30/12/2017 01:45, Peter Grehan wrote: > Hi Andriy, > >> The hardware is AMD. > >  Ryzen ? No, a pretty old family 10h. >> But what I see suggests that at this point a Local APIC timer interrupt gets >> delivered to the thread.  And that causes all the mess as the thread holding the >> spinlock gets preempted. >> >> Does this ring a bell to anyone? > >  I have seen something similar to this after about ~20 mins when doing a current > -j 16 buildworld in a guest, with the symptom being a spinlock timeout, with one > vCPU spinning in > > smp_targeted_tlb_shootdown() at smp_targeted_tlb_shootdown+0x352/frame > 0xfffffe02c80098d0 > smp_masked_invlpg() at smp_masked_invlpg+0x4c/frame 0xfffffe02c8009900 > pmap_invalidate_page() at pmap_invalidate_page+0x191/frame 0xfffffe02c8009950 > pmap_ts_referenced() at pmap_ts_referenced+0x7b3/frame 0xfffffe02c8009a00 > vm_pageout() at vm_pageout+0xe04/frame 0xfffffe02c8009a70 > > ... and all the others eventually spinning on that held lock. This looks very similar to what I see as well. > However, NMIs are still able to get through (the post-panic ddb NMI IPI) so the > VM isn't completely locked up - either an interrupt is missed, or a write isn't > seen by the vCPU issuing the tlb shootdown. Yes, I can also enter ddb and remote kgdb. >> Is there any suspect code? > >  Not sure yet, but the interrupt-injection path could do with a close inspection. > >> It seems that we set v_intr_masking bit, so the rFLAGS / eFLAGS should be >> completely virtualized.  So, maybe a hardware issue? > >  Hard to say. Running with all vCPUs pinned makes the problem go away, but that > could just mean the issue is isolated to when vCPUs migrate. I was going to try pinned CPUs and wired memory next, so your observation narrows the problem scope. Thank you! -- Andriy Gapon From owner-freebsd-virtualization@freebsd.org Sat Dec 30 16:08:00 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1C032E9F93B for ; Sat, 30 Dec 2017 16:08:00 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B10481979 for ; Sat, 30 Dec 2017 16:07:59 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (mh0.gentlemail.de [IPv6:2a00:e10:2800::a135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id vBUG7ufk090172; Sat, 30 Dec 2017 17:07:56 +0100 (CET) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 77D6C438; Sat, 30 Dec 2017 17:07:56 +0100 (CET) Message-ID: <5A47B9DC.4070503@omnilan.de> Date: Sat, 30 Dec 2017 17:07:56 +0100 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Shawn Webb CC: freebsd-virtualization@freebsd.org Subject: Re: Unable to use renamed tap device References: <20171230061053.ui4wc4yqw7szsbuw@mutt-hbsd> In-Reply-To: <20171230061053.ui4wc4yqw7szsbuw@mutt-hbsd> Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]); Sat, 30 Dec 2017 17:07:56 +0100 (CET) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 30 Dec 2017 16:08:00 -0000 Bezüglich Shawn Webb's Nachricht vom 30.12.2017 07:10 (localtime): > Hey All, > > I'm in the process of reorganizing my bhyve setup on my development > laptop. I'd like to have rename the tap devices to match the name of > the VM so that it's easier to keep track of. Otherwise, I have to keep > a spreadsheet of (tap3 -> win10-vm, tap4 -> fbsd-vm). > > It appears bhyve doesn't attach renamed tap devices. Here's the steps > I used: > > ifconfig bridge0 create > ifconfig tap0 create name fbsd-01 > ifconfig bridge0 addm em0 addm fbsd-01 up > sh /usr/share/examples/bhyve/vmrun.sh -t fbsd-01 [normal vmrun.sh arguments here] > > (In this example, em0 is the physical network device connected to the > LAN. I want to share em0 with the host and the guest via the bridge.) > > The net.link.tap.up_on_open sysctl node is set to 1. Normally, when > bhyve starts up (with tap0 instead of fbsd-01), it opens the tap > device and UPs it. I'm not seeing that same behavior with a renamed > tap interface: > > $ ifconfig ld-03_01 > ld-03_01: flags=8903 metric 0 mtu 1500 > options=80000 > ether 00:bd:df:e9:f6:04 > groups: tap > media: Ethernet autoselect > status: no carrier > nd6 options=29 > > So, it seems to me that bhyve doesn't like it when tap devices are > renamed. Can anyone shed some light on this? Unfortunately not too much, besides this report: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=219746 As long as the control device name isn't also renamed, tap/vmnet renaming should be blocked in rc(8), since it can't work the way it is at the moment. Unfortunately my C skills don't allow me to check if renaming the control device could/should be implemented in whatever functions are responsible for IF renaming (meaning to make it a kernel task). Or if it would be better to utilize devd(8)!? -harry