From owner-freebsd-virtualization@freebsd.org Sun Mar 17 16:22:35 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id A7067154127E
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Sun, 17 Mar 2019 16:22:35 +0000 (UTC)
(envelope-from crowston@protonmail.com)
Received: from mail1.protonmail.ch (mail1.protonmail.ch [185.70.40.18])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "*.protonmail.ch",
Issuer "SwissSign Server Silver CA 2014 - G22" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id E708C86516
for <freebsd-virtualization@freebsd.org>; Sun, 17 Mar 2019 16:22:34 +0000 (UTC)
(envelope-from crowston@protonmail.com)
Date: Sun, 17 Mar 2019 16:22:29 +0000
To: "freebsd-virtualization@freebsd.org" <freebsd-virtualization@freebsd.org>
From: Robert Crowston <crowston@protonmail.com>
Reply-To: Robert Crowston <crowston@protonmail.com>
Subject: GPU passthrough: mixed success on Linux, not yet on Windows
Message-ID: <H0Gbov17YtZC1-Ao1YkjZ-nuOqPv4LPggc_mni3cS8WWOjlSLBAfOGGPf4aZEpOBiC5PAUGg6fkgeutcLrdbmXNO5QfaxFtK_ANn-Nrklws=@protonmail.com>
Feedback-ID: 2OVbcR1yHYpdkD8cgQllkFwcuMVZg_LiVMMPvptooFDfHD_03MuQO4ZaF626jWHZYFEhNR2cmIbZ53j4QGWMBQ==:Ext:ProtonMail
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Status: No, score=-1.2 required=7.0 tests=ALL_TRUSTED,DKIM_SIGNED,
DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM autolearn=ham
autolearn_force=no version=3.4.2
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mail.protonmail.ch
X-Rspamd-Queue-Id: E708C86516
X-Spamd-Bar: -------
X-Spamd-Result: default: False [-7.76 / 15.00];
HAS_REPLYTO(0.00)[crowston@protonmail.com];
R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24];
FREEMAIL_FROM(0.00)[protonmail.com];
DKIM_TRACE(0.00)[protonmail.com:+];
DMARC_POLICY_ALLOW(-0.50)[protonmail.com,quarantine];
MX_GOOD(-0.01)[cached: mailsec.protonmail.ch];
NEURAL_HAM_SHORT(-0.98)[-0.981,0]; RCVD_COUNT_ZERO(0.00)[0];
FROM_EQ_ENVFROM(0.00)[];
IP_SCORE(-3.67)[ip: (-9.48), ipnet: 185.70.40.0/24(-4.89), asn: 19905(-3.92),
country: US(-0.07)]; MIME_TRACE(0.00)[0:+];
FREEMAIL_ENVFROM(0.00)[protonmail.com];
ASN(0.00)[asn:19905, ipnet:185.70.40.0/24, country:US];
MID_RHS_MATCH_FROM(0.00)[];
RCVD_IN_DNSWL_LOW(-0.10)[18.40.70.185.list.dnswl.org : 127.0.5.1];
ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
R_DKIM_ALLOW(-0.20)[protonmail.com:s=default];
REPLYTO_EQ_FROM(0.00)[]; FROM_HAS_DN(0.00)[];
TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0];
MIME_GOOD(-0.10)[text/plain];
FREEMAIL_REPLYTO(0.00)[protonmail.com];
RCPT_COUNT_ONE(0.00)[1]; TO_DN_EQ_ADDR_ALL(0.00)[];
RCVD_TLS_ALL(0.00)[]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 17 Mar 2019 16:22:35 -0000
Hi folks, this is my first post to the group. Apologies for length.
I've been experimenting with GPU passthrough on bhyve. For background, the =
host system is FreeBSD 12.0-RELEASE on an AMD Ryzen 1700 CPU @ 3.8 GHz, 32 =
GB of ECC RAM, with two nVidia GPUs. I'm working with a Linux Debian 9 gues=
t and a Windows Server 2019 (desktop experience installed) guest. I also ha=
ve a USB controller passed-through for bluetooth and keyboard.
With some unpleasant hacks I have succeeded in starting X on the Linux gues=
t, passing-through an nVidia GT 710 under the nouveau driver. I can run the=
"mate" desktop and glxgears, both of which are smooth at 4K. The Unity Hea=
ven benchmark runs at an embarrassing 0.1 fps, and 2160p x264 video in VLC =
runs at about 5 fps. Neither appears to be CPU-bound in the host or the gue=
st.
The hack I had to make: I found that many instructions to access memory-map=
ped PCI BARs are not being executed on the CPU in guest mode but are being =
passed back for emulation in the hypervisor. This causes an assertion to fa=
il inside passthru_write() in pci_passthru.c ["pi->pi_bar[baridx].type =3D=
=3D PCIBAR_IO"] because it does not expect to perform memory-mapped IO for =
the guest. Examining the to-be-emulated instructions in vmexit_inst_emul() =
{e.g., movl (%rdi), %eax}, they look benign to me, and I have no explanatio=
n for why the CPU refused to execute them in guest mode.
As an amateur work-around, I removed the assertion and instead I obtain the=
desired offset into the guest's BAR, calculate what that guest address tra=
nslates to in the host's address space, open(2) /dev/mem, mmap(2) over to t=
hat address, and perform the write directly. I do a similar trick in passth=
ru_read(). Ugly, slow, but functional.
This code path is accessed continuously whether or not X is running, with a=
n increase in activity when running anything GPU-heavy. Always to bar 1, an=
d mostly around the same offsets. I added some logging of this event. It ru=
ns at about 100 lines per second while playing video. An excerpt is:
...
Unexpected out-of-vm passthrough write #492036 to bar 1 at offset 41100.
Unexpected out-of-vm passthrough write #492037 to bar 1 at offset 41100.
Unexpected out-of-vm passthrough read #276162 to bar 1 at offset 561280.
Unexpected out-of-vm passthrough write #492038 to bar 1 at offset 38028.
Unexpected out-of-vm passthrough write #492039 to bar 1 at offset 38028.
Unexpected out-of-vm passthrough read #276163 to bar 1 at offset 561184.
Unexpected out-of-vm passthrough read #276164 to bar 1 at offset 561184.
Unexpected out-of-vm passthrough read #276165 to bar 1 at offset 561184.
Unexpected out-of-vm passthrough read #276166 to bar 1 at offset 561184.
...
So my question here is,
1. How do I diagnose why the instructions are not being executed in guest m=
ode?
Some other problems:
2. Once the virtual machine is shut down, the passed-through GPU doesn't ge=
t turned off. Whatever message was on the screen in the final throes of Lin=
ux's shutdown stays there. Maybe there is a specific detach command which b=
hyve or nouveau hasn't yet implemented? Alternatively, maybe I could exploi=
t some power management feature to reset the card when bhyve exits.
3. It is not possible to reboot the guest and then start X again without an=
intervening host reboot. The text console works fine. Xorg.0.log has a mes=
sage like
(EE) [drm] Failed to open DRM device for pci:0000:00:06.0: -19
(EE) open /dev/dri/card0: No such file or directory
dmesg is not very helpful either.[0] I suspect that this is related to prob=
lem (2).
4. There is a known bug in the version of the Xorg server that ships with D=
ebian 9, where the switch from an animated mouse cursor back to a static cu=
rsor causes the X server to sit in a busy loop of gradually increasing stac=
k depth, if the GPU takes too long to communicate with the driver.[1] For m=
e, this consistently happens after I type my password into the Debian login=
dialog box and eventually (~ 120 minutes) locks up the host by eating all =
the swap. A work-around is to replace the guest's animated cursors with sta=
tic cursors. The bug is fixed in newer versions of X, but I haven't tested =
whether their fix works for me yet.
5. The GPU doesn't come to life until the nouveau driver kicks in. What is =
special about the driver? Why doesn't the UEFI open the GPU and send it out=
put before the boot? Any idea if the problem is on the UEFI side or the hyp=
ervisor side?
6. On Windows, the way Windows probes multi-BAR devices seems to be inconsi=
stent with bhyve's model for storing io memory mappings. Specifically, I be=
lieve Windows assigns the 0xffffffff sentinel to all BARs on a device in on=
e shot, then reads them back and assigns the true addresses afterwards. How=
ever, bhyve sees the multiple 0xffffffff assignments to different BARs as a=
clash and errors out on the second BAR probe. I removed most of the mmio_r=
b_tree error handling in mem.c and this is sufficient for Windows to boot, =
and detect and correctly identify the GPU. (A better solution might be to h=
andle the initial 0xffffffff write as a special case.) I can then install t=
he official nVidia drivers without problem over Remote Desktop. However, th=
e GPU never springs into life: I am stuck with a "Windows has stopped this =
device because it has reported problems. (Code 43)" error in the device man=
ager, a blank screen, and not much else to go on.
Is it worth me continuing to hack away at these problems---of course I'm ha=
ppy to share anything I come up with---or is there an official solution to =
GPU support in the pipe about to make my efforts redundant :)?
Thanks,
Robert Crowston.
---
Footnotes
[0] Diff'ing dmesg after successful GPU initialization (+) and after failu=
re (-), and cutting out some lines that aren't relevant:
nouveau 0000:00:06.0: bios: version 80.28.a6.00.10
+nouveau 0000:00:06.0: priv: HUB0: 085014 ffffffff (1f70820b)
nouveau 0000:00:06.0: fb: 1024 MiB DDR3
@@ -466,24 +467,17 @@
nouveau 0000:00:06.0: DRM: DCB conn 00: 00001031
nouveau 0000:00:06.0: DRM: DCB conn 01: 00002161
nouveau 0000:00:06.0: DRM: DCB conn 02: 00000200
-nouveau 0000:00:06.0: disp: chid 0 mthd 0000 data 00000400 00001000 000000=
02
-nouveau 0000:00:06.0: timeout at /build/linux-UEAD6s/linux-4.9.144/drivers=
/gpu/drm/nouveau/nvkm/engine/disp/dmacgf119.c:88/gf119_disp_dmac_init()!
-nouveau 0000:00:06.0: disp: ch 1 init: c207009b
-nouveau: DRM:00000000:0000927c: init failed with -16
-nouveau 0000:00:06.0: timeout at /build/linux-UEAD6s/linux-4.9.144/drivers=
/gpu/drm/nouveau/nvkm/engine/disp/dmacgf119.c:54/gf119_disp_dmac_fini()!
-nouveau 0000:00:06.0: disp: ch 1 fini: c2071088
-nouveau 0000:00:06.0: timeout at /build/linux-UEAD6s/linux-4.9.144/drivers=
/gpu/drm/nouveau/nvkm/engine/disp/dmacgf119.c:54/gf119_disp_dmac_fini()!
-nouveau 0000:00:06.0: disp: ch 1 fini: c2071088
+[drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
+[drm] Driver supports precise vblank timestamp query.
+nouveau 0000:00:06.0: DRM: MM: using COPY for buffer copies
+nouveau 0000:00:06.0: DRM: allocated 1920x1080 fb: 0x60000, bo ffff96fdb39=
a1800
+fbcon: nouveaufb (fb0) is primary device
-nouveau 0000:00:06.0: timeout at /build/linux-UEAD6s/linux-4.9.144/drivers=
/gpu/drm/nouveau/nvkm/engine/disp/coregf119.c:187/gf119_disp_core_fini()
-nouveau 0000:00:06.0: disp: core fini: 8d0f0088
-[TTM] Finalizing pool allocator
-[TTM] Finalizing DMA pool allocator
-[TTM] Zone kernel: Used memory at exit: 0 kiB
-[TTM] Zone dma32: Used memory at exit: 0 kiB
-nouveau: probe of 0000:00:06.0 failed with error -16
+Console: switching to colour frame buffer device 240x67
+nouveau 0000:00:06.0: fb0: nouveaufb frame buffer device
+[drm] Initialized nouveau 1.3.1 20120801 for 0000:00:06.0 on minor 0
[1] https://devtalk.nvidia.com/default/topic/1028172/linux/titan-v-ubuntu-1=
6-04lts-and-387-34-driver-crashes-badly/post/5230898/#5230898
From owner-freebsd-virtualization@freebsd.org Mon Mar 18 11:17:46 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id D4E1E153A7B3
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Mon, 18 Mar 2019 11:17:46 +0000 (UTC)
(envelope-from misc-freebsd@talk2dom.com)
Received: from server.zx81.pw (server.zx81.pw [IPv6:2001:41c9:1:426::39])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
server-signature RSA-PSS (4096 bits)
client-signature RSA-PSS (2048 bits) client-digest SHA256)
(Client CN "server.zx81.pw", Issuer "Let's Encrypt Authority X3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 026F784134
for <freebsd-virtualization@freebsd.org>; Mon, 18 Mar 2019 11:17:45 +0000 (UTC)
(envelope-from misc-freebsd@talk2dom.com)
Received: from auth (localhost [127.0.0.1]) by server.zx81.pw (8.15.2/8.15.2)
with ESMTPSA id x2IBHgxe048740
(version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO)
for <freebsd-virtualization@freebsd.org>; Mon, 18 Mar 2019 11:17:43 GMT
(envelope-from misc-freebsd@talk2dom.com)
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=talk2dom.com; s=zx81;
t=1552907863; x=1553512663;
bh=IpF0BBcQ7gKnGQGVbNaP+pO5iDi7GT+e+g8TTVg0ayg=;
h=Subject:To:References:From:Date:In-Reply-To;
b=fcHzRIudPBwiKRwXexEWRv/kBlYGkfx9CYucC8OQ8MmlKhlIT3MWrBgtzbSF+oj6r
hy2wZK4Xbedtr6xFmAxCnFZYsEP1nedpKV8ore6EQ23KB0VyKUr4lj2Wp6pKv+xxVx
WvQg7hf2dtmILfj+LaGArdA+s+4WHz96hnVPMn/U=
Subject: Re: GPU passthrough: mixed success on Linux, not yet on Windows
To: freebsd-virtualization@freebsd.org
References: <H0Gbov17YtZC1-Ao1YkjZ-nuOqPv4LPggc_mni3cS8WWOjlSLBAfOGGPf4aZEpOBiC5PAUGg6fkgeutcLrdbmXNO5QfaxFtK_ANn-Nrklws=@protonmail.com>
From: Dom <misc-freebsd@talk2dom.com>
Message-ID: <283e2783-ac4c-6ab2-465d-de370fa2feff@zx81.pw>
Date: Mon, 18 Mar 2019 11:17:42 +0000
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101
Thunderbird/60.5.3
MIME-Version: 1.0
In-Reply-To: <H0Gbov17YtZC1-Ao1YkjZ-nuOqPv4LPggc_mni3cS8WWOjlSLBAfOGGPf4aZEpOBiC5PAUGg6fkgeutcLrdbmXNO5QfaxFtK_ANn-Nrklws=@protonmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Mar 2019 11:17:47 -0000
On 17/03/2019 16:22, Robert Crowston via freebsd-virtualization wrote:
> 6. On Windows, [...] the GPU never springs into life: I am stuck with a "Windows has stopped this device because it has reported problems. (Code 43)" error in the device manager, a blank screen, and not much else to go on.
>
> Is it worth me continuing to hack away at these problems---of course I'm happy to share anything I come up with---or is there an official solution to GPU support in the pipe about to make my efforts redundant :)?
>
> Thanks,
> Robert Crowston.
As far as I know, the nVidia drivers actively try to deny use of
consumer video cards inside a VM guest and error 43 might be a symptom
of this.
Some people have managed to get round this with some non-FreeBSD VM
setups that hide the VM-guest aspect, or change their GPU's reported
hardware/vendor IDs, but I've never managed to recreate this in Bhyve.
e.g. see Question 10 at
http://vfio.blogspot.com/2014/08/vfiovga-faq.html and also the comments
at the bottom of the page.
Dom
From owner-freebsd-virtualization@freebsd.org Mon Mar 18 15:04:22 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id D56E915430AC;
Mon, 18 Mar 2019 15:04:21 +0000 (UTC)
(envelope-from tech-lists@zyxst.net)
Received: from wout1-smtp.messagingengine.com (wout1-smtp.messagingengine.com
[64.147.123.24])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 5FAFE8CAEF;
Mon, 18 Mar 2019 15:04:17 +0000 (UTC)
(envelope-from tech-lists@zyxst.net)
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44])
by mailout.west.internal (Postfix) with ESMTP id 9A95843E4;
Mon, 18 Mar 2019 11:04:08 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
by compute4.internal (MEProxy); Mon, 18 Mar 2019 11:04:08 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zyxst.net; h=
date:from:to:cc:subject:message-id:mime-version:content-type; s=
fm2; bh=ekkaHxWtDaPGNrZzNoISIO8U7su9amFHDMIIDvwGuM8=; b=sQTo9Ofu
rksUUIyY6Re5DcHZp6J6QsB/MMw2mG9vDNNCchUVZnhaO1+XhNE07u2m5HktBQr+
FDDhkctWTm8jouWUkma94f8E1m3gDSndg8aEyGK7mwXPZ8/wFGk/lp3lE0mRaUdK
cw0W7erfRzDnEptcOVGi95E0wHCYPeDZbxrFUOmNjCvXe8EB9/erW+0mKOPGGry7
H2SmYWufN9kxXQQDyLHzQF9mq5C6gzd/BkP8SRgdjl34ju0h5T7sU1Sizv/DQY/u
ld0wZkORZgnAyhmaH+3sLEiX0kAp1BINtcSeORYXRmIvRDrwAMAd7vnw2OU7uiME
8TmTy98nLiaLTw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=cc:content-type:date:from:message-id
:mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender
:x-me-sender:x-sasl-enc; s=fm2; bh=ekkaHxWtDaPGNrZzNoISIO8U7su9a
mFHDMIIDvwGuM8=; b=GSHJjNH48X02sluAk5dp3skglknZjhBGN2aFGvPxW0STH
IGlHsI4CrLX8JVGjPa61QJs8iotXL005b/Ied15ysF1SpeJh1lWxsdxV5WoQ/PzW
yvLj747Jwgfs1y54uuWuHt0SZPBZJ0+sQDQM8kbo1gA5+V0aiKL9QJI6rObLdcOD
mLB0iDncT8L4KyXAaowPa8GRPE+WNB6LgsEheYyS3jI+aEFHf23vWZqKaP4Qx/E4
HwTXHbapHQqV8z5LHsoOvQM9JlsrkausJ1A+ruIpZ20bCSe+yt4hxdzYTbUbksjy
Df/AvhE1GoeUog5V9Tl6uk+2gdWbfwaqPIRm92MwA==
X-ME-Sender: <xms:Z7OPXKgWpKBVrjq3MxMQ-cwNrXCPmz1Y6rmqrKQ0q2Er83nkYF-YNQ>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedutddriedugdejvdcutefuodetggdotefrodftvf
curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfggtggufgesghdtreertd
ervdenucfhrhhomhepthgvtghhqdhlihhsthhsuceothgvtghhqdhlihhsthhsseiihiig
shhtrdhnvghtqeenucfkphepkedvrdejtddrledurddutddunecurfgrrhgrmhepmhgrih
hlfhhrohhmpehtvggthhdqlhhishhtshesiiihgihsthdrnhgvthenucevlhhushhtvghr
ufhiiigvpedt
X-ME-Proxy: <xmx:aLOPXBgLpVb2IrgOj0PeR6WnAvQESEJemWv_gWO7-0bYOLaE09j_Mw>
<xmx:aLOPXFTqWZWNF3beoDaZndvRR56jGjRulpRQ00HiMoZM0w1_dEe1PA>
<xmx:aLOPXNTSoHs91arhG9fUzd4Ve0iYgiLyMX5ZLHl3V-5_xpvjS1_O_A>
<xmx:aLOPXLSXUvCV3w8PYrZGQmht-tMrDJ2a1qBFxeCxwJkD5pbOXz3Eng>
Received: from rpi3.zyxst.net (rpi3.zyxst.net [82.70.91.101])
by mail.messagingengine.com (Postfix) with ESMTPA id 52B9E1030F;
Mon, 18 Mar 2019 11:04:07 -0400 (EDT)
Date: Mon, 18 Mar 2019 15:04:05 +0000
From: tech-lists <tech-lists@zyxst.net>
To: freebsd-virtualization@freebsd.org
Cc: freebsd-stable@freebsd.org
Subject: bhyve zfs resizing
Message-ID: <20190318150404.GB91631@rpi3.zyxst.net>
Mail-Followup-To: freebsd-virtualization@freebsd.org,
freebsd-stable@freebsd.org
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
protocol="application/pgp-signature"; boundary="XF85m9dhOBO43t/C"
Content-Disposition: inline
User-Agent: Mutt/1.11.2 (2019-01-07)
X-Rspamd-Queue-Id: 5FAFE8CAEF
X-Spamd-Bar: -------
Authentication-Results: mx1.freebsd.org;
dkim=pass header.d=zyxst.net header.s=fm2 header.b=sQTo9Ofu;
dkim=pass header.d=messagingengine.com header.s=fm2 header.b=GSHJjNH4;
spf=pass (mx1.freebsd.org: domain of tech-lists@zyxst.net designates
64.147.123.24 as permitted sender) smtp.mailfrom=tech-lists@zyxst.net
X-Spamd-Result: default: False [-7.27 / 15.00]; ARC_NA(0.00)[];
RCVD_VIA_SMTP_AUTH(0.00)[];
R_DKIM_ALLOW(-0.20)[zyxst.net:s=fm2,messagingengine.com:s=fm2];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
R_SPF_ALLOW(-0.20)[+ip4:64.147.123.24];
TO_MATCH_ENVRCPT_ALL(0.00)[];
MIME_GOOD(-0.20)[multipart/signed,text/plain];
TO_DN_NONE(0.00)[]; DMARC_NA(0.00)[zyxst.net];
NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4];
MX_GOOD(-0.01)[in2-smtp.messagingengine.com,in1-smtp.messagingengine.com,in2-smtp.messagingengine.com,in1-smtp.messagingengine.com,in2-smtp.messagingengine.com,in1-smtp.messagingengine.com,in2-smtp.messagingengine.com,in1-smtp.messagingengine.com,in2-smtp.messagingengine.com,in1-smtp.messagingengine.com,in2-smtp.messagingengine.com,in1-smtp.messagingengine.com,in2-smtp.messagingengine.com,in1-smtp.messagingengine.com];
DKIM_TRACE(0.00)[zyxst.net:+,messagingengine.com:+];
RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.97)[-0.971,0];
SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[];
MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[];
ASN(0.00)[asn:11403, ipnet:64.147.123.0/24, country:US];
IP_SCORE(-1.59)[ipnet: 64.147.123.0/24(-4.45), asn: 11403(-3.44), country:
US(-0.07)];
RCVD_IN_DNSWL_LOW(-0.10)[24.123.147.64.list.dnswl.org : 127.0.5.1]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Mar 2019 15:04:22 -0000
--XF85m9dhOBO43t/C
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Hi,
Apart from the performance benefit as per the section for bhyve in the
handbook, can the size of the zfs-backed guest:
1. be resized from the host?
2. does the guest need to be inactive?
3. can linux guests (or even windows ones) be resized as well?
thanks,
--=20
J.
--XF85m9dhOBO43t/C
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEE8n3tWhxW11Ccvv9/s8o7QhFzNAUFAlyPs1YACgkQs8o7QhFz
NAUlFQ//WBtTcdDJmrOiIvkOmwgwS3ShzwAIVajVHwLzFClPPFIY421lWCD1odVK
MHVsBcf4gkdCzbQRQB9WLlSbHnY/1jJ7RJLptqpujxMd45Dr3DVKY9CpuQu8Gd4b
r/ORT65sMRFFtQGaBACZp2ttcEgQpV05uJgwuhSyyvftFeIfNmB8REZkxvTAlnjO
zjlQ03EHdQeNL2TRU+K/D046VJjVaAIvdpReGG85u7UsBpkTuwN55FpXf9qQv5+l
9zPkefzO5taMbhpe3Oz9souFuAvOoOouYFyJJpFi8xF1c04+cL22S+7Ff14WMjDI
gCGw4LHD/h/HYh11o8W77T6Xo1asABkZJKS8Q+pPtT6pYdRAfweAScOHkFSgiH8P
KVNzDZCK1LyNeG/x3PQIEGHutrxrO7fAVeE9Av9Q78xWC2MckMnM4PQsr67khCRD
2dlzpzOJCWmp84BuHTIhgZfgDc60V1Slk6Ek+pSkdIO061mo9EfM2SfijrT3yCPm
UZ0jSW6fVoRksH/y3A1DOiV7+oUHqI3ccdcSUaQZJFRT3mD4UT2QEoh5gn+XhUHg
hJ3W8yVQ38rQUO+8GnXonnlvLbxojwJSoFnT2R5BRSiDNwB+Li7RC3EVfHyqGbvT
1oGqKpJ7BzOQ7sK6uzggePmWlHiVw+z9BHPbVX5KL38fuj5yAtg=
=LNSz
-----END PGP SIGNATURE-----
--XF85m9dhOBO43t/C--
From owner-freebsd-virtualization@freebsd.org Mon Mar 18 15:08:45 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7F66B154343E;
Mon, 18 Mar 2019 15:08:45 +0000 (UTC)
(envelope-from asomers@gmail.com)
Received: from mail-lj1-f173.google.com (mail-lj1-f173.google.com
[209.85.208.173])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 84FFD8CDDB;
Mon, 18 Mar 2019 15:08:44 +0000 (UTC)
(envelope-from asomers@gmail.com)
Received: by mail-lj1-f173.google.com with SMTP id n18so12419001ljg.11;
Mon, 18 Mar 2019 08:08:44 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:references:in-reply-to:from:date
:message-id:subject:to;
bh=7M6FM8s1F/5jupaGN3K/AHKe4vjcbT2kz9laQUtEgbc=;
b=WjJowxElhZjN/QbCtNAdqdHEsNm6z+9r/Robvt6NVNFWh4P2zxokDzQTMYOHU2cSQg
HeyjSG8AgDHpNkCJ2ta0sbvoOpSwh+E0CA6g5Si/fP4ibtIlmglQQbUscG6qUgV2CL+W
W+XXG5FjJoJrbqc5M3ODwpEG1/9PIxacPu9euQUX+Bplr0r3ok026OfVhQLRaKYG/nYa
aSADLi0ohSmBSBPT0dfR6OUsLYrQ3LoMZaCVJPXpb02WoM5uFmsYuNxN96Z3abQJH3al
84pR36FPnecAt6gHfArTxYsfLVLkNJfDEUGxhVZA/nT4MKK6z0gy1yCzPl7bV6IMWbt2
mxMA==
X-Gm-Message-State: APjAAAUHiuYb8/wyZfvbf2N2v7w2UPS0CMb7yV3Iz15Dat4NyGq/sa55
uSY+r/dLB5+BhWXcfaKXDMFC6xLBtB52mgKqxAy5+vCg
X-Google-Smtp-Source: APXvYqw7yw6A62K2ODW0etZ5sF4yJwXtruA9nFGh+4l532y8OAAL1UBKnuWfsomSE/RSihW54N34j1Nka6Ng9Jgpfiw=
X-Received: by 2002:a2e:8582:: with SMTP id b2mr6017265lji.24.1552921722331;
Mon, 18 Mar 2019 08:08:42 -0700 (PDT)
MIME-Version: 1.0
References: <20190318150404.GB91631@rpi3.zyxst.net>
In-Reply-To: <20190318150404.GB91631@rpi3.zyxst.net>
From: Alan Somers <asomers@freebsd.org>
Date: Mon, 18 Mar 2019 09:08:31 -0600
Message-ID: <CAOtMX2j55KCrwSgoPUmGfgsRYdL4ivwC0kf=ODqHf7gwe6H3Gg@mail.gmail.com>
Subject: Re: bhyve zfs resizing
To: "freebsd-virtualization@freebsd.org" <freebsd-virtualization@freebsd.org>,
FreeBSD <freebsd-stable@freebsd.org>
Content-Type: text/plain; charset="UTF-8"
X-Rspamd-Queue-Id: 84FFD8CDDB
X-Spamd-Bar: ---
Authentication-Results: mx1.freebsd.org;
spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates
209.85.208.173 as permitted sender) smtp.mailfrom=asomers@gmail.com
X-Spamd-Result: default: False [-3.98 / 15.00]; ARC_NA(0.00)[];
TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.996,0];
FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[];
R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17];
TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain];
RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[freebsd.org];
NEURAL_HAM_LONG(-1.00)[-1.000,0];
IP_SCORE(-1.31)[ip: (-0.53), ipnet: 209.85.128.0/17(-3.88), asn: 15169(-2.09),
country: US(-0.07)];
MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com];
RCPT_COUNT_TWO(0.00)[2];
RCVD_IN_DNSWL_NONE(0.00)[173.208.85.209.list.dnswl.org : 127.0.5.0];
NEURAL_HAM_SHORT(-0.66)[-0.657,0];
FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com];
MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[];
FREEMAIL_ENVFROM(0.00)[gmail.com];
ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US];
FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com];
RCVD_COUNT_TWO(0.00)[2]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Mar 2019 15:08:45 -0000
On Mon, Mar 18, 2019 at 9:05 AM tech-lists <tech-lists@zyxst.net> wrote:
>
> Hi,
>
> Apart from the performance benefit as per the section for bhyve in the
> handbook, can the size of the zfs-backed guest:
>
> 1. be resized from the host?
> 2. does the guest need to be inactive?
> 3. can linux guests (or even windows ones) be resized as well?
>
> thanks,
> --
> J.
Do you mean using a zvol as the backing store for a VM? If so, then:
1) Yes. You can just do "zfs set volsize" on the host.
2) In theory no, but the guest may need to be rebooted to notice the
change. And I'm not sure if the current bhyve code will expose the
new size without a reboot or not.
3) Sure. But after you expand the zvol (or before you shrink it),
you'll have to change the size of the guest's filesystem using the
guest's native tools.
-Alan
From owner-freebsd-virtualization@freebsd.org Mon Mar 18 15:25:34 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B3BD1543C8F;
Mon, 18 Mar 2019 15:25:34 +0000 (UTC)
(envelope-from tech-lists@zyxst.net)
Received: from wout1-smtp.messagingengine.com (wout1-smtp.messagingengine.com
[64.147.123.24])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 7711D8D891;
Mon, 18 Mar 2019 15:25:33 +0000 (UTC)
(envelope-from tech-lists@zyxst.net)
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44])
by mailout.west.internal (Postfix) with ESMTP id A978045B0;
Mon, 18 Mar 2019 11:25:31 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
by compute4.internal (MEProxy); Mon, 18 Mar 2019 11:25:31 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zyxst.net; h=
date:from:to:subject:message-id:references:mime-version
:content-type:in-reply-to; s=fm2; bh=/yn8mfkLnZs8QtHF0yFXVC7vqx0
y05CUtyChPjrpgDo=; b=YG1uDenvgktq+mM0uB0PW31bsX8JTy0rVRR3qH8iWme
9ceTnF32rzvW2lI3oZmAqDSOmKLiKu9a0Gr2yMnv2MZ1+fHRU0mUtXVb4YjQl++C
z/aB6hzF4BDsmV+uu4ZF0yCJNMzdIkdAlv5Z5AQNspwcIHru/K0R8xCqVz2mOmjn
iO1qW7Ig22mghunpmPevX5tBomIscQ61c2VRNLRw5Wa+doLidTf0DX02lEEJSDvq
7pOiNXrutq9CBzQex3/tq2yutKDQnqkS9Ngam+M3YvT7PvnZcc0J1Q5jV8hvKsXb
3UTUFxmt7eMGEqltPRi9FmEAExJznyBfmbmwS41p87g==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=content-type:date:from:in-reply-to
:message-id:mime-version:references:subject:to:x-me-proxy
:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=/yn8mf
kLnZs8QtHF0yFXVC7vqx0y05CUtyChPjrpgDo=; b=vUC+qNn8qYa0kxRnzoI70J
vlxGgGirjAPf0+C1Po7V+nES7rxwex8Z28RsRR/pwuZTn4ctKFDiJooWXyg/HImP
d5Tda8PrSpDs3sayRvQXuQnqAF40/2z2ytYSne6eXw+IVHU6CYgA251IoydmkB1M
bbjD/3ULKIbRAdrSKqICvMPNiI93pacuJj9Qs4UTZeVwFs3A+1GnpUDyZmwXu+ek
nKMz+aFM+bzGIuI6z5bx13Lq6EfHMJO87CE9q/1hRDFBiDwoAtZt2mZeYhjx21Q5
Ek/r1QthPZhzA4W0q6/EtiN2xcBZPMHHZklgI4vnpVTihPfQXYINll7DMI6Tmn4Q
==
X-ME-Sender: <xms:ariPXGhJnCa2NKo31TxTYYasVAJLgsOGW8jWlwyu32pvRWcnrLb6Pw>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedutddriedugdejiecutefuodetggdotefrodftvf
curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjfgesghdtre
ertdervdenucfhrhhomhepthgvtghhqdhlihhsthhsuceothgvtghhqdhlihhsthhsseii
hiigshhtrdhnvghtqeenucfkphepkedvrdejtddrledurddutddunecurfgrrhgrmhepmh
grihhlfhhrohhmpehtvggthhdqlhhishhtshesiiihgihsthdrnhgvthenucevlhhushht
vghrufhiiigvpedt
X-ME-Proxy: <xmx:a7iPXO67CdH_WdV6DbZlvq8aAmlUTq3B2ZlcHl02jjBU1VJ-cyWFLw>
<xmx:a7iPXNc3GBc84WI4FtC0xvz4qJOVyZ7wf_PjwrntIBxJ979IxMaz2A>
<xmx:a7iPXHB4AJgix1Y0xg8569Rg96Bmn4Y9_3P3Gn-cJTxhnsMWk_AFUw>
<xmx:a7iPXHJbhS7g1-HVPbCgeQY0ahu9rp9qKcrf2I4gc8d40DzbYUwaew>
Received: from rpi3.zyxst.net (rpi3.zyxst.net [82.70.91.101])
by mail.messagingengine.com (Postfix) with ESMTPA id 78ECFE4684;
Mon, 18 Mar 2019 11:25:30 -0400 (EDT)
Date: Mon, 18 Mar 2019 15:25:26 +0000
From: tech-lists <tech-lists@zyxst.net>
To: freebsd-virtualization@freebsd.org,
FreeBSD <freebsd-stable@freebsd.org>
Subject: Re: bhyve zfs resizing
Message-ID: <20190318152525.GC91631@rpi3.zyxst.net>
Mail-Followup-To: freebsd-virtualization@freebsd.org,
FreeBSD <freebsd-stable@freebsd.org>
References: <20190318150404.GB91631@rpi3.zyxst.net>
<CAOtMX2j55KCrwSgoPUmGfgsRYdL4ivwC0kf=ODqHf7gwe6H3Gg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
protocol="application/pgp-signature"; boundary="Pk6IbRAofICFmK5e"
Content-Disposition: inline
In-Reply-To: <CAOtMX2j55KCrwSgoPUmGfgsRYdL4ivwC0kf=ODqHf7gwe6H3Gg@mail.gmail.com>
User-Agent: Mutt/1.11.2 (2019-01-07)
X-Rspamd-Queue-Id: 7711D8D891
X-Spamd-Bar: -------
Authentication-Results: mx1.freebsd.org;
dkim=pass header.d=zyxst.net header.s=fm2 header.b=YG1uDenv;
dkim=pass header.d=messagingengine.com header.s=fm2 header.b=vUC+qNn8;
spf=pass (mx1.freebsd.org: domain of tech-lists@zyxst.net designates
64.147.123.24 as permitted sender) smtp.mailfrom=tech-lists@zyxst.net
X-Spamd-Result: default: False [-7.30 / 15.00]; ARC_NA(0.00)[];
RCVD_VIA_SMTP_AUTH(0.00)[];
R_DKIM_ALLOW(-0.20)[zyxst.net:s=fm2,messagingengine.com:s=fm2];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.24];
TO_MATCH_ENVRCPT_ALL(0.00)[];
MIME_GOOD(-0.20)[multipart/signed,text/plain];
DMARC_NA(0.00)[zyxst.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0];
RCVD_COUNT_THREE(0.00)[4];
DKIM_TRACE(0.00)[zyxst.net:+,messagingengine.com:+];
RCPT_COUNT_TWO(0.00)[2];
MX_GOOD(-0.01)[cached: in2-smtp.messagingengine.com];
SIGNED_PGP(-2.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.991,0];
FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+];
RCVD_TLS_LAST(0.00)[];
ASN(0.00)[asn:11403, ipnet:64.147.123.0/24, country:US];
IP_SCORE(-1.60)[ipnet: 64.147.123.0/24(-4.47), asn: 11403(-3.44), country:
US(-0.07)];
RCVD_IN_DNSWL_LOW(-0.10)[24.123.147.64.list.dnswl.org : 127.0.5.1]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Mar 2019 15:25:34 -0000
--Pk6IbRAofICFmK5e
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Mon, Mar 18, 2019 at 09:08:31AM -0600, Alan Somers wrote:
>
>Do you mean using a zvol as the backing store for a VM? If so, then:
>1) Yes. You can just do "zfs set volsize" on the host.
>2) In theory no, but the guest may need to be rebooted to notice the
>change. And I'm not sure if the current bhyve code will expose the
>new size without a reboot or not.
>3) Sure. But after you expand the zvol (or before you shrink it),
>you'll have to change the size of the guest's filesystem using the
>guest's native tools.
Great, that's awesome. Thanks for clarifying=20
--=20
J.
--Pk6IbRAofICFmK5e
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEE8n3tWhxW11Ccvv9/s8o7QhFzNAUFAlyPuFsACgkQs8o7QhFz
NAUgvg//Z2bpBqYHbpl1U6fzFBF+NOw+j1iqMP46tia9TziO6FUH+B/KUe9HAx0v
N+QFyPJO6Y5EFZUdKfnAXPpSe1nCA7q5ueFLNoCgJilg13kxLccsvWh2Wo0QO3R7
kdIcGBDAgBF7DM2mWTZdyLe6fdbzKMhfU5vhsZJPw+HQjdo3dCbrd3s2VynyCVT8
XyXDgpbdbm3N3D56JQdxrc2x2CTy7EjaVbYTxD6rfsyyl4Id6s45A4zcBgibtDiE
9ZKcajQmTuxHRmnRsA/4Oax2LMuoLCTFT4nzkry2k7frjJbnQqUTGY7mmVMeeQNO
ZCvGPz/A8RhiW8lg6A6quui9wjWz8ItUhmayRZQ3XzRGi7Z685vJF5DMuyLDWY7r
A6FI4Ta+3cvCxPdsfnQXlY9Ufj3cgtkNWbHgHEoGu0p/YU/H6AEFTE9U1WooY2GX
rJxc+pzt6AJaDXVuj6uy3BImY8Eq3neupbmcZbdGACW3IXfksiGb6gim2N6vyhY9
GGL/S9kzzQEGP4/rcHkI7XHcaWVm7GZ/jaY2jhROhZ+MxLaYZIRntx1ShBEAtX5b
cirLCy9X7nezkk+EQl2KBLux8H52LYD8QHaJVgVFGsDrJw2Ydb+ASRlPhBxjt2H+
1aQMSu/C9i7FC7H9fptEbQ7aAkbavgHuFUBa6A/Ijo/2YCiCCM8=
=3l1R
-----END PGP SIGNATURE-----
--Pk6IbRAofICFmK5e--
From owner-freebsd-virtualization@freebsd.org Mon Mar 18 17:16:10 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 499DF154793F
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Mon, 18 Mar 2019 17:16:10 +0000 (UTC)
(envelope-from ctuffli@gmail.com)
Received: from mail-oi1-f171.google.com (mail-oi1-f171.google.com
[209.85.167.171])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 82A926B718
for <freebsd-virtualization@freebsd.org>; Mon, 18 Mar 2019 17:16:09 +0000 (UTC)
(envelope-from ctuffli@gmail.com)
Received: by mail-oi1-f171.google.com with SMTP id u12so10733262oiv.1
for <freebsd-virtualization@freebsd.org>; Mon, 18 Mar 2019 10:16:09 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
bh=UmOBIWCnN4fQmtkc89a/wEcxCX22OmpnrWaJ1YwT5xA=;
b=EWLk1ilIZAFPQW3HRrm8Wfh3FndufX6/rIch5tlpDHGtTod2wgjez04Lfq5m4VkNPo
i58KqlExAcydILjqOKsaZ48Ggn1PIJytnYWhRPeIXTtQ4iecb49HbE4QkOVTHPBO3CI1
bq/YLMP7wHFAgWn37TLoBnuy5vH50pnvIQRl4uqTQOAdf9bV3Ya53fBikVpK8ypTzwOE
myaOYgAlkGMTxVuHNI297ecXofpv0FC3QQibcoIIP7SufkNSRpikIoHMIT3yF835tI9d
bYuuj7V/5Wb1abqei13xIVruu2rk810pwsNsZt+EZ3cm1p/ozBec8aiKUiAyQo53TpOa
uNag==
X-Gm-Message-State: APjAAAUFLrOyTuX4/UpoMKGFV118l596PtOEkoeXGRwgremhM4dZm20m
A3NRMtPRcrCUL97ff/GsCDUBU5ut
X-Google-Smtp-Source: APXvYqz7yhK+bGEPEBsjNjPAZpVZuM3sbjw41F9lMFMDCAW13rZlfq+JkGea1THro2glU/52Tm0IsQ==
X-Received: by 2002:aca:3a57:: with SMTP id h84mr10099285oia.162.1552929362001;
Mon, 18 Mar 2019 10:16:02 -0700 (PDT)
Received: from mail-oi1-f170.google.com (mail-oi1-f170.google.com.
[209.85.167.170])
by smtp.gmail.com with ESMTPSA id n3sm10777ota.81.2019.03.18.10.16.01
for <freebsd-virtualization@freebsd.org>
(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
Mon, 18 Mar 2019 10:16:01 -0700 (PDT)
Received: by mail-oi1-f170.google.com with SMTP id v7so3936570oie.8
for <freebsd-virtualization@freebsd.org>; Mon, 18 Mar 2019 10:16:01 -0700 (PDT)
X-Received: by 2002:aca:407:: with SMTP id 7mr5467206oie.90.1552929361259;
Mon, 18 Mar 2019 10:16:01 -0700 (PDT)
MIME-Version: 1.0
From: Chuck Tuffli <chuck@freebsd.org>
Date: Mon, 18 Mar 2019 10:15:50 -0700
X-Gmail-Original-Message-ID: <CAKAYmMKd5o=f8Z5uzF6pQnZqWLzRLAL3U7iM5inuodQs92pudA@mail.gmail.com>
Message-ID: <CAKAYmMKd5o=f8Z5uzF6pQnZqWLzRLAL3U7iM5inuodQs92pudA@mail.gmail.com>
Subject: emulated PCI device BAR size
To: freebsd-virtualization@freebsd.org
Content-Type: text/plain; charset="UTF-8"
X-Rspamd-Queue-Id: 82A926B718
X-Spamd-Bar: ----
Authentication-Results: mx1.freebsd.org;
spf=pass (mx1.freebsd.org: domain of ctuffli@gmail.com designates
209.85.167.171 as permitted sender) smtp.mailfrom=ctuffli@gmail.com
X-Spamd-Result: default: False [-4.22 / 15.00]; ARC_NA(0.00)[];
RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17];
TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain];
PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org];
TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1];
NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4];
MIME_TRACE(0.00)[0:+];
MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com];
NEURAL_HAM_SHORT(-0.91)[-0.909,0];
RCVD_IN_DNSWL_NONE(0.00)[171.167.85.209.list.dnswl.org : 127.0.5.0];
RCVD_TLS_LAST(0.00)[];
FORGED_SENDER(0.30)[chuck@freebsd.org,ctuffli@gmail.com];
DMARC_NA(0.00)[freebsd.org]; R_DKIM_NA(0.00)[];
FREEMAIL_ENVFROM(0.00)[gmail.com];
ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US];
FROM_NEQ_ENVFROM(0.00)[chuck@freebsd.org,ctuffli@gmail.com];
IP_SCORE(-1.30)[ip: (-0.45), ipnet: 209.85.128.0/17(-3.88), asn: 15169(-2.09),
country: US(-0.07)]; TO_DOM_EQ_FROM_DOM(0.00)[]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Mar 2019 17:16:10 -0000
In debugging why Windows doesn't like bhyve's NVMe device emulation,
another developer mentioned that the NVMe specification defines the
minimum size of BAR[0] to be 16K bytes. And while most OS don't
enforce this requirement, evidently, Windows does.
On the surface, a "minimum BAR size" sounds odd, but the MMIO BAR in
NVMe includes the doorbell registers for each queue. And since the
maximum number of queues is implementation specific, there isn't a
fixed size for the BAR.
So my question is how to best fix this. The easiest would be to add a
check to the BAR size calculation in pci_nvme.c along the lines of:
#define NVME_MMIO_SPACE_MIN (1 << 14)
...
pci_membar_sz = MAX( <calculation based on queue count>,
NVME_MMIO_SPACE_MIN);
While this makes Windows happy, are there any implications to this
approach other than chewing up more memory than the device actually
needs? Alternatively, is there a better mechanism to use in this case?
--chuck
From owner-freebsd-virtualization@freebsd.org Mon Mar 18 17:56:11 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2CBCD154899C;
Mon, 18 Mar 2019 17:56:11 +0000 (UTC)
(envelope-from SRS0=eii1=RV=quip.cz=000.fbsd@elsa.codelab.cz)
Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id C4C7C6D0CE;
Mon, 18 Mar 2019 17:56:09 +0000 (UTC)
(envelope-from SRS0=eii1=RV=quip.cz=000.fbsd@elsa.codelab.cz)
Received: from elsa.codelab.cz (localhost [127.0.0.1])
by elsa.codelab.cz (Postfix) with ESMTP id E31D628416;
Mon, 18 Mar 2019 18:56:06 +0100 (CET)
Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz
[86.49.16.209])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(No client certificate requested)
by elsa.codelab.cz (Postfix) with ESMTPSA id BC69728417;
Mon, 18 Mar 2019 18:56:05 +0100 (CET)
Subject: Re: bhyve zfs resizing
To: freebsd-virtualization@freebsd.org, FreeBSD <freebsd-stable@freebsd.org>
References: <20190318150404.GB91631@rpi3.zyxst.net>
<CAOtMX2j55KCrwSgoPUmGfgsRYdL4ivwC0kf=ODqHf7gwe6H3Gg@mail.gmail.com>
<20190318152525.GC91631@rpi3.zyxst.net>
From: Miroslav Lachman <000.fbsd@quip.cz>
Message-ID: <f7b4a7d7-4c07-22f2-8d56-9a62c685ea5c@quip.cz>
Date: Mon, 18 Mar 2019 18:56:03 +0100
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101
Firefox/52.0 SeaMonkey/2.49.3
MIME-Version: 1.0
In-Reply-To: <20190318152525.GC91631@rpi3.zyxst.net>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 8bit
X-Rspamd-Queue-Id: C4C7C6D0CE
X-Spamd-Bar: ++++
Authentication-Results: mx1.freebsd.org
X-Spamd-Result: default: False [4.24 / 15.00]; ARC_NA(0.00)[];
RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[];
TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[];
NEURAL_SPAM_SHORT(0.91)[0.914,0]; MIME_GOOD(-0.10)[text/plain];
RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[quip.cz];
AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.96)[0.957,0];
RCVD_COUNT_THREE(0.00)[3];
IP_SCORE(0.21)[ip: (0.50), ipnet: 94.124.104.0/21(0.25), asn: 42000(0.20),
country: CZ(0.07)]; MX_GOOD(-0.01)[cached: elsa.codelab.cz];
RCPT_COUNT_TWO(0.00)[2];
RCVD_IN_DNSWL_NONE(0.00)[4.105.124.94.list.dnswl.org : 127.0.10.0];
NEURAL_SPAM_LONG(0.97)[0.971,0]; R_SPF_NA(0.00)[];
FORGED_SENDER(0.30)[000.fbsd@quip.cz,SRS0=eii1=RV=quip.cz=000.fbsd@elsa.codelab.cz];
RECEIVED_SPAMHAUS_PBL(0.00)[209.16.49.86.zen.spamhaus.org : 127.0.0.11];
R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+];
ASN(0.00)[asn:42000, ipnet:94.124.104.0/21, country:CZ];
FROM_NEQ_ENVFROM(0.00)[000.fbsd@quip.cz,SRS0=eii1=RV=quip.cz=000.fbsd@elsa.codelab.cz];
MID_RHS_MATCH_FROM(0.00)[]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 18 Mar 2019 17:56:11 -0000
tech-lists wrote on 2019/03/18 16:25:
> On Mon, Mar 18, 2019 at 09:08:31AM -0600, Alan Somers wrote:
>>
>> Do you mean using a zvol as the backing store for a VM? If so, then:
>> 1) Yes. You can just do "zfs set volsize" on the host.
>> 2) In theory no, but the guest may need to be rebooted to notice the
>> change. And I'm not sure if the current bhyve code will expose the
>> new size without a reboot or not.
>> 3) Sure. But after you expand the zvol (or before you shrink it),
>> you'll have to change the size of the guest's filesystem using the
>> guest's native tools.
I did it 2 month ago on FreeBSD 11.2.
On the host with running guest:
# zfs set volsize=200G tank1/vol1/bhyve/kotel/disk1
Even if I unmounted disk in the guest it still does not see the new size
until I rebooted the guest.
After reboot of the guest, you will see corrupted GPT:
# gpart show -p vtbd1
=> 40 209715120 vtbd1 GPT (200G) [CORRUPT]
40 8 - free - (4.0K)
48 1024 vtbd1p1 freebsd-boot (512K)
1072 976 - free - (488K)
2048 203423744 vtbd1p2 freebsd-ufs (97G)
203425792 6289368 - free - (3.0G)
And after running recover, the guest will see the added space
# gpart recover vtbd1
vtbd1 recovered
# gpart show -p vtbd1
=> 40 419430320 vtbd1 GPT (200G)
40 8 - free - (4.0K)
48 1024 vtbd1p1 freebsd-boot (512K)
1072 976 - free - (488K)
2048 203423744 vtbd1p2 freebsd-ufs (97G)
203425792 216004568 - free - (103G)
After this, the partition can finally be enlarged
# gpart resize -a 1M -s 197G -i 2 vtbd1
# growfs /vol0
Kind regards
Miroslav Lachman
From owner-freebsd-virtualization@freebsd.org Tue Mar 19 00:23:34 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id D08A81551FEC;
Tue, 19 Mar 2019 00:23:33 +0000 (UTC)
(envelope-from tech-lists@zyxst.net)
Received: from wout1-smtp.messagingengine.com (wout1-smtp.messagingengine.com
[64.147.123.24])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 849F685B52;
Tue, 19 Mar 2019 00:23:32 +0000 (UTC)
(envelope-from tech-lists@zyxst.net)
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44])
by mailout.west.internal (Postfix) with ESMTP id 3E49438AA;
Mon, 18 Mar 2019 20:23:30 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
by compute4.internal (MEProxy); Mon, 18 Mar 2019 20:23:30 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zyxst.net; h=
date:from:to:subject:message-id:references:mime-version
:content-type:in-reply-to; s=fm2; bh=n4DsVE1Vk7yxDlJ8c6KpqfJbQAS
XDBUQz0B5HlP/dYI=; b=VO67TFcZPZfPKy58xPuix9pYQyjAL1Mg/WzNcIZh+Kd
X+cxcza2r2XINJb2JnR4wKI7FK63byR9qXpKq6LaJ9kNTpl5jEjulRX0EBz5rPkp
HHG5zHZng5YNPENmjGjmgaJp4OEzt/EceYNZ4wGY96Eudquqescz38vuMn6lFha+
kRV5eEvf1L1Rpc+/EFG1lypIvDkC8ItK2cD6NnlqiL5QdNqUi+slwv0ZaVAaP9b+
XKKA83mRoztSX88kSHy/xNKQHON0O7FIrVd2/nsSzLf2Nwcsz2fERTn95RbsL8iD
MZuPX87bOyI1XMnoigxj6mms69QGz1f0uP8rQSjWdkg==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=content-type:date:from:in-reply-to
:message-id:mime-version:references:subject:to:x-me-proxy
:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=n4DsVE
1Vk7yxDlJ8c6KpqfJbQASXDBUQz0B5HlP/dYI=; b=LhXMQWmeey+S3sZxiFENKi
EDxd4yaeZn6hF3HdoxZiP2OdnV5K8JuwD5NdfJIblBZNxF+KglaO6Dm633hXVUOA
xA9Hh99UIxAUL9kt4gay1wBMaYnXkXo/Pe6mtL8N5nX6W3SbfEQ1H/nts9X34vvE
+rT6wvCojG6SBI+6y7RbSDCjsEBDe7VoYRCkciKbBXZn3lVaZxgJwKwQEWrbmkOX
KweEhv7DpBa5eWnB3VODlxhrx9Kq6KFlKkCGr6aoVoPES76h1GcNjXoMnjMjQoTW
ZDeeNNcRkYkDXYj6nziNpYD1rt0f+0bCSlZDsF1icgB33HY2jpKBaHznrOQ4qpkw
==
X-ME-Sender: <xms:gTaQXMdO6b7v9lyXdmScMSc-2uk1Qj4S6r2tHTpx4SfPmYhqaBUoIw>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedutddrieefgddufecutefuodetggdotefrodftvf
curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfhfgggtuggjfgesghdtre
ertdervdenucfhrhhomhepthgvtghhqdhlihhsthhsuceothgvtghhqdhlihhsthhsseii
hiigshhtrdhnvghtqeenucfkphepkedvrdejtddrledurddutddunecurfgrrhgrmhepmh
grihhlfhhrohhmpehtvggthhdqlhhishhtshesiiihgihsthdrnhgvthenucevlhhushht
vghrufhiiigvpedt
X-ME-Proxy: <xmx:gTaQXFpMY8k3sc1ugHhwKQUkzIMWuMwqOSG2IFToFF3QoPAugVX5ng>
<xmx:gTaQXK5mVplRmd2ZdlON2oNnVliAFsgOXLaQURwrAeNmSrtxLGlSmg>
<xmx:gTaQXLqPPF_88Meu0UUQBOOVugbxoFzL-Tjkr-njg2FRB8LnXC2zaA>
<xmx:gTaQXEkc1ptz-UwlgrBH07WJ1WXWCkOfLK5ebsCqvsM5eeerY9kbIA>
Received: from rpi3.zyxst.net (rpi3.zyxst.net [82.70.91.101])
by mail.messagingengine.com (Postfix) with ESMTPA id 0F420E465F;
Mon, 18 Mar 2019 20:23:28 -0400 (EDT)
Date: Tue, 19 Mar 2019 00:23:26 +0000
From: tech-lists <tech-lists@zyxst.net>
To: freebsd-virtualization@freebsd.org,
FreeBSD <freebsd-stable@freebsd.org>
Subject: Re: bhyve zfs resizing
Message-ID: <20190319002325.GD91631@rpi3.zyxst.net>
Mail-Followup-To: freebsd-virtualization@freebsd.org,
FreeBSD <freebsd-stable@freebsd.org>
References: <20190318150404.GB91631@rpi3.zyxst.net>
<CAOtMX2j55KCrwSgoPUmGfgsRYdL4ivwC0kf=ODqHf7gwe6H3Gg@mail.gmail.com>
<20190318152525.GC91631@rpi3.zyxst.net>
<f7b4a7d7-4c07-22f2-8d56-9a62c685ea5c@quip.cz>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
protocol="application/pgp-signature"; boundary="vni90+aGYgRvsTuO"
Content-Disposition: inline
In-Reply-To: <f7b4a7d7-4c07-22f2-8d56-9a62c685ea5c@quip.cz>
User-Agent: Mutt/1.11.2 (2019-01-07)
X-Rspamd-Queue-Id: 849F685B52
X-Spamd-Bar: -------
Authentication-Results: mx1.freebsd.org;
dkim=pass header.d=zyxst.net header.s=fm2 header.b=VO67TFcZ;
dkim=pass header.d=messagingengine.com header.s=fm2 header.b=LhXMQWme;
spf=pass (mx1.freebsd.org: domain of tech-lists@zyxst.net designates
64.147.123.24 as permitted sender) smtp.mailfrom=tech-lists@zyxst.net
X-Spamd-Result: default: False [-7.15 / 15.00]; ARC_NA(0.00)[];
RCVD_VIA_SMTP_AUTH(0.00)[];
R_DKIM_ALLOW(-0.20)[zyxst.net:s=fm2,messagingengine.com:s=fm2];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.24];
TO_MATCH_ENVRCPT_ALL(0.00)[];
MIME_GOOD(-0.20)[multipart/signed,text/plain];
DMARC_NA(0.00)[zyxst.net]; NEURAL_HAM_LONG(-1.00)[-1.000,0];
RCVD_COUNT_THREE(0.00)[4];
MX_GOOD(-0.01)[cached: in2-smtp.messagingengine.com];
DKIM_TRACE(0.00)[zyxst.net:+,messagingengine.com:+];
RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.84)[-0.839,0];
SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[];
MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[];
ASN(0.00)[asn:11403, ipnet:64.147.123.0/24, country:US];
IP_SCORE(-1.60)[ipnet: 64.147.123.0/24(-4.49), asn: 11403(-3.44), country:
US(-0.07)];
RCVD_IN_DNSWL_LOW(-0.10)[24.123.147.64.list.dnswl.org : 127.0.5.1]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 19 Mar 2019 00:23:34 -0000
--vni90+aGYgRvsTuO
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Mon, Mar 18, 2019 at 06:56:03PM +0100, Miroslav Lachman wrote:
[...]
Thanks for the example, I've saved it.
Ok just one other question, which I might have found the answer to, or
might not. I'm new to this virtualising on zfs even though ive used zfs
for years. It's basically:
I made a zvol, installed 12-R into it,. Where the disks option came up I
chose the auto defaults for *ZFS* in the guest. I also selected
encryption for both the virtual disk and swap. I think perhaps I
shouldn't have done all this together in the same vm because with apache
running in it, httpd got wedged (and then everything got wedged. sync
wouldn't return). I think the top zfs layer and the encryption layer and
the zfs underneath got too busy. Happily the server still responded to a
shutdown -r and came back up. It's scrubbing the zpool to be on the safe
side.
Am I correct? In that I should have used UFS in the guest rather than
zfs? Or was it the encryption?
thanks,
--=20
J.
--vni90+aGYgRvsTuO
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEE8n3tWhxW11Ccvv9/s8o7QhFzNAUFAlyQNnIACgkQs8o7QhFz
NAVQPw/+M6VlF4Ypb6WbYP9RWYaNjmtM78W2YFnLEzVHTGud2NRALXf5L/da2QJ5
SsRJbV6VshSGs3sWIeYOaCq6JviHBEf8V8UvOHDWSRfztoA35oZGaXntsGhu9gse
qea34NzRQSnbgSP84/SGncMTF6SMNQGOzepcO1zpDOVbeidZwvBfq/yvRD98jjoR
kBmcCR/z1Tg3Ib1tkXCdR1PHIM5XQE/DzSWxmRQIFk29Pwdif/ptsWKODVJUBAHj
NSlLQeT3j8Rq+V3qffAM7xtKsXsb7ga4B+8boGdwM13SPyteCm0FAiVvN0oklkL0
YcaxlWEdpPGLSCK21iRwJJ65azfD4rTa+M7sv4/kcNI10lEBh2EaNw4j9gws87Fx
wtQMZTcdljkv2ARYySXD/e6XEsHJ/bs6kwYb7vlgRY4bY7oScgs6CxiM5FijA63k
WshkwmRRGZ5XuM8vJGIT6j89bS3cxnWLpv8A6AuLAXfkmqci9J2oPaO0G7i9v6BW
+bphPbPCsVPCUAr2SLDMD3j3woGdXnYwb5VaRXqjgPFXj/iLqp100O4HTp3NNsDw
LaFiYHZ6tnWCYCHwc0j7cxgndKfEkvsNs69lOZBdOuxX2/47deVu53t+BF0J+bH8
7J2oDjZts9SWbrAEnXQFzeM3HuFpEubxQI5z/lz6glApDlsA8uk=
=PkcC
-----END PGP SIGNATURE-----
--vni90+aGYgRvsTuO--
From owner-freebsd-virtualization@freebsd.org Tue Mar 19 00:28:33 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 22365152A57A;
Tue, 19 Mar 2019 00:28:33 +0000 (UTC)
(envelope-from asomers@gmail.com)
Received: from mail-lf1-f45.google.com (mail-lf1-f45.google.com
[209.85.167.45])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 42E15860E4;
Tue, 19 Mar 2019 00:28:32 +0000 (UTC)
(envelope-from asomers@gmail.com)
Received: by mail-lf1-f45.google.com with SMTP id p1so12995544lfk.9;
Mon, 18 Mar 2019 17:28:32 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:references:in-reply-to:from:date
:message-id:subject:to;
bh=iOhevDRY+hweLBoticrI6QocbmRP1h4+75kC+bSCKto=;
b=KVrUWlsg96Wv302lQvQeFz8dE0GzKt2gg0xD0KiuNaYDcoSYJyH7qiIVGQ3uLSNx4b
NycD9mK0zHokXGKSxzH3M9mIKqIN+nuanQVBX+t51KNKy/6FTLOG1SQ/KpWb5eqazdD/
RR7IVIxtFz8AGvuw8Y4kmswanSniHS20tuJh9nGnzJf444o+tyah787t9FVtrIWvvu+h
FtWHl/kg2/yfZnCnKvU5L9yyCfCJq+/cjUha1LFJtxW9G0MJkW/ZHq1MRCKcuJvRgmBN
+lHauUuwZD7PYypfO9KsmdVpDMbSHiAjdOIQKb8xEEA+tL9Xv6qTuW4QAJhHJ5DQI3hF
tEYQ==
X-Gm-Message-State: APjAAAXGsu7GHbxCI0sLljtjSMWD5e7nnGRmSH0k+8BtOmtywUwEnt3j
rNjbqTPAL/wsRjS4ceNBEPshB1cLLPITGr5JBSTQKc/Z
X-Google-Smtp-Source: APXvYqzgLR+MrOSBSqJ7l7eUg9B6nSuKVaXCGACinYKM2lJRq5bajWiaPGTDXQAmRpx3PwEq677WIDH8yC/yu5JoBUs=
X-Received: by 2002:ac2:43ba:: with SMTP id t26mr11184041lfl.10.1552955304427;
Mon, 18 Mar 2019 17:28:24 -0700 (PDT)
MIME-Version: 1.0
References: <20190318150404.GB91631@rpi3.zyxst.net>
<CAOtMX2j55KCrwSgoPUmGfgsRYdL4ivwC0kf=ODqHf7gwe6H3Gg@mail.gmail.com>
<20190318152525.GC91631@rpi3.zyxst.net>
<f7b4a7d7-4c07-22f2-8d56-9a62c685ea5c@quip.cz>
<20190319002325.GD91631@rpi3.zyxst.net>
In-Reply-To: <20190319002325.GD91631@rpi3.zyxst.net>
From: Alan Somers <asomers@freebsd.org>
Date: Mon, 18 Mar 2019 18:28:13 -0600
Message-ID: <CAOtMX2hRPpeW0=DLQApRf3roqFtiYwzRWtv9yB-PrZO_A6k_dA@mail.gmail.com>
Subject: Re: bhyve zfs resizing
To: "freebsd-virtualization@freebsd.org" <freebsd-virtualization@freebsd.org>,
FreeBSD <freebsd-stable@freebsd.org>
Content-Type: text/plain; charset="UTF-8"
X-Rspamd-Queue-Id: 42E15860E4
X-Spamd-Bar: ---
Authentication-Results: mx1.freebsd.org;
spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates
209.85.167.45 as permitted sender) smtp.mailfrom=asomers@gmail.com
X-Spamd-Result: default: False [-3.85 / 15.00]; ARC_NA(0.00)[];
TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0];
FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[];
R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17];
TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain];
RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[freebsd.org];
NEURAL_HAM_LONG(-1.00)[-1.000,0];
IP_SCORE(-1.32)[ip: (-0.59), ipnet: 209.85.128.0/17(-3.87), asn: 15169(-2.10),
country: US(-0.07)];
MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com];
RCPT_COUNT_TWO(0.00)[2];
RCVD_IN_DNSWL_NONE(0.00)[45.167.85.209.list.dnswl.org : 127.0.5.0];
NEURAL_HAM_SHORT(-0.51)[-0.512,0];
FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com];
MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[];
FREEMAIL_ENVFROM(0.00)[gmail.com];
ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US];
FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com];
RCVD_COUNT_TWO(0.00)[2]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 19 Mar 2019 00:28:33 -0000
On Mon, Mar 18, 2019 at 6:24 PM tech-lists <tech-lists@zyxst.net> wrote:
>
> On Mon, Mar 18, 2019 at 06:56:03PM +0100, Miroslav Lachman wrote:
>
> [...]
>
> Thanks for the example, I've saved it.
>
> Ok just one other question, which I might have found the answer to, or
> might not. I'm new to this virtualising on zfs even though ive used zfs
> for years. It's basically:
>
> I made a zvol, installed 12-R into it,. Where the disks option came up I
> chose the auto defaults for *ZFS* in the guest. I also selected
> encryption for both the virtual disk and swap. I think perhaps I
> shouldn't have done all this together in the same vm because with apache
> running in it, httpd got wedged (and then everything got wedged. sync
> wouldn't return). I think the top zfs layer and the encryption layer and
> the zfs underneath got too busy. Happily the server still responded to a
> shutdown -r and came back up. It's scrubbing the zpool to be on the safe
> side.
>
> Am I correct? In that I should have used UFS in the guest rather than
> zfs? Or was it the encryption?
>
> thanks,
> --
> J.
Running ZFS inside of a ZFS-backed guest will be slower than using
UFS, but it should work just fine. I doubt that it was the cause of
your problem.
-Alan
From owner-freebsd-virtualization@freebsd.org Tue Mar 19 00:49:34 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id CD535152C3F9;
Tue, 19 Mar 2019 00:49:34 +0000 (UTC)
(envelope-from SRS0=58Nm=RW=quip.cz=000.fbsd@elsa.codelab.cz)
Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 339C186F97;
Tue, 19 Mar 2019 00:49:34 +0000 (UTC)
(envelope-from SRS0=58Nm=RW=quip.cz=000.fbsd@elsa.codelab.cz)
Received: from elsa.codelab.cz (localhost [127.0.0.1])
by elsa.codelab.cz (Postfix) with ESMTP id F2F1C28417;
Tue, 19 Mar 2019 01:49:22 +0100 (CET)
Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz
[86.49.16.209])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(No client certificate requested)
by elsa.codelab.cz (Postfix) with ESMTPSA id D6E1D28416;
Tue, 19 Mar 2019 01:49:15 +0100 (CET)
Subject: Re: bhyve zfs resizing
To: freebsd-virtualization@freebsd.org, FreeBSD <freebsd-stable@freebsd.org>
References: <20190318150404.GB91631@rpi3.zyxst.net>
<CAOtMX2j55KCrwSgoPUmGfgsRYdL4ivwC0kf=ODqHf7gwe6H3Gg@mail.gmail.com>
<20190318152525.GC91631@rpi3.zyxst.net>
<f7b4a7d7-4c07-22f2-8d56-9a62c685ea5c@quip.cz>
<20190319002325.GD91631@rpi3.zyxst.net>
From: Miroslav Lachman <000.fbsd@quip.cz>
Message-ID: <49d8d837-b734-0770-20fd-a439c6436a4d@quip.cz>
Date: Tue, 19 Mar 2019 01:49:10 +0100
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101
Firefox/52.0 SeaMonkey/2.49.3
MIME-Version: 1.0
In-Reply-To: <20190319002325.GD91631@rpi3.zyxst.net>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-Rspamd-Queue-Id: 339C186F97
X-Spamd-Bar: ++++
Authentication-Results: mx1.freebsd.org
X-Spamd-Result: default: False [4.29 / 15.00]; ARC_NA(0.00)[];
RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[];
TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[];
NEURAL_SPAM_SHORT(0.98)[0.980,0]; MIME_GOOD(-0.10)[text/plain];
RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[quip.cz];
AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.96)[0.962,0];
RCVD_COUNT_THREE(0.00)[3];
IP_SCORE(0.19)[ip: (0.48), ipnet: 94.124.104.0/21(0.24), asn: 42000(0.19),
country: CZ(0.07)]; MX_GOOD(-0.01)[cached: elsa.codelab.cz];
RCPT_COUNT_TWO(0.00)[2];
RCVD_IN_DNSWL_NONE(0.00)[4.105.124.94.list.dnswl.org : 127.0.10.0];
NEURAL_SPAM_LONG(0.97)[0.967,0]; R_SPF_NA(0.00)[];
FORGED_SENDER(0.30)[000.fbsd@quip.cz,SRS0=58Nm=RW=quip.cz=000.fbsd@elsa.codelab.cz];
RECEIVED_SPAMHAUS_PBL(0.00)[209.16.49.86.zen.spamhaus.org : 127.0.0.11];
R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+];
ASN(0.00)[asn:42000, ipnet:94.124.104.0/21, country:CZ];
FROM_NEQ_ENVFROM(0.00)[000.fbsd@quip.cz,SRS0=58Nm=RW=quip.cz=000.fbsd@elsa.codelab.cz];
MID_RHS_MATCH_FROM(0.00)[]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 19 Mar 2019 00:49:35 -0000
tech-lists wrote on 2019/03/19 01:23:
> Am I correct? In that I should have used UFS in the guest rather than
> zfs? Or was it the encryption?
As Alan already wrote - you can use ZFS inside of the guest but I would
never choose ZFS in zvol backed guest. I prefere UFS. It is faster and
does not need so much memory as ZFS does. My VirtualBox and Bhyve guests
are small. Sometimes <1GB of RAM. Sometimes 2GB of RAM and that is very
small for ZFS.
May be you can try to limit ZFS ARC size in /etc/sysctl.conf or in
/boot/loader.conf
vfs.zfs.arc_max
Choose about 1/4 of your guest's RAM size and test it again.
Kind regards
Miroslav Lachman
From owner-freebsd-virtualization@freebsd.org Tue Mar 19 02:46:41 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 71103153A5EC
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Tue, 19 Mar 2019 02:46:41 +0000 (UTC)
(envelope-from vas@mpeks.tomsk.su)
Received: from admin.sibptus.ru (admin.sibptus.ru
[IPv6:2001:19f0:5001:21dc::10])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id A49FA8E473
for <freebsd-virtualization@freebsd.org>; Tue, 19 Mar 2019 02:46:40 +0000 (UTC)
(envelope-from vas@mpeks.tomsk.su)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sibptus.ru;
s=20181118; h=Message-ID:Subject:To:From:Date:In-Reply-To;
bh=JdKe2TYUcbcASPvurfcBUEw9d+8KaN7PYbtLGqrCQME=; b=kMAQPTNHjKIYJ1+ZIWjd5VanS3
v+STZSW5bfjpwmEFQ/HVcuXODguQJZbejsXlpQL7bba6IruXYTLd7WkD5jq2IAkqigiQlDzcvlmWe
vroc8HEOH0XUHMq/YqMZtMbIPmIIo/QsmZrEU6AgKiMscOnOf2jNuJwJoHOBO0yJDokY=;
Received: from vas by admin.sibptus.ru with local (Exim 4.92 (FreeBSD))
(envelope-from <vas@mpeks.tomsk.su>) id 1h64ly-0002HY-UA
for freebsd-virtualization@freebsd.org; Tue, 19 Mar 2019 09:46:38 +0700
Date: Tue, 19 Mar 2019 09:46:38 +0700
From: Victor Sudakov <vas@mpeks.tomsk.su>
To: freebsd-virtualization@freebsd.org
Subject: bhyve and vfs.zfs.arc_max, and zfs tuning for a hypervisor
Message-ID: <20190319024638.GA8193@admin.sibptus.ru>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
protocol="application/pgp-signature"; boundary="bg08WKrSYDhXBjb5"
Content-Disposition: inline
X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov
X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634
User-Agent: Mutt/1.11.3 (2019-02-01)
Sender: Victor Sudakov <vas@sibptus.ru>
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 19 Mar 2019 02:46:41 -0000
--bg08WKrSYDhXBjb5
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Dear Colleagues,
I have a FreeBSD 11.2 system with 32G RAM which I'm going to use as a
bhyve host with zvols and sparse zvols, with (almost) no
services/daemons of its own.=20
Could you please clarify some points for me?
1. Does ARC actually cache zfs volumes (not files/datasets)?
2. If ARC does cache volumes, does this cache make sense on a hypervisor,
because guest OSes will probably have their own disk cache anyway.
3. Would it make sense to limit vfs.zfs.arc_max to 1/8 or even less of
total RAM, so that most RAM is available to guest machines?
4. What other zfs tuning measures can you suggest for a bhyve
hypervisor?
--=20
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/
--bg08WKrSYDhXBjb5
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQEcBAEBAgAGBQJckFgOAAoJEA2k8lmbXsY0A24H/3GW0qbH0lZ2em6XApGBXBqX
pid3dfr2kBGoNLNlMhdcJgGE1uyouVIfFTrq+rma4ZDE8jWmLd0OBIHLxSH6WvV4
0IoycEKUrJP59StI1zk8UCsBXDYplHxDEMsXGrsdbUFP4BET4r9K4E19vHBak8PQ
eiXGEgjD3ctfV7AuaBBJnQqRBvfa+pTakmTqzv081YQFm6kF0DNph35YlLDEWRhv
H+cv079/ckHUQl5G+WihmhlRaWYKsGRm7adhMw+BJbQJ0wZf9A3w5HXpvDh0rMYd
SnjIqy5QCpqfsueHX0NIZrEMzacu0WnZ9NNRxDTSakMocm4Txdo43X5+YrhsWlE=
=uMHF
-----END PGP SIGNATURE-----
--bg08WKrSYDhXBjb5--
From owner-freebsd-virtualization@freebsd.org Tue Mar 19 08:06:17 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1C0BB154597C
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Tue, 19 Mar 2019 08:06:17 +0000 (UTC) (envelope-from hausen@punkt.de)
Received: from kagate.punkt.de (kagate.punkt.de [217.29.33.131])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 8975671336
for <freebsd-virtualization@freebsd.org>; Tue, 19 Mar 2019 08:06:15 +0000 (UTC)
(envelope-from hausen@punkt.de)
Received: from hugo10.ka.punkt.de (hugo10.ka.punkt.de [217.29.44.10])
by gate2.intern.punkt.de with ESMTP id x2J86DRB005753;
Tue, 19 Mar 2019 09:06:13 +0100 (CET)
Received: from [217.29.44.250] ([217.29.44.250])
by hugo10.ka.punkt.de (8.14.2/8.14.2) with ESMTP id x2J86DOM074797;
Tue, 19 Mar 2019 09:06:13 +0100 (CET) (envelope-from hausen@punkt.de)
Content-Type: text/plain;
charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 12.0 \(3445.100.39\))
Subject: Re: bhyve and vfs.zfs.arc_max, and zfs tuning for a hypervisor
From: "Patrick M. Hausen" <hausen@punkt.de>
In-Reply-To: <20190319024638.GA8193@admin.sibptus.ru>
Date: Tue, 19 Mar 2019 09:06:11 +0100
Cc: freebsd-virtualization@freebsd.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <AA997A20-4EBF-46DF-A517-FD2072682545@punkt.de>
References: <20190319024638.GA8193@admin.sibptus.ru>
To: Victor Sudakov <vas@mpeks.tomsk.su>
X-Mailer: Apple Mail (2.3445.100.39)
X-Rspamd-Queue-Id: 8975671336
X-Spamd-Bar: ------
Authentication-Results: mx1.freebsd.org
X-Spamd-Result: default: False [-6.97 / 15.00];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[];
NEURAL_HAM_SHORT(-0.97)[-0.971,0]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 19 Mar 2019 08:06:17 -0000
Hi!
> Am 19.03.2019 um 03:46 schrieb Victor Sudakov <vas@mpeks.tomsk.su>:
> 1. Does ARC actually cache zfs volumes (not files/datasets)?
Yes it does.
> 2. If ARC does cache volumes, does this cache make sense on a =
hypervisor,
> because guest OSes will probably have their own disk cache anyway.
IMHO not much, because the guest OS is relying on the fact that when
it writes it=E2=80=99s own cached data out to =E2=80=9Edisk=E2=80=9C, it =
will be committed to
stable storage.
> 3. Would it make sense to limit vfs.zfs.arc_max to 1/8 or even less of
> total RAM, so that most RAM is available to guest machines?
Yes if you build your own solution on plain FreeBSD. No if you are =
running
FreeNAS which already tries to autotune the ARC size according to the
memory committed to VMs.
> 4. What other zfs tuning measures can you suggest for a bhyve
> hypervisor?
e.g.
zfs set sync=3Dalways zfs/vm
if zfs/vm is the dataset under which you create the ZVOLs for your =
emulated
disks.
I=E2=80=99m using this for all my VM =E2=80=9Edisks=E2=80=9C and have =
added a 16 GB SLOG device
to my spinning disk pool - seems to work great. This is on a home =
system.
Our new data centre systems feature all NVME SSDs and no spinning rust.
So no need for a separate SLOG.
HTH,
Patrick
--=20
punkt.de GmbH Internet - Dienstleistungen - Beratung
Kaiserallee 13a Tel.: 0721 9109-0 Fax: -100
76133 Karlsruhe info@punkt.de http://punkt.de
AG Mannheim 108285 Gf: Juergen Egeling
From owner-freebsd-virtualization@freebsd.org Tue Mar 19 13:08:11 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 61B501550811
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Tue, 19 Mar 2019 13:08:11 +0000 (UTC)
(envelope-from bugzilla-noreply@freebsd.org)
Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org
[IPv6:2001:1900:2254:206a::50:5])
by mx1.freebsd.org (Postfix) with ESMTP id EA39D84988
for <freebsd-virtualization@freebsd.org>; Tue, 19 Mar 2019 13:08:10 +0000 (UTC)
(envelope-from bugzilla-noreply@freebsd.org)
Received: by mailman.ysv.freebsd.org (Postfix)
id AAB6C1550810; Tue, 19 Mar 2019 13:08:10 +0000 (UTC)
Delivered-To: virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 94F4E155080F
for <virtualization@mailman.ysv.freebsd.org>;
Tue, 19 Mar 2019 13:08:10 +0000 (UTC)
(envelope-from bugzilla-noreply@freebsd.org)
Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org
[IPv6:2001:1900:2254:206a::19:3])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
server-signature RSA-PSS (4096 bits)
client-signature RSA-PSS (4096 bits) client-digest SHA256)
(Client CN "mxrelay.ysv.freebsd.org",
Issuer "Let's Encrypt Authority X3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 2B28484985
for <virtualization@FreeBSD.org>; Tue, 19 Mar 2019 13:08:10 +0000 (UTC)
(envelope-from bugzilla-noreply@freebsd.org)
Received: from kenobi.freebsd.org (kenobi.freebsd.org
[IPv6:2001:1900:2254:206a::16:76])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 40926B063
for <virtualization@FreeBSD.org>; Tue, 19 Mar 2019 13:08:09 +0000 (UTC)
(envelope-from bugzilla-noreply@freebsd.org)
Received: from kenobi.freebsd.org ([127.0.1.118])
by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x2JD89x8072944
for <virtualization@FreeBSD.org>; Tue, 19 Mar 2019 13:08:09 GMT
(envelope-from bugzilla-noreply@freebsd.org)
Received: (from www@localhost)
by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x2JD89jw072943
for virtualization@FreeBSD.org; Tue, 19 Mar 2019 13:08:09 GMT
(envelope-from bugzilla-noreply@freebsd.org)
X-Authentication-Warning: kenobi.freebsd.org: www set sender to
bugzilla-noreply@freebsd.org using -f
From: bugzilla-noreply@freebsd.org
To: virtualization@FreeBSD.org
Subject: [Bug 236042] Windows Server 2016 Hyper-V snapshot triggers SCSI errors
Date: Tue, 19 Mar 2019 13:08:09 +0000
X-Bugzilla-Reason: AssignedTo
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: Base System
X-Bugzilla-Component: kern
X-Bugzilla-Version: 12.0-RELEASE
X-Bugzilla-Keywords:
X-Bugzilla-Severity: Affects Only Me
X-Bugzilla-Who: j.morillo@educationetformation.fr
X-Bugzilla-Status: New
X-Bugzilla-Resolution:
X-Bugzilla-Priority: ---
X-Bugzilla-Assigned-To: virtualization@FreeBSD.org
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: cc
Message-ID: <bug-236042-27103-PX3lngvkOM@https.bugs.freebsd.org/bugzilla/>
In-Reply-To: <bug-236042-27103@https.bugs.freebsd.org/bugzilla/>
References: <bug-236042-27103@https.bugs.freebsd.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 19 Mar 2019 13:08:11 -0000
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D236042
j.morillo@educationetformation.fr changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |j.morillo@educationetformat
| |ion.fr
--- Comment #2 from j.morillo@educationetformation.fr ---
I'm also having same problem with different setup:
- HyperV 2016 with OPNSense as Guest (FreeBSD 11.1-RELEASE-p17)
This VM has a replicate on another HyperV wich often failed (need to force =
sync
again).
Here is FreeBSD log:
(da0:storvsc0:0:0:0): WRITE(10). CDB: 2a 00 00 cd ac a8 00 01 00 00
(da0:storvsc0:0:0:0): CAM status: SCSI Status Error
(da0:storvsc0:0:0:0): SCSI status: Check Condition
(da0:storvsc0:0:0:0): SCSI sense: UNIT ATTENTION asc:3f,2 (Changed operating
definition)
(da0:storvsc0:0:0:0): Retrying command (per sense data)
(da0:storvsc0:0:0:0): WRITE(10). CDB: 2a 00 00 ca 90 68 00 00 40 00
(da0:storvsc0:0:0:0): CAM status: SCSI Status Error
(da0:storvsc0:0:0:0): SCSI status: Check Condition
(da0:storvsc0:0:0:0): SCSI sense: UNIT ATTENTION asc:3f,2 (Changed operating
definition)
(da0:storvsc0:0:0:0): Retrying command (per sense data)
--=20
You are receiving this mail because:
You are the assignee for the bug.=
From owner-freebsd-virtualization@freebsd.org Wed Mar 20 01:52:38 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4B0CF15443B4
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Wed, 20 Mar 2019 01:52:38 +0000 (UTC)
(envelope-from vas@mpeks.tomsk.su)
Received: from admin.sibptus.ru (admin.sibptus.ru
[IPv6:2001:19f0:5001:21dc::10])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 7D30A80451
for <freebsd-virtualization@freebsd.org>; Wed, 20 Mar 2019 01:52:37 +0000 (UTC)
(envelope-from vas@mpeks.tomsk.su)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sibptus.ru;
s=20181118; h=In-Reply-To:Message-ID:Subject:To:From:Date;
bh=53CR8DbjxNXHgU1PD9OnucoU9ZALOyPksYzTnZIcDJs=; b=Epy191gURcghI2j4DnbfZgpMpP
5Rw3A8fOUF6Ley8Km8obtjXXiQ/P2lwTn/bXUxNJxTySiMVvfjmptRrHlkHsh9XcG//lMxjBThMuT
hImLtpyTr0SK0hvUiZNn69uCtv0FGacgpyExxOoHDsLmlJt+xPmyaPKUnsg6WFC37Avk=;
Received: from vas by admin.sibptus.ru with local (Exim 4.92 (FreeBSD))
(envelope-from <vas@mpeks.tomsk.su>) id 1h6QPE-0009Hf-Dn
for freebsd-virtualization@freebsd.org; Wed, 20 Mar 2019 08:52:36 +0700
Date: Wed, 20 Mar 2019 08:52:36 +0700
From: Victor Sudakov <vas@mpeks.tomsk.su>
To: freebsd-virtualization@freebsd.org
Subject: Re: bhyve and vfs.zfs.arc_max, and zfs tuning for a hypervisor
Message-ID: <20190320015236.GA35511@admin.sibptus.ru>
References: <20190319024638.GA8193@admin.sibptus.ru>
<AA997A20-4EBF-46DF-A517-FD2072682545@punkt.de>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
protocol="application/pgp-signature"; boundary="YZ5djTAD1cGYuMQK"
Content-Disposition: inline
In-Reply-To: <AA997A20-4EBF-46DF-A517-FD2072682545@punkt.de>
X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov
X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634
User-Agent: Mutt/1.11.3 (2019-02-01)
Sender: Victor Sudakov <vas@sibptus.ru>
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 20 Mar 2019 01:52:38 -0000
--YZ5djTAD1cGYuMQK
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Patrick M. Hausen wrote:
>=20
> > 1. Does ARC actually cache zfs volumes (not files/datasets)?
>=20
> Yes it does.
>=20
> > 2. If ARC does cache volumes, does this cache make sense on a hyperviso=
r,
> > because guest OSes will probably have their own disk cache anyway.
>=20
> IMHO not much, because the guest OS is relying on the fact that when
> it writes it=E2=80=99s own cached data out to =E2=80=9Edisk=E2=80=9C, it =
will be committed to
> stable storage.
This is an important point.
> > 3. Would it make sense to limit vfs.zfs.arc_max to 1/8 or even less of
> > total RAM, so that most RAM is available to guest machines?
>=20
> Yes if you build your own solution on plain FreeBSD. No if you are running
> FreeNAS which already tries to autotune the ARC size according to the
> memory committed to VMs.
>=20
> > 4. What other zfs tuning measures can you suggest for a bhyve
> > hypervisor?
>=20
> e.g.
> zfs set sync=3Dalways zfs/vm
>=20
> if zfs/vm is the dataset under which you create the ZVOLs for your emulat=
ed
> disks.
Well, bhyve already has an option for this:
The block-device-options are:
nocache Open the file with O_DIRECT.
direct Open the file using O_SYNC.
ro Force the file to be opened read-only.
I think something like=20
"-s 4:0,virtio-blk,/dev/zvol/zroot/vm/mail/disk0,direct"
would do the same?
>=20
> I=E2=80=99m using this for all my VM =E2=80=9Edisks=E2=80=9C and have add=
ed a 16 GB SLOG device
> to my spinning disk pool - seems to work great. This is on a home system.
Is SLOG also used by zfs volumes?
--=20
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/
--YZ5djTAD1cGYuMQK
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQEbBAEBAgAGBQJckZzkAAoJEA2k8lmbXsY0uukH+JC3YmtDNjPduavG8UcnK1Dy
nMPQwavDdOyKK8v37n1K11ftUbdEOaHC+IWrrszuUxc3Ui/hPG6K3FAaLtLtwllp
6p+64AJ9w9t/nIoMfIGJfmU1BMQuE0JeXcJCaohoDXvyMfCxXApdxlK3iRRYa97T
8ch7Bc+f77cYuwvJkj0eW2pSmANObTsj5kcCLDhnWX9EuBoDpE1rmhfB1R1zxJaa
5PYpNChx8tqyb+O/Usex6pnZ9B7pKzO4lf4mDB13yg6ysWYnCGhECi54yZbE146h
OMaJsLpE1DxOe3aOCGsx19o5Fi+/mXb4qm1nwWTE5Jv+/zvtlVxjla+UySB0Dg==
=dqCO
-----END PGP SIGNATURE-----
--YZ5djTAD1cGYuMQK--
From owner-freebsd-virtualization@freebsd.org Wed Mar 20 09:34:51 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 94715154D8A9
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Wed, 20 Mar 2019 09:34:51 +0000 (UTC)
(envelope-from matt.churchyard@userve.net)
Received: from smtp-a.userve.net (smtp-outbound.userve.net [217.196.1.22])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "*.userve.net", Issuer "Thawte RSA CA 2018" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 0F0208D68A
for <freebsd-virtualization@freebsd.org>; Wed, 20 Mar 2019 09:34:50 +0000 (UTC)
(envelope-from matt.churchyard@userve.net)
Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2])
(using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits))
(No client certificate requested)
by smtp-a.userve.net (Postfix) with ESMTPS id A3828428;
Wed, 20 Mar 2019 09:34:41 +0000 (GMT)
Received: from SERVER.ad.usd-group.com (192.168.0.1) by
SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id
15.0.847.32; Wed, 20 Mar 2019 09:34:41 +0000
Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by
SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id
15.00.0847.030; Wed, 20 Mar 2019 09:34:41 +0000
From: Matt Churchyard <matt.churchyard@userve.net>
To: Victor Sudakov <vas@mpeks.tomsk.su>, "freebsd-virtualization@freebsd.org"
<freebsd-virtualization@freebsd.org>
Subject: RE: bhyve and vfs.zfs.arc_max, and zfs tuning for a hypervisor
Thread-Topic: bhyve and vfs.zfs.arc_max, and zfs tuning for a hypervisor
Thread-Index: AQHU3f4eR9n6J576SkWf85jIrU+hAKYSmT6AgAEp9ACAAHlQEA==
Date: Wed, 20 Mar 2019 09:34:41 +0000
Message-ID: <6388ebd5a12949179ff6da33eb107205@SERVER.ad.usd-group.com>
References: <20190319024638.GA8193@admin.sibptus.ru>
<AA997A20-4EBF-46DF-A517-FD2072682545@punkt.de>
<20190320015236.GA35511@admin.sibptus.ru>
In-Reply-To: <20190320015236.GA35511@admin.sibptus.ru>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [192.168.0.10]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-Rspamd-Queue-Id: 0F0208D68A
X-Spamd-Bar: ------
X-Spamd-Result: default: False [-6.99 / 15.00];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
NEURAL_HAM_SHORT(-0.99)[-0.989,0]; REPLY(-4.00)[];
NEURAL_HAM_LONG(-1.00)[-1.000,0]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 20 Mar 2019 09:34:51 -0000
UGF0cmljayBNLiBIYXVzZW4gd3JvdGU6DQo+IA0KPiA+IDEuIERvZXMgQVJDIGFjdHVhbGx5IGNh
Y2hlIHpmcyB2b2x1bWVzIChub3QgZmlsZXMvZGF0YXNldHMpPw0KPiANCj4gWWVzIGl0IGRvZXMu
DQoNCkkgZmluZCB0aGlzIGRpc3RpbmN0aW9uIGJldHdlZW4gdm9sdW1lcy9maWxlcy9ldGMgYW5k
IHdoYXQgaXMgY2FjaGVkIGNhdXNlcyBjb25mdXNpb24gKGFzIHdlbGwgYXMgInZvbHVtZXMgbm90
IGRhdGFzZXRzIikuDQoNCkJvdGggWlZPTHMgYW5kIFogZmlsZSBzeXN0ZW1zIGFyZSB0eXBlcyBv
ZiBkYXRhc2V0LiBBIGRhdGFzZXQgc3RvcmVzIGRhdGEgaW4gcmVjb3JkcyAodXN1YWxseSB1cCB0
byAxMjhrYiBpbiBzaXplKS4NCkl0J3MgdGhlc2UgcmVjb3JkcyB0aGF0IGFyZSBjYWNoZWQgKGFu
ZCB0aGF0IG1vc3QgWkZTIGZ1bmN0aW9ucyBzdWNoIGFzIGNvbXByZXNzaW9uL3JhaWR6L3ppbC9l
dGMgd29yayB3aXRoKQ0KDQpBcyBmYXIgYXMgdGhlIFpGUyBsb3dlciBsZXZlbHMgYXJlIGNvbmNl
cm5lZCwgdGhlcmUgaXMgbm8gZGlmZmVyZW5jZSBiZXR3ZWVuIGEgdm9sdW1lIGFuZCBhIGZpbGUg
c3lzdGVtLg0KDQo+IA0KPiA+IDIuIElmIEFSQyBkb2VzIGNhY2hlIHZvbHVtZXMsIGRvZXMgdGhp
cyBjYWNoZSBtYWtlIHNlbnNlIG9uIGEgDQo+ID4gaHlwZXJ2aXNvciwgYmVjYXVzZSBndWVzdCBP
U2VzIHdpbGwgcHJvYmFibHkgaGF2ZSB0aGVpciBvd24gZGlzayBjYWNoZSBhbnl3YXkuDQo+IA0K
PiBJTUhPIG5vdCBtdWNoLCBiZWNhdXNlIHRoZSBndWVzdCBPUyBpcyByZWx5aW5nIG9uIHRoZSBm
YWN0IHRoYXQgd2hlbiANCj4gaXQgd3JpdGVzIGl04oCZcyBvd24gY2FjaGVkIGRhdGEgb3V0IHRv
IOKAnmRpc2vigJwsIGl0IHdpbGwgYmUgY29tbWl0dGVkIHRvIA0KPiBzdGFibGUgc3RvcmFnZS4N
Cg0KTWF5YmUgSSd2ZSBtaXNzZWQgc29tZXRoaW5nIGJ1dCBJIGRvbid0IHF1aXRlIGdldCB0aGUg
bGluayBiZXR3ZWVuIHJlYWQgY2FjaGUgKEFSQykgYW5kIGd1ZXN0IHdyaXRlcyBoZXJlPw0KDQo+
IFRoaXMgaXMgYW4gaW1wb3J0YW50IHBvaW50Lg0KDQo+ID4gMy4gV291bGQgaXQgbWFrZSBzZW5z
ZSB0byBsaW1pdCB2ZnMuemZzLmFyY19tYXggdG8gMS84IG9yIGV2ZW4gbGVzcyANCj4gPiBvZiB0
b3RhbCBSQU0sIHNvIHRoYXQgbW9zdCBSQU0gaXMgYXZhaWxhYmxlIHRvIGd1ZXN0IG1hY2hpbmVz
Pw0KPiANCj4gWWVzIGlmIHlvdSBidWlsZCB5b3VyIG93biBzb2x1dGlvbiBvbiBwbGFpbiBGcmVl
QlNELiBObyBpZiB5b3UgYXJlIA0KPiBydW5uaW5nIEZyZWVOQVMgd2hpY2ggYWxyZWFkeSB0cmll
cyB0byBhdXRvdHVuZSB0aGUgQVJDIHNpemUgYWNjb3JkaW5nIA0KPiB0byB0aGUgbWVtb3J5IGNv
bW1pdHRlZCB0byBWTXMuDQo+IA0KPiA+IDQuIFdoYXQgb3RoZXIgemZzIHR1bmluZyBtZWFzdXJl
cyBjYW4geW91IHN1Z2dlc3QgZm9yIGEgYmh5dmUgDQo+ID4gaHlwZXJ2aXNvcj8NCj4gDQo+IGUu
Zy4NCj4gCXpmcyBzZXQgc3luYz1hbHdheXMgemZzL3ZtDQo+IA0KPiBpZiB6ZnMvdm0gaXMgdGhl
IGRhdGFzZXQgdW5kZXIgd2hpY2ggeW91IGNyZWF0ZSB0aGUgWlZPTHMgZm9yIHlvdXIgDQo+IGVt
dWxhdGVkIGRpc2tzLg0KDQo+V2VsbCwgYmh5dmUgYWxyZWFkeSBoYXMgYW4gb3B0aW9uIGZvciB0
aGlzOg0KDQo+VGhlIGJsb2NrLWRldmljZS1vcHRpb25zIGFyZToNCg0KPm5vY2FjaGUgICBPcGVu
IHRoZSBmaWxlIHdpdGggT19ESVJFQ1QuDQo+ZGlyZWN0ICAgIE9wZW4gdGhlIGZpbGUgdXNpbmcg
T19TWU5DLg0KPnJvICAgICAgICBGb3JjZSB0aGUgZmlsZSB0byBiZSBvcGVuZWQgcmVhZC1vbmx5
Lg0KDQo+SSB0aGluayBzb21ldGhpbmcgbGlrZQ0KPiItcyA0OjAsdmlydGlvLWJsaywvZGV2L3p2
b2wvenJvb3Qvdm0vbWFpbC9kaXNrMCxkaXJlY3QiDQo+d291bGQgZG8gdGhlIHNhbWU/DQoNCj4g
DQo+IEnigJltIHVzaW5nIHRoaXMgZm9yIGFsbCBteSBWTSDigJ5kaXNrc+KAnCBhbmQgaGF2ZSBh
ZGRlZCBhIDE2IEdCIFNMT0cgDQo+IGRldmljZSB0byBteSBzcGlubmluZyBkaXNrIHBvb2wgLSBz
ZWVtcyB0byB3b3JrIGdyZWF0LiBUaGlzIGlzIG9uIGEgaG9tZSBzeXN0ZW0uDQoNCj4gSXMgU0xP
RyBhbHNvIHVzZWQgYnkgemZzIHZvbHVtZXM/DQoNCkFzIGFib3ZlLCB0aGUgY29yZSBvZiBaRlMg
ZG9lc24ndCBjYXJlIHdoYXQgdHlwZSBvZiBkYXRhc2V0IGl0IGlzIHdvcmtpbmcgd2l0aC4gQVJD
L1pJTC9ldGMgYWxsIHdvcmsgZXhhY3RseSB0aGUgc2FtZS4NCg0KLS0NClZpY3RvciBTdWRha292
LCAgVkFTNC1SSVBFLCBWQVM0Ny1SSVBODQoyOjUwMDUvNDlAZmlkb25ldCBodHRwOi8vdmFzLnRv
bXNrLnJ1Lw0K
From owner-freebsd-virtualization@freebsd.org Wed Mar 20 17:11:41 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9791515596FB
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Wed, 20 Mar 2019 17:11:41 +0000 (UTC)
(envelope-from freebsd@omnilan.de)
Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org
[IPv6:2001:1900:2254:206a::50:5])
by mx1.freebsd.org (Postfix) with ESMTP id E6BC26FBD0
for <freebsd-virtualization@freebsd.org>; Wed, 20 Mar 2019 17:11:40 +0000 (UTC)
(envelope-from freebsd@omnilan.de)
Received: by mailman.ysv.freebsd.org (Postfix)
id 9E29315596FA; Wed, 20 Mar 2019 17:11:40 +0000 (UTC)
Delivered-To: virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6141515596F9
for <virtualization@mailman.ysv.freebsd.org>;
Wed, 20 Mar 2019 17:11:40 +0000 (UTC)
(envelope-from freebsd@omnilan.de)
Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 222796FB03
for <virtualization@freebsd.org>; Wed, 20 Mar 2019 17:11:32 +0000 (UTC)
(envelope-from freebsd@omnilan.de)
Received: from mh0.gentlemail.de (mh0.gentlemail.de [IPv6:2a00:e10:2800::a135])
by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id x2KHBTbR013742;
Wed, 20 Mar 2019 18:11:29 +0100 (CET)
(envelope-from freebsd@omnilan.de)
Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(No client certificate requested)
by mh0.gentlemail.de (Postfix) with ESMTPSA id 60109C60;
Wed, 20 Mar 2019 18:11:29 +0100 (CET)
Subject: Re: NVMe and Bhyve
To: Victor Sudakov <vas@mpeks.tomsk.su>,
"freebsd-virtualization@freebsd.org" <virtualization@freebsd.org>
References: <20190207174453.Horde.hWBGfoLlymCpipxLv8WUJo9@app.eeeit.de>
<5121268a-1ead-9858-86a7-27f75048feb2@yuripv.net>
<20190212084333.GA46685@admin.sibptus.ru>
<CACLnyCL+RC7d+2diTKV-v08vymJ4f8irtFvv7qppUL_i1JdkSA@mail.gmail.com>
<20190212130506.GA62989@admin.sibptus.ru>
<CAOfEmZh1MKmiFEhoKtyr--J2ygDkPTJ3UTyHsDUewiCugFmpxg@mail.gmail.com>
<CACLnyC+Fs=oO3xUZD9CdCjVMw+teCG_R7a_ARVbiM8i2+YU7eg@mail.gmail.com>
<20190215044134.GA52633@admin.sibptus.ru>
<D55C1528-037F-45B7-835C-8D1D50485D0C@lassitu.de>
<20190216041134.GA75357@admin.sibptus.ru>
From: Harry Schmalzbauer <freebsd@omnilan.de>
Organization: OmniLAN
Message-ID: <21d661a4-d51d-d685-599f-c5430dea83ff@omnilan.de>
Date: Wed, 20 Mar 2019 18:11:28 +0100
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101
Thunderbird/52.7.0
MIME-Version: 1.0
In-Reply-To: <20190216041134.GA75357@admin.sibptus.ru>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7
(mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]);
Wed, 20 Mar 2019 18:11:29 +0100 (CET)
X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ;
Sender-helo: mh0.gentlemail.de; )
X-Rspamd-Queue-Id: 222796FB03
X-Spamd-Bar: ------
Authentication-Results: mx1.freebsd.org;
spf=pass (mx1.freebsd.org: domain of freebsd@omnilan.de designates
2a00:e10:2800::a130 as permitted sender) smtp.mailfrom=freebsd@omnilan.de
X-Spamd-Result: default: False [-6.77 / 15.00]; ARC_NA(0.00)[];
RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx];
NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain];
RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[omnilan.de];
HAS_ORG_HEADER(0.00)[]; RCVD_COUNT_THREE(0.00)[3];
TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[];
MX_GOOD(-0.01)[mx0.gentlemail.de]; RCPT_COUNT_TWO(0.00)[2];
NEURAL_HAM_SHORT(-0.93)[-0.934,0];
IP_SCORE(-3.53)[ip: (-9.24), ipnet: 2a00:e10:2800::/64(-4.71), asn:
25074(-3.69), country: DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[];
R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+];
ASN(0.00)[asn:25074, ipnet:2a00:e10:2800::/64, country:DE];
MID_RHS_MATCH_FROM(0.00)[]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 20 Mar 2019 17:11:41 -0000
Am 16.02.2019 um 05:11 schrieb Victor Sudakov:
> Stefan Bethke wrote:
>>> I was just worried that using ahci-hd instead of paravirtualized disks
>>> causes suboptimal performance of guests. Do you think there is a hope to get
>>> paravirtualized disks support for Windows guests in bhyve?
>> I was under the impression that the Windows VirtIO disk driver works
>> with the Bhyve virtio-blk device. You need to provide these drivers to
>> Windows somehow (for example, as a second CD image during
>> installation):
>> https://www.linux-kvm.org/page/WindowsGuestDrivers/Download_Drivers
> I use NetKVM (VirtIO network) driver from there all the time, because
> e1000 is bad.
>
> However, I've read a number of bhyve resources including
> https://wiki.freebsd.org/bhyve/Windows and I've never found a success
> story with paravirtualized Windows disk drivers.
>
> If anyone has been successful with viostor(?), please share your experience.
You'll need
https://github.com/freenas/os/commit/0e4d6e1826f8aa7041cbeeb4365c797eeec5c5f4
if I remember correctly.
I asked for commit, since I'm using this in production for a long time,
but once upon a time, there has been a linux-guest specific side effect.
Don't know much about it and haven't had a chance to reproduce it.
In my bhyve world, any guest runs fine with the higher BLOCKIF_IOV limit.
-harry
From owner-freebsd-virtualization@freebsd.org Wed Mar 20 23:30:13 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id D9CFB153E6CB
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Wed, 20 Mar 2019 23:30:13 +0000 (UTC) (envelope-from hausen@punkt.de)
Received: from kagate.punkt.de (kagate.punkt.de [217.29.33.131])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 5C01086739
for <freebsd-virtualization@freebsd.org>; Wed, 20 Mar 2019 23:30:12 +0000 (UTC)
(envelope-from hausen@punkt.de)
Received: from hugo10.ka.punkt.de (hugo10.ka.punkt.de [217.29.44.10])
by gate2.intern.punkt.de with ESMTP id x2KNUAMP042638;
Thu, 21 Mar 2019 00:30:10 +0100 (CET)
Received: from [217.29.46.66] (unassigned [217.29.46.66] (may be forged))
by hugo10.ka.punkt.de (8.14.2/8.14.2) with ESMTP id x2KNUA9p063833;
Thu, 21 Mar 2019 00:30:10 +0100 (CET) (envelope-from hausen@punkt.de)
Content-Type: text/plain;
charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\))
Subject: Re: bhyve and vfs.zfs.arc_max, and zfs tuning for a hypervisor
From: "Patrick M. Hausen" <hausen@punkt.de>
In-Reply-To: <20190320015236.GA35511@admin.sibptus.ru>
Date: Thu, 21 Mar 2019 00:30:09 +0100
Cc: freebsd-virtualization@freebsd.org
Content-Transfer-Encoding: quoted-printable
Message-Id: <0520BA4F-FA4A-4BE9-85D9-2CECF2018282@punkt.de>
References: <20190319024638.GA8193@admin.sibptus.ru>
<AA997A20-4EBF-46DF-A517-FD2072682545@punkt.de>
<20190320015236.GA35511@admin.sibptus.ru>
To: Victor Sudakov <vas@mpeks.tomsk.su>
X-Mailer: Apple Mail (2.3445.102.3)
X-Rspamd-Queue-Id: 5C01086739
X-Spamd-Bar: ------
Authentication-Results: mx1.freebsd.org
X-Spamd-Result: default: False [-6.99 / 15.00];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
NEURAL_HAM_SHORT(-0.99)[-0.989,0]; REPLY(-4.00)[];
NEURAL_HAM_LONG(-1.00)[-1.000,0]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 20 Mar 2019 23:30:14 -0000
Hi!
> Am 20.03.2019 um 02:52 schrieb Victor Sudakov <vas@mpeks.tomsk.su>:
> Is SLOG also used by zfs volumes?
Yes, but for synchronous writes only, if I=E2=80=99m not mistaken.
So fundamentally yes, but in most cases no.
Patrick
--=20
punkt.de GmbH Internet - Dienstleistungen - Beratung
Kaiserallee 13a Tel.: 0721 9109-0 Fax: -100
76133 Karlsruhe info@punkt.de http://punkt.de
AG Mannheim 108285 Gf: Juergen Egeling
From owner-freebsd-virtualization@freebsd.org Wed Mar 20 23:30:49 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id E464F153E881
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Wed, 20 Mar 2019 23:30:48 +0000 (UTC) (envelope-from hausen@punkt.de)
Received: from kagate.punkt.de (kagate.punkt.de [217.29.33.131])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 81C00867B6
for <freebsd-virtualization@freebsd.org>; Wed, 20 Mar 2019 23:30:47 +0000 (UTC)
(envelope-from hausen@punkt.de)
Received: from hugo10.ka.punkt.de (hugo10.ka.punkt.de [217.29.44.10])
by gate2.intern.punkt.de with ESMTP id x2KNUkxh042658;
Thu, 21 Mar 2019 00:30:46 +0100 (CET)
Received: from [217.29.46.66] (unassigned [217.29.46.66] (may be forged))
by hugo10.ka.punkt.de (8.14.2/8.14.2) with ESMTP id x2KNUA9q063833;
Thu, 21 Mar 2019 00:30:46 +0100 (CET) (envelope-from hausen@punkt.de)
Content-Type: text/plain;
charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\))
Subject: Re: bhyve and vfs.zfs.arc_max, and zfs tuning for a hypervisor
From: "Patrick M. Hausen" <hausen@punkt.de>
In-Reply-To: <6388ebd5a12949179ff6da33eb107205@SERVER.ad.usd-group.com>
Date: Thu, 21 Mar 2019 00:30:45 +0100
Cc: Victor Sudakov <vas@mpeks.tomsk.su>,
"freebsd-virtualization@freebsd.org" <freebsd-virtualization@freebsd.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <B02887DA-D1B8-48A8-8F4B-C02A02F0726A@punkt.de>
References: <20190319024638.GA8193@admin.sibptus.ru>
<AA997A20-4EBF-46DF-A517-FD2072682545@punkt.de>
<20190320015236.GA35511@admin.sibptus.ru>
<6388ebd5a12949179ff6da33eb107205@SERVER.ad.usd-group.com>
To: Matt Churchyard <matt.churchyard@userve.net>
X-Mailer: Apple Mail (2.3445.102.3)
X-Rspamd-Queue-Id: 81C00867B6
X-Spamd-Bar: ---
Authentication-Results: mx1.freebsd.org;
spf=pass (mx1.freebsd.org: domain of hausen@punkt.de designates 217.29.33.131
as permitted sender) smtp.mailfrom=hausen@punkt.de
X-Spamd-Result: default: False [-3.15 / 15.00]; ARC_NA(0.00)[];
TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0];
FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3];
R_SPF_ALLOW(-0.20)[+ip4:217.29.32.0/20]; MV_CASE(0.50)[];
MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[];
DMARC_NA(0.00)[punkt.de]; TO_DN_SOME(0.00)[];
NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[3];
TO_MATCH_ENVRCPT_SOME(0.00)[];
MX_GOOD(-0.01)[cached: mailin.pluspunkthosting.de];
NEURAL_HAM_SHORT(-0.96)[-0.957,0];
RCVD_IN_DNSWL_NONE(0.00)[131.33.29.217.list.dnswl.org : 127.0.10.0];
IP_SCORE(-0.38)[ipnet: 217.29.32.0/20(-1.06), asn: 16188(-0.85), country:
DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[];
MIME_TRACE(0.00)[0:+];
ASN(0.00)[asn:16188, ipnet:217.29.32.0/20, country:DE];
MID_RHS_MATCH_FROM(0.00)[]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 20 Mar 2019 23:30:49 -0000
Hi Matt,
> Am 20.03.2019 um 10:34 schrieb Matt Churchyard =
<matt.churchyard@userve.net>:
>>> 2. If ARC does cache volumes, does this cache make sense on a=20
>>> hypervisor, because guest OSes will probably have their own disk =
cache anyway.
>>=20
>> IMHO not much, because the guest OS is relying on the fact that when=20=
>> it writes it=E2=80=99s own cached data out to =E2=80=9Edisk=E2=80=9C, =
it will be committed to=20
>> stable storage.
>=20
> Maybe I've missed something but I don't quite get the link between =
read cache (ARC) and guest writes here?
You are correct - I confused ARC and ZIL. I still recommend to
set sync=3Dalways for hypervisor =E2=80=9Edisk=E2=80=9C ZVOLs.
Kind regards
Patrick
--=20
punkt.de GmbH Internet - Dienstleistungen - Beratung
Kaiserallee 13a Tel.: 0721 9109-0 Fax: -100
76133 Karlsruhe info@punkt.de http://punkt.de
AG Mannheim 108285 Gf: Juergen Egeling
From owner-freebsd-virtualization@freebsd.org Thu Mar 21 01:47:44 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0FEF81546522
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Thu, 21 Mar 2019 01:47:44 +0000 (UTC)
(envelope-from mike.gerdts@joyent.com)
Received: from mail-pf1-x42e.google.com (mail-pf1-x42e.google.com
[IPv6:2607:f8b0:4864:20::42e])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 0F8328CD49
for <freebsd-virtualization@freebsd.org>; Thu, 21 Mar 2019 01:47:41 +0000 (UTC)
(envelope-from mike.gerdts@joyent.com)
Received: by mail-pf1-x42e.google.com with SMTP id 8so3235327pfr.4
for <freebsd-virtualization@freebsd.org>; Wed, 20 Mar 2019 18:47:41 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joyent.com; s=google;
h=mime-version:references:in-reply-to:from:date:message-id:subject:to
:cc; bh=K2z6GqCzoNkhrL9ZT2QTRKiHADBD+R2KKvTNGe9bPCs=;
b=M84TeUV1V8hpC/Nv7BmSFxkrmLW16pDFVBHoFyNoQCUfRQ7ew2oxWHC8IBJmGW/MFp
ZX3+62kFWKSyqqwzRLctVyTqHf4wIS+AVTFDs1fiaVYeXfXDVb7vWH0/2NL9CJxCkBz5
DaIUVRFAKBUmqNtJ1kB9YW7jZkwMMGwllVXmw=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:references:in-reply-to:from:date
:message-id:subject:to:cc;
bh=K2z6GqCzoNkhrL9ZT2QTRKiHADBD+R2KKvTNGe9bPCs=;
b=CZf2iuuIWPQaYiWARrpI2oQVbphoiqJs37tFeMRvMihqO2gEFdH661F0q/h15HrFJU
QNZKj8x8FwOhL8b0hLIv3uAeeSWgm3iip96ZL0/4j7vCUqgFzRbgAtVqgzS5jNfPJV4X
zeDuXfjB71iTeh2umo4+JFT0rttdjC+5kB3L6ZLAXPSf8nBF5hDcTeI4QNvJLNK8Bn5p
dYKHOVSL0wA9Y4wLKn9RijL4Ma2qSdT2F43p8zHwbfHgpb84B/OQ7dbg/ArdbZ21n4tf
Bbl/Nr+uTTTRpXpykRcwlT3mKQNpE1QTI/ba8z+aiwFd1MgePe41HCDYICzuqx9uizu3
IZEg==
X-Gm-Message-State: APjAAAUnYyvrdI+i+/Ib9w9/eQOgxsvD/2qevHvEXSaiolnT0OuyEEk3
NyQVdXhxTU311EViSePLjcGeFfrEf0Af3tFsRlWRYHIQKEE=
X-Google-Smtp-Source: APXvYqyj1XoFf0aXdNGgy/dYHzAe4gyE+MRxMt2OiDq00JhgxzoDiaWY78hB6kPJ5VGHEKY4gUatjR90IwIYJLVG+bo=
X-Received: by 2002:a65:50c4:: with SMTP id s4mr1008041pgp.33.1553132860604;
Wed, 20 Mar 2019 18:47:40 -0700 (PDT)
MIME-Version: 1.0
References: <20190319024638.GA8193@admin.sibptus.ru>
<AA997A20-4EBF-46DF-A517-FD2072682545@punkt.de>
In-Reply-To: <AA997A20-4EBF-46DF-A517-FD2072682545@punkt.de>
From: Mike Gerdts <mike.gerdts@joyent.com>
Date: Wed, 20 Mar 2019 20:47:29 -0500
Message-ID: <CAAxFYUF_BrdDz=uDNTwnJWH4tKmMfiQuS3XowYtQ68cQy96fxg@mail.gmail.com>
Subject: Re: bhyve and vfs.zfs.arc_max, and zfs tuning for a hypervisor
To: "Patrick M. Hausen" <hausen@punkt.de>
Cc: Victor Sudakov <vas@mpeks.tomsk.su>, freebsd-virtualization@freebsd.org
X-Rspamd-Queue-Id: 0F8328CD49
X-Spamd-Bar: ------
Authentication-Results: mx1.freebsd.org;
dkim=pass header.d=joyent.com header.s=google header.b=M84TeUV1;
spf=pass (mx1.freebsd.org: domain of mike.gerdts@joyent.com designates
2607:f8b0:4864:20::42e as permitted sender)
smtp.mailfrom=mike.gerdts@joyent.com
X-Spamd-Result: default: False [-6.30 / 15.00]; ARC_NA(0.00)[];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
R_DKIM_ALLOW(-0.20)[joyent.com:s=google]; FROM_HAS_DN(0.00)[];
RCPT_COUNT_THREE(0.00)[3];
R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36];
NEURAL_HAM_LONG(-1.00)[-1.000,0];
MIME_GOOD(-0.10)[multipart/alternative,text/plain];
PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org];
DMARC_NA(0.00)[joyent.com]; TO_DN_SOME(0.00)[];
TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[joyent.com:+];
MX_GOOD(-0.01)[alt1.aspmx.l.google.com,aspmx.l.google.com,aspmx2.googlemail.com,alt2.aspmx.l.google.com,aspmx3.googlemail.com];
RCVD_IN_DNSWL_NONE(0.00)[e.2.4.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org
: 127.0.5.0]; NEURAL_HAM_SHORT(-0.91)[-0.907,0];
IP_SCORE(-2.88)[ip: (-9.40), ipnet: 2607:f8b0::/32(-2.81), asn: 15169(-2.12),
country: US(-0.07)]; FROM_EQ_ENVFROM(0.00)[];
MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[];
ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US];
RCVD_COUNT_TWO(0.00)[2]
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 21 Mar 2019 01:47:44 -0000
On Tue, Mar 19, 2019 at 3:07 AM Patrick M. Hausen <hausen@punkt.de> wrote:
> Hi!
>
> > Am 19.03.2019 um 03:46 schrieb Victor Sudakov <vas@mpeks.tomsk.su>:
> > 1. Does ARC actually cache zfs volumes (not files/datasets)?
>
> Yes it does.
>
> > 2. If ARC does cache volumes, does this cache make sense on a hyperviso=
r,
> > because guest OSes will probably have their own disk cache anyway.
>
> IMHO not much, because the guest OS is relying on the fact that when
> it writes it=E2=80=99s own cached data out to =E2=80=9Edisk=E2=80=9C, it =
will be committed to
> stable storage.
>
I'd recommend caching at least metadata (primarycache=3Dmetadata). The gue=
st
will not cache zfs metadata and not having metadata in the cache can lead
to a big hit in performance. The metadata in question here are things like
block pointers that keep track of where the data is at - zfs can't find the
data without metadata.
I think the key decision as to whether you use primarycache=3Dmetadata or
primarycache=3Dall comes down to whether you are after predictable
performance or optimal performance. You will likely get worse performance
with primarycache=3Dmetaadata (or especially with primarycache=3Dnone),
presuming the host has RAM to spare. As you pack the system with more VMs
or allocate more disk to existing VMs, you will probably find that
primarycache=3Dmetadata leads steadier performance regardless of how much
storage is used or how active other VMs are.
> > 3. Would it make sense to limit vfs.zfs.arc_max to 1/8 or even less of
> > total RAM, so that most RAM is available to guest machines?
>
> Yes if you build your own solution on plain FreeBSD. No if you are runnin=
g
> FreeNAS which already tries to autotune the ARC size according to the
> memory committed to VMs.
>
> > 4. What other zfs tuning measures can you suggest for a bhyve
> > hypervisor?
>
> e.g.
> zfs set sync=3Dalways zfs/vm
>
> if zfs/vm is the dataset under which you create the ZVOLs for your emulat=
ed
> disks.
>
I'm not sure what the state of this is in FreeBSD, but in SmartOS we allow
the guests to benefit from write caching if they negotiate FLUSH. Guests
that do negotiate flush are expected to use proper barriers to flush the
cache at critical times. When a FLUSH arrives, SmartOS bhyve issues an
fsync(). To be clear - SmartOS bhyve is not actually caching writes in
memory, it is just delaying transaction group commits. This avoids
significant write inflation and associated latency. Support for FLUSH
negotiation has greatly improved I/O performance - to the point that some
tests show parity with running directly on the host pool. If not already
in FreeBSD, this would probably be something of relatively high value to
pull in.
If you do go the route of sync=3Dalways and primarycache=3D<none|metadata>,=
be
sure your guest block size and host volblocksize match. ZFS (on platforms
I'm more familiar with, at least) defaults to volblocksize=3D8k. Most gues=
t
file systems these days seem to default to a block size of 4 KiB. If the
guest file system issues a 4 KiB aligned write, that will turn into a
read-modify-write cycle to stitch that 4 KiB guest block into the host's 8
KB block. If the adjacent guest block that is in the same 8 KiB host block
is written in the next write, it will also turn into a read-modify-write
cycle.
If you are using ZFS in the guest, this can be particularly problematic
because the guest ZFS will align writes with the guest pool's ashift, not
with a guest dataset's recordsize or volblocksize. I discovered this in an
extended benchmarking of zfs-on-zfs - primarily with primarycache=3Dmetadat=
a
and sync=3Dalways. The write inflation was quite significant: 3x was
common. I tracked some of this down to alignment issues and part of it was
due to sync writes causing the data to be written twice.
George Wilson has a great talk where he describes the same issues I hit.
https://www.youtube.com/watch?v=3D_-QAnKtIbGc
I've mentioned write inflation related to sync writes a few times. One
point that I think is poorly understood is that when ZFS is rushed into
committing a write with fsync or similar, the immediate write is of ZIL
blocks to the intent log. The intent log can be on a separate disk (slog,
log=3D<device>) or it can be on the disks that hold the pool's data. When
the intent log is on the data disks, the data is written to the same disks
multiple times: once as ZIL blocks and once as data blocks. Between these
writes there will be full-disk head movement as the uberblocks are updated
at the beginning and end of the disk.
What I say above is based on experience with kernel zones on Solaris and
bhyve on SmartOS. There are enough similarities that I expect bhyve on
FreeBSD will be the same, but FreeBSD may have some strange-to-me zfs
caching changes.
Regards,
Mike
From owner-freebsd-virtualization@freebsd.org Thu Mar 21 02:38:35 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1F9C71548CDF
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Thu, 21 Mar 2019 02:38:35 +0000 (UTC)
(envelope-from vas@mpeks.tomsk.su)
Received: from admin.sibptus.ru (admin.sibptus.ru
[IPv6:2001:19f0:5001:21dc::10])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 1E8DB8EC50
for <freebsd-virtualization@freebsd.org>; Thu, 21 Mar 2019 02:38:33 +0000 (UTC)
(envelope-from vas@mpeks.tomsk.su)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sibptus.ru;
s=20181118; h=In-Reply-To:Message-ID:Subject:To:From:Date;
bh=/S9FwYoBtoHYmMEBCz1R5wze8Meiy0OPtz2C5WwI4es=; b=D5LQIGKdzB3xMjnLrAfjvwJqK9
vx8LxARzFVUh+jzlM/btQZIACPTcupsv0e55c/lXmpCT1rxM8luoUnigfdai7X9zjqCSpGJ/hjHvP
9YoL4ie4bQsqAENeufAshWCznRH+GOfxQKz/L71qnNIVZ6d+ILT7BOyQkOrGHys5iDJ8=;
Received: from vas by admin.sibptus.ru with local (Exim 4.92 (FreeBSD))
(envelope-from <vas@mpeks.tomsk.su>) id 1h6nbD-000F4C-W5
for freebsd-virtualization@freebsd.org; Thu, 21 Mar 2019 09:38:31 +0700
Date: Thu, 21 Mar 2019 09:38:31 +0700
From: Victor Sudakov <vas@mpeks.tomsk.su>
To: "freebsd-virtualization@freebsd.org" <freebsd-virtualization@freebsd.org>
Subject: Re: bhyve and vfs.zfs.arc_max, and zfs tuning for a hypervisor
Message-ID: <20190321023831.GA57807@admin.sibptus.ru>
References: <20190319024638.GA8193@admin.sibptus.ru>
<AA997A20-4EBF-46DF-A517-FD2072682545@punkt.de>
<20190320015236.GA35511@admin.sibptus.ru>
<6388ebd5a12949179ff6da33eb107205@SERVER.ad.usd-group.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
protocol="application/pgp-signature"; boundary="bg08WKrSYDhXBjb5"
Content-Disposition: inline
In-Reply-To: <6388ebd5a12949179ff6da33eb107205@SERVER.ad.usd-group.com>
X-PGP-Key: http://www.dreamwidth.org/pubkey?user=victor_sudakov
X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634
User-Agent: Mutt/1.11.4 (2019-03-13)
Sender: Victor Sudakov <vas@sibptus.ru>
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 21 Mar 2019 02:38:35 -0000
--bg08WKrSYDhXBjb5
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Matt Churchyard wrote:
> >=20
> > > 1. Does ARC actually cache zfs volumes (not files/datasets)?
> >=20
> > Yes it does.
>=20
> I find this distinction between volumes/files/etc and what is cached
> causes confusion (as well as "volumes not datasets").
>=20
> Both ZVOLs and Z file systems are types of dataset. A dataset stores data
> in records (usually up to 128kb in size). It's these records that are
> cached (and that most ZFS functions such as compression/raidz/zil/etc
> work with)
>=20
> As far as the ZFS lower levels are concerned, there is no difference
> between a volume and a file system.
Thank you Matt, this was very instructive.
> > > 2. If ARC does cache volumes, does this cache make sense on a=20
> > > hypervisor, because guest OSes will probably have their own disk cach=
e anyway.
> >=20
> > IMHO not much, because the guest OS is relying on the fact that when=20
> > it writes it=E2=80=99s own cached data out to =E2=80=9Edisk=E2=80=9C, i=
t will be committed to=20
> > stable storage.
>=20
> Maybe I've missed something but I don't quite get the link between
> read cache (ARC) and guest writes here?
Maybe there was a confusion between read and write caches, but my
question still stands:
Does it make sense to cache the same data (for reading too) twice: one time
in host's RAM (ZFS ARC) and the other time in guest's RAM (whatever fs the
guest uses, all modern OSes have disk caches)?
What do VMWare or VirtualBox do for this situation? Do they ever cache
their volumes in the hypervisor's RAM?
--=20
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
2:5005/49@fidonet http://vas.tomsk.ru/
--bg08WKrSYDhXBjb5
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQEcBAEBAgAGBQJckvknAAoJEA2k8lmbXsY05lYIAJQUH3L5cyzR/TEuzoHE1esh
LqSJqNDKDAJllrlKZ9NNPe5g1ylLiEEtc0WwrvesrWecIX20cJqJaamz2uEVk/iC
EUWaCKkOEN8LvOoVMMY8GvJ5r0DezdfDyKp01tSNTZY/HlsF63u5UNyNlHXDGYeU
pSCZhl+8qWWoZxk51G3znGU03QKPD6R9BFBtp6Yukp/cGb3E95jOb8wmX2z/3CJQ
bsY2SUlcGdeRnM+bifDBQaH9oCktcYWqpejEEUNLKx5Gmvz1KuG0X1yrH5mtqanY
3EFm9EPsqvn/QWnAxLqEwBmdoZokLtYKXTUPV16LRLrL8Jy19gklACBNEevc+Yo=
=9k/W
-----END PGP SIGNATURE-----
--bg08WKrSYDhXBjb5--
From owner-freebsd-virtualization@freebsd.org Thu Mar 21 10:24:23 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2EC3C155B48C
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Thu, 21 Mar 2019 10:24:23 +0000 (UTC)
(envelope-from matt.churchyard@userve.net)
Received: from smtp-a.userve.net (smtp-outbound.userve.net [217.196.1.22])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "*.userve.net", Issuer "Thawte RSA CA 2018" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id BE15A84195
for <freebsd-virtualization@freebsd.org>; Thu, 21 Mar 2019 10:24:22 +0000 (UTC)
(envelope-from matt.churchyard@userve.net)
Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2])
(using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits))
(No client certificate requested)
by smtp-a.userve.net (Postfix) with ESMTPS id A006E6C4;
Thu, 21 Mar 2019 10:24:14 +0000 (GMT)
Received: from SERVER.ad.usd-group.com (192.168.0.1) by
SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id
15.0.847.32; Thu, 21 Mar 2019 10:24:14 +0000
Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by
SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id
15.00.0847.030; Thu, 21 Mar 2019 10:24:14 +0000
From: Matt Churchyard <matt.churchyard@userve.net>
To: Victor Sudakov <vas@mpeks.tomsk.su>, "freebsd-virtualization@freebsd.org"
<freebsd-virtualization@freebsd.org>
Subject: RE: bhyve and vfs.zfs.arc_max, and zfs tuning for a hypervisor
Thread-Topic: bhyve and vfs.zfs.arc_max, and zfs tuning for a hypervisor
Thread-Index: AQHU3f4eR9n6J576SkWf85jIrU+hAKYSmT6AgAEp9ACAAHlQEIABJdmAgAB9PfA=
Date: Thu, 21 Mar 2019 10:24:13 +0000
Message-ID: <99457cf7758b4f7ebacafce5f40af220@SERVER.ad.usd-group.com>
References: <20190319024638.GA8193@admin.sibptus.ru>
<AA997A20-4EBF-46DF-A517-FD2072682545@punkt.de>
<20190320015236.GA35511@admin.sibptus.ru>
<6388ebd5a12949179ff6da33eb107205@SERVER.ad.usd-group.com>
<20190321023831.GA57807@admin.sibptus.ru>
In-Reply-To: <20190321023831.GA57807@admin.sibptus.ru>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [192.168.0.10]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-Rspamd-Queue-Id: BE15A84195
X-Spamd-Bar: ------
X-Spamd-Result: default: False [-6.98 / 15.00];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
NEURAL_HAM_SHORT(-0.98)[-0.975,0]; REPLY(-4.00)[];
NEURAL_HAM_LONG(-1.00)[-1.000,0]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 21 Mar 2019 10:24:23 -0000
PiA+IA0KPiA+ID4gMS4gRG9lcyBBUkMgYWN0dWFsbHkgY2FjaGUgemZzIHZvbHVtZXMgKG5vdCBm
aWxlcy9kYXRhc2V0cyk/DQo+ID4gDQo+ID4gWWVzIGl0IGRvZXMuDQo+IA0KPiBJIGZpbmQgdGhp
cyBkaXN0aW5jdGlvbiBiZXR3ZWVuIHZvbHVtZXMvZmlsZXMvZXRjIGFuZCB3aGF0IGlzIGNhY2hl
ZCANCj4gY2F1c2VzIGNvbmZ1c2lvbiAoYXMgd2VsbCBhcyAidm9sdW1lcyBub3QgZGF0YXNldHMi
KS4NCj4gDQo+IEJvdGggWlZPTHMgYW5kIFogZmlsZSBzeXN0ZW1zIGFyZSB0eXBlcyBvZiBkYXRh
c2V0LiBBIGRhdGFzZXQgc3RvcmVzIA0KPiBkYXRhIGluIHJlY29yZHMgKHVzdWFsbHkgdXAgdG8g
MTI4a2IgaW4gc2l6ZSkuICBJdCdzIHRoZXNlIHJlY29yZHMgDQo+IHRoYXQgYXJlIGNhY2hlZCAo
YW5kIHRoYXQgbW9zdCBaRlMgZnVuY3Rpb25zIHN1Y2ggYXMgDQo+IGNvbXByZXNzaW9uL3JhaWR6
L3ppbC9ldGMgd29yayB3aXRoKQ0KPiANCj4gQXMgZmFyIGFzIHRoZSBaRlMgbG93ZXIgbGV2ZWxz
IGFyZSBjb25jZXJuZWQsIHRoZXJlIGlzIG5vIGRpZmZlcmVuY2UgDQo+IGJldHdlZW4gYSB2b2x1
bWUgYW5kIGEgZmlsZSBzeXN0ZW0uDQoNCj5UaGFuayB5b3UgTWF0dCwgdGhpcyB3YXMgdmVyeSBp
bnN0cnVjdGl2ZS4NCg0KPiA+ID4gMi4gSWYgQVJDIGRvZXMgY2FjaGUgdm9sdW1lcywgZG9lcyB0
aGlzIGNhY2hlIG1ha2Ugc2Vuc2Ugb24gYSANCj4gPiA+IGh5cGVydmlzb3IsIGJlY2F1c2UgZ3Vl
c3QgT1NlcyB3aWxsIHByb2JhYmx5IGhhdmUgdGhlaXIgb3duIGRpc2sgY2FjaGUgYW55d2F5Lg0K
PiA+IA0KPiA+IElNSE8gbm90IG11Y2gsIGJlY2F1c2UgdGhlIGd1ZXN0IE9TIGlzIHJlbHlpbmcg
b24gdGhlIGZhY3QgdGhhdCB3aGVuIA0KPiA+IGl0IHdyaXRlcyBpdOKAmXMgb3duIGNhY2hlZCBk
YXRhIG91dCB0byDigJ5kaXNr4oCcLCBpdCB3aWxsIGJlIGNvbW1pdHRlZCANCj4gPiB0byBzdGFi
bGUgc3RvcmFnZS4NCj4gDQo+IE1heWJlIEkndmUgbWlzc2VkIHNvbWV0aGluZyBidXQgSSBkb24n
dCBxdWl0ZSBnZXQgdGhlIGxpbmsgYmV0d2VlbiANCj4gcmVhZCBjYWNoZSAoQVJDKSBhbmQgZ3Vl
c3Qgd3JpdGVzIGhlcmU/DQoNCj5NYXliZSB0aGVyZSB3YXMgYSBjb25mdXNpb24gYmV0d2VlbiBy
ZWFkIGFuZCB3cml0ZSBjYWNoZXMsIGJ1dCBteSBxdWVzdGlvbiBzdGlsbCBzdGFuZHM6DQoNCj5E
b2VzIGl0IG1ha2Ugc2Vuc2UgdG8gY2FjaGUgdGhlIHNhbWUgZGF0YSAoZm9yIHJlYWRpbmcgdG9v
KSB0d2ljZTogb25lIHRpbWUgaW4gaG9zdCdzIFJBTSAoWkZTIEFSQykgYW5kIHRoZSBvdGhlciB0
aW1lIGluIGd1ZXN0J3MgUkFNICh3aGF0ZXZlciBmcyB0aGUgZ3Vlc3QgdXNlcywgYWxsIG1vZGVy
biBPU2VzIGhhdmUgZGlzayBjYWNoZXMpPw0KDQo+V2hhdCBkbyBWTVdhcmUgb3IgVmlydHVhbEJv
eCBkbyBmb3IgdGhpcyBzaXR1YXRpb24/IERvIHRoZXkgZXZlciBjYWNoZSB0aGVpciB2b2x1bWVz
IGluIHRoZSBoeXBlcnZpc29yJ3MgUkFNPw0KDQpWaXJ0dWFsYm94IHdvdWxkIGJlIG5vIGRpZmZl
cmVudCB0byBiaHl2ZSBpbiB0aGF0IGl0IGRvZXNuJ3QgY2FyZSB3aGF0IHN0b3JhZ2Ugc3lzdGVt
IHlvdSBhcmUgdXNpbmcgb3IgaG93IGl0IGlzIGNvbmZpZ3VyZWQsIHRoYXQncyB1cCB0byB0aGUg
c3lzdGVtIGFkbWluLg0KSSBiZWxpZXZlIFZNRlMgaXMgbW9yZSBha2luIHRvIG90aGVyICJ0cmFk
aXRpb25hbCIgZmlsZSBzeXN0ZW1zLCBhbmQgZG9lc24ndCBkbyBSQU0gY2FjaGluZyB0byBhbnl3
aGVyZSBuZWFyIHRoZSBleHRlbnQgb2YgWkZTLiBJIGRvIHRoaW5rIHlvdSBjYW4gdXNlIFNTRC9O
Vk1lL2V0YyBhcyBjYWNoZSBpbiBWTVdhcmUuDQoNCk15IGluaXRpYWwgaW5zdGluY3Qgd291bGQg
YmUgdG8ga2VlcCBjYWNoZSBvbiwgYnV0IHJlZHVjZSB0aGUgbGltaXQgdG8gYWxsb2NhdGUgdGhl
IG1ham9yaXR5IG9mIHRoZSBSQU0gZm9yIGd1ZXN0cy4gKEknZCBzdGlsbCB3YW50IGF0IGxlYXN0
IDRHQiBhcyBhbiBhYnNvbHV0ZSBtaW5pbXVtIHRob3VnaCwgcHJvYmFibHkgbW9yZSBvbiBzeXN0
ZW1zIHdpdGggMTAwR0IrIHRvdGFsKS4gT2YgY291cnNlIHlvdSBjb3VsZCBwcm9iYWJseSB0ZXN0
IHdpdGggY2FjaGUgc2V0IHRvIGFsbC9tZXRhZGF0YSBhbmQgc2VlIHdoYXQgZWZmZWN0IGl0IGhh
cy4gQWRkaW5nIEwyQVJDIG1heSBiZSB1c2VmdWwgaWYgdGhlIG1haW4gcG9vbCBpcyBzcGlubmlu
ZyBkaXNrcywgYnV0IHRoZW4gSSd2ZSBoZWFyZCB0aGVyZSdzIGEgcnVsZSBvZiB0aHVtYiBmb3Ig
cmVxdWlyaW5nIFggYW1vdW50IG9mIEFSQyBmb3IgWSBhbW91bnQgb2YgTDJBUkMsIGJ1dCBJJ20g
bm90IHN1cmUgd2hhdCB0aGF0IHJ1bGUgaXMuDQoNCkknZCBhbHNvIGJlIGludHJpZ3VlZCB0byBr
bm93IHdoYXQgdGhlIGxvZ2ljIGluIEZyZWVOQVMgaXMgZm9yIGl0LiBJdCBpcyBzaW1wbHkgYSBj
YXNlIG9mICIoYXJjID0gdG90YWxfcmFtIC0gZ3Vlc3RfYWxsb2NhdGVkKSI/DQpJcyB0aGVyZSBh
IGxvd2VyIGxpbWl0IGJhc2VkIG9uIGEgcGVyY2VudGFnZSBvciB0b3RhbCBSQU0sIGFuZC9vciBh
IGhhcmQgbG93ZXIgbGltaXQ/DQoNCk1hdHQNCg0K
From owner-freebsd-virtualization@freebsd.org Thu Mar 21 13:10:34 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 984F2153FA69
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Thu, 21 Mar 2019 13:10:34 +0000 (UTC) (envelope-from hausen@punkt.de)
Received: from kagate.punkt.de (kagate.punkt.de [217.29.33.131])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 2F6328A806
for <freebsd-virtualization@freebsd.org>; Thu, 21 Mar 2019 13:10:27 +0000 (UTC)
(envelope-from hausen@punkt.de)
Received: from hugo10.ka.punkt.de (hugo10.ka.punkt.de [217.29.44.10])
by gate2.intern.punkt.de with ESMTP id x2LDAIIn053336;
Thu, 21 Mar 2019 14:10:18 +0100 (CET)
Received: from [217.29.44.158] ([217.29.44.158])
by hugo10.ka.punkt.de (8.14.2/8.14.2) with ESMTP id x2LDAHrF089852;
Thu, 21 Mar 2019 14:10:18 +0100 (CET) (envelope-from hausen@punkt.de)
Content-Type: text/plain;
charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\))
Subject: Re: bhyve and vfs.zfs.arc_max, and zfs tuning for a hypervisor
From: "Patrick M. Hausen" <hausen@punkt.de>
In-Reply-To: <99457cf7758b4f7ebacafce5f40af220@SERVER.ad.usd-group.com>
Date: Thu, 21 Mar 2019 14:10:17 +0100
Cc: Victor Sudakov <vas@mpeks.tomsk.su>,
"freebsd-virtualization@freebsd.org" <freebsd-virtualization@freebsd.org>
Content-Transfer-Encoding: quoted-printable
Message-Id: <758301FE-77BC-4D98-AF13-CD2A7E2D276A@punkt.de>
References: <20190319024638.GA8193@admin.sibptus.ru>
<AA997A20-4EBF-46DF-A517-FD2072682545@punkt.de>
<20190320015236.GA35511@admin.sibptus.ru>
<6388ebd5a12949179ff6da33eb107205@SERVER.ad.usd-group.com>
<20190321023831.GA57807@admin.sibptus.ru>
<99457cf7758b4f7ebacafce5f40af220@SERVER.ad.usd-group.com>
To: Matt Churchyard <matt.churchyard@userve.net>
X-Mailer: Apple Mail (2.3445.102.3)
X-Rspamd-Queue-Id: 2F6328A806
X-Spamd-Bar: --
Authentication-Results: mx1.freebsd.org;
spf=pass (mx1.freebsd.org: domain of hausen@punkt.de designates 217.29.33.131
as permitted sender) smtp.mailfrom=hausen@punkt.de
X-Spamd-Result: default: False [-2.93 / 15.00]; ARC_NA(0.00)[];
TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.994,0];
FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3];
R_SPF_ALLOW(-0.20)[+ip4:217.29.32.0/20]; MV_CASE(0.50)[];
MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[];
DMARC_NA(0.00)[punkt.de]; TO_DN_SOME(0.00)[];
NEURAL_HAM_LONG(-1.00)[-0.999,0]; RCVD_COUNT_THREE(0.00)[3];
TO_MATCH_ENVRCPT_SOME(0.00)[];
MX_GOOD(-0.01)[mailin.pluspunkthosting.de,mailin.pluspunkthosting.de];
NEURAL_HAM_SHORT(-0.76)[-0.761,0];
RCVD_IN_DNSWL_NONE(0.00)[131.33.29.217.list.dnswl.org : 127.0.10.0];
IP_SCORE(-0.37)[ipnet: 217.29.32.0/20(-1.02), asn: 16188(-0.82), country:
DE(-0.01)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[];
MIME_TRACE(0.00)[0:+];
ASN(0.00)[asn:16188, ipnet:217.29.32.0/20, country:DE];
MID_RHS_MATCH_FROM(0.00)[]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 21 Mar 2019 13:10:34 -0000
Hi all,
> Am 21.03.2019 um 11:24 schrieb Matt Churchyard via =
freebsd-virtualization <freebsd-virtualization@freebsd.org>:
> I'd also be intrigued to know what the logic in FreeNAS is for it. It =
is simply a case of "(arc =3D total_ram - guest_allocated)"?
> Is there a lower limit based on a percentage or total RAM, and/or a =
hard lower limit?
The relevant code can be found here:
=
https://github.com/freenas/freenas/blob/master/src/middlewared/middlewared=
/plugins/vm.py
Kind regards,
Patrick
--=20
punkt.de GmbH Internet - Dienstleistungen - Beratung
Kaiserallee 13a Tel.: 0721 9109-0 Fax: -100
76133 Karlsruhe info@punkt.de http://punkt.de
AG Mannheim 108285 Gf: Juergen Egeling
From owner-freebsd-virtualization@freebsd.org Fri Mar 22 00:44:55 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id B8C531554615
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Fri, 22 Mar 2019 00:44:55 +0000 (UTC)
(envelope-from d.scott.phillips@intel.com)
Received: from mga18.intel.com (mga18.intel.com [134.134.136.126])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "orsmga106.jf.intel.com",
Issuer "COMODO RSA Organization Validation Secure Server CA" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id B515687B7C
for <freebsd-virtualization@freebsd.org>; Fri, 22 Mar 2019 00:44:53 +0000 (UTC)
(envelope-from d.scott.phillips@intel.com)
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from orsmga007.jf.intel.com ([10.7.209.58])
by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
21 Mar 2019 17:44:46 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.60,254,1549958400"; d="scan'208";a="124803722"
Received: from dsp-dsk1.jf.intel.com (HELO localhost) ([10.54.70.10])
by orsmga007.jf.intel.com with ESMTP; 21 Mar 2019 17:44:46 -0700
From: D Scott Phillips <d.scott.phillips@intel.com>
To: freebsd-virtualization@freebsd.org
Subject: Updating uefi-edk2-bhyve
Date: Thu, 21 Mar 2019 17:44:46 -0700
Message-ID: <86muln68ld.fsf@intel.com>
MIME-Version: 1.0
Content-Type: text/plain
X-Rspamd-Queue-Id: B515687B7C
X-Spamd-Bar: --------
Authentication-Results: mx1.freebsd.org;
dmarc=pass (policy=none) header.from=intel.com;
spf=pass (mx1.freebsd.org: domain of d.scott.phillips@intel.com designates
134.134.136.126 as permitted sender) smtp.mailfrom=d.scott.phillips@intel.com
X-Spamd-Result: default: False [-8.79 / 15.00]; ARC_NA(0.00)[];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
R_SPF_ALLOW(-0.20)[+ip4:134.134.136.126/32];
TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain];
TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0];
RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3];
RCVD_TLS_LAST(0.00)[];
RCVD_IN_DNSWL_MED(-0.20)[126.136.134.134.list.dnswl.org : 127.0.9.2];
MX_GOOD(-0.01)[mga04.intel.com,mga03.intel.com,mga14.intel.com,mga09.intel.com,mga12.intel.com,mga07.intel.com,mga06.intel.com,mga17.intel.com,mga01.intel.com,mga02.intel.com,mga11.intel.com,mga18.intel.com,mga05.intel.com];
DMARC_POLICY_ALLOW(-0.50)[intel.com,none];
NEURAL_HAM_SHORT(-0.98)[-0.984,0];
IP_SCORE(-1.79)[ipnet: 134.134.136.0/24(-4.94), asn: 4983(-3.96), country:
US(-0.07)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[];
MIME_TRACE(0.00)[0:+];
ASN(0.00)[asn:4983, ipnet:134.134.136.0/24, country:US];
MID_RHS_MATCH_FROM(0.00)[];
WHITELIST_SPF_DKIM(-3.00)[intel.com:s:+]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 22 Mar 2019 00:44:55 -0000
Hi freebsd-virtualization,
Recently I wanted to be able to do UEFI HTTP Boot in bhyve, so I've
rebased the bhyve firmware up to the latest upstream tag,
edk2-stable201903. You can find the firmware here:
https://gitlab.com/scott-ph/edk2/tree/wip/2019-03/v2-bhyve-rebase-edk2-stable201903
and a ports patch to help with testing here:
https://gitlab.com/scott-ph/freebsd-ports/tree/wip/2019-03/v2-uefi-edk2-bhyve
I've successfully run FreeBSD, Linux, and Windows with this firmware,
and of course HTTP Booting is working. If you're interested you can give
it a try, and I'd be glad to hear any reports of bugs you find with the
new firmware. Hopefully after some testing we can stabilize this and
push it as an update to the port. Thanks,
Scott
From owner-freebsd-virtualization@freebsd.org Fri Mar 22 01:38:01 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE22C1555A6B
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Fri, 22 Mar 2019 01:38:01 +0000 (UTC)
(envelope-from rebecca@bluestop.org)
Received: from muon.bluestop.org (muon.bluestop.org [65.103.231.193])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
server-signature RSA-PSS (4096 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 14F6C896B2
for <freebsd-virtualization@freebsd.org>; Fri, 22 Mar 2019 01:37:59 +0000 (UTC)
(envelope-from rebecca@bluestop.org)
Received: from muon.bluestop.org (localhost [127.0.0.1])
by muon.bluestop.org (Postfix) with ESMTP id 2DDFD7F395;
Thu, 21 Mar 2019 19:39:02 -0600 (MDT)
Received: from muon.bluestop.org ([127.0.0.1])
by muon.bluestop.org (muon.bluestop.org [127.0.0.1]) (amavisd-new, port 10026)
with ESMTP id RXoVHMPKhSWS; Thu, 21 Mar 2019 19:39:01 -0600 (MDT)
Received: from [10.0.10.197] (unknown [65.103.231.197])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by muon.bluestop.org (Postfix) with ESMTPSA;
Thu, 21 Mar 2019 19:39:01 -0600 (MDT)
Date: Thu, 21 Mar 2019 19:34:51 -0600
From: rebecca@bluestop.org
To: freebsd-virtualization@freebsd.org, D Scott Phillips
<d.scott.phillips@intel.com>
Message-ID: <ccedded1-94ae-464b-98f2-ce932a80a0c7@Spark>
In-Reply-To: <86muln68ld.fsf@intel.com>
References: <86muln68ld.fsf@intel.com>
Subject: Re: Updating uefi-edk2-bhyve
X-Readdle-Message-ID: ccedded1-94ae-464b-98f2-ce932a80a0c7@Spark
MIME-Version: 1.0
X-Rspamd-Queue-Id: 14F6C896B2
X-Spamd-Bar: ---
X-Spamd-Result: default: False [-3.25 / 15.00]; ARC_NA(0.00)[];
R_SPF_FAIL(0.00)[-all];
R_DKIM_ALLOW(-0.20)[bluestop.org:s=mail];
RCVD_VIA_SMTP_AUTH(0.00)[];
IP_SCORE(-0.99)[ipnet: 65.100.0.0/14(-4.87), asn: 209(0.01), country:
US(-0.07)]; TO_DN_SOME(0.00)[];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
NEURAL_HAM_LONG(-1.00)[-1.000,0];
MIME_GOOD(-0.10)[multipart/alternative,text/plain];
URI_COUNT_ODD(1.00)[3]; RCVD_COUNT_THREE(0.00)[4];
TO_MATCH_ENVRCPT_SOME(0.00)[];
DKIM_TRACE(0.00)[bluestop.org:+]; RCPT_COUNT_TWO(0.00)[2];
FROM_NO_DN(0.00)[]; MX_GOOD(-0.01)[mail.bluestop.org];
DMARC_POLICY_ALLOW(0.00)[bluestop.org,quarantine];
DMARC_POLICY_ALLOW_WITH_FAILURES(-0.50)[];
NEURAL_HAM_SHORT(-0.95)[-0.953,0]; FROM_EQ_ENVFROM(0.00)[];
MID_RHS_NOT_FQDN(0.50)[]; MIME_TRACE(0.00)[0:+,1:+];
ASN(0.00)[asn:209, ipnet:65.100.0.0/14, country:US];
RCVD_TLS_LAST(0.00)[]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 22 Mar 2019 01:38:02 -0000
This is awesome=21 Updating to a newer edk2 has been on my todo list for =
a while.
However I don=E2=80=99t see a BhyvePkg in the first link you posted. And =
could you confirm that you got HTTP boot working on =46reeBSD as well as =
Linux and Windows=3F I thought loader changes were needed for that to wor=
k, to enable the kernel to be fetched via http.
=E2=80=94
Rebecca Cran
On Mar 21, 2019, 6:46 PM -0600, D Scott Phillips <d.scott.phillips=40inte=
l.com>, wrote:
> Hi freebsd-virtualization,
>
> Recently I wanted to be able to do UE=46I HTTP Boot in bhyve, so I've
> rebased the bhyve firmware up to the latest upstream tag,
> edk2-stable201903. You can find the firmware here:
>
> https://gitlab.com/scott-ph/edk2/tree/wip/2019-03/v2-bhyve-rebase-edk2-=
stable201903
>
> and a ports patch to help with testing here:
>
> https://gitlab.com/scott-ph/freebsd-ports/tree/wip/2019-03/v2-uefi-edk2=
-bhyve
>
> I've successfully run =46reeBSD, Linux, and Windows with this firmware,=
> and of course HTTP Booting is working. If you're interested you can giv=
e
> it a try, and I'd be glad to hear any reports of bugs you find with the=
> new firmware. Hopefully after some testing we can stabilize this and
> push it as an update to the port. Thanks,
>
> Scott
> =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
> freebsd-virtualization=40freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to =22freebsd-virtualization-unsubscribe=40=
freebsd.org=22
From owner-freebsd-virtualization@freebsd.org Fri Mar 22 01:44:39 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 801941555E4A
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Fri, 22 Mar 2019 01:44:39 +0000 (UTC)
(envelope-from araujobsdport@gmail.com)
Received: from mail-lf1-x131.google.com (mail-lf1-x131.google.com
[IPv6:2a00:1450:4864:20::131])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 493ED89EF7
for <freebsd-virtualization@freebsd.org>; Fri, 22 Mar 2019 01:44:38 +0000 (UTC)
(envelope-from araujobsdport@gmail.com)
Received: by mail-lf1-x131.google.com with SMTP id r25so242669lfn.13
for <freebsd-virtualization@freebsd.org>; Thu, 21 Mar 2019 18:44:38 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
h=mime-version:references:in-reply-to:reply-to:from:date:message-id
:subject:to:cc;
bh=HMSiyZP1kBF8wcRwPE88oo9JzJ1QmvU/XESrE3vsrRI=;
b=Vd6L6c63nbymofu1k5ehRkbnSUA55hgpE0kzxtPgpx0VqH7BU58ee5BW0mWh0r4TRH
wMRhGthAfJLAJ/JZBaXOXjxCNngKdUvl1r2bI+Odj8JAqyLOGHCDB34+kvLOLnWEI5mO
y+56MalZOZeoraJWoqIfJMQFeRyEhEPSYAAsWBhe+eBdxSEiSAjvtS+YFbYufgM+4moS
qVZlFnxFEzAROPveqMIHGIh4Cd1fqHshHtLJuSTokJ+n7ScO/p2rrglCAPI8too4SMBO
8trWE+1psAw250UgSMCgPTFgPUZSmKifwNoB+Lxy5YPacgeCDgpPROTxkNuVfAFMFrDm
p79g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:references:in-reply-to:reply-to
:from:date:message-id:subject:to:cc;
bh=HMSiyZP1kBF8wcRwPE88oo9JzJ1QmvU/XESrE3vsrRI=;
b=Ew7PmBVxVbg07KKOsI6qrJtMg7EsFPFDD/2DnKISd/DzpU82XMaRScLqDc9mPBLldX
g7VcfcaNLQpj5I7URTIapeUe6X2fnxxgFVQmnC2NkiX4dOGy7ZT21iWZxSUHKZ/wM+Q+
IdnuXu2S0ieYFSJSWe6LJgNHhenhK4K40V5wsFeThQv4Ppbk6g60Of7/ebxmi5g2h9dC
ltH/sEWRpKCUdU/ZZ/mO6ITnurIl39ydS172PrJVekxLm9EEz8Jn8wyRIqFtsu5xwSan
f4qhHFDpbF1fKx1t5L7RaDrkEDEFVmCgBy2LW1QS8I79bjzTFWIu4lR2amKBxsenVUtF
sPDQ==
X-Gm-Message-State: APjAAAURz9uYFcoy8APabKCc4Cgv3R0SbAhbxtVwwBLvdvk9BoBJhV3K
3y589chEaip3x3EeEwbUxyxytSwQVStyciTnxj8=
X-Google-Smtp-Source: APXvYqzdFOEhBJoLrPr+dQ+bpFS+r4WxTLc2T+Gzo5Rdhh3Tm6rWETJoc7IfyaF9eTDdQyyyHNG14SF5qi76mlcd4tY=
X-Received: by 2002:a19:ed19:: with SMTP id y25mr3858328lfy.80.1553219076835;
Thu, 21 Mar 2019 18:44:36 -0700 (PDT)
MIME-Version: 1.0
References: <86muln68ld.fsf@intel.com>
In-Reply-To: <86muln68ld.fsf@intel.com>
Reply-To: araujo@freebsd.org
From: Marcelo Araujo <araujobsdport@gmail.com>
Date: Fri, 22 Mar 2019 09:44:25 +0800
Message-ID: <CAOfEmZi9zOcpN+kpvSb-gXma=FG5g3v3eS+msX=QdK1dnq5PnQ@mail.gmail.com>
Subject: Re: Updating uefi-edk2-bhyve
To: D Scott Phillips <d.scott.phillips@intel.com>
Cc: "freebsd-virtualization@freebsd.org" <freebsd-virtualization@freebsd.org>
X-Rspamd-Queue-Id: 493ED89EF7
X-Spamd-Bar: -----
Authentication-Results: mx1.freebsd.org;
dkim=pass header.d=gmail.com header.s=20161025 header.b=Vd6L6c63;
dmarc=pass (policy=none) header.from=gmail.com;
spf=pass (mx1.freebsd.org: domain of araujobsdport@gmail.com designates
2a00:1450:4864:20::131 as permitted sender)
smtp.mailfrom=araujobsdport@gmail.com
X-Spamd-Result: default: False [-5.74 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[];
HAS_REPLYTO(0.00)[araujo@freebsd.org]; TO_DN_SOME(0.00)[];
R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36];
FREEMAIL_FROM(0.00)[gmail.com]; URI_COUNT_ODD(1.00)[11];
DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2];
DMARC_POLICY_ALLOW(-0.50)[gmail.com,none];
MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com];
NEURAL_HAM_SHORT(-0.99)[-0.992,0]; FROM_EQ_ENVFROM(0.00)[];
RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+];
FREEMAIL_ENVFROM(0.00)[gmail.com];
ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US];
DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0];
ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[];
NEURAL_HAM_LONG(-1.00)[-1.000,0];
MIME_GOOD(-0.10)[multipart/alternative,text/plain];
PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org];
REPLYTO_DOM_NEQ_FROM_DOM(0.00)[];
TO_MATCH_ENVRCPT_SOME(0.00)[];
RCVD_IN_DNSWL_NONE(0.00)[1.3.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org
: 127.0.5.0];
IP_SCORE(-2.73)[ip: (-9.10), ipnet: 2a00:1450::/32(-2.38), asn: 15169(-2.13),
country: US(-0.07)]; RCVD_COUNT_TWO(0.00)[2]
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 22 Mar 2019 01:44:39 -0000
Em sex, 22 de mar de 2019 =C3=A0s 08:45, D Scott Phillips <
d.scott.phillips@intel.com> escreveu:
> Hi freebsd-virtualization,
>
> Recently I wanted to be able to do UEFI HTTP Boot in bhyve, so I've
> rebased the bhyve firmware up to the latest upstream tag,
> edk2-stable201903. You can find the firmware here:
>
>
> https://gitlab.com/scott-ph/edk2/tree/wip/2019-03/v2-bhyve-rebase-edk2-st=
able201903
>
> and a ports patch to help with testing here:
>
>
> https://gitlab.com/scott-ph/freebsd-ports/tree/wip/2019-03/v2-uefi-edk2-b=
hyve
>
> I've successfully run FreeBSD, Linux, and Windows with this firmware,
> and of course HTTP Booting is working. If you're interested you can give
> it a try, and I'd be glad to hear any reports of bugs you find with the
> new firmware. Hopefully after some testing we can stabilize this and
> push it as an update to the port. Thanks,
>
> Scott
> _______________________________________________
> freebsd-virtualization@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to "
> freebsd-virtualization-unsubscribe@freebsd.org"
>
This is a fantastic news!!!
I think after more tests done by the community, you should make a PR to
this repo: https://github.com/freebsd/uefi-edk2 and then I can update the
ports.
Well done job, thanks!
--=20
--=20
Marcelo Araujo (__)araujo@FreeBSD.org
\\\'',)http://www.FreeBSD.org <http://www.freebsd.org/> \/ \ ^
Power To Server. .\. /_)
From owner-freebsd-virtualization@freebsd.org Fri Mar 22 03:19:46 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 625591557F2E
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Fri, 22 Mar 2019 03:19:46 +0000 (UTC)
(envelope-from rebecca@bluestop.org)
Received: from muon.bluestop.org (muon.bluestop.org [65.103.231.193])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
server-signature RSA-PSS (4096 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 37E0D8D804
for <freebsd-virtualization@freebsd.org>; Fri, 22 Mar 2019 03:19:45 +0000 (UTC)
(envelope-from rebecca@bluestop.org)
Received: from muon.bluestop.org (localhost [127.0.0.1])
by muon.bluestop.org (Postfix) with ESMTP id 477BF7F4AF;
Thu, 21 Mar 2019 21:20:53 -0600 (MDT)
Received: from muon.bluestop.org ([127.0.0.1])
by muon.bluestop.org (muon.bluestop.org [127.0.0.1]) (amavisd-new, port 10026)
with ESMTP id 9MflVRjab_cS; Thu, 21 Mar 2019 21:20:52 -0600 (MDT)
Received: from photon.int.bluestop.org (unknown [65.103.231.197])
(using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by muon.bluestop.org (Postfix) with ESMTPSA;
Thu, 21 Mar 2019 21:20:52 -0600 (MDT)
Subject: Re: Updating uefi-edk2-bhyve
To: D Scott Phillips <d.scott.phillips@intel.com>,
freebsd-virtualization@freebsd.org
References: <86muln68ld.fsf@intel.com>
From: Rebecca Cran <rebecca@bluestop.org>
Message-ID: <1fe3ca3f-be70-99db-e7c0-35c9194c97e4@bluestop.org>
Date: Thu, 21 Mar 2019 21:19:42 -0600
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101
Thunderbird/60.5.3
MIME-Version: 1.0
In-Reply-To: <86muln68ld.fsf@intel.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-Rspamd-Queue-Id: 37E0D8D804
X-Spamd-Bar: ----
X-Spamd-Result: default: False [-4.93 / 15.00]; ARC_NA(0.00)[];
RCVD_VIA_SMTP_AUTH(0.00)[];
R_DKIM_ALLOW(-0.20)[bluestop.org:s=mail];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx];
NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain];
IP_SCORE(-0.99)[ipnet: 65.100.0.0/14(-4.87), asn: 209(0.00), country:
US(-0.07)]; RCVD_COUNT_THREE(0.00)[4];
TO_MATCH_ENVRCPT_SOME(0.00)[];
MX_GOOD(-0.01)[cached: mail.bluestop.org];
DKIM_TRACE(0.00)[bluestop.org:+]; RCPT_COUNT_TWO(0.00)[2];
NEURAL_HAM_SHORT(-0.93)[-0.934,0];
DMARC_POLICY_ALLOW(-0.50)[bluestop.org,quarantine];
FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+];
RCVD_TLS_LAST(0.00)[];
ASN(0.00)[asn:209, ipnet:65.100.0.0/14, country:US];
MID_RHS_MATCH_FROM(0.00)[]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 22 Mar 2019 03:19:46 -0000
On 3/21/19 6:44 PM, D Scott Phillips wrote:
> Hi freebsd-virtualization,
>
> Recently I wanted to be able to do UEFI HTTP Boot in bhyve, so I've
> rebased the bhyve firmware up to the latest upstream tag,
> edk2-stable201903. You can find the firmware here:
>
> https://gitlab.com/scott-ph/edk2/tree/wip/2019-03/v2-bhyve-rebase-edk2-stable201903
Thanks! I realized you've removed BhyvePkg and ported OvmfPkg to run
under Bhyve, which is perfect. I've successfully built OvmfPkgX64 with
your changes and can boot Bhyve to the UEFI Shell.
The RELEASE build with the GCC5 toolset (but using an installation of
gcc 4.8) works great for me, though I ran into problems with the DEBUG
build and the GCC48 toolset.
--
Rebecca Cran
From owner-freebsd-virtualization@freebsd.org Fri Mar 22 03:34:39 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5CF62155861F;
Fri, 22 Mar 2019 03:34:39 +0000 (UTC) (envelope-from ler@lerctr.org)
Received: from thebighonker.lerctr.org (unknown [IPv6:2001:470:1f0f:3ad::53:2])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
server-signature RSA-PSS (4096 bits)
client-signature RSA-PSS (2048 bits) client-digest SHA256)
(Client CN "*.lerctr.org", Issuer "Let's Encrypt Authority X3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 2F6868E151;
Fri, 22 Mar 2019 03:34:37 +0000 (UTC) (envelope-from ler@lerctr.org)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org;
s=lerami;
h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date:
Content-Transfer-Encoding:Content-Type:MIME-Version:Sender:Reply-To:
Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender:
Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:
List-Subscribe:List-Post:List-Owner:List-Archive;
bh=TJDBcLeLh0weWT64mU1791/LfMSypBfSk3z93BBnU1c=; b=LszWD306aEoG8jR0t/yKjDsSAw
Y/UO/8rIjzZI/sctEAVfyFsmqYBe3eS2ajOGHZXrxY+Krqwz/qBF4sstYZjSy2egJIWr53dGmwn9p
IVo0Wp8zHdRccjN1oURvrsXoVL6uw6ZiBz3q9Aw2MCRWDdA5iGvswz1vsSbbKGWuw2TA=;
Received: from thebighonker.lerctr.org
([2001:470:1f0f:3ad:bb:dcff:fe50:d900]:11805 helo=webmail.lerctr.org)
by thebighonker.lerctr.org with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256)
(Exim 4.92 (FreeBSD)) (envelope-from <ler@lerctr.org>)
id 1h7Aww-000DSd-2q; Thu, 21 Mar 2019 22:34:30 -0500
Received: from 2600:1700:210:b18f:e89d:e7f9:22a:2fc4 by webmail.lerctr.org
with HTTP (HTTP/1.1 POST); Thu, 21 Mar 2019 22:34:29 -0500
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII;
format=flowed
Content-Transfer-Encoding: 7bit
Date: Thu, 21 Mar 2019 22:34:29 -0500
From: Larry Rosenman <ler@lerctr.org>
To: Rebecca Cran <rebecca@bluestop.org>
Cc: D Scott Phillips <d.scott.phillips@intel.com>,
freebsd-virtualization@freebsd.org, owner-freebsd-virtualization@freebsd.org
Subject: Re: Updating uefi-edk2-bhyve
In-Reply-To: <1fe3ca3f-be70-99db-e7c0-35c9194c97e4@bluestop.org>
References: <86muln68ld.fsf@intel.com>
<1fe3ca3f-be70-99db-e7c0-35c9194c97e4@bluestop.org>
Message-ID: <7e84fd01c3f46268c26f9bab8b9fb9bc@lerctr.org>
X-Sender: ler@lerctr.org
User-Agent: Roundcube Webmail/1.3.8
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 22 Mar 2019 03:34:39 -0000
On 03/21/2019 10:19 pm, Rebecca Cran via freebsd-virtualization wrote:
> On 3/21/19 6:44 PM, D Scott Phillips wrote:
>> Hi freebsd-virtualization,
>>
>> Recently I wanted to be able to do UEFI HTTP Boot in bhyve, so I've
>> rebased the bhyve firmware up to the latest upstream tag,
>> edk2-stable201903. You can find the firmware here:
>>
>> https://gitlab.com/scott-ph/edk2/tree/wip/2019-03/v2-bhyve-rebase-edk2-stable201903
>
>
> Thanks! I realized you've removed BhyvePkg and ported OvmfPkg to run
> under Bhyve, which is perfect. I've successfully built OvmfPkgX64 with
> your changes and can boot Bhyve to the UEFI Shell.
>
> The RELEASE build with the GCC5 toolset (but using an installation of
> gcc 4.8) works great for me, though I ran into problems with the DEBUG
> build and the GCC48 toolset.
out of curiosity, is there any work being done to move this to later
LLVM/CLANG?
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 E-Mail: ler@lerctr.org
US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
From owner-freebsd-virtualization@freebsd.org Fri Mar 22 05:30:16 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1F696155AC86;
Fri, 22 Mar 2019 05:30:16 +0000 (UTC)
(envelope-from rebecca@bluestop.org)
Received: from muon.bluestop.org (muon.bluestop.org [65.103.231.193])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
server-signature RSA-PSS (4096 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id A5F006AB08;
Fri, 22 Mar 2019 05:30:15 +0000 (UTC)
(envelope-from rebecca@bluestop.org)
Received: from muon.bluestop.org (localhost [127.0.0.1])
by muon.bluestop.org (Postfix) with ESMTP id 7854D7F595;
Thu, 21 Mar 2019 23:31:23 -0600 (MDT)
Received: from muon.bluestop.org ([127.0.0.1])
by muon.bluestop.org (muon.bluestop.org [127.0.0.1]) (amavisd-new, port 10026)
with ESMTP id E3DS4sAfhr3J; Thu, 21 Mar 2019 23:31:22 -0600 (MDT)
Received: from [10.0.10.197] (unknown [65.103.231.197])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(No client certificate requested)
by muon.bluestop.org (Postfix) with ESMTPSA;
Thu, 21 Mar 2019 23:31:22 -0600 (MDT)
Date: Thu, 21 Mar 2019 23:26:44 -0600
From: rebecca@bluestop.org
To: Larry Rosenman <ler@lerctr.org>
Cc: D Scott Phillips <d.scott.phillips@intel.com>,
freebsd-virtualization@freebsd.org,
owner-freebsd-virtualization@freebsd.org
Message-ID: <33fcf111-fe00-4ec7-8a2f-7c53246d756f@Spark>
In-Reply-To: <7e84fd01c3f46268c26f9bab8b9fb9bc@lerctr.org>
References: <86muln68ld.fsf@intel.com>
<1fe3ca3f-be70-99db-e7c0-35c9194c97e4@bluestop.org>
<7e84fd01c3f46268c26f9bab8b9fb9bc@lerctr.org>
Subject: Re: Updating uefi-edk2-bhyve
X-Readdle-Message-ID: 33fcf111-fe00-4ec7-8a2f-7c53246d756f@Spark
MIME-Version: 1.0
X-Rspamd-Queue-Id: A5F006AB08
X-Spamd-Bar: ------
X-Spamd-Result: default: False [-6.94 / 15.00];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0];
NEURAL_HAM_SHORT(-0.94)[-0.942,0];
NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
X-Content-Filtered-By: Mailman/MimeDel 2.1.29
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 22 Mar 2019 05:30:16 -0000
On Mar 21, 2019, 9:54 PM -0600, Larry Rosenman <ler=40lerctr.org>, wrote:=
> out of curiosity, is there any work being done to move this to later
> LLVM/CLANG=3F
That=E2=80=99s something else I=E2=80=99m hoping to work on, since gcc 4.=
8 is very outdated now. It seems to build fine using gcc 7, but I need to=
test that it still runs fine. And that=E2=80=99s complicated by the fact=
that the DEBUG build generates a triple fault even with gcc 4.8.
=E2=80=94
Rebecca Cran
From owner-freebsd-virtualization@freebsd.org Fri Mar 22 12:19:36 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7B01D1544427
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Fri, 22 Mar 2019 12:19:36 +0000 (UTC)
(envelope-from bugzilla-noreply@freebsd.org)
Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org
[IPv6:2001:1900:2254:206a::50:5])
by mx1.freebsd.org (Postfix) with ESMTP id 0E127807BC
for <freebsd-virtualization@freebsd.org>; Fri, 22 Mar 2019 12:19:36 +0000 (UTC)
(envelope-from bugzilla-noreply@freebsd.org)
Received: by mailman.ysv.freebsd.org (Postfix)
id C06301544426; Fri, 22 Mar 2019 12:19:35 +0000 (UTC)
Delivered-To: virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id AEBB61544425
for <virtualization@mailman.ysv.freebsd.org>;
Fri, 22 Mar 2019 12:19:35 +0000 (UTC)
(envelope-from bugzilla-noreply@freebsd.org)
Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org
[IPv6:2001:1900:2254:206a::19:3])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
server-signature RSA-PSS (4096 bits)
client-signature RSA-PSS (4096 bits) client-digest SHA256)
(Client CN "mxrelay.ysv.freebsd.org",
Issuer "Let's Encrypt Authority X3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 4A7EC807BA
for <virtualization@FreeBSD.org>; Fri, 22 Mar 2019 12:19:35 +0000 (UTC)
(envelope-from bugzilla-noreply@freebsd.org)
Received: from kenobi.freebsd.org (kenobi.freebsd.org
[IPv6:2001:1900:2254:206a::16:76])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 713911249B
for <virtualization@FreeBSD.org>; Fri, 22 Mar 2019 12:19:34 +0000 (UTC)
(envelope-from bugzilla-noreply@freebsd.org)
Received: from kenobi.freebsd.org ([127.0.1.118])
by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x2MCJYYp063190
for <virtualization@FreeBSD.org>; Fri, 22 Mar 2019 12:19:34 GMT
(envelope-from bugzilla-noreply@freebsd.org)
Received: (from www@localhost)
by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x2MCJYJZ063181
for virtualization@FreeBSD.org; Fri, 22 Mar 2019 12:19:34 GMT
(envelope-from bugzilla-noreply@freebsd.org)
X-Authentication-Warning: kenobi.freebsd.org: www set sender to
bugzilla-noreply@freebsd.org using -f
From: bugzilla-noreply@freebsd.org
To: virtualization@FreeBSD.org
Subject: [Bug 231756] Vagrant boxes don't work with vagrant-bhyve by default
Date: Fri, 22 Mar 2019 12:19:32 +0000
X-Bugzilla-Reason: AssignedTo
X-Bugzilla-Type: changed
X-Bugzilla-Watch-Reason: None
X-Bugzilla-Product: Base System
X-Bugzilla-Component: bin
X-Bugzilla-Version: CURRENT
X-Bugzilla-Keywords: feature
X-Bugzilla-Severity: Affects Many People
X-Bugzilla-Who: 0mp@FreeBSD.org
X-Bugzilla-Status: Open
X-Bugzilla-Resolution:
X-Bugzilla-Priority: ---
X-Bugzilla-Assigned-To: virtualization@FreeBSD.org
X-Bugzilla-Flags:
X-Bugzilla-Changed-Fields: keywords cc bug_status
Message-ID: <bug-231756-27103-f6VUgfKJ1L@https.bugs.freebsd.org/bugzilla/>
In-Reply-To: <bug-231756-27103@https.bugs.freebsd.org/bugzilla/>
References: <bug-231756-27103@https.bugs.freebsd.org/bugzilla/>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/
Auto-Submitted: auto-generated
MIME-Version: 1.0
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 22 Mar 2019 12:19:36 -0000
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D231756
Mateusz Piotrowski <0mp@FreeBSD.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Keywords| |feature
CC| |0mp@FreeBSD.org
Status|New |Open
--=20
You are receiving this mail because:
You are the assignee for the bug.=
From owner-freebsd-virtualization@freebsd.org Fri Mar 22 16:04:50 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 60FDE1549F76
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Fri, 22 Mar 2019 16:04:50 +0000 (UTC)
(envelope-from d.scott.phillips@intel.com)
Received: from mga18.intel.com (mga18.intel.com [134.134.136.126])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "orsmga106.jf.intel.com",
Issuer "COMODO RSA Organization Validation Secure Server CA" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id B3E9B890E1
for <freebsd-virtualization@freebsd.org>; Fri, 22 Mar 2019 16:04:48 +0000 (UTC)
(envelope-from d.scott.phillips@intel.com)
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from fmsmga004.fm.intel.com ([10.253.24.48])
by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
22 Mar 2019 09:04:45 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.60,256,1549958400"; d="scan'208";a="154842341"
Received: from dsp-dsk1.jf.intel.com (HELO localhost) ([10.54.70.10])
by fmsmga004.fm.intel.com with ESMTP; 22 Mar 2019 09:04:45 -0700
From: D Scott Phillips <d.scott.phillips@intel.com>
To: Rebecca Cran <rebecca@bluestop.org>, freebsd-virtualization@freebsd.org
Subject: Re: Updating uefi-edk2-bhyve
In-Reply-To: <1fe3ca3f-be70-99db-e7c0-35c9194c97e4@bluestop.org>
References: <86muln68ld.fsf@intel.com>
<1fe3ca3f-be70-99db-e7c0-35c9194c97e4@bluestop.org>
Date: Fri, 22 Mar 2019 09:04:45 -0700
Message-ID: <86r2ayao9u.fsf@intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Rspamd-Queue-Id: B3E9B890E1
X-Spamd-Bar: --------
Authentication-Results: mx1.freebsd.org;
dmarc=pass (policy=none) header.from=intel.com;
spf=pass (mx1.freebsd.org: domain of d.scott.phillips@intel.com designates
134.134.136.126 as permitted sender) smtp.mailfrom=d.scott.phillips@intel.com
X-Spamd-Result: default: False [-8.79 / 15.00]; ARC_NA(0.00)[];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
TO_DN_SOME(0.00)[];
R_SPF_ALLOW(-0.20)[+ip4:134.134.136.126/32];
NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain];
RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_THREE(0.00)[3];
TO_MATCH_ENVRCPT_SOME(0.00)[];
RCVD_IN_DNSWL_MED(-0.20)[126.136.134.134.list.dnswl.org : 127.0.9.2];
MX_GOOD(-0.01)[cached: mga04.intel.com];
RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.99)[-0.989,0];
DMARC_POLICY_ALLOW(-0.50)[intel.com,none];
IP_SCORE(-1.79)[ipnet: 134.134.136.0/24(-4.94), asn: 4983(-3.96), country:
US(-0.07)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[];
MIME_TRACE(0.00)[0:+];
ASN(0.00)[asn:4983, ipnet:134.134.136.0/24, country:US];
MID_RHS_MATCH_FROM(0.00)[];
WHITELIST_SPF_DKIM(-3.00)[intel.com:s:+]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 22 Mar 2019 16:04:50 -0000
Rebecca Cran <rebecca@bluestop.org> writes:
> This is awesome! Updating to a newer edk2 has been on my todo list for
> a while.
>
> However I don=E2=80=99t see a BhyvePkg in the first link you posted. And =
could
> you confirm that you got HTTP boot working on FreeBSD as well as Linux
> and Windows? I thought loader changes were needed for that to work, to
> enable the kernel to be fetched via http.
Sorry, I didn't write that very clearly, what I meant was that I tested
local boot with those 3 os's, and then I also tested HTTP Boot and it
was working well enough to fetch and start loader_lua.efi. So I haven't
fully booted FreeBSD over HTTP, just got the loader running.
Just using the loader's nfs support will probably be good enough for my
use, so I don't plan on looking at http support for the loader at the
moment.
> On 3/21/19 6:44 PM, D Scott Phillips wrote:
> > Hi freebsd-virtualization,
> >
> > Recently I wanted to be able to do UEFI HTTP Boot in bhyve, so I've
> > rebased the bhyve firmware up to the latest upstream tag,
> > edk2-stable201903. You can find the firmware here:
> >
> > https://gitlab.com/scott-ph/edk2/tree/wip/2019-03/v2-bhyve-rebase-edk2-=
stable201903
>
>
> Thanks! I realized you've removed BhyvePkg and ported OvmfPkg to run
> under Bhyve, which is perfect. I've successfully built OvmfPkgX64 with
> your changes and can boot Bhyve to the UEFI Shell.
Right, my understanding from looking at the history is that BhyvePkg
started life as a copy of the OvmfPkg directory. So I rewrote the
patches to be deltas against OvmfPkg so that rebasing would be
tractable. In the end I chose to leave them that way to make the life of
the next rebaser easier.
From owner-freebsd-virtualization@freebsd.org Fri Mar 22 16:08:50 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id DF753154A090
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Fri, 22 Mar 2019 16:08:50 +0000 (UTC)
(envelope-from d.scott.phillips@intel.com)
Received: from mga11.intel.com (mga11.intel.com [192.55.52.93])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "fmsmga102.fm.intel.com",
Issuer "COMODO RSA Organization Validation Secure Server CA" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 61E6489237;
Fri, 22 Mar 2019 16:08:49 +0000 (UTC)
(envelope-from d.scott.phillips@intel.com)
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from orsmga004.jf.intel.com ([10.7.209.38])
by fmsmga102.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
22 Mar 2019 09:07:37 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.60,256,1549958400"; d="scan'208";a="285005973"
Received: from dsp-dsk1.jf.intel.com (HELO localhost) ([10.54.70.10])
by orsmga004.jf.intel.com with ESMTP; 22 Mar 2019 09:07:37 -0700
From: D Scott Phillips <d.scott.phillips@intel.com>
To: araujo@freebsd.org
Cc: "freebsd-virtualization\@freebsd.org" <freebsd-virtualization@freebsd.org>
Subject: Re: Updating uefi-edk2-bhyve
In-Reply-To: <CAOfEmZi9zOcpN+kpvSb-gXma=FG5g3v3eS+msX=QdK1dnq5PnQ@mail.gmail.com>
References: <86muln68ld.fsf@intel.com>
<CAOfEmZi9zOcpN+kpvSb-gXma=FG5g3v3eS+msX=QdK1dnq5PnQ@mail.gmail.com>
Date: Fri, 22 Mar 2019 09:07:37 -0700
Message-ID: <86mulmao52.fsf@intel.com>
MIME-Version: 1.0
Content-Type: text/plain
X-Rspamd-Queue-Id: 61E6489237
X-Spamd-Bar: ---------
Authentication-Results: mx1.freebsd.org;
dmarc=pass (policy=none) header.from=intel.com;
spf=pass (mx1.freebsd.org: domain of d.scott.phillips@intel.com designates
192.55.52.93 as permitted sender) smtp.mailfrom=d.scott.phillips@intel.com
X-Spamd-Result: default: False [-9.07 / 15.00]; ARC_NA(0.00)[];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:192.55.52.93/32];
TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain];
RCVD_TLS_LAST(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0];
RCVD_COUNT_THREE(0.00)[3];
IP_SCORE(-1.80)[ipnet: 192.55.52.0/24(-4.96), asn: 4983(-3.96), country:
US(-0.07)]; MX_GOOD(-0.01)[cached: mga04.intel.com];
RCPT_COUNT_TWO(0.00)[2];
DMARC_POLICY_ALLOW(-0.50)[intel.com,none];
NEURAL_HAM_SHORT(-0.97)[-0.968,0]; FROM_EQ_ENVFROM(0.00)[];
R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+];
ASN(0.00)[asn:4983, ipnet:192.55.52.0/24, country:US];
RCVD_IN_DNSWL_HI(-0.50)[93.52.55.192.list.dnswl.org : 127.0.9.3];
MID_RHS_MATCH_FROM(0.00)[];
WHITELIST_SPF_DKIM(-3.00)[intel.com:s:+]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 22 Mar 2019 16:08:51 -0000
Marcelo Araujo <araujobsdport@gmail.com> writes:
> This is a fantastic news!!!
>
> I think after more tests done by the community, you should make a PR
> to this repo: https://github.com/freebsd/uefi-edk2 and then I can
> update the ports.
Yep, that sounds like a good plan to me. Is there anything else I can be
doing to get a "call for testing" out there, beyond this mail thread?
From owner-freebsd-virtualization@freebsd.org Fri Mar 22 16:12:28 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0C342154A34F;
Fri, 22 Mar 2019 16:12:28 +0000 (UTC)
(envelope-from d.scott.phillips@intel.com)
Received: from mga12.intel.com (mga12.intel.com [192.55.52.136])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "fmsmga106.fm.intel.com",
Issuer "COMODO RSA Organization Validation Secure Server CA" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id B86A889620;
Fri, 22 Mar 2019 16:12:26 +0000 (UTC)
(envelope-from d.scott.phillips@intel.com)
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from fmsmga002.fm.intel.com ([10.253.24.26])
by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
22 Mar 2019 09:12:19 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.60,256,1549958400"; d="scan'208";a="154223641"
Received: from dsp-dsk1.jf.intel.com (HELO localhost) ([10.54.70.10])
by fmsmga002.fm.intel.com with ESMTP; 22 Mar 2019 09:12:19 -0700
From: D Scott Phillips <d.scott.phillips@intel.com>
To: rebecca@bluestop.org, Larry Rosenman <ler@lerctr.org>
Cc: freebsd-virtualization@freebsd.org,
owner-freebsd-virtualization@freebsd.org
Subject: Re: Updating uefi-edk2-bhyve
In-Reply-To: <33fcf111-fe00-4ec7-8a2f-7c53246d756f@Spark>
References: <86muln68ld.fsf@intel.com>
<1fe3ca3f-be70-99db-e7c0-35c9194c97e4@bluestop.org>
<7e84fd01c3f46268c26f9bab8b9fb9bc@lerctr.org>
<33fcf111-fe00-4ec7-8a2f-7c53246d756f@Spark>
Date: Fri, 22 Mar 2019 09:12:19 -0700
Message-ID: <86k1gqanx8.fsf@intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Rspamd-Queue-Id: B86A889620
X-Spamd-Bar: --------
Authentication-Results: mx1.freebsd.org;
dmarc=pass (policy=none) header.from=intel.com;
spf=pass (mx1.freebsd.org: domain of d.scott.phillips@intel.com designates
192.55.52.136 as permitted sender) smtp.mailfrom=d.scott.phillips@intel.com
X-Spamd-Result: default: False [-8.79 / 15.00]; ARC_NA(0.00)[];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
RCPT_COUNT_THREE(0.00)[4];
R_SPF_ALLOW(-0.20)[+ip4:192.55.52.136/32];
NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain];
RCVD_TLS_LAST(0.00)[]; TO_DN_SOME(0.00)[];
RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[];
RCVD_IN_DNSWL_MED(-0.20)[136.52.55.192.list.dnswl.org : 127.0.9.2];
MX_GOOD(-0.01)[cached: mga04.intel.com];
DMARC_POLICY_ALLOW(-0.50)[intel.com,none];
NEURAL_HAM_SHORT(-0.98)[-0.984,0];
IP_SCORE(-1.80)[ipnet: 192.55.52.0/24(-4.96), asn: 4983(-3.96), country:
US(-0.07)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[];
MIME_TRACE(0.00)[0:+];
ASN(0.00)[asn:4983, ipnet:192.55.52.0/24, country:US];
MID_RHS_MATCH_FROM(0.00)[];
WHITELIST_SPF_DKIM(-3.00)[intel.com:s:+]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 22 Mar 2019 16:12:28 -0000
rebecca@bluestop.org writes:
> On Mar 21, 2019, 9:54 PM -0600, Larry Rosenman <ler@lerctr.org>, wrote:
> > out of curiosity, is there any work being done to move this to later
> > LLVM/CLANG?
>
> That=E2=80=99s something else I=E2=80=99m hoping to work on, since gcc 4.=
8 is very
> outdated now. It seems to build fine using gcc 7, but I need to test
> that it still runs fine. And that=E2=80=99s complicated by the fact that =
the
> DEBUG build generates a triple fault even with gcc 4.8.
You're seeing this firmware triple fault in the DEBUG build of this new
firmware? It seems to be functioning properly for me. Do you get any
serial output before the fault?
From owner-freebsd-virtualization@freebsd.org Fri Mar 22 20:00:25 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id D8D241550378;
Fri, 22 Mar 2019 20:00:24 +0000 (UTC)
(envelope-from rebecca@bluestop.org)
Received: from muon.bluestop.org (muon.bluestop.org [65.103.231.193])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
server-signature RSA-PSS (4096 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id E8801912AD;
Fri, 22 Mar 2019 20:00:23 +0000 (UTC)
(envelope-from rebecca@bluestop.org)
Received: from muon.bluestop.org (localhost [127.0.0.1])
by muon.bluestop.org (Postfix) with ESMTP id 997F37FCB1;
Fri, 22 Mar 2019 14:01:30 -0600 (MDT)
Received: from muon.bluestop.org ([127.0.0.1])
by muon.bluestop.org (muon.bluestop.org [127.0.0.1]) (amavisd-new, port 10026)
with ESMTP id 3voT63tTumfm; Fri, 22 Mar 2019 14:01:29 -0600 (MDT)
Received: from photon.int.bluestop.org (unknown [65.103.231.197])
(using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by muon.bluestop.org (Postfix) with ESMTPSA;
Fri, 22 Mar 2019 14:01:29 -0600 (MDT)
Subject: Re: Updating uefi-edk2-bhyve
To: D Scott Phillips <d.scott.phillips@intel.com>,
Larry Rosenman <ler@lerctr.org>
Cc: freebsd-virtualization@freebsd.org,
owner-freebsd-virtualization@freebsd.org
References: <86muln68ld.fsf@intel.com>
<1fe3ca3f-be70-99db-e7c0-35c9194c97e4@bluestop.org>
<7e84fd01c3f46268c26f9bab8b9fb9bc@lerctr.org>
<33fcf111-fe00-4ec7-8a2f-7c53246d756f@Spark> <86k1gqanx8.fsf@intel.com>
From: Rebecca Cran <rebecca@bluestop.org>
Message-ID: <8c63eb87-e3b8-2365-2eaf-a6e36424407c@bluestop.org>
Date: Fri, 22 Mar 2019 14:00:21 -0600
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101
Thunderbird/60.5.3
MIME-Version: 1.0
In-Reply-To: <86k1gqanx8.fsf@intel.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
X-Rspamd-Queue-Id: E8801912AD
X-Spamd-Bar: ----
X-Spamd-Result: default: False [-4.91 / 15.00]; ARC_NA(0.00)[];
RCVD_VIA_SMTP_AUTH(0.00)[];
R_DKIM_ALLOW(-0.20)[bluestop.org:s=mail];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+mx];
NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain];
IP_SCORE(-0.99)[ipnet: 65.100.0.0/14(-4.88), asn: 209(-0.01), country:
US(-0.07)]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[4];
TO_MATCH_ENVRCPT_SOME(0.00)[];
DKIM_TRACE(0.00)[bluestop.org:+];
MX_GOOD(-0.01)[cached: mail.bluestop.org];
DMARC_POLICY_ALLOW(-0.50)[bluestop.org,quarantine];
NEURAL_HAM_SHORT(-0.91)[-0.907,0]; FROM_EQ_ENVFROM(0.00)[];
MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[];
ASN(0.00)[asn:209, ipnet:65.100.0.0/14, country:US];
MID_RHS_MATCH_FROM(0.00)[]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 22 Mar 2019 20:00:25 -0000
On 3/22/19 10:12 AM, D Scott Phillips wrote:
>
> You're seeing this firmware triple fault in the DEBUG build of this new
> firmware? It seems to be functioning properly for me. Do you get any
> serial output before the fault?
I do get quite a bit of output before the fault. I'm running 13-CURRENT
(on an Skylake PC), built today, so there may be some new issues that
aren't in a release.
I'm building OVMF with "build -p OvmfPkg/OvmfPkgX64.dsc -a X64 -b DEBUG
-DDEBUG_ON_SERIAL_PORT=TRUE -t GCC5"
root@cube:# bhyve -AHP -s 0:0,hostbridge -s 31:0,lpc -c 4 -m 4G -s
29,fbuf,tcp=0.0.0.0:5900,w=800,h=600 -l bootrom,OVMF_CODE.fd -l
com1,stdio guest
fbuf frame buffer base: 0x941e00000 [sz 16777216]
SecCoreStartupWithStack(0xFFFCC000, 0x820000)
SEC: Normal boot
DecompressMemFvs: OutputBuffer@A00000+0xBE0090
ScratchBuffer@1600000+0x10000 PcdOvmfDecompressionScratchEnd=0x1610000
Register PPI Notify: DCD0BE23-9586-40F4-B643-06522CED4EDE
Install PPI: 8C8CE578-8A3D-4F1C-9935-896185C32DD3
Install PPI: 5473C07A-3DCB-4DCA-BD6F-1E9689E7349A
The 0th FV start address is 0x00000820000, size is 0x000E0000, handle is
0x820000
Register PPI Notify: 49EDB1C1-BF21-4761-BB12-EB0031AABB39
Register PPI Notify: EA7CA24B-DED5-4DAD-A389-BF827E8F9B38
Install PPI: B9E0ABFE-5979-4914-977F-6DEE78C278A6
Install PPI: DBE23AA9-A345-4B97-85B6-B226F1617389
DiscoverPeimsAndOrderWithApriori(): Found 0x6 PEI FFS files in the 0th FV
Evaluate PEI DEPEX for FFS(9B3ADA4F-AE56-4C24-8DEA-F03B7558AE50)
 RESULT = TRUE (Apriori)
Loading PEIM 9B3ADA4F-AE56-4C24-8DEA-F03B7558AE50
Loading PEIM at 0x0000082C340 EntryPoint=0x0000082F812 PcdPeim.efi
Install PPI: 06E81C58-4AD7-44BC-8390-F10265F72480
Install PPI: 01F34D25-4DE2-23AD-3FF3-36353FF323F1
Install PPI: 4D8B155B-C059-4C8F-8926-06FD4331DB8A
Install PPI: A60C6B59-E459-425D-9C69-0BCC9CB27D81
Register PPI Notify: 605EA650-C65C-42E1-BA80-91A52AB618C6
Evaluate PEI DEPEX for FFS(A3610442-E69F-4DF3-82CA-2360C4031A23)
 TRUE
 END
 RESULT = TRUE
Loading PEIM A3610442-E69F-4DF3-82CA-2360C4031A23
Loading PEIM at 0x00000831140 EntryPoint=0x00000832848
ReportStatusCodeRouterPei.efi
Install PPI: 0065D394-9951-4144-82A3-0AFC8579C251
Install PPI: 229832D3-7A30-4B36-B827-F40CB7D45436
Evaluate PEI DEPEX for FFS(9D225237-FA01-464C-A949-BAABC02D31D0)
 PUSH GUID(0065D394-9951-4144-82A3-0AFC8579C251) = TRUE
 END
 RESULT = TRUE
Loading PEIM 9D225237-FA01-464C-A949-BAABC02D31D0
Loading PEIM at 0x000008335C0 EntryPoint=0x00000834B6F
StatusCodeHandlerPei.efi
Evaluate PEI DEPEX for FFS(222C386D-5ABC-4FB4-B124-FBB82488ACF4)
 PUSH GUID(01F34D25-4DE2-23AD-3FF3-36353FF323F1) = TRUE
 END
 RESULT = TRUE
Loading PEIM 222C386D-5ABC-4FB4-B124-FBB82488ACF4
Loading PEIM at 0x00000835BC0 EntryPoint=0x00000835E00 PlatformPei.efi
Platform PEIM Loaded
CMOS:
00: 50 00 47 00 13 00 06 22 03 19 20 02 00 80 00 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 20 00 00 BF 00 00 00 00 00 00 00 00 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 40 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Install PPI: 7408D748-FC8C-4EE6-9288-C4BEC092A410
GetFirstNonAddress: Pci64Base=0x800000000 Pci64Size=0x800000000
PublishPeiMemory: mPhysMemAddressWidth=36 PeiMemoryCap=65800 KB
PeiInstallPeiMemory MemoryBegin 0xBBFBE000, MemoryLength 0x4042000
QemuInitializeRam called
Reserved variable store memory: 0xBFF7C000; size: 528kb
Platform PEI Firmware Volume Initialization
Install PPI: 49EDB1C1-BF21-4761-BB12-EB0031AABB39
Notify: PPI Guid: 49EDB1C1-BF21-4761-BB12-EB0031AABB39, Peim notify
entry point: 8292C1
The 1th FV start address is 0x00000900000, size is 0x00B00000, handle is
0x900000
Register PPI Notify: EE16160A-E8BE-47A6-820A-C6900DB0250A
Temp Stack : BaseAddress=0x818000 Length=0x8000
Temp Heap : BaseAddress=0x810000 Length=0x8000
Total temporary memory:Â Â Â 65536 bytes.
 temporary memory stack ever used:      5096 bytes.
 temporary memory heap used for HobList: 6328 bytes.
 temporary memory heap occupied by memory pages: 0 bytes.
Memory Allocation 0x00000004 0x806000 - 0x806FFF
Memory Allocation 0x00000006 0xBFF7C000 - 0xBFFFFFFF
Memory Allocation 0x00000004 0x820000 - 0x8FFFFF
Memory Allocation 0x00000004 0x900000 - 0x13FFFFF
Old Stack size 32768, New stack size 131072
Stack Hob: BaseAddress=0xBBFBE000 Length=0x20000
Heap Offset = 0xBB7CE000 Stack Offset = 0xBB7BE000
TemporaryRamMigration(0x810000, 0xBBFD6000, 0x10000)
vm exit[0]
reason         VMX
rip            0x0000000000827359
inst_length    5
status         0
exit_reason    2 (Triple fault)
qualification   0x0000000000000000
inst_type              0
inst_error             0
Abort
On the build problem side, if I run "build -p OvmfPkg/OvmfPkgX64.dsc -a
X64 -t GCC48 -b RELEASE" then I get an error (that I don't get if I
replace GCC48 with GCC5):
/usr/local/bin/ld: warning: cannot find entry symbol _ModuleEntryPoint;
not setting start address
/usr/local/bin/ld: warning: cannot find entry symbol _ModuleEntryPoint;
not setting start address
"objcopy"
/home/bcran/workspace/bhyveedk2/Build/OvmfX64/RELEASE_GCC48/X64/OvmfPkg/AmdSevDxe/AmdSevDxe/DEBUG/AmdSevDxe.dll
"objcopy"
/home/bcran/workspace/bhyveedk2/Build/OvmfX64/RELEASE_GCC48/X64/OvmfPkg/EmuVariableFvbRuntimeDxe/Fvb/DEBUG/EmuVariableFvbRuntimeDxe.dll
objcopy: error: the input file
'/home/bcran/workspace/bhyveedk2/Build/OvmfX64/RELEASE_GCC48/X64/OvmfPkg/EmuVariableFvbRuntimeDxe/Fvb/DEBUG/EmuVariableFvbRuntimeDxe.dll'
has no sections
objcopy: error: the input file
'/home/bcran/workspace/bhyveedk2/Build/OvmfX64/RELEASE_GCC48/X64/OvmfPkg/AmdSevDxe/AmdSevDxe/DEBUG/AmdSevDxe.dll'
has no sections
make: *** [GNUmakefile:383:
/home/bcran/workspace/bhyveedk2/Build/OvmfX64/RELEASE_GCC48/X64/OvmfPkg/AmdSevDxe/AmdSevDxe/DEBUG/AmdSevDxe.dll]
Error 1
--
Rebecca Cran
From owner-freebsd-virtualization@freebsd.org Fri Mar 22 20:25:58 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4689A1550D00;
Fri, 22 Mar 2019 20:25:58 +0000 (UTC)
(envelope-from d.scott.phillips@intel.com)
Received: from mga01.intel.com (mga01.intel.com [192.55.52.88])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "fmsmga101.fm.intel.com",
Issuer "COMODO RSA Organization Validation Secure Server CA" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id BCE5B9203D;
Fri, 22 Mar 2019 20:25:56 +0000 (UTC)
(envelope-from d.scott.phillips@intel.com)
X-Amp-Result: SKIPPED(no attachment in message)
X-Amp-File-Uploaded: False
Received: from orsmga007.jf.intel.com ([10.7.209.58])
by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384;
22 Mar 2019 13:25:48 -0700
X-ExtLoop1: 1
X-IronPort-AV: E=Sophos;i="5.60,256,1549958400"; d="scan'208";a="125060312"
Received: from dsp-dsk1.jf.intel.com (HELO localhost) ([10.54.70.10])
by orsmga007.jf.intel.com with ESMTP; 22 Mar 2019 13:25:48 -0700
From: D Scott Phillips <d.scott.phillips@intel.com>
To: Rebecca Cran <rebecca@bluestop.org>, Larry Rosenman <ler@lerctr.org>
Cc: freebsd-virtualization@freebsd.org,
owner-freebsd-virtualization@freebsd.org
Subject: Re: Updating uefi-edk2-bhyve
In-Reply-To: <8c63eb87-e3b8-2365-2eaf-a6e36424407c@bluestop.org>
References: <86muln68ld.fsf@intel.com>
<1fe3ca3f-be70-99db-e7c0-35c9194c97e4@bluestop.org>
<7e84fd01c3f46268c26f9bab8b9fb9bc@lerctr.org>
<33fcf111-fe00-4ec7-8a2f-7c53246d756f@Spark> <86k1gqanx8.fsf@intel.com>
<8c63eb87-e3b8-2365-2eaf-a6e36424407c@bluestop.org>
Date: Fri, 22 Mar 2019 13:25:48 -0700
Message-ID: <86ftremzar.fsf@intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Rspamd-Queue-Id: BCE5B9203D
X-Spamd-Bar: ---------
Authentication-Results: mx1.freebsd.org;
dmarc=pass (policy=none) header.from=intel.com;
spf=pass (mx1.freebsd.org: domain of d.scott.phillips@intel.com designates
192.55.52.88 as permitted sender) smtp.mailfrom=d.scott.phillips@intel.com
X-Spamd-Result: default: False [-9.09 / 15.00]; ARC_NA(0.00)[];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
RCPT_COUNT_THREE(0.00)[4];
R_SPF_ALLOW(-0.20)[+ip4:192.55.52.88/32];
NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain];
RCVD_TLS_LAST(0.00)[]; TO_DN_SOME(0.00)[];
RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[];
MX_GOOD(-0.01)[cached: mga04.intel.com];
DMARC_POLICY_ALLOW(-0.50)[intel.com,none];
NEURAL_HAM_SHORT(-0.98)[-0.984,0];
IP_SCORE(-1.80)[ipnet: 192.55.52.0/24(-4.96), asn: 4983(-3.96), country:
US(-0.07)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[];
MIME_TRACE(0.00)[0:+];
ASN(0.00)[asn:4983, ipnet:192.55.52.0/24, country:US];
RCVD_IN_DNSWL_HI(-0.50)[88.52.55.192.list.dnswl.org : 127.0.9.3];
MID_RHS_MATCH_FROM(0.00)[];
WHITELIST_SPF_DKIM(-3.00)[intel.com:s:+]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 22 Mar 2019 20:25:58 -0000
Rebecca Cran <rebecca@bluestop.org> writes:
> On 3/22/19 10:12 AM, D Scott Phillips wrote:
>
> >
> > You're seeing this firmware triple fault in the DEBUG build of this new
> > firmware? It seems to be functioning properly for me. Do you get any
> > serial output before the fault?
>
> I do get quite a bit of output before the fault. I'm running 13-CURRENT=20
> (on an Skylake PC), built today, so there may be some new issues that=20
> aren't in a release.
>
> I'm building OVMF with "build -p OvmfPkg/OvmfPkgX64.dsc -a X64 -b DEBUG=20
> -DDEBUG_ON_SERIAL_PORT=3DTRUE -t GCC5"
>
> root@cube:# bhyve -AHP -s 0:0,hostbridge -s 31:0,lpc -c 4 -m 4G -s=20
> 29,fbuf,tcp=3D0.0.0.0:5900,w=3D800,h=3D600 -l bootrom,OVMF_CODE.fd -l=20
> com1,stdio guest
>
> [..snip..]
>
> Memory Allocation 0x00000004 0x806000 - 0x806FFF
> Memory Allocation 0x00000006 0xBFF7C000 - 0xBFFFFFFF
> Memory Allocation 0x00000004 0x820000 - 0x8FFFFF
> Memory Allocation 0x00000004 0x900000 - 0x13FFFFF
> Old Stack size 32768, New stack size 131072
> Stack Hob: BaseAddress=3D0xBBFBE000 Length=3D0x20000
> Heap Offset =3D 0xBB7CE000 Stack Offset =3D 0xBB7BE000
> TemporaryRamMigration(0x810000, 0xBBFD6000, 0x10000)
> vm exit[0]
> reason=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 VMX
> rip=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0 0x0000000000827359
> inst_length=C2=A0=C2=A0=C2=A0=C2=A0 5
> status=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0
> exit_reason=C2=A0=C2=A0=C2=A0=C2=A0 2 (Triple fault)
> qualification=C2=A0=C2=A0=C2=A0 0x0000000000000000
> inst_type=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 0
> inst_error=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0 0
> Abort
Hmm, I guess it might be some diference in the code generation between
gcc 4.8 and gcc 5.
> On the build problem side, if I run "build -p OvmfPkg/OvmfPkgX64.dsc -a=20
> X64 -t GCC48 -b RELEASE" then I get an error (that I don't get if I=20
> replace GCC48 with GCC5):
>
> /usr/local/bin/ld: warning: cannot find entry symbol _ModuleEntryPoint;=20
> not setting start address
> /usr/local/bin/ld: warning: cannot find entry symbol _ModuleEntryPoint;=20
> not setting start address
> "objcopy"=20
> /home/bcran/workspace/bhyveedk2/Build/OvmfX64/RELEASE_GCC48/X64/OvmfPkg/A=
mdSevDxe/AmdSevDxe/DEBUG/AmdSevDxe.dll
> "objcopy"=20
> /home/bcran/workspace/bhyveedk2/Build/OvmfX64/RELEASE_GCC48/X64/OvmfPkg/E=
muVariableFvbRuntimeDxe/Fvb/DEBUG/EmuVariableFvbRuntimeDxe.dll
> objcopy: error: the input file=20
> '/home/bcran/workspace/bhyveedk2/Build/OvmfX64/RELEASE_GCC48/X64/OvmfPkg/=
EmuVariableFvbRuntimeDxe/Fvb/DEBUG/EmuVariableFvbRuntimeDxe.dll'=20
> has no sections
> objcopy: error: the input file=20
> '/home/bcran/workspace/bhyveedk2/Build/OvmfX64/RELEASE_GCC48/X64/OvmfPkg/=
AmdSevDxe/AmdSevDxe/DEBUG/AmdSevDxe.dll'=20
> has no sections
> make: *** [GNUmakefile:383:=20
> /home/bcran/workspace/bhyveedk2/Build/OvmfX64/RELEASE_GCC48/X64/OvmfPkg/A=
mdSevDxe/AmdSevDxe/DEBUG/AmdSevDxe.dll]=20
> Error 1
I'm not seeing this problem with the build steps I'm doing, which are
just the ones in the ports patch. Perhaps some system version of one of
the build tools is getting used at some point? I had to mess with
${PATH} to get the build to work right. I'd say look back through the
build log and make sure it's using the expected executables. On my side
I'll run the build through poudriere to make sure it's not working due
to some specific quirk of my system.
From owner-freebsd-virtualization@freebsd.org Fri Mar 22 22:29:58 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id A18A61553B8F;
Fri, 22 Mar 2019 22:29:58 +0000 (UTC)
(envelope-from rebecca@bluestop.org)
Received: from muon.bluestop.org (muon.bluestop.org [65.103.231.193])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
server-signature RSA-PSS (4096 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 45C8A96A61;
Fri, 22 Mar 2019 22:29:57 +0000 (UTC)
(envelope-from rebecca@bluestop.org)
Received: from muon.bluestop.org (localhost [127.0.0.1])
by muon.bluestop.org (Postfix) with ESMTP id 7ACB37FD8C;
Fri, 22 Mar 2019 16:31:03 -0600 (MDT)
Received: from muon.bluestop.org ([127.0.0.1])
by muon.bluestop.org (muon.bluestop.org [127.0.0.1]) (amavisd-new, port 10026)
with ESMTP id NOtdbY29lRiI; Fri, 22 Mar 2019 16:31:02 -0600 (MDT)
Received: from photon.int.bluestop.org (unknown [65.103.231.197])
(using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by muon.bluestop.org (Postfix) with ESMTPSA;
Fri, 22 Mar 2019 16:31:02 -0600 (MDT)
Subject: Re: Updating uefi-edk2-bhyve
To: D Scott Phillips <d.scott.phillips@intel.com>,
Larry Rosenman <ler@lerctr.org>
Cc: freebsd-virtualization@freebsd.org,
owner-freebsd-virtualization@freebsd.org
References: <86muln68ld.fsf@intel.com>
<1fe3ca3f-be70-99db-e7c0-35c9194c97e4@bluestop.org>
<7e84fd01c3f46268c26f9bab8b9fb9bc@lerctr.org>
<33fcf111-fe00-4ec7-8a2f-7c53246d756f@Spark> <86k1gqanx8.fsf@intel.com>
<8c63eb87-e3b8-2365-2eaf-a6e36424407c@bluestop.org>
<86ftremzar.fsf@intel.com>
From: Rebecca Cran <rebecca@bluestop.org>
Message-ID: <b2f90a4c-d73f-6797-aa1e-62161ad011f5@bluestop.org>
Date: Fri, 22 Mar 2019 16:29:54 -0600
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101
Thunderbird/60.5.3
MIME-Version: 1.0
In-Reply-To: <86ftremzar.fsf@intel.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
X-Rspamd-Queue-Id: 45C8A96A61
X-Spamd-Bar: ----
X-Spamd-Result: default: False [-4.96 / 15.00]; ARC_NA(0.00)[];
RCVD_VIA_SMTP_AUTH(0.00)[];
R_DKIM_ALLOW(-0.20)[bluestop.org:s=mail];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+mx];
NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain];
IP_SCORE(-0.99)[ipnet: 65.100.0.0/14(-4.88), asn: 209(-0.01), country:
US(-0.07)]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[4];
TO_MATCH_ENVRCPT_SOME(0.00)[];
MX_GOOD(-0.01)[cached: mail.bluestop.org];
DKIM_TRACE(0.00)[bluestop.org:+];
DMARC_POLICY_ALLOW(-0.50)[bluestop.org,quarantine];
NEURAL_HAM_SHORT(-0.96)[-0.959,0]; FROM_EQ_ENVFROM(0.00)[];
MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[];
ASN(0.00)[asn:209, ipnet:65.100.0.0/14, country:US];
MID_RHS_MATCH_FROM(0.00)[]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 22 Mar 2019 22:29:58 -0000
On 3/22/19 2:25 PM, D Scott Phillips wrote:
> Hmm, I guess it might be some diference in the code generation between
> gcc 4.8 and gcc 5.
I've just tested switching from gcc 4.8 to 8.3.0 and everything seems to
work fine - both build and runtime - so I think it may be more
productive to upgrade and use that instead, since it's a supported
version (gcc 4.8 is pretty old now). Would you agree?
The DEBUG build did complain about unused definitions and functions in
FeatureControl.c though, so I ended up with the following patch:
diff --git a/OvmfPkg/PlatformPei/FeatureControl.c
b/OvmfPkg/PlatformPei/FeatureControl.c
index 09f33fe5b1..051dbbdac0 100644
--- a/OvmfPkg/PlatformPei/FeatureControl.c
+++ b/OvmfPkg/PlatformPei/FeatureControl.c
@@ -24,7 +24,7 @@
 //
 // The value to be written to the Feature Control MSR, retrieved from
fw_cfg.
 //
-STATIC UINT64 mFeatureControlValue;
+//STATIC UINT64 mFeatureControlValue;^M
 /**
  Write the Feature Control MSR on an Application Processor or the Boot
@@ -36,6 +36,7 @@ STATIC UINT64 mFeatureControlValue;
  @param[in,out] WorkSpace Pointer to the input/output argument workspace
                            shared by all processors.
 **/
+#if 0^M
 STATIC
 VOID
 EFIAPI
@@ -45,7 +46,7 @@ WriteFeatureControl (
 {
  AsmWriteMsr64 (MSR_CORE2_FEATURE_CONTROL, mFeatureControlValue);
 }
-
+#endif^M
 /**
  Notification function called when EFI_PEI_MP_SERVICES_PPI becomes
available.
@@ -57,6 +58,7 @@ WriteFeatureControl (
  @return Status of the notification. The status code returned from this
           function is ignored.
 **/
+#if 0^M
 STATIC
 EFI_STATUS
 EFIAPI
@@ -94,17 +96,17 @@ OnMpServicesAvailable (
  WriteFeatureControl (NULL);
  return EFI_SUCCESS;
 }
-
+#endif^M
 //
 // Notification object for registering the callback, for when
 // EFI_PEI_MP_SERVICES_PPI becomes available.
 //
-STATIC CONST EFI_PEI_NOTIFY_DESCRIPTOR mMpServicesNotify = {
-Â EFI_PEI_PPI_DESCRIPTOR_NOTIFY_CALLBACK | // Flags
-Â EFI_PEI_PPI_DESCRIPTOR_TERMINATE_LIST,
- &gEfiPeiMpServicesPpiGuid,              // Guid
- OnMpServicesAvailable                   // Notify
-};
+//STATIC CONST EFI_PEI_NOTIFY_DESCRIPTOR mMpServicesNotify = {^M
+//Â EFI_PEI_PPI_DESCRIPTOR_NOTIFY_CALLBACK | // Flags^M
+//Â EFI_PEI_PPI_DESCRIPTOR_TERMINATE_LIST,^M
+// &gEfiPeiMpServicesPpiGuid,              // Guid^M
+// OnMpServicesAvailable                   // Notify^M
+//};^M
 VOID
 InstallFeatureControlCallback (
From owner-freebsd-virtualization@freebsd.org Sat Mar 23 12:54:27 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id E54F9154E994
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Sat, 23 Mar 2019 12:54:26 +0000 (UTC)
(envelope-from tech-lists@zyxst.net)
Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com
[66.111.4.27])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id 7D6928FC77
for <freebsd-virtualization@freebsd.org>; Sat, 23 Mar 2019 12:54:24 +0000 (UTC)
(envelope-from tech-lists@zyxst.net)
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44])
by mailout.nyi.internal (Postfix) with ESMTP id 4A2D6200E3
for <freebsd-virtualization@freebsd.org>; Sat, 23 Mar 2019 08:54:18 -0400 (EDT)
Received: from mailfrontend1 ([10.202.2.162])
by compute4.internal (MEProxy); Sat, 23 Mar 2019 08:54:18 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zyxst.net; h=
date:from:to:subject:message-id:mime-version:content-type; s=
fm2; bh=SNj3HRC2RHv8GPDky3/UXB3c/PVEje57bEgakfqWaYM=; b=MfX6FVyD
vgZmubdvvLq63VVtPI6ude+oOozyJHLiCnxM4M36K671WykJKdnOiqS7R76qeGsk
siFBxerRWEMhtEqtN1bR5fWwccmafA1h63xin/9Wk0NIEgoy5AkVhAKwHRzIMl3I
4LCmxwiOm5fmNeMjysxX4iHuI7yDtdCz77396ExfyhOgHGeMMhjhB4ZbrlimSoS1
XmO1sXJyIAjpT+WTOhvtNNeKmjbruCM0eaWHDRc1nGBYWYqB/DPG2mNOYhQ1Gp10
E7v1w7K2jRhrxk8HRpKsngJJKKdqgKOWSPknHJ8EienYT1lXpODeI79SgiETPrPL
lv72RMQFxqtsRA==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=content-type:date:from:message-id
:mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender
:x-me-sender:x-sasl-enc; s=fm2; bh=SNj3HRC2RHv8GPDky3/UXB3c/PVEj
e57bEgakfqWaYM=; b=wM+FPO/ZkjqGN6fO+i+hJMlnZb7QhfZOR7ET5IeY1N498
uPnHiZ2Fl1JtU4Sr/a7DtCLndozA8mtEY299A8Px61kptDSxQQjbW04n0x6fZVFX
yLq4tknOd1aZ4hZLEIhwiW1NhFCfHz7tMakpcGpFjMwlWtzrw/UMIUkyhm4xjw2y
UOU3VDKmab/liT2M/XlHNIdNyn7y8bEGagiup26LCBhVzpxzM48NVjuY8V1c4YOQ
p4UboEP+75k7mEUAkU2tPViALxjJ/elv4SwabwHaNZkCqy2fzDcfWjIDvjhRlibI
yTjw0z362l7SpgP6tzDe6EhTe6KKWbqt5OkmgNh0A==
X-ME-Sender: <xms:eSyWXJB4xsgCcWT-vVlLxNKGEsup-NX5-b8khOR7UVUE9u4UqTIYpg>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedutddrjeefgdegjecutefuodetggdotefrodftvf
curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu
uegrihhlohhuthemuceftddtnecunecujfgurhepfffhvffukfggtggufgesghdtreertd
ervdenucfhrhhomhepthgvtghhqdhlihhsthhsuceothgvtghhqdhlihhsthhsseiihiig
shhtrdhnvghtqeenucfkphepkedvrdejtddrledurddutddunecurfgrrhgrmhepmhgrih
hlfhhrohhmpehtvggthhdqlhhishhtshesiiihgihsthdrnhgvthenucevlhhushhtvghr
ufhiiigvpedt
X-ME-Proxy: <xmx:eiyWXPz_D0AV9h1Zs7YF5LQZqaeoLr2M7ZVS_7TP9Vy7NsKuFzlbGw>
<xmx:eiyWXKe2-SgDNvOuTtlzcLyzfjrLz66wIMm9N9T0ei8X9mvp3TcCOw>
<xmx:eiyWXFfrunuNeH0yo7qTYkZVuN2SkVcg7ik-N8hYOlN1-U0eMZaGvQ>
<xmx:eiyWXDBGY-lGUjP4tKwnlEH-WrEq1q0aHwfXP6cKfVnFh7nKr107kA>
Received: from rpi3.zyxst.net (rpi3.zyxst.net [82.70.91.101])
by mail.messagingengine.com (Postfix) with ESMTPA id 70721E4176
for <freebsd-virtualization@freebsd.org>; Sat, 23 Mar 2019 08:54:17 -0400 (EDT)
Date: Sat, 23 Mar 2019 12:54:15 +0000
From: tech-lists <tech-lists@zyxst.net>
To: freebsd-virtualization@freebsd.org
Subject: freebsd guest on azure
Message-ID: <20190323125414.GA43521@rpi3.zyxst.net>
Mail-Followup-To: freebsd-virtualization@freebsd.org
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
protocol="application/pgp-signature"; boundary="NzB8fVQJ5HfG6fxh"
Content-Disposition: inline
User-Agent: Mutt/1.11.2 (2019-01-07)
X-Rspamd-Queue-Id: 7D6928FC77
X-Spamd-Bar: ---------
Authentication-Results: mx1.freebsd.org;
dkim=pass header.d=zyxst.net header.s=fm2 header.b=MfX6FVyD;
dkim=pass header.d=messagingengine.com header.s=fm2 header.b=wM+FPO/Z;
spf=pass (mx1.freebsd.org: domain of tech-lists@zyxst.net designates
66.111.4.27 as permitted sender) smtp.mailfrom=tech-lists@zyxst.net
X-Spamd-Result: default: False [-9.20 / 15.00]; ARC_NA(0.00)[];
RCVD_VIA_SMTP_AUTH(0.00)[];
R_DKIM_ALLOW(-0.20)[zyxst.net:s=fm2,messagingengine.com:s=fm2];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
R_SPF_ALLOW(-0.20)[+ip4:66.111.4.27];
TO_MATCH_ENVRCPT_ALL(0.00)[];
MIME_GOOD(-0.20)[multipart/signed,text/plain];
PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org];
TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1];
NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4];
DMARC_NA(0.00)[zyxst.net];
DKIM_TRACE(0.00)[zyxst.net:+,messagingengine.com:+];
MX_GOOD(-0.01)[in2-smtp.messagingengine.com,in1-smtp.messagingengine.com,in2-smtp.messagingengine.com,in1-smtp.messagingengine.com,in2-smtp.messagingengine.com,in1-smtp.messagingengine.com];
NEURAL_HAM_SHORT(-0.96)[-0.965,0]; SIGNED_PGP(-2.00)[];
FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+];
RCVD_TLS_LAST(0.00)[];
ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US];
IP_SCORE(-3.53)[ip: (-9.55), ipnet: 66.111.4.0/24(-4.60), asn: 11403(-3.42),
country: US(-0.07)];
RCVD_IN_DNSWL_LOW(-0.10)[27.4.111.66.list.dnswl.org : 127.0.5.1]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 23 Mar 2019 12:54:27 -0000
--NzB8fVQJ5HfG6fxh
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Hi,
If you're running a freebsd instance on azure,
1. can you back up the instance
-without azure's own tools
-or with azures own tools (and I imagine it costs extra)
2. or if you can't directly back up the instance, apart from tarring up
the directories you need to keep and downloading them from the instance,=20
what do you do?
thanks,
--=20
J.
--NzB8fVQJ5HfG6fxh
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEE8n3tWhxW11Ccvv9/s8o7QhFzNAUFAlyWLG0ACgkQs8o7QhFz
NAUcSw//ZXJBH+0MM9PQu6M63lk84Q3mfVC+dBjFB0+aE+M/kGRtTAaEg4k2bc2T
Fzx2o1iXgXyEs+zJQ1wlF8f1Q50gr/TtCDWwEukkrXggyvHvWNUnS9qerJWM8vIQ
sd7pWVstoGCcWylD1L92ynEKTl2xcB7XRh7nCWYfMn9qmGbVPtF9wr4bRydDubTK
jyRFCSvQnUcrhNYWMMHYSsS7XJB2eO36I3ACmuBR6PmxwWbO23qBUm9M2HbpIzXr
W8JFS+NhRVkokAccarV13d2jAzADidj8DqQjU65VDqBq/ccDlyZ5+dwmgSZAIFul
F/80+H0lZBzZXAPe7fEOwtYpMNKCtHCs9EdWujTOXWCIC3cLAL78MzjikCWB1Uuj
mY2kH+AAG9OVGvYTC5TWJ8kMnR5TAs5tZ2jnoAfk3MvdONpL/GZgq7048HgQhemP
Z6DDi2M6LaeLyOKccEVIXX+VB5OLD+ry9pNFypFfFcm4GIm1AR2XVm+4YUORlDvf
c4fC3hXMcsGPjgm9fJ2cgAxAI1JVLDiK04NehwVx2xtj4moCJXKmA+B+xwJdxVUT
WH6tHOro6BtIKnnWwWztAa4HxjTUJ5UbBh0VZ0nhaN34UCHfBJRx/nBKlBOzI7cd
YmLqZxHU76t9PYgo0W9n0IrBoHeWbKJuVdfgg64LW2TWj56wgUw=
=pJI8
-----END PGP SIGNATURE-----
--NzB8fVQJ5HfG6fxh--
From owner-freebsd-virtualization@freebsd.org Sat Mar 23 13:52:41 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id EAEEF15514AD
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Sat, 23 Mar 2019 13:52:40 +0000 (UTC)
(envelope-from frank2@fjl.co.uk)
Received: from bs1.fjl.org.uk (bs1.fjl.org.uk [84.45.41.196])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(Client CN "bs1.fjl.org.uk", Issuer "bs1.fjl.org.uk" (not verified))
by mx1.freebsd.org (Postfix) with ESMTPS id B11E86B9B9
for <freebsd-virtualization@freebsd.org>; Sat, 23 Mar 2019 13:52:33 +0000 (UTC)
(envelope-from frank2@fjl.co.uk)
Received: from [10.80.23.167] ([103.14.185.11]) (authenticated bits=0)
by bs1.fjl.org.uk (8.14.4/8.14.4) with ESMTP id x2NDVLgw022002
for <freebsd-virtualization@freebsd.org>; Sat, 23 Mar 2019 13:31:22 GMT
(envelope-from frank2@fjl.co.uk)
Date: Sat, 23 Mar 2019 21:31:17 +0800
User-Agent: K-9 Mail for Android
In-Reply-To: <20190323125414.GA43521@rpi3.zyxst.net>
References: <20190323125414.GA43521@rpi3.zyxst.net>
MIME-Version: 1.0
Content-Type: text/plain;
charset=utf-8
Content-Transfer-Encoding: quoted-printable
Subject: Re: freebsd guest on azure
To: freebsd-virtualization@freebsd.org
From: Frank Leonhardt <frank2@fjl.co.uk>
Message-ID: <610D9FF5-891C-4657-BA2A-1E38D41AFF35@fjl.co.uk>
X-Rspamd-Queue-Id: B11E86B9B9
X-Spamd-Bar: -
Authentication-Results: mx1.freebsd.org;
spf=pass (mx1.freebsd.org: domain of frank2@fjl.co.uk designates 84.45.41.196
as permitted sender) smtp.mailfrom=frank2@fjl.co.uk
X-Spamd-Result: default: False [-1.16 / 15.00]; ARC_NA(0.00)[];
NEURAL_HAM_MEDIUM(-0.53)[-0.530,0]; FROM_HAS_DN(0.00)[];
R_SPF_ALLOW(-0.20)[+ip4:84.45.41.196];
TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain];
PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org];
TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1];
NEURAL_HAM_LONG(-0.85)[-0.850,0]; RCVD_TLS_LAST(0.00)[];
NEURAL_SPAM_SHORT(0.75)[0.750,0];
RCVD_IN_DNSWL_MED(-0.20)[196.41.45.84.list.dnswl.org : 127.0.6.2];
MX_GOOD(-0.01)[bs1.fjl.org.uk]; DMARC_NA(0.00)[fjl.co.uk];
IP_SCORE(-0.02)[country: GB(-0.09)]; FROM_EQ_ENVFROM(0.00)[];
R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+];
ASN(0.00)[asn:25577, ipnet:84.45.0.0/17, country:GB];
MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 23 Mar 2019 13:52:41 -0000
On 23 March 2019 20:54:15 GMT+08:00, tech-lists <tech-lists@zyxst=2Enet> w=
rote:
>Hi,
>
>If you're running a freebsd instance on azure,
>
>1=2E can you back up the instance
>
>-without azure's own tools
>-or with azures own tools (and I imagine it costs extra)
>
>2=2E or if you can't directly back up the instance, apart from tarring up
>the directories you need to keep and downloading them from the
>instance,=20
>what do you do?
First off, if I'm doing this on Azure then someone with deep pockets and a=
Microsoft fixation is paying=2E I thought I'd make that clear!
As to your question, what I've done is run a ZFS pool and simply send a sn=
apshot=2E I can't remember now if I made it boot from ZFS as I prefer to bo=
ot from UFS and use ZFS for the rest=2E But I think I did this with Azure e=
xactly because I wanted to copy the disk around=2E It was early last year=
so you can't quite me on it=2E
If you're not sure about sending datasets on ZFS then ask=2E
Regards, Frank=2E
--=20
Sent from my Android device with K-9 Mail=2E Please excuse my brevity=2E
From owner-freebsd-virtualization@freebsd.org Sat Mar 23 18:57:31 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3ED4E1559BB6
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Sat, 23 Mar 2019 18:57:31 +0000 (UTC)
(envelope-from tech-lists@zyxst.net)
Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com
[66.111.4.27])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id A60C077239
for <freebsd-virtualization@freebsd.org>; Sat, 23 Mar 2019 18:57:29 +0000 (UTC)
(envelope-from tech-lists@zyxst.net)
Received: from compute4.internal (compute4.nyi.internal [10.202.2.44])
by mailout.nyi.internal (Postfix) with ESMTP id 8B31E20ED2
for <freebsd-virtualization@freebsd.org>; Sat, 23 Mar 2019 14:57:28 -0400 (EDT)
Received: from mailfrontend2 ([10.202.2.163])
by compute4.internal (MEProxy); Sat, 23 Mar 2019 14:57:28 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zyxst.net; h=
date:from:to:subject:message-id:references:mime-version
:content-type:in-reply-to; s=fm2; bh=f7igJgBwCYcxPfO+IBIrSzyihyi
GOtkXScAEMO9ImZU=; b=SidS4q6iH72gRR9l9Fi/uxWoM8c6wjDi2jm69ToQO1H
CK8XcGZ3kYZ2mo4fPZXWxok/YBKIIIpeR8LBE8zMfK47ppYdRi9iFCta8x0Gk6Ly
+gNohUyaeAcATbulvaaIt5Hf/0YHXolpNju6KmrOcp/EmabCrzfQGYbw5I3h7pQa
LNzLbRzDLHhf+vePKeUB52r7U2PtXJXo+Lcu3W+mWsCAx0qSBGs1an40MYFXWIDL
0FaHKb92UyaXJwzolqXn6kdXLax5HTF28iXZLw1g2NIHOdASPCgObBGajg5Zs6uq
8JoQQDDFVzN5dc92J+5l2nTbGF1zxQ+Ptk0BkJFY3jw==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
messagingengine.com; h=content-type:date:from:in-reply-to
:message-id:mime-version:references:subject:to:x-me-proxy
:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=f7igJg
BwCYcxPfO+IBIrSzyihyiGOtkXScAEMO9ImZU=; b=lcg5ErnAq4LAzU/DK2SXaw
Pt5KKPeq6gIo8zZYILm2r4x0snmaw1DpBIp+MfT7oVQAft2RjWNrGPLbnn7xAy2z
7iRjz019h69ivoNgvdFbVVib1v79H3ewujhp+D36KI/XQo1crzPngTo/eEPP9/AI
7zhfR6aqzVTLlG9fqi5yOCxmoL8/k9Zua1DzniHeu+2d/Y4zgKURsvIXHFiuTb1b
5Q8TCQecbWE4wjmigGbL6tbiXPXXxCVgtDzP46t3J/TFpiJX79uXVG/2LAEEQjMg
5+yjqt25MvcadoKQ2re0bKXkYMofaS9rGKAm1Z0nt3heVEcoOiIoV8h8qIL8v3qw
==
X-ME-Sender: <xms:mIGWXH7qdv93gYrOaSg0Ldb7LWY8X5J6SOUQWfDhM98T_5yYdGG6Qw>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedutddrjeefgdduvddvucetufdoteggodetrfdotf
fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
uceurghilhhouhhtmecufedttdenucenucfjughrpeffhffvuffkfhggtggujggfsehgtd
erredtredvnecuhfhrohhmpehtvggthhdqlhhishhtshcuoehtvggthhdqlhhishhtshes
iiihgihsthdrnhgvtheqnecukfhppeekvddrjedtrdeluddruddtudenucfrrghrrghmpe
hmrghilhhfrhhomhepthgvtghhqdhlihhsthhsseiihiigshhtrdhnvghtnecuvehluhhs
thgvrhfuihiivgeptd
X-ME-Proxy: <xmx:mIGWXJ_BxhD17x2wQiLXpTSzbMTN3c_farJjjdmqMCqz5F4sh6abgw>
<xmx:mIGWXGMkdGCZrvidoWbGzfNOhnNKTnQaO0l5KMOpsdfBGvJUa9C_Uw>
<xmx:mIGWXExrWwQdO0EpHR1q6WtWx6MaE0WYkkf6S1AKi5K6_FXMXk_xGQ>
<xmx:mIGWXGuSx8q93Pqxj1ujMRzt3mVTcGUkPNth3R8tknfBbJ4O7mMt0Q>
Received: from rpi3.zyxst.net (rpi3.zyxst.net [82.70.91.101])
by mail.messagingengine.com (Postfix) with ESMTPA id 911BF1030F
for <freebsd-virtualization@freebsd.org>; Sat, 23 Mar 2019 14:57:27 -0400 (EDT)
Date: Sat, 23 Mar 2019 18:57:25 +0000
From: tech-lists <tech-lists@zyxst.net>
To: freebsd-virtualization@freebsd.org
Subject: Re: freebsd guest on azure
Message-ID: <20190323185724.GA71155@rpi3.zyxst.net>
Mail-Followup-To: freebsd-virtualization@freebsd.org
References: <20190323125414.GA43521@rpi3.zyxst.net>
<610D9FF5-891C-4657-BA2A-1E38D41AFF35@fjl.co.uk>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
protocol="application/pgp-signature"; boundary="oyUTqETQ0mS9luUI"
Content-Disposition: inline
In-Reply-To: <610D9FF5-891C-4657-BA2A-1E38D41AFF35@fjl.co.uk>
User-Agent: Mutt/1.11.2 (2019-01-07)
X-Rspamd-Queue-Id: A60C077239
X-Spamd-Bar: ---------
Authentication-Results: mx1.freebsd.org;
dkim=pass header.d=zyxst.net header.s=fm2 header.b=SidS4q6i;
dkim=pass header.d=messagingengine.com header.s=fm2 header.b=lcg5ErnA;
spf=pass (mx1.freebsd.org: domain of tech-lists@zyxst.net designates
66.111.4.27 as permitted sender) smtp.mailfrom=tech-lists@zyxst.net
X-Spamd-Result: default: False [-9.24 / 15.00]; ARC_NA(0.00)[];
RCVD_VIA_SMTP_AUTH(0.00)[];
R_DKIM_ALLOW(-0.20)[zyxst.net:s=fm2,messagingengine.com:s=fm2];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
R_SPF_ALLOW(-0.20)[+ip4:66.111.4.27];
TO_MATCH_ENVRCPT_ALL(0.00)[];
MIME_GOOD(-0.20)[multipart/signed,text/plain];
PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org];
TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1];
NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4];
DMARC_NA(0.00)[zyxst.net];
MX_GOOD(-0.01)[cached: in2-smtp.messagingengine.com];
DKIM_TRACE(0.00)[zyxst.net:+,messagingengine.com:+];
NEURAL_HAM_SHORT(-1.00)[-0.999,0]; SIGNED_PGP(-2.00)[];
FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+];
RCVD_TLS_LAST(0.00)[];
ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US];
IP_SCORE(-3.54)[ip: (-9.59), ipnet: 66.111.4.0/24(-4.60), asn: 11403(-3.42),
country: US(-0.07)];
RCVD_IN_DNSWL_LOW(-0.10)[27.4.111.66.list.dnswl.org : 127.0.5.1]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 23 Mar 2019 18:57:31 -0000
--oyUTqETQ0mS9luUI
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Sat, Mar 23, 2019 at 09:31:17PM +0800, Frank Leonhardt wrote:
>As to your question, what I've done is run a ZFS pool and simply send a sn=
apshot.=20
That's a good idea. I presumed I'd be stuck just with UFS. My most
favoured way of doing this is having a zfs-vol backed vm on a freebsd
host but it's not my choice.
> I can't remember now if I made it boot from ZFS as I prefer to boot from =
UFS=20
this is me. I like to boot off something fast and simple.
> and use ZFS for the rest. But I think I did this with Azure exactly becau=
se =20
> I wanted to copy the disk around. It was early last year so you can't=20
> quite me on it.
>
> If you're not sure about sending datasets on ZFS then ask.
Yes please how are you invoking sending a dataset in this context? For
example, are you using compression for the data stream? If so, what
sort? Is there anything else pertinent to the azure context when using
zfs send?
thanks,
--=20
J.
--oyUTqETQ0mS9luUI
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEE8n3tWhxW11Ccvv9/s8o7QhFzNAUFAlyWgYoACgkQs8o7QhFz
NAVK4A//WrRcPcRoZi5QsoKAvazL2rc5LSz2HyVteOP6EJIjGLp+wH/5OZD8Ywj2
hlKZ50Vg30PvgYT/UvFuVswjcxZ8nqjsizYjfcsDL2MHDLzsAP2vPgiaMs0w/vd4
gzMyjplVPIb/tBDBZJ4yl3bSx1a04VX78pgXw9toJJMO0xDtjJIK8cfDe5znPaFt
vHoV2mR8RgeVTT0o7WzLf5W6R3MLj6UkCwjzI2BAd6dPCSfBGD6lEuv8/wUWbgLH
C5HsqSxdswjdjxvhFNTZY//pHHTBnPVjiR9UFqtCousxvDvbXTUh9Bjs3Mvw8HMy
Dg53voaGktm6fxmA+cDQK9+tnATf0LOu8sQ1klFa2/w634Z2LD7Nb0BvH/1crzTn
POwMGvonXLqpUOyOUVpeuxr3EuPXqfQ+hyjY7v1foQQf6OB2asiCzdM+prsfQUyd
tN62j/nAFAQ9bHuvfm9aso9NqTB+D5LgavqBxrnvO3jf/6maGASnY16kVxGcD69i
kVTeAfUzfbRJCSSQBaNQoFWJbqK70NoZl3T/HUM2um2RjLPipkYMttsRz2J5u8Uj
Su31LnMiSB56JHxPAVN76FF8T2Fw4alrM6zjeVkefN26WlolXnTw9sSPgIxOHT5K
42GJ54ERnk/M9RpAFoVQmfe8wniF4RMIM/6EnOMkPPnuAX3Pn+U=
=qIfD
-----END PGP SIGNATURE-----
--oyUTqETQ0mS9luUI--
From owner-freebsd-virtualization@freebsd.org Sat Mar 23 21:35:56 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id DC757155D802
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Sat, 23 Mar 2019 21:35:55 +0000 (UTC)
(envelope-from ctuffli@gmail.com)
Received: from mail-ot1-f43.google.com (mail-ot1-f43.google.com
[209.85.210.43])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id DE44083E9C
for <freebsd-virtualization@freebsd.org>; Sat, 23 Mar 2019 21:35:54 +0000 (UTC)
(envelope-from ctuffli@gmail.com)
Received: by mail-ot1-f43.google.com with SMTP id d24so4879501otl.11
for <freebsd-virtualization@freebsd.org>; Sat, 23 Mar 2019 14:35:54 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:references:in-reply-to:from:date
:message-id:subject:to;
bh=u6lOTcyNnam9+Vs17xD2NKpN3YAhECJThGzDBAVSE6g=;
b=CAeZ4IQyhEWipspTQCRPZf2vdr5LLzSd13VUJ6WJKye4Va2ytQS5DgwsmCFVx1p/VP
esceUxDjLKYXH121AZpjFlImMMfWbbr9mhPHSimkIoN1+Hs7FCbzIXREGR6HsA4AvExV
/QigjOlzxBOF7YCcpbU4P3sGo3Cyoc8dOsfvrPprKm+/MgswkQtox5OzPM3yhPqTFSQ1
/7lkmTmr29kp199iLHwkLNcQ3IlMtXnBbl/WFV2qqp0srLMqSWDWa+elqytzbDBuQWL3
Y88wRj13a7N9q5e5jljjXe5oBC9HPr57NP+M48aE1k65DCN/Z7dP8ydj13sdFb7wxCkC
DbiA==
X-Gm-Message-State: APjAAAUJ7xkZ4hXktxaFVZWX4lbWi+ySykcg/8VRwQhcEpuZN41PEfqW
w5MFBt2Iz0Tygv4KPc/6W34NM5yp
X-Google-Smtp-Source: APXvYqznNVLEHkmmgRqypHH/LGME6qIpS9vy2OpRl8uWOJvDIa5pZYZPZN2fo8UyhV9vp2oL+1TuUw==
X-Received: by 2002:a9d:1b2f:: with SMTP id l44mr11984620otl.217.1553376947901;
Sat, 23 Mar 2019 14:35:47 -0700 (PDT)
Received: from mail-ot1-f44.google.com (mail-ot1-f44.google.com.
[209.85.210.44])
by smtp.gmail.com with ESMTPSA id q6sm4504167oia.4.2019.03.23.14.35.47
for <freebsd-virtualization@freebsd.org>
(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
Sat, 23 Mar 2019 14:35:47 -0700 (PDT)
Received: by mail-ot1-f44.google.com with SMTP id m10so4930945otp.2
for <freebsd-virtualization@freebsd.org>; Sat, 23 Mar 2019 14:35:47 -0700 (PDT)
X-Received: by 2002:a9d:75cc:: with SMTP id c12mr8057084otl.77.1553376947435;
Sat, 23 Mar 2019 14:35:47 -0700 (PDT)
MIME-Version: 1.0
References: <CAKAYmMKd5o=f8Z5uzF6pQnZqWLzRLAL3U7iM5inuodQs92pudA@mail.gmail.com>
In-Reply-To: <CAKAYmMKd5o=f8Z5uzF6pQnZqWLzRLAL3U7iM5inuodQs92pudA@mail.gmail.com>
From: Chuck Tuffli <chuck@freebsd.org>
Date: Sat, 23 Mar 2019 14:35:36 -0700
X-Gmail-Original-Message-ID: <CAKAYmML9jAc4WfJ8NP9iiFWY4R7DgU9SKyF_ZiNxt5FvSXQfhA@mail.gmail.com>
Message-ID: <CAKAYmML9jAc4WfJ8NP9iiFWY4R7DgU9SKyF_ZiNxt5FvSXQfhA@mail.gmail.com>
Subject: Re: emulated PCI device BAR size
To: freebsd-virtualization@freebsd.org
Content-Type: text/plain; charset="UTF-8"
X-Rspamd-Queue-Id: DE44083E9C
X-Spamd-Bar: ----
Authentication-Results: mx1.freebsd.org;
spf=pass (mx1.freebsd.org: domain of ctuffli@gmail.com designates
209.85.210.43 as permitted sender) smtp.mailfrom=ctuffli@gmail.com
X-Spamd-Result: default: False [-4.32 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[];
R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; TO_DN_NONE(0.00)[];
RCVD_COUNT_THREE(0.00)[4];
MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com];
NEURAL_HAM_SHORT(-0.99)[-0.989,0];
FORGED_SENDER(0.30)[chuck@freebsd.org,ctuffli@gmail.com];
MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[];
FREEMAIL_ENVFROM(0.00)[gmail.com];
ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US];
FROM_NEQ_ENVFROM(0.00)[chuck@freebsd.org,ctuffli@gmail.com];
TO_DOM_EQ_FROM_DOM(0.00)[]; ARC_NA(0.00)[];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0];
MIME_GOOD(-0.10)[text/plain];
PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org];
DMARC_NA(0.00)[freebsd.org]; RCPT_COUNT_ONE(0.00)[1];
IP_SCORE(-1.32)[ip: (-0.55), ipnet: 209.85.128.0/17(-3.86), asn: 15169(-2.13),
country: US(-0.07)];
RCVD_IN_DNSWL_NONE(0.00)[43.210.85.209.list.dnswl.org : 127.0.5.0];
RCVD_TLS_LAST(0.00)[];
RWL_MAILSPIKE_POSSIBLE(0.00)[43.210.85.209.rep.mailspike.net : 127.0.0.17]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 23 Mar 2019 21:35:56 -0000
On Mon, Mar 18, 2019 at 10:15 AM Chuck Tuffli <chuck@freebsd.org> wrote:
>
> In debugging why Windows doesn't like bhyve's NVMe device emulation,
> another developer mentioned that the NVMe specification defines the
> minimum size of BAR[0] to be 16K bytes. And while most OS don't
> enforce this requirement, evidently, Windows does.
>
> On the surface, a "minimum BAR size" sounds odd, but the MMIO BAR in
> NVMe includes the doorbell registers for each queue. And since the
> maximum number of queues is implementation specific, there isn't a
> fixed size for the BAR.
>
> So my question is how to best fix this. The easiest would be to add a
> check to the BAR size calculation in pci_nvme.c along the lines of:
> #define NVME_MMIO_SPACE_MIN (1 << 14)
> ...
> pci_membar_sz = MAX( <calculation based on queue count>,
> NVME_MMIO_SPACE_MIN);
I went down this route and have a patch up for review
(https://reviews.freebsd.org/D19676) if anyone is interested.
--chuck
From owner-freebsd-virtualization@freebsd.org Sat Mar 23 21:59:22 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 10768155E2B8;
Sat, 23 Mar 2019 21:59:22 +0000 (UTC)
(envelope-from rebecca@bluestop.org)
Received: from muon.bluestop.org (muon.bluestop.org [65.103.231.193])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
server-signature RSA-PSS (4096 bits))
(Client did not present a certificate)
by mx1.freebsd.org (Postfix) with ESMTPS id E1F03849E8;
Sat, 23 Mar 2019 21:59:20 +0000 (UTC)
(envelope-from rebecca@bluestop.org)
Received: from muon.bluestop.org (localhost [127.0.0.1])
by muon.bluestop.org (Postfix) with ESMTP id 8C381C0455;
Sat, 23 Mar 2019 16:00:19 -0600 (MDT)
Received: from muon.bluestop.org ([127.0.0.1])
by muon.bluestop.org (muon.bluestop.org [127.0.0.1]) (amavisd-new, port 10026)
with ESMTP id gsYg5jBhqRNa; Sat, 23 Mar 2019 16:00:18 -0600 (MDT)
Received: from photon.int.bluestop.org (unknown [65.103.231.197])
(using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by muon.bluestop.org (Postfix) with ESMTPSA;
Sat, 23 Mar 2019 16:00:18 -0600 (MDT)
Subject: Re: Updating uefi-edk2-bhyve
To: D Scott Phillips <d.scott.phillips@intel.com>
Cc: Larry Rosenman <ler@lerctr.org>,
owner-freebsd-virtualization@freebsd.org, freebsd-virtualization@freebsd.org
References: <86muln68ld.fsf@intel.com>
<1fe3ca3f-be70-99db-e7c0-35c9194c97e4@bluestop.org>
<7e84fd01c3f46268c26f9bab8b9fb9bc@lerctr.org>
<33fcf111-fe00-4ec7-8a2f-7c53246d756f@Spark> <86k1gqanx8.fsf@intel.com>
<8c63eb87-e3b8-2365-2eaf-a6e36424407c@bluestop.org>
<86ftremzar.fsf@intel.com>
<b2f90a4c-d73f-6797-aa1e-62161ad011f5@bluestop.org>
From: Rebecca Cran <rebecca@bluestop.org>
Message-ID: <4fb370c4-0358-8ca7-67ee-c98fd83d2740@bluestop.org>
Date: Sat, 23 Mar 2019 15:59:11 -0600
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101
Thunderbird/60.5.3
MIME-Version: 1.0
In-Reply-To: <b2f90a4c-d73f-6797-aa1e-62161ad011f5@bluestop.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US
X-Rspamd-Queue-Id: E1F03849E8
X-Spamd-Bar: ----
X-Spamd-Result: default: False [-4.97 / 15.00]; ARC_NA(0.00)[];
RCVD_VIA_SMTP_AUTH(0.00)[];
R_DKIM_ALLOW(-0.20)[bluestop.org:s=mail];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+mx];
NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain];
IP_SCORE(-0.99)[ipnet: 65.100.0.0/14(-4.88), asn: 209(-0.02), country:
US(-0.07)]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[4];
TO_MATCH_ENVRCPT_SOME(0.00)[];
DKIM_TRACE(0.00)[bluestop.org:+];
DMARC_POLICY_ALLOW(-0.50)[bluestop.org,quarantine];
MX_GOOD(-0.01)[mail.bluestop.org];
NEURAL_HAM_SHORT(-0.97)[-0.966,0]; FROM_EQ_ENVFROM(0.00)[];
MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[];
ASN(0.00)[asn:209, ipnet:65.100.0.0/14, country:US];
MID_RHS_MATCH_FROM(0.00)[]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 23 Mar 2019 21:59:22 -0000
On 3/22/19 4:29 PM, Rebecca Cran via freebsd-virtualization wrote:
> On 3/22/19 2:25 PM, D Scott Phillips wrote:
>
>> Hmm, I guess it might be some diference in the code generation between
>> gcc 4.8 and gcc 5.
>
>
> I've just tested switching from gcc 4.8 to 8.3.0 and everything seems
> to work fine - both build and runtime - so I think it may be more
> productive to upgrade and use that instead, since it's a supported
> version (gcc 4.8 is pretty old now). Would you agree?
I've submitted a couple of merge requests to update your edk2 tree and
port for gcc 8:
https://gitlab.com/scott-ph/edk2/merge_requests/1
https://gitlab.com/scott-ph/freebsd-ports/merge_requests/1
--
Rebecca Cran
From owner-freebsd-virtualization@freebsd.org Sat Mar 23 22:07:44 2019
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.ysv.freebsd.org (Postfix) with ESMTP id 68038155E5EB
for <freebsd-virtualization@mailman.ysv.freebsd.org>;
Sat, 23 Mar 2019 22:07:44 +0000 (UTC)
(envelope-from ctuffli@gmail.com)
Received: from mail-ot1-f42.google.com (mail-ot1-f42.google.com
[209.85.210.42])
(using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
(Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 8F39A84E4E
for <freebsd-virtualization@freebsd.org>; Sat, 23 Mar 2019 22:07:43 +0000 (UTC)
(envelope-from ctuffli@gmail.com)
Received: by mail-ot1-f42.google.com with SMTP id x8so4926710otg.7
for <freebsd-virtualization@freebsd.org>; Sat, 23 Mar 2019 15:07:43 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
bh=y/2fPSg+1/qeeQ2PHA59wiw9DdJBw5QbpkJBqLR7gGs=;
b=IztBze2EjGEsZ8K9wnY5+b9WmnC68T+m9bHX6w7t4QfqUx3GcWw8+QV9xXt3FZS+SG
j8hs/Nh1maHtNxp8a7vF7KNsVHncnUNAdtpE1ZetGH0OENW84hpIn8rxn78zq0247K0m
wSdluvNwO+lm3OfPY0hgsQyozJ6iF5JljgBIR1Kaf05ppw3dpSJmciuajr0q1DgB8kiF
r/IFttZt8QT3/nbdcxVDd2WEMnDHrCCU7sUmhTKrYOYcQDQ9i7Wzgz2lBH36n6setKhI
a+VCaKbktVwnx733EHnimxwMSAR/lFi0rtcZ5H4U43BOTqe7XgtLET4I2g5WHiIjiYUM
crbQ==
X-Gm-Message-State: APjAAAX6wSumiWSpfq4Qlx7retWi8trgBIlKGw3l3F8zBJjA5s55JzeW
42nkfzdXktOfc8xZOV8XdQGdGj5A
X-Google-Smtp-Source: APXvYqx+1iWOI7DwHeizsBmsHWXBHMIM7HLU7Kla5P3b8p+ZvOXaMP4YKZ3RtXVpoW6oyh+HtFWilA==
X-Received: by 2002:a9d:73c2:: with SMTP id m2mr11811049otk.338.1553378857289;
Sat, 23 Mar 2019 15:07:37 -0700 (PDT)
Received: from mail-oi1-f172.google.com (mail-oi1-f172.google.com.
[209.85.167.172])
by smtp.gmail.com with ESMTPSA id 65sm4678723ota.35.2019.03.23.15.07.36
for <freebsd-virtualization@freebsd.org>
(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
Sat, 23 Mar 2019 15:07:36 -0700 (PDT)
Received: by mail-oi1-f172.google.com with SMTP id v7so4299095oie.8
for <freebsd-virtualization@freebsd.org>; Sat, 23 Mar 2019 15:07:36 -0700 (PDT)
X-Received: by 2002:aca:7541:: with SMTP id q62mr6803313oic.49.1553378856519;
Sat, 23 Mar 2019 15:07:36 -0700 (PDT)
MIME-Version: 1.0
From: Chuck Tuffli <chuck@freebsd.org>
Date: Sat, 23 Mar 2019 15:07:25 -0700
X-Gmail-Original-Message-ID: <CAKAYmMLkdEsq_=yu4=xGCKb=-Lije-xSOhCpkEMrXYON4xiyXg@mail.gmail.com>
Message-ID: <CAKAYmMLkdEsq_=yu4=xGCKb=-Lije-xSOhCpkEMrXYON4xiyXg@mail.gmail.com>
Subject: bhyve threading model?
To: freebsd-virtualization@freebsd.org
Content-Type: text/plain; charset="UTF-8"
X-Rspamd-Queue-Id: 8F39A84E4E
X-Spamd-Bar: ---
Authentication-Results: mx1.freebsd.org;
spf=pass (mx1.freebsd.org: domain of ctuffli@gmail.com designates
209.85.210.42 as permitted sender) smtp.mailfrom=ctuffli@gmail.com
X-Spamd-Result: default: False [-3.15 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[];
RWL_MAILSPIKE_GOOD(0.00)[42.210.85.209.rep.mailspike.net : 127.0.0.18];
R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; TO_DN_NONE(0.00)[];
RCVD_COUNT_THREE(0.00)[4];
MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com];
NEURAL_HAM_SHORT(-0.93)[-0.925,0];
FORGED_SENDER(0.30)[chuck@freebsd.org,ctuffli@gmail.com];
MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[];
FREEMAIL_ENVFROM(0.00)[gmail.com];
ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US];
FROM_NEQ_ENVFROM(0.00)[chuck@freebsd.org,ctuffli@gmail.com];
TO_DOM_EQ_FROM_DOM(0.00)[]; ARC_NA(0.00)[];
NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[];
TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0];
MIME_GOOD(-0.10)[text/plain];
PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org];
DMARC_NA(0.00)[freebsd.org]; RCPT_COUNT_ONE(0.00)[1];
SUBJECT_ENDS_QUESTION(1.00)[];
RCVD_IN_DNSWL_NONE(0.00)[42.210.85.209.list.dnswl.org : 127.0.5.0];
IP_SCORE(-1.21)[ipnet: 209.85.128.0/17(-3.86), asn: 15169(-2.14), country:
US(-0.07)]; RCVD_TLS_LAST(0.00)[]
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
<freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>,
<mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sat, 23 Mar 2019 22:07:44 -0000
In the process of refactoring some of bhyve's NVMe emulation code, I'm
noticing there are a handful of mutexes. But this leads me to believe
I don't understand how the emulated devices work. Is there a paper /
video / code comment / etc. somewhere that explains bhyve's threading
model, specifically around the emulated devices? TIA
--chuck