Date: Fri, 16 Feb 2024 12:45:47 -0500 From: Paul Procacci <pprocacci@gmail.com> To: Chuck Tuffli <chuck@tuffli.net> Cc: Matthew Grooms <mgrooms@shrew.net>, FreeBSD virtualization <freebsd-virtualization@freebsd.org> Subject: Re: bhyve disk performance issue Message-ID: <CAFbbPuhqtUcDnt=7PdgOqspr4b2T3zyFuWF8fuhNd5G6VZN=%2Bw@mail.gmail.com> In-Reply-To: <28ea168c-1211-4104-b8b4-daed0e60950d@app.fastmail.com> References: <6a128904-a4c1-41ec-a83d-56da56871ceb@shrew.net> <28ea168c-1211-4104-b8b4-daed0e60950d@app.fastmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000adcbaa06118350cb
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
On Fri, Feb 16, 2024 at 12:43=E2=80=AFPM Chuck Tuffli <chuck@tuffli.net> wr=
ote:
> On Fri, Feb 16, 2024, at 9:19 AM, Matthew Grooms wrote:
>
> Hi All,
>
>
> I'm in the middle of a project that involves building out a handful of
> servers to host virtual Linux instances. Part of that includes testing
> bhyve to see how it performs. The intent is to compare host storage optio=
ns
> such as raw vs zvol block devices and ufs vs zfs disk images using hardwa=
re
> raid vs zfs managed disks. It would also involve
>
>
> =E2=80=A6
>
> Here is a list of a few other things I'd like to try:
>
> 1) Wiring guest memory ( unlikely as it's 32G of 256G )
> 2) Downgrading the host to 13.2-RELEASE
>
>
> FWIW we recently did a similar exercise and saw significant performance
> differences on ZFS backed disk images when comparing 14.0 and 13.2. We
> didn=E2=80=99t have time to root cause the difference, so it could simply=
be some
> tuning difference needed for 14.
>
> =E2=80=94chuck
>
I myself am actually doing something very very similar.
I was seeing atrocious disk performance until I set the disk type to nvme.
Now it's screaming fast.
disk0_type=3D"nvme"
Not sure what yours is set at, but it might be worth looking into.
~Paul
--=20
__________________
:(){ :|:& };:
--000000000000adcbaa06118350cb
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div><div dir=3D"ltr"><br></div><br><div class=3D"gmail_qu=
ote"><div dir=3D"ltr" class=3D"gmail_attr">On Fri, Feb 16, 2024 at 12:43=E2=
=80=AFPM Chuck Tuffli <<a href=3D"mailto:chuck@tuffli.net">chuck@tuffli.=
net</a>> wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"marg=
in:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1e=
x"><div class=3D"msg-6751001795835766023"><u></u><div><div>On Fri, Feb 16, =
2024, at 9:19 AM, Matthew Grooms wrote:<br></div><blockquote type=3D"cite" =
id=3D"m_-6751001795835766023qt"><p>Hi All,<br></p><p><br></p><div>I'm i=
n the middle of a project that involves building out a
handful of servers to host virtual Linux instances. Part of that
includes testing bhyve to see how it performs. The intent is to
compare host storage options such as raw vs zvol block devices and
ufs vs zfs disk images using hardware raid vs zfs managed disks.
It would also involve<br></div><p><br></p></blockquote><div>=E2=80=A6=
<br></div><blockquote type=3D"cite" id=3D"m_-6751001795835766023qt"><p><spa=
n style=3D"color:rgb(27,30,32)">Here is a list of a few other things I'=
d like to try:</span><br></p><div> <br></div><div> 1) Wiring guest memory (=
unlikely as it's 32G of 256G )<br></div><div> 2) Downgrading the host =
to 13.2-RELEASE<br></div></blockquote><div><br></div><div>FWIW we recently =
did a similar exercise and saw significant performance differences on ZFS b=
acked disk images when comparing 14.0 and 13.2. We didn=E2=80=99t have time=
to root cause the difference, so it could simply be some tuning difference=
needed for 14.=C2=A0<br></div><div><br></div><div>=E2=80=94chuck</div></di=
v></div></blockquote></div><br clear=3D"all"></div><div>I myself am actuall=
y doing something very very similar.</div><div>I was seeing atrocious disk =
performance until I set the disk type to nvme.<br></div><div>Now it's s=
creaming fast.<br><br>disk0_type=3D"nvme"<br><br></div><div>Not s=
ure what yours is set at, but it might be worth looking into.</div><div><br=
></div><div>~Paul<br></div><div><span class=3D"gmail_signature_prefix">-- <=
/span><br><div dir=3D"ltr" class=3D"gmail_signature">__________________<br>=
<br>:(){ :|:& };:</div></div></div>
--000000000000adcbaa06118350cb--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFbbPuhqtUcDnt=7PdgOqspr4b2T3zyFuWF8fuhNd5G6VZN=%2Bw>