Date: Tue, 27 Feb 2024 13:21:43 +0300 From: Vitaliy Gusev <gusev.vitaliy@gmail.com> To: Matthew Grooms <mgrooms@shrew.net> Cc: virtualization@freebsd.org Subject: Re: bhyve disk performance issue Message-ID: <BE794E98-7B69-4626-BB66-B56F23D6A67E@gmail.com> In-Reply-To: <e45c95df-4858-48aa-a274-ba1bf8e599d5@shrew.net> References: <6a128904-a4c1-41ec-a83d-56da56871ceb@shrew.net> <28ea168c-1211-4104-b8b4-daed0e60950d@app.fastmail.com> <0ff6f30a-b53a-4d0f-ac21-eaf701d35d00@shrew.net> <6f6b71ac-2349-4045-9eaf-5c50d42b89be@shrew.net> <50614ea4-f0f9-44a2-b5e6-ebb33cfffbc4@shrew.net> <6a4e7e1d-cca5-45d4-a268-1805a15d9819@shrew.net> <f01a9bca-7023-40c0-93f2-8cdbe4cd8078@tubnor.net> <edb80fff-561b-4dc5-95ee-204e0c6d95df@shrew.net> <a07d070b-4dc1-40c9-bc80-163cd59a5bfc@Duedinghausen.eu> <e45c95df-4858-48aa-a274-ba1bf8e599d5@shrew.net>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_2DCCC81D-8089-4C12-92E2-6A195E662DF6 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi, > On 23 Feb 2024, at 18:37, Matthew Grooms <mgrooms@shrew.net> wrote: >=20 >> ... > The problem occurs when an image file is used on either ZFS or UFS. = The problem also occurs when the virtual disk is backed by a raw disk = partition or a ZVOL. This issue isn't related to a specific underlying = filesystem. >=20 Do I understand right, you ran testing inside VM inside guest VM on = ext4 filesystem? If so you should be aware about additional overhead in = comparison when you were running tests on the hosts. I would suggest to run fio (or even dd) on raw disk device inside VM, = i.e. without filesystem at all. Just do not forget do =E2=80=9Cecho 3 > = /proc/sys/vm/drop_caches=E2=80=9D in Linux Guest VM before you run = tests.=20 Could you also give more information about: 1. What results did you get (decode bonnie++ output)? 2. What results expecting? 3. VM configuration, virtio-blk disk size, etc. 4. Full command for tests (including size of test-set), bhyve, etc. 5. Did you pass virtio-blk as 512 or 4K ? If 512, probably you should = try 4K. 6. Linux has several read-ahead options for IO schedule, and it could = be related too. Additionally could also you play with =E2=80=9Csync=3Ddisabled=E2=80=9D = volume/zvol option? Of course it is only for write testing. =E2=80=94=E2=80=94 Vitaliy --Apple-Mail=_2DCCC81D-8089-4C12-92E2-6A195E662DF6 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"content-type" content=3D"text/html; = charset=3Dutf-8"></head><body style=3D"overflow-wrap: break-word; = -webkit-nbsp-mode: space; line-break: = after-white-space;">Hi,<div><br></div><div><div><br><blockquote = type=3D"cite"><div>On 23 Feb 2024, at 18:37, Matthew Grooms = <mgrooms@shrew.net> wrote:</div><br = class=3D"Apple-interchange-newline"><div><div><blockquote = type=3D"cite">...</blockquote>The problem occurs when an image file is = used on either ZFS or UFS. The problem also occurs when the virtual disk = is backed by a raw disk partition or a ZVOL. This issue isn't related to = a specific underlying = filesystem.<br><br></div></div></blockquote><div><br></div>Do I = understand right, you ran testing inside VM inside guest VM on = ext4 filesystem? If so you should be aware about additional overhead in = comparison when you were running tests on the = hosts.</div><div><br></div><div>I would suggest to run fio (or even dd) = on raw disk device inside VM, i.e. without filesystem at all. Just = do not forget do =E2=80=9C<span style=3D"font-family: Menlo; font-size: = 12px; background-color: rgb(231, 238, 238);">echo 3 > = /proc/sys/vm/drop_caches</span>=E2=80=9D in Linux Guest VM before you = run tests. </div><div><br></div><div>Could you also give more = information about:</div><div><br></div><div> 1. What results did = you get (decode bonnie++ output)?</div><div> 2. What results = expecting?</div><div> 3. VM configuration, virtio-blk disk size, = etc.</div><div> 4. Full command for tests (including size of = test-set), bhyve, etc.</div><div> 5. Did you pass virtio-blk as 512 = or 4K ? If 512, probably you should try 4K.</div><div> 6. Linux has = several read-ahead options for IO schedule, and it could be related = too.</div><div><br></div><div>Additionally could also you play with = =E2=80=9Csync=3Ddisabled=E2=80=9D volume/zvol option? Of course it is = only for write = testing.</div><div><br></div><div>=E2=80=94=E2=80=94</div><div>Vitaliy</di= v><div><br></div></div></body></html>= --Apple-Mail=_2DCCC81D-8089-4C12-92E2-6A195E662DF6--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?BE794E98-7B69-4626-BB66-B56F23D6A67E>