Date: Sun, 14 Jul 2024 08:11:26 +0100 From: David Chisnall <theraven@FreeBSD.org> To: Warner Losh <imp@bsdimp.com> Cc: Alan Somers <asomers@freebsd.org>, FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: Is anyone working on VirtFS (FUSE over VirtIO) Message-ID: <B926C3DA-A9F0-47EC-96BD-68EAA7A5A3B9@FreeBSD.org> In-Reply-To: <CANCZdfp0h%2BXhGoTk43e3ycZSg2Qh-6Fih9y5zfssGjRpoJaS-A@mail.gmail.com> References: <CAOtMX2gmc6L4H8L9107D84xofmd-idDgtVg8nkFkXPaPX1E8wg@mail.gmail.com> <A14C40DA-15EE-4777-B47F-2B342CE787EA@freebsd.org> <CANCZdfr7kKxTgBJ_LSKxAGsMUN9%2B=fiw1Fwy7Oxrc4G2mdSdYQ@mail.gmail.com> <E46B9396-7679-4C03-B144-BF4F9431F649@FreeBSD.org> <CANCZdfp0h%2BXhGoTk43e3ycZSg2Qh-6Fih9y5zfssGjRpoJaS-A@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_E777BA92-4526-42F6-81C1-95C53597DD06 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Wow, that looks incredibly useful. Not needing bhyve / qemu (nested, if = your main development is a VM) to test virtio drivers would be a huge = productivity win. =20 David > On 13 Jul 2024, at 23:06, Warner Losh <imp@bsdimp.com> wrote: >=20 > Hey David, >=20 > You might want to check out https://reviews.freebsd.org/D45370 which = has the testing framework as well as hints at other work that's been = done for virtiofs by Emil Tsalapatis. It looks quite interesting. = Anything he's done that's at odds with what I've said just shows where = my analysis was flawed :) This looks quite promising, but I've not had = the time to look at it in detail yet. >=20 > Warner >=20 > On Sat, Jul 13, 2024 at 2:44=E2=80=AFAM David Chisnall = <theraven@freebsd.org <mailto:theraven@freebsd.org>> wrote: >> On 31 Dec 2023, at 16:19, Warner Losh <imp@bsdimp.com = <mailto:imp@bsdimp.com>> wrote: >>>=20 >>> Yea. The FUSE protocol is going to be the challenge here. For this = to be useful, the VirtioFS support on the FreeBSD needs to be 100% in = the kernel, since you can't have userland in the loop. This isn't so = terrible, though, since our VFS interface provides a natural breaking = point for converting the requests into FUSE requests. The trouble, I = fear, is a mismatch between FreeBSD's VFS abstraction layer and Linux's = will cause issues (many years ago, the weakness of FreeBSD VFS caused = problems for a company doing caching, though things have no doubt = improved from those days). Second, there's a KVM tie-in for the direct = mapped pages between the VM and the hypervisor. I'm not sure how that = works on the client (FreeBSD) side (though the description also says = it's mapped via a PCI bar, so maybe the VM OS doesn't care). >>=20 >> =46rom what I can tell from a little bit of looking at the code, our = FUSE implementation has a fairly cleanly abstracted layer (in = fuse_ipc.c) for handling the message queue. For VirtioFS, it would = 'just' be necessary to factor out the bits here that do uio into = something that talked to a VirtIO ring. I don=E2=80=99t know what the = VFS limitations are, but since the protocol for VirtioFS is the kernel = <-> userspace protocol for FUSE, it seems that any functionality that = works with FUSE filesystems in userspace would work with VirtioFS = filesystems. >>=20 >> The shared buffer cache bits are nice, but are optional, so could be = done in a later version once the basic functionality worked. =20 >>=20 >> David >>=20 --Apple-Mail=_E777BA92-4526-42F6-81C1-95C53597DD06 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"content-type" content=3D"text/html; = charset=3Dutf-8"></head><body style=3D"overflow-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;">Wow, that = looks incredibly useful. Not needing bhyve / qemu (nested, if your = main development is a VM) to test virtio drivers would be a huge = productivity win. <div><br></div><div>David<br = id=3D"lineBreakAtBeginningOfMessage"><div><br><blockquote = type=3D"cite"><div>On 13 Jul 2024, at 23:06, Warner Losh = <imp@bsdimp.com> wrote:</div><br = class=3D"Apple-interchange-newline"><div><div dir=3D"ltr"><div>Hey = David,</div><div><br></div><div>You might want to check out <a = href=3D"https://reviews.freebsd.org/D45370">https://reviews.freebsd.org/D4= 5370</a> which has the testing framework as well as hints at other work = that's been done for virtiofs by Emil Tsalapatis. It looks = quite interesting. Anything he's done that's at odds with what I've said = just shows where my analysis was flawed :) This looks quite promising, = but I've not had the time to look at it in detail = yet.</div><div><br></div><div>Warner</div></div><br><div = class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Sat, Jul = 13, 2024 at 2:44=E2=80=AFAM David Chisnall <<a = href=3D"mailto:theraven@freebsd.org">theraven@freebsd.org</a>> = wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px = 0px 0px 0.8ex;border-left:1px solid = rgb(204,204,204);padding-left:1ex"><div>On 31 Dec 2023, at 16:19, Warner = Losh <<a href=3D"mailto:imp@bsdimp.com" = target=3D"_blank">imp@bsdimp.com</a>> wrote:<br><div><blockquote = type=3D"cite"><br><div><div = style=3D"font-family:Helvetica;font-size:12px;font-style:normal;font-varia= nt-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text= -indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-d= ecoration:none">Yea. The FUSE protocol is going to be the challenge = here. For this to be useful, the VirtioFS support on the = FreeBSD needs to be 100% in the kernel, since you can't have = userland in the loop. This isn't so terrible, though, since our VFS = interface provides a natural breaking point for converting the requests = into FUSE requests. The trouble, I fear, is a mismatch between FreeBSD's = VFS abstraction layer and Linux's will cause issues (many years ago, the = weakness of FreeBSD VFS caused problems for a company doing caching, = though things have no doubt improved from those days). Second, there's a = KVM tie-in for the direct mapped pages between the VM and the = hypervisor. I'm not sure how that works on the client (FreeBSD) side = (though the description also says it's mapped via a PCI bar, so maybe = the VM OS doesn't care).</div></div></blockquote><br></div><div>=46rom = what I can tell from a little bit of looking at the code, our FUSE = implementation has a fairly cleanly abstracted layer (in fuse_ipc.c) for = handling the message queue. For VirtioFS, it would 'just' be = necessary to factor out the bits here that do uio into something that = talked to a VirtIO ring. I don=E2=80=99t know what the VFS = limitations are, but since the protocol for VirtioFS is the kernel = <-> userspace protocol for FUSE, it seems that any functionality = that works with FUSE filesystems in userspace would work with VirtioFS = filesystems.</div><div><br></div><div>The shared buffer cache bits are = nice, but are optional, so could be done in a later version once the = basic functionality worked. = </div><div><br></div><div>David</div><div><br></div></div></blockquo= te></div> </div></blockquote></div><br></div></body></html>= --Apple-Mail=_E777BA92-4526-42F6-81C1-95C53597DD06--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?B926C3DA-A9F0-47EC-96BD-68EAA7A5A3B9>