From nobody Sat Jul 13 22:06:10 2024 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WM2bz5RZhz5RDgZ for ; Sat, 13 Jul 2024 22:06:23 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WM2bz21cJz4K5F for ; Sat, 13 Jul 2024 22:06:23 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pl1-x62b.google.com with SMTP id d9443c01a7336-1fb0d7e4ee9so21005175ad.3 for ; Sat, 13 Jul 2024 15:06:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1720908382; x=1721513182; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=Kwpc8i3irvphvXp1O1pjQmCocJAz5PnxZ2pVTzwNuIM=; b=a1vthkMfJN/oyyrZYBMHGgoxVrlmXhWHrz37miC+K8tn9Xqw65f5keF1tt0h8ofXSo ep2iNRYACX0CplJrq+bSsN8CVr7PShOXJsHii0NZACmwie9yw1FMQb7o3ai5tKVUm0uf 13zT0+JN8QFJKbAWjbCldvrHYVaRQItWDXRgR+mKgoVLnThynBibUBKIVi542IQ2D6Ij 44ELNgU/jbhj/DIHmm11QB+cdVSJicF+gV+EJTvUd5HIihjrSKOR4jPObsAvzfnsKvBA YW7vmeNNUPLuHJzwM8GxPPLh2R66AaxoKVGcM05NWhNNiDLitamRDkHlQ2tt+7ildQ9h 9lxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720908382; x=1721513182; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Kwpc8i3irvphvXp1O1pjQmCocJAz5PnxZ2pVTzwNuIM=; b=kBJqlSmtbDUVsn9HohcWPty/N/3Oz3hcvP3yf6oUURZl+1HI/cuzAa9vcmIwn4OMJE h1gUcQorQdlSTV1BWDJva6hX5FF38RQMva9A5uGAoU2k8HgRW3HdnRYDFKeylVgTOXdN WdRXe66/au6e3MJzSBL3vh+lMFF0+TRtFlmQjr9jCUsCm43YH6CSTCVUbnqB3YLFYaPY OzgzKjwVspJxdUI6piqwdfChFU4WEamHfhkrOof3dNSwFFt1w7I+qKMyR1r3AxfbGcN4 i6wQGJzH+ocafWSF2x7YpOCkWteN0M4/4ZmJYJGBUrccAaW6bSlF3Eya9NGuwc7H1KzJ Ajsw== X-Forwarded-Encrypted: i=1; AJvYcCWEbzI0zxk2Uxf5AnHxqTCNTppW699H3CrnbkXRryMVf3wAvBLMDdEwl70FPQ8JZuzAEFCYvG4JaNI9LMjHomJZ/BQFs0RppcevE28= X-Gm-Message-State: AOJu0YxlEgaWOIqbJIdki39ZMvoCve8BMufVhjRouJ7PJwafvgsRWWTy sGs07KeCh48Ua5XacCPcxI8wi/GAz6LXG69mgHn2FCKncU8WIyQ5G1EPlFenI/GPYaIo4EiNY44 oE3A8OiuYKPJRgUKdFxmH1b790bjEdPMXCWE2gQ== X-Google-Smtp-Source: AGHT+IGAzw741Cxo67ryoklVBilvA6IEvpaBpqqeYj3wx/mt+bqW5IIq3O0HgbhBaZWX8sA4oUvNxlLLCEEd0p/sLqg= X-Received: by 2002:a17:903:1110:b0:1f8:6d9e:fa9c with SMTP id d9443c01a7336-1fbb6d34b01mr142659455ad.6.1720908381634; Sat, 13 Jul 2024 15:06:21 -0700 (PDT) List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Sat, 13 Jul 2024 16:06:10 -0600 Message-ID: Subject: Re: Is anyone working on VirtFS (FUSE over VirtIO) To: David Chisnall Cc: Alan Somers , FreeBSD Hackers Content-Type: multipart/alternative; boundary="000000000000c75fa9061d283433" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Queue-Id: 4WM2bz21cJz4K5F --000000000000c75fa9061d283433 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hey David, You might want to check out https://reviews.freebsd.org/D45370 which has the testing framework as well as hints at other work that's been done for virtiofs by Emil Tsalapatis. It looks quite interesting. Anything he's done that's at odds with what I've said just shows where my analysis was flawed :) This looks quite promising, but I've not had the time to look at it in detail yet. Warner On Sat, Jul 13, 2024 at 2:44=E2=80=AFAM David Chisnall wrote: > On 31 Dec 2023, at 16:19, Warner Losh wrote: > > > Yea. The FUSE protocol is going to be the challenge here. For this to be > useful, the VirtioFS support on the FreeBSD needs to be 100% in the > kernel, since you can't have userland in the loop. This isn't so terrible= , > though, since our VFS interface provides a natural breaking point for > converting the requests into FUSE requests. The trouble, I fear, is a > mismatch between FreeBSD's VFS abstraction layer and Linux's will cause > issues (many years ago, the weakness of FreeBSD VFS caused problems for a > company doing caching, though things have no doubt improved from those > days). Second, there's a KVM tie-in for the direct mapped pages between t= he > VM and the hypervisor. I'm not sure how that works on the client (FreeBSD= ) > side (though the description also says it's mapped via a PCI bar, so mayb= e > the VM OS doesn't care). > > > From what I can tell from a little bit of looking at the code, our FUSE > implementation has a fairly cleanly abstracted layer (in fuse_ipc.c) for > handling the message queue. For VirtioFS, it would 'just' be necessary t= o > factor out the bits here that do uio into something that talked to a Virt= IO > ring. I don=E2=80=99t know what the VFS limitations are, but since the p= rotocol > for VirtioFS is the kernel <-> userspace protocol for FUSE, it seems that > any functionality that works with FUSE filesystems in userspace would wor= k > with VirtioFS filesystems. > > The shared buffer cache bits are nice, but are optional, so could be done > in a later version once the basic functionality worked. > > David > > --000000000000c75fa9061d283433 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hey David,

You might want to= check out=C2=A0 https://rev= iews.freebsd.org/D45370 which has the testing framework as well as hint= s at other work that's been done for virtiofs=C2=A0by Emil=C2=A0Tsalapa= tis. It looks quite interesting. Anything he's done that's at odds = with what I've said just shows where my analysis was flawed :) This loo= ks quite promising, but I've not had the time to look at it in detail y= et.

Warner

On Sat, Jul 13, 2024 at 2:44=E2=80= =AFAM David Chisnall <theraven@f= reebsd.org> wrote:
On 31 Dec 2023, at 16:19, Warner Losh <imp@bsdimp.com> wrote:

Yea. The FUSE protocol i= s going to be the challenge here. For this to be useful, the VirtioFS=C2=A0= support on=C2=A0the FreeBSD=C2=A0 needs to be 100% in the kernel, since you= can't have userland in the loop. This isn't so terrible, though, s= ince our VFS interface provides a natural breaking point for converting the= requests into FUSE requests. The trouble, I fear, is a mismatch between Fr= eeBSD's VFS abstraction layer and Linux's will cause issues (many y= ears ago, the weakness of FreeBSD VFS caused problems for a company doing c= aching, though things have no doubt improved from those days). Second, ther= e's a KVM tie-in for the direct mapped pages between the VM and the hyp= ervisor. I'm not sure how that works on the client (FreeBSD) side (thou= gh the description also says it's mapped via a PCI bar, so maybe the VM= OS doesn't care).

From what I c= an tell from a little bit of looking at the code, our FUSE implementation h= as a fairly cleanly abstracted layer (in fuse_ipc.c) for handling the messa= ge queue.=C2=A0 For VirtioFS, it would 'just' be necessary to facto= r out the bits here that do uio into something that talked to a VirtIO ring= .=C2=A0 I don=E2=80=99t know what the VFS limitations are, but since the pr= otocol for VirtioFS is the kernel <-> userspace protocol for FUSE, it= seems that any functionality that works with FUSE filesystems in userspace= would work with VirtioFS filesystems.

The shared = buffer cache bits are nice, but are optional, so could be done in a later v= ersion once the basic functionality worked. =C2=A0

David

--000000000000c75fa9061d283433--