Date: Sun, 24 Aug 2025 19:20:00 -0700 From: Rick Macklem <rick.macklem@gmail.com> To: Rob Norris <robn@despairlabs.com> Cc: freebsd-hackers@freebsd.org Subject: Re: NFSv4.2 READ_PLUS support? Message-ID: <CAM5tNy49m9z3h5vtcqo3OFZ3O2epXU8nqCLWzuyt2ez60TxHPg@mail.gmail.com> In-Reply-To: <99aea3ba-73d3-4f0b-be2c-ae06752da4bb@app.fastmail.com> References: <CALXu0UfzyaZti2dSbZHPf_YXU88_2oCP98cXJPM6mQ9Fsr6ZuQ@mail.gmail.com> <CAM5tNy7x=AttZiu=ZLFtiQS3U2JD=gZOgjeGE-VoB1Obp9eZxQ@mail.gmail.com> <aKh3ZpxPYSIfMFfX@kib.kiev.ua> <CAM5tNy6WL93tVX18RsEUymYXON-W50i_-xm1CxrN6n0nbecmYg@mail.gmail.com> <CALXu0Uf%2B=GD8ZkZmN1v3s2GKp8Y5VDoiTO-cjay3Ypf1wmPMQw@mail.gmail.com> <CAM5tNy7nNCJBukrsC0MZJz9oQsGk5=CUaGhMJYkue6VL0yEmww@mail.gmail.com> <99aea3ba-73d3-4f0b-be2c-ae06752da4bb@app.fastmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Aug 24, 2025 at 6:49 PM Rob Norris <robn@despairlabs.com> wrote:
>
> On Sat, 23 Aug 2025, at 1:24 AM, Rick Macklem wrote:
> > There is a performance problem for ZFS related to holes and recently
> > written data (if vfs.zfs.dmu_offset_next_sync=1 recently created holes
> > will be found, but it really slows things down).
>
> To be clear, "recently created" here means that the record has not yet
> been written to disk, and so hasn't been through the compression stage
> in the IO pipeline yet, which is where things are turned into holes.
> dmu_offset_next_sync=1 is slow because it forces the entire transaction
> to be written down before doing the hole check. It's a very blunt
> instrument.
>
> There is some work happening to improve this situation (GH#16025[1] and
> followup[2]) that I hope to have time to assist with before the end of
> the year, so I think at least it's ok to plan for a future where ZFS is
> rather less slow at hole detection.
Yes, that sounds like good news.
Btw, here's the background to how I found out about dmu_offset_next_sync.
Garrett Wollman runs a bunch (around 20?) FreeBSD NFS servers using
ZFS at MIT (at least he did last I heard from him). He noticed some
copy_file_range() { called Copy in NFSv4.2 } were taking a long time
(one case was 25sec). I (maybe mistakenly) implemented copy_file_range(2)
to try and maintain sparseness when copying large files by using
SEEK_DATA/SEEK_HOLE, which was causing the slow copying.
(He found that setting dmu_offset_next_sync=0 fixed the problem for him.)
In this case, that is no problem, since copy_file_range(2) on FreeBSD
tries to maintain sparseness, but does not guarantee to do so.
rick
>
> > To get this right, it will take someone that really knows ZFS to
> > figure out how to do a VOP_READPLUS() well.
>
> I'd be happy to work on the ZFS side with a partner on the FreeBSD side.
>
> Cheers,
> Rob.
>
> 1. https://github.com/openzfs/zfs/pull/16025
> 2. https://github.com/openzfs/zfs/compare/master...rrevans:zfs:find_dirty
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAM5tNy49m9z3h5vtcqo3OFZ3O2epXU8nqCLWzuyt2ez60TxHPg>