Date: Sat, 28 Aug 2021 13:37:47 +1000 From: Peter Jeremy <peter@rulingia.com> To: Johannes Totz <johannes@jo-t.de> Cc: freebsd-fs@freebsd.org Subject: Re: ZFS on high-latency devices Message-ID: <YSmviwpiox9Z/QTw@server.rulingia.com> In-Reply-To: <sfmool$vq4$1@ciao.gmane.io> References: <YR4mY%2Bb6o7fBJqEN@server.rulingia.com> <sfmool$vq4$1@ciao.gmane.io>
next in thread | previous in thread | raw e-mail | index | archive | help
--sc46ZKMZH17jPmMr
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On 2021-Aug-20 00:16:28 +0100, Johannes Totz <johannes@jo-t.de> wrote:
>Do you have geli included in those perf tests? Any difference if you=20
>leave it out?
Yes, I mentioned geli (and I also have IPSEC, which I forgot to
mention). I haven't tried taking them out but dd(1) tests suggest
they aren't a problem.
>What's making the throughout slow? zfs issuing a bunch of small writes=20
>and then trying to read something (unrelated)? Is there just not enough=20
>data to be written to saturate the link?
At least from eyeballing gstat, there are basically no reads involved
in the zfs recv. The problem seems to be that writes aren't evenly
spread across all the vdevs, combined with very long delays associated
with flushing snapshots. I have considered instrumenting ggate{c,d}
to see if I can identify any issues.
>Totally random thought: there used to be a vdev cache (not sure if=20
>that's still around) that would inflate read requests to hopefully drag=20
>in more data that might be useful soon.
ZFS includes that functionality itself.
>Have you tried hastd?
I haven't but hastd also uses GEOM_GATE so I wouldn't expect
significantly different behaviour.
--=20
Peter Jeremy
--sc46ZKMZH17jPmMr
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQKTBAEBCgB9FiEE7rKYbDBnHnTmXCJ+FqWXoOSiCzQFAmEpr4VfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEVF
QjI5ODZDMzA2NzFFNzRFNjVDMjI3RTE2QTU5N0EwRTRBMjBCMzQACgkQFqWXoOSi
CzSVew//fZTheVP8ILPkpNBjJN7E5qEn3UPZFydrf0Og2oF/vHf9KYcWgM1f9w3m
c27H31kI1I0025/XXBE8fAv4XsjvhfaNI3bIzq4M9fNMMErPkGAqTd/396GJIoNW
LYcqpjOGGDWmZWtB1WwVOwLYsPPjOGpbt8cQ6YDf1USGWN0U9KmBzfBryZvGluaH
CVDQlxXQSr+mWfKE0dGhgfYPK5pwxv0WJRPTMLOnnyf2BeoLl5OxCfdTaO9mpB4v
5xoJCfHKh3udX9MJXtOaKboSUBw9t5pQhcluqPwoYoQk2c0kyawxUITfBGZqejGU
9BU3yD/l6m+8Mln7P8vE9oPhYnpE7ikZ3tW9YT0dmlYFjrjG0gQKysWc6G2RFJ7Q
2iRbOghT/Cc5wu59sNj/enfsCZ3E6apk7RhDXEzd8bNgwadD+vC3SF58lNyz0ItM
zRxvjR/1BGMeIFG/ksMDmPLnvLhCD6P/O+OJ2oYWiu2FYROFyUBkBPT7+OS9qgJg
59RLb8iT4ITixf3XF8mHHXfyl+OMaJSh/dSkOnRNPep84CiZdA1ncpdxtH/dDuOr
tpg4S1qs1AzE1zJB2+rpmc+YpJ9ueRZkqu2btcrL6twrUfTa0PWBu/yOhCsbq7oD
B43grMzLA/CJS7A4v9SRbsdqUsN9B7utXqKYgjuFFad2BkmRAXg=
=IEmC
-----END PGP SIGNATURE-----
--sc46ZKMZH17jPmMr--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?YSmviwpiox9Z/QTw>