Date: Thu, 19 Aug 2021 19:37:39 +1000 From: Peter Jeremy <peter@rulingia.com> To: freebsd-fs@freebsd.org Subject: ZFS on high-latency devices Message-ID: <YR4mY%2Bb6o7fBJqEN@server.rulingia.com>
index | next in thread | raw e-mail
[-- Attachment #1 --] I'm looking at backing up my local ZFS pools to a remote system without the remote system having access to the pool content. The options for the backup pools seem to be: a) Use OpenZFS native encryption with ZFS running on the remote system backed up using "zfs send --raw". b) Run ZFS over geli locally over geom_gate[1] to the remote system. The first approach removes RTT as an issue but requires that the local pools first be converted to native encryption - a process that seems to be generously defined as "very difficult". The second approach removes the need to encrypt the local pool but is far more sensitive to RTT issues and I've been unable to get a reasonable throughput. The main problems I've found are: * Even with a quite high write aggregation limit, I still get lots of relatively small writes. * Snapshot boundaries appear to wait for all queued writes to be flushed. I've found https://openzfs.org/wiki/ZFS_on_high_latency_devices but I can't get the procedure to work. "zfs send" of a zvol seems to bear very little resemblance to a "zfs send" of a "normal" filesystem. Sending a zvol, I can't get ndirty _down_ to the suggested 70-80%, whereas with (eg) my mail spool, I can't get ndirty _up_ to the suggested 70-80%. And most of the suggested adjustments are system- wide so the suggested changes are likely to adversely impact local ZFS performance. Does anyone have any suggestions as to a way forward? Either a reliable process to encrypt an existing pool or a way to improve throughput doing "zfs recv" to a pool with a high RTT. -- Peter Jeremy [-- Attachment #2 --] -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEE7rKYbDBnHnTmXCJ+FqWXoOSiCzQFAmEeJlxfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEVF QjI5ODZDMzA2NzFFNzRFNjVDMjI3RTE2QTU5N0EwRTRBMjBCMzQACgkQFqWXoOSi CzS6LRAAiBR1BHdBtlf7jnNmPuvMn+oaYwW1HeEmLzdcD/YrGwJN6vP0d7LV2AP9 ceY3E2SiRTN/VDcYAYuWBk4UzNbV16tYbs6f6qQpAYH0diF0lFzjYe+hsnvDZFMo NXskXqAyhGPqciDTn+lRbNZETDJbnonNVmGUw7HfzG7wTPZOyr/5odq+YXyep6s0 zmpFTheLStYKMXnbaxBazoV6ydzQ9y4Elc/aCqswMV29vHxDSp5j/fdDwG6ZQRFs B2+4AqzF9Ea8eASzDeX0qzwhiH8jXNDCP0Xp0uPusSnzYJBu2i/x42wbT7Dp9DVE oTIXYMYlX2WZTLrUyB7evgEdce/DLH01LP/ZPLTPSK+vHPtoLIV/FiVre0AqUO3j ilJli+O+OKeITyWFjI60bXOoWWJpl+zyirwCtC2viKwtwyj4nj1jTrSHK9d8dCbq KI///r89DivnU2tKgSIbvCgwtE8YUi4T5dc0VJzr2c+b0Vf39R3LbC+C4FdvperW AqU6kuN9sVgqsJK5kLUBh4buAWDGhpfNxllbEh9sGioEodnnEi1nM5MmWqPhLnAM Lz44UMeSJEu4+XoQt9MO5QObWfi86382oN9qA7khMIad1aEN4eqwLKH9f3QutfR5 kkCbG+BRq7StBkeV8YzqPBJe2BKgBDA3BUHZe5p+5/kIXe+OC+k= =CRZo -----END PGP SIGNATURE-----help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?YR4mY%2Bb6o7fBJqEN>