Date: Fri, 18 Feb 2022 19:36:18 +1100 From: Peter Jeremy <peterj@freebsd.org> To: FreeBSD FS <freebsd-fs@freebsd.org>, "freebsd-geom@FreeBSD.org" <freebsd-geom@freebsd.org> Subject: Re: bio re-ordering Message-ID: <Yg9agkeypdDOwKWm@server.rulingia.com> In-Reply-To: <20220218014814.GJ97875@funkthat.com> References: <YfTCs7j3TPZFcFCD@server.rulingia.com> <YfTEj1KLhQhoR3xP@kib.kiev.ua> <CANCZdfoqQ3Ze%2BcMTsk_ho9x8hsSM9=fTavSao%2BUtwc2nSAEJpQ@mail.gmail.com> <Yfo3i9Yy/uCUpss1@server.rulingia.com> <CANCZdfqBQOvzMCrJxWq9GzqCKyK_AubBE1CxAW5FULnE7D_jrg@mail.gmail.com> <b75872f4-521b-5eab-68d0-4b1c04a10add@FreeBSD.org> <CANCZdfp=0rbBkr4SoXhvn7hrQniPQzTeZra2HGBwXDGsJjN8XQ@mail.gmail.com> <9848cde6-5c12-cdd4-e722-42fe26fa0349@FreeBSD.org> <Yf5IUCWW/tgI/Cse@server.rulingia.com> <20220218014814.GJ97875@funkthat.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--q22ohTV72yYk0JEy Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2022-Feb-17 17:48:14 -0800, John-Mark Gurney <jmg@funkthat.com> wrote: >Peter Jeremy wrote this message on Sat, Feb 05, 2022 at 20:50 +1100: >> I've raised https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D261731 to >> make geom_gate support BIO_ORDERED. Exposing the BIO_ORDERED flag to >> userland is quite easy (once a decision is made as to how to do that). >> Enhancing the geom_gate clients to correctly implement BIO_ORDERED is >> somewhat harder. > >The clients are single threaded wrt IOs, so I don't think updating them >are required. ggatec(8) and ggated(8) will not reorder I/Os. I'm not sure about hast. >I do have patches to improve things by making ggated multithreaded to >improve IOPs, and so making this improvement would allow those patches >to be useful. Likewise, I found ggatec and ggated to be too slow for my purposes and so I've implemented my own variant (not network API compatible) that can/does reorder requests. That was when I noticed that BIO_ORDERED wasn't implemented. >I do have a question though, what is the exact semantics of _ORDERED? I can't authoritatively answer this, sorry. >And right now, the ggate protocol (from what I remember) doesn't have >a way to know when the remote kernel has received notification that an >IO is complete. A G_GATE_CMD_START write request will be sent to the remote system and issued as a pwrite(2) then an acknowledgement packet will be returned and passed back to the local kernel via G_GATE_CMD_DONE. There's no support for BIO_FLUSH or BIO_ORDERED so there's no way for the local kernel to know when the write has been written to non-volatile store. >> I've done some experiments and OpenZFS doesn't generate BIO_ORDERED >> operations so I've also raised https://github.com/openzfs/zfs/issues/130= 65 >> I haven't looked into how difficult that would be to fix. Unrelated to the above but for completeness: OpenZFS avoids the need for BIO_ORDERED by not issuing additional I/Os until previous I/Os have been retired when ordering is important. (It does rely on BIO_FLUSH). --=20 Peter Jeremy --q22ohTV72yYk0JEy Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEE7rKYbDBnHnTmXCJ+FqWXoOSiCzQFAmIPWn1fFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEVF QjI5ODZDMzA2NzFFNzRFNjVDMjI3RTE2QTU5N0EwRTRBMjBCMzQACgkQFqWXoOSi CzQoug/9HBdIH8xPRej9/4/cO0136I3bxqIynTIvQiIvDcdy9E1DbqrfSV5yB7dR GyvVaIFMCvR3PljmDpGU9J7Gr/uzohcOOX3oU7mZ0XENGYFztbZrQ7ZAsw9KGgPP WgTNqrXlGkMDx1RALf/KRU/YhiRg7alQAl/ulTfwxyG9OixzDQT5JcD4tOv3SIzR ouhLo3MaXvy0NbNERAzelLNJApE+P5SM1i7XUFEY2ri5KmSkfV2ogV+E4ZZCu5Jj 6pb5u2gFEQDMeH+tfDxSGdjqV3ssP5hwY2wjzweMEpS1cJ2hvE/oLMAo9+/Z+mQN PtlYxYqIWtV6R+ZArbcyHUWXctvSO0lRTcwe62PqEAcNANLUl4RNrK3RZ12xsNwT HRyZE8LOK1dTHratF02UJ6dLaajR4CXRFkiJu6xfN8Hy8R9oejTcezICiWEYsZV1 l9QHKW2Vr8xwJjVjAz+YZBvWRoD7MQS8Am4/6zMfBYvosJU9KGHXMKroi0v43Hl0 UDC0yQtaHhIE4qPM64UtonkFKR1bC8fJJYVeO+hj9s1rA1dZG+c+tIuLbVPVQ8Hg 8fk18jy7IeksOHM9maO54kf5Oj77alrs1XjpX6wEW+0/pRFula4OluV+fDoZTv7J uoa0fVm+uEDlRd3bzOitVdXh3vMvMmiF/4yCLhiTET58e6CQTjc= =RNi/ -----END PGP SIGNATURE----- --q22ohTV72yYk0JEy--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Yg9agkeypdDOwKWm>