Date: Wed, 15 Sep 2021 12:20:45 -0700 From: Navdeep Parhar <np@FreeBSD.org> To: Alan Somers <asomers@freebsd.org>, John Baldwin <jhb@freebsd.org> Cc: src-committers <src-committers@freebsd.org>, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Subject: Re: git: 1ecbc1d8e9d3 - main - cxgbe tom: Don't queue AIO requests on listen sockets. Message-ID: <bf633b17-7652-ff09-d93d-924356392c69@FreeBSD.org> In-Reply-To: <CAOtMX2jjB8T6780HarbCR6rsSkUf%2BQUN55JCGu8h8-=oJTx7zA@mail.gmail.com> References: <202109142046.18EKkfEN043871@gitrepo.freebsd.org> <CAOtMX2iGyPvZvhLFhp9tf2aX=NUnoPLezE5RELxhqJqKGRf2Jw@mail.gmail.com> <fa92e982-4311-fa90-d37f-f8d78042c482@FreeBSD.org> <CAOtMX2hJZ3svh9V6b57LX0xxXeNa1m_zWkYCr2vejNvb0y4AyA@mail.gmail.com> <0cec1c04-0b42-6297-37c5-43d1dcb0f8d5@FreeBSD.org> <CAOtMX2jjB8T6780HarbCR6rsSkUf%2BQUN55JCGu8h8-=oJTx7zA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 9/15/21 11:05 AM, Alan Somers wrote: > On Wed, Sep 15, 2021 at 11:32 AM John Baldwin <jhb@freebsd.org > <mailto:jhb@freebsd.org>> wrote: > > On 9/15/21 8:47 AM, Alan Somers wrote: > > On Wed, Sep 15, 2021 at 9:21 AM John Baldwin <jhb@freebsd.org > <mailto:jhb@freebsd.org>> wrote: > > > >> On 9/14/21 1:53 PM, Alan Somers wrote: > >>> On Tue, Sep 14, 2021 at 2:46 PM John Baldwin <jhb@freebsd.org > <mailto:jhb@freebsd.org>> wrote: > >>> > >>>> The branch main has been updated by jhb: > >>>> > >>>> URL: > >>>> > >> > https://cgit.FreeBSD.org/src/commit/?id=1ecbc1d8e9d3fbcd8e68fc68f0a32944a12ddb1e > <https://cgit.FreeBSD.org/src/commit/?id=1ecbc1d8e9d3fbcd8e68fc68f0a32944a12ddb1e>; > >>>> > >>>> commit 1ecbc1d8e9d3fbcd8e68fc68f0a32944a12ddb1e > >>>> Author: John Baldwin <jhb@FreeBSD.org> > >>>> AuthorDate: 2021-09-14 20:46:14 +0000 > >>>> Commit: John Baldwin <jhb@FreeBSD.org> > >>>> CommitDate: 2021-09-14 20:46:14 +0000 > >>>> > >>>> cxgbe tom: Don't queue AIO requests on listen sockets. > >>>> > >>>> This is similar to the fixes in 141fe2dceeae. One > difference is > >> that > >>>> TOE sockets do not change states (listen vs non-listen) once > >> created, > >>>> so no lock is needed for SOLISTENING(). > >>>> > >>>> Sponsored by: Chelsio Communications > >>>> > >>> > >>> I've always wondered: what's the point to using AIO with > sockets? Can't > >>> everything socket-related be done better with non-blocking > read/write and > >>> kqueue? > >> > >> Zero-copy operation with TOE is why TOE uses AIO. Zero-copy of user > >> buffers > >> can't really work with the non-AIO APIs because the user buffer > is free to > >> be reused immediately after write(2) (and on the read side you > don't know > >> the buffer in advance to allow the NIC to write directly into > the use > >> buffer). > >> > >> In theory we could support zero-copy using mb_ext_pgs for > aio_write() for > >> the non-TOE case similar to what sendfile() does. > >> > >> -- > >> John Baldwin > >> > > > > Interesting. Do you know of any common applications that include > this > > optimization? I've been working on the AIO ecosystem for Rust. > It would > > be good to ensure that this use case works, especially if > zero-copy ever > > works for non-TOE. > > I do not, and I rely on patches I merged upstream to netperf (-a and > -A flags) > to test it. I believe there might be some proprietary bits in some > FreeBSD > downstreams that might make use of this. > > -- > John Baldwin > > > Do you mean these -a and -A flags, or am I looking in the wrong place? > -a sizespec > Alter the send and receive buffer alignments on the local > system. This defaults to 8 bytes. > > -A sizespec > As -a, but for the remote system. The aio options are available for the TCP tests but you might have to build netperf with a non-default option (that's what I had to do a long time back and I haven't checked if it's still needed). # netperf -t TCP_STREAM -- -h ... TCP/UDP BSD Sockets Test Options: -a Use aio_write(2) -A Use aio_read(2) ... # pkg options netperf netperf - EXAMPLES: off netperf - EXS: on netperf - HISTOGRAM: on netperf - INFO: off netperf - OMNI: off netperf - SCTP: on netperf - SOCKETS: on Regards, Navdeep
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bf633b17-7652-ff09-d93d-924356392c69>