From owner-dev-commits-src-all@freebsd.org Wed Sep 15 17:32:25 2021 Return-Path: Delivered-To: dev-commits-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 63D0266AEA6; Wed, 15 Sep 2021 17:32:25 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4H8nN92MLJz4Qy1; Wed, 15 Sep 2021 17:32:25 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from John-Baldwins-MacBook-Pro.local (ralph.baldwin.cx [66.234.199.215]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 96E0826970; Wed, 15 Sep 2021 17:32:24 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Subject: Re: git: 1ecbc1d8e9d3 - main - cxgbe tom: Don't queue AIO requests on listen sockets. To: Alan Somers Cc: src-committers , dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org References: <202109142046.18EKkfEN043871@gitrepo.freebsd.org> From: John Baldwin Message-ID: <0cec1c04-0b42-6297-37c5-43d1dcb0f8d5@FreeBSD.org> Date: Wed, 15 Sep 2021 10:32:23 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: dev-commits-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Sep 2021 17:32:25 -0000 On 9/15/21 8:47 AM, Alan Somers wrote: > On Wed, Sep 15, 2021 at 9:21 AM John Baldwin wrote: > >> On 9/14/21 1:53 PM, Alan Somers wrote: >>> On Tue, Sep 14, 2021 at 2:46 PM John Baldwin wrote: >>> >>>> The branch main has been updated by jhb: >>>> >>>> URL: >>>> >> https://cgit.FreeBSD.org/src/commit/?id=1ecbc1d8e9d3fbcd8e68fc68f0a32944a12ddb1e >>>> >>>> commit 1ecbc1d8e9d3fbcd8e68fc68f0a32944a12ddb1e >>>> Author: John Baldwin >>>> AuthorDate: 2021-09-14 20:46:14 +0000 >>>> Commit: John Baldwin >>>> CommitDate: 2021-09-14 20:46:14 +0000 >>>> >>>> cxgbe tom: Don't queue AIO requests on listen sockets. >>>> >>>> This is similar to the fixes in 141fe2dceeae. One difference is >> that >>>> TOE sockets do not change states (listen vs non-listen) once >> created, >>>> so no lock is needed for SOLISTENING(). >>>> >>>> Sponsored by: Chelsio Communications >>>> >>> >>> I've always wondered: what's the point to using AIO with sockets? Can't >>> everything socket-related be done better with non-blocking read/write and >>> kqueue? >> >> Zero-copy operation with TOE is why TOE uses AIO. Zero-copy of user >> buffers >> can't really work with the non-AIO APIs because the user buffer is free to >> be reused immediately after write(2) (and on the read side you don't know >> the buffer in advance to allow the NIC to write directly into the use >> buffer). >> >> In theory we could support zero-copy using mb_ext_pgs for aio_write() for >> the non-TOE case similar to what sendfile() does. >> >> -- >> John Baldwin >> > > Interesting. Do you know of any common applications that include this > optimization? I've been working on the AIO ecosystem for Rust. It would > be good to ensure that this use case works, especially if zero-copy ever > works for non-TOE. I do not, and I rely on patches I merged upstream to netperf (-a and -A flags) to test it. I believe there might be some proprietary bits in some FreeBSD downstreams that might make use of this. -- John Baldwin