From owner-freebsd-arch@freebsd.org Mon Feb 1 20:14:40 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B4675A9857D for ; Mon, 1 Feb 2016 20:14:40 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id A3A4D14B0 for ; Mon, 1 Feb 2016 20:14:40 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id A0AFBA9857C; Mon, 1 Feb 2016 20:14:40 +0000 (UTC) Delivered-To: arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A0550A9857B for ; Mon, 1 Feb 2016 20:14:40 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 80B1814AE for ; Mon, 1 Feb 2016 20:14:40 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from ralph.baldwin.cx (c-73-231-226-104.hsd1.ca.comcast.net [73.231.226.104]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 8C2CEB946; Mon, 1 Feb 2016 15:14:39 -0500 (EST) From: John Baldwin To: Jilles Tjoelker Cc: arch@freebsd.org Subject: Re: Refactoring asynchronous I/O Date: Mon, 01 Feb 2016 12:14:28 -0800 Message-ID: <4485804.9PRGDHMp0I@ralph.baldwin.cx> User-Agent: KMail/4.14.3 (FreeBSD/10.2-STABLE; KDE/4.14.3; amd64; ; ) In-Reply-To: <20160131230214.GA37435@stack.nl> References: <2793494.0Z1kBV82mT@ralph.baldwin.cx> <20160131230214.GA37435@stack.nl> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 01 Feb 2016 15:14:39 -0500 (EST) X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Feb 2016 20:14:40 -0000 On Monday, February 01, 2016 12:02:14 AM Jilles Tjoelker wrote: > On Tue, Jan 26, 2016 at 05:39:03PM -0800, John Baldwin wrote: > > Note that binding the AIO support to a new fileop does mean that the AIO code > > now becomes mandatory (rather than optional). We could perhaps make the > > system calls continue to be optional if people really need that, but the guts > > of the code will now need to always be in the kernel. > > Enabling this by default is OK with me as long as the easy ways to get a > stuck process are at least disabled by default. Currently, a process > gets stuck forever if it has an AIO request from or to a pipe that will > never complete. An AIO daemon should not be allowed to perform an > unbounded sleep such as for a pipe (NFS server should be OK). Mmm, I don't currently fix this for pipes, but my changes do fix this for sockets (right now if you queue multiple reads for a socket both are woken up when data arrives and if the data only satifies the first read request, the second will block an AIO daemon forever). However, having fo_aio_queue() should make this fixable for pipes as they could use their own queueing logic and handling function. It may be that pipes need to work more like sockets (where the handling is object-centric rather than request-centric, so a pipe would queue a task when it was ready and would drain any requests queued to that pipe). Pipes could either share the same AIO daemon pool as sockets or use a private pool. I'd probably like to avoid an explosion of daemon pools though. I considered having a single AIO pool, but it's kind of messy to keep the per-process job limits in the request-centric pool while also permitting object-centric handlers on the internal job queue. OTOH, if enough backends end up using object-centric handlers then the job limits might end up meaningless and we could just drop them. -- John Baldwin