From owner-freebsd-hackers Mon Jan 17 20:27:41 2000 Delivered-To: freebsd-hackers@freebsd.org Received: from maxwell.syr.edu (maxwell.syr.edu [128.230.129.5]) by hub.freebsd.org (Postfix) with ESMTP id 9901014BD8 for ; Mon, 17 Jan 2000 20:27:34 -0800 (PST) (envelope-from cmsedore@maxwell.syr.edu) Received: from qwerty.maxwell.syr.edu (qwerty.maxwell.syr.edu [128.230.129.248]) by maxwell.syr.edu (8.9.3/8.9.1) with ESMTP id EAA90719 for ; Tue, 18 Jan 2000 04:27:34 GMT Date: Mon, 17 Jan 2000 23:27:33 -0500 (EST) From: Chris Sedore To: freebsd-hackers@FreeBSD.ORG Subject: Re: Concept check: iothreads addition to pthreads for MYSQL+FreeBSD. In-Reply-To: <77EC013A8765D311974D00A0C9B413A1013A86E9-100000@exchange.maxwell.syr.edu> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Mon, 10 Jan 2000, Chris Sedore wrote: > > > On Mon, 10 Jan 2000 jasone@canonware.com wrote: > > [...] > > > 1) Does this seem like a reasonable approach? [It _works_, and well. But > > > it tastes strongly of hack.] > > > > I'm not very fond of this approach to the problem, though it can work, as > > you note. Asynchronous I/O is in my opinion a much cleaner solution. > > > > > 2) Does anyone have suggestions for a solution that will be cleaner and > > > won't take man-months to implement? [Which is the redeeming quality of > > > what I've got - it took me two days to zero in on a very workable > > > solution.] > > > > Have you tried the linuxthreads port lately? With linuxthreads, each > > thread is an rfork()ed process, which means that blocking on file I/O only > > blocks the thread that attempts the I/O. There are various > > advantages/disadvantages between libc_r and linuxthreads, but you may find > > linuxthreads to meet your needs at the moment. > > > > > 3) Is anyone working on something I might leverage off of in this area? > > > [For instance, a select()-based interface to async I/O? Or support for > > > blocking disk I/O in select() and read()?] > > > > Though not ideal, it seems to me that using asynchronous I/O will work > > reasonably well inside libc_r. If there weren't plans to build a much more > > scalable threads library than what we currently have, this modification > > would be high on my priority list. > > I was just poking around a bit, and I don't think it would take too much > to add an aio_select() call, that would do an async io select (that is, > queue a select as just another aio operation). I've cheated around this > a bit, since you can get away (for instance) with executing an aio_read() > against a listening socket, waiting for the completion (an error, since > you can't read from such a thing), then doing the accept(). > > It seems that a fair amount of benefit could be had from async select, > since it would facilitate using aio in pthreads, innd, etc, which were > originally designed for select(), but might like to add async io for disk > operations. > > Maybe it is just bloat in the long-run, since with kernel threads one > could just fire off another thread to do select(). Or maybe someone will > pipe up and say that select() won't work that way (i.e., just pass in the > select parameters and let one of the aiod's call select() on behalf of > the main process). (yes, I'm following up on my own message) I had some free time this weekend, so I wrote aio_select. It basically allows you to call select asynchronously, and pickup results the same way you would for aio_read or aio_write. It might be just the write kind of solution for the problem(s) above. The implementation is not what I'd call ideal, mostly because I wanted to get on to aio_cancel. I did finish a reasonable implementation of aio_cancel, which should fill out FreeBSD aio routines to be reasonably POSIX compliant. We're in 'feature freeze' for now, so I'm guessing that it will be a while before aio_cancel gets checked in. -Chris To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message