Date: Mon, 10 Jan 2000 15:39:17 -0800 From: jasone@canonware.com To: Scott Hess <scott@avantgo.com> Cc: freebsd-hackers@freebsd.org, developer@lists.mysql.com Subject: Re: Concept check: iothreads addition to pthreads for MYSQL+FreeBSD. Message-ID: <20000110153917.D4938@sturm.canonware.com> In-Reply-To: <1a6101bf5bc1$4e364b20$1e80000a@avantgo.com>; from scott@avantgo.com on Mon, Jan 10, 2000 at 03:20:38PM -0800 References: <1a6101bf5bc1$4e364b20$1e80000a@avantgo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Jan 10, 2000 at 03:20:38PM -0800, Scott Hess wrote: > I've implemented a rough fix, which is to rfork() processes which I label > "iothreads" to handle the disk I/O. The parent process running pthreads > has a socketpair() to each of the iothreads. The iothreads wait for > requests on the socketpair, and since socketpairs can block, pthreads can > handle them efficiently. This essentially allows me to turn blocking disk > I/O calls into non-blocking calls. Having multiple pending seeks turns out > to be a huge win for MYSQL, allowing it to scale much better as disks are > added to the RAID5 array. > > Unfortunately, I'm concerned about using this code in production, because > it needs a fair amount of cleanup to handle signals and administrative > functions correctly. For this reason and others, I'm starting a project to > move it into the pthreads library itself. Before I embark on that effort, > I have a couple questions: > > 1) Does this seem like a reasonable approach? [It _works_, and well. But > it tastes strongly of hack.] I'm not very fond of this approach to the problem, though it can work, as you note. Asynchronous I/O is in my opinion a much cleaner solution. > 2) Does anyone have suggestions for a solution that will be cleaner and > won't take man-months to implement? [Which is the redeeming quality of > what I've got - it took me two days to zero in on a very workable > solution.] Have you tried the linuxthreads port lately? With linuxthreads, each thread is an rfork()ed process, which means that blocking on file I/O only blocks the thread that attempts the I/O. There are various advantages/disadvantages between libc_r and linuxthreads, but you may find linuxthreads to meet your needs at the moment. > 3) Is anyone working on something I might leverage off of in this area? > [For instance, a select()-based interface to async I/O? Or support for > blocking disk I/O in select() and read()?] Though not ideal, it seems to me that using asynchronous I/O will work reasonably well inside libc_r. If there weren't plans to build a much more scalable threads library than what we currently have, this modification would be high on my priority list. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20000110153917.D4938>