Date: Mon, 10 Jan 2000 15:20:38 -0800 From: "Scott Hess" <scott@avantgo.com> To: <freebsd-hackers@freebsd.org>, <developer@lists.mysql.com> Subject: Concept check: iothreads addition to pthreads for MYSQL+FreeBSD. Message-ID: <1a6101bf5bc1$4e364b20$1e80000a@avantgo.com>
next in thread | raw e-mail | index | archive | help
Recently I was tasked to find a way to scale up our MYSQL server, running MYSQL3.22.15 on FreeBSD3.3. I've been testing a hardware RAID solution, and found that with 6 disks in a RAID5 configuration, the system was only perhaps 30% faster than when running on a single disk. [The 6 disks in the RAID5 are the same model as the single-disk test I was comparing against.] Experimentation determined that pthreads was the problem. FreeBSD's implementation of pthreads using a select() loop, and select() always says that disk I/O is ready to proceed, and disk I/O never return EWOULDBLOCK. Essentially, pthreads was serializing the MYSQL read() requests, and if the dataset exceeds memory size, performance becomes entirely seek bound. I've implemented a rough fix, which is to rfork() processes which I label "iothreads" to handle the disk I/O. The parent process running pthreads has a socketpair() to each of the iothreads. The iothreads wait for requests on the socketpair, and since socketpairs can block, pthreads can handle them efficiently. This essentially allows me to turn blocking disk I/O calls into non-blocking calls. Having multiple pending seeks turns out to be a huge win for MYSQL, allowing it to scale much better as disks are added to the RAID5 array. Unfortunately, I'm concerned about using this code in production, because it needs a fair amount of cleanup to handle signals and administrative functions correctly. For this reason and others, I'm starting a project to move it into the pthreads library itself. Before I embark on that effort, I have a couple questions: 1) Does this seem like a reasonable approach? [It _works_, and well. But it tastes strongly of hack.] 2) Does anyone have suggestions for a solution that will be cleaner and won't take man-months to implement? [Which is the redeeming quality of what I've got - it took me two days to zero in on a very workable solution.] 3) Is anyone working on something I might leverage off of in this area? [For instance, a select()-based interface to async I/O? Or support for blocking disk I/O in select() and read()?] 4) Is there anyone willing to commit to testing my modified pthreads library against MYSQL? [I'll be stress testing it quite heavily, of course. It would probably also be testable against Squid with async I/O and multithreaded Apache 2.0.] Thanks, scott To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1a6101bf5bc1$4e364b20$1e80000a>