Date: Tue, 11 Jan 2000 11:25:35 +1100 From: Patryk Zadarnowski <patrykz@ilion.eu.org> To: "Scott Hess" <scott@avantgo.com> Cc: freebsd-hackers@FreeBSD.ORG, developer@lists.mysql.com Subject: Re: Concept check: iothreads addition to pthreads for MYSQL+FreeBSD. Message-ID: <200001110025.LAA31411@mycenae.ilion.eu.org> In-Reply-To: Your message of "Mon, 10 Jan 2000 15:20:38 -0800." <1a6101bf5bc1$4e364b20$1e80000a@avantgo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> Recently I was tasked to find a way to scale up our MYSQL server, running > MYSQL3.22.15 on FreeBSD3.3. I've been testing a hardware RAID solution, > and found that with 6 disks in a RAID5 configuration, the system was only > perhaps 30% faster than when running on a single disk. [The 6 disks in the > RAID5 are the same model as the single-disk test I was comparing against.] > > Experimentation determined that pthreads was the problem. FreeBSD's > implementation of pthreads using a select() loop, and select() always says > that disk I/O is ready to proceed, and disk I/O never return EWOULDBLOCK. > Essentially, pthreads was serializing the MYSQL read() requests, and if the > dataset exceeds memory size, performance becomes entirely seek bound. > > I've implemented a rough fix, which is to rfork() processes which I label > "iothreads" to handle the disk I/O. The parent process running pthreads > has a socketpair() to each of the iothreads. The iothreads wait for > requests on the socketpair, and since socketpairs can block, pthreads can > handle them efficiently. This essentially allows me to turn blocking disk > I/O calls into non-blocking calls. Having multiple pending seeks turns out > to be a huge win for MYSQL, allowing it to scale much better as disks are > added to the RAID5 array. > > Unfortunately, I'm concerned about using this code in production, because > it needs a fair amount of cleanup to handle signals and administrative > functions correctly. For this reason and others, I'm starting a project to > move it into the pthreads library itself. Before I embark on that effort, > I have a couple questions: > > 1) Does this seem like a reasonable approach? [It _works_, and well. But > it tastes strongly of hack.] > > 2) Does anyone have suggestions for a solution that will be cleaner and > won't take man-months to implement? [Which is the redeeming quality of > what I've got - it took me two days to zero in on a very workable > solution.] > > 3) Is anyone working on something I might leverage off of in this area? > [For instance, a select()-based interface to async I/O? Or support for > blocking disk I/O in select() and read()?] > > 4) Is there anyone willing to commit to testing my modified pthreads > library against MYSQL? [I'll be stress testing it quite heavily, of > course. It would probably also be testable against Squid with async I/O > and multithreaded Apache 2.0.] I'm no expert on pthreads, but, if you decide to proceed with implementing a mixed user-land/rfork pthread implementation, may I suggest that you implement is through POSIX pthread_attr_setscope() interfaces instead of some local extension. pthread_attr_setscope() sounds like a portable interface to precisely what you're trying to achieve. Pat. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200001110025.LAA31411>