Date: Sun, 23 Mar 2008 19:06:40 -1000 (HST) From: Jeff Roberson <jroberson@chesapeake.net> To: arch@freebsd.org Subject: buffer queue lock contention Message-ID: <20080323185402.C910@desktop>
next in thread | raw e-mail | index | archive | help
http://people.freebsd.org/~jeff/bqlock.diff This patch makes a seperate lock for each queue of the buffer cache. It is intended to reduce contention on the single bqlock. In most cases this is quite trivial however getnewbuffer() warrants some discussion. In getnewbuffer() we walk several queues in order looking for a buffer to recycle or use directly. With a single lock you're guaranteed that as you're looking at one queue a new buffer won't come available on an earlier queue. You're also guaranteed that you can sleep atomically while waiting for new buffers. In my patch the queues are independently synchronized and so these guarantees are no longer present. A thread could scan each list sucessively and not find any suitable buffers and then go to sleep even after a new buffer had been posted to one of the other queues. I don't believe this is a significant problem because the very next brelse is likely to wakeup the getnewbuf sleeper. With a buffer cache under suitable pressure the wakeups are likely to happen quite frequently. The other part of this diff is to avoid acquiring the global 'needs buffer' and 'running buf request' locks when the variables they protect are not set. This would make it possible for one thread to go to sleep simultaneously with another thread releasing a buffer that would wake it up and this wakeup would be missed. However, as in the bqlock case, another brelse is not far behind to unstick this process. Presently we call bufcountwakeup() and bufspacewakeup() on almost every call to brelse() and thus acquire these global locks several times. When we've actually done io we acquire a third global lock with runningbufwakeup(). All of these global locks effectively add latency and cost to each io. Thanks, Jeff
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080323185402.C910>