Date: Wed, 17 Aug 2005 06:00:10 +1000 From: Peter Jeremy <PeterJeremy@optushome.com.au> To: Luigi Rizzo <rizzo@icir.org> Cc: freebsd-arch@freebsd.org Subject: Re: Special schedulers, one CPU only kernel, one only userland Message-ID: <20050816200010.GJ13959@cirb503493.alcatel.com.au> In-Reply-To: <20050816051231.D66550@xorpc.icir.org> References: <42F9ECF2.8080809@freebsd.org> <200508101638.27087.jhb@FreeBSD.org> <42FA6E0E.4070205@samsco.org> <200508111121.46546.jhb@FreeBSD.org> <20050816051231.D66550@xorpc.icir.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 2005-Aug-16 05:12:31 -0700, Luigi Rizzo wrote: >reading this thread, and at times looking at some of the kernel code, >with plenty of places where you have to drop a lock that you >already have, do some small thing and then reacquire the lock itself, >makes me wonder if we don't need a better mechanism/abtraction for >this kind of programming. There are two distinct cases where this is done: 1) The thread needs to call another function which can potentially sleep and therefore needs to drop any locks it is holding before calling the function and re-acquire them later. 2) The thread needs to execute for an excessive period whilst holding the lock and explicitly releases it occasionally to allow other threads to run (eg much of the vnode/buffer/page scanning code). For the second case, we could create a function that checked if another thread was waiting on the lock and, if so, released the lock and grabbed it again after sleeping. This is fairly easy for a single lock but would be quite messy to implement for multiple locks - since you need to release/acquire them in the correct order to prevent deadlocks. The first case again seems easy at first glance: Add a "I'm happy to release this lock" flag to each lock. Before calling a function that might block, you set the flag. When the function calls sleep(9), the code releases the locks and re-acquires them before returning. There are three large gotchas: 1) Since sleeping might have changed the state of some structures that the thread is working on, it needs to re-validate its internal state to ensure that it's not working with stale data. This means the thread needs to check if it slept or not after each point where it could potentially sleep - and I suspect this amounts to most of the non-boilerplate code in the existing re-acquire lock section. 2) You have to ensure that either the release/acquire is atomic for all locks or the ordering is correct. 3) You need some way to pass back a permanent failure to re-acquire a requested lock (maybe a piece of hardware went away). >In a way, this seems similar to the handling of interrupts: >if we want a thread to be interrupted we don't check for interrupts >(and save and restore state) explicitly at every instruction, but >rely on the processor doing the right thing for us. In the case of locks, restoring the previous state may not be correct. Whilst I hold a lock on object foo, I know its state will remain unchanged (unless I change it) and can therefore safely cache information about the object. If I release/reacquire the lock, the object may have changed in arbitrary ways and I need to ensure that any cached information is still correct (and correct it if it's not). Overall, you could probably make the code cleaner (and maybe more efficient) but you can't make it totally transparent. -- Peter Jeremy
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20050816200010.GJ13959>