Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 8 Nov 2007 20:35:07 +0100
From:      "Attilio Rao" <attilio@freebsd.org>
To:        freebsd-arch@freebsd.org
Cc:        Robert Watson <rwatson@freebsd.org>, Pawel Jakub Dawidek <pjd@freebsd.org>
Subject:   [RFC] callout overhaul: part I
Message-ID:  <3bbf2fe10711081135y3473817ejbc72574e3e8c763d@mail.gmail.com>

next in thread | raw e-mail | index | archive | help
Hello,
Some benchmarks posted by rwatson some time ago (
http://lists.freebsd.org/pipermail/freebsd-arch/2007-October/006945.html
) evicted callout_lock spinlock as an highly contented lock, in
particular for network paths. Honestly, this is not a surprise at all
:). This moved me in the direction of working on callouts, mainly for
improving the scalability respect a large number of CPUs but, since
callout are rather sensitive for the whole system, a general overhaul
to the whole mecanism would be a good idea.
In particular, I found phk's ideas about it (
http://lists.freebsd.org/pipermail/freebsd-arch/2006-November/005730.html
) very valuable and I have on-going discussions with him and rwatson
about these.

phk's proposal is however not complete at 100% and I'm trying to
integrate with other missing supports.
This patch, for example:
http://people.freebsd.org/~attilio/callout_rwlock.diff

would add the support for callout working with rwlock. Currently, mutex are the
only possible kind of lock to be used in conjuction with callouts and
this is a little bit limitative.

The support is implemented using the jhb's abstraction layer for
locks. Even if it is rather generic to be used with any kind of
abstracted lock, only rwlock and mutex are allowed to work because of
softclock() running in a kthread.
A new function is added to the KPI called callout_init_rw() which
basically does the same of callout_init_mtx() but with a rwlock.
Addictionally, it can get the new flag CALLOUT_SHAREDLOCK which let
acquiring the rwlock in shared mode while running the callout.
Using CALLOUT_SHAREDLOCK with callout_init_mtx() has no effect. We
could panic for it if someone it worths, but it would end up polluting
callout.h or alternatively complicating new _callout_init_lock().

Please, also note that the mtx_owned() stuff from the callout_stop()
function is removed as theoretically the lock, when specified, should
always be acquired when running callout_stop(). However, as there is
no simple way to do assertions with the lock abstraction class, and
there is no way to retrieve a child object from the lock_object class,
don't do any assertion for the moment, just comment the situation.
I'm eager to hear for your feedbacks.

Thanks,
Attilio

PS: It seems that other developers alredy did something similar (like
pjd. It seems we discussed about it some time ago but I completely
forgot about that... :)), please report if you have had better ideas
on it.


-- 
Peace can only be achieved by understanding - A. Einstein



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3bbf2fe10711081135y3473817ejbc72574e3e8c763d>