From owner-freebsd-arch@FreeBSD.ORG Sat Mar 29 07:03:28 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5CF8D1065672 for ; Sat, 29 Mar 2008 07:03:28 +0000 (UTC) (envelope-from jroberson@chesapeake.net) Received: from webaccess-cl.virtdom.com (webaccess-cl.virtdom.com [216.240.101.25]) by mx1.freebsd.org (Postfix) with ESMTP id 24B3E8FC1D for ; Sat, 29 Mar 2008 07:03:28 +0000 (UTC) (envelope-from jroberson@chesapeake.net) Received: from [10.0.1.200] (cpe-24-94-72-120.hawaii.res.rr.com [24.94.72.120]) (authenticated bits=0) by webaccess-cl.virtdom.com (8.13.6/8.13.6) with ESMTP id m2T73Pe7028289 for ; Sat, 29 Mar 2008 03:03:27 -0400 (EDT) (envelope-from jroberson@chesapeake.net) Date: Fri, 28 Mar 2008 21:04:17 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: arch@freebsd.org In-Reply-To: <20080312152744.I29518@fledge.watson.org> Message-ID: <20080328202602.N72156@desktop> References: <10004.1205307334@critter.freebsd.dk> <200803120908.10403.jhb@freebsd.org> <20080312140422.B29518@fledge.watson.org> <200803121059.45614.jhb@freebsd.org> <20080312152744.I29518@fledge.watson.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Subject: timeout/callout small step forward X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Mar 2008 07:03:28 -0000 http://people.freebsd.org/~jeff/callout.diff This patch takes the current callout implementation and makes it per-cpu. It also hides callout details from the rest of the kernel by making the callwheel structure private to kern_timeout.c among other things. All callouts default to the first softclock thread to maintain ordering and concurrency guarantees of the older code. The only new interface that is implemented is 'callout_reset_on' which can specify a cpu to run the callout on. This can force a callout which is not presently being dispatched to migrate to a new cpu. The threads are not hard bound to each cpu so that they can't be starved into not processing their queues. They are 'medium' bound by ULE and probably tend to run where they are scheduled on 4BSD but there is no guarantee. The ULE rules will run them on their native CPU if it's running something of lesser priority or idle. If it's running something of greater priority it will look for a more idle cpu. This prevents the softclock thread from being starved by other interrupt thread activity. I believe this helps phk's effort rather than hindering it because: 1) It collects up more implementation details and pushes them into kern_timeout.c 2) It sounds like we're keeping the same current/waiting/cancel algorithm so this packages it up into a per-instances structure as would need to happen in phk's proposal as well. 3) The only new interface, a varient of callout_reset() should be trivial to implement whatever the new scheme is. Also included is changes to kern_event.c and subr_sleepq.c so sleepq() timeouts happen on the cpu that generated them and so do kqueue() poll timeouts. You can see that the additional interface is quite simple. This has been tested and debugged by nokia. Thanks, Jeff