From owner-freebsd-arch@FreeBSD.ORG Thu Nov 8 20:01:39 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C923616A421 for ; Thu, 8 Nov 2007 20:01:39 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.187]) by mx1.freebsd.org (Postfix) with ESMTP id 4665513C4BA for ; Thu, 8 Nov 2007 20:01:39 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by nf-out-0910.google.com with SMTP id b2so228148nfb for ; Thu, 08 Nov 2007 12:01:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:cc:mime-version:content-type:content-transfer-encoding:content-disposition:x-google-sender-auth; bh=jzmrghTcmXZMrFhALwp8y617WIMMfP3jEVgtfelp2iA=; b=hRkrBlV3/l2BckX6oUPFDfovMeywgDlABWTP1exnizFlW0HzpNsDLOmp9hDAIlXNdpvs1HrFVflOb9SnMCuHXfM+KwnvE/JfvU1qcivk/RnNb+r5qm/pgaDT9IeKcu+/+Obdn3omN5x0I4PSx2XpvoCIUAnVxoUz4783GZ+XH7U= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:cc:mime-version:content-type:content-transfer-encoding:content-disposition:x-google-sender-auth; b=HySyfrs9z6V0APvRq6bmJfTnzI3O0Nsl+KBtLA2VbQHciYkx/i9i+FfEUlbIV7FLYjvVlA48onRMOuYWRgnCR60NBdivm4sKAf5VU1UxUaYP+Jhn85y+5Z0AR1cl3fd/+9DmLgKohW/bBMJtjIx6zr6oXgEtN9JISvhT8fe85t0= Received: by 10.86.77.5 with SMTP id z5mr771367fga.1194550507967; Thu, 08 Nov 2007 11:35:07 -0800 (PST) Received: by 10.86.90.4 with HTTP; Thu, 8 Nov 2007 11:35:07 -0800 (PST) Message-ID: <3bbf2fe10711081135y3473817ejbc72574e3e8c763d@mail.gmail.com> Date: Thu, 8 Nov 2007 20:35:07 +0100 From: "Attilio Rao" Sender: asmrookie@gmail.com To: freebsd-arch@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Google-Sender-Auth: 8476ae5a7da20049 Cc: Robert Watson , Pawel Jakub Dawidek Subject: [RFC] callout overhaul: part I X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Nov 2007 20:01:39 -0000 Hello, Some benchmarks posted by rwatson some time ago ( http://lists.freebsd.org/pipermail/freebsd-arch/2007-October/006945.html ) evicted callout_lock spinlock as an highly contented lock, in particular for network paths. Honestly, this is not a surprise at all :). This moved me in the direction of working on callouts, mainly for improving the scalability respect a large number of CPUs but, since callout are rather sensitive for the whole system, a general overhaul to the whole mecanism would be a good idea. In particular, I found phk's ideas about it ( http://lists.freebsd.org/pipermail/freebsd-arch/2006-November/005730.html ) very valuable and I have on-going discussions with him and rwatson about these. phk's proposal is however not complete at 100% and I'm trying to integrate with other missing supports. This patch, for example: http://people.freebsd.org/~attilio/callout_rwlock.diff would add the support for callout working with rwlock. Currently, mutex are the only possible kind of lock to be used in conjuction with callouts and this is a little bit limitative. The support is implemented using the jhb's abstraction layer for locks. Even if it is rather generic to be used with any kind of abstracted lock, only rwlock and mutex are allowed to work because of softclock() running in a kthread. A new function is added to the KPI called callout_init_rw() which basically does the same of callout_init_mtx() but with a rwlock. Addictionally, it can get the new flag CALLOUT_SHAREDLOCK which let acquiring the rwlock in shared mode while running the callout. Using CALLOUT_SHAREDLOCK with callout_init_mtx() has no effect. We could panic for it if someone it worths, but it would end up polluting callout.h or alternatively complicating new _callout_init_lock(). Please, also note that the mtx_owned() stuff from the callout_stop() function is removed as theoretically the lock, when specified, should always be acquired when running callout_stop(). However, as there is no simple way to do assertions with the lock abstraction class, and there is no way to retrieve a child object from the lock_object class, don't do any assertion for the moment, just comment the situation. I'm eager to hear for your feedbacks. Thanks, Attilio PS: It seems that other developers alredy did something similar (like pjd. It seems we discussed about it some time ago but I completely forgot about that... :)), please report if you have had better ideas on it. -- Peace can only be achieved by understanding - A. Einstein