From owner-freebsd-smp@FreeBSD.ORG Sun Sep 23 02:57:50 2007 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F11CE16A417 for ; Sun, 23 Sep 2007 02:57:50 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mu-out-0910.google.com (mu-out-0910.google.com [209.85.134.191]) by mx1.freebsd.org (Postfix) with ESMTP id 9194213C459 for ; Sun, 23 Sep 2007 02:57:50 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by mu-out-0910.google.com with SMTP id w9so1495646mue for ; Sat, 22 Sep 2007 19:57:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:cc:mime-version:content-type:content-transfer-encoding:content-disposition:x-google-sender-auth; bh=iiUvjAMtMydCs3r99oSiL8ZVSK+ENH6vXGacQdqDpQM=; b=g1iP1YJJkN0bZX0zG6kpRUL0G28h99XNXoc29uUxIEd/IjbnDP8ftcKd8USxTcqu+JhvIwjQl1Jrm3UL/FR+ItWu/f2LB7wZ8avmP1BZ3vwtB3mwzLrpBB6WxmFr1U90iktaxg/zEDFc8Y+cWCGp9cHbwoh03+LXgh9iTNNBlnY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:sender:to:subject:cc:mime-version:content-type:content-transfer-encoding:content-disposition:x-google-sender-auth; b=iWgivIQ7NXNN0XY5Jkaqr5xq/lcUGOjxvROnqBlWyXWPktg3E5q1lwgalmZVsy1CyN4hnqzpdKb7lnuc7C9zvoO73MlMsNChlfbb8kp0+o96ISr41ys1SK5Tas7lpY4RBvXQZGV3vcHPATRvTCChrT2a77oj7j+4pn0D6LowUkM= Received: by 10.78.149.15 with SMTP id w15mr2190939hud.1190514726280; Sat, 22 Sep 2007 19:32:06 -0700 (PDT) Received: by 10.78.97.18 with HTTP; Sat, 22 Sep 2007 19:32:06 -0700 (PDT) Message-ID: <3bbf2fe10709221932i386f65b9h6f47ab4bee08c528@mail.gmail.com> Date: Sun, 23 Sep 2007 04:32:06 +0200 From: "Attilio Rao" Sender: asmrookie@gmail.com To: freebsd-smp@freebsd.org, freebsd-arch@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Google-Sender-Auth: 2e21c3221234266c Cc: Subject: rwlocks: poor performance with adaptive spinning X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Sep 2007 02:57:51 -0000 Recently several people have reported problems of starvation with rwlocks. In particular, users which tried to use rwlock on big SMP environment (16+ CPUs) found them rather subjected to poor performances and to starvation of waiters. Inspecting the code, something strange about adaptive spinning popped up: basically, for rwlocks, adaptive spinning stubs seem to be customed too down in the decisioning-loop. The desposition of the stub will let the thread that would adaptively spin, to set the respecitve (both read or write) waiters flag on, which means that the owner of the lock will go down in the hard path of locking functions and will performe a full wakeup even if the waiters queues can result empty. This is a big penalty for adaptive spinning which can make it completely useless. In addiction to this, adaptive spinning only runs in the turnstile spinlock path which is not ideal. This patch ports the approach alredy used for adaptive spinning in sx locks to rwlocks: http://users.gufi.org/~rookie/works/patches/kern_rwlock.diff In sx it is unlikely to see big benefits because they are held for too long times, but for rwlocks situation is rather different. I would like to see if people can do benchmarks with this patch (maybe in private environments?) as I'm not able to do them in short times. Adaptive spinning in rwlocks can be improved further with other tricks (like adding a backoff counter, for example, or trying to spin with the lock held in read mode too), but we first should be sure to start with a solid base. Thanks, Attilio -- Peace can only be achieved by understanding - A. Einstein