From owner-freebsd-current@FreeBSD.ORG Tue Apr 10 18:43:32 2007 Return-Path: X-Original-To: current@FreeBSD.org Delivered-To: freebsd-current@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 49ABA16A406; Tue, 10 Apr 2007 18:43:32 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 1A73813C4E8; Tue, 10 Apr 2007 18:43:32 +0000 (UTC) (envelope-from kris@obsecurity.org) Received: from obsecurity.dyndns.org (elvis.mu.org [192.203.228.196]) by elvis.mu.org (Postfix) with ESMTP id 3BA471A4D86; Tue, 10 Apr 2007 11:43:12 -0700 (PDT) Received: by obsecurity.dyndns.org (Postfix, from userid 1000) id BD2E8513AE; Tue, 10 Apr 2007 14:43:04 -0400 (EDT) Date: Tue, 10 Apr 2007 14:43:04 -0400 From: Kris Kennaway To: Mark Kirkwood Message-ID: <20070410184304.GB44123@xor.obsecurity.org> References: <20070226002234.GA80974@xor.obsecurity.org> <461B69C0.4060707@paradise.net.nz> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="huq684BweRXVnRxX" Content-Disposition: inline In-Reply-To: <461B69C0.4060707@paradise.net.nz> User-Agent: Mutt/1.4.2.2i Cc: pgsql-hackers , performance@FreeBSD.org, current@FreeBSD.org, Kris Kennaway Subject: Re: Anyone interested in improving postgresql scaling? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2007 18:43:32 -0000 --huq684BweRXVnRxX Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Apr 10, 2007 at 10:41:04PM +1200, Mark Kirkwood wrote: > Kris Kennaway wrote: > >If so, then your task is the following: > > > >Make SYSV semaphores less dumb about process wakeups. Currently > >whenever the semaphore state changes, all processes sleeping on the > >semaphore are woken, even if we only have released enough resources > >for one waiting process to claim. i.e. there is a thundering herd > >wakeup situation which destroys performance at high loads. Fixing > >this will involve replacing the wakeup() calls with appropriate > >amounts of wakeup_one(). >=20 > I'm forwarding this to the pgsql-hackers list so that folks more=20 > qualified than I can comment, but as I understand the way postgres=20 > implements locking each process has it *own* semaphore it waits on -=20 > and who is waiting for what is controlled by an in (shared) memory hash= =20 > of lock structs (access to these is controlled via platform Dependant=20 > spinlock code). So a given semaphore state change should only involve=20 > one process wakeup. I have not studied the exact code path, but there are indeed multiple wakeups happening from the semaphore code (as many as the number of active postgresql processes). It is easy to instrument sleepq_broadcast() and log them when they happen. Anyway mux@ fixed this some time ago, which indeed helped scaling for traffic over a local domain socket (particularly at higher loads), but I saw some anomalous results when using loopback TCP traffic. I think this is unrelated (in this situation TCP is highly contended, and it is often the case that fixing one bottleneck can make a highly contended situation perform worse, because you were effectively serializing a bit before, and reducing the non-linear behaviour) but am still investigating, so the patch has not yet been committed. Kris --huq684BweRXVnRxX Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFGG9q4Wry0BWjoQKURAgj5AKD8GphymMDpkMqiJsyxu77xXZN5RACbBlbV OxZZdXcUrbW7nwz2Ac/srxo= =UDMf -----END PGP SIGNATURE----- --huq684BweRXVnRxX--