From owner-freebsd-net@FreeBSD.ORG Sun Nov 25 12:36:29 2007 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 253E416A468; Sun, 25 Nov 2007 12:36:29 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.186]) by mx1.freebsd.org (Postfix) with ESMTP id A5B3C13C4EB; Sun, 25 Nov 2007 12:36:28 +0000 (UTC) (envelope-from max@love2party.net) Received: from amd64.laiers.local (dslb-088-066-008-048.pools.arcor-ip.net [88.66.8.48]) by mrelayeu.kundenserver.de (node=mrelayeu0) with ESMTP (Nemesis) id 0MKwh2-1IwGiT1vxZ-0002z3; Sun, 25 Nov 2007 13:36:26 +0100 From: Max Laier Organization: FreeBSD To: Darren Reed Date: Sun, 25 Nov 2007 13:36:20 +0100 User-Agent: KMail/1.9.7 References: <200711231232.04447.max@love2party.net> <200711251047.44778.max@love2party.net> <474964CF.90308@freebsd.org> In-Reply-To: <474964CF.90308@freebsd.org> X-Face: ,,8R(x[kmU]tKN@>gtH1yQE4aslGdu+2]; R]*pL,U>^H?)gW@49@wdJ`H<=?utf-8?q?=25=7D*=5FBD=0A=09U=5For=3D=5CmOZf764=26nYj=3DJYbR1PW0ud?=>|!~,,CPC.1-D$FG@0h3#'5"k{V]a~.<=?utf-8?q?mZ=7D44=23Se=7Em=0A=09Fe=7E=5C=5DX5B=5D=5Fxj?=(ykz9QKMw_l0C2AQ]}Ym8)fU MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart16846558.i6R9qJmlUT"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200711251336.28161.max@love2party.net> X-Provags-ID: V01U2FsdGVkX18RS52xtco7cAiEwCXhR5q0ghA1ZVUr/V3LOV/ XRr8ZelUuE1ngrMgtnft/e06A4Aqq4s7vIolbV5LRf/Qtlx0vb wiXrtZX7sp0nYMJGP3wSpyZIEJGthYS+7ZgXCkFyc0= Cc: freebsd-net@freebsd.org, freebsd-current@freebsd.org, Robert Watson Subject: Re: Switch pfil(9) to rmlocks X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Nov 2007 12:36:29 -0000 --nextPart16846558.i6R9qJmlUT Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Sunday 25 November 2007, Darren Reed wrote: > Max Laier wrote: > > On Sunday 25 November 2007, Darren Reed wrote: > > > Max Laier wrote: > > > > On Friday 23 November 2007, Robert Watson wrote: > > > > > On Fri, 23 Nov 2007, Max Laier wrote: > > > > > > attached is a diff to switch the pfil(9) subsystem to > > > > > > rmlocks, which are more suited for the task. I'd like some > > > > > > exposure before doing the switch, but I don't expect any > > > > > > fallout. This email is going through the patched pfil > > > > > > already - twice. > > > > > > > > > > Max, > > > > > > > > > > Have you done performance measurements that show rmlocks to be > > > > > a win in this scenario? I did some patchs for UNIX domain > > > > > sockets to replace the rwlock there but it appeared not to have > > > > > a measurable impact on SQL benchmarks, presumbaly because the > > > > > read/write blend wasn't right and/or that wasnt a significant > > > > > source of overhead in the benchmark. I'd anticipate a much > > > > > more measurable improvement for pfil, but would be interested > > > > > in learning how much is seen? > > > > > > > > I had to roll an artificial benchmark in order to see a > > > > significant change (attached - it's a hack!). > > > > > > > > Using 3 threads on a 4 CPU machine I get the following results: > > > > null hook: ~13% +/- 2 > > > > mtx hook: up to 40% [*] > > > > rw hook: ~5% +/- 1 > > > > rm hook: ~35% +/- 5 > > > > > > Is that 13%/5%/35% faster or slower or improvement or degradation? > > > If "rw hook" (using rwlock like we have today?) is 5%, whas is the > > > baseline? > > > > > > I'm expecting that at least one of these should be a 0%... > > > > Sorry for the sparse explanation. All numbers above are gain with > > rmlocks i.e. rmlocks are faster in all scenarios. The test cases are > > different hook functions. Every hook has a DELAY(1) and a > > lock/unlock call around it of the respective lock type. read lock > > acquisitions for rw and rm. Please look at the code I posted a bit > > later for more details. > > Thanks for the clarification. > That makes rmlocks very interesting. > And the kind of lock that both ipf and ipfw could benefit from, > especially since you're planning on changing the pfil locks to be > this way. Are you (or is someone else?) intending on following > up moving ipfw to them, where appropriate? It's unclear yet, where they are appropriate. As the name suggests (read=20 mostly locks) they are for cases where you acquire a write lock only once=20 in a blue moon. As such the write lock acquisition is a very expensive=20 operation. This might be appropriate for some cases of IPFW rulesets,=20 but certainly not the general case. Things like address tables and=20 dynamic rules (states) take a lot of writes. It's not yet clear where=20 the balance for rmlocks really is and my benchmark doesn't answer that. =46or pfil the case is obvious as you hardly ever change the hooks. > I'm tempted to suggest them to other platforms...although I'd expect > some amount nay-saying because of NIH, it would be good if others > also picked up on them, if the benefits are this clear cut... Again ... only for a select set of cases. But they are really great for=20 cases where you don't want to use proper synchronization because of the=20 slow down and the relative small chance of ever hitting the race. Now we=20 have a tool to properly protect against these races without sacrificing=20 performance. =2D-=20 /"\ Best regards, | mlaier@freebsd.org \ / Max Laier | ICQ #67774661 X http://pf4freebsd.love2party.net/ | mlaier@EFnet / \ ASCII Ribbon Campaign | Against HTML Mail and News --nextPart16846558.i6R9qJmlUT Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.4 (FreeBSD) iD8DBQBHSWxMXyyEoT62BG0RAhQkAJ9XuE5DZegAIh789220yZQ+ttkphQCfTv3R M1RvV1uZKZANACUGRq5feYE= =XyvE -----END PGP SIGNATURE----- --nextPart16846558.i6R9qJmlUT--