From owner-cvs-all@FreeBSD.ORG Mon Nov 13 02:27:28 2006 Return-Path: X-Original-To: cvs-all@freebsd.org Delivered-To: cvs-all@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AED8F16A415 for ; Mon, 13 Nov 2006 02:27:28 +0000 (UTC) (envelope-from kip.macy@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3297743D72 for ; Mon, 13 Nov 2006 02:27:18 +0000 (GMT) (envelope-from kip.macy@gmail.com) Received: by nf-out-0910.google.com with SMTP id l23so319522nfc for ; Sun, 12 Nov 2006 18:27:17 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:references; b=jM4uWxnYvTZAazChhRsFUfHc7nmF/ZMaabIJ7A+i5Pdc9n4SOQ++IHD3bk6LM0BmYLIRUtmXmCmJRLUzQsK3yqySbwdWdC8KWm0DfEaoS+iKUNCxNQCJpf2HCEmHjxSR46T/xSAVt1TpXMk62ZKAhIPBPXnfZPUg30/Mm+vOqQU= Received: by 10.82.172.15 with SMTP id u15mr558113bue.1163384836683; Sun, 12 Nov 2006 18:27:16 -0800 (PST) Received: by 10.82.191.20 with HTTP; Sun, 12 Nov 2006 18:27:16 -0800 (PST) Message-ID: Date: Sun, 12 Nov 2006 18:27:16 -0800 From: "Kip Macy" To: "Alan Cox" In-Reply-To: <200611122148.kACLmYDV006108@repoman.freebsd.org> MIME-Version: 1.0 References: <200611122148.kACLmYDV006108@repoman.freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: cvs-src@freebsd.org, src-committers@freebsd.org, cvs-all@freebsd.org Subject: Re: cvs commit: src/sys/amd64/amd64 pmap.c src/sys/arm/arm pmap.c src/sys/i386/i386 pmap.c src/sys/ia64/ia64 pmap.c src/sys/powerpc/powerpc mmu_oea.c src/sys/sparc64/sparc64 pmap.c src/sys/sun4v/sun4v pmap.c src/sys/vm vm_fault.c vm_kern.c X-BeenThere: cvs-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: CVS commit messages for the entire tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Nov 2006 02:27:28 -0000 Thanks Alan, this is a definite improvement. For the benefit of other readers: below I have the ten locks with longest max_wait (total time in microseconds spent waiting for the lock) during the compile phase of a "make -j32 buildkernel" on a T2000 (8 cores w/ 4 threads per core). The reduction in turnstile contention is a direct result of reducing contention on the page queue mutex. before: max total wait_total count avg wait_avg cnt_hold cnt_lock name 5 7266196 206805560 7522452 0 27 48266619 0 /usr/flatstor/shared/freebsd/kmacy/src/sys/kern/subr_turnstile.c:487 (turnstile chain) 457 528521 180592127 550284 0 328 1469872 0 /usr/flatstor/shared/freebsd/kmacy/src/sys/vm/vm_fault.c:844 (vm page queue mutex) 15057461 1679582934 117520488 87978 19090 1335 0 0 /usr/flatstor/shared/freebsd/kmacy/src/sys/ufs/ffs/ffs_vnops.c:366 (ufs) 214 1076256 112489341 559032 1 201 1520471 0 /usr/flatstor/shared/freebsd/kmacy/src/sys/vm/vm_fault.c:902 (vm page queue mutex) 424 8250360 105249196 559031 14 188 1767340 0 /usr/flatstor/shared/freebsd/kmacy/src/sys/sun4v/sun4v/pmap.c:956 (vm page queue mutex) 72563452121 218316084315 94216669 452713 482239 208 0 0 /usr/flatstor/shared/freebsd/kmacy/src/sys/kern/vfs_default.c:263 (nfs) 23 1349030 14049785 280685 4 50 923679 0 /usr/flatstor/shared/freebsd/kmacy/src/sys/kern/kern_idle.c:121 (sched lock) 73 214117 11078161 63944 3 173 2505 0 /usr/flatstor/shared/freebsd/kmacy/src/sys/nfsclient/nfs_socket.c:1235 (Giant) 42 92768 10431233 40012 2 260 122966 0 /usr/flatstor/shared/freebsd/kmacy/src/sys/vm/vm_fault.c:342 (vm page queue mutex) 6429 2155581 6645086 18297 117 363 105550 0 /usr/flatstor/shared/freebsd/kmacy/src/sys/vm/vm_object.c:651 (vm page queue mutex) after: 15311133 1125368615 119287635 94194 11947 1266 0 0 /usr/flatstor/shared/freebsd/kmacy/src/sys/ufs/ffs/ffs_vnops.c:366 (lockmgr:ufs) 546 4708767 93794515 326286 14 287 533239 713529 /usr/flatstor/shared/freebsd/kmacy/src/sys/sun4v/sun4v/pmap.c:955 (sleep mutex:vm page queue mutex) 36217304 520567907 53857789 243928 2134 220 0 0 /usr/flatstor/shared/freebsd/kmacy/src/sys/kern/vfs_default.c:263 (lockmgr:nfs) 210 590989 38380260 326286 1 117 335955 160913 /usr/flatstor/shared/freebsd/kmacy/src/sys/vm/vm_fault.c:899 (sleep mutex:vm page queue mutex) 16 1702987 25490112 2068283 0 12 7329498 0 /usr/flatstor/shared/freebsd/kmacy/src/sys/kern/subr_turnstile.c:487 (spin mutex:turnstile chain) 22 962236 7498290 194839 4 38 665502 0 /usr/flatstor/shared/freebsd/kmacy/src/sys/kern/kern_idle.c:121 (spin mutex:sched lock) 9 28157 4130053 7242 3 570 9959 3040 /usr/flatstor/shared/freebsd/kmacy/src/sys/sun4v/sun4v/pmap.c:1162 (sleep mutex:vm page queue mutex) 39 44908 2629172 21967 2 119 21879 11174 /usr/flatstor/shared/freebsd/kmacy/src/sys/vm/vm_fault.c:342 (sleep mutex:vm page queue mutex) 4 20673 2265592 21708 0 104 88367 0 /usr/flatstor/shared/freebsd/kmacy/src/sys/kern/subr_trap.c:191 (spin mutex:sched lock) 31 11117 2231209 7243 1 308 7419 3023 /usr/flatstor/shared/freebsd/kmacy/src/sys/kern/sys_pipe.c:827 (sleep mutex:vm page queue mutex) On 11/12/06, Alan Cox wrote: > > alc 2006-11-12 21:48:34 UTC > > FreeBSD src repository > > Modified files: > sys/amd64/amd64 pmap.c > sys/arm/arm pmap.c > sys/i386/i386 pmap.c > sys/ia64/ia64 pmap.c > sys/powerpc/powerpc mmu_oea.c > sys/sparc64/sparc64 pmap.c > sys/sun4v/sun4v pmap.c > sys/vm vm_fault.c vm_kern.c > Log: > Make pmap_enter() responsible for setting PG_WRITEABLE instead > of its caller. (As a beneficial side-effect, a high-contention > acquisition of the page queues lock in vm_fault() is eliminated.) > > Revision Changes Path > 1.573 +3 -1 src/sys/amd64/amd64/pmap.c > 1.73 +4 -1 src/sys/arm/arm/pmap.c > 1.577 +3 -1 src/sys/i386/i386/pmap.c > 1.180 +2 -0 src/sys/ia64/ia64/pmap.c > 1.112 +3 -2 src/sys/powerpc/powerpc/mmu_oea.c > 1.160 +4 -1 src/sys/sparc64/sparc64/pmap.c > 1.6 +3 -1 src/sys/sun4v/sun4v/pmap.c > 1.223 +2 -7 src/sys/vm/vm_fault.c > 1.124 +1 -1 src/sys/vm/vm_kern.c >