From owner-svn-src-all@freebsd.org Mon Nov 30 17:28:28 2015 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B625BA3CD7C for ; Mon, 30 Nov 2015 17:28:28 +0000 (UTC) (envelope-from mckusick@mckusick.com) Received: from chez.mckusick.com (chez.mckusick.com [IPv6:2001:5a8:4:7e72:d250:99ff:fe57:4030]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9BF08157F; Mon, 30 Nov 2015 17:28:28 +0000 (UTC) (envelope-from mckusick@mckusick.com) Received: from chez.mckusick.com (localhost [IPv6:::1]) by chez.mckusick.com (8.15.2/8.14.9) with ESMTP id tAUHSQjO021076; Mon, 30 Nov 2015 09:28:27 -0800 (PST) (envelope-from mckusick@chez.mckusick.com) Message-Id: <201511301728.tAUHSQjO021076@chez.mckusick.com> From: Kirk McKusick To: koobs@FreeBSD.org Subject: Re: svn commit: r291460 - head/sys/kern cc: svn-src-all@freebsd.org In-reply-to: <565BCA11.4090600@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <21074.1448904506.1@chez.mckusick.com> Date: Mon, 30 Nov 2015 09:28:26 -0800 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Nov 2015 17:28:28 -0000 > To: Kirk McKusick , src-committers@freebsd.org, > svn-src-all@freebsd.org, svn-src-head@freebsd.org > From: Kubilay Kocak > Subject: Re: svn commit: r291460 - head/sys/kern > Date: Mon, 30 Nov 2015 15:01:21 +1100 > > On 30/11/2015 8:42 AM, Kirk McKusick wrote: >> Author: mckusick >> Date: Sun Nov 29 21:42:26 2015 >> New Revision: 291460 >> URL: https://svnweb.freebsd.org/changeset/base/291460 >> >> Log: >> As the kernel allocates and frees vnodes, it fully initializes them >> on every allocation and fully releases them on every free. These >> are not trivial costs: it starts by zeroing a large structure then >> initializes a mutex, a lock manager lock, an rw lock, four lists, >> and six pointers. And looking at vfs.vnodes_created, these operations >> are being done millions of times an hour on a busy machine. >> >> As a performance optimization, this code update uses the uma_init >> and uma_fini routines to do these initializations and cleanups only >> as the vnodes enter and leave the vnode_zone. With this change the >> initializations are only done kern.maxvnodes times at system startup >> and then only rarely again. The frees are done only if the vnode_zone >> shrinks which never happens in practice. For those curious about the >> avoided work, look at the vnode_init() and vnode_fini() functions in >> kern/vfs_subr.c to see the code that has been removed from the main >> vnode allocation/free path. >> >> Reviewed by: kib >> Tested by: Peter Holm >> >> Modified: >> head/sys/kern/vfs_subr.c >> > > Kirk, > > Very interesting. > > Any estimation / expectation on the performance benefit of this, > and in what scenario's / use-cases it might be particularly > beneficial? Any benchmarks or tests you can share? > > ./koobs The senario in which it will be beneficial is one in which many files are being accessed (independent of filesystem type(s) on which they are being accessed). The key value to check is: sysctl vfs.vnodes_created This is the count of vnode creations that are no longer incurring the creation/free overhead. If this number is large and/or growing quickly, you are getting the savings. An example that will drive this number through the roof is: find / -type f -ls >/dev/null Most systems have enough vnodes that they can cache all the files that they access (see `sysctl kern.maxvnodes'). But when the cache is not big enough, this change will ease the pain of reallocating vnodes. Kirk McKusick