From owner-freebsd-stable@FreeBSD.ORG Thu Sep 25 16:08:24 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 050F5106568D for ; Thu, 25 Sep 2008 16:08:24 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from QMTA04.emeryville.ca.mail.comcast.net (qmta04.emeryville.ca.mail.comcast.net [76.96.30.40]) by mx1.freebsd.org (Postfix) with ESMTP id DC0468FC13 for ; Thu, 25 Sep 2008 16:08:23 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from OMTA07.emeryville.ca.mail.comcast.net ([76.96.30.59]) by QMTA04.emeryville.ca.mail.comcast.net with comcast id K06N1a00d1GXsucA448P1X; Thu, 25 Sep 2008 16:08:23 +0000 Received: from koitsu.dyndns.org ([67.180.253.227]) by OMTA07.emeryville.ca.mail.comcast.net with comcast id K48M1a00N4v8bD78T48Nql; Thu, 25 Sep 2008 16:08:23 +0000 X-Authority-Analysis: v=1.0 c=1 a=pl5uMaLHwrUA:10 a=20ZdTaP2r60A:10 a=QycZ5dHgAAAA:8 a=Sy3vyHCys7K8U2cHiLsA:9 a=zrkFD7_zZH1qNqOpONgA:7 a=jBntfdy1STXsoBrwz3du4-rSBtsA:4 a=EoioJ0NPDVgA:10 a=LY0hPdMaydYA:10 Received: by icarus.home.lan (Postfix, from userid 1000) id D7CC6C9419; Thu, 25 Sep 2008 09:08:21 -0700 (PDT) Date: Thu, 25 Sep 2008 09:08:21 -0700 From: Jeremy Chadwick To: Alexandre Sunny Kovalenko Message-ID: <20080925160821.GC16220@icarus.home.lan> References: <1222357601.978.11.camel@RabbitsDen> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1222357601.978.11.camel@RabbitsDen> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-stable@freebsd.org, admin@kkip.pl, Pete French Subject: Re: vm.kmem_size settings doesn't affect loader? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Sep 2008 16:08:24 -0000 On Thu, Sep 25, 2008 at 11:46:41AM -0400, Alexandre Sunny Kovalenko wrote: > On Thu, 2008-09-25 at 16:10 +0100, Pete French wrote: > > > These are the tuning settings I use: > > > > > > vm.kmem_size="1536M" > > > vm.kmem_size_max="1536M" > > > vfs.zfs.arc_min="16M" > > > vfs.zfs.arc_max="64M" > > > > > > > > The entire copying process took almost 2 hours. Not once did I > > > experience kmem exhaustion. I can *guarantee* that I would have crashed > > > the box numerous times had I not tuned the machine with the values > > > above. > > > > I am interested in the last two values you have there - I also use ZFS and > > tune to prevent memory exhaustion. I tune kmem_size, but I have not bothered > > toi set the arc_min - and the default appears to be 16M anyway. The arc_max > > value I do set, but I got dreadful performance when I had it at 6M, so > > I upped it to 128M. This has not resulted in any panics, and gives much better > > performance. As I understand it there is only one ARC, and it comes from > > kernel space too, so setting it to an extra 64 meg should not use that much > > more memory, yes ? > On my RELENG_7 (circa August 14th) > > vfs.zfs.arc_min: 16777216 > vfs.zfs.arc_max: 402653184 > > seem to be set by default. Yes, that is "384M". The default arc_max value is calculated based on numerous things which are system-specific. The code is in arc_init() here, and the variable is called arc_c_max: src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c > I have > > vm.kmem_size_max="512M" > vm.kmem_size="512M" > > set in my /boot/loader.conf, and Your kmem is set to 512MB, yet your maximum ARC size is 384MB. That leaves 128MB for the entire rest of the kmap, including other pieces of ZFS. I can assure you that with a heavier ZFS load, your machine will easily panic. > vfs.ufs.dirhash_maxmem=16777216 > kern.maxvnodes=100000 > > set in /etc/sysctl.conf vfs.ufs.dirhash_maxmem is for UFS/UFS2. Speculative: kern.maxvnodes is vfs-related, and ZFS does use vfs (duh). Increasing kern.maxvnodes means using more kernel memory (no idea if that's kmem or not), which could mean you're decreasing the amount of memory available for ZFS/ARC as well (e.g. panic more likely). > This is amd64 with 3GB of real memory. ZFS pool is used to serve music, > digital pictures and video to the rest of household, as well as provide > target for the weekly backups from couple of Windows desktops, so use is > not that heavy. Machine has been up for 22 days and has shown no ill > effects that I can notice. The lesser load on ZFS, and possibly the system in general, is probably what's keeping you afloat. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |