From owner-freebsd-stable@FreeBSD.ORG Thu Sep 25 14:14:10 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1D5711065696 for ; Thu, 25 Sep 2008 14:14:10 +0000 (UTC) (envelope-from admin@kkip.pl) Received: from mainframe.kkip.pl (kkip.pl [87.105.164.78]) by mx1.freebsd.org (Postfix) with ESMTP id B541B8FC1D for ; Thu, 25 Sep 2008 14:14:09 +0000 (UTC) (envelope-from admin@kkip.pl) Received: from [192.168.0.10] by mainframe.kkip.pl with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.69 (FreeBSD)) (envelope-from ) id 1KirbW-00047i-Ex for freebsd-stable@freebsd.org; Thu, 25 Sep 2008 16:14:07 +0200 Message-ID: <48DB9CAA.9060807@kkip.pl> Date: Thu, 25 Sep 2008 16:14:02 +0200 From: Bartosz Stec User-Agent: Thunderbird 2.0.0.16 (Windows/20080708) MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <48DB6772.1060400@kkip.pl> <20080925130227.GA13497@icarus.home.lan> In-Reply-To: <20080925130227.GA13497@icarus.home.lan> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Authenticated-User: admin@kkip.pl X-Authenticator: plain X-Sender-Verify: SUCCEEDED (sender exists & accepts mail) X-Spam-Score: -8.9 X-Spam-Score-Int: -88 X-Exim-Version: 4.69 (build at 26-Jun-2008 18:19:28) X-Date: 2008-09-25 16:14:07 X-Connected-IP: 192.168.0.10:4011 X-Message-Linecount: 84 X-Body-Linecount: 71 X-Message-Size: 3328 X-Body-Size: 2772 X-Received-Count: 1 X-Recipient-Count: 1 X-Local-Recipient-Count: 1 X-Local-Recipient-Defer-Count: 0 X-Local-Recipient-Fail-Count: 0 Subject: Re: vm.kmem_size settings doesn't affect loader? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Sep 2008 14:14:10 -0000 Jeremy Chadwick pisze: > On Thu, Sep 25, 2008 at 12:26:58PM +0200, Bartosz Stec wrote: > >> Today I've experienced zfs-related kernel panic. Log says: >> >> savecore: reboot after panic: kmem_malloc(131072): kmem_map too >> small: 327684096 total allocated >> >> Reported amount of memory (327684096) is wrong, because i made suggested >> tuning in my loader.conf: >> >> vm.kmem_size="512M" >> vm.kmem_size_max="512M" >> >> Just to be sure: >> >> # sysctl vm | grep kmem >> vm.kmem_size: 536870912 >> vm.kmem_size_min: 0 >> vm.kmem_size_max: 536870912 >> vm.kmem_size_scale: 3 >> >> Am I missing something? >> > > I believe this is normal. The amount shown in "total allocated" will > not match what you have vm.kmem_size or vm.kmem_size_max set to. > Someone more familiar with the VM can explain why this is, but as I > understand it, it's 100% normal. > Thanks for explaining. This part of tuning guide confused me: "I was able to generate the following kernel panic in less than a minute by copying files from a linux server connected via gigabit crossover: Apr 8 06:46:08 nas savecore: reboot after panic: kmem_malloc(131072): kmem_map too small: 528273408 total allocated As you can see, this is *with* vm.kmem_size="512M"!" I've just expected similiar numbers in my case. > Your options are: > > 1) Consider increasing it from 512M to something like 1.5GB; do not > increase it past that on RELENG_7, as there isn't support for more than > 2GB total. For example, on a 1GB memory machine, I often recommend > 768M. On 2GB machines, 1536M. You will need to run -CURRENT if you > want more. > > 2) Tune ZFS aggressively. Start by setting vfs.zfs.arc_min="16M" > and vfs.zfs.arc_max="64M". > > If your machine has some small amount of memory (768MB, 1GB, etc.), > then you probably shouldn't be using ZFS. > > Problem occured on i386 machine with 1GB of memory and 7.1-pre (3HDD, 40GB, RAIDZ1). I know that i386 is highly unrecommended for ZFS, but it's just a home box for testing and learning purposes - I just want to know what I'm doing and what should I expect when I decide to put ZFS on server machines :) Currently, from posts on freebsd-fs, I conclude that even with a gigs of kmem and using AMD64, we still can experience panic from kmem_malloc. Manual tuning is hard for me because I'm not familiar with BSD kernel code nor kernel memory management. I'm just an end-user who love concepts of ZFS and wait for it to be (more) stable. Of course I've followed tuning guide carefully. So, for now, I will add another 512MB of memory and increse vm.kmem_size and do more tests. Thanks once again for explaining and suggestions. Good luck with ZFS everyone! :) -- Bartosz Stec