From owner-freebsd-stable@FreeBSD.ORG Tue Jan 12 10:06:28 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 890FB1065672 for ; Tue, 12 Jan 2010 10:06:28 +0000 (UTC) (envelope-from mail@maxlor.com) Received: from mxout002.mail.hostpoint.ch (mxout002.mail.hostpoint.ch [217.26.49.181]) by mx1.freebsd.org (Postfix) with ESMTP id 451DD8FC14 for ; Tue, 12 Jan 2010 10:06:27 +0000 (UTC) Received: from [10.0.2.10] (helo=asmtp001.mail.hostpoint.ch) by mxout002.mail.hostpoint.ch with esmtp (Exim 4.69 (FreeBSD)) (envelope-from ) id 1NUdO2-0002LK-7C for freebsd-stable@freebsd.org; Tue, 12 Jan 2010 10:50:06 +0100 Received: from [82.136.101.114] (helo=atlantis.intranet) by asmtp001.mail.hostpoint.ch with esmtpa (Exim 4.69 (FreeBSD)) (envelope-from ) id 1NUdO1-000CBn-VD for freebsd-stable@freebsd.org; Tue, 12 Jan 2010 10:50:06 +0100 Received: from ws-mni.intranet.assentis.info (pub212004072186.fx-hfc.datazug.ch [212.4.72.186]) by atlantis.intranet (Postfix) with ESMTPSA id BD167184F0 for ; Tue, 12 Jan 2010 10:50:05 +0100 (CET) X-Authenticated-Sender-Id: mail@maxlor.com From: Benjamin Lutz To: freebsd-stable@freebsd.org Date: Tue, 12 Jan 2010 10:50:04 +0100 User-Agent: KMail/1.9.9 X-Face: $Ov27?7*N,h60fIEfNJdb!m,@#4T/d; 1hw|W0zvsHM(a$Yn6BYQ0^SEEXvi8>D`|V*F"=?utf-8?q?=5F+=0A=09R2?=@Aq>+mNb4`,'[[%z9v0Fa~]AD1}xQO3|>b.z&}l#R-_(P`?@Mz"kS; XC>Eti,i3>%@=?utf-8?q?g=3F=0A=094f?=,\c7|Ghwb&ky$b2PJ^\0b83NkLsFKv|smL/cI4UD%Tu8alAD MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <201001121050.04982.mail@maxlor.com> Subject: Please explain FreeBSD 8.0/ZFS memory usage patterns X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Jan 2010 10:06:28 -0000 Hello, I've set up a FreeBSD-8.0/amd64 system with ZFS as a home server, and I'm a bit puzzled by it's memory usage pattern; I'm seeing total memory usage oscillate between roughly 50% and 90% of my RAM. So I've tracked memory usage over the course of about 12 hours and graphed it here: http://www.maxlor.com/temp/mem.png The unit is MiB; kern = vm.kmem_size, proc = sum of RSS of all processes (since some memory will be shared between processes, this is an upper bound), active = vm.stats.vm.v_active_count, wire = vm.stats.vm.v_wire_count, arc = kstat.zfs.misc.arcstats.size A few other details about the machine in question: avail memory = 3581587456 (3415 MiB) kstat.zfs.misc.arcstats.c_min: 93800960 (89 MiB) kstat.zfs.misc.arcstats.c_max: 750407680 (716 MiB) The machine runs many services (mostly mail, filesharing via NFS, apache, squeezebox server, rtorrent), but is generally very lightly loaded (as I said, it's a home server with mostly 1 user). As you can see from the graph and numbers, memory pressure seems to be about nonexistant. There is a root UFS file system, but it sees little use; /home, /usr and /var are on ZFS. Now, I have a couple of questions: After subtracting the ARC size from the wired memory size, there are still about 500 MiB of wired memory left that I can't account for. This is much higher than I'm used to seeing on FreeBSD 7.2/i386 systems without ZFS, where the wired amount is typically in the 50 MiB range. What are those 500 MiB used for, or how do I find out? Same goes for active memory: if I subtract the "proc" amount from the "active" amount, there are 500 MiB left over (probably more, since proc is an upper bound) which I presume are used by the kernel... but what for? Does ZFS require that much memory for things other than the ARC? In the first half of the graph, there's a torrent download running, which I assume causes the large fluctuations in memory usage. However, memory usage of the torrent program is pretty much constant while it runs; is FreeBSD simply behaving lazily here, not bothering to free memory as long as there isn't any memory pressure? Any insight into why the numbers are as they are is appreciated. Cheers Benjamin