From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 18 19:29:51 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A65C57FB; Tue, 18 Mar 2014 19:29:51 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 80FF71A4; Tue, 18 Mar 2014 19:29:51 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 6C877B946; Tue, 18 Mar 2014 15:29:50 -0400 (EDT) From: John Baldwin To: freebsd-hackers@freebsd.org Subject: Re: Tracking down what has inact pages locked up Date: Tue, 18 Mar 2014 15:05:47 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <53260B36.2070409@denninger.net> In-Reply-To: <53260B36.2070409@denninger.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Message-Id: <201403181505.47349.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 18 Mar 2014 15:29:50 -0400 (EDT) Cc: Alan Cox , Karl Denninger X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Mar 2014 19:29:51 -0000 On Sunday, March 16, 2014 4:36:06 pm Karl Denninger wrote: > Is there a reasonable way to determine who or what has that memory > locked up -- and thus why the vm system is not demoting that space into > the cache bucket so it can be freed (which, if my understanding is > correct, should be happening long before now!) I have a hackish thing (for 8.x, might work on 10.x) to let you figure out what is using up RAM. This should perhaps go into the base system at some point. Grab the bits at http://people.freebsd.org/~jhb/vm_objects/ You will want to build the kld first and use 'make load' to load it. It adds a new sysctl that dumps info about all the VM objects in the system. You can then build the 'vm_objects' tool and run it. It can take a while to run if you have NFS mounts, so I typically save its output to a file first and then use sort on the results. sort -n will show you the largest consumer of RAM, sort -n -k 3 will show you the largest consumer of inactive pages. Note that 'df' and 'ph' objects are anonymous, and that filename paths aren't always reliable, but this can still be useful. -- John Baldwin