From owner-freebsd-stable@FreeBSD.ORG Mon Apr 4 22:43:37 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 465A4106564A for ; Mon, 4 Apr 2011 22:43:37 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta01.westchester.pa.mail.comcast.net (qmta01.westchester.pa.mail.comcast.net [76.96.62.16]) by mx1.freebsd.org (Postfix) with ESMTP id 042338FC12 for ; Mon, 4 Apr 2011 22:43:36 +0000 (UTC) Received: from omta23.westchester.pa.mail.comcast.net ([76.96.62.74]) by qmta01.westchester.pa.mail.comcast.net with comcast id TaWX1g0041c6gX851ajdfQ; Mon, 04 Apr 2011 22:43:37 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta23.westchester.pa.mail.comcast.net with comcast id Tajb1g00r1t3BNj3jajcxa; Mon, 04 Apr 2011 22:43:36 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 197EF9B429; Mon, 4 Apr 2011 15:43:34 -0700 (PDT) Date: Mon, 4 Apr 2011 15:43:34 -0700 From: Jeremy Chadwick To: Boris Kochergin Message-ID: <20110404224334.GA64297@icarus.home.lan> References: <4D972FF7.6010901@acm.poly.edu> <20110402153315.GP78089@deviant.kiev.zoral.com.ua> <4D974393.80606@acm.poly.edu> <4D9A307F.9070408@acm.poly.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4D9A307F.9070408@acm.poly.edu> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Kostik Belousov , FreeBSD-STABLE Mailing List Subject: Re: Kernel memory leak in 8.2-PRERELEASE? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Apr 2011 22:43:37 -0000 On Mon, Apr 04, 2011 at 04:56:31PM -0400, Boris Kochergin wrote: > On 04/02/11 11:41, Boris Kochergin wrote: > >On 04/02/11 11:33, Kostik Belousov wrote: > >>On Sat, Apr 02, 2011 at 10:17:27AM -0400, Boris Kochergin wrote: > >>>Ahoy. This morning, I awoke to the following on one of my servers: > >>> > >>>pid 59630 (httpd), uid 80, was killed: out of swap space > >>>pid 59341 (find), uid 0, was killed: out of swap space > >>>pid 23134 (irssi), uid 1001, was killed: out of swap space > >>>pid 49332 (sshd), uid 1001, was killed: out of swap space > >>>pid 69074 (httpd), uid 0, was killed: out of swap space > >>>pid 11879 (eggdrop-1.6.19), uid 1001, was killed: out of swap space > >>>... > >>> > >>>And so on. > >>> > >>>The machine is: > >>> > >>>FreeBSD exodus.poly.edu 8.2-PRERELEASE FreeBSD 8.2-PRERELEASE #2: Thu > >>>Dec 2 11:39:21 EST 2010 > >>>spawk@exodus.poly.edu:/usr/obj/usr/src/sys/EXODUS amd64 > >>> > >>>10:13AM up 120 days, 20:06, 2 users, load averages: 0.00, 0.01, 0.00 > >>> > >>>The memory line from top intrigued me: > >>> > >>>Mem: 16M Active, 48M Inact, 6996M Wired, 229M Cache, 828M Buf, > >>>605M Free > >>> > >>>The machine has 8 gigs of memory, and I don't know what all that wired > >>>memory is being used for. There is a large-ish (6 x 1.5-TB) ZFS RAID-Z2 > >>>on it which has had a disk in the UNAVAIL state for a few months: > >>> > >>># zpool status > >>> pool: home > >>> state: DEGRADED > >>>status: One or more devices could not be used because the label is > >>>missing or > >>> invalid. Sufficient replicas exist for the pool to continue > >>> functioning in a degraded state. > >>>action: Replace the device using 'zpool replace'. > >>> see: http://www.sun.com/msg/ZFS-8000-4J > >>> scrub: none requested > >>>config: > >>> > >>> NAME STATE READ WRITE CKSUM > >>> home DEGRADED 0 0 0 > >>> raidz2 DEGRADED 0 0 0 > >>> ada0 ONLINE 0 0 0 > >>> ada1 ONLINE 0 0 0 > >>> ada2 ONLINE 0 0 0 > >>> ada3 ONLINE 0 0 0 > >>> ada4 ONLINE 0 0 0 > >>> ada5 UNAVAIL 0 85 11 experienced > >>>I/O failures > >>> > >>>errors: No known data errors > >>> > >>>"vmstat -m" and "vmstat -z" output: > >>> > >>>http://acm.poly.edu/~spawk/vmstat-m.txt > >>>http://acm.poly.edu/~spawk/vmstat-z.txt > >>> > >>>Anyone have a clue? I know it's just going to happen again if I reboot > >>>the machine. It is still up in case there are diagnostics for > >>>me to run. > >>Try r218795. Most likely, your issue is not leak. > > > >Thanks. Will update to today's 8-STABLE and report back. > > > >-Boris > > The problem persists, I'm afraid, and seems to have crept up a lot > more quickly than before: > > # uname -a > FreeBSD exodus.poly.edu 8.2-STABLE FreeBSD 8.2-STABLE #3: Sat Apr 2 > 11:48:43 EDT 2011 > spawk@exodus.poly.edu:/usr/obj/usr/src/sys/EXODUS amd64 > > Mem: 314M Active, 955M Inact, 6356M Wired, 267M Cache, 828M Buf, 18M Free > > Any ideas for a diagnostic recourse? Can you please provide the details I requested here? Thanks. http://lists.freebsd.org/pipermail/freebsd-stable/2011-April/062147.html -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB |