From owner-freebsd-stable@FreeBSD.ORG Tue May 15 08:35:47 2007 Return-Path: X-Original-To: stable@freebsd.org Delivered-To: freebsd-stable@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9DC8616A402 for ; Tue, 15 May 2007 08:35:47 +0000 (UTC) (envelope-from uspoerlein@gmail.com) Received: from ik-out-1112.google.com (ik-out-1112.google.com [66.249.90.177]) by mx1.freebsd.org (Postfix) with ESMTP id 2746F13C44B for ; Tue, 15 May 2007 08:35:46 +0000 (UTC) (envelope-from uspoerlein@gmail.com) Received: by ik-out-1112.google.com with SMTP id c29so1202090ika for ; Tue, 15 May 2007 01:35:46 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=Al5KrbxvMdWNGgJnqCsgDUUWV9l89juWXVrraeeKXIeC/7FfkwWLpJEaksK1dWziS9EaPO45SB2SHVOAdXv5/iX/zbhBuMDb5xyVeSHrRze+WdtGsNP/gh5GTrIcRyAofsM4q2dikIlx5ZkfnLXjfWBN56SdmurhKldud72eoCY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=SdebvO8WPyq7YN+eiFEs5U2v2TUsVjjfMjUEDaY38elOw5BDhU+u0VJj6GcY4JdSBrFh6syTAB/idH2dH/ab+qeI06EkiHTWTxCh/aWj63gitjIdVdx/+j6uBpdLlhJT12WyGAGTbbErOzGpVMkGUDX6q2nKG+6rHbNwne+VWWU= Received: by 10.78.160.4 with SMTP id i4mr2541454hue.1179218145488; Tue, 15 May 2007 01:35:45 -0700 (PDT) Received: by 10.78.173.19 with HTTP; Tue, 15 May 2007 01:35:45 -0700 (PDT) Message-ID: <7ad7ddd90705150135i7d5c5d7ey1ed7001cf069ccb4@mail.gmail.com> Date: Tue, 15 May 2007 10:35:45 +0200 From: "Ulrich Spoerlein" To: "Marc G. Fournier" In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <7ad7ddd90705141007k77d7c335u3de54363a0eb8eaa@mail.gmail.com> Cc: stable@freebsd.org, Kris Kennaway Subject: Re: Known memory leak in 6-STABLE from April 1st? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 May 2007 08:35:47 -0000 On 5/14/07, Marc G. Fournier wrote: > > Now after doing some heavy IMAP testing (cyrus reconstruct of big > > maildirs) the system froze to a complete halt. Stupid me already > > rebooted the machine, tomorrow I'll try to break into DDB when it > > happens again. I also started recording top(1) memory output and > > sysctl vm.zone output. > > > > The main questions is: Were there any known memory leaks at the start > > of April? Any patches I should blindly try before spending several > > days on debugging this? > > Hrmmm ... long shot here, but what does: > > sysctl kern.ipc.numopensockets > > show over that period of time ... just wondering if we are somehow related on > problems here, just different symptoms ... Sorry no, nothing suspicous there. It bounces up and down, after killing all amavis, cyrus and postfix processes it came down to about 80. Right now it's at 280 again, and the memory is so small, that I can no longer grep(1) a 600MB file or do other useful stuff. This is the last vm.zone output, anything suspicous? What commands should I run (in DDB?) to see where the memory is going? Tue May 15 10:33:38 CEST 2007 vm.zone: ITEM SIZE LIMIT USED FREE REQUESTS FFS2 dinode: 256, 0, 83286, 10179, 1273390 FFS1 dinode: 128, 0, 0, 0, 0 FFS inode: 132, 0, 83286, 10761, 1273390 Mountpoints: 664, 0, 7, 11, 8 SWAPMETA: 276, 121576, 11, 17, 22 pfosfp: 28, 0, 0, 0, 0 pfospfen: 108, 0, 0, 0, 0 pfiaddrpl: 92, 0, 0, 0, 0 pfstatescrub: 28, 0, 0, 0, 0 pffrcent: 12, 50141, 0, 0, 0 pffrcache: 48, 10062, 0, 0, 0 pffrag: 48, 0, 0, 0, 0 pffrent: 16, 5075, 0, 0, 0 pfrkentry2: 156, 0, 0, 0, 0 pfrkentry: 156, 0, 0, 0, 0 pfrktable: 1240, 0, 0, 0, 0 pfpooladdrpl: 68, 0, 0, 0, 0 pfaltqpl: 128, 0, 0, 0, 0 pfstatepl: 260, 10005, 0, 0, 0 pfrulepl: 604, 0, 0, 0, 0 pfsrctrpl: 100, 0, 0, 0, 0 rtentry: 132, 0, 37, 108, 3848 ripcb: 180, 25608, 0, 110, 33 sackhole: 20, 0, 0, 676, 3672 tcpreass: 20, 1690, 0, 676, 1365 hostcache: 76, 15400, 5, 245, 317 syncache: 100, 15366, 0, 195, 27209 tcptw: 48, 5148, 0, 624, 34253 tcpcb: 464, 25600, 29, 203, 212261 inpcb: 180, 25608, 29, 389, 212261 udpcb: 180, 25608, 19, 179, 493092 ipq: 32, 904, 0, 0, 0 unpcb: 144, 25623, 213, 3405, 300681 socket: 356, 25608, 263, 5567, 1006076 KNOTE: 68, 0, 0, 616, 724767 PIPE: 408, 0, 10, 818, 1544936 DIRHASH: 1024, 0, 861, 247, 4143 NFSNODE: 460, 0, 1, 23, 7 NFSMOUNT: 480, 0, 1, 15, 2 L VFS Cache: 291, 0, 402, 326, 6124 S VFS Cache: 68, 0, 87410, 16638, 1831436 NAMEI: 1024, 0, 0, 660, 105375628 VNODEPOLL: 76, 0, 0, 250, 8 VNODE: 272, 0, 83341, 12405, 1273709 ata_composit: 196, 0, 0, 0, 0 ata_request: 204, 0, 0, 76, 34 g_bio: 132, 0, 0, 696, 16904466 ACL UMA zone: 388, 0, 0, 0, 0 mbuf_jumbo_1: 16384, 0, 0, 0, 0 mbuf_jumbo_9: 9216, 0, 0, 0, 0 mbuf_jumbo_p: 4096, 0, 0, 0, 0 mbuf_cluster: 2048, 25600, 929, 325, 1276052 mbuf: 256, 0, 931, 554, 141685396 mbuf_packet: 256, 0, 806, 679, 104799211 VMSPACE: 296, 0, 127, 328, 1617479 UPCALL: 44, 0, 5, 229, 10 KSEGRP: 88, 0, 386, 174, 556 THREAD: 376, 0, 394, 286, 292710 PROC: 536, 0, 170, 215, 1617545 Files: 72, 0, 580, 4296, 40984983 4096: 4096, 0, 229, 542, 3118296 2048: 2048, 0, 229, 547, 4814216 1024: 1024, 0, 347, 389, 9329214 512: 512, 0, 186, 398, 1319246 256: 256, 0, 919, 3941, 5484210 128: 128, 0, 2516, 6034, 28789911 64: 64, 0, 5339, 7110, 21344537 32: 32, 0, 2687, 7257, 10126384 16: 16, 0, 2861, 996, 9327126 mt_zone: 1024, 0, 189, 3, 189 DP fakepg: 72, 0, 0, 0, 0 PV ENTRY: 24, 2067410, 126063, 334892, 1426534413 MAP ENTRY: 68, 0, 8613, 7795, 104088221 KMAP ENTRY: 68, 57344, 1001, 1127, 208001 MAP: 192, 0, 7, 13, 7 VM OBJECT: 132, 0, 82483, 14058, 43364599 128 Bucket: 524, 0, 2145, 865, 8052 64 Bucket: 268, 0, 88, 80, 274 32 Bucket: 140, 0, 69, 99, 180 16 Bucket: 76, 0, 42, 58, 118 UMA Hash: 128, 0, 3, 27, 6 UMA RCntSlab: 104, 0, 627, 2, 7918 UMA Slabs: 64, 0, 1933, 427, 31248 UMA Zones: 480, 0, 83, 5, 83 UMA Kegs: 140, 0, 83, 13, 83 Uli