From owner-freebsd-bugs@FreeBSD.ORG Fri Jan 2 21:01:13 2009 Return-Path: Delivered-To: freebsd-bugs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0F4F110656D4; Fri, 2 Jan 2009 21:01:13 +0000 (UTC) (envelope-from jason.dicioccio@ods.org) Received: from mail-ew0-f21.google.com (mail-ew0-f21.google.com [209.85.219.21]) by mx1.freebsd.org (Postfix) with ESMTP id 99B598FC1E; Fri, 2 Jan 2009 21:01:11 +0000 (UTC) (envelope-from jason.dicioccio@ods.org) Received: by ewy14 with SMTP id 14so7793936ewy.19 for ; Fri, 02 Jan 2009 13:01:10 -0800 (PST) Received: by 10.210.119.16 with SMTP id r16mr14077795ebc.47.1230930070340; Fri, 02 Jan 2009 13:01:10 -0800 (PST) Received: by 10.210.80.20 with HTTP; Fri, 2 Jan 2009 13:01:10 -0800 (PST) Message-ID: Date: Fri, 2 Jan 2009 13:01:10 -0800 From: "Jason DiCioccio" To: vwe@freebsd.org In-Reply-To: MIME-Version: 1.0 References: <200812311353.mBVDraLJ042040@freefall.freebsd.org> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net@freebsd.org, freebsd-bugs@freebsd.org, bugs-followup@freebsd.org Subject: Re: kern/130059: [panic] Leaking 50k mbufs/hour X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jan 2009 21:01:13 -0000 Adding a bit more info.. Here's the netstat -m and vmstat -z output after the mbufs have had a while to build up: -- netstat -m -- 959510/235/959745 mbufs in use (current/cache/total) 257/133/390/25600 mbuf clusters in use (current/cache/total/max) 257/127 mbuf+clusters out of packet secondary zone in use (current/cache) 0/117/117/12800 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) 240391K/792K/241184K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/152/6656 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 2716 requests for I/O initiated by sendfile 0 calls to protocol drain routines -- vmstat -z -- ITEM SIZE LIMIT USED FREE REQUESTS FAILURES UMA Kegs: 128, 0, 99, 21, 99, 0 UMA Zones: 120, 0, 99, 21, 99, 0 UMA Slabs: 64, 0, 1132, 48, 15590, 0 UMA RCntSlabs: 104, 0, 312, 21, 856, 0 UMA Hash: 128, 0, 4, 26, 7, 0 16 Bucket: 76, 0, 21, 29, 64, 0 32 Bucket: 140, 0, 32, 24, 107, 0 64 Bucket: 268, 0, 34, 8, 142, 0 128 Bucket: 524, 0, 628, 9, 45698, 5387 VM OBJECT: 124, 0, 9136, 32404, 20553589, 0 MAP: 140, 0, 7, 49, 7, 0 KMAP ENTRY: 68, 56560, 17, 487, 153898, 0 MAP ENTRY: 68, 0, 15360, 2336, 53481139, 0 DP fakepg: 72, 0, 0, 0, 0, 0 mt_zone: 72, 0, 194, 71, 194, 0 16: 16, 0, 1996, 643, 81641919, 0 32: 32, 0, 2542, 283, 9470304, 0 64: 64, 0, 4584, 7039, 61275174, 0 128: 128, 0, 1101, 549, 6280868, 0 256: 256, 0, 1299, 366, 20697494, 0 512: 512, 0, 145, 279, 79529, 0 1024: 1024, 0, 173, 71, 532899, 0 2048: 2048, 0, 104, 146, 14198, 0 4096: 4096, 0, 266, 63, 1242097, 0 Files: 76, 0, 1595, 655, 16940144, 0 TURNSTILE: 76, 0, 295, 41, 295, 0 umtx pi: 52, 0, 0, 0, 0, 0 PROC: 696, 0, 229, 46, 541535, 0 THREAD: 552, 0, 283, 11, 952, 0 UPCALL: 44, 0, 0, 0, 0, 0 SLEEPQUEUE: 32, 0, 295, 157, 295, 0 VMSPACE: 232, 0, 196, 59, 541501, 0 cpuset: 40, 0, 2, 182, 2, 0 mbuf_packet: 256, 0, 258, 126, 20757971, 0 mbuf: 256, 0, 960618, 33, 116817015, 0 mbuf_cluster: 2048, 25600, 384, 6, 384, 0 mbuf_jumbo_pagesize: 4096, 12800, 0, 117, 19593255, 0 mbuf_jumbo_9k: 9216, 6400, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 3200, 0, 0, 0, 0 mbuf_ext_refcnt: 4, 0, 0, 406, 26807, 0 ACL UMA zone: 388, 0, 0, 20, 65073710, 0 g_bio: 132, 0, 12, 423, 10043465, 0 ata_request: 192, 0, 3, 337, 3212743, 0 ata_composite: 184, 0, 0, 0, 0, 0 cryptop: 60, 0, 0, 0, 0, 0 cryptodesc: 56, 0, 0, 0, 0, 0 VNODE: 276, 0, 19330, 33618, 3235279, 0 VNODEPOLL: 64, 0, 0, 0, 0, 0 NAMEI: 1024, 0, 0, 28, 38128632, 0 S VFS Cache: 68, 0, 20134, 23490, 10825579, 0 L VFS Cache: 291, 0, 27, 207, 175327, 0 NFSMOUNT: 560, 0, 0, 0, 0, 0 NFSNODE: 452, 0, 0, 0, 0, 0 DIRHASH: 1024, 0, 1946, 170, 28110, 0 pipe: 396, 0, 89, 31, 229902, 0 ksiginfo: 80, 0, 248, 40, 248, 0 itimer: 220, 0, 0, 36, 1, 0 KNOTE: 68, 0, 175, 217, 2157098, 0 socket: 416, 25605, 404, 127, 3609347, 0 unpcb: 168, 25622, 278, 136, 1352116, 0 ipq: 32, 904, 0, 226, 42, 0 udpcb: 180, 25608, 13, 75, 1774380, 0 inpcb: 180, 25608, 155, 197, 482839, 0 tcpcb: 464, 25600, 92, 60, 482839, 0 tcptw: 52, 5184, 63, 297, 96034, 0 syncache: 104, 15392, 9, 102, 474336, 0 hostcache: 76, 15400, 626, 124, 4537, 0 tcpreass: 20, 1690, 0, 169, 92838, 0 sackhole: 20, 0, 0, 169, 7789, 0 sctp_ep: 804, 25600, 0, 0, 0, 0 sctp_asoc: 1412, 40000, 0, 0, 0, 0 sctp_laddr: 24, 80040, 0, 145, 28, 0 sctp_raddr: 400, 80000, 0, 0, 0, 0 sctp_chunk: 92, 400008, 0, 0, 0, 0 sctp_readq: 76, 400000, 0, 0, 0, 0 sctp_stream_msg_out: 60, 400050, 0, 0, 0, 0 sctp_asconf: 24, 400055, 0, 0, 0, 0 sctp_asconf_ack: 24, 400055, 0, 0, 0, 0 ripcb: 180, 25608, 1, 43, 2, 0 rtentry: 124, 0, 78, 46, 3549, 0 SWAPMETA: 276, 121576, 158, 108, 19610, 0 Mountpoints: 716, 0, 6, 4, 6, 0 FFS inode: 128, 0, 19299, 19191, 3235170, 0 FFS1 dinode: 128, 0, 0, 0, 0, 0 FFS2 dinode: 256, 0, 19299, 16011, 3235170, 0 pfsrctrpl: 124, 10013, 0, 0, 0, 0 pfrulepl: 828, 0, 12, 8, 12, 0 pfstatepl: 284, 10010, 0, 0, 0, 0 pfaltqpl: 224, 0, 0, 0, 0, 0 pfpooladdrpl: 68, 0, 6, 106, 6, 0 pfrktable: 1240, 1002, 1, 5, 2, 0 pfrkentry: 156, 200000, 3, 47, 3, 0 pfrkentry2: 156, 0, 0, 0, 0, 0 pffrent: 16, 5075, 0, 0, 0, 0 pffrag: 48, 0, 0, 0, 0, 0 pffrcache: 48, 10062, 0, 0, 0, 0 pffrcent: 12, 50141, 0, 0, 0, 0 pfstatescrub: 28, 0, 0, 0, 0, 0 pfiaddrpl: 100, 0, 0, 0, 0, 0 pfospfen: 108, 0, 696, 24, 696, 0 pfosfp: 28, 0, 407, 228, 407, 0