Date: Thu, 29 Mar 2007 23:37:00 -0300 From: "Marc G. Fournier" <freebsd@hub.org> To: freebsd-stable@freebsd.org Subject: More on: No buffer space available Message-ID: <12F4628FC7F68F8CBC37B104@ganymede.hub.org>
next in thread | raw e-mail | index | archive | help
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 'k, here's a bit more data, see if something jumps out at anyone ... first, a bit of a timeline ... Tonight, after ~75hrs uptime, I got an SMS, which is what generally indicates the problem has begun ... the SMS is a result of a cron job running on the machine that pings another server on the same network ... if it can't ping it, it sends me an SMS (ie. the other server is down) ... the neat thing is that I'm also sent one when it comes back up again ... normally, I get the 'down' before I get the 'up' ... when this problem starts to manifest itself, I get the 'up' message, followed a bit later by the 'down' ... the reason: since the network is down (it can't ping), it also can't send out the 'down' message, so it queues it up ... as soon as it comes back up again, mail works again, so the 'up' goes out before queue is re-processed ... That said ... When I got my first SMS, I logged in and ran the two commands Robert suggested: 142/968/1110 mbufs in use (current/cache/total) 136/552/688/25600 mbuf clusters in use (current/cache/total/max) 136/388 mbuf+clusters out of packet secondary zone in use (current/cache) 0/0/0/0 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/0 9k jumbo clusters in use (current/cache/total/max) 0/0/0/0 16k jumbo clusters in use (current/cache/total/max) 307K/1346K/1653K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/38/6656 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 396 requests for I/O initiated by sendfile 713 calls to protocol drain routines Now, just over 4 hours later (I'm just about to reboot): 421/914/1335 mbufs in use (current/cache/total) 233/617/850/25600 mbuf clusters in use (current/cache/total/max) 233/361 mbuf+clusters out of packet secondary zone in use (current/cache) 0/0/0/0 4k (page size) jumbo clusters in use (current/cache/total/max) 0/0/0/0 9k jumbo clusters in use (current/cache/total/max) 0/0/0/0 16k jumbo clusters in use (current/cache/total/max) 571K/1462K/2033K bytes allocated to network (current/cache/total) 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) 0/0/0 requests for jumbo clusters denied (4k/9k/16k) 0/38/6656 sfbufs in use (current/peak/max) 0 requests for sfbufs denied 0 requests for sfbufs delayed 406 requests for I/O initiated by sendfile 733 calls to protocol drain routines Now, the vmstat -z is a wee bit longer: ITEM SIZE LIMIT USED FREE REQUESTS FAILURES UMA Kegs: 140, 0, 64, 8, 64, 0 UMA Zones: 480, 0, 64, 0, 64, 0 UMA Slabs: 64, 0, 3069, 1238, 128266, 0 UMA RCntSlabs: 104, 0, 344, 100, 14486, 0 UMA Hash: 128, 0, 4, 26, 7, 0 16 Bucket: 76, 0, 15, 35, 70, 0 32 Bucket: 140, 0, 22, 34, 87, 0 64 Bucket: 268, 0, 25, 59, 220, 9 128 Bucket: 524, 0, 2975, 147, 80119, 1572874 VM OBJECT: 132, 0, 74837, 29795, 94095430, 0 MAP: 192, 0, 7, 13, 7, 0 KMAP ENTRY: 68, 57344, 747, 877, 623962, 0 MAP ENTRY: 68, 0, 89827, 13941, 192935836, 0 PV ENTRY: 24, 2245615, 1255065, 247860, 3181072639, 0 DP fakepg: 72, 0, 0, 0, 0, 0 mt_zone: 1024, 0, 134, 126, 134, 0 16: 16, 0, 8483, 449, 19117061, 0 32: 32, 0, 4673, 3915, 8407030, 0 64: 64, 0, 8889, 3029, 24622501, 0 128: 128, 0, 19660, 3620, 88038870, 0 256: 256, 0, 3891, 2319, 9037915, 0 512: 512, 0, 847, 993, 1874043, 0 1024: 1024, 0, 631, 357, 2814842, 0 2048: 2048, 0, 293, 381, 812570, 0 4096: 4096, 0, 1130, 439, 4342201, 0 Files: 72, 0, 11119, 3191, 72134326, 0 PROC: 536, 0, 1073, 971, 2510920, 0 THREAD: 376, 0, 2188, 372, 352225, 0 KSEGRP: 88, 0, 2137, 383, 32084, 0 UPCALL: 44, 0, 119, 349, 30104, 0 VMSPACE: 296, 0, 1034, 812, 2510845, 0 mbuf_packet: 256, 0, 142, 382, 33140922, 0 mbuf: 256, 0, 44, 542, 65939839, 0 mbuf_cluster: 2048, 25600, 524, 164, 345305, 0 mbuf_jumbo_pagesize: 4096, 0, 0, 0, 0, 0 mbuf_jumbo_9k: 9216, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 0, 0, 0, 0, 0 ACL UMA zone: 388, 0, 0, 0, 0, 0 g_bio: 132, 0, 0, 4205, 87153652, 0 VNODE: 272, 0, 71264, 22158, 1241560352, 0 VNODEPOLL: 76, 0, 0, 100, 3, 0 S VFS Cache: 68, 0, 73121, 29135, 1248334482, 0 L VFS Cache: 291, 0, 124, 1085, 682683, 0 NAMEI: 1024, 0, 0, 304, 1434961352, 0 DIRHASH: 1024, 0, 1810, 258, 18000204, 0 PIPE: 408, 0, 1981, 602, 1091976, 0 KNOTE: 68, 0, 32, 360, 3972127, 0 socket: 356, 12331, 12271, 60, 8439626, 1141 unpcb: 144, 12339, 11561, 373, 5337418, 0 ipq: 32, 904, 0, 226, 2, 0 udpcb: 180, 12342, 74, 146, 2173707, 0 inpcb: 180, 12342, 678, 1478, 927361, 0 tcpcb: 464, 12328, 619, 717, 927361, 0 tcptw: 48, 2496, 59, 1501, 256613, 0 syncache: 100, 15366, 0, 195, 676224, 0 hostcache: 76, 15400, 512, 688, 34850, 0 tcpreass: 20, 1690, 0, 507, 53830, 0 sackhole: 20, 0, 0, 507, 20912, 0 ripcb: 180, 12342, 0, 88, 1127, 0 rtentry: 132, 0, 203, 319, 6656, 0 g_stripe_zone: 131072, 100, 0, 0, 0, 0 SWAPMETA: 276, 121576, 957, 429, 17641, 0 Mountpoints: 664, 0, 197, 19, 200, 0 FFS inode: 132, 0, 70901, 17491, 1239034732, 0 FFS1 dinode: 128, 0, 0, 0, 0, 0 FFS2 dinode: 256, 0, 70901, 7924, 1239034732, 0 If the '4 hour later' version is of any use, please ask, I did save a copy before rebooting ... Does this provide anything? Is there something else I should do/try? Thanks ... - ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email . scrappy@hub.org MSN . scrappy@hub.org Yahoo . yscrappy Skype: hub.org ICQ . 7615664 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (FreeBSD) iD8DBQFGDHfM4QvfyHIvDvMRAhZlAJ4sR9Xe3fuC5egjtt9o9dX8Ek+opACcCu3H euSZyKGB9/HVcuwilQicfMM= =bQo7 -----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?12F4628FC7F68F8CBC37B104>