From owner-freebsd-bugs@FreeBSD.ORG Thu May 24 20:10:31 2012 Return-Path: Delivered-To: freebsd-bugs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E8C001065677 for ; Thu, 24 May 2012 20:10:31 +0000 (UTC) (envelope-from kes-kes@yandex.ru) Received: from forward17.mail.yandex.net (forward17.mail.yandex.net [IPv6:2a02:6b8:0:1402::2]) by mx1.freebsd.org (Postfix) with ESMTP id D73208FC21 for ; Thu, 24 May 2012 20:10:30 +0000 (UTC) Received: from smtp18.mail.yandex.net (smtp18.mail.yandex.net [95.108.252.18]) by forward17.mail.yandex.net (Yandex) with ESMTP id E5DA21060F3A for ; Fri, 25 May 2012 00:10:20 +0400 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1337890220; bh=Uns36Y2S32eFsTOQUMajwlWXrFtWpCrocy2imtYvP3I=; h=Date:From:Reply-To:Message-ID:To:Subject:MIME-Version: Content-Type:Content-Transfer-Encoding; b=im30IsOzcQ128sG41/PGw9PIVUaQMhSmAFGgm1+UuZkbsURMy3RWxZ4zGeDRzuWcD g/L5ZMkC5wsO6uBBmOwWuh0iuvPqtUw8qHmNKtZ0fwA2iqbGUkdJ8dkEarf+nGOS8/ 2ngL7tWuBaP2bpTsdoREgVGbe/kImCGco4XDnVc8= Received: from smtp18.mail.yandex.net (localhost [127.0.0.1]) by smtp18.mail.yandex.net (Yandex) with ESMTP id A3C7C18A00F2 for ; Fri, 25 May 2012 00:10:20 +0400 (MSK) Received: from unknown (unknown [77.93.52.20]) by smtp18.mail.yandex.net (nwsmtp/Yandex) with ESMTP id AH34XMnD-AK3e9NnL; Fri, 25 May 2012 00:10:20 +0400 X-Yandex-Rcpt-Suid: freebsd-bugs@FreeBSD.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1337890220; bh=Uns36Y2S32eFsTOQUMajwlWXrFtWpCrocy2imtYvP3I=; h=Date:From:X-Mailer:Reply-To:Organization:X-Priority:Message-ID:To: Subject:MIME-Version:Content-Type:Content-Transfer-Encoding; b=Wq+f7k6RY8s9SK5WYIkQDveCzJNd1KwF/N8Intz2MhNNOohHez6nFubLmxbACwU86 GaDB8d08paMM1x76CmJfHYyUx8iBWb6HWSObLbJ5WG4YAnxdowsAa7maGc+myfr572 UqRpVIkUKJy783KxP9H2zuwp+K6WUvL6HEOKuN9s= Date: Thu, 24 May 2012 23:10:18 +0300 From: Eugen Konkov X-Mailer: The Bat! (v4.0.24) Professional Organization: ISP FreeLine X-Priority: 3 (Normal) Message-ID: <1012364286.20120524231018@yandex.ru> To: freebsd-bugs@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=windows-1251 Content-Transfer-Encoding: 8bit Cc: Subject: re0 eat all CPU, but only 300-1000 Interrupts occured X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Eugen Konkov List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 May 2012 20:10:32 -0000 this server is about 1year running, 50days uptime but this occur only once and gone away after one or two hours. last pid: 74164; load averages: 17.97, 13.81, 14.40 up 47+09:31:54 21:50:58 110 processes: 9 running, 87 sleeping, 14 waiting CPU: 0.5% user, 0.0% nice, 1.6% system, 97.9% interrupt, 0.0% idle Mem: 194M Active, 449M Inact, 204M Wired, 20K Cache, 112M Buf, 1143M Free Swap: 512M Total, 512M Free PID USERNAME PRI NICE SIZE RES STATE TIME WCPU COMMAND 12 root -76 - 0K 112K WAIT 65.3H 100.00% {irq256: re0} 65795 root 39 0 13420K 5804K select 19:58 0.63% snmpd 2664 root 36 0 14608K 6812K RUN 75:49 0.54% bgpd 0 root -76 0 0K 80K - 31.6H 0.44% {dummynet} 7758 bind 36 0 75268K 62880K RUN 178:21 0.24% {named} 12 root -44 - 0K 112K WAIT 29.4H 0.10% {swi4: clock} # systat -v 2 users Load 13.61 12.24 13.96 May 24 21:49 Mem:KB REAL VIRTUAL VN PAGER SWAP PAGER Tot Share Tot Share Free in out in out Act 256100 6368 626872 8116 1171172 count All 353960 9500 2792812 26356 pages Proc: Interrupts r p d s w Csw Trp Sys Int Sof Flt 3 cow 4404 total 12 32 2207 106 4216 4404 3983 48 28 zfod 1 ata0 14 ozfod 4119 hpet0 20 4.1%Sys 93.4%Intr 2.6%User 0.0%Nice 0.0%Idle %ozfod uhci0 ehci | | | | | | | | | | | daefr 284 re0 256 ==+++++++++++++++++++++++++++++++++++++++++++++++> 7 prcfr 40 dtbuf 44 totfr Namei Name-cache Dir-cache 111725 desvn react Calls hits % hits % 64097 numvn pdwak 3177 3177 100 27926 frevn pdpgs intrn Disks ad1 da0 pass0 209000 wire KB/t 14.85 0.00 0.00 198616 act tps 1 0 0 459200 inact MB/s 0.01 0.00 0.00 20 cache %busy 2 0 0 1169956 free # vmstat -i interrupt total rate irq14: ata0 5946124 1 irq20: hpet0 3979660990 971 irq23: uhci0 ehci0 85 0 irq256: re0 3847013663 939 Total 7832620862 1912 # uptime 9:44PM up 47 days, 9:25, 2 users, load averages: 9.05, 14.17, 15.61 #bwm-ng -I re0 bwm-ng v0.6 (probing every 0.500s), press 'h' for help input: getifaddrs type: rate - iface Rx Tx Total ============================================================================== re0: 13.32 Mb/s 12.42 Mb/s 25.74 Mb/s ------------------------------------------------------------------------------ total: 14.95 Mb/s 13.94 Mb/s 28.89 Mb/s # vmstat -z ITEM SIZE LIMIT USED FREE REQ FAIL SLEEP UMA Kegs: 128, 0, 92, 28, 92, 0, 0 UMA Zones: 224, 0, 92, 10, 92, 0, 0 UMA Slabs: 284, 0, 1158, 4, 29780, 0, 0 UMA RCntSlabs: 544, 0, 919, 5, 919, 0, 0 UMA Hash: 128, 0, 2, 28, 3, 0, 0 16 Bucket: 76, 0, 36, 14, 36, 0, 0 32 Bucket: 140, 0, 43, 13, 44, 0, 0 64 Bucket: 268, 0, 59, 11, 67, 90, 0 128 Bucket: 524, 0, 895, 1, 27120, 585, 0 VM OBJECT: 136, 0, 37976, 130,40520105, 0, 0 MAP: 140, 0, 7, 49, 7, 0, 0 KMAP ENTRY: 72, 57505, 32, 127, 79899, 0, 0 MAP ENTRY: 72, 0, 2094, 1192,100480394, 0, 0 DP fakepg: 72, 0, 0, 0, 0, 0, 0 SG fakepg: 72, 0, 0, 0, 0, 0, 0 mt_zone: 2060, 0, 268, 11, 268, 0, 0 16: 16, 0, 5106, 375,251057215, 0, 0 32: 32, 0, 26924, 196,1931894354, 0, 0 64: 64, 0, 6372, 413,172309547, 0, 0 128: 128, 0, 23931, 41019,195195229, 0, 0 256: 256, 0, 2786, 289,2302309652, 0, 0 512: 512, 0, 638, 58, 4189195, 0, 0 1024: 1024, 0, 533, 15, 1586826, 0, 0 2048: 2048, 0, 251, 59, 17903, 0, 0 4096: 4096, 0, 154, 29, 2738963, 0, 0 Files: 56, 0, 180, 289,224690270, 0, 0 TURNSTILE: 72, 0, 156, 24, 156, 0, 0 umtx pi: 52, 0, 0, 0, 0, 0, 0 MAC labels: 20, 0, 0, 0, 0, 0, 0 PROC: 688, 0, 57, 23, 1272718, 0, 0 THREAD: 728, 0, 124, 31, 206859, 0, 0 SLEEPQUEUE: 44, 0, 156, 80, 156, 0, 0 VMSPACE: 232, 0, 38, 30, 1272743, 0, 0 cpuset: 40, 0, 2, 182, 2, 0, 0 audit_record: 816, 0, 0, 0, 0, 0, 0 mbuf_packet: 256, 0, 256, 1536,5400971809, 0, 0 mbuf: 256, 0, 6, 1412,9019029846, 0, 0 mbuf_cluster: 2048, 25600, 1792, 6, 1792, 0, 0 mbuf_jumbo_page: 4096, 12800, 0, 20, 828696, 0, 0 mbuf_jumbo_9k: 9216, 6400, 0, 0, 0, 0, 0 mbuf_jumbo_16k: 16384, 3200, 0, 0, 0, 0, 0 mbuf_ext_refcnt: 4, 0, 0, 0, 0, 0, 0 NetGraph items: 36, 4130, 2, 175,1535407748, 0, 0 NetGraph data items: 36, 531, 0, 531,3322859915,26160, 0 g_bio: 140, 0, 0, 4144,23787066, 0, 0 ttyinq: 152, 0, 300, 12, 675, 0, 0 ttyoutq: 256, 0, 157, 8, 353, 0, 0 ata_request: 204, 0, 0, 1045, 5946685, 0, 0 ata_composite: 180, 0, 0, 0, 0, 0, 0 VNODE: 272, 0, 64112, 134, 7873012, 0, 0 VNODEPOLL: 60, 0, 0, 0, 0, 0, 0 S VFS Cache: 72, 0, 66367, 466, 8051562, 0, 0 L VFS Cache: 292, 0, 595, 198, 109305, 0, 0 NAMEI: 1024, 0, 0, 12,171046456, 0, 0 NFSMOUNT: 532, 0, 0, 0, 0, 0, 0 NFSNODE: 460, 0, 0, 0, 0, 0, 0 DIRHASH: 1024, 0, 485, 11, 497, 0, 0 pipe: 392, 0, 10, 30, 784482, 0, 0 Mountpoints: 648, 0, 6, 6, 6, 0, 0 ksiginfo: 80, 0, 66, 990, 210783, 0, 0 itimer: 220, 0, 0, 0, 0, 0, 0 KNOTE: 72, 0, 8, 310,92381338, 0, 0 socket: 416, 25605, 73, 242,131092823, 0, 0 unpcb: 172, 25622, 26, 43, 3331350, 0, 0 ipq: 32, 904, 0, 226, 8756, 0, 0 udp_inpcb: 220, 25614, 7, 245,127020097, 0, 0 udpcb: 8, 25781, 7, 399,127020097, 0, 0 tcp_inpcb: 220, 25614, 33, 57, 644727, 0, 0 tcpcb: 692, 25600, 26, 39, 644727, 0, 0 tcptw: 52, 5184, 7, 137, 92058, 0, 0 syncache: 112, 15365, 0, 70, 360404, 0, 0 hostcache: 76, 15400, 30, 70, 3515, 0, 0 tcpreass: 20, 1690, 0, 169, 1927, 0, 0 sackhole: 20, 0, 0, 169, 620, 0, 0 sctp_ep: 880, 25600, 0, 0, 0, 0, 0 sctp_asoc: 1524, 40000, 0, 0, 0, 0, 0 sctp_laddr: 24, 80040, 0, 145, 45, 0, 0 sctp_raddr: 464, 80000, 0, 0, 0, 0, 0 sctp_chunk: 96, 400000, 0, 0, 0, 0, 0 sctp_readq: 76, 400000, 0, 0, 0, 0, 0 sctp_stream_msg_out: 72, 400044, 0, 0, 0, 0, 0 sctp_asconf: 24, 400055, 0, 0, 0, 0, 0 sctp_asconf_ack: 24, 400055, 0, 0, 0, 0, 0 ripcb: 220, 25614, 6, 48, 78019, 0, 0 rtentry: 108, 0, 24066, 18, 288480, 0, 0 IPFW dynamic rule: 108, 0, 0, 0, 0, 0, 0 divcb: 220, 25614, 0, 0, 0, 0, 0 selfd: 28, 0, 158, 223,1786944834, 0, 0 ip4flow: 40, 25668, 680, 24160,36376221, 0, 0 ip6flow: 64, 25636, 0, 174, 20, 0, 0 SWAPMETA: 276, 121576, 0, 0, 0, 0, 0 FFS inode: 116, 0, 64073, 46, 7872685, 0, 0 FFS1 dinode: 128, 0, 0, 0, 0, 0, 0 FFS2 dinode: 256, 0, 64073, 22, 7872685, 0, 0 # uname -a FreeBSD bor 9.0-CURRENT FreeBSD 9.0-CURRENT #1: Wed Feb 23 09:39:22 UTC 2011 @:/usr/obj/usr/src/sys/KES_KERN_v9 i386 #tail /var/log/messages May 24 21:58:57 bor kernel: Limiting icmp unreach response from 273 to 200 packets/sec May 24 21:58:58 bor kernel: Limiting icmp unreach response from 223 to 200 packets/sec May 24 21:58:59 bor kernel: Limiting icmp unreach response from 235 to 200 packets/sec May 24 21:59:00 bor kernel: Limiting icmp unreach response from 278 to 200 packets/sec May 24 21:59:01 bor kernel: Limiting icmp unreach response from 283 to 200 packets/sec May 24 21:59:02 bor kernel: Limiting icmp unreach response from 255 to 200 packets/sec May 24 21:59:03 bor kernel: Limiting icmp unreach response from 269 to 200 packets/sec May 24 21:59:04 bor kernel: Limiting icmp unreach response from 286 to 200 packets/sec May 24 21:59:05 bor kernel: Limiting icmp unreach response from 225 to 200 packets/sec May 24 21:59:06 bor kernel: Limiting icmp unreach response from 328 to 200 packets/sec May 24 21:59:07 bor kernel: Limiting icmp unreach response from 285 to 200 packets/sec May 24 21:59:08 bor kernel: Limiting icmp unreach response from 286 to 200 packets/sec May 24 21:59:09 bor kernel: Limiting icmp unreach response from 304 to 200 packets/sec May 24 21:59:10 bor kernel: Limiting icmp unreach response from 304 to 200 packets/sec May 24 21:59:11 bor kernel: Limiting icmp unreach response from 311 to 200 packets/sec Can any tell me what that was and how I can fight that at next time it happen? Thank you.