Date: Wed, 30 Sep 2009 01:37:54 -0700 From: Rudy <crapsh@monkeybrains.net> To: freebsd-stable@freebsd.org Subject: Re: em0 watchdog timeouts Message-ID: <4AC318E2.70306@monkeybrains.net> In-Reply-To: <4AB9638B.8040607@monkeybrains.net> References: <4AB9638B.8040607@monkeybrains.net>
next in thread | previous in thread | raw e-mail | index | archive | help
Rudy wrote: > I am having watchdog timeout issues with my Intel 82573 Pro/1000 ... > http://lists.freebsd.org/pipermail/freebsd-net/2008-May/018075.html > > link to dcgdis.zip didn't work. Do you have a copy? > Thanks, Jack. Got the file and flashed -- no upgrade needed. So, while the router was offline, I flashed the motherboards bios (Supermicro X7Sbi), upgraded to 7.2-STABLE, and downloaded the 6.9.12 version of the em driver. Still, watchdog timeouts. Sigh. Will the Intel Gigabit ET Quad Port Adapter make my the timeouts go away??? Should I be using amd64??? Should tx_int_delay=0? Summary: 2 Nics on Motherboard + quad card in PCIe slot. Watchdog timeouts on motherboard nics and on quad card nic when bandwidth > 10Mbps There is minimal (bgp session) TCP to the box... it only forwards packets between interfaces. # uname -r -m 7.2-STABLE i386 # dmesg | grep ^em em0: <Intel(R) PRO/1000 Network Connection 6.9.12> port 0x2000-0x201f mem 0xd0220000-0xd023ffff,0xd0200000-0xd021ffff irq 16 at device 0.0 on pci5 em0: Using MSI interrupt em0: [FILTER] em0: Ethernet address: 00:15:17:78:99:70 em1: <Intel(R) PRO/1000 Network Connection 6.9.12> port 0x2020-0x203f mem 0xd0260000-0xd027ffff,0xd0240000-0xd025ffff irq 17 at device 0.1 on pci5 em1: Using MSI interrupt em1: [FILTER] em1: Ethernet address: 00:15:17:78:99:71 em2: <Intel(R) PRO/1000 Network Connection 6.9.12> port 0x3000-0x301f mem 0xd0320000-0xd033ffff,0xd0300000-0xd031ffff irq 17 at device 0.0 on pci6 em2: Using MSI interrupt em2: [FILTER] em2: Ethernet address: 00:15:17:78:99:72 em3: <Intel(R) PRO/1000 Network Connection 6.9.12> port 0x3020-0x303f mem 0xd0360000-0xd037ffff,0xd0340000-0xd035ffff irq 18 at device 0.1 on pci6 em3: Using MSI interrupt em3: [FILTER] em3: Ethernet address: 00:15:17:78:99:73 em4: <Intel(R) PRO/1000 Network Connection 6.9.12> port 0x4000-0x401f mem 0xd0400000-0xd041ffff irq 16 at device 0.0 on pci13 em4: Using MSI interrupt em4: [FILTER] em4: Ethernet address: 00:30:48:67:14:50 em5: <Intel(R) PRO/1000 Network Connection 6.9.12> port 0x5000-0x501f mem 0xd0500000-0xd051ffff irq 17 at device 0.0 on pci15 em5: Using MSI interrupt em5: [FILTER] em5: Ethernet address: 00:30:48:67:14:51 # vmstat -i interrupt total rate irq1: atkbd0 710 0 irq4: sio0 3 0 irq23: atapci0 14943 0 cpu0: timer 929753417 2000 irq256: em0 702754836 1511 irq257: em1 2 0 irq260: em4 469338728 1009 irq261: em5 78605337 169 cpu1: timer 929753403 2000 Total 3110221379 6690 # sysctl dev.em.0.stats=1 Sep 30 01:08:20 mango kernel: em0: Excessive collisions = 0 Sep 30 01:08:20 mango kernel: em0: Sequence errors = 0 Sep 30 01:08:20 mango kernel: em0: Defer count = 0 Sep 30 01:08:20 mango kernel: em0: Missed Packets = 101469 Sep 30 01:08:20 mango kernel: em0: Receive No Buffers = 0 Sep 30 01:08:20 mango kernel: em0: Receive Length Errors = 0 Sep 30 01:08:20 mango kernel: em0: Receive errors = 0 Sep 30 01:08:20 mango kernel: em0: Crc errors = 0 Sep 30 01:08:20 mango kernel: em0: Alignment errors = 0 Sep 30 01:08:20 mango kernel: em0: Collision/Carrier extension errors = 0 Sep 30 01:08:20 mango kernel: em0: RX overruns = 0 Sep 30 01:08:20 mango kernel: em0: watchdog timeouts = 15 Sep 30 01:08:20 mango kernel: em0: RX MSIX IRQ = 0 TX MSIX IRQ = 0 LINK MSIX IRQ = 0 Sep 30 01:08:20 mango kernel: em0: XON Rcvd = 0 Sep 30 01:08:20 mango kernel: em0: XON Xmtd = 0 Sep 30 01:08:20 mango kernel: em0: XOFF Rcvd = 0 Sep 30 01:08:20 mango kernel: em0: XOFF Xmtd = 0 Sep 30 01:08:20 mango kernel: em0: Good Packets Rcvd = 1056196797 Sep 30 01:08:20 mango kernel: em0: Good Packets Xmtd = 1088726903 Sep 30 01:08:20 mango kernel: em0: TSO Contexts Xmtd = 4088 Sep 30 01:08:20 mango kernel: em0: TSO Contexts Failed = 0 # sysctl dev.em.0.debug=1 Sep 30 01:34:59 mango kernel: em0: Adapter hardware address = 0xc5159420 Sep 30 01:34:59 mango kernel: em0: CTRL = 0x401c0241 RCTL = 0x8002 Sep 30 01:34:59 mango kernel: em0: Packet buffer = Tx=16k Rx=32k Sep 30 01:34:59 mango kernel: em0: Flow control watermarks high = 30720 low = 29220 Sep 30 01:34:59 mango kernel: em0: tx_int_delay = 66, tx_abs_int_delay = 66 Sep 30 01:34:59 mango kernel: em0: rx_int_delay = 0, rx_abs_int_delay = 66 Sep 30 01:34:59 mango kernel: em0: fifo workaround = 0, fifo_reset_count = 0 Sep 30 01:34:59 mango kernel: em0: hw tdh = 980, hw tdt = 980 Sep 30 01:34:59 mango kernel: em0: hw rdh = 203, hw rdt = 202 Sep 30 01:34:59 mango kernel: em0: Num Tx descriptors avail = 1024 Sep 30 01:34:59 mango kernel: em0: Tx Descriptors not avail1 = 0 Sep 30 01:34:59 mango kernel: em0: Tx Descriptors not avail2 = 0 Sep 30 01:34:59 mango kernel: em0: Std mbuf failed = 0 Sep 30 01:34:59 mango kernel: em0: Std mbuf cluster failed = 0 Sep 30 01:34:59 mango kernel: em0: Driver dropped packets = 0 Sep 30 01:34:59 mango kernel: em0: Driver tx dma failure in encap = 0 # grep watchdog /var/log/messages Sep 28 19:52:24 mango kernel: em0: watchdog timeout -- resetting Sep 28 20:09:56 mango kernel: em4: watchdog timeout -- resetting Sep 28 22:43:15 mango kernel: em0: watchdog timeout -- resetting Sep 29 02:20:12 mango kernel: em0: watchdog timeout -- resetting Sep 29 02:35:59 mango kernel: em4: watchdog timeout -- resetting Sep 29 03:46:10 mango kernel: em0: watchdog timeout -- resetting Sep 29 04:16:11 mango kernel: em0: watchdog timeout -- resetting Sep 29 05:30:29 mango kernel: em0: watchdog timeout -- resetting Sep 29 06:29:04 mango kernel: em0: watchdog timeout -- resetting Sep 29 09:50:38 mango kernel: em0: watchdog timeout -- resetting Sep 29 11:23:19 mango kernel: em0: watchdog timeout -- resetting Sep 29 11:28:31 mango kernel: em0: watchdog timeout -- resetting Sep 29 13:11:21 mango kernel: em0: watchdog timeout -- resetting Sep 29 13:26:08 mango kernel: em0: watchdog timeout -- resetting Sep 29 14:54:37 mango kernel: em0: watchdog timeout -- resetting Sep 29 15:28:28 mango kernel: em0: watchdog timeout -- resetting Sep 29 15:34:50 mango kernel: em0: watchdog timeout -- resetting I did reconfigure the network to move 20Mbps off of em0 (which is now doing about 40Mbps) and I haven't had a watchdog for 6 hours. # cat /etc/sysctl.conf net.inet.ip.fastforwarding=1 net.inet.ip.intr_queue_maxlen=92 net.inet.icmp.drop_redirect=1 dev.em.0.rx_processing_limit=200 dev.em.1.rx_processing_limit=200 dev.em.2.rx_processing_limit=200 dev.em.4.rx_processing_limit=200 dev.em.5.rx_processing_limit=200 # cat /boot/loader.conf ### dynamic module to get latest driver if_em_load="YES" ### add more descriptors to em devices. hw.em.rxd=1024 hw.em.txd=1024 Rudy
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4AC318E2.70306>