Date: Tue, 04 Oct 2005 16:27:35 +0200 From: Ferdinand Goldmann <ferdinand.goldmann@jku.at> To: net@freebsd.org Subject: Re: dummynet, em driver, device polling issues :-(( Message-ID: <43429157.90606@jku.at> In-Reply-To: <9CD8C672-1EF2-42FE-A61E-83DC684C893D@dragondata.com> References: <4341089F.7010504@jku.at> <20051003104548.GB70355@cell.sick.ru> <4341242F.9060602@jku.at> <20051003123210.GF70355@cell.sick.ru> <43426EF3.3020404@jku.at> <9CD8C672-1EF2-42FE-A61E-83DC684C893D@dragondata.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Kevin Day wrote:
> This is pretty odd. We've got dozens of servers using various versions
> of 5.x, and many different em cards, and have no problem, even when
> shoving near line rate speeds out of them.
Maximum transfer rates we see in MRTG were around 320Mbit/s (with polling disabled)
> em0: <Intel(R) PRO/1000 Network Connection, Version - 1.7.35> port
> 0xecc0-0xecff mem 0xdfae0000-0xdfafffff irq 64 at device 7.0 on pci6
em0: <Intel(R) PRO/1000 Network Connection, Version - 1.7.35> port
0x2280-0x22bf mem 0xeffc0000-0xeffdffff
irq 20 at device 5.0 on pci1
Pretty much the same here, even the driver version.
em0@pci1:5:0: class=0x020000 card=0x10028086 chip=0x10268086 rev=0x04 hdr=0x00
vendor = 'Intel Corporation'
device = '82545GM Gigabit Ethernet Controller'
> After you experience your problems, can you do "sysctl -w
> hw.em0.stats=1" and "sysctl -w hw.em0.debug_info=1" and post what gets
> dumped to your syslog/dmesg output?
em0: Excessive collisions = 0
em0: Symbol errors = 0
em0: Sequence errors = 0
em0: Defer count = 11
em0: Missed Packets = 0
em0: Receive No Buffers = 0
em0: Receive length errors = 0
em0: Receive errors = 0
em0: Crc errors = 0
em0: Alignment errors = 0
em0: Carrier extension errors = 0
em0: XON Rcvd = 11
em0: XON Xmtd = 0
em0: XOFF Rcvd = 11
em0: XOFF Xmtd = 0
em0: Good Packets Rcvd = 283923273
em0: Good Packets Xmtd = 272613648
em0: Adapter hardware address = 0xc12cfb48
em0:CTRL = 0x58f00249
em0:RCTL = 0x8002 PS=(0x8402)
em0:tx_int_delay = 66, tx_abs_int_delay = 66
em0:rx_int_delay = 0, rx_abs_int_delay = 66
em0: fifo workaround = 0, fifo_reset = 0
em0: hw tdh = 173, hw tdt = 173
em0: Num Tx descriptors avail = 256
em0: Tx Descriptors not avail1 = 0
em0: Tx Descriptors not avail2 = 0
em0: Std mbuf failed = 0
em0: Std mbuf cluster failed = 0
em0: Driver dropped packets = 0
> We're using polling on nearly all the servers, and don't see ierrs at
> all.
Hm. That's strange. The above values were gathered with polling disabled. As
soon as I enable polling, ierrs on the em0 interface are rising:
em0: Excessive collisions = 0
em0: Symbol errors = 0
em0: Sequence errors = 0
em0: Defer count = 11
em0: Missed Packets = 39
em0: Receive No Buffers = 2458
em0: Receive length errors = 0
em0: Receive errors = 0
em0: Crc errors = 0
em0: Alignment errors = 0
em0: Carrier extension errors = 0
em0: XON Rcvd = 11
em0: XON Xmtd = 4
em0: XOFF Rcvd = 11
em0: XOFF Xmtd = 43
em0: Good Packets Rcvd = 315880003
em0: Good Packets Xmtd = 303985941
em0: Adapter hardware address = 0xc12cfb48
em0:CTRL = 0x58f00249
em0:RCTL = 0x8002 PS=(0x8402)
em0:tx_int_delay = 66, tx_abs_int_delay = 66
em0:rx_int_delay = 0, rx_abs_int_delay = 66
em0: fifo workaround = 0, fifo_reset = 0
em0: hw tdh = 57, hw tdt = 57
em0: Num Tx descriptors avail = 249
em0: Tx Descriptors not avail1 = 0
em0: Tx Descriptors not avail2 = 0
em0: Std mbuf failed = 0
em0: Std mbuf cluster failed = 0
em0: Driver dropped packets = 0
Can you tell me what settings you are using for polling? I have set it to
HZ=1000 and burst_max=300.
I have now noticed another thing which might indicate one of the possible
causes for the problem - this box until now ran FreeBSD 4.x and did not support
ipfw tables to lock out whole lists of IP adresses. So there were quite a few
inefficient rules for this. I now put all the locked IP addresses in a table
which is referenced by only one rule. Since I did this, the ierrs seem to rise
slower with polling enabled.
> Have you tried contacting Intel directly about this?
> freebsdnic@mailbox.intel.com has been pretty helpful with em specific
> problems in the past.
Not yet, thank you for the hint.
--
>> Ferdinand Goldmann //// | |
>> EMail: Ferdinand.Goldmann@zid.uni-linz.ac.at |--00 | UNIX |
>> Tel. : +43/732/2468/9398 Fax. : +43/732/2468/9397 C ^ | |
>> EMail: Ferdinand.Goldmann@zid.uni-linz.ac.at \ ~/ ~~~|~~~~~~~~
>> PGP D4CF 8AA4 4B2A 7B88 65CA 5EDC 0A9B FA9A 13EA B993| |-----3
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?43429157.90606>