Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 8 Sep 2014 22:05:42 +0200
From:      "=?iso-8859-2?Q?Ing._Ale=B9_Nechv=E1tal?=" <alesnechvatal@videon-znojmo.cz>
To:        "'Marcelo Gondim'" <gondim@bsdinfo.com.br>, "'Eric Joyner'" <ricera10@gmail.com>
Cc:        'Jack F Vogel' <jfv@freebsd.org>, 'FreeBSD Net' <freebsd-net@freebsd.org>, 'Adrian Chadd' <adrian@freebsd.org>
Subject:   RE: ixgbe CRITICAL: ECC ERROR!! Please Reboot!!
Message-ID:  <002001cfcba0$4812c7f0$d83857d0$@videon-znojmo.cz>
In-Reply-To: <540DFAE3.8070001@bsdinfo.com.br>
References:  <5408F23C.2030309@bsdinfo.com.br> <CAJ-VmomOECsnRt98ULijSkj9JmwnaLEnZ1ny7fqazsM3WJDNvA@mail.gmail.com> <54091607.9010100@bsdinfo.com.br> <5409CA44.8070203@bsdinfo.com.br> <CAJ-VmomnzTfHzw2i0C01kmfg1rKLePcTHhZqbPWsAezg%2Be9q0g@mail.gmail.com> <540A1A3E.2040306@bsdinfo.com.br> <540A8200.5010404@bsdinfo.com.br> <CA%2Bb0zg9%2Bhhz6sCzrTAHjhc5VzTbDWdnQ%2BBcfdMPsyTGsPeo3gQ@mail.gmail.com> <540DFAE3.8070001@bsdinfo.com.br>

next in thread | previous in thread | raw e-mail | index | archive | help
I was experiencing similar behaviour on a machine acting as a gateway. It
has one X520-SR2, both SPF+ ports used to connet to another machines with
the same NICs. One of the NICs suddenly stopped to pass data through and the
the ifconfing down up cycle was needed to make it work again, that in better
case. In the worse case, the machine rebooted without leaving any message in
syslog (it's on a remote site). This happened in intervals of 5 mins. up to
1 hour (I am not sure about the exact correlation between traffic going
through and the number of these events, but during traffic peaks, it seems
to have happened more often).

First I checked the mbufs, but didn't see any problem. Then I tried to
disable TSO (on both sides of the links), but without observing any
improvement.
I was playing with different versions of the kernel, the newer the kernel (I
went up to 9.3 STABLE, didn't try 10 or HEAD) the more instability. It
seemed, that the most stable was 9.1 PRERELEASE with 2.4.8 driver (at least
I didn't see reboots, a cron script could keep the gateway passing data by
shutting down and bringing up the interfaces).  Now I am running 9.3 RELEASE
r271227M with the 2.5.25 driver from Intel's website. This configuration was
not stable either, until I discovered high interrupt rate on each queue (the
sum on all queues reached over 400k in some moments). I turned off the
limit of interrupts by setting hw.intr_storm_threshold=0 (previously set to
150000). Since the following reboot (involuntary, btw.) the gateway keeps
running smoothly for 7 hours now with peak throughput of approx. 1.6 Gbit/s.

The sum of interrupt rates on all your queues looks quite high too, so there
could be some similarity and disabling the mentioned limit could help.

My sysctl.conf:
kern.ipc.somaxconn=1024
net.inet.icmp.icmplim=100
net.inet.tcp.blackhole=2
net.inet.udp.blackhole=1
net.inet.tcp.sendspace=262144
net.inet.tcp.recvspace=262144
net.inet.ip.fastforwarding=1
kern.ipc.maxsockbuf=16777216
net.inet.tcp.sendbuf_max=16777216
net.inet.tcp.recvbuf_max=16777216
hw.intr_storm_threshold=0

#sysctl -a dev.ix.1

dev.ix.1.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version -
2.5.25
dev.ix.1.%driver: ix
dev.ix.1.%location: slot=0 function=1
dev.ix.1.%pnpinfo: vendor=0x8086 device=0x10fb subvendor=0x8086
subdevice=0x7a11 class=0x020000
dev.ix.1.%parent: pci1
dev.ix.1.fc: 3
dev.ix.1.enable_aim: 1
dev.ix.1.advertise_speed: 0
dev.ix.1.dropped: 0
dev.ix.1.mbuf_defrag_failed: 0
dev.ix.1.watchdog_events: 0
dev.ix.1.link_irq: 2
dev.ix.1.queue0.interrupt_rate: 31250
dev.ix.1.queue0.irqs: 152182132
dev.ix.1.queue0.txd_head: 526
dev.ix.1.queue0.txd_tail: 526
dev.ix.1.queue0.tso_tx: 2
dev.ix.1.queue0.no_tx_dma_setup: 0
dev.ix.1.queue0.no_desc_avail: 119373
dev.ix.1.queue0.tx_packets: 274418209
dev.ix.1.queue0.rxd_head: 441
dev.ix.1.queue0.rxd_tail: 440
dev.ix.1.queue0.rx_packets: 103852222
dev.ix.1.queue0.rx_bytes: 25483398737
dev.ix.1.queue0.rx_copies: 87237192
dev.ix.1.queue0.lro_queued: 0
dev.ix.1.queue0.lro_flushed: 0
dev.ix.1.queue1.interrupt_rate: 31250
dev.ix.1.queue1.irqs: 152973172
dev.ix.1.queue1.txd_head: 1282
dev.ix.1.queue1.txd_tail: 1282
dev.ix.1.queue1.tso_tx: 50
dev.ix.1.queue1.no_tx_dma_setup: 0
dev.ix.1.queue1.no_desc_avail: 114840
dev.ix.1.queue1.tx_packets: 271827311
dev.ix.1.queue1.rxd_head: 495
dev.ix.1.queue1.rxd_tail: 494
dev.ix.1.queue1.rx_packets: 103292003
dev.ix.1.queue1.rx_bytes: 20681070454
dev.ix.1.queue1.rx_copies: 89490795
dev.ix.1.queue1.lro_queued: 0
dev.ix.1.queue1.lro_flushed: 0
dev.ix.1.queue2.interrupt_rate: 21739
dev.ix.1.queue2.irqs: 81502157
dev.ix.1.queue2.txd_head: 313
dev.ix.1.queue2.txd_tail: 313
dev.ix.1.queue2.tso_tx: 50
dev.ix.1.queue2.no_tx_dma_setup: 0
dev.ix.1.queue2.no_desc_avail: 111923
dev.ix.1.queue2.tx_packets: 23189156
dev.ix.1.queue2.rxd_head: 349
dev.ix.1.queue2.rxd_tail: 348
dev.ix.1.queue2.rx_packets: 94457797
dev.ix.1.queue2.rx_bytes: 18053240855
dev.ix.1.queue2.rx_copies: 82430878
dev.ix.1.queue2.lro_queued: 0
dev.ix.1.queue2.lro_flushed: 0
dev.ix.1.queue3.interrupt_rate: 83333
dev.ix.1.queue3.irqs: 81678068
dev.ix.1.queue3.txd_head: 1990
dev.ix.1.queue3.txd_tail: 1990
dev.ix.1.queue3.tso_tx: 3
dev.ix.1.queue3.no_tx_dma_setup: 0
dev.ix.1.queue3.no_desc_avail: 110545
dev.ix.1.queue3.tx_packets: 21258249
dev.ix.1.queue3.rxd_head: 1544
dev.ix.1.queue3.rxd_tail: 1543
dev.ix.1.queue3.rx_packets: 99140619
dev.ix.1.queue3.rx_bytes: 19476001717
dev.ix.1.queue3.rx_copies: 86228489
dev.ix.1.queue3.lro_queued: 0
dev.ix.1.queue3.lro_flushed: 0
dev.ix.1.mac_stats.crc_errs: 0
dev.ix.1.mac_stats.ill_errs: 0
dev.ix.1.mac_stats.byte_errs: 0
dev.ix.1.mac_stats.short_discards: 0
dev.ix.1.mac_stats.local_faults: 0
dev.ix.1.mac_stats.remote_faults: 2
dev.ix.1.mac_stats.rec_len_errs: 0
dev.ix.1.mac_stats.xon_txd: 0
dev.ix.1.mac_stats.xon_recvd: 0
dev.ix.1.mac_stats.xoff_txd: 0
dev.ix.1.mac_stats.xoff_recvd: 0
dev.ix.1.mac_stats.total_octets_rcvd: 85345731961
dev.ix.1.mac_stats.good_octets_rcvd: 85345731961
dev.ix.1.mac_stats.total_pkts_rcvd: 400734734
dev.ix.1.mac_stats.good_pkts_rcvd: 400734734
dev.ix.1.mac_stats.mcast_pkts_rcvd: 0
dev.ix.1.mac_stats.bcast_pkts_rcvd: 0
dev.ix.1.mac_stats.rx_frames_64: 190985269
dev.ix.1.mac_stats.rx_frames_65_127: 142879679
dev.ix.1.mac_stats.rx_frames_128_255: 18114498
dev.ix.1.mac_stats.rx_frames_256_511: 5882972
dev.ix.1.mac_stats.rx_frames_512_1023: 7407080
dev.ix.1.mac_stats.rx_frames_1024_1522: 35465236
dev.ix.1.mac_stats.recv_undersized: 0
dev.ix.1.mac_stats.recv_fragmented: 0
dev.ix.1.mac_stats.recv_oversized: 0
dev.ix.1.mac_stats.recv_jabberd: 0
dev.ix.1.mac_stats.management_pkts_rcvd: 0
dev.ix.1.mac_stats.management_pkts_drpd: 0
dev.ix.1.mac_stats.checksum_errs: 462621
dev.ix.1.mac_stats.good_octets_txd: 705061464268
dev.ix.1.mac_stats.total_pkts_txd: 590658400
dev.ix.1.mac_stats.good_pkts_txd: 590658400
dev.ix.1.mac_stats.bcast_pkts_txd: 2907
dev.ix.1.mac_stats.mcast_pkts_txd: 0
dev.ix.1.mac_stats.management_pkts_txd: 0
dev.ix.1.mac_stats.tx_frames_64: 24533923
dev.ix.1.mac_stats.tx_frames_65_127: 65221733
dev.ix.1.mac_stats.tx_frames_128_255: 24274012
dev.ix.1.mac_stats.tx_frames_256_511: 12092320
dev.ix.1.mac_stats.tx_frames_512_1023: 10358523
dev.ix.1.mac_stats.tx_frames_1024_1522: 454177889

> -----Original Message-----
> From: owner-freebsd-net@freebsd.org [mailto:owner-freebsd-
> net@freebsd.org] On Behalf Of Marcelo Gondim
> Sent: Monday, September 08, 2014 8:52 PM
> To: Eric Joyner
> Cc: Jack F Vogel; FreeBSD Net; Adrian Chadd
> Subject: Re: ixgbe CRITICAL: ECC ERROR!! Please Reboot!!
> 
> On 08/09/2014 15:38, Eric Joyner wrote:
> > Getting local / remote faults is strange; are these stats from after you
> > rebooted?
> 
> When the crash happens I just do:
> 
> # ifconfig ix0 down; ifconfig ix0 up; ifconfig ix1 down; ifconfig up ixi1
> 
> After that the two interface ports return to work.
> 
> In no time I had to restart the server.
> 
> # uptime
>   3:51PM  up 11 days, 15:44, 2 users, load averages: 5.36, 5.68, 5.69
> 
> Tomorrow will be going to the Datacenter to check on the temperature of
> the network interface. As the crashes occur when traffic is high, I
> believe that the temperature increases because processing is high.
> 
> 
> >
> > ---
> > - Eric Joyner
> >
> > On Fri, Sep 5, 2014 at 8:39 PM, Marcelo Gondim <gondim@bsdinfo.com.br>
> > wrote:
> >
> >> On 05/09/2014 17:17, Marcelo Gondim wrote:
> >>
> >>> On 05/09/2014 16:49, Adrian Chadd wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> But is the airflow in the unit sufficient?
> >>>>
> >>>> I had this problem at a previous job - the box was running fine, the
> >>>> room was very cold, but the internal fans in the server were set to
> >>>> "be very quiet". It wasn't enough to keep the ixgbe NICs happy. I had
> >>>> to change the fan settings to "just always run full speed".
> >>>>
> >>>> The fan temperature feedback loop was based on sensors on the CPU,
> >>>> _not_ on the peripherals.
> >>>>
> >>> Hi Adrian,
> >>>
> >>> Ummm. I'll check it and improve internal cooling.  :)
> >>> She is not happy and I'm also not. rsrsrsr
> >>>
> >>> Cheers,
> >>>
> >> Besides the problem of heating of the network interface, I am putting
> some
> >> information here. Could you tell me if there is something strange or is
it
> >> normal?
> >>
> >> dev.ix.0.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version
> -
> >> 2.5.15
> >> dev.ix.0.%driver: ix
> >> dev.ix.0.%location: slot=0 function=0 handle=\_SB_.PCI1.BR48.S3F0
> >> dev.ix.0.%pnpinfo: vendor=0x8086 device=0x154d subvendor=0x8086
> >> subdevice=0x7b11 class=0x020000
> >> dev.ix.0.%parent: pci131
> >> dev.ix.0.fc: 3
> >> dev.ix.0.enable_aim: 1
> >> dev.ix.0.advertise_speed: 0
> >> dev.ix.0.dropped: 0
> >> dev.ix.0.mbuf_defrag_failed: 0
> >> dev.ix.0.watchdog_events: 0
> >> dev.ix.0.link_irq: 121769
> >> dev.ix.0.queue0.interrupt_rate: 5319
> >> dev.ix.0.queue0.irqs: 7900830877
> >> dev.ix.0.queue0.txd_head: 1037
> >> dev.ix.0.queue0.txd_tail: 1037
> >> dev.ix.0.queue0.tso_tx: 142
> >> dev.ix.0.queue0.no_tx_dma_setup: 0
> >> dev.ix.0.queue0.no_desc_avail: 0
> >> dev.ix.0.queue0.tx_packets: 9725701450
> >> dev.ix.0.queue0.rxd_head: 1175
> >> dev.ix.0.queue0.rxd_tail: 1174
> >> dev.ix.0.queue0.rx_packets: 13069276955
> >> dev.ix.0.queue0.rx_bytes: 3391061018
> >> dev.ix.0.queue0.rx_copies: 8574407
> >> dev.ix.0.queue0.lro_queued: 0
> >> dev.ix.0.queue0.lro_flushed: 0
> >> dev.ix.0.queue1.interrupt_rate: 41666
> >> dev.ix.0.queue1.irqs: 7681141208
> >> dev.ix.0.queue1.txd_head: 219
> >> dev.ix.0.queue1.txd_tail: 221
> >> dev.ix.0.queue1.tso_tx: 57
> >> dev.ix.0.queue1.no_tx_dma_setup: 0
> >> dev.ix.0.queue1.no_desc_avail: 44334
> >> dev.ix.0.queue1.tx_packets: 10196891433
> >> dev.ix.0.queue1.rxd_head: 1988
> >> dev.ix.0.queue1.rxd_tail: 1987
> >> dev.ix.0.queue1.rx_packets: 13210132242
> >> dev.ix.0.queue1.rx_bytes: 4317357059
> >> dev.ix.0.queue1.rx_copies: 8131936
> >> dev.ix.0.queue1.lro_queued: 0
> >> dev.ix.0.queue1.lro_flushed: 0
> >> dev.ix.0.queue2.interrupt_rate: 5319
> >> dev.ix.0.queue2.irqs: 7647486080
> >> dev.ix.0.queue2.txd_head: 761
> >> dev.ix.0.queue2.txd_tail: 761
> >> dev.ix.0.queue2.tso_tx: 409
> >> dev.ix.0.queue2.no_tx_dma_setup: 0
> >> dev.ix.0.queue2.no_desc_avail: 54207
> >> dev.ix.0.queue2.tx_packets: 10161246425
> >> dev.ix.0.queue2.rxd_head: 1874
> >> dev.ix.0.queue2.rxd_tail: 1872
> >> dev.ix.0.queue2.rx_packets: 13175551880
> >> dev.ix.0.queue2.rx_bytes: 4472798418
> >> dev.ix.0.queue2.rx_copies: 7488876
> >> dev.ix.0.queue2.lro_queued: 0
> >> dev.ix.0.queue2.lro_flushed: 0
> >> dev.ix.0.queue3.interrupt_rate: 500000
> >> dev.ix.0.queue3.irqs: 7641129521
> >> dev.ix.0.queue3.txd_head: 2039
> >> dev.ix.0.queue3.txd_tail: 2039
> >> dev.ix.0.queue3.tso_tx: 9
> >> dev.ix.0.queue3.no_tx_dma_setup: 0
> >> dev.ix.0.queue3.no_desc_avail: 150346
> >> dev.ix.0.queue3.tx_packets: 10619971896
> >> dev.ix.0.queue3.rxd_head: 1055
> >> dev.ix.0.queue3.rxd_tail: 1054
> >> dev.ix.0.queue3.rx_packets: 13137835529
> >> dev.ix.0.queue3.rx_bytes: 4063197306
> >> dev.ix.0.queue3.rx_copies: 8188713
> >> dev.ix.0.queue3.lro_queued: 0
> >> dev.ix.0.queue3.lro_flushed: 0
> >> dev.ix.0.queue4.interrupt_rate: 5319
> >> dev.ix.0.queue4.irqs: 7439824996
> >> dev.ix.0.queue4.txd_head: 26
> >> dev.ix.0.queue4.txd_tail: 26
> >> dev.ix.0.queue4.tso_tx: 553912
> >> dev.ix.0.queue4.no_tx_dma_setup: 0
> >> dev.ix.0.queue4.no_desc_avail: 0
> >> dev.ix.0.queue4.tx_packets: 10658683718
> >> dev.ix.0.queue4.rxd_head: 684
> >> dev.ix.0.queue4.rxd_tail: 681
> >> dev.ix.0.queue4.rx_packets: 13204786830
> >> dev.ix.0.queue4.rx_bytes: 3700845239
> >> dev.ix.0.queue4.rx_copies: 8193379
> >> dev.ix.0.queue4.lro_queued: 0
> >> dev.ix.0.queue4.lro_flushed: 0
> >> dev.ix.0.queue5.interrupt_rate: 15151
> >> dev.ix.0.queue5.irqs: 7456613396
> >> dev.ix.0.queue5.txd_head: 603
> >> dev.ix.0.queue5.txd_tail: 603
> >> dev.ix.0.queue5.tso_tx: 17
> >> dev.ix.0.queue5.no_tx_dma_setup: 0
> >> dev.ix.0.queue5.no_desc_avail: 0
> >> dev.ix.0.queue5.tx_packets: 10639139790
> >> dev.ix.0.queue5.rxd_head: 404
> >> dev.ix.0.queue5.rxd_tail: 403
> >> dev.ix.0.queue5.rx_packets: 13144301293
> >> dev.ix.0.queue5.rx_bytes: 3986784766
> >> dev.ix.0.queue5.rx_copies: 8256195
> >> dev.ix.0.queue5.lro_queued: 0
> >> dev.ix.0.queue5.lro_flushed: 0
> >> dev.ix.0.queue6.interrupt_rate: 125000
> >> dev.ix.0.queue6.irqs: 7466940576
> >> dev.ix.0.queue6.txd_head: 1784
> >> dev.ix.0.queue6.txd_tail: 1784
> >> dev.ix.0.queue6.tso_tx: 2001
> >> dev.ix.0.queue6.no_tx_dma_setup: 0
> >> dev.ix.0.queue6.no_desc_avail: 0
> >> dev.ix.0.queue6.tx_packets: 9784312967
> >> dev.ix.0.queue6.rxd_head: 395
> >> dev.ix.0.queue6.rxd_tail: 394
> >> dev.ix.0.queue6.rx_packets: 13103079970
> >> dev.ix.0.queue6.rx_bytes: 3581485264
> >> dev.ix.0.queue6.rx_copies: 7336569
> >> dev.ix.0.queue6.lro_queued: 0
> >> dev.ix.0.queue6.lro_flushed: 0
> >> dev.ix.0.queue7.interrupt_rate: 5319
> >> dev.ix.0.queue7.irqs: 7486391989
> >> dev.ix.0.queue7.txd_head: 1549
> >> dev.ix.0.queue7.txd_tail: 1549
> >> dev.ix.0.queue7.tso_tx: 2052
> >> dev.ix.0.queue7.no_tx_dma_setup: 0
> >> dev.ix.0.queue7.no_desc_avail: 0
> >> dev.ix.0.queue7.tx_packets: 9758335509
> >> dev.ix.0.queue7.rxd_head: 1990
> >> dev.ix.0.queue7.rxd_tail: 1986
> >> dev.ix.0.queue7.rx_packets: 13141436559
> >> dev.ix.0.queue7.rx_bytes: 3877881546
> >> dev.ix.0.queue7.rx_copies: 8505244
> >> dev.ix.0.queue7.lro_queued: 0
> >> dev.ix.0.queue7.lro_flushed: 0
> >> dev.ix.0.mac_stats.crc_errs: 159
> >> dev.ix.0.mac_stats.ill_errs: 5
> >> dev.ix.0.mac_stats.byte_errs: 10
> >> dev.ix.0.mac_stats.short_discards: 0
> >> dev.ix.0.mac_stats.local_faults: 213
> >> dev.ix.0.mac_stats.remote_faults: 37
> >> dev.ix.0.mac_stats.rec_len_errs: 0
> >> dev.ix.0.mac_stats.xon_txd: 33623392288
> >> dev.ix.0.mac_stats.xon_recvd: 0
> >> dev.ix.0.mac_stats.xoff_txd: 110161
> >> dev.ix.0.mac_stats.xoff_recvd: 0
> >> dev.ix.0.mac_stats.total_octets_rcvd: 37967032023930
> >> dev.ix.0.mac_stats.good_octets_rcvd: 37965112076546
> >> dev.ix.0.mac_stats.total_pkts_rcvd: 105201314224
> >> dev.ix.0.mac_stats.good_pkts_rcvd: 18446743122750685300
> >> dev.ix.0.mac_stats.mcast_pkts_rcvd: 43835
> >> dev.ix.0.mac_stats.bcast_pkts_rcvd: 960566
> >> dev.ix.0.mac_stats.rx_frames_64: 454067
> >> dev.ix.0.mac_stats.rx_frames_65_127: 75350664570
> >> dev.ix.0.mac_stats.rx_frames_128_255: 4726321485
> >> dev.ix.0.mac_stats.rx_frames_256_511: 2282382994
> >> dev.ix.0.mac_stats.rx_frames_512_1023: 3293330145
> >> dev.ix.0.mac_stats.rx_frames_1024_1522: 19541190234
> >> dev.ix.0.mac_stats.recv_undersized: 0
> >> dev.ix.0.mac_stats.recv_fragmented: 0
> >> dev.ix.0.mac_stats.recv_oversized: 0
> >> dev.ix.0.mac_stats.recv_jabberd: 4
> >> dev.ix.0.mac_stats.management_pkts_rcvd: 0
> >> dev.ix.0.mac_stats.management_pkts_drpd: 0
> >> dev.ix.0.mac_stats.checksum_errs: 527716650
> >> dev.ix.0.mac_stats.good_octets_txd: 81355567084088
> >> dev.ix.0.mac_stats.total_pkts_txd: 81545370171
> >> dev.ix.0.mac_stats.good_pkts_txd: 47921867720
> >> dev.ix.0.mac_stats.bcast_pkts_txd: 298646
> >> dev.ix.0.mac_stats.mcast_pkts_txd: 18446744040086169062
> >> dev.ix.0.mac_stats.management_pkts_txd: 0
> >> dev.ix.0.mac_stats.tx_frames_64: 18446744045326220993
> >> dev.ix.0.mac_stats.tx_frames_65_127: 15629697019
> >> dev.ix.0.mac_stats.tx_frames_128_255: 4141756940
> >> dev.ix.0.mac_stats.tx_frames_256_511: 2195917491
> >> dev.ix.0.mac_stats.tx_frames_512_1023: 2717769904
> >> dev.ix.0.mac_stats.tx_frames_1024_1522: 51620056990
> >>
> >> dev.ix.1.%desc: Intel(R) PRO/10GbE PCI-Express Network Driver, Version
> -
> >> 2.5.15
> >> dev.ix.1.%driver: ix
> >> dev.ix.1.%location: slot=0 function=1 handle=\_SB_.PCI1.BR48.S3F1
> >> dev.ix.1.%pnpinfo: vendor=0x8086 device=0x154d subvendor=0x8086
> >> subdevice=0x7b11 class=0x020000
> >> dev.ix.1.%parent: pci131
> >> dev.ix.1.fc: 3
> >> dev.ix.1.enable_aim: 1
> >> dev.ix.1.advertise_speed: 0
> >> dev.ix.1.dropped: 0
> >> dev.ix.1.mbuf_defrag_failed: 0
> >> dev.ix.1.watchdog_events: 0
> >> dev.ix.1.link_irq: 46692
> >> dev.ix.1.queue0.interrupt_rate: 15151
> >> dev.ix.1.queue0.irqs: 2634341620
> >> dev.ix.1.queue0.txd_head: 1101
> >> dev.ix.1.queue0.txd_tail: 1101
> >> dev.ix.1.queue0.tso_tx: 0
> >> dev.ix.1.queue0.no_tx_dma_setup: 0
> >> dev.ix.1.queue0.no_desc_avail: 4142
> >> dev.ix.1.queue0.tx_packets: 4749431326
> >> dev.ix.1.queue0.rxd_head: 812
> >> dev.ix.1.queue0.rxd_tail: 811
> >> dev.ix.1.queue0.rx_packets: 21135516
> >> dev.ix.1.queue0.rx_bytes: 33229682
> >> dev.ix.1.queue0.rx_copies: 119434
> >> dev.ix.1.queue0.lro_queued: 0
> >> dev.ix.1.queue0.lro_flushed: 0
> >> dev.ix.1.queue1.interrupt_rate: 15151
> >> dev.ix.1.queue1.irqs: 2616347302
> >> dev.ix.1.queue1.txd_head: 797
> >> dev.ix.1.queue1.txd_tail: 797
> >> dev.ix.1.queue1.tso_tx: 0
> >> dev.ix.1.queue1.no_tx_dma_setup: 0
> >> dev.ix.1.queue1.no_desc_avail: 4129
> >> dev.ix.1.queue1.tx_packets: 4730356550
> >> dev.ix.1.queue1.rxd_head: 1345
> >> dev.ix.1.queue1.rxd_tail: 1344
> >> dev.ix.1.queue1.rx_packets: 20844457
> >> dev.ix.1.queue1.rx_bytes: 35014827
> >> dev.ix.1.queue1.rx_copies: 125167
> >> dev.ix.1.queue1.lro_queued: 0
> >> dev.ix.1.queue1.lro_flushed: 0
> >> dev.ix.1.queue2.interrupt_rate: 5555
> >> dev.ix.1.queue2.irqs: 2603166062
> >> dev.ix.1.queue2.txd_head: 1428
> >> dev.ix.1.queue2.txd_tail: 1432
> >> dev.ix.1.queue2.tso_tx: 0
> >> dev.ix.1.queue2.no_tx_dma_setup: 0
> >> dev.ix.1.queue2.no_desc_avail: 34368
> >> dev.ix.1.queue2.tx_packets: 4681339909
> >> dev.ix.1.queue2.rxd_head: 1153
> >> dev.ix.1.queue2.rxd_tail: 1152
> >> dev.ix.1.queue2.rx_packets: 21263862
> >> dev.ix.1.queue2.rx_bytes: 46251419
> >> dev.ix.1.queue2.rx_copies: 125156
> >> dev.ix.1.queue2.lro_queued: 0
> >> dev.ix.1.queue2.lro_flushed: 0
> >> dev.ix.1.queue3.interrupt_rate: 5319
> >> dev.ix.1.queue3.irqs: 2658483423
> >> dev.ix.1.queue3.txd_head: 179
> >> dev.ix.1.queue3.txd_tail: 179
> >> dev.ix.1.queue3.tso_tx: 0
> >> dev.ix.1.queue3.no_tx_dma_setup: 0
> >> dev.ix.1.queue3.no_desc_avail: 4189
> >> dev.ix.1.queue3.tx_packets: 4811705668
> >> dev.ix.1.queue3.rxd_head: 583
> >> dev.ix.1.queue3.rxd_tail: 582
> >> dev.ix.1.queue3.rx_packets: 20472051
> >> dev.ix.1.queue3.rx_bytes: 29823943
> >> dev.ix.1.queue3.rx_copies: 129450
> >> dev.ix.1.queue3.lro_queued: 0
> >> dev.ix.1.queue3.lro_flushed: 0
> >> dev.ix.1.queue4.interrupt_rate: 5319
> >> dev.ix.1.queue4.irqs: 2629295642
> >> dev.ix.1.queue4.txd_head: 592
> >> dev.ix.1.queue4.txd_tail: 594
> >> dev.ix.1.queue4.tso_tx: 0
> >> dev.ix.1.queue4.no_tx_dma_setup: 0
> >> dev.ix.1.queue4.no_desc_avail: 4106
> >> dev.ix.1.queue4.tx_packets: 4774679355
> >> dev.ix.1.queue4.rxd_head: 259
> >> dev.ix.1.queue4.rxd_tail: 258
> >> dev.ix.1.queue4.rx_packets: 22561633
> >> dev.ix.1.queue4.rx_bytes: 40196457
> >> dev.ix.1.queue4.rx_copies: 135966
> >> dev.ix.1.queue4.lro_queued: 0
> >> dev.ix.1.queue4.lro_flushed: 0
> >> dev.ix.1.queue5.interrupt_rate: 18518
> >> dev.ix.1.queue5.irqs: 2628129098
> >> dev.ix.1.queue5.txd_head: 1070
> >> dev.ix.1.queue5.txd_tail: 1072
> >> dev.ix.1.queue5.tso_tx: 0
> >> dev.ix.1.queue5.no_tx_dma_setup: 0
> >> dev.ix.1.queue5.no_desc_avail: 4127
> >> dev.ix.1.queue5.tx_packets: 4750613588
> >> dev.ix.1.queue5.rxd_head: 595
> >> dev.ix.1.queue5.rxd_tail: 594
> >> dev.ix.1.queue5.rx_packets: 21309940
> >> dev.ix.1.queue5.rx_bytes: 29264004
> >> dev.ix.1.queue5.rx_copies: 134128
> >> dev.ix.1.queue5.lro_queued: 0
> >> dev.ix.1.queue5.lro_flushed: 0
> >> dev.ix.1.queue6.interrupt_rate: 125000
> >> dev.ix.1.queue6.irqs: 2615444123
> >> dev.ix.1.queue6.txd_head: 1008
> >> dev.ix.1.queue6.txd_tail: 1008
> >> dev.ix.1.queue6.tso_tx: 0
> >> dev.ix.1.queue6.no_tx_dma_setup: 0
> >> dev.ix.1.queue6.no_desc_avail: 4106
> >> dev.ix.1.queue6.tx_packets: 4703519800
> >> dev.ix.1.queue6.rxd_head: 1209
> >> dev.ix.1.queue6.rxd_tail: 1208
> >> dev.ix.1.queue6.rx_packets: 22783319
> >> dev.ix.1.queue6.rx_bytes: 28091721
> >> dev.ix.1.queue6.rx_copies: 131653
> >> dev.ix.1.queue6.lro_queued: 0
> >> dev.ix.1.queue6.lro_flushed: 0
> >> dev.ix.1.queue7.interrupt_rate: 6024
> >> dev.ix.1.queue7.irqs: 2644640254
> >> dev.ix.1.queue7.txd_head: 1313
> >> dev.ix.1.queue7.txd_tail: 1313
> >> dev.ix.1.queue7.tso_tx: 0
> >> dev.ix.1.queue7.no_tx_dma_setup: 0
> >> dev.ix.1.queue7.no_desc_avail: 4106
> >> dev.ix.1.queue7.tx_packets: 4824584039
> >> dev.ix.1.queue7.rxd_head: 1463
> >> dev.ix.1.queue7.rxd_tail: 1462
> >> dev.ix.1.queue7.rx_packets: 21464418
> >> dev.ix.1.queue7.rx_bytes: 38133966
> >> dev.ix.1.queue7.rx_copies: 163180
> >> dev.ix.1.queue7.lro_queued: 0
> >> dev.ix.1.queue7.lro_flushed: 0
> >> dev.ix.1.mac_stats.crc_errs: 0
> >> dev.ix.1.mac_stats.ill_errs: 0
> >> dev.ix.1.mac_stats.byte_errs: 0
> >> dev.ix.1.mac_stats.short_discards: 0
> >> dev.ix.1.mac_stats.local_faults: 0
> >> dev.ix.1.mac_stats.remote_faults: 2
> >> dev.ix.1.mac_stats.rec_len_errs: 0
> >> dev.ix.1.mac_stats.xon_txd: 11207820571
> >> dev.ix.1.mac_stats.xon_recvd: 0
> >> dev.ix.1.mac_stats.xoff_txd: 3735958948
> >> dev.ix.1.mac_stats.xoff_recvd: 0
> >> dev.ix.1.mac_stats.total_octets_rcvd: 47828274751
> >> dev.ix.1.mac_stats.good_octets_rcvd: 47225376551
> >> dev.ix.1.mac_stats.total_pkts_rcvd: 177931572
> >> dev.ix.1.mac_stats.good_pkts_rcvd: 18446743954331576567
> >> dev.ix.1.mac_stats.mcast_pkts_rcvd: 7410
> >> dev.ix.1.mac_stats.bcast_pkts_rcvd: 8907
> >> dev.ix.1.mac_stats.rx_frames_64: 0
> >> dev.ix.1.mac_stats.rx_frames_65_127: 134619798
> >> dev.ix.1.mac_stats.rx_frames_128_255: 7996096
> >> dev.ix.1.mac_stats.rx_frames_256_511: 3996945
> >> dev.ix.1.mac_stats.rx_frames_512_1023: 5145938
> >> dev.ix.1.mac_stats.rx_frames_1024_1522: 21331154
> >> dev.ix.1.mac_stats.recv_undersized: 0
> >> dev.ix.1.mac_stats.recv_fragmented: 0
> >> dev.ix.1.mac_stats.recv_oversized: 0
> >> dev.ix.1.mac_stats.recv_jabberd: 0
> >> dev.ix.1.mac_stats.management_pkts_rcvd: 0
> >> dev.ix.1.mac_stats.management_pkts_drpd: 0
> >> dev.ix.1.mac_stats.checksum_errs: 219681
> >> dev.ix.1.mac_stats.good_octets_txd: 38874487735713
> >> dev.ix.1.mac_stats.total_pkts_txd: 38026147390
> >> dev.ix.1.mac_stats.good_pkts_txd: 27377335167
> >> dev.ix.1.mac_stats.bcast_pkts_txd: 67411
> >> dev.ix.1.mac_stats.mcast_pkts_txd: 18446744063060778699
> >> dev.ix.1.mac_stats.management_pkts_txd: 0
> >> dev.ix.1.mac_stats.tx_frames_64: 18446744065500352065
> >> dev.ix.1.mac_stats.tx_frames_65_127: 6755049420
> >> dev.ix.1.mac_stats.tx_frames_128_255: 1914523761
> >> dev.ix.1.mac_stats.tx_frames_256_511: 1038553757
> >> dev.ix.1.mac_stats.tx_frames_512_1023: 1240890942
> >> dev.ix.1.mac_stats.tx_frames_1024_1522: 24637516838
> >>
> >>
> >> Cheers,
> >>
> >>
> >>>> -a
> >>>>
> >>>>
> >>>> On 5 September 2014 07:35, Marcelo Gondim
> <gondim@bsdinfo.com.br> wrote:
> >>>>
> >>>>> Hi Adrian,
> >>>>>
> >>>>> I confirmed with the support staff of the room where the server is,
> >>>>> that the
> >>>>> ambient temperature was normal.
> >>>>>
> >>>>>
> >>>>> On 04/09/2014 22:46, Marcelo Gondim wrote:
> >>>>>
> >>>>>> On 04/09/2014 20:48, Adrian Chadd wrote:
> >>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> The only time this has happened to me is because the card
> overheated.
> >>>>>>> Can you check that?
> >>>>>>>
> >>>>>> Hi Adrian,
> >>>>>>
> >>>>>> The room where the equipment is located is very cold but I'll check
it
> >>>>>> out.
> >>>>>> Also seen at the time of the problem, a lot of dropped packets.
> >>>>>>
> >>>>>> # netstat -idn
> >>>>>> ...
> >>>>>> ix0    1500 <Link#9>      a0:36:9f:2a:6d:ac 18446743423829095869
159
> >>>>>> 750924631703 53285910688     0 0     0
> >>>>>> ix0       - fe80::a236:9f fe80::a236:9fff:f        0     - -
2
> >>>>>> -     -     -
> >>>>>> ix1    1500 <Link#10>     a0:36:9f:2a:6d:ae 18446743954328745465
0
> >>>>>> 119550050209 20178077451     0 0     0
> >>>>>> ix1       - fe80::a236:9f fe80::a236:9fff:f        0     - -
1
> >>>>>> -     -     -
> >>>>>> ...
> >>>>>>
> >>>>>> 119550050209 droped packets on ix1 and 750924631703 droped on ix0
> >>>>>>
> >>>>>> Could be interesting I upgrade to10.1-PRERELEASE?
> >>>>>> Could there be a problem with the driver?
> >>>>>>
> >>>>>> Traffic on ix0: 1.4Gbps output / 600Mbps input
> >>>>>> Traffic on ix1: 1.2Gbps output
> >>>>>>
> >>>>>> PPS on ix0: 163Kpps output / 215Kpps input
> >>>>>> PPS on ix1: 131Kpps output
> >>>>>>
> >>>>>> Thanks for your help.
> >>>>>>
> >>>>>>>
> >>>>>>> -a
> >>>>>>>
> >>>>>>>
> >>>>>>> On 4 September 2014 16:14, Marcelo Gondim
> <gondim@bsdinfo.com.br>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hi All,
> >>>>>>>>
> >>>>>>>> I have an Intel X520-SR2and today was working when all traffic
> >>>>>>>> stopped.
> >>>>>>>> I looked in the logs and found this message:
> >>>>>>>>
> >>>>>>>> Sep  4 18:29:53 rt01 kernel: ix1:
> >>>>>>>> Sep  4 18:29:53 rt01 kernel: CRITICAL: ECC ERROR!! Please
Reboot!!
> >>>>>>>>
> >>>>>>>> # uname -a
> >>>>>>>> FreeBSD rt01.xxxxx.com.br 10.0-STABLE FreeBSD 10.0-STABLE #10
> >>>>>>>> r267839:
> >>>>>>>> Thu
> >>>>>>>> Jul 10 15:35:04 BRT 2014
> >>>>>>>> root@rt01.xxxxx.com.br:/usr/obj/usr/src/sys/GONDIM10 amd64
> >>>>>>>>
> >>>>>>>> # netstat -m
> >>>>>>>> 98324/53476/151800 mbufs in use (current/cache/total)
> >>>>>>>> 98301/44951/143252/1014370 mbuf clusters in use
> >>>>>>>> (current/cache/total/max)
> >>>>>>>> 98301/44897 mbuf+clusters out of packet secondary zone in use
> >>>>>>>> (current/cache)
> >>>>>>>> 0/421/421/507184 4k (page size) jumbo clusters in use
> >>>>>>>> (current/cache/total/max)
> >>>>>>>> 0/0/0/150276 9k jumbo clusters in use (current/cache/total/max)
> >>>>>>>> 0/0/0/84530 16k jumbo clusters in use (current/cache/total/max)
> >>>>>>>> 221183K/104955K/326138K bytes allocated to network
> >>>>>>>> (current/cache/total)
> >>>>>>>> 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
> >>>>>>>> 0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
> >>>>>>>> 0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
> >>>>>>>> 0/0/0 requests for jumbo clusters denied (4k/9k/16k)
> >>>>>>>> 0 requests for sfbufs denied
> >>>>>>>> 0 requests for sfbufs delayed
> >>>>>>>> 0 requests for I/O initiated by sendfile
> >>>>>>>>
> >>>>>>>> Best regards,
> >>>>>>>>
> 
> _______________________________________________
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?002001cfcba0$4812c7f0$d83857d0$>