Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 09 May 2010 05:54:19 -0400
From:      joe <joe@hostedcontent.com>
To:        Jack Vogel <jfvogel@gmail.com>
Cc:        Ian FREISLICH <ianf@clue.co.za>, Fabien Thomas <fabien.thomas@netasq.com>, freebsd-current@freebsd.org
Subject:   Re: igb broken? Unexplained weirdness with intel 82576 nics on a supermicro board.
Message-ID:  <4BE6864B.5060906@hostedcontent.com>
In-Reply-To: <AANLkTincmLc5qGkfnQsGWO8OGjaotR_wquRxIiBBLQz9@mail.gmail.com>
References:  <4BE565E5.9030505@hostedcontent.com>	 <4BE529FF.5000008@hostedcontent.com>	 <C15B47BC-AF5F-4AF4-899C-98155EC5A4D4@netasq.com>	 <E1OAhhO-00021o-DH@clue.co.za> <E1OAlmh-0002Dj-Mm@clue.co.za>	 <4BE59434.9070308@hostedcontent.com>	 <AANLkTikLmEwr15mPz9Hz1kB97dXUNhkOS8R47d68rm1z@mail.gmail.com>	 <4BE599B0.60203@hostedcontent.com>	 <AANLkTikVu2mfnLXzYEms4RO0toTvBctARq1lQ2ZmnVOT@mail.gmail.com>	 <4BE59DBD.4000105@hostedcontent.com> <AANLkTincmLc5qGkfnQsGWO8OGjaotR_wquRxIiBBLQz9@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On 05/08/2010 02:21 PM, Jack Vogel wrote:
> The cable, its a simple thing but make SURE you try that, a slightly
> damaged one can do weird things and its quick to check, don't overlook
> it.
>
> Jack
>
>
> On Sat, May 8, 2010 at 10:22 AM, joe <joe@hostedcontent.com
> <mailto:joe@hostedcontent.com>> wrote:
>
>     On 05/08/2010 01:53 PM, Jack Vogel wrote:
>
>         I still am not clear on this system, how many ports are on it,
>         and its
>         an 82576?
>         Sounds to me like you've proven its not on the box if you can do
>         fine
>         when its
>         on its own. So change ports in the switch, as I said, change
>         cables, must be
>         something in that environment.
>
>         Jack
>
>
>         On Sat, May 8, 2010 at 10:04 AM, joe <joe@hostedcontent.com
>         <mailto:joe@hostedcontent.com>
>         <mailto:joe@hostedcontent.com <mailto:joe@hostedcontent.com>>>
>         wrote:
>
>             On 05/08/2010 01:31 PM, Jack Vogel wrote:
>
>                 Looks like something to do with system C, you might
>         isolate it,
>                 and try
>                 a back
>                 to back connection with its NICs, change cables, look at
>         BIOS
>                 settings,
>                 change
>                 the slot the nic is in... All just off the top of my head.
>
>                 Jack
>
>
>                 On Sat, May 8, 2010 at 9:41 AM, joe
>         <joe@hostedcontent.com <mailto:joe@hostedcontent.com>
>         <mailto:joe@hostedcontent.com <mailto:joe@hostedcontent.com>>
>         <mailto:joe@hostedcontent.com <mailto:joe@hostedcontent.com>
>         <mailto:joe@hostedcontent.com <mailto:joe@hostedcontent.com>>>>
>
>                 wrote:
>
>                     On 05/08/2010 11:17 AM, Ian FREISLICH wrote:
>
>                         joe wrote:
>
>                             On 05/08/2010 06:55 AM, Ian FREISLICH wrote:
>
>                                 joe wrote:
>
>                                            I have just tried your
>         suggeston and
>                 it has
>                                     no effect for me ;(
>
>
>                                 Do you have another brand of NIC that
>         you can
>                 try?  At
>                                 least that
>                                 will isolate whether it's igb(4) or
>         something else.
>
>
>                             I will grab a new nic today and try...my
>         options are
>                 limited
>                             though.
>                             Here are the nics i can get my hands on
>
>                             TP-LINK TL-TG3468, 10/100/1000Mbps PCIe Adapter
>                 (supported
>                             by fbsd?)
>
>
>                         Based on the RTL8168B chip.  Should be supported
>         by the
>                 re(4)
>                         driver.
>
>                             Intel (EXPI9301CT) Gigabit CT Desktop
>         Adapter (yet
>                 another
>                             intel nic)
>
>
>                         i82574L chip.  Should be supported by the em(4)
>         driver.
>                   I have had
>                         good performance in the past with this driver
>         and less than
>                         satisfactory performance with the igb(4) driver.
>
>                         That may not be your problem though.  Before you
>         go out
>                 and buy,
>                         have a look at the amount of interrupt time your
>         slow
>                 machine spends
>                         in 'top' or 'systat -vm'.  systat will also show the
>                 interrupt rate
>                         for each driver, perhaps it's not doing
>         interrupt moderation
>                         properly.
>                         This will manifest as more than about a 1000 per
>         second.
>                   There are
>                         loader tunables for the driver to increase the
>         number of
>                 transfer
>                         descriptors and to tune interrupt moderation.
>
>                         You could try running trafshow (port) on the
>         interface while
>                         performing the transfer.  Perhaps promiscuous
>         mode will
>                 turn off
>                         some hardware feature that will improve things.
>           It may
>                 however
>                         break hardware vlanning as it does on my 82575GB
>         4 port
>                 igb card.
>
>                         Ian
>
>                         --
>                         Ian Freislich
>
>
>                     I bought those two cards anyways, im in a rush to
>         figure out
>                 this
>                     problem. That being said i am still encountering the
>         exact same
>                     problem regardless on which network card i am
>         running. I am at a
>                     complete loss. I am about to try a raid card to see
>         if the
>                 problem
>                     might lay within the onboard sata ports. I did pull the
>                 server and
>                     brought it home so that i can test more things quicker.
>
>                     I am going to try using a raid card instead of the
>         onboard sata
>                     ports and see if i still encounter the same problem.
>         I would
>                 love
>                     any suggestions you may have on where to go from here to
>                 figure out
>                     where the problem might be.
>
>                     joe
>
>                     _______________________________________________
>         freebsd-current@freebsd.org <mailto:freebsd-current@freebsd.org>
>         <mailto:freebsd-current@freebsd.org
>         <mailto:freebsd-current@freebsd.org>>
>         <mailto:freebsd-current@freebsd.org
>         <mailto:freebsd-current@freebsd.org>
>         <mailto:freebsd-current@freebsd.org
>         <mailto:freebsd-current@freebsd.org>>>
>
>                     mailing list
>         http://lists.freebsd.org/mailman/listinfo/freebsd-current
>                     To unsubscribe, send any mail to
>         "freebsd-current-unsubscribe@freebsd.org
>         <mailto:freebsd-current-unsubscribe@freebsd.org>
>         <mailto:freebsd-current-unsubscribe@freebsd.org
>         <mailto:freebsd-current-unsubscribe@freebsd.org>>
>         <mailto:freebsd-current-unsubscribe@freebsd.org
>         <mailto:freebsd-current-unsubscribe@freebsd.org>
>         <mailto:freebsd-current-unsubscribe@freebsd.org
>         <mailto:freebsd-current-unsubscribe@freebsd.org>>>"
>
>
>
>             I think it might have something to so with the nics /
>         switch, and
>             their features. I brought the box home, plugged into my gb
>         switch,
>             and i am able to FTP data to the server at around 35MB/sec.
>
>             I dont know what would cause this other than some sort of
>         issue with
>             the the 3 different types of nics and the switch i am using.
>
>             Any suggestions?
>
>
>
>     There are two embedded intel 82576 nics on this motherboard. I do
>     believe i have proven it is not the box itself as it is capable of
>     high incoming throughput. I have other servers on the switch which
>     do 55MB/sec without issues. I believe it is a combination of this
>     server and/or the nics i have and the switch i am using. It's the
>     only logical explanation if i get the desired throughput on my home
>     switch but not on the switch that is collocated. I will try updating
>     the firmware of the switch tonight as well as bringing the switch i
>     use at home with me.
>
>

Here is a follow-up just incase anyone ever encounters this problem 
again. I updated the firmware on the switch, made sure jumbo packets 
were enabled, and the switch was restarted. I'm now seeing the 
throughput i had expected on this box!

897755008 bytes received in 00:11 (72.74 MB/s)

Once again, thank you all for the help!

Joe



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4BE6864B.5060906>