Date: Fri, 1 Aug 1997 14:27:14 -0400 (EDT) From: Andrew Gallatin <gallatin@CS.Duke.EDU> To: Chris Csanady <ccsanady@friley01.res.iastate.edu> Cc: freebsd-current@freebsd.org Subject: Re: Myrinet, etc.. (Re: code talks: announcing EIDE bus master patches) Message-ID: <199708011827.OAA27698@hurricane.cs.duke.edu> In-Reply-To: <199708010252.VAA15322@friley01.res.iastate.edu> References: <199707312354.TAA29288@hurricane.cs.duke.edu> <199708010252.VAA15322@friley01.res.iastate.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
Chris Csanady writes: > > > > I'd also like to point out that the 300Mb's is acheived using an IP > > > stack layered on top of some active messaging protocol. (which is > > > implemented on the io processor on the nic.) > > > > > > As far as the TCP/IP stack under FreeBSD, all you can push through it > > > is about 150Mb/s. This is somewhat unfortunate, although surprisingly > > > linux doesn't seem to manage much better. I think it would be quite > > > nice if we could correctly implement a zero copy architecture.. > > > > > > Chris Csanady > > > >Actually, that's not true. I'm using an in-kernel IP driver layered > >on top of Myricom's MyriApi general-purpose messaging software (which > >is kernel resident). It hooks into the network stack the way any > >ethernet driver would (and gets hit with a copy for each xmit from or > >receive into user space like anything else does). There is software > >that's downloaded onto the interface card, but it has no protocol > >specific knowledge. When its sitting under the IP driver, it is used > >the same way the firwmware on any ethernet card would be used -- the > >driver tells it a MAC address that it wants something sent to, and it > >sends it, etc. > > Sorry, I guess I assumed wrong. Me too.. I hope I didn't over-react. > However, the numbers you are > reporting are vastly different from he ones mentioned on myrinet's > web page for 2.2.1. Yeah.. the UDP numbers on Myri's page are a little out of date, plus the TCP numbers are from a benchmark run w/TCP send & receive space at their 16k defaults. The b/w nearly doubles if you crank them up to 64k. Also, my 356Mb/sec is taken when receiving from an alpha. I cannot transmit that fast with an Intel box yet. > >The fact that we're running over Myrinet & not ethernet is transparent > >to any application. In this configuration, with the hardware I > >currently have available (memory b/w challenged Pentium Pros) I can > >receive UDP traffic at about 356 Mb/sec and send UDP traffic at a rate > >of a about 280Mb/sec. TCP streams are around 275Mb/sec. > > That really is quite impressive--is your driver available? I would > like to play with it when I get a chance. :) I'm also quite > interested in how your interrupt/buffer architecture looks. Sure. There are pointers to both the distribution tarball & a CVS-web browsable interface at http://www.cs.duke.edu/ari/manic/ip/ > As for memory bandwith limitations, I can't wait for the alpha > port. ;) > > Chris Csanady In terms of memory b/w, the alphas are ... "interesting". The older 21171 core-logic chipset alphas (AlphaStation 500, AlphaStation 600, EB164, PC164) leave a bit to be desired, especially when both the CPU and a gigabit Myrinet interface are competing for the memory system. The newer 21174 core-logic chipset alphas (Personal Workstation series, PC164lx) are much better. See http://www.cs.duke.edu/ari/manic/ip/dma-perf/dma-bcopy.html for some graphs that illustrate this point. Drew ------------------------------------------------------------------------------ Andrew Gallatin http://www.cs.duke.edu/~gallatin Duke University Email: gallatin@cs.duke.edu Department of Computer Science Phone: (919) 660-6590
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199708011827.OAA27698>