Date: Thu, 3 Feb 2000 21:29:27 -0500 (EST) From: Bill Paul <wpaul@skynet.ctr.columbia.edu> To: ken@kdm.org (Kenneth D. Merry) Cc: freebsd-hackers@FreeBSD.ORG, freebsd-current@FreeBSD.ORG Subject: Re: Suggestions for Gigabit cards for -CURRENT Message-ID: <200002040229.VAA10541@skynet.ctr.columbia.edu> In-Reply-To: <20000202113259.A43505@panzer.kdm.org> from "Kenneth D. Merry" at Feb 2, 2000 11:32:59 am
next in thread | previous in thread | raw e-mail | index | archive | help
Of all the gin joints in all the towns in all the world, Kenneth D. Merry had to walk into mine and say: > On Wed, Feb 02, 2000 at 13:03:09 -0500, Thomas Stromberg wrote: > > We're currently looking at upgrading several of our FreeBSD servers > > (dual PIII-600's, 66MHz PCI) and some Sun Ultra's to Gigabit Ethernet. > > We plan to hook these machines into our Cisco Catalyst 5000 server. They > > will most likely move to be running FreeBSD 4.x by the time that we > > actually get our budget approved. What experiences do you guys have with > > the cards? > > > > Currently we're looking at the ~$1000 range, specifically at Alteon > > 512k's ($1000) for the FreeBSD servers and Sun Gigabit 2.0's ($2000) for > > the Sun servers. I was interested in the Myrinet cards (for obvious > > reasons), but they appear to require a Myrinet switch (though I found > > myself slightly confused so I may be wrong) rather then being able to > > hook into our Catalyst 5000. The Intel PRO/1000 Gigabit cards look > > rather nice too, but I haven't seen drivers yet for FreeBSD (Linux yes). > > > > I'm pretty much purchasing on marketing and reputation rather then any > > experience here, so any help would be much appreciated. > > I would recommend getting Alteon boards. It is likely that the Sun boards > are Alteon OEM, although I'm not positive. I think the first gigabit cards Sun had on the market were OEMed from Alteon, but I've been told that their newer cards are something else entirely. I don't know exactly what, but they're not Tigon-based. > One thing to keep in mind is that both Netgear and 3Com are OEMing Alteon > boards, and you'll get them much cheaper that way. The boards are pretty > much identical to the Alteon branded boards (which have no identifying > marks on them). The performance is the same, at least for the Netgear > boards. (I don't have any 3Com boards.) There are a number of companies selling OEM'ed alteon boards for various prices. IBM sells two cards, one for PC-based hardware and one for RS/6000s which I think are basically the same hardware with different driver kits. Of course, the RS/6000 card is $2100 while the PC-based one is probably around $600 or so. My guess is they're Alteon cards with different PCI device IDs, but I can't confirm this as I don't have one. The SGI gigabit adapter, NEC gigabit adapter, DEC EtherWORKS/1000, 3Com 3c985 and 3c985B, and the Netgear GA620 are all Tigon boards (not to mention the Alteon ACEnic) and should all work fine with the ti driver. Oh, I found another one recently: Farallon also sells a gigabit PCI NIC for the Mac which is Tigon-based. > The Netgear GA620 is a 512K Tigon 2 board, and generally goes for around > $300 or so. The 3Com boards have 1MB of SRAM, but I'm not sure whether > they're Tigon 1 or Tigon 2. You really want a Tigon 2 board. Maybe > someone who has one can comment. The original 3Com 3c985 was a Tigon 1 board (I have one) and the 3c985B is a Tigon 2. The Tigon 1 is no longer in production, though of course I try to maintain support for it for those people who still have them. The Tigon 1 had only a single R4000 CPU in it while the Tigon 2 has two. The Netgear GA620 is by far the cheapest at about $320. The various OEM cards sold for the PC are usually around $600, give or take $100. The GA620 only has 512K of SRAM compared to 1MB on most of the others, however you're not likely to notice a problem with that unless you try to push the card really hard with a really big TCP window size and jumbo frames. > The Intel cards may look nice, and there is a FreeBSD driver for them, but > I wouldn't get one. The first problem with the Intel boards is that there > are no docs for them. Supposedly they're using a Cisco chip, and the specs > for the chip are top secret. This is why I don't buy or recommend Intel NICs. But that's just my personal bias. > The FreeBSD driver (written by Matt Jacob) is based on the Linux driver, > which Intel wrote, and he hasn't yet managed to get decent throughput > through the cards. (Maybe Matt will comment.) They also only have 64K of > memory on board, which is insufficient for a heavily loaded server, IMO. > > Even with the 512K Alteon boards, you have a minimum of about 200K, and > probably more like 300K of cache for transmit and receive. The Alteon cards also need a certain amount of SRAM to run the firmware. > The Intel boards also don't have the features necessary to really support > zero copy TCP receive. > > The Alteon boards, on the other hand, have most of the features necessary, > and if I get some time, I may add the last feature (header splitting) to > the firmware. > > The other alternative is SysKonnect, and that might actually be a good > alternative. I haven't seen the boards, don't know how much they cost, > etc. etc. You might want to ask Bill Paul about them, he wrote the driver. The SysKonnect cards aren't bad. A single port multimode fiber card is around $700, I think. The single mode cards are more expensive. However SysKonnect also makes the only dual port gigabit card on the market right now (though it costs around $1100). The SysKonnect cards also average about 1MB of SRAM. The dual port card that I have has 1MB, which is I divide up between both ports. The SysKonnect boards have two main parts: a XaQti XMAC II gigabit MAC and a controller that interfaces the XMAC to the PCI bus called the GEnesis. (The GEnesis also provides buffering, arbitration and other things.) Dual port cards have one GEnesis and two XMACs, so they appear as a single PCI device to the host. The drivers that SysKonnect currently provides are set up to use the dual port cards in a failover configuration, meaning that you get only one virtual interface, and the driver monitors the link state and such and switches from one port to the other if one of the links goes down. The FreeBSD driver is different: each port on a dual port NIC is a separate interface and can operate independently of the other. So you can have sk0 and sk1 on the same card connected to two networks both running at the same time. Both the Alteon and SysKonnect NICs are 64-bit PCI cards. (Actually, I'm pretty sure all of the PCI gigabit NICs are 64-bit.) Both kinds of cards can do jumbograms on FreeBSD. Also, both vendors have released pretty good hardware documentation, which makes them good choices for custom applications, if you're into that sort of thing. -Bill -- ============================================================================= -Bill Paul (212) 854-6020 | System Manager, Master of Unix-Fu Work: wpaul@ctr.columbia.edu | Center for Telecommunications Research Home: wpaul@skynet.ctr.columbia.edu | Columbia University, New York City ============================================================================= "It is not I who am crazy; it is I who am mad!" - Ren Hoek, "Space Madness" ============================================================================= To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200002040229.VAA10541>