Date: Mon, 12 Jan 1998 02:13:34 -0600 From: "Jeffrey J. Mountin" <mountin.man@mixcom.com> To: Lukas Wunner <lukas@design.de>, =?iso-8859-1?Q?=22Luis_E=2E_Mu=F1oz=22?= <lem@cantv.net> Cc: freebsd-isp@FreeBSD.ORG Subject: Re: [fbsd-isp] Designing for a very large ISP Message-ID: <3.0.3.32.19980112021334.006fe3dc@mixcom.com> In-Reply-To: <19980104141146.32430@reactor> References: <3.0.5.32.19980103121611.007af8f0@pop.can tv.net> <3.0.5.32.19980103121611.007af8f0@pop.cantv.net>
next in thread | previous in thread | raw e-mail | index | archive | help
At 02:11 PM 1/4/98 +0100, Lukas Wunner wrote: >> During the first year, it'll have to support some 100000 users. >> Of course, we've already chosen FreeBSD as the core OS for this >> after looking to Digital Unix, Solaris and Windows NT. > >My suggestion would be to look at an SGI Origin 200/2000 or Sun U2 box >instead of a PC box running FreeBSD. We have experienced severe problems >with PCs wrt expandability/scalability. E.g., we currently have 256MB of >RAM in our news box but would love to go to 512MB, but the motherboard >is not capable of doing that although the documentation and the webpages >state the contrary (it's a Tyan Tomcat III or IV). If you want to go to >something like 1GB or 2GB of RAM, you're stuck with a PC. The only >solution seem to be PPro based machines, but the chipsets available so >far are really ugly wrt memory and PCI performance in my opinion (as >compared to good old Pentium based boards). The only chipset which seems >to be able to support lots of RAM and more than 4 PCI cards seems to be Quite a few PPro boards have 5 PCI slots and Tyan's S1571 is one Pentium board with 5 PCI slots. Many of the replies to this thread mention memory capacity and depending on what you want to use either 384, 512, or 1024 Mb can be used. Not sure why this is a factor, since in dealing PC based ISP's load sharing and distribution of services is why it works and is cost effective *and* scalable. One factor I see with PCs is mb design with limits on the number of PCI slots and max memory, but then one can use a lower performance CPU and spread the load/services around. Pentium boards do seem to be (by design) intended for home/desktop workstations or low range servers, but work just fine with FBSD. Take VIA's new VP3, it can address a 2Mb L2 cache, but the largest I know of is with FIC at 1 Mb @6ns. It can also address up to 2 Gb of memory and can cache up to 1 Gb, but is there a board that does this? The PPro cache up to 4 Gb of memory. The limits of Intel's VX and TX are ridiculous at 64 Mb, especially when they can take up to 384/512 Mb total. Personally I'd love to see a board that can take more than 384 Mb of SDRAM, but would be willing to accept this limit since more servers can be deployed. FWIW, comparing a P100 with 64 Mb of SDRAM and a P166 with 64 Mb of FPM (didn't try it with EDO and see no reason why not to use SDRAM) show the P100 faster with memory intensive operations. Both are almost identical TX chipset boards. I've know a few people that have run news on Sun's and find FBSD to be better, so you might wish to try an FIC 2007 with a 1 Mb L2 cache that will take 512 Mb before spending big bucks. And I am aware of the architectural differences between PC systems with Sun and SGI, but see PCs as cost effective. Since you have a Tyan, what about the S1668 Titan Pro for the PPro with 5 PCI slots and up to 1 Gb of memory? For my use I am thinking of the FIC, either the 2007 or 2012 and possibly the Tyan. Or both. All 3? ;) I'm not just answering your post but trying to cover a few of the others that I have seen here as well. :) >the Orion GX, one implementation being the AMI Goliath board (cf. www.ami.com). >However, I have not been able so far to get my hands on one of these >boards *without* buying a large expensive box from Compaq et al (it seems >AMI only sells these boards to OEMs, at least I have not found a >distributor here in Germany, if you know of one, please drop me a line). >The Goliath board also requires special (read: expensive) DIMMs, you >can not use standard PS/2s or SDRAMs. Expensive? Compare the price of a 64 Mb EDO SIMM, 64 Mb DIMM, 64 Mb SDRAM DIMM, the latter with SPD, and 64 Mb EDO ECC today. Even though SDRAM w/SPD is very recent they are comparable to plain SDRAM, and slightly more than EDO SIMMs and cheaper than EDO w/ECC, but EDO/FPM DIMMs fall torwards the lower end. Are you talking propriatary? Not fond of any system/mb that has a propriatary configuration. >Experience also has shown that usually you have to invest more time >into getting a PC based server running as compared to a machine which >was designed from scratch to be a high performance/bandwidth server >like the SGI Origin 2000, and time is money (I have spent *several* hours >trying to get the Tomcat board running with more than 256MB of RAM). Pardon my ignorance, but I've always thought of SGI boxes as grahpics designers dreams, not a server for an ISP. I've dealt with them on occasion (regretfully) and know a grahpics designer that now has his own SGI to use and couldn't be happier, but I'd pass. And depending on the services/security it takes me anywhere from an hour to several days (gotta sleep) getting a system running. >Note: this is no FreeBSD-bashing. I'd always prefer a FreeBSD-based >machine to an SGI Origin 2000 if it wasn't that difficult to find >PC hardware which suits our needs wrt scalability/performance. >Any recommendations for high-end PC hardware are welcome. Maybe there is and I made a few, but then even if I had unlimited funds I see no reason to spend more than necessary to get the job done. If the board can handle 128 Mb SIMMs, the Titan pro must since 8 * 128 = 1 Gb and either EDO or EDO/ECC are comparable in price. I worked with a Dell PPro server that had DIMM slots and built in SCSI that was blazingly fast @180MHz w/128Mb ECC compared to a Asus PPro @200Mhz w/256Mb EDO and 2940UW (not mine, but built to my specs). Must say the Dell was a well built system and had only slighty faster drives. The Dell cost about 40% more though. Both ran NT, not my choice. <g> The news server I set up had an Asus T2P4, P120, 128 Mb EDO, 2940UW, Intel EtherXpress Pro100B, and only 2 drives. It would take a full feed, but was a bit slow reading. What was unique is that it ran at 125Mhz, 1.5 multiplier with 83MHz bus, and was able to do some serious disk IO. Also very stable, as much as any news server. Doubling the memory and adding more drives would have helped the reading issue, but news was outsourced. You can still get the T2P4, but it seems to be superceeded by the the TXP4. Plenty of alternatives to the SGI. Jeff Mountin - Unix Systems TCP/IP networking mountin.man@mixcom.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3.0.3.32.19980112021334.006fe3dc>