Date: Wed, 15 Feb 1995 03:11:59 -0500 (EST) From: Rashid Karimov <rashid@rk.ios.com> To: taob@io.org (Brian Tao) Cc: lray@aurora.liunet.edu, freebsd-questions@freebsd.org, freebsd-isp@freebsd.org Subject: Re: Building a large FreeBSD news server Message-ID: <199502150811.DAA00277@rk.ios.com> In-Reply-To: <Pine.BSF.3.91.960215014407.24170d-100000@zip.io.org> from "Brian Tao" at Feb 15, 96 02:00:27 am
next in thread | previous in thread | raw e-mail | index | archive | help
Hi again :), > > > what is expirience with running FreeBSD PCs as INND > > servers with full news feed ? Is it capable of handling > > the load ? Any IO/CPU limitations ? > > I have the same questions, since I'm just starting out here. My > server will be going into full production mode within a few days (it's > been running with about a dozen beta testers for a week now). > > > I run P6-200 here as news server - runs just fine, load > > averages are ~0.3-0.5 , readers peak at ~200-300, total > > of ~23.000 newsgroups in active, 3 days expire average, > > 370.000 - 420.000 inodes used in /usr/spool/uucp. > > P133, 128 megabytes RAM, 3 NCR53c810 controllers and nine disks > to be split up thusly (I don't have all the controllers plugged in > yet): > At the moment, I have the news spool spread over two 2GB Quantum > Atlas drives and a 4GB Quantum Grand Prix, running off a single > Buslogics BT-946C controller. It receives about 120,000 articles a > day (900MB or so), with 20000 groups in the active file. Only about > 7000 of those actually have articles in them so far. ;-) > > I have the history database, the overview files and the > alt.binaries spool on a single disk on the current machine, and that > disk is getting hit pretty hard according to iostat. That's being > split across three spindles and three controllers in the production > machine, so disk I/O should not be a problem. > > A delayed expire run takes less than 5 minutes to generate the > list of articles to delete, which fastrm processes in about 30 > minutes. I expect that to rise once we get a hundred readers on it at > once, and more articles to expire. > > It seems INN is only spooling about 2 articles per second, peaking > at around 5/sec for very short periods of time. Perhaps this will > improve once I'm using multiple controllers and more disks? It's interesting thing , actually ... I was trying to get some estimates on articles/sec rate. Folx at n.s.n say that it should indeed peak at 5 arts/sec for a decent feed, so I decided to check what an ftp rate would be on a local 10Mb LAN. I've tried it on FreeBSD <> FreeBSD ( P166 <> P6-200) Sun20 <> FreeBSD Sun20 <> Sun10 LocalFreeBSD <> LocalFreeBSD with 100 small files(2K) with prompt off. Guess what ! It was steady (!!!) 100 files in 20 secs , or ~5 files/sec in ALL CASES. I've tried to do ftp to the same PC from 2 different machines at the same time. It took ~23 sec to upload 200 files (100 from each PC). So it looks like for small articles/multiple feeds (and fast links :) the upper limit could be bigger than 5art/sec ( putting aside the difference " ftp versus nntp"). Other thing is that on th eame link it will take ~0.25 sec to transfer single 200K file , and even less if the thing is compressed , so batched newsfeed looks like reasonable alternative, especially for a stub news server, when one knows for sure that there won't be no "blanks" in the batch - I mean the articles the site already has. With multiple feeds the effective loss of the BW will be much bigger , especially if the server ( say, small ISP) will provide transit for the newsfeed from one major peer to another. Probably it has sense to inject only articles which were originated locally , but it's often not the point when all peers are equally interested in getting full newsfeed from each other trying to get news articles as fast as possible. Rashid
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199502150811.DAA00277>