Date: Tue, 11 Mar 97 9:50:35 CST From: Joe Greco <jgreco@solaria.sol.net> To: mmead@goof.com (matthew c. mead) Cc: msmith@atrad.adelaide.edu.au, julian@whistle.com, tom@sdf.com, isp@freebsd.org, hackers@freebsd.org Subject: Re: freebsd as a news server? Message-ID: <199703111550.JAA27921@solaria.sol.net> In-Reply-To: <199703090439.XAA12069@goof.com> from "matthew c. mead" at Mar 8, 97 11:39:17 pm
next in thread | previous in thread | raw e-mail | index | archive | help
> > No async support in 2.1.6, but I believe that noatime is supported. > > Hmm. No wonder I don't see any improvement in 2.1.6 > mounting async and building the kernel. > > > You should _really_ go search the archives of the -hackers list for > > postings from Joe Greco regarding big news servers. In particular, > > you are using (probably) slow disks, your layout is likely to be > > wrong, and you don't have enough SCSI busses 8) > > Hmm, the disks are far from slow. They're 9G micropolis s/far from slow/slow as hell for news/ > SCSI-II fast, "ultra wide" disks. There is, however, one bus. > There's only 5 drives on it. I'm interested in figuring out > what's wrong with the layout. I've got the striping factor set > to 255 blocks (per a suggestion in the docs for ccd). I'll try > to have a looksee at the archives of -hackers. Would be nice to > be able to get mailbox file format archives, though. :-) Thanks > for your reply! Don't feel bad, this is one of many hard-learned news lessons. Usenet news deals in hundreds of thousands of small bits of information that need to get on your system in a time dependent fashio on a daily basis. Your drives are only so fast, and there is a point at which the drive is practically saturated with accesses (even if the drive light is not on solid, you have passed a point of "reasonable response time"). Once you pass this point, your news server suffers. As you stuff more articles on a drive, the implicit expectation is that you will be able to perform more accesses on that drive in the same period of time. This is false: see above. In general, today's drives are capable of storing LOTS of data but the access times are abysmal. That means that you can not plan on taking full advantage of a 9GB drive's capacity... you can only practically store a few GB on it before the drive becomes saturated. Based on real world implementations, a 4GB drive "might" be OK for smaller installations (several dozen readers), but 2GB drives are mandatory for real installations. The exception is alt.binaries, where the number of accesses you will be expected to perform is much lower in comparison to the average article size. I use 2 x 2GB drives striped for /news, plus 2 x 2GB drives for /news/alt, plus 2 x 9GB drives for /news/alt/binaries. I also have 2 x 1GB drives for /nov (overviews) plus 2 x 1GB drives for the newslib. This is a good recipe for a news server's spool. Use fast drives - you don't need ultra-wide or any other stuff, standard fast SCSI works great. You are not moving big amounts of data around, after all. If you ask for a 4K chunk of data from a disk, 98% of the time you spend waiting for it will be for the heads to seek to the data, so if it takes a teeny bit longer to fetch the data over the bus, who cares? Use good fast drives: the new Seagate Hawk-II's are fab, the Barracudas are pretty good too. Get the ST-31055N's for 1GB drives, 32155N's, 32171N's, or 32155N's for 2GB drives. As noted, I maintain three spool filesystems on most news spools, I stripe a pair of disks together with a stripe size equal to the size of a cylinder group: there is some theory involved there but it works. My stripe size here is 65536 - except for newslib, where you do want a smaller stripe size (I use 128). Final note: You don't want to make one big filesystem. That's a mistake. ... JG
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199703111550.JAA27921>