Date: Thu, 19 Sep 1996 11:00:22 -0700 (PDT) From: "Rodney W. Grimes" <rgrimes@GndRsh.aac.dev.com> To: jgreco@brasil.moneng.mei.com (Joe Greco) Cc: henrich@crh.cl.msu.edu, jgreco@brasil.moneng.mei.com, freebsd-isp@FreeBSD.ORG Subject: Re: News server... Message-ID: <199609191800.LAA07104@GndRsh.aac.dev.com> In-Reply-To: <199609191538.KAA10990@brasil.moneng.mei.com> from Joe Greco at "Sep 19, 96 10:38:43 am"
next in thread | previous in thread | raw e-mail | index | archive | help
> > Joe, what parameters do you give to newfs for your news spools? > > I used to use -b 4096 -f 512... the Usenet traditional... but I am > going to be doing some tests this week because I am seeing some unusual > bottlenecking in article throughput on some of the systems I maintain.. > > > > (And Im still skeptical on your stripe size of a CG, but im being convinced > > slowly :) The problem as I see it is that we have millions of 4k articles. > > Okay, so maybe a stripe size of 4k is poor because of directory look ups and > > what not, but why not a stripe size in the range of 128k or 256k, why jump all > > the way up to 32MB, seems that you will miss the "sweet" spot that striping can > > get you.. > > You can experiment if you want :-) Yes, 128k or 256k may be better (bear in > mind that many binaries articles are 1_MB_) and I have seen 1.5MB junk > directories.. so I would probably not select a number < 2MB. > > Obviously the sweet spot is probably going to be someplace between > 1MB and the size of the disk ;-) But I suspect it is somehow going to > be tied to how ffs works internally. > > I also do not have a GOOD set of tools with which to measure concurrency > within a news filesystem. I have some basic programs that I run several > of simultaneously, but how fair they are? Dunno. It is certainly a > problem that could use a researcher. All I am interested in is convincing > people that a stripe size of 8K is _foolish_. :-) Perhaps some one should dig up the 1987 UCB paper by Patterson, Gibson and Katz titled ``A Case for Redundant Arrays of Inexpensive Disks (RAID)'', I am sure they covered something about configuring the stripe size very large for high TPS rates, and very small for high bandwidth. Ah.. I did find the UCB report number: UCB/CSD 87/391. I did find this in some other sales lit I have: ``In I/O intensive environments, performance is optimized by striping the drives in the array with stripes large enough so that each record potentially falls entierely within on stripe.'' ... Unfortunately, small stripes rule out multiple overlapped I/O operations, since each I/O will typically involve all drives.'' I think in the usenet news engine case we can safely call the update record a ``cylinder group'', since any file add/remove/read is almost always going to fall within 1 cylinder group (except when UFS's optimizations fail.) -- Rod Grimes rgrimes@gndrsh.aac.dev.com Accurate Automation Company Reliable computers for FreeBSD
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199609191800.LAA07104>