From owner-freebsd-isp Fri Feb 19 8: 1:41 1999 Delivered-To: freebsd-isp@freebsd.org Received: from nomis.simon-shapiro.org (nomis.simon-shapiro.org [209.86.126.163]) by hub.freebsd.org (Postfix) with SMTP id B6EFD116C9 for ; Fri, 19 Feb 1999 08:01:37 -0800 (PST) (envelope-from shimon@simon-shapiro.org) Received: (qmail 56915 invoked by uid 1000); 19 Feb 1999 17:09:03 -0000 Message-ID: X-Mailer: XFMail 1.3 [p0] on FreeBSD X-Priority: 3 (Normal) Content-Type: text/plain; charset=iso-8859-8 Content-Transfer-Encoding: 8bit MIME-Version: 1.0 In-Reply-To: <19990219113330.C14890@lemis.com> Date: Fri, 19 Feb 1999 12:09:03 -0500 (EST) X-Face: (&r=uR0&yvh>h^ZL4"-TH61PD}/|Y'~58Z# Gz&BK'&uLAf:2wLb~L7YcWfau{;N(#LR2)\i.l8'ZqVhv~$rNx$]Om6Sv36S'\~5m/U'"i/L)&t$R0&?,)tm0l5xZ!\hZU^yMyCdt!KTcQ376cCkQ^Q_n.GH;Dd-q+ O51^+.K-1Kq?WsP9;cw-Ki+b.iY-5@3!YB5{I$h;E][Xlg*sPO61^5=:5k)JdGet,M|$"lq!1!j_>? $0Yc? Reply-To: shimon@simon-shapiro.org Organization: The Simon Shapiro Foundation From: Simon Shapiro To: Greg Lehey Subject: Re: DPT 3334UW RAID-5 Slowness / Weird FS problems Cc: freebsd-isp@FreeBSD.ORG, Bryn Wm.Moslow Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Greg Lehey, On 19-Feb-99 you wrote: > On Wednesday, 17 February 1999 at 12:07:01 -0500, Shimon Shapiro wrote: > > > > Greg Lehey, On 16-Feb-99 you wrote: > >> On Monday, 15 February 1999 at 13:08:22 -0800, Bryn Wm. Moslow wrote: > >>> I recently installed a DPT 3334UW with 64MB cache in a mail server > >>> running RAID-5 with a 32K stripe on an external case on which the > >>> user > >>> mail spool is mounted. The array is comprised of 6 Seagate Ultra Wide > >>> 4.5GB SCA drives. The system is a P2 300 with 384MB and uses an > >>> additional Seagate UW drive for boot, /usr, /var, swap, and staff > >>> home > >>> directories. It doesn't go into swap often but if it does it only > >>> hits > >>> about 5 to 10 MB. The system is running FreeBSD 2.2.8. > >> > >> I don't know the DPT controllers, but 32 kB stripes are far too > >> small. > >> For better performance, you should increase them to between 256 kB > >> and > >> 512 kB. Small stripe sizes create many more I/O requests at the > >> drive > >> level. > > > > It highly depends on the amount of cache memory on the card and the > > nature > > of the access. If I/O is sequential in nature, then Greg is correct. > > If > > it is highly fragmented and random (as in most DBMS), then Greg is only > > correct if there is a lot of memory and access is to large blocks. > > As I said, FreeBSD I/O requests range in size between 512 bytes and 60 > kB. With a 16 kB stripe size, you would need to have an average block > size of about 1 kB in order to avoid significant performance impact. > DBMSs tend to read and write blocks of 16 kB or so. Oracle default (was at one time) 8K, with large/busy systems set sometimes to the max of 64KB. Never saw any benefit in much larger stripes. Bus data times become excessively long. I have someplace a graph depicting FreeBSD block size distribution. The pre-CAM driver collected such data. I may put it back in. > >>> If I format with the default settings to newfs, everything works > >>> fine. The same hang also occurs if I try to do 16K block size. I > >>> haven't tried anything bigger as I had an extremely tight window in > >>> which to get the machine online. > >> > >> You should consider that once you have set the stripe size, you're > >> stuck with it. Unless the DPTs have a good reason (like "not > >> supported"), take a 256 kB stripe size. > > > > a. Newfs parameters have nothing to do with the SCSI HBA (DSPT or > > otherwise. If 16k block size hangs, it is a bug elsewhere. > > I studied such complaint long, long ago (on FreeBSD 3.0) and the > > requests did not even hit the DPT. > > But, like Greg said, this has almost no bearing on perfromance... > > > > b. The DPT firmware will take up to 1MB stripes. Beware there is no > > more > > than 64MB of cache on the card (if that much), and very large > > stripes > > may consume all the cache quickly and inefficiently. > > Do you have any information about how the controller uses the cache? highly proprietary. You can control percentage of read-ahead, but not much more. > >From the design of Vinum, I would expect that these memory areas would > not be cached. Vinum will never use more memory than is needed to > satisfy the request, so this makes a maximum of 60 kB per request. > Memory use for fragmented requests is higher, not lower. I actually saw 64K blocks, but 60 is close enough :-) > > Greg > -- > See complete headers for address, home page and phone numbers > finger grog@lemis.com for PGP public key Sincerely Yours, Shimon@Simon-Shapiro.ORG 770.265.7340 Simon Shapiro Unwritten code has no bugs and executes at twice the speed of mouth To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message