From owner-freebsd-hackers Wed Mar 4 07:13:26 1998 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id HAA17645 for freebsd-hackers-outgoing; Wed, 4 Mar 1998 07:13:26 -0800 (PST) (envelope-from owner-freebsd-hackers@FreeBSD.ORG) Received: from Kitten.mcs.com (Kitten.mcs.com [192.160.127.90]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id HAA17571 for ; Wed, 4 Mar 1998 07:13:00 -0800 (PST) (envelope-from karl@Mars.mcs.net) Received: from Mars.mcs.net (karl@Mars.mcs.net [192.160.127.85]) by Kitten.mcs.com (8.8.7/8.8.2) with ESMTP id IAA16855; Wed, 4 Mar 1998 08:15:24 -0600 (CST) Received: (from karl@localhost) by Mars.mcs.net (8.8.7/8.8.2) id IAA13916; Wed, 4 Mar 1998 08:15:23 -0600 (CST) Message-ID: <19980304081523.61560@mcs.net> Date: Wed, 4 Mar 1998 08:15:23 -0600 From: Karl Denninger To: shimon@simon-shapiro.org Cc: Wilko Bulte , sbabkin@dcn.att.com, tlambert@primenet.com, jdn@acp.qiv.com, blkirk@float.eli.net, hackers@FreeBSD.ORG, grog@lemis.com Subject: Re: SCSI Bus redundancy... References: <19980303232444.59397@mcs.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.84 In-Reply-To: ; from Simon Shapiro on Tue, Mar 03, 1998 at 10:23:32PM -0800 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, Mar 03, 1998 at 10:23:32PM -0800, Simon Shapiro wrote: > > > RAID 5, in particular, benefits enormously from writeback, as it allows > > it to defer writes until an entire stripe is ready, which means no > > read/compute/write cycle. This is a monstrous win for performance. > > I played, on the DPT with RAID{0,1,5} stripe size vs. perfromance. The > numbers really move around. I used to even know how to compute this > stuff... :-) > > The best I've seen off our RAID systems right now is about 11MB/sec > > (that's > > megaBYTES, not bits). That's on an Ultra bus, with 2 ultra busses going > > to > > the RAID disks. > > About right. SCSI-II used to be 4-5 MB/bus. Ultra-wide is about 5-6, for > small O/S-type blocks. I see about 18 MB/Sec on the DPT on three busses. > The difficulty is in having FreeBSD capable of producing this traffic on > small blocks (dd if=/dev/zero of=/dev/something bs=64k is NOT typical > application). Hmmm.... Well, I made some adjustments to the queueing algorythm in the controller this morning, and guess what - I now get ~17MB/Sec on two SCSI busses in RAID 0+1 mode. Now *that's* not bad. In RAID 5 mode I'm getting ~10MB/sec still, and I think I'm hitting the wall now on the disk I/O (since RAID 5 doesn't stripe data) rather than on the interface! Curiously enough, turning read-ahead in the controller on actually slows it *down* a bit. Not much, but a little bit. > Hook up a DPT to one of these boxes. Will be interesting to see what > happens. Seriously. I'll have to give that a shot. > > I could run two host channels on this thing across two RAID sets into two > > Adaptec adapters. That might be a big win. > > > > I suspect the bottleneck is in the AIC code at this point, or the bus > > itself, or the interrupt latency on the DMA completion is killing me. > > There is no appreciable difference between running at 40MB/sec (ultra > > full-bore) and 20MB/sec, indicating that perhaps the hold-up is in the > > Adaptec microcode, driver, and/or the Adaptec/PCI bus interface. > > I read the AIC code quite carefully when writing the DPT code. There is > nothing obviously wrong with it. Justin is a very careful engineer. > It is either the sequencer itself, or the SCSI layer, or FreeBSD. To get a > DPT to saturate, you need about (with 4KB transfers, random across the > entire array), 1,900 transactions per second. To reach 1,740 or so, from > userspace, I have to run about 256 copies of st.c. The LA is about 220 at > this point. Not very good for interactive work. > > Trying the same with SMP either crashes, or goes down to about 880 TPS. > > Simon -- -- Karl Denninger (karl@MCS.Net)| MCSNet - Serving Chicagoland and Wisconsin http://www.mcs.net/ | T1's from $600 monthly to FULL DS-3 Service | NEW! K56Flex support on ALL modems Voice: [+1 312 803-MCS1 x219]| EXCLUSIVE NEW FEATURE ON ALL PERSONAL ACCOUNTS Fax: [+1 312 803-4929] | *SPAMBLOCK* Technology now included at no cost To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message