Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 4 Mar 1998 08:15:23 -0600
From:      Karl Denninger  <karl@mcs.net>
To:        shimon@simon-shapiro.org
Cc:        Wilko Bulte <wilko@yedi.iaf.nl>, sbabkin@dcn.att.com, tlambert@primenet.com, jdn@acp.qiv.com, blkirk@float.eli.net, hackers@FreeBSD.ORG, grog@lemis.com
Subject:   Re: SCSI Bus redundancy...
Message-ID:  <19980304081523.61560@mcs.net>
In-Reply-To: <XFMail.980303222332.shimon@simon-shapiro.org>; from Simon Shapiro on Tue, Mar 03, 1998 at 10:23:32PM -0800
References:  <19980303232444.59397@mcs.net> <XFMail.980303222332.shimon@simon-shapiro.org>

next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Mar 03, 1998 at 10:23:32PM -0800, Simon Shapiro wrote:
> 
> > RAID 5, in particular, benefits enormously from writeback, as it allows 
> > it to defer writes until an entire stripe is ready, which means no 
> > read/compute/write cycle.  This is a monstrous win for performance.
> 
> I played, on the DPT with RAID{0,1,5} stripe size vs. perfromance.  The
> numbers really move around.  I used to even know how to compute this
> stuff...

:-)

> > The best I've seen off our RAID systems right now is about 11MB/sec
> > (that's
> > megaBYTES, not bits).  That's on an Ultra bus, with 2 ultra busses going
> > to
> > the RAID disks.
> 
> About right.  SCSI-II used to be 4-5 MB/bus.  Ultra-wide is about 5-6, for
> small O/S-type blocks.  I see about 18 MB/Sec on the DPT on three busses.
> The difficulty is in having FreeBSD capable of producing this traffic on
> small blocks (dd if=/dev/zero of=/dev/something bs=64k is NOT typical
> application).

Hmmm.... Well, I made some adjustments to the queueing algorythm in the
controller this morning, and guess what - I now get ~17MB/Sec on two SCSI
busses in RAID 0+1 mode.  Now *that's* not bad.  In RAID 5 mode I'm getting
~10MB/sec still, and I think I'm hitting the wall now on the disk I/O (since
RAID 5 doesn't stripe data) rather than on the interface!  

Curiously enough, turning read-ahead in the controller on actually slows 
it *down* a bit.  Not much, but a little bit.

> Hook up a DPT to one of these boxes.  Will be interesting to see what
> happens. Seriously.

I'll have to give that a shot.

> > I could run two host channels on this thing across two RAID sets into two
> > Adaptec adapters.  That might be a big win.
> > 
> > I suspect the bottleneck is in the AIC code at this point, or the bus 
> > itself, or the interrupt latency on the DMA completion is killing me.  
> > There is no appreciable difference between running at 40MB/sec (ultra 
> > full-bore) and 20MB/sec, indicating that perhaps the hold-up is in the 
> > Adaptec microcode, driver, and/or the Adaptec/PCI bus interface.
> 
> I read the AIC code quite carefully when writing the DPT code.  There is
> nothing obviously wrong with it.  Justin is a very careful engineer.
> It is either the sequencer itself, or the SCSI layer, or FreeBSD.  To get a
> DPT to saturate, you need about (with 4KB transfers, random across the
> entire array), 1,900 transactions per second.  To reach 1,740 or so, from
> userspace, I have to run about 256 copies of st.c.  The LA is about 220 at
> this point.  Not very good for interactive work.
> 
> Trying the same with SMP either crashes, or goes down to about 880 TPS.
> 
> Simon

--
-- 
Karl Denninger (karl@MCS.Net)| MCSNet - Serving Chicagoland and Wisconsin
http://www.mcs.net/          | T1's from $600 monthly to FULL DS-3 Service
			     | NEW! K56Flex support on ALL modems
Voice: [+1 312 803-MCS1 x219]| EXCLUSIVE NEW FEATURE ON ALL PERSONAL ACCOUNTS
Fax:   [+1 312 803-4929]     | *SPAMBLOCK* Technology now included at no cost

To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?19980304081523.61560>