Date: Wed, 4 Mar 1998 08:15:23 -0600 From: Karl Denninger <karl@mcs.net> To: shimon@simon-shapiro.org Cc: Wilko Bulte <wilko@yedi.iaf.nl>, sbabkin@dcn.att.com, tlambert@primenet.com, jdn@acp.qiv.com, blkirk@float.eli.net, hackers@FreeBSD.ORG, grog@lemis.com Subject: Re: SCSI Bus redundancy... Message-ID: <19980304081523.61560@mcs.net> In-Reply-To: <XFMail.980303222332.shimon@simon-shapiro.org>; from Simon Shapiro on Tue, Mar 03, 1998 at 10:23:32PM -0800 References: <19980303232444.59397@mcs.net> <XFMail.980303222332.shimon@simon-shapiro.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, Mar 03, 1998 at 10:23:32PM -0800, Simon Shapiro wrote:
>
> > RAID 5, in particular, benefits enormously from writeback, as it allows
> > it to defer writes until an entire stripe is ready, which means no
> > read/compute/write cycle. This is a monstrous win for performance.
>
> I played, on the DPT with RAID{0,1,5} stripe size vs. perfromance. The
> numbers really move around. I used to even know how to compute this
> stuff...
:-)
> > The best I've seen off our RAID systems right now is about 11MB/sec
> > (that's
> > megaBYTES, not bits). That's on an Ultra bus, with 2 ultra busses going
> > to
> > the RAID disks.
>
> About right. SCSI-II used to be 4-5 MB/bus. Ultra-wide is about 5-6, for
> small O/S-type blocks. I see about 18 MB/Sec on the DPT on three busses.
> The difficulty is in having FreeBSD capable of producing this traffic on
> small blocks (dd if=/dev/zero of=/dev/something bs=64k is NOT typical
> application).
Hmmm.... Well, I made some adjustments to the queueing algorythm in the
controller this morning, and guess what - I now get ~17MB/Sec on two SCSI
busses in RAID 0+1 mode. Now *that's* not bad. In RAID 5 mode I'm getting
~10MB/sec still, and I think I'm hitting the wall now on the disk I/O (since
RAID 5 doesn't stripe data) rather than on the interface!
Curiously enough, turning read-ahead in the controller on actually slows
it *down* a bit. Not much, but a little bit.
> Hook up a DPT to one of these boxes. Will be interesting to see what
> happens. Seriously.
I'll have to give that a shot.
> > I could run two host channels on this thing across two RAID sets into two
> > Adaptec adapters. That might be a big win.
> >
> > I suspect the bottleneck is in the AIC code at this point, or the bus
> > itself, or the interrupt latency on the DMA completion is killing me.
> > There is no appreciable difference between running at 40MB/sec (ultra
> > full-bore) and 20MB/sec, indicating that perhaps the hold-up is in the
> > Adaptec microcode, driver, and/or the Adaptec/PCI bus interface.
>
> I read the AIC code quite carefully when writing the DPT code. There is
> nothing obviously wrong with it. Justin is a very careful engineer.
> It is either the sequencer itself, or the SCSI layer, or FreeBSD. To get a
> DPT to saturate, you need about (with 4KB transfers, random across the
> entire array), 1,900 transactions per second. To reach 1,740 or so, from
> userspace, I have to run about 256 copies of st.c. The LA is about 220 at
> this point. Not very good for interactive work.
>
> Trying the same with SMP either crashes, or goes down to about 880 TPS.
>
> Simon
--
--
Karl Denninger (karl@MCS.Net)| MCSNet - Serving Chicagoland and Wisconsin
http://www.mcs.net/ | T1's from $600 monthly to FULL DS-3 Service
| NEW! K56Flex support on ALL modems
Voice: [+1 312 803-MCS1 x219]| EXCLUSIVE NEW FEATURE ON ALL PERSONAL ACCOUNTS
Fax: [+1 312 803-4929] | *SPAMBLOCK* Technology now included at no cost
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?19980304081523.61560>