Date: Fri, 19 Feb 1999 12:09:03 -0500 (EST) From: Simon Shapiro <shimon@simon-shapiro.org> To: Greg Lehey <grog@lemis.com> Cc: freebsd-isp@FreeBSD.ORG, Bryn Wm.Moslow <bryn@spacemonster.org> Subject: Re: DPT 3334UW RAID-5 Slowness / Weird FS problems Message-ID: <XFMail.990219120903.shimon@simon-shapiro.org> In-Reply-To: <19990219113330.C14890@lemis.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Greg Lehey, On 19-Feb-99 you wrote:
> On Wednesday, 17 February 1999 at 12:07:01 -0500, Shimon Shapiro wrote:
> >
> > Greg Lehey, On 16-Feb-99 you wrote:
> >> On Monday, 15 February 1999 at 13:08:22 -0800, Bryn Wm. Moslow wrote:
> >>> I recently installed a DPT 3334UW with 64MB cache in a mail server
> >>> running RAID-5 with a 32K stripe on an external case on which the
> >>> user
> >>> mail spool is mounted. The array is comprised of 6 Seagate Ultra Wide
> >>> 4.5GB SCA drives. The system is a P2 300 with 384MB and uses an
> >>> additional Seagate UW drive for boot, /usr, /var, swap, and staff
> >>> home
> >>> directories. It doesn't go into swap often but if it does it only
> >>> hits
> >>> about 5 to 10 MB. The system is running FreeBSD 2.2.8.
> >>
> >> I don't know the DPT controllers, but 32 kB stripes are far too
> >> small.
> >> For better performance, you should increase them to between 256 kB
> >> and
> >> 512 kB. Small stripe sizes create many more I/O requests at the
> >> drive
> >> level.
> >
> > It highly depends on the amount of cache memory on the card and the
> > nature
> > of the access. If I/O is sequential in nature, then Greg is correct.
> > If
> > it is highly fragmented and random (as in most DBMS), then Greg is only
> > correct if there is a lot of memory and access is to large blocks.
>
> As I said, FreeBSD I/O requests range in size between 512 bytes and 60
> kB. With a 16 kB stripe size, you would need to have an average block
> size of about 1 kB in order to avoid significant performance impact.
> DBMSs tend to read and write blocks of 16 kB or so.
Oracle default (was at one time) 8K, with large/busy systems set sometimes
to the max of 64KB. Never saw any benefit in much larger stripes.
Bus data times become excessively long. I have someplace a graph depicting
FreeBSD block size distribution. The pre-CAM driver collected such data.
I may put it back in.
> >>> If I format with the default settings to newfs, everything works
> >>> fine. The same hang also occurs if I try to do 16K block size. I
> >>> haven't tried anything bigger as I had an extremely tight window in
> >>> which to get the machine online.
> >>
> >> You should consider that once you have set the stripe size, you're
> >> stuck with it. Unless the DPTs have a good reason (like "not
> >> supported"), take a 256 kB stripe size.
> >
> > a. Newfs parameters have nothing to do with the SCSI HBA (DSPT or
> > otherwise. If 16k block size hangs, it is a bug elsewhere.
> > I studied such complaint long, long ago (on FreeBSD 3.0) and the
> > requests did not even hit the DPT.
> > But, like Greg said, this has almost no bearing on perfromance...
> >
> > b. The DPT firmware will take up to 1MB stripes. Beware there is no
> > more
> > than 64MB of cache on the card (if that much), and very large
> > stripes
> > may consume all the cache quickly and inefficiently.
>
> Do you have any information about how the controller uses the cache?
highly proprietary. You can control percentage of read-ahead, but not much
more.
> >From the design of Vinum, I would expect that these memory areas would
> not be cached. Vinum will never use more memory than is needed to
> satisfy the request, so this makes a maximum of 60 kB per request.
> Memory use for fragmented requests is higher, not lower.
I actually saw 64K blocks, but 60 is close enough :-)
>
> Greg
> --
> See complete headers for address, home page and phone numbers
> finger grog@lemis.com for PGP public key
Sincerely Yours, Shimon@Simon-Shapiro.ORG
770.265.7340
Simon Shapiro
Unwritten code has no bugs and executes at twice the speed of mouth
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-isp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?XFMail.990219120903.shimon>