Date: Tue, 17 Jun 1997 15:36:08 -0700 (PDT) From: Simon Shapiro <Shimon@i-Connect.Net> To: FreeBSd-SCSI@FreeBSD.org Subject: RAID Configuration Notes Message-ID: <XFMail.970617153608.Shimon@i-Connect.Net>
next in thread | raw e-mail | index | archive | help
Hi Y'all
I am getting several mail messages in regards to the DPT driver and RAID
configuration every day. Makes me happy and proud :-) If only I had a
penny per request :-))
Anyway, I thought, at the risk of offending some of you with the trivial,
to provide some notes on RAID configuration. Especially when it relates to
DPT controllers.
Bandwidth:
* WRITEs are always slower than READS. The ``how much'' depends
on the configuration. Details to follow.
* A single SCSI-{II,III) disk can perform about 140 disk I/Os per second.
This statememt is true for block_size < 8K. (Almost) Regardless of
narrow/wide, ultra/fast, etc. The reason being that, according to SCSI
specifications, all negotiations and handshakes happen in narrow, async
5MHz. Otherwise slow/old devices will surely hang the bus.
* A ribbon-cable SCSI bus (parallel, not FCAL) can support up to 440 or so
Tx/Sec. Yes, this means that for very high activity, much more than 4
drives per bus is a waste.
* From my observations, FreeBSD does ALL block device I/O (including all
filesystem operations) in 4K increments. Regardless of your specified
``block size''. Raw device (character) can be no larger than 64K.
* RAID-0 gives you the best performance. The risk is that ANY device
failure will take with it the entire array (data/functionality wise).
* The MTBS of ANY RAID array is the MTBF of the worst device divided by
the number of devices.
* The declared MTBF for most modern 3.5" disk drives is around 800,000
hours. If you belive that, I own a bridge in London you should invest
in.
* RAID-1 gives OK READ performance and write performance similar to a
single disk (DPT only, In-kernel RAID-1 writes at 0.5 the speed of a
single-disk). The drawback is a mximum size of disk * 2 and 50% space
utilization. Perfromance in degraded mode (a failed drive) is similar to
a single disk (plus lots of noise (DPT)).
* RAID-5 has capacity utilization of disk * (no_of_disks - 1). READ is as
fast as a RAID-0 (in optimal state) and WRITE is slowest of the bunch.
WRITE performance is about 10MB/Sec on the newest firmware (vs 15-20 for
RAID-0.
Performance:
* With software interrupts disabled, 256 parallel dd reading and writing,
with all error checking enabled we see:
RAID | READ | WRITE
------+------+------
none | 3-4 | 3-4
0 | 18-20 | 15-18
1 | 10-15 | 8-12
5 | 15-20 | 5-10
RAID-0 is two drives, RAID-5 is 5 drives, across 2 busses (see below).
Configuration:
* The DPT defaults for cache copnfiguration are fine for general
filesystem operation and quite useless for database work. They are
easily tunable. They are useless because they allocate 30% of the cache
to read-ahead buffers and use 128K stripes.
* You can tune the cache as well as the cache that disk drives have
on-board.
* When you configure RAID arrays across busses on a DPT, stripe the array
across.
Example: Three busses, bus 0 has targets 1, 2, & 3, bus 1 has 4, 5, & 6
and bus 2 has 8, 9, & 10. When configuring a 5 wide RAID-5 array select
the devices, in dptmgr) in this order: 0-1, 1-4, 2-8, 0-2, 1-5, and hot
spare on 2-9. The next array will use 0-3, 1-6, and 2-10.
This will force the DPT to ``jump'' busses when moving from stipe to
stripe and result in HUGE boost in perfromance.
* Hot spares apply to the entire controller, not just a particuloar array.
In the above example, if you defined a RAID-0 array (which includes 0-3,
1-6, and 2-10), you (can define but) do NOT need another hyot spare.
2-9 will do for either.
* To have RAID arrays span controllers and/or have the ability to exapand
an existing array, you will have to either wait or integrate these
changes yourself (messy).
I hope this provides some background and helps.
Simon
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?XFMail.970617153608.Shimon>