Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 10 Nov 1996 17:32:24 +0000 ()
From:      Brian Tao <taob@io.org>
To:        FREEBSD-SCSI-L <freebsd-scsi@freebsd.org>
Subject:   CCD vs. hardware RAIDs (more numbers)
Message-ID:  <Pine.BSF.3.95.961110163640.10143B-100000@cabal.io.org>

next in thread | raw e-mail | index | archive | help
    After much delay with vanishing U.S. distributors and customs
hassles, I now have three rackmount enclosures, each with seven 2GB
drives (eventually becoming part of a news server cluster).

    ccd in 2.2-961014-SNAP was very easy to setup, once I realized a
drive in the array somehow had its filesystem set to "SystemV" type
and not "4.2BSD".  To continue some of the benchmarks I performed with
the CMD and Streamlogic RAIDION RAIDs, I striped three of the 2GB
drives together.

    This probably isn't a fair test, but I think it's in the right
ballpark.  The test server is the same PPro200, but now upgraded to
the Oct 14 snapshot.  I used Quantum 2GB wide drives, mounted in a
DAE-800 rackmount enclosure from Connector Resources Unlimited
(http://www.cruinc.com/):

FreeBSD 2.2-961014-SNAP #0: Sun Nov 10 02:06:41 EST 1996
    root@news-in.ican.net:/mnt/sys/compile/NEWS
Calibrating clock(s) relative to mc146818A clock...
i586 clock: 199305096 Hz, i8254 clock: 1193158 Hz
CPU: Pentium Pro (199.31-MHz 686-class CPU)
  Origin = "GenuineIntel"  Id = 0x617  Stepping=7
  Features=0xf9ff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,<b11>,MTRR,PGE,MCA,CMOV>
real memory  = 134217728 (131072K bytes)
avail memory = 129789952 (126748K bytes)
Probing for devices on PCI bus 0:
chip0 <Intel 82440FX (Natoma) PCI and memory controller> rev 2 on pci0:0
chip1 <Intel 82371SB PCI-ISA bridge> rev 1 on pci0:1:0
chip2 <Intel 82371SB IDE interface> rev 0 on pci0:1:1
de0 <Digital DC21140 Fast Ethernet> rev 18 int a irq 10 on pci0:10
de0: SMC 9332 DC21140 [10-100Mb/s] pass 1.2
de0: address 00:00:c0:6b:eb:e6
de0: enabling 10baseT port
ahc0 <Adaptec 2940 Ultra SCSI host adapter> rev 0 int a irq 11 on pci0:11
ahc0: aic7880 Wide Channel, SCSI Id=15, 16 SCBs
ahc0 waiting for scsi devices to settle
(ahc0:0:0): "Quantum XP32150W L912" type 0 fixed SCSI 2
sd0(ahc0:0:0): Direct-Access 2151MB (4406960 512 byte sectors)
sd0(ahc0:0:0): with 3907 cyls, 10 heads, and an average 112 sectors/track
(ahc0:1:0): "Quantum XP32150W 81HB" type 0 fixed SCSI 2
sd1(ahc0:1:0): Direct-Access 2050MB (4199760 512 byte sectors)
sd1(ahc0:1:0): with 3907 cyls, 10 heads, and an average 107 sectors/track
(ahc0:2:0): "Quantum XP32150W L912" type 0 fixed SCSI 2
sd2(ahc0:2:0): Direct-Access 2151MB (4406960 512 byte sectors)
sd2(ahc0:2:0): with 3907 cyls, 10 heads, and an average 112 sectors/track
[...]


    Stripe size used was 64K, to match those on the hardware RAID's.
The CMD-5500 was the faster of the two hardware RAIDs, clocking in at
14MB/s writing and 11MB/s reading to a 256MB file on a RAID 0
filesystem.  The ccd filesystem hit 11MB/s writing and almost 21MB/s
reading a 256MB file.  This looked like there was a cache effect (F/W
SCSI-2 peaks at 20MB/s, does it not?).  I repeated the iozone with
512MB;  this time it reported about 11.5MB/s writing and 17.5MB/sec
reading.

    Bonnie was also quite a bit faster on the throughput tests, but a
little slower with the random seeks.  Increasing the stripe set to
five drives improved the stats (results are included at the end of
this message).

    Mounted async, the CCD filesystem is *fast*... it outperforms the
CMD-5500 (with 64MB of write-back cache) on the 10000-file
touch/touch/unlink test.  The 15556-file FreeBSD source tree can be
untarred onto it from another drive in 62 seconds flat.  :)

    So for about US$5200 (rack enclosure, seven 2GB fast/wide drives),
you get something that can be just as fast or faster than a hardware
RAID, at about one-third the $/GB cost (less if you don't go for the
fancy enclosure).  Of course, you don't get the same amount of
reliability or protection against drive failure, and depending on your
application that could easily be worth the premium on a "real" RAID.
But for raw disk performance, ccd keeps up quite nicely.

>>>>>
# iozone 256 65536
[...]
Writing the 256 Megabyte file, 'iozone.tmp'...24.000000 seconds
Reading the file...12.804688 seconds

IOZONE performance measurements:
        11184810 bytes/second for writing the file
        20963842 bytes/second for reading the file

# iozone 512 65536
[...]
Writing the 512 Megabyte file, 'iozone.tmp'...45.968750 seconds
Reading the file...30.593750 seconds

IOZONE performance measurements:
        11679040 bytes/second for writing the file
        17548385 bytes/second for reading the file


              -------Sequential Output-------- ---Sequential Input-- --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
single    256  3473 29.0  3555  7.6  1659  5.4  4782 44.6  3716  5.6 105.2  2.0
raidion   256   985  8.2   954  1.9   772  2.2  4754 44.3  5110  5.6 120.8  1.7
cmd       256  7213 59.4  7176 16.1  3521 11.1  7328 68.4  6136  6.7 187.8  2.7
ccd3      256 10965 88.2 11356 28.9  7134 23.2 10302 96.5 12596 20.3 163.7  3.0
ccd5      256 12129 97.1 14036 35.0  8299 27.0 10233 96.3 14351 23.5 181.0  3.4


SINGLE
  touch:   0.277u 56.454s 3:54.02  24.2%  10+170k 166+20314io 14pf+0w
  retouch: 0.193u  2.796s 1:49.61   2.7%  17+190k   2+10000io  0pf+0w
  unlink:  0.199u  4.792s 1:52.40   4.4% 167+226k   1+10000io  6pf+0w

RAIDION
  touch:   0.245u 57.470s 1:16.07  75.8%  10+171k 159+20314io 15pf+0w
  retouch: 0.174u  2.797s 0:11.59  25.5%  16+176k   2+10000io  0pf+0w
  unlink:  0.171u  4.838s 0:13.55  36.9% 160+216k   1+10000io  3pf+0w

CMD
  touch:   0.192u 56.159s 1:08.75  81.9%  10+169k 166+20314io 29pf+0w
  retouch: 0.187u  2.764s 0:09.25  31.7%  16+185k   1+10000io  0pf+0w
  unlink:  0.216u  4.757s 0:11.07  44.8% 164+220k   2+10000io  0pf+0w

CCD-async
  touch:   0.230u 58.327s 0:59.14  99.0%  10+170k 165+  314io  6pf+0w
  retouch: 0.203u  2.576s 0:02.61 106.1%  20+192k   1+    0io  0pf+0w
  unlink:  0.182u  4.264s 0:04.68  94.8% 168+222k   1+  305io  6pf+0w
<<<<<

--
Brian Tao (BT300, taob@io.org, taob@ican.net)
Senior Systems and Network Administrator, Internet Canada Corp.
"Though this be madness, yet there is method in't"




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.3.95.961110163640.10143B-100000>