From owner-freebsd-scsi Sun Nov 10 15:42:07 1996 Return-Path: owner-freebsd-scsi Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id PAA00736 for freebsd-scsi-outgoing; Sun, 10 Nov 1996 15:42:07 -0800 (PST) Received: from who.cdrom.com (who.cdrom.com [204.216.27.3]) by freefall.freebsd.org (8.7.5/8.7.3) with ESMTP id PAA00729 for ; Sun, 10 Nov 1996 15:42:05 -0800 (PST) Received: from ican.net (ican.net [198.133.36.9]) by who.cdrom.com (8.7.5/8.6.11) with SMTP id OAA26672 for ; Sun, 10 Nov 1996 14:33:32 -0800 (PST) Received: from gate.ican.net(really [198.133.36.2]) by ican.net via sendmail with esmtp id for ; Sun, 10 Nov 1996 17:33:19 -0500 (EST) (Smail-3.2 1996-Jul-4 #1 built 1996-Jul-10) Received: (from smap@localhost) by gate.ican.net (8.7.5/8.7.3) id RAA05638 for ; Sun, 10 Nov 1996 17:32:36 -0500 (EST) Received: from cabal.io.org(10.1.6.2) by gate.ican.net via smap (V1.3) id sma005636; Sun Nov 10 17:32:24 1996 Received: from localhost (taob@localhost) by cabal.io.org (8.7.6/8.7.3) with SMTP id RAA10274 for ; Sun, 10 Nov 1996 17:32:24 GMT X-Authentication-Warning: cabal.io.org: taob owned process doing -bs Date: Sun, 10 Nov 1996 17:32:24 +0000 () From: Brian Tao To: FREEBSD-SCSI-L Subject: CCD vs. hardware RAIDs (more numbers) Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-scsi@freebsd.org X-Loop: FreeBSD.org Precedence: bulk After much delay with vanishing U.S. distributors and customs hassles, I now have three rackmount enclosures, each with seven 2GB drives (eventually becoming part of a news server cluster). ccd in 2.2-961014-SNAP was very easy to setup, once I realized a drive in the array somehow had its filesystem set to "SystemV" type and not "4.2BSD". To continue some of the benchmarks I performed with the CMD and Streamlogic RAIDION RAIDs, I striped three of the 2GB drives together. This probably isn't a fair test, but I think it's in the right ballpark. The test server is the same PPro200, but now upgraded to the Oct 14 snapshot. I used Quantum 2GB wide drives, mounted in a DAE-800 rackmount enclosure from Connector Resources Unlimited (http://www.cruinc.com/): FreeBSD 2.2-961014-SNAP #0: Sun Nov 10 02:06:41 EST 1996 root@news-in.ican.net:/mnt/sys/compile/NEWS Calibrating clock(s) relative to mc146818A clock... i586 clock: 199305096 Hz, i8254 clock: 1193158 Hz CPU: Pentium Pro (199.31-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x617 Stepping=7 Features=0xf9ff,MTRR,PGE,MCA,CMOV> real memory = 134217728 (131072K bytes) avail memory = 129789952 (126748K bytes) Probing for devices on PCI bus 0: chip0 rev 2 on pci0:0 chip1 rev 1 on pci0:1:0 chip2 rev 0 on pci0:1:1 de0 rev 18 int a irq 10 on pci0:10 de0: SMC 9332 DC21140 [10-100Mb/s] pass 1.2 de0: address 00:00:c0:6b:eb:e6 de0: enabling 10baseT port ahc0 rev 0 int a irq 11 on pci0:11 ahc0: aic7880 Wide Channel, SCSI Id=15, 16 SCBs ahc0 waiting for scsi devices to settle (ahc0:0:0): "Quantum XP32150W L912" type 0 fixed SCSI 2 sd0(ahc0:0:0): Direct-Access 2151MB (4406960 512 byte sectors) sd0(ahc0:0:0): with 3907 cyls, 10 heads, and an average 112 sectors/track (ahc0:1:0): "Quantum XP32150W 81HB" type 0 fixed SCSI 2 sd1(ahc0:1:0): Direct-Access 2050MB (4199760 512 byte sectors) sd1(ahc0:1:0): with 3907 cyls, 10 heads, and an average 107 sectors/track (ahc0:2:0): "Quantum XP32150W L912" type 0 fixed SCSI 2 sd2(ahc0:2:0): Direct-Access 2151MB (4406960 512 byte sectors) sd2(ahc0:2:0): with 3907 cyls, 10 heads, and an average 112 sectors/track [...] Stripe size used was 64K, to match those on the hardware RAID's. The CMD-5500 was the faster of the two hardware RAIDs, clocking in at 14MB/s writing and 11MB/s reading to a 256MB file on a RAID 0 filesystem. The ccd filesystem hit 11MB/s writing and almost 21MB/s reading a 256MB file. This looked like there was a cache effect (F/W SCSI-2 peaks at 20MB/s, does it not?). I repeated the iozone with 512MB; this time it reported about 11.5MB/s writing and 17.5MB/sec reading. Bonnie was also quite a bit faster on the throughput tests, but a little slower with the random seeks. Increasing the stripe set to five drives improved the stats (results are included at the end of this message). Mounted async, the CCD filesystem is *fast*... it outperforms the CMD-5500 (with 64MB of write-back cache) on the 10000-file touch/touch/unlink test. The 15556-file FreeBSD source tree can be untarred onto it from another drive in 62 seconds flat. :) So for about US$5200 (rack enclosure, seven 2GB fast/wide drives), you get something that can be just as fast or faster than a hardware RAID, at about one-third the $/GB cost (less if you don't go for the fancy enclosure). Of course, you don't get the same amount of reliability or protection against drive failure, and depending on your application that could easily be worth the premium on a "real" RAID. But for raw disk performance, ccd keeps up quite nicely. >>>>> # iozone 256 65536 [...] Writing the 256 Megabyte file, 'iozone.tmp'...24.000000 seconds Reading the file...12.804688 seconds IOZONE performance measurements: 11184810 bytes/second for writing the file 20963842 bytes/second for reading the file # iozone 512 65536 [...] Writing the 512 Megabyte file, 'iozone.tmp'...45.968750 seconds Reading the file...30.593750 seconds IOZONE performance measurements: 11679040 bytes/second for writing the file 17548385 bytes/second for reading the file -------Sequential Output-------- ---Sequential Input-- --Random-- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks--- Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU single 256 3473 29.0 3555 7.6 1659 5.4 4782 44.6 3716 5.6 105.2 2.0 raidion 256 985 8.2 954 1.9 772 2.2 4754 44.3 5110 5.6 120.8 1.7 cmd 256 7213 59.4 7176 16.1 3521 11.1 7328 68.4 6136 6.7 187.8 2.7 ccd3 256 10965 88.2 11356 28.9 7134 23.2 10302 96.5 12596 20.3 163.7 3.0 ccd5 256 12129 97.1 14036 35.0 8299 27.0 10233 96.3 14351 23.5 181.0 3.4 SINGLE touch: 0.277u 56.454s 3:54.02 24.2% 10+170k 166+20314io 14pf+0w retouch: 0.193u 2.796s 1:49.61 2.7% 17+190k 2+10000io 0pf+0w unlink: 0.199u 4.792s 1:52.40 4.4% 167+226k 1+10000io 6pf+0w RAIDION touch: 0.245u 57.470s 1:16.07 75.8% 10+171k 159+20314io 15pf+0w retouch: 0.174u 2.797s 0:11.59 25.5% 16+176k 2+10000io 0pf+0w unlink: 0.171u 4.838s 0:13.55 36.9% 160+216k 1+10000io 3pf+0w CMD touch: 0.192u 56.159s 1:08.75 81.9% 10+169k 166+20314io 29pf+0w retouch: 0.187u 2.764s 0:09.25 31.7% 16+185k 1+10000io 0pf+0w unlink: 0.216u 4.757s 0:11.07 44.8% 164+220k 2+10000io 0pf+0w CCD-async touch: 0.230u 58.327s 0:59.14 99.0% 10+170k 165+ 314io 6pf+0w retouch: 0.203u 2.576s 0:02.61 106.1% 20+192k 1+ 0io 0pf+0w unlink: 0.182u 4.264s 0:04.68 94.8% 168+222k 1+ 305io 6pf+0w <<<<< -- Brian Tao (BT300, taob@io.org, taob@ican.net) Senior Systems and Network Administrator, Internet Canada Corp. "Though this be madness, yet there is method in't"