Date: 27 Feb 2001 15:35:42 -0600 From: Kirk Strauser <kirk@strauser.com> To: kirk@strauser.com Cc: freebsd-fs@freebsd.org, mtbear@modus.org, corwin@ghostwheel.honeypot.net Subject: A benchmark of Vinum with IDE drives Message-ID: <8766hverm9.fsf@pooh.honeypot>
next in thread | raw e-mail | index | archive | help
I decided to benchmark Vinum on various combinations of low-cost hardware that
a small-budget user might be likely to have. I thought that the results were
interesting enough that someone else might like to read them, and I certainly
thought that the FreeBSD community could use a few more actual numbers for
would-be enthusiasts to dig through. The outcome of my testing follows below.
######## An Outline ########
1. Introduction
A. Motivation
B. Setup notes
2. Identification
A. Individual Devices
B. Vinum Volume definitions
3. Benchmark Results
A. Methodology
1. Bonnie
2. rawio
B. Notes
C. Results - Individual Devices
1. Bonnie
2. rawio
D. Results - Vinum Volumes
1. Bonnie
2. rawio
4. Commentary
A. Objective
B. Subjective
######## The Presentation ########
1. Introduction
A. Motivation
I became interested in Vinum at roughly the same time as when I noticed
that my /usr partition was over 90% full. Specifically, a Linux-using
friend was talking about LVM (Logical Volume Manager) and it's
drive-concatenatio capabilities. This sounded like just the thing I
needed.
I need to explain something at this point. As you'll see below, my
testbed had an U/W SCSI 4GB drive that I used for /, /var, /tmp, etc., and
an ATA/66 13GB drive that held /usr, /usr/export, and other large
mountpoints. Given that's what I had to work with, that's what I tested.
Yes, I am *completely* aware that striping your volumes across two
completely different drives is a pretty stupid idea, but it was fun. More
impressively (to me), it worked - well, even. The point, though, is that
I really do know better than to use the setup for real work. It was just
a test; please don't make fun of me. Anyway, after the success of my
asymmetrical experiments, I went to my local TV, refrigerator, and
computer part superstore and bought two identical ATA/100 30GB drives,
which are now my production hardware.
Also, I went into this experiment knowing that SCSI would be a clear
performance leader. However, with the phenomenally low price of IDE
storage these days, the advantage simply isn't worth the extra cost for
most home or small-LAN uses. I wanted to know what kind of raw throughput
a home user or a small system administrator could get for their money if
they were willing to invest a little work.
My only criterion for these tests was sheer throughput. Some of the
setups were more robust than others by definition, but that was not the
purpose of the experiment.
My goal in running these tests was not to provide scientifically accurate
information to the world. I just wanted to find out what setup would give
the best results on my hardware. While searching the web for basic hints,
though, I discovered a total lack of hard numbers on the subject.
Furthermore, IDE drives are almost never mentioned, and then usually only
in a warning section. This is my little way to contribute to the body of
knowledge. I hope that someone finds it helpful.
B. Setup notes
The test system is an Intel Celeron 400MHz CPU on an Asus P3B-F
motherboard (BIOS version 1006) with 128MB of PC-100 ECC SDRAM. SCSI
support is via a Tekram DC-390F PCI controller (BIOS version 3.22). The
IDE controller is the motherboard's built-in Intel PIIX4 ATA33 chipset.
All IDE drives were configured as "master" for their tests. In tests
involving two IDE drives, each drive was on a separate controller. I did
a quick before-and-after Bonnie test with two drives on the same
controller, and it was sad. Pathetic. Not Good. Those results are not
posted here.
All sizes given are in "true" (2 ^ 10*n) measurements, rather than "sales"
(1000 ^ n) figures. That is, 1MB = 1,048,576 bytes (not 1,000,000!).
2. Identification
A. Individual Devices
1. ibm_scsi
a. Model DCAS-34330W S65A (4.1GB)
b. Ultra-wide SCSI, 3 platters, 8.5ms average seek time, 448KB buffer,
5400 RPM
2. ibm_ide
a. Model DJNA-371350 (12.9 GB, aka "Deskstar 22GXP")
b. Ultra ATA/66, 3 platters, 8.7ms average seek time, 1966KB buffer,
7200 RPM
3. maxtor1/2
a. Model 5T030H3 (29.3GB, aka "DiamondMax Plus 60")
b. Ultra ATA/100, 2 platters, 8.7ms average seek time, 2MB buffer,
7200 RPM
B. Vinum Volume definitions
1. A concat plex of one subdrive on a single IDE (ibm_ide) drive:
volume A_SINGLE
plex org concat
sd length 1g drive ibm_ide
2. A 256k striped plex with one subdrive on IDE (ibm_ide) and one subdrive
on SCSI (ibm_scsi):
volume A_STRIPE
plex org striped 256k
sd length 1g drive ibm_ide
sd length 1g drive ibm_scsi
3. A mirrored plex with subdrives on two IDE drives (maxtor1/2):
volume MIRROR
plex org concat
sd length 1g drive maxtor1
plex org concat
sd length 1g drive maxtor2
4. A 256k striped plex with subdrives on two IDE drives (maxtor1/2):
volume STRIPE256
plex org striped 256k
sd length 1g drive maxtor1
sd length 1g drive maxtor2
5. A 512k striped plex with subdrives on two IDE drives (maxtor1/2):
volume STRIPE512
plex org striped 512k
sd length 1g drive maxtor1
sd length 1g drive maxtor2
6. Two 256k striped plexes, each with subdrives on two IDE drives
(maxtor1/2), mirrored:
volume RAID10256
plex org striped 256k
sd length 512m drive maxtor1
sd length 512m drive maxtor2
plex org striped 256k
sd length 512m drive maxtor2
sd length 512m drive maxtor1
7. Two 512k striped plexes, each with subdrives on two IDE drives
(maxtor1/2), mirrored:
volume RAID10512
plex org striped 512k
sd length 512m drive maxtor1
sd length 512m drive maxtor2
plex org striped 512k
sd length 512m drive maxtor2
sd length 512m drive maxtor1
3. Benchmark Results
A. Methodology
Two tests were performed on the various drives and volumes: Bonnie and
rawio. Bonnie is a well-known filesystem benchmarking program that
measures the high-level throughput of a device. rawio differs in that it
is specifically designed to bypass any filesystem caching in order to
measure performance of the tested device itself.
1. Bonnie
Bonnie writes to a file on the specified volume using various methods,
then reads the contents back. In all cases, the default filelength of
100MB was used. The filesystem was initialized with newfs (without any
command line arguments) before each test. The command line used was:
bonnie -d {volume_name} > bonnie-{description}
2. rawio
rawio accessess the specified block device directly. Since it ignores
any filesystem data, newfs was not used on the device before each test
as it was with Bonnie. The command line used was:
rawio -a {volume_name} > rawio-{description}
B. Notes
All tests were not performed on all drive setups. If I had planned this a
little more before the start of the project, I would've made the tests
comprehensive.
In the results below, the highest throughtput and lowest CPU utilization in
each column are marked with an asterisk to the /right/ of the cell. No
other ordering or row-arranging was done.
C. Results - Individual Devices
1. Bonnie
-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
IBM_SCSI 7960 39.6* 7995 12.6* 3680 8.4* 7850 38.7* 7930 10.0*261.0 3.5*
IBM_IDE 14690*72.9 15050*25.5 6449 16.7 15728 78.0 17102 23.5 264.1 3.6
MAXTOR 13022 63.9 13470 22.0 7973*18.9 19455*95.7 29632*39.8 405.5* 5.2
2. rawio
Random read Sequential read Random write Sequential write
K/sec /sec K/sec /sec K/sec /sec K/sec /sec
IBM_SCSI 1062.3 67 1008.2 62 1071.5 67 1054.0 64
IBM_IDE 2013.6 125 1996.3 122 1910.5* 118* 1898.7 116
MAXTOR 2291.4* 142* 5136.4* 314* 1324.8 82 8914.6* 544*
D. Results - Vinum Volumes
1. Bonnie
-------Sequential Output-------- ---Sequential Input-- --Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
A_SINGLE 15546*77.3 15794 25.2 7029 15.7 15956 78.4 16363 16.9 237.6 3.4*
A_STRIPE 14764 76.5 15002 27.8 5220 14.2 11533 58.5*12843 18.1 336.8 5.3
MIRROR 8668 44.5 9484 16.2 3047 7.4*16318 81.4 20151 20.6 607.2 7.7
STRIPE256 13907 70.1 18756 32.0 8380*20.5 19125 97.4 27116 28.5 608.8* 7.6
STRIPE512 13789 69.4 18782*31.3 7711 18.5 19165*96.1 27918*28.4 545.8 7.2
RAID10256 8026 41.1* 9431 16.1 4055 9.6 12748 63.6 13714 14.7*439.5 5.7
RAID10512 8065 41.1* 9423 15.7* 3418 8.0 13002 64.7 14765 15.4 412.8 5.3
2. rawio
Random read Sequential read Random write Sequential write
ID K/sec /sec K/sec /sec K/sec /sec K/sec /sec
A_STRIPE 2170.2 135 7060.9 431 2262.2* 140* 3596.3 219
MIRROR 3534.5* 218* 11908.9 727 1151.2 71 7739.2 472
STRIPE256 3246.0 202 18812.1 1148 2182.3 136 16774.7 1024
STRIPE512 3336.9 207 20659.3* 1261* 2240.7 139 16862.8* 1029*
RAID10256 3004.9 187 10564.1 645 1041.7 64 8011.8 489
RAID10512 3029.9 189 13488.9 823 1058.1 66 8400.0 513
4. Commentary
A. Objective
The numbers pretty well speak for themselves, but I'm going to chip in
anyway. In a nutshell, mirroring is very slow on this setup. I was
disappointed to see that it didn't hold up to striping even for reads,
except for it's slight lead in rawio's random test. Striping, on the other
hand, was an all-around winner. In fact, except for CPU loading, I found
no weak spot in this scheme at all.
The RAID-10 setups offered no bonus at all. I suspect that anyone needing
the robustness offered by such a setup would be using SCSI.
Unsurprisingly, the U/W SCSI drive was always the must CPU-friendly device
tested (and often by a huge margin). I make to claim to causality, though
- it may have been more cycle-hungry if it were able to achieve the raw
transfer rates of the newer IDE drives.
B. Subjective
I may never use a single-drive setup again. After settling into a
256k-striped configuration at the conclusion of the tests, everything seems
to fly along. Doing a "make search" in /usr/ports for the letter 'a' takes
.483 real seconds. NFS flies along. "make clean" is no longer agonizing.
Yes, there is a definite place for SCSI drives, particularly in any real
server. For the home user or administrator of a small file server, though,
Vinum on IDE is a spectacular combination of blazing performance and
rock-bottom price. If you haven't tried it already, make a backup of your
data and start experimenting!
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-fs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?8766hverm9.fsf>