From owner-freebsd-current@FreeBSD.ORG Sun Oct 31 22:51:26 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 938A916A4CE for ; Sun, 31 Oct 2004 22:51:26 +0000 (GMT) Received: from picard.newmillennium.net.au (static-114.250.240.220.dsl.comindico.com.au [220.240.250.114]) by mx1.FreeBSD.org (Postfix) with ESMTP id DF43243D53 for ; Sun, 31 Oct 2004 22:51:22 +0000 (GMT) (envelope-from alastair@newmillennium.net.au) Received: from riker (riker.nmn.cafn [10.0.1.2])i9VMqrqR000990 for ; Mon, 1 Nov 2004 09:52:54 +1100 (EST) (envelope-from alastair@newmillennium.net.au) From: Sender: "Alastair D'Silva" To: Date: Mon, 1 Nov 2004 09:55:50 +1100 Organization: New Millennium Networking Message-ID: <002401c4bf9c$c4fee8e0$0201000a@riker> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.2616 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Importance: Normal Subject: Gvinum RAID5 performance X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 31 Oct 2004 22:51:26 -0000 I've just added 4 x Seagate 200GB IDE hard drives to my system, and have created a RAID5 setup (279KB stripe size) across all 4 new disks with the volume name 'newexport'. Each drive is a single drive with its own IDE channel. The current gvinum config is below: gvinum -> l 6 drives: D drive5 State: up /dev/ad10s1d A: 0/190654 MB (0%) D drive4 State: up /dev/ad8s1d A: 0/190654 MB (0%) D drive2 State: up /dev/ad6s1d A: 0/190654 MB (0%) D drive3 State: up /dev/ad4s1d A: 0/190654 MB (0%) D drive1 State: up /dev/ad2s1d A: 0/114345 MB (0%) D drive0 State: up /dev/ad0s1d A: 0/114345 MB (0%) 8 volumes: V newexport State: up Plexes: 1 Size: 558 GB V root State: up Plexes: 2 Size: 160 MB V home State: up Plexes: 2 Size: 8192 MB V usr State: up Plexes: 2 Size: 8192 MB V var State: up Plexes: 2 Size: 512 MB V pgsql State: up Plexes: 2 Size: 512 MB V squid State: up Plexes: 1 Size: 2048 MB V export State: up Plexes: 1 Size: 187 GB 13 plexes: P newexport.p0 R5 State: up Subdisks: 4 Size: 558 GB P root.p0 C State: up Subdisks: 1 Size: 160 MB P root.p1 C State: up Subdisks: 1 Size: 160 MB P home.p0 C State: up Subdisks: 1 Size: 8192 MB P home.p1 C State: up Subdisks: 1 Size: 8192 MB P usr.p0 C State: up Subdisks: 1 Size: 8192 MB P usr.p1 C State: up Subdisks: 1 Size: 8192 MB P var.p0 C State: up Subdisks: 1 Size: 512 MB P var.p1 C State: up Subdisks: 1 Size: 512 MB P pgsql.p0 C State: up Subdisks: 1 Size: 512 MB P pgsql.p1 C State: up Subdisks: 1 Size: 512 MB P squid.p0 S State: up Subdisks: 2 Size: 2048 MB P export.p0 S State: up Subdisks: 2 Size: 187 GB 18 subdisks: S newexport.p0.s3 State: up D: drive5 Size: 186 GB S newexport.p0.s2 State: up D: drive4 Size: 186 GB S newexport.p0.s1 State: up D: drive3 Size: 186 GB S newexport.p0.s0 State: up D: drive2 Size: 186 GB S root.p0.s0 State: up D: drive0 Size: 160 MB S root.p1.s0 State: up D: drive1 Size: 160 MB S home.p0.s0 State: up D: drive0 Size: 8192 MB S home.p1.s0 State: up D: drive1 Size: 8192 MB S usr.p0.s0 State: up D: drive0 Size: 8192 MB S usr.p1.s0 State: up D: drive1 Size: 8192 MB S var.p0.s0 State: up D: drive0 Size: 512 MB S var.p1.s0 State: up D: drive1 Size: 512 MB S pgsql.p0.s0 State: up D: drive0 Size: 512 MB S pgsql.p1.s0 State: up D: drive1 Size: 512 MB S squid.p0.s0 State: up D: drive0 Size: 1024 MB S squid.p0.s1 State: up D: drive1 Size: 1024 MB S export.p0.s0 State: up D: drive0 Size: 93 GB S export.p0.s1 State: up D: drive1 Size: 93 GB So far so good. Now, running a dd from a plex gives me less performance than running a dd from one of the subdisks, even though the array is not running in degraded mode. picard# dd if=/dev/gvinum/sd/newexport.p0.s0 of=/dev/null bs=16M count=100 100+0 records in 100+0 records out 1677721600 bytes transferred in 25.873485 secs (64843279 bytes/sec) picard# dd if=/dev/gvinum/plex/newexport.p0 of=/dev/null bs=16M count=100 100+0 records in 100+0 records out 1677721600 bytes transferred in 28.513923 secs (58838680 bytes/sec) Also, something seems to blow up when running a newfs, but I don't get a panic. It could be a hardware issue, and I'll try reading/writing from the raw drives to verify this later. picard# newfs -L export -U /dev/gvinum/newexport /dev/gvinum/newexport: 571962.0MB (1171378176 sectors) block size 16384, fragment size 2048 using 3113 cylinder groups of 183.77MB, 11761 blks, 23552 inodes. with soft updates super-block backups (for fsck -b #) at: 160, 376512, 752864, 1129216, 1505568, 1881920, 2258272, 2634624, 3010976, 3387328, 3763680, 4140032, 4516384, 4892736, 5269088, 5645440, 6021792, 6398144, 6774496, 7150848, 7527200, 7903552, 8279904, 8656256, 9032608, 9408960, 9785312, 10161664, 10538016, 10914368, 11290720, 11667072, 12043424, 12419776, 12796128, 13172480, 13548832, 13925184, 14301536, 14677888, 15054240, 15430592, 15806944, 16183296, 16559648, 16936000, 17312352, 17688704, 18065056, 18441408, 18817760, 19194112, 19570464, 19946816, 20323168, 20699520, 21075872, 21452224, 21828576, 22204928, 22581280, 22957632, 23333984, 23710336, 24086688, 24463040, 24839392, 25215744, 25592096, 25968448, 26344800, 26721152, 27097504, 27473856, 27850208, 28226560, 28602912, 28979264, 29355616, 29731968, 30108320, 30484672, 30861024, 31237376, 31613728, 31990080, 32366432, 32742784, 33119136, On a final note, I'd like to implement the ability to grow RAID5 plexes - any suggestions on where to start? -- Alastair D'Silva mob: 0413 485 733 Networking Consultant fax: 0413 181 661 New Millennium Networking web: http://www.newmillennium.net.au