From owner-freebsd-stable@FreeBSD.ORG Fri Aug 17 23:12:36 2007 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A6BCD16A41A for ; Fri, 17 Aug 2007 23:12:36 +0000 (UTC) (envelope-from bsam@ipt.ru) Received: from mail.kuban.ru (mail.kuban.ru [62.183.66.246]) by mx1.freebsd.org (Postfix) with ESMTP id 1468413C48A for ; Fri, 17 Aug 2007 23:12:35 +0000 (UTC) (envelope-from bsam@ipt.ru) Received: from bsam.ru ([85.172.12.136]) by mail.kuban.ru (8.9.1/8.9.1) with ESMTP id l7HNCD9K064177; Sat, 18 Aug 2007 03:12:24 +0400 (MSD) Received: (from bsam@localhost) by bsam.ru (8.14.1/8.14.1/Submit) id l7HNCdgw001985; Sat, 18 Aug 2007 03:12:39 +0400 (MSD) (envelope-from bsam@ipt.ru) X-Authentication-Warning: bsam.ru: bsam set sender to bsam@ipt.ru using -f To: Vivek Khera References: <31BB09D7-B58A-47AC-8DD1-6BB8141170D8@khera.org> <20575075@bsam.ru> From: Boris Samorodov Date: Sat, 18 Aug 2007 03:12:39 +0400 In-Reply-To: <20575075@bsam.ru> (Boris Samorodov's message of "Sat\, 18 Aug 2007 02\:26\:04 +0400") Message-ID: <54492280@bsam.ru> User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.1 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: FreeBSD Stable Subject: Re: large RAID volume partition strategy X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Aug 2007 23:12:36 -0000 On Sat, 18 Aug 2007 02:26:04 +0400 Boris Samorodov wrote: > On Fri, 17 Aug 2007 17:42:55 -0400 Vivek Khera wrote: > > I have a shiny new big RAID array. 16x500GB SATA 300+NCQ drives > > connected to the host via 4Gb fibre channel. This gives me 6.5Tb of > > raw disk. > > I've come up with three possibilities on organizing this disk. My > > needs are really for a single 1Tb file system on which I will run > > postgres. However, in the future I'm not sure what I'll really need. > > I don't plan to ever connect any other servers to this RAID unit. > > The three choices I've come with so far are: > > 1) Make one RAID volume of 6.5Tb (in a RAID6 + hot spare > > configuration), and make one FreeBSD file system on the whole > > partition. > > 2) Make one RAID volume of 6.5Tb (in a RAID6 + hot spare > > configuration), and make 6 FreeBSD partitions with one file system > > each. > > 3) Make 6 RAID volumes and expose them to FreeBSD as multiple drives, > > then make one partition + file system on each "disk". Each RAID > > volume would span across all 16 drives, and I could make the volumes > > of differing RAID levels, if needed, but I'd probably stick with RAID6 > > +spare. > > I'm not keen on option 1 because of the potentially long fsck times > > after a crash. > > What advantage/disadvantage would I have between 2 and 3? The only > > thing I can come up with is that the disk scheduling algorithm in > > FreeBSD might not be optimal if the drives really are not truly > > independent as they are really backed by the same 16 drives, so > > option 2 might be better. However, with option 3, if I do ever end > > up connecting another host to the array, I can assign some of the > > volumes to the other host(s). > > My goal is speed, speed, speed. > Seems that RAID[56] may be too sloooow. I'd suggest RAID10. > I have 6 SATA-II 300MB/s disks at 3WARE adapter. My (very!) simple > tests gave about 170MB/s for dd. BTW, I tested (OK, very fast) > RAID5, RAID6, gmirror+gstripe and noone get close to RAID10. (Well, as > expected, I suppose). > > I'm running FreeBSD 6.2/amd64 and > > using an LSI fibre card. > If you have time you may do your own tests... And in case RAID0 you ^^^^^ RAID10 > shouldn't have problems with long fsck. Leave a couple of your disks > for hot-swapping and you'll get 7Tb. ;-) ^^^ 3.5TB > > Thanks for any opinions and recommendations. sorry, not my night... WBR -- bsam