From owner-freebsd-stable@FreeBSD.ORG Fri Aug 17 23:05:35 2007 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D207816A41A for ; Fri, 17 Aug 2007 23:05:35 +0000 (UTC) (envelope-from bsam@ipt.ru) Received: from mail.kuban.ru (mail.kuban.ru [62.183.66.246]) by mx1.freebsd.org (Postfix) with ESMTP id 4014F13C48E for ; Fri, 17 Aug 2007 23:05:34 +0000 (UTC) (envelope-from bsam@ipt.ru) Received: from bsam.ru ([85.172.12.136]) by mail.kuban.ru (8.9.1/8.9.1) with ESMTP id l7HMPcrv054859; Sat, 18 Aug 2007 02:25:49 +0400 (MSD) Received: (from bsam@localhost) by bsam.ru (8.14.1/8.14.1/Submit) id l7HMQ4c5001609; Sat, 18 Aug 2007 02:26:04 +0400 (MSD) (envelope-from bsam@ipt.ru) X-Authentication-Warning: bsam.ru: bsam set sender to bsam@ipt.ru using -f To: Vivek Khera References: <31BB09D7-B58A-47AC-8DD1-6BB8141170D8@khera.org> From: Boris Samorodov Date: Sat, 18 Aug 2007 02:26:04 +0400 In-Reply-To: <31BB09D7-B58A-47AC-8DD1-6BB8141170D8@khera.org> (Vivek Khera's message of "Fri\, 17 Aug 2007 17\:42\:55 -0400") Message-ID: <20575075@bsam.ru> User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.1 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: FreeBSD Stable Subject: Re: large RAID volume partition strategy X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Aug 2007 23:05:35 -0000 On Fri, 17 Aug 2007 17:42:55 -0400 Vivek Khera wrote: > I have a shiny new big RAID array. 16x500GB SATA 300+NCQ drives > connected to the host via 4Gb fibre channel. This gives me 6.5Tb of > raw disk. > I've come up with three possibilities on organizing this disk. My > needs are really for a single 1Tb file system on which I will run > postgres. However, in the future I'm not sure what I'll really need. > I don't plan to ever connect any other servers to this RAID unit. > The three choices I've come with so far are: > 1) Make one RAID volume of 6.5Tb (in a RAID6 + hot spare > configuration), and make one FreeBSD file system on the whole > partition. > 2) Make one RAID volume of 6.5Tb (in a RAID6 + hot spare > configuration), and make 6 FreeBSD partitions with one file system > each. > 3) Make 6 RAID volumes and expose them to FreeBSD as multiple drives, > then make one partition + file system on each "disk". Each RAID > volume would span across all 16 drives, and I could make the volumes > of differing RAID levels, if needed, but I'd probably stick with RAID6 > +spare. > I'm not keen on option 1 because of the potentially long fsck times > after a crash. > What advantage/disadvantage would I have between 2 and 3? The only > thing I can come up with is that the disk scheduling algorithm in > FreeBSD might not be optimal if the drives really are not truly > independent as they are really backed by the same 16 drives, so > option 2 might be better. However, with option 3, if I do ever end > up connecting another host to the array, I can assign some of the > volumes to the other host(s). > My goal is speed, speed, speed. Seems that RAID[56] may be too sloooow. I'd suggest RAID10. I have 6 SATA-II 300MB/s disks at 3WARE adapter. My (very!) simple tests gave about 170MB/s for dd. BTW, I tested (OK, very fast) RAID5, RAID6, gmirror+gstripe and noone get close to RAID10. (Well, as expected, I suppose). > I'm running FreeBSD 6.2/amd64 and > using an LSI fibre card. If you have time you may do your own tests... And in case RAID0 you shouldn't have problems with long fsck. Leave a couple of your disks for hot-swapping and you'll get 7Tb. ;-) > Thanks for any opinions and recommendations. WBR -- bsam