From owner-freebsd-scsi@FreeBSD.ORG Fri Sep 24 22:45:47 2010 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8A5BC106566C for ; Fri, 24 Sep 2010 22:45:47 +0000 (UTC) (envelope-from des@des.no) Received: from smtp.des.no (smtp.des.no [194.63.250.102]) by mx1.freebsd.org (Postfix) with ESMTP id 4BAF38FC17 for ; Fri, 24 Sep 2010 22:45:47 +0000 (UTC) Received: from ds4.des.no (des.no [84.49.246.2]) by smtp.des.no (Postfix) with ESMTP id 777101FFC53; Fri, 24 Sep 2010 22:45:46 +0000 (UTC) Received: by ds4.des.no (Postfix, from userid 1001) id 3D9E6844A3; Sat, 25 Sep 2010 00:45:46 +0200 (CEST) From: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= To: Niklas Saers References: <2EA9CBBC-3F97-4AF2-BFB5-96DF39FDE376@saers.com> Date: Sat, 25 Sep 2010 00:45:46 +0200 In-Reply-To: <2EA9CBBC-3F97-4AF2-BFB5-96DF39FDE376@saers.com> (Niklas Saers's message of "Thu, 23 Sep 2010 14:55:02 +0200") Message-ID: <86aan67obp.fsf@ds4.des.no> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.2 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Cc: freebsd-scsi@freebsd.org Subject: Re: mfi - setting up disks X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Sep 2010 22:45:47 -0000 Niklas Saers writes: > In the SuperMicro system where I had problems with the mpt controller, > I switched it for a mfi-based controller. I had it set up with 36x > RAID0 volumes with each their own disk (no way to access the disk > otherwise I found), and added them to a ZFS system. The numbering > became a bit weird, so I pulled the disks out one by one and put them > back to figure out and note down what disk number was in what > slot. Only test data on my ZFS volume, so I didn't mind that crashing. You can wire down SCSI buses and disks in /boot/device.hints so each disk always gets the same device number regardless of the order in which the disks spin up. The syntax is documented in /sys/conf/NOTES (search for "SCSI DEVICE CONFIGURATION"). It's a CAM feature, and mfi uses CAM, so I *think* it should work for mfi as well, but what you'll actually be wiring down are mfi volumes, not individual disks, so it's up to you to assign the right disk to the right volume. I am very close to suggesting that you just let the controller handle the RAID part of things and just build your zfs pool on top of that, i.e. use mfiutil to divide your disks into 5 x 6+1, 5 x 5+2 or 7 x 4+1 volumes plus one spare, and use each volume as a separate vdev in your zfs pool. You may lose a small amount of performance, and rebuilds will be slower, but the main argument in favor of zfs, the write hole, is moot if your controller has battery-backed cache. DES --=20 Dag-Erling Sm=C3=B8rgrav - des@des.no