From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 15 05:19:24 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 076CC106564A for ; Thu, 15 Dec 2011 05:19:24 +0000 (UTC) (envelope-from janm@transactionware.com) Received: from midgard.transactionware.com (mail2.transactionware.com [203.14.245.36]) by mx1.freebsd.org (Postfix) with SMTP id 5E4238FC13 for ; Thu, 15 Dec 2011 05:19:22 +0000 (UTC) Received: (qmail 88889 invoked by uid 907); 15 Dec 2011 04:52:41 -0000 Received: from jmmacpro.transactionware.com (HELO jmmacpro.transactionware.com) (192.168.1.33) by midgard.transactionware.com (qpsmtpd/0.82) with ESMTP; Thu, 15 Dec 2011 15:52:41 +1100 Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=us-ascii From: Jan Mikkelsen In-Reply-To: <9317551F-CBE0-4368-B798-498E58E240B2@sarenet.es> Date: Thu, 15 Dec 2011 15:52:41 +1100 Content-Transfer-Encoding: quoted-printable Message-Id: <4391B719-FABD-4E13-B328-D55EA9E7D74D@transactionware.com> References: <4EE8A005.5030607@barafranca.com> <9317551F-CBE0-4368-B798-498E58E240B2@sarenet.es> To: Borja Marcos X-Mailer: Apple Mail (2.1251.1) X-Mailman-Approved-At: Sun, 18 Dec 2011 22:24:05 +0000 Cc: freebsd-hackers@freebsd.org, Hugo Silva Subject: Re: mfi (Dell H700) + hot swapping doesn't appear to work with RC1 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Dec 2011 05:19:24 -0000 On 15/12/2011, at 2:16 AM, Borja Marcos wrote: >=20 > On Dec 14, 2011, at 2:09 PM, Hugo Silva wrote: >=20 >> Hello, >>=20 >> First of all apologies if this has been fixed in RC3. I set this = server >> up with mfsbsd, which is RC1, and didn't get to update the system = yet. >>=20 >> This box has 6 hdds, a 2-mirror zpool was set up as the root pool, = with >> 2 spares. >>=20 >> While testing hot swapping I noticed that while the controller = detects >> disk removal/insertion, the zpool will never recover. The problem = seems >> to be deeper than ZFS, as disklabel/fdisk/etc also fail on the >> removed-and-reinserted disk. >>=20 >> At the ZFS level, doing a zpool clear yields more errors on the = removed >> disk; rebooting becomes the only option to make the pool healthy = again. >>=20 >>=20 >> Is this normal? Did I miss any step? >=20 > I assume that you have tried to use the H700 as a "JBOD" card, = defining logical volume for each hard disk. >=20 > The problem is: that gorgeous, fantastic, masterful, Nobel award = candidate card, has a wonderful behavior in that case. If you extract = one of the disks, the logical volume associated to it is invalidated. = So, you insert a replacement disk, and the card refuses to recognize the = volume. What is even worse, in order to recover it's mandatory to reboot = the complete system *AND* go through the RAID configuration utility. >=20 > That's the problem. The card refuses to work as a simple disk = controller without frills, and the frills get in the way. >=20 > To summarize: it isn't FreeBSD's fault, no matter which version you = use. It's a "feature" coming directly from the geniuses who designed the = card. Hugo: You missed a step. Borja: No reboot required. For the mfi controllers I have been testing recently (MegaRAID 9261-8i), = you need to install the sysutils/megacli port, and use that to clear the = "foreignness" of the disk you just added. Something like: MegaCli -CfgForeign -Clear -a0 You should be able to then recreate it as a JBOD device, and progress = through whatever higher level recovery you need to do. Regards, Jan Mikkelsen