From owner-freebsd-fs@FreeBSD.ORG Tue Jan 22 15:40:24 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id AC856D0D for ; Tue, 22 Jan 2013 15:40:24 +0000 (UTC) (envelope-from claudiu.vasadi@gmail.com) Received: from mail-ie0-f178.google.com (mail-ie0-f178.google.com [209.85.223.178]) by mx1.freebsd.org (Postfix) with ESMTP id 56F82E36 for ; Tue, 22 Jan 2013 15:40:24 +0000 (UTC) Received: by mail-ie0-f178.google.com with SMTP id c12so11646001ieb.23 for ; Tue, 22 Jan 2013 07:40:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:in-reply-to:references:date:message-id :subject:from:to:cc:content-type; bh=1ciVddKAnkarB86VkrstRmL+G5j25tdaYeub/w6zWu0=; b=s56RSmqldG0KyRZn91DmMe3n+su7eDyVLqbKmWbUZiI1YO1G6AJ/bdhaZI316YYtzP GcTY1lIxifnHWhCNY3zYFNamDOpTWmkiKuVtoowjQfK/W1zv0KtuJby9jiA1YvNtSe4Y Ed5wDVtMkX6AQgiYXq530Zf4qILkXRNDUhLzA9juPFAfkpuSZ7vlttmRh4gkAeCIOpNZ yukWr4+61vAP2HhuWLFa2TwcX1i8Dx7AtEvWoje9iXRalEo2tNpIz1U7YRqPgzaMf6+d wu7a1jGLl7VriqWxl8PZEcI1I2MzSdgs581LOVUBajVlqdMRexd6/aDWoZGKstIe4mKJ T0Gw== MIME-Version: 1.0 X-Received: by 10.50.170.66 with SMTP id ak2mr12402662igc.38.1358869223676; Tue, 22 Jan 2013 07:40:23 -0800 (PST) Received: by 10.64.33.110 with HTTP; Tue, 22 Jan 2013 07:40:23 -0800 (PST) In-Reply-To: <16B2C50C-DD36-4375-A002-F866A612D842@sarenet.es> References: <314B600D-E8E6-4300-B60F-33D5FA5A39CF@sarenet.es> <16B2C50C-DD36-4375-A002-F866A612D842@sarenet.es> Date: Tue, 22 Jan 2013 16:40:23 +0100 Message-ID: Subject: Re: RFC: Suggesting ZFS "best practices" in FreeBSD From: claudiu vasadi To: Borja Marcos Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jan 2013 15:40:24 -0000 On Tue, Jan 22, 2013 at 4:36 PM, Borja Marcos wrote: > > On Jan 22, 2013, at 4:04 PM, Warren Block wrote: > > > I'm a proponent of using various types of labels, but my impression > after a recent experience was that ZFS metadata was enough to identify the > drives even if they were moved around. That is, ZFS bare metadata on a > drive with no other partitioning or labels. > > > > Is that incorrect? > > I'm afraid it's inconvenient unless you enjoy reboots ;) > > This is a patologic and likely example I just demonstrated to a friend. > > We were testing a new server with 12 hard disks and a proper HBA. > > The disks are, unspririsingly, da0-da11. There is a da12 used (for now) > for the OS, so that there's no problem to create and destroy pools at > leisure. > > My friend had created a pool with two raidz vdevs nothing rocket science. > da0-5, da6-11. > > So, we were doing some tests and I've pulled one of the disks. Nothing > special, ZFS recovers nicely. > > Now it comes the fun part. > > I reboot the machine with the missing disk. > > What happens now? > > I had pulled da4 I think. So, disks with an ID > 4 have been renamed to N > - 1. da5 became da4, da6 became da5, da7 became da6... and, critically, > da12 became da11. > > The reboot begun by failing to mount the root filesystem, but that one is > trivial. Just tell the kernel where it is now (da11) and it boots happily. > > Now, we have a degraded pool with a missing disk (da4) and a da4 that > previously was da5. It works of course, but in degraded state. > > OK, we found a replacement disk, and we plugged it. It became, guess! Yes, > da12. > > Now: I cannot "online" da4, because it exists. I cannot online da12 > because it didn't belong to the pool. I cannot replace da4 with da12, > because it is there. > > Now that I think of it, in this case: > 15896790606386444480 OFFLINE 0 0 0 was /dev/da4 > > Is it possible to say zpool replace 15896790606386444480 da12? I haven't > tried it. > > Anyway, seems to be a bit confusing. The logical, albeit cumbersome > approach is to reboot the machine with the new da4 in place, and after > rebooting, onlining or replacing. > > Using names prevents this kind of confusion. > > > > > Same thing happened to me on a production server that crashed. Sometimes, it;s not easy to reboot again simply because you need to insert a disk. This, of course, make s the hot-swappable capability of the hardware, useless. -- Best regards, Claudiu Vasadi