From owner-freebsd-questions@freebsd.org Wed Jun 14 16:44:27 2017 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B8D79BEF1C8 for ; Wed, 14 Jun 2017 16:44:27 +0000 (UTC) (envelope-from dpchrist@holgerdanske.com) Received: from holgerdanske.com (holgerdanske.com [184.105.128.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "holgerdanske.com", Issuer "holgerdanske.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A838770926 for ; Wed, 14 Jun 2017 16:44:27 +0000 (UTC) (envelope-from dpchrist@holgerdanske.com) Received: from 99.100.19.101 ([99.100.19.101]) by holgerdanske.com with ESMTPSA (ECDHE-RSA-AES128-GCM-SHA256:TLSv1.2:Kx=ECDH:Au=RSA:Enc=AESGCM(128):Mac=AEAD) (SMTP-AUTH username dpchrist@holgerdanske.com, mechanism PLAIN) for ; Wed, 14 Jun 2017 09:44:23 -0700 Subject: Re: Drive labelling with ZFS To: freebsd-questions@freebsd.org References: <03643051-38e8-87ef-64ee-5284e2567cb8@fjl.co.uk> <7fa67076-3ec8-4c25-67b9-a1b8a0aa5afc@holgerdanske.com> <5940EE63.2080904@fjl.co.uk> From: David Christensen Message-ID: Date: Wed, 14 Jun 2017 09:43:46 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <5940EE63.2080904@fjl.co.uk> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jun 2017 16:44:27 -0000 On 06/14/2017 01:05 AM, Frank Leonhardt wrote: > On 14/06/2017 03:02, David Christensen wrote: >> On 06/13/2017 04:32 PM, David Christensen wrote: >>> Both [1] and [3] discuss the fact that a given drive, partition, >>> file system, etc., can be identified in various ways, manual or >>> automatic, but the kernel will pick one and "wither" the rest. >>> Once a GPT label is set manually, other methods should be >>> disabled via settings in /boot/loader.conf and the system >>> rebooted ([1] p. 35): >>> >>> kern.geom.label.disk_ident.enable="0" >>> kern.geom.label.gptid.enable="0" >> >> Beware that all your disks need to have GPT labels, and those >> labels need to be carried forward into /etc/fstab, etc., before you >> reboot, as the kernel won't be able to find the disks using Disk ID >> or GPT GUID labels once those methods are disabled. >> >> > Thanks David. I'd actually tried all the things you suggested, and > read and re-read the Lucas books which blithely suggest setting GEOM > labels but without going in to detail. The first chapter is all over > the place in structure. However, I didn't try the sysctrl tweaks you > suggest to disable the other methods. I recall the books suggesting > that other methods are disabled, but without telling you how. > > You may well have supplied the missing piece of the jigsaw here. It's > a shame ZFS can't be told which labelling method to use (or can it?) > Current situation is less than helpful. > > The new SAS enclosure utility in 11.0 is great. It can flash the > light on any drive you like, but it only takes device names, not > GUIDs. And if ZFS fails /dev/da87p3 it immediately changes to > referring to it by the GUID only. I can see why assuming the drive is > completely off-line but in most cases it's JUST failed, and therefore > knowing where it was is the same as knowing where it is. > > Part of the problem is that zpools created by sysinstall during > installation are on unlabelled partitions. Actually it does label > them, but not in any helpful way. > > Regards, Frank. On 06/14/2017 07:22 AM, Frank Leonhardt wrote: > Hi David, > > It turns out that these options were set anyway. The problem turned > out be be that I was assuming that geom label played nice with GPT. > It doesn't! Well it does display labels set on GPT partitions, but > it doesn't change them. It took a look at the GPT blocks to confirm > this. It does, however, mask the GPT version with its own, sometimes, > leading to much monkeyhouse. > > So ignore glabel completely and set the labels using gpart instead. > > Having got this sorted out, it turns out that it's really not as > useful as it sounds. On a new array you can find a broken drive this > way, but when it comes to moving a drive around (e.g. from the spare > slot to its correct location) life isn't so simple. First off, ZFS > does a good job of locating pool components wherever in the array you > move them using the GUID. However, if you change the GPT label and > move it, ZFS will refer to it by the device name instead. Nothing I > have tried will persuade it otherwise. If you leave the label intact > it's now pointing to the wrong slot, which ZFS really doesn't mind > about but this could really ruin your day if you don't know. > > Now FreeBSD 11.0 can flash the ident light on any drive you choose, > by device name (as used by ZFS), I'm seriously wondering if labels > are worth the bother if they can't be relied on. Consider what happen > if a tech pulls two drives and puts them back in the wrong order. ZFS > will carry on regardless, but the label will now identify the wrong > slot. Dangerous! > > Anyone got any thoughts on this? > > Regards, Frank. I'm glad I was able to provide you with one useful clue. The Lucas books assume a fair amount of reader knowledge and follow-up, but they gave me a nice boost up the learning curve and were worth every penny. I probably would not have understood glabel vs. gpart without them. The /boot/loader.conf settings are also present on my FreeBSD 11.0 system. The installer must have set them for me. I agree with the idea of having some kind of identifier other than the automatically generated interface based device node (e.g. /dev/ada0s1) for devices/ virtual devices. It sounds like FreeBSD provides multiple choices and the various subsystems are not well coordinated on their usage (?). I am a SOHO user who has only built a few JBOD and RAID0 arrays. But, now I have four 1.5 TB drives and would like to put them to use with FreeBSD ZFS ZRAID1 or striped mirrors. If you figure out a "one label to rule them all" solution, please post it. (My preference at this point would be whitespace-free strings set by the administrator based on drive function -- e.g. "zraid1a", "zraid1b", "zraid1c", and "zraid1d", or "zmirror0a", "zmirror0b", "zmirror1a", and "zmirror1b" in my case; I plan to attach matching physical labels on the drives themselves. Failing free-form strings, I prefer make/model/serial number.) David