From owner-freebsd-fs@FreeBSD.ORG Thu Nov 20 10:46:23 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B1CF4B90 for ; Thu, 20 Nov 2014 10:46:23 +0000 (UTC) Received: from smtp-sofia.digsys.bg (smtp-sofia.digsys.bg [193.68.21.123]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "smtp-sofia.digsys.bg", Issuer "Digital Systems Operational CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DCFB5FD for ; Thu, 20 Nov 2014 10:46:22 +0000 (UTC) Received: from dcave.digsys.bg (dcave.digsys.bg [193.68.6.1]) (authenticated bits=0) by smtp-sofia.digsys.bg (8.14.6/8.14.6) with ESMTP id sAKAXava024662 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Thu, 20 Nov 2014 12:33:37 +0200 (EET) (envelope-from daniel@digsys.bg) Message-ID: <546DC380.1040707@digsys.bg> Date: Thu, 20 Nov 2014 12:33:36 +0200 From: Daniel Kalchev User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.2.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: Re: ZFS and glabel References: <1422065A4E115F409E22C1EC9EDAFBA4220D0DB7@sofdc01exc02.postbank.bg> In-Reply-To: <1422065A4E115F409E22C1EC9EDAFBA4220D0DB7@sofdc01exc02.postbank.bg> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Nov 2014 10:46:23 -0000 Hi Ivailo, The FreeBSD glabel is in bit of a mess, indeed. It is a mess not because the tech is bad or buggy (although there are caveats), but because the glabel tool had made it all too confusing by displaying them all together. Or perhaps our assumptions are wrong if one needs to be more precise. We have several kinds of labels. Each of them lives under it's own namespace (a subdir of /dev). There are the glabel type, which you manipulate with glabel - this lives under /dev/label. There are the geom labels that you manipulate via gpart that live under /dev/gpt. There are the gmirror labels that you manipulate with gmirror and live under /dev/mirror. There are the disk ID labels that live under /dev/diskid. There are the UFS labels that you manipulate with newfs/tunefs that live under /dev/ufs. Perhaps there are others I missed.. Then comes ZFS. For it's own sanity, ZFS would label the devices it's given with it's own labels -- so that when you reboot or move the pool to another machine it still finds it's members and structure. If it can find its own labels, that is... As a consequence of this, the safest way to use ZFS is with whole devices. This pretty much guarantees your ZFS pool will be portable across any system and ZFS will *always* be able to find it, no matter what. The drawback is you might not know for sure which device id is which physical drive, because many factors might influence device name reordering. But this is pretty much the only drawback. The diskid should work in a similar way. On systems that don't have disk ids, you will fall back to the device name, so no big deal. The next "safest" thing is the GPT label, which you create with gpart. Many systems (non FreeBSD) support it and your pool will be just fine there. Worst are glabel and gmirror, mostly because they have trouble being nested. But as long as you stick to some simple rules, these work ok too. What you are seeing is when you destroy the label, ZFS can no longer find it's own labels. This is because when you destroy the label ZFS has no idea w where to look for it -- what the offset would be. If in your example, you recreate the label again, that pool will suddenly work again -- even if you use different name for the new label -- the ZFS's own label will be then discoverable again. I myself prefer either raw disks or GPT. The later especially in smaller systems, where I would use GPT for boot partitions anyway. But also on systems with tens of drives, where I need to know the physical location of the drive (and not care much about it's serial number at that moment, which would be the case of using diskid labels). On these systems, I would label the GPT partition with chasis/position name. By the way, I still have few systems that use glabels (dev/label). Daniel On 17.11.14 14:42, Ivailo A. Tanusheff wrote: > Dear all, > > I run to an interesting issue and I would like to discuss it with all of you. > The whole thing began with me trying to identify available HDD to include in a zfs pool through a script/program. > I assumed that the easiest way of doing this is using glabel. For example: > > root@FreeBSD:~ # glabel status > Name Status Components > gptid/248e758c-e267-11e3-95bb-08002796202b N/A ada0p1 > diskid/DISK-VBdd471206-91164057 N/A ada5 > diskid/DISK-VBe98b5e75-0d8cf6dc N/A ada8 > diskid/DISK-VB7d006584-01beca12 N/A ada6 > diskid/DISK-VB721029c3-66a60156 N/A ada7 > diskid/DISK-VB31481dbb-639540a1 N/A ada2 > diskid/DISK-VB95921208-4eb19f41 N/A ada4 > > So far it is OK and if I create pool like zpool create xxx ada4 then the line for ada4 will disappear from the glabel status. > As far as I remember though it is not recommended to use production pools based on the device naming, so I wanted to switch to gpt lable, i.e. diskid/DISK-VB95921208-4eb19f41. > When I recreate pool like: > zpool create xxx diskid/DISK-VB95921208-4eb19f41 the pool is created without problems, but the device does not disappear from the glabel status list, thus making my program running wrong. > Is this a problem with the zfs implementation, my server or the general idea is wrong? > > BTW, if I label the disk additionally, like: > glabel create VB95921208-4eb19f41 ada4 > zpool create xxx label/VB95921208-4eb19f41 > > The glabel status again shows the right information. The problem with the latest approach is that if someone executes: > glabel destroy -f VB95921208-4eb19f41 > > The result becomes: > pool: xxx > state: UNAVAIL > status: One or more devices are faulted in response to IO failures. > action: Make sure the affected devices are connected, then run 'zpool clear'. > see: http://illumos.org/msg/ZFS-8000-HC > scan: none requested > config: > > NAME STATE READ WRITE CKSUM > xxx UNAVAIL 0 0 0 > 6968348230421469155 REMOVED 0 0 0 was /dev/label/VB95921208-4eb19f41 > > And the data is practically unrecoverable. > > So my questions are: > - Is there a way to make glabel to show the right data when I use diskid/DISK-VB95921208-4eb19f41 > - Which is the most proper way of creating vdevs - with disk name (ada4), diskid (diskid/DISK-VB95921208-4eb19f41) or manual labeling? > - How may I found which disks are free, if the diskid approach is the best solution? > > > Regards, > > Ivailo Tanusheff > > Disclaimer: > > This communication is confidential. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication by mistake, please notify us immediately by responding to this email and then delete it from your system. > Eurobank Bulgaria AD is not responsible for, nor endorses, any opinion, recommendation, conclusion, solicitation, offer or agreement or any information contained in this communication. > Eurobank Bulgaria AD cannot accept any responsibility for the accuracy or completeness of this message as it has been transmitted over a public network. If you suspect that the message may have been intercepted or amended, please call the sender. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >