From owner-freebsd-fs@FreeBSD.ORG Wed Jan 23 02:43:24 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 21726674 for ; Wed, 23 Jan 2013 02:43:23 +0000 (UTC) (envelope-from jas@cse.yorku.ca) Received: from nm18-vm3.bullet.mail.gq1.yahoo.com (nm18-vm3.bullet.mail.gq1.yahoo.com [98.136.217.218]) by mx1.freebsd.org (Postfix) with SMTP id 69A762D5 for ; Wed, 23 Jan 2013 02:43:22 +0000 (UTC) Received: from [98.137.12.191] by nm18.bullet.mail.gq1.yahoo.com with NNFMP; 23 Jan 2013 02:40:06 -0000 Received: from [208.71.42.214] by tm12.bullet.mail.gq1.yahoo.com with NNFMP; 23 Jan 2013 02:40:06 -0000 Received: from [127.0.0.1] by smtp225.mail.gq1.yahoo.com with NNFMP; 23 Jan 2013 02:40:06 -0000 X-Yahoo-Newman-Id: 97992.44294.bm@smtp225.mail.gq1.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: wjSIrusVM1nrNDmCG_kkLmu6QGd2XeaXwDAy0SYNIPG4GEl WZnzE_ZRTvnjdaQl2BL5vMmxxGJkqtWDoW_fPBK6FnFaYbAnUmEgCnFY6fNM N0fSKYEsuvGVWkKD3BhDbkM7Qvk.TH79_6.iX7.M7q4R383G5yk7BlzPPz_i a5_N5kdqRowoMbyupWDoTnTCcrmrihicQ5b0v3Y0LfXVDTldzSFSRM3Q4RJ_ sp24BeOW36C5I6d8mBP3EqgfbvVY4bNnrotMnpUpTOwRemltaiqTW3En4imP 1gTFDg5abuWn1cNScU7mFj7wHmSXC.T1LJffyKB7LA7M39LsE9XDxXb6E3Nr 45wW7Jydjp8l7X2yhtLgQRLc2V543nHtrkLDYG7fT.TlkuW_iEWVwsJfazIg bvRRtA0ydtP8EIllw1bTgC3YjHmkfLCQgxsWJhtoMkl8v_ga3B52Pxkb7HpY fZS1UJ8L3XB5EQy2Wyf9h2LQgZx.Yg6U- X-Yahoo-SMTP: hdvk3SuswBDjqWuLIhjJ7cQT_83YtZNiMmKQOSuhvZGxXQ-- Received: from [192.168.1.100] (jas@99.237.135.117 with plain) by smtp225.mail.gq1.yahoo.com with SMTP; 22 Jan 2013 18:40:05 -0800 PST Message-ID: <50FF4D87.5080404@cse.yorku.ca> Date: Tue, 22 Jan 2013 21:40:07 -0500 From: Jason Keltz User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: Warren Block Subject: Re: RFC: Suggesting ZFS "best practices" in FreeBSD References: <314B600D-E8E6-4300-B60F-33D5FA5A39CF@sarenet.es> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Filesystems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2013 02:43:24 -0000 On 22/01/2013 9:16 PM, Warren Block wrote: > On Tue, 22 Jan 2013, Michael DeMan wrote: > >> On Jan 22, 2013, at 7:04 AM, Warren Block wrote: >>> >>> I'm a proponent of using various types of labels, but my impression >>> after a recent experience was that ZFS metadata was enough to >>> identify the drives even if they were moved around. That is, ZFS >>> bare metadata on a drive with no other partitioning or labels. >>> >>> Is that incorrect? >> >> I don't know if it is correct or not, but the best I could figure out >> was to both label the drives and also force the mapping so the >> physical and logical drives always show up associated correctly. I >> also ended up deciding I wanted the hostname as a prefix for the >> labels - so if they get moved around to say another machine I can >> look and know what is going on - 'oh yeah, those disks are from the >> ones we moved over to this machine'... > > It helps to avoid duplicate labels, a good idea. > >> #1. Map the physical drive slots to how they show up in FBSD so if a >> disk is removed and the machine is rebooted all the disks after that >> removed one do not have an 'off by one error'. i.e. if you have >> ada0-ada14 and remove ada8 then reboot - normally FBSD skips that >> missing ada8 drive and the next drive (that used to be ada9) is now >> called ada8 and so on... > > How do you do that? If I'm in that situation, I think I could find > the bad drive, or at least the good ones, with diskinfo and the drive > serial number. One suggestion I saw somewhere was to use disk serial > numbers for label values. I think that was using /boot/device.hints. Unfortunately it only works for some systems, and not for all.. and someone shared an experience with me where a kernel update caused the card probe order to change, the devices to change, and then it all broke... It worked for one card, not for the other... I gave up because I wanted consistency across different systems.. In my own opinion, the whole process of partitioning drives, labelling them, all kinds of tricks for dealing with 4k drives, manually configuring /boot/device.hints, etc. is something that we have to do, but honestly, I really believe there *has* to be a better way.... Years back when I was using a 3ware/AMCC RAID card (actually, I AM still using a few), none of this was an issue... every disk just appeared in order.. I didn't have to configure anything specially .. ordering never changed when I removed a drive, I didn't need to partition or do anything with the disks - just give it the raw disks, and it knew what to do... If anything, I took my labeller and labelled the disk bays with a numeric label so when I got an error, I knew which disk to pull, but order never changed, and I always pulled the right drive... Now, I look at my pricey "new" system, see disks ordered by default in what seems like an almost "random" order... I dded each drive to figure out the exact ordering, and labelled the disks, but it just gets really annoying.... > >> #2. Use gpart+gnop to deal with 4K disk sizes in a standardized way >> and also to leave a little extra room so if when doing a replacement >> disk and that disk is a few MB smaller than the original - it all >> 'just works'. (All disks are partitioned to a slightly smaller size >> than their physical capacity). > > I've been told (but have not personally verified) that newer versions > of ZFS actually leaves some unused space at the end of a drive to > allow for variations in nominally-sized drives. Don't know how much. You see... this point has been mentioned on the list a whole bunch of times, and is exactly the type of information that needs to make it into a "best practices". Does anyone know if this applies to ZFS in FreeBSD? From what version? Who knows, maybe a whole bunch of people are partitioning devices that don't need to be! :) Jason. -- Jason Keltz Manager of Development Department of Computer Science and Engineering York University, Toronto, Canada Tel: 416-736-2100 x. 33570 Fax: 416-736-5872