Date: Mon, 4 Mar 2013 12:32:07 -0500 (EST) From: "Lawrence K. Chen, P.Eng." <lkchen@ksu.edu> To: Gustau =?utf-8?Q?P=C3=A9rez?= i Querol <gperez@entel.upc.edu> Cc: FreeBSD FS <freebsd-fs@freebsd.org> Subject: Re: benefit of GEOM labels for ZFS, was Hard drive device names... serial numbers Message-ID: <629654633.22396164.1362418327720.JavaMail.root@k-state.edu> In-Reply-To: <513216C6.6030108@entel.upc.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
----- Original Message ----- > Al 01/03/2013 23:52, En/na Lawrence K. Chen, P.Eng. ha escrit: > > I only have 15 drives...(12 HDDs and 3 SSDs) but the ordering of > > drives seemed to randomize on every boot (wonder now if the > > controller was doing some kind of staggering in spin ups. And, > > their other drivers cope with it. They provide a v1.1 driver for > > FreeBSD 7.2 or source to the v1.0 driver.) And, then everything > > moved around when I changed controllers a few times. > > > I had resorted at one point to putting device.hints to force all > > the > > drives to keep their mapping. Which caused problems elsewhere, and > > a mess when I added another controller. But, then I changed to > > more > > meaningful GPT labels and exported and re-imported my zpools with > > '-d /dev/gpt', and now things are ok. > > That reordering issue is what made me switch to geom labels first > (IIRC I did by 5.x era) and next switched to GPT labels. The GPT > allows me first to easily used those drives with ZFS and specially > because those labels belong to the partition scheme and thus are > filesystem independent. > What I found specially useful is to be able to identify those drives. > When I drive fails I know where it is because of the simple name > convention I use; but having a blinking led helps a lot, specially > when writing the recovery plan for the rest of the team. > Gus The GPT labels have come to be even more handy, because I had to switch out one (of two 5-bay) enclosures for a cheaper Rosewill one, due to a failing powersupply. The Rosewill one doesn't have LEDs. L From owner-freebsd-fs@FreeBSD.ORG Mon Mar 4 18:05:25 2013 Return-Path: <owner-freebsd-fs@FreeBSD.ORG> Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 7199BD81 for <freebsd-fs@freebsd.org>; Mon, 4 Mar 2013 18:05:25 +0000 (UTC) (envelope-from lkchen@k-state.edu) Received: from ksu-out.merit.edu (ksu-out.merit.edu [207.75.117.132]) by mx1.freebsd.org (Postfix) with ESMTP id 3E8C1B3F for <freebsd-fs@freebsd.org>; Mon, 4 Mar 2013 18:05:25 +0000 (UTC) X-Merit-ExtLoop1: 1 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Av8EAN/hNFHPS3TT/2dsb2JhbABEhlC9AhZzgh8BAQQBI0QSBQcPDgoCAg0ZAlkGiCAGsi+JM4g6gSOQHoETA4hsnkaBUoFUggk X-IronPort-AV: E=Sophos;i="4.84,781,1355115600"; d="scan'208";a="906285647" X-MERIT-SOURCE: KSU Received: from ksu-sfpop-mailstore02.merit.edu ([207.75.116.211]) by sfpop-ironport05.merit.edu with ESMTP; 04 Mar 2013 13:05:13 -0500 Date: Mon, 4 Mar 2013 13:05:13 -0500 (EST) From: "Lawrence K. Chen, P.Eng." <lkchen@ksu.edu> To: Charles Sprickman <spork@bway.net> Message-ID: <1234083675.22411680.1362420313611.JavaMail.root@k-state.edu> In-Reply-To: <6F419054-DC4E-43A7-8879-37BE54D10A47@bway.net> Subject: Re: benefit of GEOM labels for ZFS, was Hard drive device names... serial numbers MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [129.130.0.181] X-Mailer: Zimbra 7.2.2_GA_2852 (ZimbraWebClient - GC25 ([unknown])/7.2.2_GA_2852) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems <freebsd-fs.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-fs>, <mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>; List-Post: <mailto:freebsd-fs@freebsd.org> List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>, <mailto:freebsd-fs-request@freebsd.org?subject=subscribe> X-List-Received-Date: Mon, 04 Mar 2013 18:05:25 -0000 ----- Original Message ----- > On Mar 4, 2013, at 5:38 AM, Daniel Kalchev wrote: > > > What you do when the disk is dead, and you don't know which one it > > is in a rather big rack full of disks? > > > > Perhaps, you offline each and every disk in the system until you > > eliminate all but one? :) > > One thing that I recently discovered is led(4). > > I have no idea how ubiquitous support for this is, but I see it on > both a Supermicro and a Tyan board. In /dev/led, things are > labelled like so: > > [spork@util2 ~]$ ls -l /dev/led/ > total 0 > crw------- 1 root wheel 0, 40 Feb 9 00:20 ahcich0.fault > crw------- 1 root wheel 0, 39 Feb 9 00:20 ahcich0.locate > crw------- 1 root wheel 0, 42 Feb 9 00:20 ahcich1.fault > crw------- 1 root wheel 0, 41 Feb 9 00:20 ahcich1.locate > crw------- 1 root wheel 0, 44 Feb 9 00:20 ahcich2.fault > crw------- 1 root wheel 0, 43 Feb 9 00:20 ahcich2.locate > crw------- 1 root wheel 0, 46 Feb 9 00:20 ahcich3.fault > crw------- 1 root wheel 0, 45 Feb 9 00:20 ahcich3.locate > crw------- 1 root wheel 0, 48 Feb 9 00:20 ahcich4.fault > crw------- 1 root wheel 0, 47 Feb 9 00:20 ahcich4.locate > crw------- 1 root wheel 0, 50 Feb 9 00:20 ahcich5.fault > crw------- 1 root wheel 0, 49 Feb 9 00:20 ahcich5.locate > > If you pair that up with boot messages, you can probably sort out > which drive is which: > > [spork@util2 ~]$ grep "at ahcich1" /var/run/dmesg.boot > ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 > > And then you can blink the "locate" LED on the sas/sata backplane: > > root@util2:/home/spork # echo f > /dev/led/ahcich0.locate > > And disable the blinking after you're done: > > root@util2:/home/spork # echo 0 > /dev/led/ahcich0.locate > > I'm sure this is all very hardware dependent, but if you have > supported hardware, it's an easy way to find what's where. > > Charles > > Hmmm, interesting.... > ls -l /dev/led total 0 crw------- 1 root wheel 0, 59 Feb 27 23:28 ahcich0.fault crw------- 1 root wheel 0, 58 Feb 27 23:28 ahcich0.locate crw------- 1 root wheel 0, 61 Feb 27 23:28 ahcich1.fault crw------- 1 root wheel 0, 60 Feb 27 23:28 ahcich1.locate crw------- 1 root wheel 0, 63 Feb 27 23:28 ahcich2.fault crw------- 1 root wheel 0, 62 Feb 27 23:28 ahcich2.locate crw------- 1 root wheel 0, 65 Feb 27 23:28 ahcich3.fault crw------- 1 root wheel 0, 64 Feb 27 23:28 ahcich3.locate crw------- 1 root wheel 0, 67 Feb 27 23:28 ahcich4.fault crw------- 1 root wheel 0, 66 Feb 27 23:28 ahcich4.locate crw------- 1 root wheel 0, 69 Feb 27 23:28 ahcich5.fault crw------- 1 root wheel 0, 68 Feb 27 23:28 ahcich5.locate crw------- 1 root wheel 0, 48 Feb 27 23:28 siisch0 crw------- 1 root wheel 0, 49 Feb 27 23:28 siisch1 crw------- 1 root wheel 0, 54 Feb 27 23:28 siisch2 crw------- 1 root wheel 0, 55 Feb 27 23:28 siisch3 crw------- 1 root wheel 0, 56 Feb 27 23:28 siisch4 crw------- 1 root wheel 0, 57 Feb 27 23:28 siisch5 ahcich is the mobo controller, wonder where the LEDs are...don't recall seeing any when I've been poking around inside. But, the 10 external drives are on the siisch controllers. Actually 5 on siisch0 and 5 on siisch1. > dmesg | grep 'siisch0' siisch0: <SIIS channel> at channel 0 on siis0 pmp0 at siisch0 bus 0 scbus4 target 15 lun 0 ada4 at siisch0 bus 0 scbus4 target 0 lun 0 ada5 at siisch0 bus 0 scbus4 target 1 lun 0 ada6 at siisch0 bus 0 scbus4 target 2 lun 0 ada7 at siisch0 bus 0 scbus4 target 3 lun 0 ada8 at siisch0 bus 0 scbus4 target 4 lun 0 The enclosure (SANS Digital) on siisch0 has 2 LEDs at each drive position and a LED on the front for each. The enclosure on siisch1 was cheaper, and lacks the LEDs, but uses the same carriers, etc. I suppose if it really mattered, I could move the SANS Digital enclosure from my old fileserver to here, making both enclosures on old server the cheaper Rosewill ones.... Though I haven't figured out how to get persistent drive identifiers or such on ubuntu....and, since I'm just doing RAID1 across, and separate VGs....when a drive fails in a RAID set...either there's an LED or its the corresponding one in the other enclosure.... siisch[01] is a Sil3132 and siisch[2345] is a Sil3124 Though I'm thinking of switch one or both for asm1061 based controllers. Had put the idea on hold at first, but now that my video card is no longer supported by the latest nvidia-driver...I'm considering whether I want to risk getting a newer card (it was the 6th card I tried before I had something mostly working...) Only some of my drives are SATA III though. But all 3 SSDs are....and think the ones I'm using for ZIL/L2ARC might benefit. Perhaps I need 3 asm1061's.... L
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?629654633.22396164.1362418327720.JavaMail.root>