From owner-freebsd-proliant@FreeBSD.ORG Mon Oct 30 20:05:17 2006 Return-Path: X-Original-To: freebsd-proliant@freebsd.org Delivered-To: freebsd-proliant@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8911516A415 for ; Mon, 30 Oct 2006 20:05:17 +0000 (UTC) (envelope-from mikhollo@cisco.com) Received: from sj-iport-6.cisco.com (sj-iport-6.cisco.com [171.71.176.117]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2AAB843D4C for ; Mon, 30 Oct 2006 20:05:15 +0000 (GMT) (envelope-from mikhollo@cisco.com) Received: from sj-dkim-3.cisco.com ([171.71.179.195]) by sj-iport-6.cisco.com with ESMTP; 30 Oct 2006 12:05:15 -0800 Received: from sj-core-5.cisco.com (sj-core-5.cisco.com [171.71.177.238]) by sj-dkim-3.cisco.com (8.12.11.20060308/8.12.11) with ESMTP id k9UK5EfT015318 for ; Mon, 30 Oct 2006 12:05:14 -0800 Received: from [64.101.130.181] ([64.101.130.181]) by sj-core-5.cisco.com (8.12.10/8.12.6) with ESMTP id k9UK5EW4010953 for ; Mon, 30 Oct 2006 12:05:14 -0800 (PST) Mime-Version: 1.0 (Apple Message framework v752.2) In-Reply-To: References: <20061029043926.GI90772@k7.mavetju> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Mike Holloway Date: Mon, 30 Oct 2006 14:03:19 -0600 To: freebsd-proliant@freebsd.org X-Mailer: Apple Mail (2.752.2) DKIM-Signature: a=rsa-sha1; q=dns; l=4086; t=1162238714; x=1163102714; c=relaxed/simple; s=sjdkim3002; h=Content-Type:From:Subject:Content-Transfer-Encoding:MIME-Version; d=cisco.com; i=mikhollo@cisco.com; z=From:Mike=20Holloway=20 |Subject:Re=3A=20RAID=20monitoring=20tools; X=v=3Dcisco.com=3B=20h=3D+pgVBaOaSKnCytkW0o7PqW/IAx0=3D; b=M/u55AQKPctW+lnxkFqyrPdTe2xsTzHLOXPNh1GxZxlYleOix9t8A6AypiYDYXEyfKv2ITZ7 UXsZEcDYxmEayc9yAIdCIiqZGn2UtsE79nhXrxyXEomCW5rRDUEhyu3C; Authentication-Results: sj-dkim-3.cisco.com; header.From=mikhollo@cisco.com; dkim=pass ( sig from cisco.com verified; ); Subject: Re: RAID monitoring tools X-BeenThere: freebsd-proliant@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "Technical discussion of FreeBSD on HP ProLiant server platforms." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Oct 2006 20:05:17 -0000 Whoops, meant to copy the list... Appreciate the pointer to camcontrol, I previously had just been using swatch to watch syslog and send messages to nagios via nsca. My problem was that I never knew the initial state of the disks until an event happened in syslog. For reference, here's what I saw from camcontrol during my tests (FreeBSD 6.0 rel): During normal operation of the raid: # camcontrol inquiry da0 -D pass0: Fixed Direct Access SCSI-0 device After removing one of the raid member disks: # camcontrol inquiry da0 -D pass0: Fixed Direct Access SCSI-0 device After re-inserting the raid member disk: # camcontrol inquiry da0 -D pass0: Fixed Direct Access SCSI-0 device And about 45 minutes later: # camcontrol inquiry da0 -D pass0: Fixed Direct Access SCSI-0 device And here's the configuration I use for swatch to feed nsca in realtime: watchfor /ciss0.*removed/ exec "/usr/local/bin/nsca_report 2 \"Disk Array\" Hot-plug drive removed" watchfor /ciss0.*failure/ exec "/usr/local/bin/nsca_report 2 \"Disk Array\" Physical drive failure" watchfor /ciss0.*inserted/ exec "/usr/local/bin/nsca_report 1 \"Disk Array\" Hot-plug drive inserted" watchfor /ciss0.*recovery->recovering/ exec "/usr/local/bin/nsca_report 1 \"Disk Array\" Drive is rebuilding..." watchfor /ciss0.*recovering->OK/ exec "/usr/local/bin/nsca_report 0 \"Disk Array\" Drive has successfully rebuilt." For completeness, here's the nsca_report script that I use to send the alarms to nagios, substitute your own thishost and -H: #!/bin/bash outcode=$1 thisservice=$2 thishost=`echo $HOSTNAME | sed -e "s/\./ /g" | cut -f 1 -d ' '` shift shift echo -e "${thishost}\t${thisservice}\t${outcode}\t$*\n" | /usr/local/ bin/send_nsca -H www -c /usr/local/etc/send_nsca.cfg 2>&1 >> /dev/null -mike On Oct 29, 2006, at 12:51 AM, Marc G. Fournier wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > camcontrol devlist: > > at scbus0 target 0 lun 0 > (pass0,da0) > > I don't have *regular* monitoring on it, mind you, just do it > periodically, by > hand ... > > > > - --On Sunday, October 29, 2006 15:39:26 +1100 Edwin Groothuis > wrote: > >> Greetings, >> >> Last week we had two failing disks, and if it wasn't for a walk >> through the datacenter (which is off-site, and ten dollars away) >> we wouldn't have noticed it. I've read the thread about hpacucli, >> and have had my failed attempts to get it up and running under the >> linuxolator. >> >> So the question is: how do *you* monitor the status of your disks >> and RAID arrays? Any suggestions will be appriciated. >> >> Edwin >> >> -- >> Edwin Groothuis | Personal website: http:// >> www.mavetju.org >> edwin@mavetju.org | Weblog: http:// >> weblog.barnet.com.au/edwin/ >> _______________________________________________ >> freebsd-proliant@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-proliant >> To unsubscribe, send any mail to "freebsd-proliant- >> unsubscribe@freebsd.org" > > > > - ---- > Marc G. Fournier Hub.Org Networking Services (http:// > www.hub.org) > Email . scrappy@hub.org MSN . > scrappy@hub.org > Yahoo . yscrappy Skype: hub.org ICQ . 7615664 > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.5 (FreeBSD) > > iD8DBQFFRDM94QvfyHIvDvMRAtNdAKC+AYhavYxQ4qZzP4/zqsBfLirE6gCbBebW > Oxd406ykkw1tElrfzn1Y/zM= > =fgIA > -----END PGP SIGNATURE----- > > _______________________________________________ > freebsd-proliant@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-proliant > To unsubscribe, send any mail to "freebsd-proliant- > unsubscribe@freebsd.org"