From owner-freebsd-questions@FreeBSD.ORG Fri Apr 24 22:23:56 2009 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 718511065672; Fri, 24 Apr 2009 22:23:56 +0000 (UTC) (envelope-from psteele@maxiscale.com) Received: from exprod7og103.obsmtp.com (exprod7og103.obsmtp.com [64.18.2.159]) by mx1.freebsd.org (Postfix) with SMTP id 141318FC0A; Fri, 24 Apr 2009 22:23:55 +0000 (UTC) (envelope-from psteele@maxiscale.com) Received: from source ([209.85.200.170]) by exprod7ob103.postini.com ([64.18.6.12]) with SMTP ID DSNKSfI7+5Tgi+ylUhf8lK96clLug5G4F4VT@postini.com; Fri, 24 Apr 2009 15:23:56 PDT Received: by wf-out-1314.google.com with SMTP id 25so1073037wfc.30 for ; Fri, 24 Apr 2009 15:23:55 -0700 (PDT) Received: by 10.142.216.18 with SMTP id o18mr856990wfg.312.1240611835303; Fri, 24 Apr 2009 15:23:55 -0700 (PDT) Received: from localhost ([76.231.178.131]) by mx.google.com with ESMTPS id 24sm433261wff.31.2009.04.24.15.23.54 (version=SSLv3 cipher=RC4-MD5); Fri, 24 Apr 2009 15:23:54 -0700 (PDT) Date: Fri, 24 Apr 2009 15:23:53 -0700 (PDT) From: Peter Steele To: Ivan Voras Message-ID: <30737365.3061240611830997.JavaMail.HALO$@halo> In-Reply-To: <14044628.3041240611737554.JavaMail.HALO$@halo> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-questions@freebsd.org Subject: Re: Unexpected gmirror behavior: Is this a bug? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 Apr 2009 22:23:56 -0000 > By "kicked out" you mean "overwritten"? > >You should definitely look at "gmirror list" before and after. Sorry for the confusion. By "kicked out", what I meant was as gmirror started up it took ad4 as the principal member, saw that it was previously part of a mirror with three other drives and tried to add those drives. These drives could not be added for some reason so the system eventually completed the process leaving a degraded mirror with only 1/4 members active. When the system completed booted, a gmirror list showed that the mirror consisted of only a single member of the expected four members. We have software that runs automatically when a system has booted to make sure all drives are partipating in the mirror. In this case it discovered 3 of the 4 drives were missing and proceeded to add them back in. This is where their old data gets destroyed of course. If I go through this exact same process with any of the other drives everything works as it should--that drive gets reinserted and none of the other drives lose any data. The problem only occurs when the drive that's pulled is the first drive, which in our case is ad4.