From owner-freebsd-fs@FreeBSD.ORG Wed Jun 12 08:44:59 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id CA585B42 for ; Wed, 12 Jun 2013 08:44:59 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from mail-la0-x233.google.com (mail-la0-x233.google.com [IPv6:2a00:1450:4010:c03::233]) by mx1.freebsd.org (Postfix) with ESMTP id 5371219CF for ; Wed, 12 Jun 2013 08:44:59 +0000 (UTC) Received: by mail-la0-f51.google.com with SMTP id fq12so7489521lab.10 for ; Wed, 12 Jun 2013 01:44:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=vNvD5TNxquZYbGiKpfDWpMnowuHfmxgp/1BvQYFDZcc=; b=t4WXZ5o0CxZv9/kwfNrSqH5h5F0utLJsEh/pcNSbT1P3yo/Qh1uox58y1RVM8xua/8 6w8o2NQOW9bb1hrSNLWdbLhCWkcEI39DBUnIGMLOlIlvRt42w6cc3iL0DIIzJS3s2j+N 5lbgCqGwd61IY+1SKWO6OftgKDX3cnf3nvuQ/NXFgEJtZIFAHlJJ0s9YFBPZwdpnAuZE SfWcbcTgnrluj7Sp6tBqdY2DUYv/SCPR4hohQAnNCMojLCBH/AY+Y0q2KGPEl7LuJ87h NhDs3LVnLtbAbXqY/AQVucIRpyMEuxFa4lIMaLRgMLK4TZBxt8bH3aaziVaUVwzyOeeL rmoA== X-Received: by 10.112.150.170 with SMTP id uj10mr5406999lbb.93.1371026697991; Wed, 12 Jun 2013 01:44:57 -0700 (PDT) Received: from localhost ([188.230.122.226]) by mx.google.com with ESMTPSA id a3sm8737907lbg.2.2013.06.12.01.44.55 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 12 Jun 2013 01:44:56 -0700 (PDT) Date: Wed, 12 Jun 2013 11:44:54 +0300 From: Mikolaj Golub To: Dmitry Morozovsky Subject: Re: hast: can't restore after disk failure Message-ID: <20130612084453.GA55502@gmail.com> References: <20130610201650.GA2823@gmail.com> <20130611060741.GA42231@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 12 Jun 2013 08:44:59 -0000 On Wed, Jun 12, 2013 at 12:23:52AM +0400, Dmitry Morozovsky wrote: > On Tue, 11 Jun 2013, Mikolaj Golub wrote: > > > On Tue, Jun 11, 2013 at 12:40:08AM +0400, Dmitry Morozovsky wrote: > > > On Mon, 10 Jun 2013, Mikolaj Golub wrote: > > > > > > [snipall] > > > > > > > > Jun 10 16:56:20 cthulhu3 kernel: Jun 10 16:56:20 > > > > > cthulhu3 hastd[765]: [d1] (secondary) Worker process exited ungracefully > > > > > (pid=14380, exitcode=66). > > > > > > > > > > Any hints? Thanks! > > > > > > > > Have you run hastctl create to initialize metadata? > > > > > > Yes, but did it naively: > > > > > > hastctl create d1 > > > > No errors? > > no visible, but hast instance ungracefully exits > > > > and status still reported 0 as provider size... > > > > I assume /dev/ada1p1 is present and readable/writable? > > > > Symptoms are like if it did not exist. > > nope, it does: > > root@cthulhu3:/# diskinfo /dev/ada1p1 > /dev/ada1p1 512 999654686720 1952450560 0 1048576 1936954 16 63 > root@cthulhu3:/# diskinfo /dev/ada0p1 > /dev/ada0p1 512 999653638144 1952448512 0 1048576 1936952 16 63 > Hm, looking in the source where this error is generated: cthulhu3 hastd[14379]: [d1] (secondary) Unable to read metadata from /dev/ada1p1: No such file or directory. it looks like hastd successfully read metadata from disk but failed to parse it (did not found an entry). This usually happens when metadata is not initialized by `hastctl create`. Does `hastctl dump d1' not work too? -- Mikolaj Golub