From owner-freebsd-fs@FreeBSD.ORG Fri Oct 14 09:00:03 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6CCF61065672 for ; Fri, 14 Oct 2011 09:00:03 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from QMTA11.westchester.pa.mail.comcast.net (qmta11.westchester.pa.mail.comcast.net [76.96.59.211]) by mx1.freebsd.org (Postfix) with ESMTP id 06D348FC0C for ; Fri, 14 Oct 2011 09:00:02 +0000 (UTC) Received: from omta20.westchester.pa.mail.comcast.net ([76.96.62.71]) by QMTA11.westchester.pa.mail.comcast.net with comcast id kZ011h0021YDfWL5BZ03BA; Fri, 14 Oct 2011 09:00:03 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta20.westchester.pa.mail.comcast.net with comcast id kZ021h00F1t3BNj3gZ0201; Fri, 14 Oct 2011 09:00:02 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id BC00C102C1C; Fri, 14 Oct 2011 02:00:00 -0700 (PDT) Date: Fri, 14 Oct 2011 02:00:00 -0700 From: Jeremy Chadwick To: Attila Nagy Message-ID: <20111014090000.GA66602@icarus.home.lan> References: <4E97F710.8000004@fsn.hu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4E97F710.8000004@fsn.hu> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@freebsd.org Subject: Re: cache devices come up as dsk/original_device_name in zpools X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Oct 2011 09:00:03 -0000 On Fri, Oct 14, 2011 at 10:47:12AM +0200, Attila Nagy wrote: > Hi, > > A have a zpool with cache devices on 8-STABLE (csuped and compiled > at Sep 14 15:01:25 CEST 2011). The problem is every time I reboot, > the cache devices turn to UNAVAIL (because device name changes to > dsk/daXX): > dsk/da37 UNAVAIL 0 0 0 cannot open > dsk/da38 UNAVAIL 0 0 0 cannot open > > After removing and re-adding them, everyting goes back to normal, > until the next reboot. I have no /boot/zfs/zpool.cache (because the > machine is netbooted), maybe this is the cause? In previous versions > everything was fine. Obviously at some point when you built this system you entered "dsk/da37" and "dsk/da38". So the metadata on those drives probably contains references to those strings. You need to clear/change that. I'm not sure how to go about doing that, especially on a system which lacks /boot/zfs/zpool.cache. A one-time "zpool export" then a reboot, I imagine, would suffice, but I'm not sure if export actually changes the metadata on the disk itself or just updates the zpool.cache file. If you ran "zdb" on this system (the output will be HUGE given the number of vdevs and devices you have!), you should see some relevant information under each disk (child), specifically "path" vs. "phys_path". Maybe these differ? You might also try tinkering about with the loader.conf(5) variables zpool_cache_*. Depending on your setup, you might be able to move the zpool.cache file to a different location -- I realise you PXE boot, but if you have any sort of storage media on that system that isn't under ZFS that *is* available (e.g. a small UFS partition, etc.) then you might consider storing it there. See /boot/defaults/loader.conf. Otherwise I'm not sure how to go about changing the actual strings in the disk metadata. Maybe remove the cache devices entirely, zero out the first and last ~16MBytes of the da37 and da38 disks (using dd), then re-add them using their "daXX" name? That might suffice. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB |