From owner-freebsd-stable@FreeBSD.ORG Thu Sep 11 08:31:37 2014 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8EC12CA7 for ; Thu, 11 Sep 2014 08:31:37 +0000 (UTC) Received: from isis.morrow.me.uk (isis.morrow.me.uk [204.109.63.142]) by mx1.freebsd.org (Postfix) with ESMTP id 57C5F153 for ; Thu, 11 Sep 2014 08:31:37 +0000 (UTC) Received: from anubis.morrow.me.uk (unknown [93.89.81.46]) (Authenticated sender: mauzo) by isis.morrow.me.uk (Postfix) with ESMTPSA id 0183645087 for ; Thu, 11 Sep 2014 07:12:39 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.8.3 isis.morrow.me.uk 0183645087 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=morrow.me.uk; s=dkim201101; t=1410419560; bh=ADrYwbcUeW2ZvLCEpvdJLQTha4QDGw7Ky2nuIMkl5x8=; h=Date:From:To:Subject:References:In-Reply-To; b=R8k16yGDN17L7cvsKr+u1QRpgoZqRYkRcx6roOLHft9bzLpwtsBEOyICDKFEfUscX hK5DoK6OiTQKvWmiehLKD3R0D4BVmE1w97VYs3HFfibU3X7SHSDibIWY76e8LGAYIv 8am73fMtjrm2OKwucnUCDL3K4hWBbCyOtOGMGQXc= X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.98.1 at isis.morrow.me.uk Received: by anubis.morrow.me.uk (Postfix, from userid 5001) id 2335A153D9; Thu, 11 Sep 2014 08:12:37 +0100 (BST) Date: Thu, 11 Sep 2014 08:12:37 +0100 From: Ben Morrow To: freebsd-stable@freebsd.org Subject: Snapshots that won't delete [was: Re: ZFS on root booting...] Message-ID: <20140911071233.GA50585@anubis.morrow.me.uk> Mail-Followup-To: freebsd-stable@freebsd.org References: <7F008C560B48412AB66A1EBD9382DDAE@multiplay.co.uk> <9315C209-701A-49EF-85D3-ACCCD1513EC3@icloud.com> <959C54D2C8EB4AC8983DC1DA3CE042E3@multiplay.co.uk> <9F24DD48FBEA46C39F98DF600D46DA1A@multiplay.co.uk> <4450778127F4407EB6566A0FE11CD651@multiplay.co.uk> <090135D4-8B1F-42B4-82FC-6FD2F1DBDDA8@icloud.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5411159F.6060608@egr.msu.edu> X-Newsgroups: gmane.os.freebsd.stable User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Sep 2014 08:31:37 -0000 Quoth Adam McDougall : > > Were you running a newer kernel with an older format zpool? I heard > ixsystems had customers doing that and ran into corruption when they > tried to modify the zpool in some way (expand? I don't remember). > http://www.bsdnow.tv/episodes/2014_07_09-zfs_war_stories Oh! Might this be what's causing a problem I've been meaning to ask about? My desktop at home is running (a patched, but not anywhere to do with ZFS) 10-STABLE from a while ago, with a zpool that was created under 8.2-R and is still at version 15. I have been deliberately not upgrading it, because I saw no reason to and it seemed safer to leave things as they were. Recently, though, my dump script has started having occasional problems with snapshots that won't delete. Pending further investigation I have been renaming them to allow the recursive delete to succeed, and (so far) rebooting has always made it possible to get rid of them. /home/mauzo% sudo zfs destroy zroot/DATA/R@broken-dump-20140906 cannot destroy snapshot zroot/DATA/R@broken-dump-20140906: dataset is busy /home/mauzo% mount | grep @ /home/mauzo% zfs list -o name,origin | grep broken-dump /home/mauzo% zfs list -o name,userrefs zroot/DATA/R@broken-dump-20140906 NAME USERREFS zroot/DATA/R@broken-dump-20140906 0 /home/mauzo% zfs holds zroot/DATA/R@broken-dump-20140906 NAME TAG TIMESTAMP /home/mauzo% Presumably this is indicative of some sort of serious problem with either the pool or the filesystem, and the only permanent solution is to rebuild the whole thing? ZFS doesn't seem to have a fsck-equivalent for the ZFS layer. I've run a scrub, and it found no problems. I was about to upgrade the machine to the latest 10-STABLE, but I can put that off if anyone thinks this is worth investigating. Once I reboot I'm fairly sure the current stuck snapshots will fix themselves, and of course I can't be sure when this will happen again. [Also: it would be really nice to have an alias for 'zfs destroy' that will only destroy snapshots. When running something like 'zfs destroy -r zroot/DATA@whatever' I find I have to check at least three times that I have definitely put that @ in the right place before pressing Return. *Especially* when I'm doing it because I'm running a dump, and it's just fallen over because it can't delete a snapshot.] Ben