From owner-freebsd-current@FreeBSD.ORG  Wed Jun  9 04:08:00 2004
Return-Path: <owner-freebsd-current@FreeBSD.ORG>
Delivered-To: freebsd-current@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id C6B8C16A4CE
	for <freebsd-current@freebsd.org>;
	Wed,  9 Jun 2004 04:08:00 +0000 (GMT)
Received: from fledge.watson.org (fledge.watson.org [204.156.12.50])
	by mx1.FreeBSD.org (Postfix) with ESMTP id 7AAE243D4C
	for <freebsd-current@freebsd.org>;
	Wed,  9 Jun 2004 04:08:00 +0000 (GMT)
	(envelope-from robert@fledge.watson.org)
Received: from fledge.watson.org (localhost [127.0.0.1])
	by fledge.watson.org (8.12.11/8.12.11) with ESMTP id i59470oF093628;
	Wed, 9 Jun 2004 00:07:00 -0400 (EDT)
	(envelope-from robert@fledge.watson.org)
Received: from localhost (robert@localhost)i59470UX093625;
	Wed, 9 Jun 2004 00:07:00 -0400 (EDT)
	(envelope-from robert@fledge.watson.org)
Date: Wed, 9 Jun 2004 00:07:00 -0400 (EDT)
From: Robert Watson <rwatson@freebsd.org>
X-Sender: robert@fledge.watson.org
To: Randy Bush <randy@psg.com>
In-Reply-To: <16582.31631.362797.734568@ran.psg.com>
Message-ID: <Pine.NEB.3.96L.1040609000437.93490B-100000@fledge.watson.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
cc: FreeBSD Current <freebsd-current@freebsd.org>
Subject: Re: snapshot dump hangs
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.1
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
	<freebsd-current.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>,
	<mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>,
	<mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 09 Jun 2004 04:08:00 -0000


On Tue, 8 Jun 2004, Randy Bush wrote:

> i try to dump a live filesys with
> 
>    /sbin/rdump 0Luaf remote.sys:/backup/foo/var.dump  /dev/twed0s1d
> 
> the screen does
> 
>   DUMP: Connection to remote.sys established.
>   DUMP: Date of this level 0 dump: Wed Jun  9 01:52:32 2004
>   DUMP: Date of last level 0 dump: the epoch
>   DUMP: Dumping snapshot of /dev/twed0s1d (/var) to /backup/foo/var.dump on host remote.sys
>   DUMP: mapping (Pass I) [regular files]
>   DUMP: mapping (Pass II) [directories]
>   DUMP: estimated 513616 tape blocks.
>   DUMP: dumping (Pass III) [directories]
>   DUMP: dumping (Pass IV) [regular files]
> 
> and then just hangs forever
> 
> there is nothing in /var/.snap

dump unlinks the snapshot as soon as it opens it so that it will be GC'd
when dump is done...

> i can
> 
>     mount -u -o snapshot /var/.snap/snap1 /var

And this doesn't hang, or does?

> this is horrifyingly reproducable and happens with other source
> filesystems. 

I'm using dump with -L regularly with a recent -CURRENT box at work for
backups.  On the whole, it's worked without problems, but I did once
observe dump hanging, in the sbwait state (suggesting it's blocked on
socket I/O?).  At the time, I was unable to do much diagnosis.  Could you
do a couple of things:

(1) Could you do a "ps awxl" and see what wait channel dump is blocked on?

(2) Could you break into DDB and generate a stack trace for dump?

(3) Could you run "show lockedvnods" in DDB and show the results?

(4) Could you run "show locks <pid>" on the dump process?

Thanks!

Robert N M Watson             FreeBSD Core Team, TrustedBSD Projects
robert@fledge.watson.org      Senior Research Scientist, McAfee Research