From owner-freebsd-current@FreeBSD.ORG Wed Dec 1 11:12:12 2010 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AC418106566C for ; Wed, 1 Dec 2010 11:12:12 +0000 (UTC) (envelope-from yanegomi@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 3E2508FC1F for ; Wed, 1 Dec 2010 11:12:11 +0000 (UTC) Received: by wwf26 with SMTP id 26so2605257wwf.31 for ; Wed, 01 Dec 2010 03:12:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=CRWu7SRJ0gSNUgxpEZTPluewke630mNBtQq29BieCDE=; b=rOtZzYkQu3gZaf9ILl2yIxM/fcxuUEVkGNUTzXFwE73b+Xntvuf81UfiFuA/iuEYtN KIeC+yax/dZSHD2jzk5q5dtCs/iADl1yqiD2+0FjDnpskXy4epthcQteLbrEzJbpgR9Z Xh6fBxB51g82TEaivnnz1iCX41fuDt3jibHL4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=LJz4jzXJxPsdG9vL38thFE6CdB/sgafcMiw46dN6edtDFxfxlQo1I440egSVTOU203 6ScE1n3xOYpos4weQsVnq1bvjNVb4AXCZybl98jZNacFs4x1OVAJAPpsO4rUI6Pct1Ju WD5x0yndp3RTwV1LCvTZZMhKz13uFxxDKw2+M= MIME-Version: 1.0 Received: by 10.216.175.18 with SMTP id y18mr8779496wel.30.1291201930870; Wed, 01 Dec 2010 03:12:10 -0800 (PST) Received: by 10.216.198.27 with HTTP; Wed, 1 Dec 2010 03:12:10 -0800 (PST) In-Reply-To: <20101201110008.GA50719@x2.osted.lan> References: <1FA8A18C-9350-4C2D-B034-768566ACB718@gmail.com> <20101201110008.GA50719@x2.osted.lan> Date: Wed, 1 Dec 2010 03:12:10 -0800 Message-ID: From: Garrett Cooper To: Peter Holm Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Marshall Kirk McKusick , Kostik Belousov , current@freebsd.org Subject: Re: How a full fsck screwed up my SU+J filesystem X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Dec 2010 11:12:12 -0000 On Wed, Dec 1, 2010 at 3:00 AM, Peter Holm wrote: > On Wed, Dec 01, 2010 at 01:28:06AM -0800, Garrett Cooper wrote: >> =A0 =A0 =A0 So... I was doing a portmaster -af today because vlc stopped= playing audio (for some reason ... I kind of went on a pkg_cutleaves rampa= ge and probably deinstalled too much stuff), and the machine hardlocked dur= ing an upgrade. I did a soft reboot and saw messages along the lines of "yo= ur journal and filesystem mount time mismatched; running a full fsck". I fi= gured "ok, sure..." and let it do it's thing. Problem was that it pruned a = lot of stuff from my /usr partition -- including the .sujournal !!! So now = it's stuck at Mounting local file systems: stating: >> >> Failed to find journal. =A0 Use tunefs to create one >> Failed to start journal: 2 >> >> =A0 =A0 =A0 (I assume the 2 means ENOENT). All of the above were printf(= 9)'s from the kernel. >> =A0 =A0 =A0 Now the machine won't continue in multiuser mode (doesn't re= spond to interrupts, no panic, etc). Going into ddb, I don't see anything i= n info_threads (just a bunch of references to sched_switch, a few to fork_t= rampoline, cpustop_handler, and kdb_enter). I'm going to try and massage th= e machine back to life from single user mode, but the fact that this died i= n this way (i.e. .sujournal getting nuked by a full fsck) is a bit disheart= ening for SU+J :(... It would be nice if at least the fsck aborted before g= oing and nuking the journal :/... (or at the very least if the file wasn't = removable -- i.e. SF_NOUNLINK). >> =A0 =A0 =A0 Here's to hoping I can resuscitate the filesystem... > > Thank you for reporting this. > > I was able to reproduce the problem by: > > tunefs -j enable /dev/md5a > mount /dev/md5a /mnt > chflags 0 /mnt/.sujournal > rm -f /mnt/.sujournal > umount /mnt > mount /dev/md5a /mnt > > The mount(1) is now stuck in mntref. > > http://people.freebsd.org/~pho/stress/log/kostik404.txt > > A sequence of "tunefs -j disable" + "tunefs -j enable" should get > you going. I had to do fsck -y again, unmount the partition, then tunefs -j enable it, and then I was able to get it to work. Looks like some files were lost -- it would be interesting to see what the degree of damage was. Thankfully it was just /usr (mostly /usr/local because I was recompiling ports) and not my more critical data partition(s). Thanks! -Garrett