From owner-freebsd-current@FreeBSD.ORG Thu Oct 18 09:59:05 2007 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0C38716A419 for ; Thu, 18 Oct 2007 09:59:05 +0000 (UTC) (envelope-from avleeuwen@gmail.com) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.180]) by mx1.freebsd.org (Postfix) with ESMTP id C803F13C465 for ; Thu, 18 Oct 2007 09:59:04 +0000 (UTC) (envelope-from avleeuwen@gmail.com) Received: by wa-out-1112.google.com with SMTP id k17so142571waf for ; Thu, 18 Oct 2007 02:59:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:references; bh=1y89v1IDavm1egQu0Cj/3Yv6a7YU7REkDBhOM+2K250=; b=Ehs2Nfrp1I8CqwrQeD3zKDxoD7MU87kFoT25FQWjvGicCMGrTVHJsWtVHyO12IcmEeJj7hbnjZT18ZH07AqHVdpHHy6jXyYK77kFcP7lgv58kZLkyd4SDaQBrbl6yWWnE1K7lV5NdbgOGqlDvbOnM9wbF0gPpLHaFlOFeyV+02s= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:references; b=pcX2uGujnURrJdxR+0CCgRYF+ZoL6GKcOT36zvFX10L8dFjJDkZPQ4ImnKmo9tJLQROmnoiNOlzH2Y7qCdKmKX+n1Lsv195xOxMBVqISyKjeOZNJgnUkKlFXMzyoF45U+FITArUqUpwsh0215ksokfUY5qIzKfaDGsWD79GB2D8= Received: by 10.114.152.17 with SMTP id z17mr418310wad.1192701543533; Thu, 18 Oct 2007 02:59:03 -0700 (PDT) Received: by 10.114.133.11 with HTTP; Thu, 18 Oct 2007 02:59:03 -0700 (PDT) Message-ID: Date: Thu, 18 Oct 2007 11:59:03 +0200 From: "Arjan van Leeuwen" To: "Kostik Belousov" In-Reply-To: <20071018093948.GN6511@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 References: <1191175387.92510.6.camel@shumai.marcuscom.com> <200710131857.46963.pieter@degoeje.nl> <471170E1.2090205@freebsd.org> <200710140408.46121.pieter@degoeje.nl> <471427BE.10500@freebsd.org> <20071016112907.GD6511@deviant.kiev.zoral.com.ua> <4715FAA7.7030404@freebsd.org> <20071018093948.GN6511@deviant.kiev.zoral.com.ua> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Pieter de Goeje , freebsd-current@freebsd.org, Rolf Witt Subject: Re: panic: ffs_blkfree: freeing free block X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: avleeuwen@piwebs.com List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Oct 2007 09:59:05 -0000 Hey Kostik, 2007/10/18, Kostik Belousov : > > On Wed, Oct 17, 2007 at 07:05:59AM -0500, Eric Anderson wrote: > > Kostik Belousov wrote: > > >On Tue, Oct 16, 2007 at 12:20:59PM +0200, Arjan van Leeuwen wrote: > > >>2007/10/16, Eric Anderson : > > >>>Pieter de Goeje wrote: > > >>>>On Sunday 14 October 2007, Eric Anderson wrote: > > >>>>>Pieter de Goeje wrote: > > >>>>>>On Saturday 13 October 2007, Rolf Witt wrote: > > >>>>>>>Backtrace from crash tonight (after portsnap cron): > > >>>>>> > > >>>>>> > > >>>>>>># uname -a > > >>>>>>>FreeBSD peanuts.homenet.local 7.0-PRERELEASE FreeBSD > 7.0-PRERELEASE > > >>>>>>>#121: Thu Oct 11 16:29:05 CEST 2007 > > >>>>>>>rowi@peanuts.homenet.local:/usr/obj/usr/src/sys/PEANUTS i386 > > >>>>>>> > > >>>>>>>Sources from Builddate. > > >>>>>>> > > >>>>>>>Rowi > > >>>>>>Eric sent me this patch: > > >>>>>>http://www.googlebit.com/freebsd/patches/ffs_softdep.c-patch > > >>>>>>which seems to be working great so far. I am still testing it, but > I > > >>>>>>think it fixed the problem. > > >>>>>>To apply, cd /usr/src; patch < /path/to/patch and then rebuild the > > >>>>>>kernel. > > >>>>>> > > >>>>>>Cheers, > > >>>>>>Pieter de Goeje > > >>>>>It doesn't actually 'fix' the problem, but I think it helps > identify > > >>>it. > > >>>>>I'm not 100% certain if this is the right fix our not, but so far > > >>>>>feedback has been good when running with this patch. > > >>>>> > > >>>>>Can somebody confirm that this patch is ok? > > >>>>> > > >>>>>Eric > > >>>>Can you elaborate on what this patch exactly does / what the problem > is? > > >>>> > > >>>>Pieter > > >>> > > >>>I think the problem is that blocks are being put in the worklist > twice, > > >>>but I'm not certain why yet. The patch reduces the chance of this > > >>>happening by more often removing the block from the worklist instead > of > > >>>leaving it on. I think actually the patch is hiding the real issue > more > > >>>than anything, which is why I said it isn't fixing the problem, but > > >>>because of the reports I've seen, I think we're sniffing in the right > > >>>area. I'll be looking more into this soon, when I get another few > > >>>minutes of free time and my debugger.. > > >>I actually seem to be getting the panic more often with the patch (i.e > . > > >>twice in just 30 minutes with the patch, about twice per day without > the > > >>patch), while doing the same things as usual. The panic usually > happens > > >>when > > >>I'm compiling or linking. Is that strange? > > > > > >Patch is wrong. Actually, it does put the dirrem to the proccessing > twice > > >when xp != NULL. > > > > Yes, exactly why I said 'it doesn't fix the problem'. In fact, I > > actually did not post it to the list, and didn't intend for it to be > > posted. I was hoping for feedback for further examination. > > > > Kostik - do you have any ideas on what is going on here? Did you see > > the message in another thread that looked similar (it was regarding > > gjournal)? > > No, I have no idea. BTW, some further information from the people that > experience the problem could be helpful. For instance, > UFS1/UFS2 UFS2, filesystem created with FreeBSD 6-STABLE size of the problematic fs/% of the space used /dev/ad4s1g 90613368 22619950 60744350 27% /home are quotas enabled in kernel/active for the fs ? No to both. are softupdates on ? Yes. Turning them off seems to fix the problem (fingers crossed - I only turned it off this morning, didn't have a panic yet) what block number is reported as being freed twice ? is it always the same ? > what is the full fsck diagnostic after reboot ? I'm in X when it happens, so I can't see the message. It doesn't say which block is being freed on reboot. I haven't saved the fsck output either. I can turn softupdates back on tonight and reproduce the panic to get a full fsck log, if you want me to. The range of questions is so wide because I try to somewhat localize the > search field down from "anywhere". > > np :) Arjan