From owner-freebsd-current@freebsd.org Fri Dec 4 13:23:21 2015 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 259A0A413B1 for ; Fri, 4 Dec 2015 13:23:21 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id CC21C1B9F for ; Fri, 4 Dec 2015 13:23:19 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:RLko3xb7DSZatbTagvYYd4H/LSx+4OfEezUN459isYplN5qZpcqybnLW6fgltlLVR4KTs6sC0LqI9fi4EUU7or+/81k6OKRWUBEEjchE1ycBO+WiTXPBEfjxciYhF95DXlI2t1uyMExSBdqsLwaK+i760zceF13FOBZvIaytQ8iJ35rxj7j60qaQSjsLrQL1Wal1IhSyoFeZnegtqqwmFJwMzADUqGBDYeVcyDAgD1uSmxHh+pX4p8Y7oGx48sgs/M9YUKj8Y79wDfkBVGxnYCgJ45jOvAXOUBCdrlQBQGQV2k5NAQfY/hD5dpj0uyr+8OF63X/JE9fxSOUOWD+hp4JiQxzshSJPYyQ8+WrUjsF1pL9crw+sowR/hYXdNtLGfMFid7/QKItJDVFKWdxcAmkYWtux X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2DQAQAKk2FW/61jaINWCIQNbga9PgENgW4XCoUjSgKBYRQBAQEBAQEBAYEJgi2CBwEBAQMBAQEBIAQnIAsFCwIBCA4DAwECAQICDRkCAiEGAQkeCAIECAcEARwEh3kDCggNr3mMDA2EQQEBAQEBAQEBAQEBAQEBAQEBAQEagQGFU4R9glOBYwUBAYM6gUQFjh+IQoUthSN1g1JJg3qJFIVih1cCHwEBQoIRHYF0IDQHhCc6gQcBAQE X-IronPort-AV: E=Sophos;i="5.20,380,1444708800"; d="scan'208";a="255736395" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-annu.net.uoguelph.ca with ESMTP; 04 Dec 2015 08:23:18 -0500 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id A33EB15F55D; Fri, 4 Dec 2015 08:23:18 -0500 (EST) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id GHDtrndRnAfE; Fri, 4 Dec 2015 08:23:17 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id CBB6615F565; Fri, 4 Dec 2015 08:23:17 -0500 (EST) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id YnT2c_waYhEO; Fri, 4 Dec 2015 08:23:17 -0500 (EST) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id ADA5215F55D; Fri, 4 Dec 2015 08:23:17 -0500 (EST) Date: Fri, 4 Dec 2015 08:23:17 -0500 (EST) From: Rick Macklem To: Mateusz Guzik Cc: Kirk McKusick , FreeBSD Current Message-ID: <43332766.118512946.1449235397400.JavaMail.zimbra@uoguelph.ca> In-Reply-To: <20151204025102.GA15514@dft-labs.eu> References: <20151203224752.GA19134@dft-labs.eu> <201512032307.tB3N7mMl001027@chez.mckusick.com> <20151204025102.GA15514@dft-labs.eu> Subject: Re: panic "ffs_checkblk: bad block" on recent -head kernels MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.12] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF34 (Win)/8.0.9_GA_6191) Thread-Topic: panic "ffs_checkblk: bad block" on recent -head kernels Thread-Index: 8bp8r+BfzQp/jWQL+d7eMmHwmKGtUg== X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Dec 2015 13:23:21 -0000 Mateusz Guzik wrote: > On Thu, Dec 03, 2015 at 03:07:48PM -0800, Kirk McKusick wrote: > > > Date: Thu, 3 Dec 2015 23:47:52 +0100 > > > From: Mateusz Guzik > > > To: Rick Macklem > > > Cc: FreeBSD Current > > > Subject: Re: panic "ffs_checkblk: bad block" on recent -head kernels > > > > > > On Thu, Dec 03, 2015 at 05:08:27PM -0500, Rick Macklem wrote: > > >> Hi, > > >> > > >> I get a fairly reproducible panic when doing a full kernel build > > >> on a 256Mbyte single core i386 when running recent kernels from -head. > > >> > > >> The panic is "ffs_checkblk: bad block ..". I don't actually have the > > >> block # (although I think it's just 0xfffffffffffffff, given the > > >> backtrace), > > >> because it runs off the screen. (I looked up the message via the > > >> debugger > > >> from the first arg. to panic.) > > >> > > >> Here's the backtrace without all the numbers: > > >> panic(c14f4b55, ffffffff, ffffffff, 0, 64,...) > > >> ffs_checkblk(ffffffff, 8000, fffffff9c, ffffffff, c4a02454,...) > > >> ffs_reallocblks > > >> VOP_REALLOCBLKS_APV > > >> cluster_write > > >> ffs_write > > >> VOP_WRITE_APV > > >> vn_write > > >> vn_io_fault_doio > > >> vn_io_fault1 > > >> vn_io_fault > > >> dofilewrite > > >> kern_writev > > >> sys_write > > >> syscall > > >> > > >> It doesn't happen on a kernel dated Sep. 30, but does happen on a Nov. > > >> 30 one. > > >> (I was away from home, so I didn't upgrade kernels for 2 months.) > > >> > > >> I am slowly doing a binary search for the first kernel rev. where it > > >> occurs, > > >> but since each build takes hours, it's going to take a while;-). > > >> > > >> At this point, it doesn't appear to happen on r289278 (just before > > >> jeff@'s buffer > > >> cache patch). > > >> With kernels between r289279-->r290480, I get into the "R" state that > > >> was fixed by r290481 before I get a crash. > > >> I tried reverting r289405 and r290047 from a recent kernel and the > > >> crashes still > > >> occurred, so it doesn't appear to be these commits. > > >> > > >> I am currently testing r290481 to see if the crash occurs for this rev. > > >> > > >> If anyone has some insight into which commit might cause this, > > >> please let me know. > > > > > > Well, did it crash with r291460 or later? > > > > > > If so, try the kernel just before that and if that helps, try: > > > > > > diff --git a/sys/kern/vfs_subr.c b/sys/kern/vfs_subr.c > > > index ff37de8..0ad6ef7 100644 > > > --- a/sys/kern/vfs_subr.c > > > +++ b/sys/kern/vfs_subr.c > > > @@ -2783,6 +2783,7 @@ _vdrop(struct vnode *vp, bool locked) > > > vp->v_op = NULL; > > > #endif > > > bzero(&vp->v_un, sizeof(vp->v_un)); > > > + vp->v_lasta = vp->v_clen = vp->v_cstart = vp->v_lastw = 0; > > > vp->v_iflag = 0; > > > vp->v_vflag = 0; > > > bo->bo_flag = 0; > > > > > > -- > > > Mateusz Guzik > > > > I concur with trying this suggestion. starting with r291460 these > > fields were no longer zero'ed when allocating the vnode. So you may > > have some residual values in there that are causing trouble. > Good work. This does seem to fix the problem. Without this patch, the crash would occur on pretty well every full kernel build attaempt. With the patch, I've done 3 full kernel builds without a crash. I see the fix has already hit head. Thanks guys, rick > I reviewed the rest of the structure, looks like this is the rest of the > fallout. > > -- > Mateusz Guzik > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >