From owner-freebsd-current@FreeBSD.ORG Thu Mar 12 07:23:29 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3F2B41065670; Thu, 12 Mar 2009 07:23:29 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-bw0-f164.google.com (mail-bw0-f164.google.com [209.85.218.164]) by mx1.freebsd.org (Postfix) with ESMTP id 356BF8FC1F; Thu, 12 Mar 2009 07:23:27 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by bwz8 with SMTP id 8so265264bwz.43 for ; Thu, 12 Mar 2009 00:23:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=nXUZ6MjxtkFCO9Rv5x5aJZP7hwdIxkvXiMjQKHZPjaw=; b=eQepRHMTOGJszMCIOxGu/vSglUT/NkdtgVT/4QOQ7D8yzfo/i6Lk3amQwQTblQePWJ uWKF0Cc+tTiX8JV4xip30nlTwEX38iiJ6WscKpy3vxcc9b8XuxcGciLU3BWdxmMTs1Yo IjS14AZ5USzSdCvYXlvq9VLTpp2O/I61uil04= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=Z+53fZLBQbYk6ONr62CBHDHzW9p4zJ8TTNv0rotbKWprcZIJI4PzmWaGXVjcOquI9W xAJhJTLlqCXqYVwp6S8fsc/lytzYwlXjPXYoHbGY5ZJX5RYGNsPkfEd+GkrjrY8WcHfQ KqFikRMqlkzJBkqoac0e8lEC3ONU7C/JR5QQg= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.223.115.146 with SMTP id i18mr7056657faq.6.1236842605872; Thu, 12 Mar 2009 00:23:25 -0700 (PDT) In-Reply-To: <47d0403c0901161242ifa1f121p5a3583caddd40483@mail.gmail.com> References: <49613379.8040901@vwsoft.com> <47d0403c0901161242ifa1f121p5a3583caddd40483@mail.gmail.com> Date: Thu, 12 Mar 2009 08:23:25 +0100 X-Google-Sender-Auth: a8c6644a0bcd18a6 Message-ID: <3bbf2fe10903120023r12cb29cdmbca516044431dd25@mail.gmail.com> From: Attilio Rao To: Ben Kaduk Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: uwe@grohnwaldt.eu, Martin Wilke , freebsd-current@freebsd.org, kib@freebsd.org, Volker , christoph.mallon@gmx.de Subject: Re: panic in bundirty X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 12 Mar 2009 07:23:29 -0000 2009/1/16, Ben Kaduk : > On Sun, Jan 4, 2009 at 5:08 PM, Volker wrote: > > Gentlemen, > > > > I've had the pleasure to inspect miwi's new tinderbox machine in a panic > > situation. > > > > The debugging info we're able to get out of the box is the following: > > > > panic: bundirty: buffer 0xffffffff9a475438 still on queue 1 > > > > #9 0xffffffff8049f196 in panic (fmt=Variable "fmt" is not available. > > ) at /usr/src/sys/kern/kern_shutdown.c:556 > > #10 0xffffffff8050b540 in bundirty (bp=Variable "bp" is not available. > > ) at /usr/src/sys/kern/vfs_bio.c:1068 > > #11 0xffffffff8050d684 in brelse (bp=0xffffffff9a475438) > > at /usr/src/sys/kern/vfs_bio.c:1388 > > #12 0xffffffff8050f07c in bufdone (bp=0xffffffff9a475438) > > at /usr/src/sys/kern/vfs_bio.c:3157 > > #13 0xffffffff8069cb6c in ffs_backgroundwritedone (bp=0xffffffff9a475438) > > ---Type to continue, or q to quit--- > > at /usr/src/sys/ufs/ffs/ffs_vfsops.c:1679 > > #14 0xffffffff8050f051 in bufdone (bp=0xffffffff9a475438) > > at /usr/src/sys/kern/vfs_bio.c:3151 > > #15 0xffffffff80452b51 in g_io_schedule_up (tp=Variable "tp" is not > > available. > > ) > > at /usr/src/sys/geom/geom_io.c:587 > > #16 0xffffffff8045323f in g_up_procbody () at > > /usr/src/sys/geom/geom_kern.c:95 > > #17 0xffffffff8048037a in fork_exit ( > > callout=0xffffffff804531d0 , arg=0x0, > > frame=0xffffffff80e63c80) at /usr/src/sys/kern/kern_fork.c:789 > > > > in frame 11, 'p *bp' gives: > > $1 = {b_bufobj = 0xffffff00034fe958, b_bcount = 16384, b_caller1 = 0x0, > > b_data = 0xffffffff9f4b7000 "", b_error = 5, b_iocmd = 2 '\002', > > b_ioflags = 2 '\002', b_iooffset = 7127891968, b_resid = 16384, > > b_iodone = 0, b_blkno = 13921664, b_offset = 7127891968, b_bobufs = { > > tqe_next = 0xffffffff9a505a38, tqe_prev = 0xffffff00034fe9a8}, > > b_left = 0x0, b_right = 0xffffffff9a505a38, b_vflags = 0, b_freelist = { > > tqe_next = 0xffffffff9a472db8, tqe_prev = 0xffffffff80b56210}, > > b_qindex = 1, b_flags = 41092, b_xflags = 33 '!', b_lock = > > {lock_object = { > > lo_name = 0xffffffff8083ce18 "bufwait", > > lo_type = 0xffffffff8083ce18 "bufwait", lo_flags = 91947008, > > lo_witness_data = {lod_list = {stqe_next = 0xffffffff80ae6f80}, > > lod_witness = 0xffffffff80ae6f80}}, lk_lock = 18446744073709551608, > > lk_recurse = 0, lk_timo = 0, lk_pri = 80}, b_bufsize = 16384, > > b_runningbufspace = 0, b_kvabase = 0xffffffff9f4b7000 "", b_kvasize = > > 16384, > > b_lblkno = 13921664, b_vp = 0xffffff00034fe820, b_dirtyoff = 0, > > b_dirtyend = 0, b_rcred = 0x0, b_wcred = 0x0, > > b_saveaddr = 0xffffffff9f4b7000, b_pager = {pg_reqpage = 0}, b_cluster = { > > cluster_head = {tqh_first = 0xffffffff9a479c68, > > tqh_last = 0xffffffff9a4901b0}, cluster_entry = { > > tqe_next = 0xffffffff9a479c68, tqe_prev = 0xffffffff9a4901b0}}, > > b_pages = {0xffffff00de0eafa0, 0xffffff00de0eb010, 0xffffff00de0eb080, > > 0xffffff00de0eb0f0, 0x0 }, b_npages = 4, b_dep = { > > lh_first = 0x0}, b_fsprivate1 = 0x0, b_fsprivate2 = 0x0, > > b_fsprivate3 = 0x0, b_pin_count = 0} > > > > This panic does not occur before commit 176708. > > > > The panic is in sys/kern/vfs_bio.c bundirty 1055: > > KASSERT( bp->b_flags & B_REMFREE || bp->b_qindex == QUEUE_NONE ... > > > > bp->b_qindex = 0x01 (QUEUE_FREE) > > bp->b_flags = 0xa084 (B_ASYNC | B_DELWRI | B_NOCACHE | B_INVAL) > > > > My assumption is: > > > > brele knows about the QUEUE_FREE but bundirty only wants to operate on > > QUEUE_NONE. Either this is a race condition, brele should not call > > bundirty or bundirty should operate not just on QUEUE_NONE buffers. That's right. It seems that this bug crept in while adds to the brelse() have been done during the time. IMHO the right thing to do is to move the cleanup of B_DELWRI and vnode diassociation upside. This is the patch: http://www.freebsd.org/~attilio/vfs_bio.diff Let me know if you can review and test this patch. Thanks, Attilio -- Peace can only be achieved by understanding - A. Einstein