From owner-freebsd-stable@FreeBSD.ORG Sat Feb 21 09:52:32 2009 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3BF82106564A for ; Sat, 21 Feb 2009 09:52:32 +0000 (UTC) (envelope-from daichi@freebsd.org) Received: from natial.ongs.co.jp (natial.ongs.co.jp [202.216.246.90]) by mx1.freebsd.org (Postfix) with ESMTP id A0E848FC0C for ; Sat, 21 Feb 2009 09:52:31 +0000 (UTC) (envelope-from daichi@freebsd.org) Received: from parancell.ongs.co.jp (dullmdaler.ongs.co.jp [202.216.246.94]) by natial.ongs.co.jp (Postfix) with ESMTPSA id 1562C125422; Sat, 21 Feb 2009 18:35:17 +0900 (JST) Message-ID: <499FCAD4.8000006@freebsd.org> Date: Sat, 21 Feb 2009 18:35:16 +0900 From: Daichi GOTO User-Agent: Thunderbird 2.0.0.19 (X11/20090201) MIME-Version: 1.0 To: Steve Wills References: <95E82F15-2535-4C67-BDF0-44CFC7EB9FBB@mouf.net> In-Reply-To: <95E82F15-2535-4C67-BDF0-44CFC7EB9FBB@mouf.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Cc: stable@freebsd.org, Masanori OZAWA Subject: Re: unionfs panic in 7.1 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Feb 2009 09:52:32 -0000 Steve Wills wrote: > Hi, > > I've found an reproducable panic in unionfs on 7.1-R. > > /usr/src/sys/fs/unionfs/union_subr.c is: > $FreeBSD: src/sys/fs/unionfs/union_subr.c,v 1.92.2.7.2.2 2008/12/15 > 03:58:55 daichi Exp $ > > kgdb output is below. > > kgdb /boot/kernel/kernel.symbols /var/crash/vmcore.0 > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you > are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"... > > Unread portion of the kernel message buffer: > > stack pointer = 0x10:0xffffffffb0e1a8b0 > frame pointer = 0x10:0xffffffffb0e1a8d0 > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, long 1, def32 0, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 72036 (make) > panic: from debugger > cpuid = 0 > Uptime: 49m32s > Physical memory: 2034 MB > Dumping 441 MB: 426 410 394 378 362 346 330 314 298 282 266 250 234 218 > 202 186 170 154 138 122 106 90 74 58 42 26 10 > > Reading symbols from /boot/kernel/if_tap.ko...Reading symbols from > /boot/kernel/if_tap.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/if_tap.ko > Reading symbols from /boot/kernel/snd_hda.ko...Reading symbols from > /boot/kernel/snd_hda.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/snd_hda.ko > Reading symbols from /boot/kernel/sound.ko...Reading symbols from > /boot/kernel/sound.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/sound.ko > Reading symbols from /boot/kernel/atapicam.ko...Reading symbols from > /boot/kernel/atapicam.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/atapicam.ko > Reading symbols from /boot/kernel/smb.ko...Reading symbols from > /boot/kernel/smb.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/smb.ko > Reading symbols from /boot/kernel/smbus.ko...Reading symbols from > /boot/kernel/smbus.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/smbus.ko > Reading symbols from /boot/kernel/linprocfs.ko...Reading symbols from > /boot/kernel/linprocfs.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/linprocfs.ko > Reading symbols from /boot/kernel/linux.ko...Reading symbols from > /boot/kernel/linux.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/linux.ko > Reading symbols from /boot/kernel/zfs.ko...Reading symbols from > /boot/kernel/zfs.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/zfs.ko > Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from > /boot/kernel/opensolaris.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/opensolaris.ko > Reading symbols from /boot/kernel/if_bridge.ko...Reading symbols from > /boot/kernel/if_bridge.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/if_bridge.ko > Reading symbols from /boot/kernel/bridgestp.ko...Reading symbols from > /boot/kernel/bridgestp.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/bridgestp.ko > Reading symbols from /usr/local/modules/fuse.ko...done. > Loaded symbols for /usr/local/modules/fuse.ko > Reading symbols from /boot/kernel/radeon.ko...Reading symbols from > /boot/kernel/radeon.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/radeon.ko > Reading symbols from /boot/kernel/drm.ko...Reading symbols from > /boot/kernel/drm.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/drm.ko > Reading symbols from /boot/kernel/unionfs.ko...Reading symbols from > /boot/kernel/unionfs.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/unionfs.ko > Reading symbols from /boot/kernel/nullfs.ko...Reading symbols from > /boot/kernel/nullfs.ko.symbols...done. > done. > Loaded symbols for /boot/kernel/nullfs.ko > #0 doadump () at pcpu.h:195 > 195 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) bt > #0 doadump () at pcpu.h:195 > #1 0xffffffff804c70a8 in boot (howto=260) at > /usr/src/sys/kern/kern_shutdown.c:418 > #2 0xffffffff804c750c in panic (fmt=Variable "fmt" is not available. > ) at /usr/src/sys/kern/kern_shutdown.c:574 > #3 0xffffffff801c1707 in db_panic (addr=Variable "addr" is not available. > ) at /usr/src/sys/ddb/db_command.c:446 > #4 0xffffffff801c1d6f in db_command (last_cmdp=0xffffffff80aa6448, > cmd_table=0x0, dopager=1) at /usr/src/sys/ddb/db_command.c:413 > #5 0xffffffff801c1f80 in db_command_loop () at > /usr/src/sys/ddb/db_command.c:466 > #6 0xffffffff801c3b69 in db_trap (type=Variable "type" is not available. > ) at /usr/src/sys/ddb/db_main.c:228 > #7 0xffffffff804f3955 in kdb_trap (type=12, code=0, > tf=0xffffffffb0e1a800) at /usr/src/sys/kern/subr_kdb.c:524 > #8 0xffffffff807a3180 in trap_fatal (frame=0xffffffffb0e1a800, > eva=Variable "eva" is not available. > ) at /usr/src/sys/amd64/amd64/trap.c:759 > #9 0xffffffff807a3554 in trap_pfault (frame=0xffffffffb0e1a800, > usermode=0) at /usr/src/sys/amd64/amd64/trap.c:680 > #10 0xffffffff807a3eda in trap (frame=0xffffffffb0e1a800) at > /usr/src/sys/amd64/amd64/trap.c:449 > #11 0xffffffff80788dde in calltrap () at > /usr/src/sys/amd64/amd64/exception.S:209 > #12 0xffffffff804b9f0e in _mtx_lock_sleep (m=0xffffff006c384638, > tid=18446742974280188784, opts=Variable "opts" is not available. > ) at /usr/src/sys/kern/kern_mutex.c:339 > #13 0xffffffff80555767 in vn_start_write (vp=0xffffff006c4983f0, > mpp=Variable "mpp" is not available. > ) at /usr/src/sys/kern/vfs_vnops.c:902 > #14 0xffffffff805574a1 in vn_close (vp=0xffffff006c4983f0, flags=1, > file_cred=0xffffff0015947500, td=0xffffff0004e74370) at > /usr/src/sys/kern/vfs_vnops.c:287 > #15 0xffffffff80557589 in vn_closefile (fp=0xffffff006c020e00, > td=0xffffff0004e74370) at /usr/src/sys/kern/vfs_vnops.c:867 > #16 0xffffffff8049331f in fdrop (fp=0xffffff006c020e00, > td=0xffffff0004e74370) at file.h:299 > #17 0xffffffff80494579 in closef (fp=0xffffff006c020e00, > td=0xffffff0004e74370) at /usr/src/sys/kern/kern_descrip.c:2033 > #18 0xffffffff80494d5d in kern_close (td=0xffffff0004e74370, fd=Variable > "fd" is not available. > ) at /usr/src/sys/kern/kern_descrip.c:1125 > #19 0xffffffff807a37d6 in syscall (frame=0xffffffffb0e1ac80) at > /usr/src/sys/amd64/amd64/trap.c:907 > #20 0xffffffff80788feb in Xfast_syscall () at > /usr/src/sys/amd64/amd64/exception.S:330 > #21 0x000000000044030c in ?? () > Previous frame inner to this frame (corrupt stack?) > (kgdb) frame 12 > #12 0xffffffff804b9f0e in _mtx_lock_sleep (m=0xffffff006c384638, > tid=18446742974280188784, opts=Variable "opts" is not available. > ) at /usr/src/sys/kern/kern_mutex.c:339 > 339 owner = (struct thread *)(v & ~MTX_FLAGMASK); > (kgdb) list > 334 * If the owner is running on another CPU, spin until the > 335 * owner stops running or the state of the lock changes. > 336 */ > 337 v = m->mtx_lock; > 338 if (v != MTX_UNOWNED) { > 339 owner = (struct thread *)(v & ~MTX_FLAGMASK); > 340 #ifdef ADAPTIVE_GIANT > 341 if (TD_IS_RUNNING(owner)) { > 342 #else > 343 if (m != &Giant && TD_IS_RUNNING(owner)) { > (kgdb) p v > $1 = 0 > (kgdb) p m > $2 = (struct mtx *) 0xffffff006c384638 > (kgdb) p *m > $3 = {lock_object = {lo_name = 0x0, lo_type = 0x0, lo_flags = > 1281410792, lo_witness_data = {lod_list = {stqe_next = 0xffffff004c60c708}, > lod_witness = 0xffffff004c60c708}}, mtx_lock = 0, mtx_recurse = 0} > (kgdb) frame 13 > #13 0xffffffff80555767 in vn_start_write (vp=0xffffff006c4983f0, > mpp=Variable "mpp" is not available. > ) at /usr/src/sys/kern/vfs_vnops.c:902 > 902 MNT_ILOCK(mp); > (kgdb) list > 897 return (0); > 898 } > 899 } > 900 if ((mp = *mpp) == NULL) > 901 return (0); > 902 MNT_ILOCK(mp); > 903 if (vp == NULL) > 904 MNT_REF(mp); > 905 /* > 906 * Check on status of suspension. > (kgdb) p mp > $4 = (struct mount *) 0xffffff006c3845e8 > (kgdb) p *mp > $5 = {mnt_lock = {lk_object = {lo_name = 0x1
bounds>, lo_type = 0xffffffffb1029552 "unionfs", lo_flags = 2969738112, > lo_witness_data = { > lod_list = {stqe_next = 0xffffff003c980000}, lod_witness = > 0xffffff003c980000}}, lk_interlock = 0xffffff0015d46378, lk_flags = 0, > lk_sharecount = 0, > lk_waitcount = 775638512, lk_exclusivecount = -256, lk_prio = -1, > lk_timo = 0, lk_lockholder = 0x0, lk_newlock = 0x0}, mnt_mtx = > {lock_object = { > lo_name = 0x0, lo_type = 0x0, lo_flags = 1281410792, > lo_witness_data = {lod_list = {stqe_next = 0xffffff004c60c708}, > lod_witness = 0xffffff004c60c708}}, > mtx_lock = 0, mtx_recurse = 0}, mnt_gen = 0, mnt_list = {tqe_next = > 0x0, tqe_prev = 0x0}, mnt_op = 0xffffffffb1029552, mnt_vfc = > 0xffffffffb1029552, > mnt_vnodecovered = 0x4390000, mnt_syncer = 0x0, mnt_ref = -2136101936, > mnt_nvnodelist = {tqh_first = 0x80, tqh_last = 0x50000000000000}, > mnt_nvnodelistsize = 51, mnt_writeopcount = 0, mnt_kern_flag = -1, > mnt_flag = 4294967295, mnt_noasync = 0, mnt_opt = 0xffffffff8086e910, > mnt_optnew = 0xffffffff8086e910, mnt_maxsymlinklen = 16973824, > mnt_stat = {f_version = 0, f_type = 0, f_flags = 4, f_bsize = 0, > f_iosize = 18446742976017063016, f_blocks = 4294967297, f_bfree = 0, > f_bavail = 0, f_files = 0, f_ffree = 0, f_syncwrites = 0, > f_asyncwrites = 18446742976013551312, f_syncreads = 0, f_asyncreads > = 18446742976013551424, f_spare = {0, 0, 0, 18446742976013551456, 0, 0, > 0, 0, > 18446744071572912256, 16384}, f_namemax = 784962992, f_owner = > 4294967040, f_fsid = {val = {0, 0}}, > f_charspare = > "\000\000\000\000\000\000\000\000ËE8l\000ˇˇˇËE8l\000ˇˇˇ", '\0' 24 times>, > "\001\000\000\000\000\000\000\000\023\016\206\200ˇˇˇˇ\2001ß\200ˇˇˇˇXy⁄l\000ˇˇˇ", > f_fstypename = "xsv\004\000ˇˇˇ–K8l\000ˇˇˇ", > f_mntfromname = "\030D8l\000ˇˇˇ", '\0' , > "\200o\034\201ˇˇˇˇ∂L\003\003", '\0' , > "∏\227\tJ\000ˇˇˇ∏¨`L\000ˇˇˇ\000\000\000\000\000\000\000\000\b\000\000\000\000\000\000\000 > \031L0\000\000\000", > f_mntonname = "\a", '\0' , > "\023\016\206\200ˇˇˇˇ\023\016\206\200ˇˇˇˇ\000\0009\004", '\0' 12 times>, "‡\230≠\200ˇˇˇˇ@", '\0' , > "P\0003\000\000\000\000\000\000\000ˇˇˇˇˇˇˇˇ"}, mnt_cred = 0x0, mnt_data > = 0xffffffff8086e910, mnt_time = -2138642160, mnt_iosize_max = 16973824, > mnt_export = 0x0, mnt_label = 0x4, mnt_hashseed = 0, mnt_markercnt = > 0, mnt_holdcnt = 1815627896, mnt_holdcntwaiters = -256, > mnt_secondary_writes = 10, > mnt_secondary_accwrites = 0, mnt_gjprovider = 0x0, mnt_explock = > {lk_object = {lo_name = 0x0, lo_type = 0x0, lo_flags = 0, > lo_witness_data = {lod_list = { > stqe_next = 0x0}, lod_witness = 0x0}}, lk_interlock = > 0xffffff006c3848c8, lk_flags = 2589843616, lk_sharecount = -1, > lk_waitcount = -1707370640, > lk_exclusivecount = -1, lk_prio = -1, lk_timo = -1707370720, > lk_lockholder = 0x9, lk_newlock = 0x0}} > (kgdb) > > I reproduce this by unionfs mounting a ports dir used by the ports > tinderbox. If anyone would like more info, please let me know, I have > the core around, and can reproduce, help debug, resend in case my mailer > has mangled this, etc. I have some research around that issue. First I should say, I cannot judge that unionfs leads that problem or not. JIMO, I guess that is not depends on unionfs itself. Very similar panis is reported without unionfs. At least, I cannot figure out what is the main cause of this problem. sorry. > Thanks, > Steve > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" -- Daichi GOTO, http://people.freebsd.org/~daichi