From owner-freebsd-current@FreeBSD.ORG Mon Jan 20 16:46:37 2014 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5F575695 for ; Mon, 20 Jan 2014 16:46:37 +0000 (UTC) Received: from mail-vc0-x231.google.com (mail-vc0-x231.google.com [IPv6:2607:f8b0:400c:c03::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 1A12A127E for ; Mon, 20 Jan 2014 16:46:37 +0000 (UTC) Received: by mail-vc0-f177.google.com with SMTP id if11so2879306vcb.22 for ; Mon, 20 Jan 2014 08:46:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=jJ1ciaXFvoq9DGPLnerZhgfI5tyKMscmYt/p5sHy3pQ=; b=Chm1MD0ZG5JVp2G72gIB0VrakB7BqUQuENaF2tIEmpABgH1Usv+Tl/rOcxtx5gbKsA 8nMdMMWe9y+8llWAJ37jcRTAWkBdZe7SCczdHlzLD9N7Kf/CTmXHitPoL+IFNP1gaHQJ j2uNgU046A9bpVox0euqzNtLMod4GDmMxu9035XN+QE9RI//Y41H8HiSD0rbPKd7CYW7 CrNgta9GZRaKk2SN1WxcQRtMD7wpRNKoIBdzK4/umL3buQs3qZBgm8IN4O1Y0xzSr1AI JPMI8F0v1Idn5Sh8dZZGtq94DDaDNcYIl0Df2KYJtbjbxuy1kgeQXYY6EdP+O6j1BlpB isjQ== MIME-Version: 1.0 X-Received: by 10.58.201.169 with SMTP id kb9mr200493vec.42.1390236396103; Mon, 20 Jan 2014 08:46:36 -0800 (PST) Received: by 10.220.168.135 with HTTP; Mon, 20 Jan 2014 08:46:36 -0800 (PST) In-Reply-To: <20140120163750.GC24664@kib.kiev.ua> References: <20140120153615.GS1789@albert.catwhisker.org> <20140120163750.GC24664@kib.kiev.ua> Date: Mon, 20 Jan 2014 11:46:36 -0500 Message-ID: Subject: Re: panic: mtx_lock_spin: recursed on non-recursive mutex uart_hwmtx... From: Thomas Hoffmann To: current@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.17 Cc: Konstantin Belousov X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jan 2014 16:46:37 -0000 On Mon, Jan 20, 2014 at 11:37 AM, Konstantin Belousov wrote: > On Mon, Jan 20, 2014 at 07:36:15AM -0800, David Wolfskill wrote: > > Jan 20 07:02:16 freebeast syslogd: exiting on signal 15 > > Waiting (max 60 seconds) for system process `vnlru' to stop...done > > Waiting (max 60 seconds) for system process `syncer' to stop... > > Syncing disks, vnodes remaining...2 4 4 4 3 3 3 2 1 1 0 0 0 0 done > > Waiting (max 60 seconds) for system process `bufdaemon' to stop...done > > All buffers synced. > > lock order reversal: > > 1st 0xc7336c68 ufs (ufs) @ /usr/src/sys/kern/vfs_mount.c:1237 > > 2nd 0xc76846dc devfs (devfs) @ /usr/src/sys/ufs/ffs/ffs_vfsops.c:1412 > > KDB: stack backtrace: > > db_trace_self_wrapper(c112d57c,a3231,e1fb9888,c0b1bfe7,1,...) at > db_trace_self_wrapper+0x2d/frame 0xe1fb9828 > > kdb_backtrace(c1131131,c76846dc,c1124024,c6dad2d0,c1162dc1,...) at > kdb_backtrace+0x30/frame 0xe1fb9890 > > witness_checkorder(c76846dc,9,c1162dc1,584,0,...) at > witness_checkorder+0xc8a/frame 0xe1fb98e0 > > __lockmgr_args(c76846dc,80400,c76846fc,0,0,0,c1162dc1,584) at > __lockmgr_args+0x844/frame 0xe1fb99b4 > > vop_stdlock(e1fb9a28,c6dad3a0,c6dad408,c169bc38,e1fb9a4c,...) at > vop_stdlock+0x4d/frame 0xe1fb99e4 > > VOP_LOCK1_APV(c13828dc,e1fb9a28,0,0,c13cded0,...) at > VOP_LOCK1_APV+0x104/frame 0xe1fb9a10 > > _vn_lock(c76846a8,80400,c1162dc1,584,c1393aac,...) at > _vn_lock+0xa1/frame 0xe1fb9a50 > > ffs_flushfiles(c753f000,a,c6f0a310,e1fb9b38,c0d416e3,...) at > ffs_flushfiles+0x14c/frame 0xe1fb9a9c > > softdep_flushfiles(c753f000,2,c6f0a310,0,0,...) at > softdep_flushfiles+0x6e/frame 0xe1fb9af0 > > ffs_unmount(c753f000,80000,e1fb9b70,518,c6f0a310,...) at > ffs_unmount+0x194/frame 0xe1fb9b38 > > dounmount(c753f000,80000,c6f0a310,0,e1db3eb0,...) at > dounmount+0x4dc/frame 0xe1fb9b98 > > vfs_unmountall(e1db4058,0,c112895a,144,c6d8b1a0,...) at > vfs_unmountall+0x4e/frame 0xe1fb9bb8 > > kern_reboot(4008,0,c112895a,c3,0,...) at kern_reboot+0x5af/frame > 0xe1fb9c20 > > sys_reboot(c6f0a310,e1fb9cc8,14,c112b5b7,7a2,...) at > sys_reboot+0x6f/frame 0xe1fb9c40 > > syscall(e1fb9d08) at syscall+0x2de/frame 0xe1fb9cfc > > Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xe1fb9cfc > > --- syscall (55, FreeBSD ELF32, sys_reboot), eip = 0x805a9df, esp = > 0xbfbfd88c, ebp = 0xbfbfd960 --- > > Uptime: 4m28s > > panic: mtx_lock_spin: recursed on non-recursive mutex uart_hwmtx @ > /usr/src/sys/dev/uart/uart_cpu.h:94 > > > > cpuid = 0 > > KDB: stack backtrace: > > db_trace_self_wrapper(c112d57c,2f727375,2f637273,2f737973,2f766564,...) > at db_trace_self_wrapper+0x2d/frame 0xe1fb9a60 > > kdb_backtrace(c12ec1f3,0,c11269a2,e1fb9b38,e1fb9b08,...) at > kdb_backtrace+0x30/frame 0xe1fb9ac8 > > vpanic(c140d0c8,100,c11269a2,e1fb9b38,e1fb9b38,...) at > vpanic+0x11f/frame 0xe1fb9b08 > > kassert_panic(c11269a2,c10f66d8,c10f6758,5e,8,...) at > kassert_panic+0xea/frame 0xe1fb9b2c > > __mtx_lock_spin_flags(c7291830,0,c10f6758,5e,c137d684,...) at > __mtx_lock_spin_flags+0x190/frame 0xe1fb9b5c > > ns8250_bus_grab(c7291800,c7062b28,c137d684,5e,c140b9a4,...) at > ns8250_bus_grab+0x40/frame 0xe1fb9b78 > > uart_grab(c13fd070) at uart_grab+0x83/frame 0xe1fb9b9c > > uart_cngrab(c137d69c,88888889,e1fb9c20,c0ac7d43,c1128e43,...) at > uart_cngrab+0x12/frame 0xe1fb9ba8 > > cngrab(c1128e43,1c,c112895a,144,c6d8b1a0,...) at cngrab+0x33/frame > 0xe1fb9bb8 > > kern_reboot(4008,0,c112895a,c3,0,...) at kern_reboot+0x6d3/frame > 0xe1fb9c20 > > sys_reboot(c6f0a310,e1fb9cc8,14,c112b5b7,7a2,...) at > sys_reboot+0x6f/frame 0xe1fb9c40 > > syscall(e1fb9d08) at syscall+0x2de/frame 0xe1fb9cfc > > Xint0x80_syscall() at Xint0x80_syscall+0x21/frame 0xe1fb9cfc > > --- syscall (55, FreeBSD ELF32, sys_reboot), eip = 0x805a9df, esp = > 0xbfbfd88c, ebp = 0xbfbfd960 --- > > KDB: enter: panic > > [ thread pid 1 tid 100002 ] > > Stopped at kdb_enter+0x3d: movl $0,kdb_why > > db> show locks > > exclusive sleep mutex Giant (Giant) r = 0 (0xc1724910) locked @ > /usr/src/sys/vm/swap_pager.c:2365 > > db> > > > > > > When I issued "reset", the machine came back up normally (in slice > > 1 -- stable/9). I then rebooted to slice 4 (head), and issued the > > same commands I usually do to set the default boot slice back to 1, then > > "shutdown -p now" -- and re-created the panic. > > > > I can leave the machine up for a while if anyone has suggestions for me > > to poke around. I have a local private mirror of the FreeBSD SVN repos > > and a spare bootable slice; I'm williing to try patches. The machine > > isn't especially fast, but it's generally OK. > > > > If it would be worthwhile, I could reboot my laptop to slice 4 (head) & > > attempt the same reset-to-slice-1-default & power-off. > > > > This definitely did not happen @r260875, and it appears to be quite > > repeatable @r260904 > > You do not have option WITNESS_SKIPSPIN in your kernel config, do you ? > My recent source upgrade from 10.0-RELEASE r260689 to 11.0-CURRENT r260850 left me with a GENERIC kernel with the following defined: options WITNESS # Enable checks to detect deadlocks and cycles options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed why? -Tom