From owner-freebsd-current@FreeBSD.ORG Wed Sep 9 19:08:58 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2D10E1065676 for ; Wed, 9 Sep 2009 19:08:58 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id F19698FC0C for ; Wed, 9 Sep 2009 19:08:57 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id A0CE346B46; Wed, 9 Sep 2009 15:08:57 -0400 (EDT) Received: from jhbbsd.hudson-trading.com (unknown [209.249.190.8]) by bigwig.baldwin.cx (Postfix) with ESMTPA id CD11B8A020; Wed, 9 Sep 2009 15:08:56 -0400 (EDT) From: John Baldwin To: freebsd-current@freebsd.org Date: Wed, 9 Sep 2009 15:07:32 -0400 User-Agent: KMail/1.9.7 References: <4AA7D4BA.40002@demig.de> In-Reply-To: <4AA7D4BA.40002@demig.de> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-15" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200909091507.32600.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Wed, 09 Sep 2009 15:08:56 -0400 (EDT) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=4.2 tests=AWL,BAYES_00,RDNS_NONE autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Norbert Koch Subject: Re: lock order reversal etc. X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Sep 2009 19:08:58 -0000 On Wednesday 09 September 2009 12:15:54 pm Norbert Koch wrote: > Hello. > > Is this the right mailing list to post > kernel stack traces from Freebsd-8-BETA4? > If yes, see below, otherwise just ignore this message. > > uma_zalloc_arg: zone "64" with the following non-sleepable locks held: > exclusive sleep mutex jail mutex (jail mutex) r = 0 (0xc0b91328) locked > @ > /usr/src/sys/modules/syscons/daemon/../../../dev/syscons/daemon/daemon_saver.c:355 > KDB: stack backtrace: > db_trace_self_wrapper(c0abf50f,e69b391c,c07ab3b5,c4a0580a,163,...) at > db_trace_self_wrapper+0x26 > kdb_backtrace(c4a0580a,163,ffffffff,c0d261c4,e69b3954,...) at > kdb_backtrace+0x29 > _witness_debugger(c0ac198e,e69b3968,4,1,0,...) at _witness_debugger+0x25 > witness_warn(5,0,c0ae4b92,c0add27b,e69b3984,...) at witness_warn+0x1fd > uma_zalloc_arg(c148b000,0,2,2,14,...) at uma_zalloc_arg+0x34 > malloc(29,c0b92dd0,2,163,c46562e4,...) at malloc+0xe4 > daemon_init(c0d61940,c0aba83e,e69b3a28,c46f5600,c4a06cc0,...) at > daemon_init+0x7b > splash_test(7b,c46f5600,c4a06cc0,c46f5600,e69b3a50,...) at splash_test+0x91 > splash_register(c4a06c40,0,0,76,0,...) at splash_register+0x48 Try this patch for this one: Index: daemon_saver.c =================================================================== --- daemon_saver.c (revision 196974) +++ daemon_saver.c (working copy) @@ -351,11 +351,23 @@ static int daemon_init(video_adapter_t *adp) { + size_t hostlen; mtx_lock(&prison0.pr_mtx); - messagelen = strlen(prison0.pr_hostname) + 3 + strlen(ostype) + 1 + - strlen(osrelease); - message = malloc(messagelen + 1, M_DEVBUF, M_WAITOK); + for (;;) { + hostlen = strlen(prison0.pr_hostname); + mtx_unlock(&prison0.pr_mtx); + + messagelen = hostlen + 3 + strlen(ostype) + 1 + + strlen(osrelease); + message = malloc(messagelen + 1, M_DEVBUF, M_WAITOK); + mtx_lock(&prison0.pr_mtx); + if (hostlen < strlen(prison0.pr_hostname)) { + free(message, M_DEVBUF); + continue; + } + break; + } sprintf(message, "%s - %s %s", prison0.pr_hostname, ostype, osrelease); mtx_unlock(&prison0.pr_mtx); blanked = 0; > lock order reversal: > 1st 0xc4afca2c filedesc structure (filedesc structure) @ > /usr/src/sys/kern/kern_descrip.c:1088 > 2nd 0xc4be3488 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:4091 > KDB: stack backtrace: This is the right order. I think several of these have been fixed, but there still might be some place that uses UFS -> filedesc. You can try hardcoding that order into witness to find it. > lock order reversal: > 1st 0xd82d03f0 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2559 > 2nd 0xc4f64800 dirhash (dirhash) @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:285 This is known to be harmless. > lock order reversal: > 1st 0xc464337c ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2083 > 2nd 0xd81d14b0 bufwait (bufwait) @ /usr/src/sys/ufs/ffs/ffs_softdep.c:6177 > 3rd 0xc673a7ac ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2083 > KDB: stack backtrace: > db_trace_self_wrapper(c0abf50f,e6b103cc,c07ab3b5,c079c13b,c0ac23fb,...) > at db_trace_self_wrapper+0x26 > kdb_backtrace(c079c13b,c0ac23fb,c4122e90,c4126290,e6b10428,...) at > kdb_backtrace+0x29 > _witness_debugger(c0ac23fb,c673a7ac,c0ab4f66,c4126290,c0ac96be,...) at > _witness_debugger+0x25 > witness_checkorder(c673a7ac,9,c0ac96be,823,0,...) at > witness_checkorder+0x839 > __lockmgr_args(c673a7ac,80100,c673a7c8,0,0,...) at __lockmgr_args+0x7a7 > ffs_lock(e6b10538,c07ab15b,c0ac8bb1,80100,c673a754,...) at ffs_lock+0x8a > VOP_LOCK1_APV(c0bae360,e6b10538,c4c08524,c0bc5660,c673a754,...) at > VOP_LOCK1_APV+0xb5 > _vn_lock(c673a754,80100,c0ac96be,823,4,...) at _vn_lock+0x5e > vget(c673a754,80100,c4c08480,50,0,...) at vget+0xb9 > vfs_hash_get(c46e1508,18ef074,80000,c4c08480,e6b10694,...) at > vfs_hash_get+0xe6 > ffs_vgetf(c46e1508,18ef074,80000,e6b10694,1,...) at ffs_vgetf+0x49 > softdep_sync_metadata(c4643324,0,c0ae2efa,146,0,...) at > softdep_sync_metadata+0x5ba > ffs_syncvnode(c4643324,1,c0abaa2c,c0ab45da,3,...) at ffs_syncvnode+0x3e2 > ffs_truncate(c4643324,600,0,880,c465d280,...) at ffs_truncate+0x66a > ufs_direnter(c4643324,c673a754,e6b10a1c,e6b10c00,d82ec1d0,...) at > ufs_direnter+0x8f6 > ufs_mkdir(e6b10c28,e6b10c3c,0,0,e6b10b6c,...) at ufs_mkdir+0x8a7 > VOP_MKDIR_APV(c0bae360,e6b10c28,e6b10c00,e6b10b6c,0,...) at > VOP_MKDIR_APV+0xa5 > kern_mkdirat(c4c08480,ffffff9c,804c2e8,0,41fd,...) at kern_mkdirat+0x22b > kern_mkdir(c4c08480,804c2e8,0,41fd,e6b10d2c,...) at kern_mkdir+0x2e > mkdir(c4c08480,e6b10cf8,8,c0ac2cbd,c0b8d920,...) at mkdir+0x29 > syscall(e6b10d38) at syscall+0x2a3 > Xint0x80_syscall() at Xint0x80_syscall+0x20 This is probably harmless. -- John Baldwin