From owner-freebsd-current@FreeBSD.ORG Wed Jul 8 14:52:32 2009 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 67A8B10656B2 for ; Wed, 8 Jul 2009 14:52:32 +0000 (UTC) (envelope-from hopet@ics.muni.cz) Received: from minas.ics.muni.cz (minas.ics.muni.cz [147.251.4.40]) by mx1.freebsd.org (Postfix) with ESMTP id 1DC4D8FC12 for ; Wed, 8 Jul 2009 14:52:29 +0000 (UTC) (envelope-from hopet@ics.muni.cz) Received: from KLOBOUCEK (kloboucek.fi.muni.cz [147.251.54.33]) (authenticated user=hopet@ICS.MUNI.CZ bits=0) by minas.ics.muni.cz (8.13.8/8.13.8/SuSE Linux 0.8) with ESMTP id n68ES3q1011595 (version=TLSv1/SSLv3 cipher=RC4-MD5 bits=128 verify=NO) for ; Wed, 8 Jul 2009 16:28:03 +0200 From: "Petr Holub" To: Date: Wed, 8 Jul 2009 16:27:58 +0200 Message-ID: <005a01c9ffd8$4db1b340$e91519c0$@muni.cz> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: Acn/vcg7YJYkijBrSrOt3OLrTvU9BQAGkWjw Content-Language: cs X-Muni-Spam-TestIP: 147.251.54.33 X-Muni-Envelope-From: hopet@ics.muni.cz X-Muni-Virus-Test: Clean X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-3.0 (minas.ics.muni.cz [147.251.4.35]); Wed, 08 Jul 2009 16:28:03 +0200 (CEST) Cc: Subject: 8.0-BETA1 lock reversal X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jul 2009 14:52:38 -0000 Hi, after upgrading from 7.2 to 8.0-BETA1 today, I've noticed the lock reversal notifications in various parts of the storage subsystem. Examples are shown below. I'm running i386 with GENERIC kernel. Let me know if more info is needed. Petr -----Original Message----- From: Petr Holub [mailto:hopet@evenstar.ics.muni.cz] Sent: Wednesday, July 08, 2009 1:18 PM To: hopet@ics.muni.cz Subject: kloboucek-8.0-BETA1-problem Jul 8 13:04:02 kloboucek kernel: lock order reversal: Jul 8 13:04:02 kloboucek kernel: 1st 0xc744437c ntfs (ntfs) @ /usr/src/sys/kern /vfs_subr.c:2404 Jul 8 13:04:02 kloboucek kernel: 2nd 0xc737b724 ntnode (ntnode) @ /usr/src/sys/ modules/ntfs/../../fs/ntfs/ntfs_subr.c:361 Jul 8 13:04:02 kloboucek kernel: KDB: stack backtrace: Jul 8 13:04:02 kloboucek kernel: db_trace_self_wrapper(c0c5b564,e973b978,c08b5b 35,c08a68db,c0c5e3f9,...) at db_trace_self_wrapper+0x26 Jul 8 13:04:02 kloboucek kernel: kdb_backtrace(c08a68db,c0c5e3f9,c6d310c8,c6d30 ff8,e973b9d4,...) at kdb_backtrace+0x29 Jul 8 13:04:02 kloboucek kernel: _witness_debugger(c0c5e3f9,c737b724,c7453600,c 6d30ff8,c745391f,...) at _witness_debugger+0x25 Jul 8 13:04:02 kloboucek kernel: witness_checkorder(c737b724,9,c745391f,169,0,. ..) at witness_checkorder+0x839 Jul 8 13:04:02 kloboucek kernel: __lockmgr_args(c737b724,80100,c737b740,0,0,... ) at __lockmgr_args+0x7a7 Jul 8 13:04:02 kloboucek kernel: ntfs_ntget(c737b700,c74443f0,c74443e0,c737b700 ,e973bb04,...) at ntfs_ntget+0x75 Jul 8 13:04:02 kloboucek kernel: ntfs_reclaim(e973bb04,1,0,c7444324,e973bb28,.. .) at ntfs_reclaim+0x3b Jul 8 13:04:02 kloboucek kernel: VOP_RECLAIM_APV(c7454200,e973bb04,0,0,c7444398 ,...) at VOP_RECLAIM_APV+0xa5 Jul 8 13:04:02 kloboucek kernel: vgonel(c7444398,0,c0c6567e,986,1,...) at vgone l+0x1a4 Jul 8 13:04:02 kloboucek kernel: vflush(c73a4c94,0,1,c75e1240,c75e1240,...) at vflush+0x337 Jul 8 13:04:02 kloboucek kernel: ntfs_unmount(c73a4c94,8000000,c0c64e9d,4f4,80, ...) at ntfs_unmount+0x59 Jul 8 13:04:02 kloboucek kernel: dounmount(c73a4c94,8000000,c75e1240,479,7,...) at dounmount+0x46d Jul 8 13:04:02 kloboucek kernel: unmount(c75e1240,e973bcf8,8,c75e1240,c0d3c288, ...) at unmount+0x30f Jul 8 13:04:02 kloboucek kernel: syscall(e973bd38) at syscall+0x2a3 Jul 8 13:04:02 kloboucek kernel: Xint0x80_syscall() at Xint0x80_syscall+0x20 Jul 8 13:04:02 kloboucek kernel: --- syscall (22, FreeBSD ELF32, unmount), eip = 0x280c59af, esp = 0xbfbfe5bc, ebp = 0xbfbfe678 --- Jul 8 13:04:02 kloboucek kernel: lock order reversal: Jul 8 13:04:02 kloboucek kernel: 1st 0xc7445058 ufs (ufs) @ /usr/src/sys/kern/v fs_mount.c:1199 Jul 8 13:04:02 kloboucek kernel: 2nd 0xc74447ac devfs (devfs) @ /usr/src/sys/ke rn/vfs_subr.c:2188 Jul 8 13:04:02 kloboucek kernel: KDB: stack backtrace: Jul 8 13:04:02 kloboucek kernel: db_trace_self_wrapper(c0c5b564,e973ba2c,c08b5b 35,c08a68db,c0c5e3f9,...) at db_trace_self_wrapper+0x26 Jul 8 13:04:02 kloboucek kernel: kdb_backtrace(c08a68db,c0c5e3f9,c6d30d88,c6d30 cb8,e973ba88,...) at kdb_backtrace+0x29 Jul 8 13:04:02 kloboucek kernel: _witness_debugger(c0c5e3f9,c74447ac,c0c4d3c5,c 6d30cb8,c0c6567e,...) at _witness_debugger+0x25 Jul 8 13:04:02 kloboucek kernel: witness_checkorder(c74447ac,9,c0c6567e,88c,0,. ..) at witness_checkorder+0x839 Jul 8 13:04:02 kloboucek kernel: __lockmgr_args(c74447ac,80100,c74447c8,0,0,... ) at __lockmgr_args+0x7a7 Jul 8 13:04:02 kloboucek kernel: vop_stdlock(e973bb90,4,c0c56a67,80100,c7444754 ,...) at vop_stdlock+0x62 Jul 8 13:04:02 kloboucek kernel: VOP_LOCK1_APV(c0d38d00,e973bb90,c0da89d8,c0d75 c00,c7444754,...) at VOP_LOCK1_APV+0xb5 Jul 8 13:04:02 kloboucek kernel: _vn_lock(c7444754,80100,c0c6567e,88c,c74534f8, ...) at _vn_lock+0x5e Jul 8 13:04:02 kloboucek kernel: vrele(c7444754,c0c64e9d,469,200,c75e1240,...) at vrele+0x137 Jul 8 13:04:02 kloboucek kernel: ntfs_unmount(c73a4c94,8000000,c0c64e9d,4f4,80, ...) at ntfs_unmount+0x1a0 Jul 8 13:04:02 kloboucek kernel: dounmount(c73a4c94,8000000,c75e1240,479,7,...) at dounmount+0x46d Jul 8 13:04:02 kloboucek kernel: unmount(c75e1240,e973bcf8,8,c75e1240,c0d3c288, ...) at unmount+0x30f Jul 8 13:04:02 kloboucek kernel: syscall(e973bd38) at syscall+0x2a3 Jul 8 13:04:02 kloboucek kernel: Xint0x80_syscall() at Xint0x80_syscall+0x20 Jul 8 13:04:02 kloboucek kernel: --- syscall (22, FreeBSD ELF32, unmount), eip = 0x280c59af, esp = 0xbfbfe5bc, ebp = 0xbfbfe678 --- Jul 8 13:04:14 kloboucek kernel: lock order reversal: Jul 8 13:04:14 kloboucek kernel: 1st 0xdace6460 bufwait (bufwait) @ /usr/src/sy s/kern/vfs_bio.c:2558 Jul 8 13:04:14 kloboucek kernel: 2nd 0xc73b9c00 dirhash (dirhash) @ /usr/src/sy s/ufs/ufs/ufs_dirhash.c:285 Jul 8 13:04:14 kloboucek kernel: KDB: stack backtrace: Jul 8 13:04:14 kloboucek kernel: db_trace_self_wrapper(c0c5b564,e97a0768,c08b5b 35,c08a68db,c0c5e3f9,...) at db_trace_self_wrapper+0x26 Jul 8 13:04:14 kloboucek kernel: kdb_backtrace(c08a68db,c0c5e3f9,c6d2cf60,c6d30 df0,e97a07c4,...) at kdb_backtrace+0x29 Jul 8 13:04:14 kloboucek kernel: _witness_debugger(c0c5e3f9,c73b9c00,c0c7e6a2,c 6d30df0,c0c7e33b,...) at _witness_debugger+0x25 Jul 8 13:04:14 kloboucek kernel: witness_checkorder(c73b9c00,9,c0c7e33b,11d,0,. ..) at witness_checkorder+0x839 Jul 8 13:04:14 kloboucek kernel: _sx_xlock(c73b9c00,0,c0c7e33b,11d,c73aed24,... ) at _sx_xlock+0x85 Jul 8 13:04:14 kloboucek kernel: ufsdirhash_acquire(dace6400,e97a08dc,200,daf57 800,e97a0894,...) at ufsdirhash_acquire+0x35 Jul 8 13:04:14 kloboucek kernel: ufsdirhash_add(c73aed24,e97a08dc,800,e97a0880, e97a0884,...) at ufsdirhash_add+0x13 Jul 8 13:04:14 kloboucek kernel: ufs_direnter(c71a5c90,c75bd754,e97a08dc,e97a0b d4,0,...) at ufs_direnter+0x729 Jul 8 13:04:14 kloboucek kernel: ufs_makeinode(e97a0bd4,0,e97a0ac8,e97a0a24,c0b a67e5,...) at ufs_makeinode+0x4f8 Jul 8 13:04:14 kloboucek kernel: ufs_create(e97a0ac8,e97a0ae0,0,0,e97a0ba8,...) at ufs_create+0x30 Jul 8 13:04:14 kloboucek kernel: VOP_CREATE_APV(c0d5d160,e97a0ac8,e97a0bd4,e97a 0a60,0,...) at VOP_CREATE_APV+0xa5 Jul 8 13:04:14 kloboucek kernel: vn_open_cred(e97a0ba8,e97a0c5c,180,0,c73a8700, ...) at vn_open_cred+0x200 Jul 8 13:04:14 kloboucek kernel: vn_open(e97a0ba8,e97a0c5c,180,c73e71f8,0,...) at vn_open+0x3b Jul 8 13:04:14 kloboucek kernel: kern_openat(c75e2d80,ffffff9c,28e1e6d0,0,a03,. ..) at kern_openat+0x118 Jul 8 13:04:14 kloboucek kernel: kern_open(c75e2d80,28e1e6d0,0,a02,180,...) at kern_open+0x35 Jul 8 13:04:14 kloboucek kernel: open(c75e2d80,e97a0cf8,c,c0c3fdfb,c0d3c0ac,... ) at open+0x30 Jul 8 13:04:14 kloboucek kernel: syscall(e97a0d38) at syscall+0x2a3 Jul 8 13:04:14 kloboucek kernel: Xint0x80_syscall() at Xint0x80_syscall+0x20 Jul 8 13:04:14 kloboucek kernel: --- syscall (5, FreeBSD ELF32, open), eip = 0x 28a5b13b, esp = 0xbfbfdffc, ebp = 0xbfbfe028 --- lock order reversal: 1st 0xdacfa200 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2558 2nd 0xc73b8c00 dirhash (dirhash) @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:285 KDB: stack backtrace: db_trace_self_wrapper(c0c5b564,e97d4768,c08b5b35,c08a68db,c0c5e3f9,...) at db_tr ace_self_wrapper+0x26 kdb_backtrace(c08a68db,c0c5e3f9,c6d2cf60,c6d30df0,e97d47c4,...) at kdb_backtrace +0x29 _witness_debugger(c0c5e3f9,c73b8c00,c0c7e6a2,c6d30df0,c0c7e33b,...) at _witness_ debugger+0x25 witness_checkorder(c73b8c00,9,c0c7e33b,11d,0,...) at witness_checkorder+0x839 _sx_xlock(c73b8c00,0,c0c7e33b,11d,c7440828,...) at _sx_xlock+0x85 ufsdirhash_acquire(dacfa1a0,e97d48dc,200,db321a18,e97d4894,...) at ufsdirhash_ac quire+0x35 ufsdirhash_add(c7440828,e97d48dc,2a18,e97d4880,e97d4884,...) at ufsdirhash_add+0 x13 ufs_direnter(c744153c,c813953c,e97d48dc,e97d4bd4,0,...) at ufs_direnter+0x729 ufs_makeinode(e97d4bd4,0,e97d4ac8,e97d4a24,c0ba67e5,...) at ufs_makeinode+0x4f8 ufs_create(e97d4ac8,e97d4ae0,0,0,e97d4ba8,...) at ufs_create+0x30 VOP_CREATE_APV(c0d5d160,e97d4ac8,e97d4bd4,e97d4a60,0,...) at VOP_CREATE_APV+0xa5 vn_open_cred(e97d4ba8,e97d4c5c,180,0,c7357480,...) at vn_open_cred+0x200 vn_open(e97d4ba8,e97d4c5c,180,c73e7690,4a0c65bf,...) at vn_open+0x3b kern_openat(c9119000,ffffff9c,bfbfddec,0,a03,...) at kern_openat+0x118 kern_open(c9119000,bfbfddec,0,a02,180,...) at kern_open+0x35 open(c9119000,e97d4cf8,c,c0c5ec8a,c0d3c0ac,...) at open+0x30 syscall(e97d4d38) at syscall+0x2a3 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (5, FreeBSD ELF32, open), eip = 0x2815afe3, esp = 0xbfbfd8ec, ebp = 0xbfbfdd98 ---