From owner-freebsd-bugs@FreeBSD.ORG Tue Aug 2 18:50:10 2011 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 371A71065675 for ; Tue, 2 Aug 2011 18:50:10 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 0F4D38FC14 for ; Tue, 2 Aug 2011 18:50:10 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id p72Io9mg012707 for ; Tue, 2 Aug 2011 18:50:09 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id p72Io9px012706; Tue, 2 Aug 2011 18:50:09 GMT (envelope-from gnats) Resent-Date: Tue, 2 Aug 2011 18:50:09 GMT Resent-Message-Id: <201108021850.p72Io9px012706@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Victor Detoni Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 79817106564A for ; Tue, 2 Aug 2011 18:44:18 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22]) by mx1.freebsd.org (Postfix) with ESMTP id 68B1F8FC14 for ; Tue, 2 Aug 2011 18:44:18 +0000 (UTC) Received: from red.freebsd.org (localhost [127.0.0.1]) by red.freebsd.org (8.14.4/8.14.4) with ESMTP id p72IiI7M043719 for ; Tue, 2 Aug 2011 18:44:18 GMT (envelope-from nobody@red.freebsd.org) Received: (from nobody@localhost) by red.freebsd.org (8.14.4/8.14.4/Submit) id p72IiIj0043691; Tue, 2 Aug 2011 18:44:18 GMT (envelope-from nobody) Message-Id: <201108021844.p72IiIj0043691@red.freebsd.org> Date: Tue, 2 Aug 2011 18:44:18 GMT From: Victor Detoni To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.1 Cc: Subject: kern/159390: panic: mutex pf task mtx owned at /usr/src/sys/contrib/pf/net/if_pfsync.c:2029 X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Aug 2011 18:50:10 -0000 >Number: 159390 >Category: kern >Synopsis: panic: mutex pf task mtx owned at /usr/src/sys/contrib/pf/net/if_pfsync.c:2029 >Confidential: no >Severity: serious >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Tue Aug 02 18:50:09 UTC 2011 >Closed-Date: >Last-Modified: >Originator: Victor Detoni >Release: FreeBSD 9.0-BETA1 - CURRENT >Organization: >Environment: FreeBSD fbsd2 9.0-BETA1 FreeBSD 9.0-BETA1 #0: Thu Jul 28 17:15:31 UTC 2011 root@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 >Description: Problem occured after packets SYN sent to pf interface. lock order reversal: 1st 0xffffffff8112bba0 pf task mtx (pf task mtx) @ /usr/src/sys/contrib/pf/net/pf.c:6630 2nd 0xfffffe000357baf8 radix mode head (radix node head) @ /usr/src/sys/net/route.c:354 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x807 Aug 1 21:20:13 fbsd2 kernel: _rw_rlock() at _rw_rlock+0x6d Aug 1 21:20:13 fbsd2 kernel: rtalloc1_fib() at rtalloc1_fib+0x10c Aug 1 21:20:13 fbsd2 kernel: rtalloc_ign_fib() at rtalloc_ign_fib+0xc5 Aug 1 21:20:13 fbsd2 kernel: pf_calc_mss() at pf_calc_mss+0xb2 Aug 1 21:20:13 fbsd2 kernel: pf_test_rule() at pf_test_rule+0x1e74 Aug 1 21:20:13 fbsd2 kernel: pf_test() at pf_test+0x1051 Aug 1 21:20:13 fbsd2 kernel: pf_check_in() at pf_check_in+0x2b Aug 1 21:20:13 fbsd2 kernel: pfil_run_hooks() at pfil_run_hooks+0xd2 Aug 1 21:20:13 fbsd2 kernel: ip_input() at ip_input+0x2e7 Aug 1 21:20:13 fbsd2 kernel: netisr_dispatch_src() at netisr_dispatch_src+0x160 Aug 1 21:20:13 fbsd2 kernel: ether_demux() at ether_demux+0x17d Aug 1 21:20:13 fbsd2 kernel: ether_nh_input() at ether_nh_input+0x20e Aug 1 21:20:13 fbsd2 kernel: netisr_dispatch_src() at netisr_dispatch_src+0x160 Aug 1 21:20:13 fbsd2 kernel: em_rxeof() at em_rxeof+0x1a7 Aug 1 21:20:13 fbsd2 kernel: em_handle_que() at em_handle_que+0x50 Aug 1 21:20:13 fbsd2 kernel: taskqueue_run_locked() at taskqueue_run_locked+0x93 Aug 1 21:20:13 fbsd2 kernel: taskqueue_thread_loop() at taskqueue_thread_loop+0x3e Aug 1 21:20:13 fbsd2 kernel: fork_exit() at fork_exit+0x135 Aug 1 21:20:13 fbsd2 kernel: fork_trampoline() at fork_trampoline+0xe panic: mutex pf task mtx owned at /usr/src/sys/contrib/pf/net/if_pfsync.c:2029 cpuid = 0 KDB: enter: panic [ thread pid 0 tid 100030 ] Stopped at kdb_enter+0x3b: movq $0,0x924002(xrip) db> db> >How-To-Repeat: After reboot, it shows: (panic) Aug 2 14:58:05 fbsd2 kernel: lock order reversal: Aug 2 14:58:05 fbsd2 kernel: 1st 0xffffff80f6386258 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:2658 Aug 2 14:58:05 fbsd2 kernel: 2nd 0xfffffe00035fc600 dirhash (dirhash) @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:284 Aug 2 14:58:05 fbsd2 kernel: KDB: stack backtrace: Aug 2 14:58:05 fbsd2 kernel: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a Aug 2 14:58:05 fbsd2 kernel: kdb_backtrace() at kdb_backtrace+0x37 Aug 2 14:58:05 fbsd2 kernel: _witness_debugger() at _witness_debugger+0x2e Aug 2 14:58:05 fbsd2 kernel: witness_checkorder() at witness_checkorder+0x807 Aug 2 14:58:05 fbsd2 kernel: _sx_xlock() at _sx_xlock+0x55 Aug 2 14:58:05 fbsd2 kernel: ufsdirhash_acquire() at ufsdirhash_acquire+0x33 Aug 2 14:58:05 fbsd2 kernel: ufsdirhash_add() at ufsdirhash_add+0x19 Aug 2 14:58:05 fbsd2 kernel: ufs_direnter() at ufs_direnter+0x8c9 Aug 2 14:58:05 fbsd2 kernel: ufs_makeinode() at ufs_makeinode+0x25b Aug 2 14:58:05 fbsd2 kernel: VOP_CREATE_APV() at VOP_CREATE_APV+0x8d Aug 2 14:58:05 fbsd2 kernel: vn_open_cred() at vn_open_cred+0x46a Aug 2 14:58:05 fbsd2 kernel: kern_openat() at kern_openat+0x17f Aug 2 14:58:05 fbsd2 kernel: syscallenter() at syscallenter+0x1aa Aug 2 14:58:05 fbsd2 kernel: syscall() at syscall+0x4c Aug 2 14:58:05 fbsd2 kernel: Xfast_syscall() at Xfast_syscall+0xdd Aug 2 14:58:05 fbsd2 kernel: --- syscall (5, FreeBSD ELF64, open), rip = 0x800b3de2c, rsp = 0x7fffffffd358, rbp = 0x8 --- Aug 2 14:58:05 fbsd2 kernel: lock order reversal: Aug 2 14:58:05 fbsd2 kernel: 1st 0xfffffe00b115e638 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2134 Aug 2 14:58:05 fbsd2 kernel: 2nd 0xffffff80f6386258 bufwait (bufwait) @ /usr/src/sys/ufs/ffs/ffs_vnops.c:261 Aug 2 14:58:05 fbsd2 kernel: 3rd 0xfffffe002a4e3818 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2134 Aug 2 14:58:05 fbsd2 kernel: KDB: stack backtrace: Aug 2 14:58:05 fbsd2 kernel: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a Aug 2 14:58:05 fbsd2 kernel: kdb_backtrace() at kdb_backtrace+0x37 Aug 2 14:58:05 fbsd2 kernel: _witness_debugger() at _witness_debugger+0x2e Aug 2 14:58:05 fbsd2 kernel: witness_checkorder() at witness_checkorder+0x807 Aug 2 14:58:05 fbsd2 kernel: __lockmgr_args() at __lockmgr_args+0xd42 Aug 2 14:58:05 fbsd2 kernel: ffs_lock() at ffs_lock+0x8c Aug 2 14:58:05 fbsd2 kernel: VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b Aug 2 14:58:05 fbsd2 kernel: _vn_lock() at _vn_lock+0x47 Aug 2 14:58:05 fbsd2 kernel: vget() at vget+0x7b Aug 2 14:58:05 fbsd2 kernel: vfs_hash_get() at vfs_hash_get+0xd5 Aug 2 14:58:05 fbsd2 kernel: ffs_vgetf() at ffs_vgetf+0x48 Aug 2 14:58:05 fbsd2 kernel: softdep_sync_buf() at softdep_sync_buf+0x547 Aug 2 14:58:05 fbsd2 kernel: ffs_syncvnode() at ffs_syncvnode+0x299 Aug 2 14:58:05 fbsd2 kernel: ffs_truncate() at ffs_truncate+0x463 Aug 2 14:58:05 fbsd2 kernel: ufs_direnter() at ufs_direnter+0x6ff Aug 2 14:58:05 fbsd2 kernel: ufs_makeinode() at ufs_makeinode+0x25b Aug 2 14:58:05 fbsd2 kernel: VOP_CREATE_APV() at VOP_CREATE_APV+0x8d Aug 2 14:58:05 fbsd2 kernel: vn_open_cred() at vn_open_cred+0x46a Aug 2 14:58:05 fbsd2 kernel: kern_openat() at kern_openat+0x17f Aug 2 14:58:05 fbsd2 kernel: syscallenter() at syscallenter+0x1aa Aug 2 14:58:05 fbsd2 kernel: syscall() at syscall+0x4c Aug 2 14:58:05 fbsd2 kernel: Xfast_syscall() at Xfast_syscall+0xdd Aug 2 14:58:05 fbsd2 kernel: --- syscall (5, FreeBSD ELF64, open), rip = 0x800b3de2c, rsp = 0x7fffffffd358, rbp = 0x8 --- panic: mutex pf task mtx owned at /usr/src/sys/contrib/pf/net/if_pfsync.c:2029 cpuid = 0 KDB: enter: panic [ thread pid 0 tid 100030 ] Stopped at kdb_enter+0x3b: movq $0,0x924002(xrip) db> >Fix: Recompile kernel without "device pfsync" line. >Release-Note: >Audit-Trail: >Unformatted: