From owner-freebsd-amd64@FreeBSD.ORG Wed Jan 18 18:44:31 2012 Return-Path: Delivered-To: amd64@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CC6521065672 for ; Wed, 18 Jan 2012 18:44:31 +0000 (UTC) (envelope-from cwt@networks.cwu.edu) Received: from nsc0.cwu.edu (nsc0.cwu.edu [72.233.196.16]) by mx1.freebsd.org (Postfix) with ESMTP id A08E98FC17 for ; Wed, 18 Jan 2012 18:44:31 +0000 (UTC) Received: from n.cwu.edu (n.cwu.edu [198.104.69.57]) by nsc0.cwu.edu (8.14.3/8.14.3) with ESMTP id q0IITKmK064158 for ; Wed, 18 Jan 2012 10:29:20 -0800 (PST) (envelope-from cwt@networks.cwu.edu) Received: from n.cwu.edu (localhost [127.0.0.1]) by n.cwu.edu (8.13.3/8.13.3) with ESMTP id q0IITKk7099022 for ; Wed, 18 Jan 2012 10:29:20 -0800 (PST) (envelope-from cwt@networks.cwu.edu) Received: from localhost (cwt@localhost) by n.cwu.edu (8.13.3/8.13.1/Submit) with ESMTP id q0IITK1t099019 for ; Wed, 18 Jan 2012 10:29:20 -0800 (PST) (envelope-from cwt@networks.cwu.edu) X-Authentication-Warning: n.cwu.edu: cwt owned process doing -bs Date: Wed, 18 Jan 2012 10:29:20 -0800 (PST) From: Chris Timmons X-X-Sender: cwt@n.cwu.edu To: amd64@freebsd.org Message-ID: <20120118101741.U94209@n.cwu.edu> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender DNS name whitelisted, not delayed by milter-greylist-4.0 (nsc0.cwu.edu [72.233.196.16]); Wed, 18 Jan 2012 10:29:20 -0800 (PST) Cc: Subject: 8.2-stable, repeatable panic - nessus, bpf X-BeenThere: freebsd-amd64@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting FreeBSD to the AMD64 platform List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2012 18:44:32 -0000 When I start a large scan with Nessus, I get an immediate, repeatable panic "sleeping thread owns a non-sleepable lock". I'd been seeing this occasionally with 8.2-stable over the last year, but now it happens every time. Server hardware is a dual quad Xeon ProLiant G5 w/16GB. I've compiled a debugging kernel with WITNESS and have the following output. I've seen /usr/src/sys/net/bpf.c:2148 with WITNESS every time; /usr/src/sys/dev/usb/input/ukbd.c:2018 only appeared after I added DDB to the kernel options and began seeing more copious output. Comments/Suggestions? lock order reversal: (Giant after non-sleepable) 1st 0xffffffff80e28920 bpf global lock (bpf global lock) @ /usr/src/sys/net/bpf.c:2148 2nd 0xffffffff80c65360 Giant (Giant) @ /usr/src/sys/dev/usb/input/ukbd.c:2018 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 _witness_debugger() at _witness_debugger+0x2c witness_checkorder() at witness_checkorder+0x651 _mtx_lock_flags() at _mtx_lock_flags+0x3c ukbd_poll() at ukbd_poll+0x44 kbdmux_poll() at kbdmux_poll+0x3f sc_cngetc() at sc_cngetc+0xed cncheckc() at cncheckc+0x65 cngetc() at cngetc+0x1c db_readline() at db_readline+0x77 db_read_line() at db_read_line+0x15 db_command_loop() at db_command_loop+0x38 db_trap() at db_trap+0x89 kdb_trap() at kdb_trap+0xc1 trap() at trap+0x176 calltrap() at calltrap+0x8 --- trap 0x3, rip = 0xffffffff805f600b, rsp = 0xffffff8485c08630, rbp = 0xffffff8485c08650 --- kdb_enter() at kdb_enter+0x3b witness_warn() at witness_warn+0x2c4 trap() at trap+0x286 calltrap() at calltrap+0x8 --- trap 0xc, rip = 0xffffffff80888093, rsp = 0xffffff8485c08930, rbp = 0xffffff8485c08980 --- copyout() at copyout+0x43 bpfioctl() at bpfioctl+0xaf0 devfs_ioctl_f() at devfs_ioctl_f+0x7a kern_ioctl() at kern_ioctl+0xfe ioctl() at ioctl+0xfd amd64_syscall() at amd64_syscall+0xf9 Xfast_syscall() at Xfast_syscall+0xfc --- syscall (54, FreeBSD ELF64, ioctl), rip = 0x8010fc0dc, rsp = 0x7fffe351a598, rbp = 0x23 --- Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 01 fault virtual address = 0x805aee428 fault code = supervisor write data, protection violation instruction pointer = 0x20:0xffffffff80888093 stack pointer = 0x28:0xffffff8485c08930 frame pointer = 0x28:0xffffff8485c08980 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 2337 (nessusd) [thread pid 2337 tid 100175 ] Stopped at copyout+0x43: repe movsb (%rsi),%es:(%rdi) Tracing pid 2337 tid 100175 td 0xffffff0128e47460 copyout() at copyout+0x43 bpfioctl() at bpfioctl+0xaf0 devfs_ioctl_f() at devfs_ioctl_f+0x7a kern_ioctl() at kern_ioctl+0xfe ioctl() at ioctl+0xfd amd64_syscall() at amd64_syscall+0xf9 Xfast_syscall() at Xfast_syscall+0xfc --- syscall (54, FreeBSD ELF64, ioctl), rip = 0x8010fc0dc, rsp = 0x7fffe351a598, rbp = 0x23 --- db>