From owner-freebsd-amd64@FreeBSD.ORG Thu Jun 27 01:10:01 2013 Return-Path: Delivered-To: freebsd-amd64@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 1B2733A0 for ; Thu, 27 Jun 2013 01:10:01 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 0D0BA1A97 for ; Thu, 27 Jun 2013 01:10:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r5R1A0Ah072930 for ; Thu, 27 Jun 2013 01:10:00 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r5R1A0oA072929; Thu, 27 Jun 2013 01:10:00 GMT (envelope-from gnats) Date: Thu, 27 Jun 2013 01:10:00 GMT Message-Id: <201306270110.r5R1A0oA072929@freefall.freebsd.org> To: freebsd-amd64@FreeBSD.org Cc: From: Alan Amesbury Subject: Re: amd64/180018: System panics when bsnmpd is started X-Mailman-Approved-At: Thu, 27 Jun 2013 02:11:55 +0000 X-BeenThere: freebsd-amd64@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Alan Amesbury List-Id: Porting FreeBSD to the AMD64 platform List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jun 2013 01:10:01 -0000 The following reply was made to PR amd64/180018; it has been noted by GNATS. From: Alan Amesbury To: bug-followup@FreeBSD.org Cc: Subject: Re: amd64/180018: System panics when bsnmpd is started Date: Wed, 26 Jun 2013 20:07:57 -0500 This is a multi-part message in MIME format. --------------040001040900060009020400 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Two example console output and crash dumps are attached. The first is what happens when bsnmpd is enabled and the system is allowed to boot. The second is the result of starting bsnmpd after the system had been up for a while (about 30 minutes). -- Alan Amesbury University Information Security University of Minnesota --------------040001040900060009020400 Content-Type: text/plain; charset=us-ascii; name="crash_1_console.text" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="crash_1_console.text" Starting bsnmpd. Wed Jun 26 18:45:52 CDT 2013 ################ Fatal trap 12: page fault while in kernel mode cpuid = 30; apic id = 2e fault virtual address = 0xbff001019 fault code = supervisor write data, page not present instruction pointer = 0x20:0xffffffff8067ad6b stack pointer = 0x28:0xffffff88c41a9720 frame pointer = 0x28:0xffffff88c41a9800 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 60934 (bsnmpd) trap number = 12 panic: page fault cpuid = 27 KDB: stack backtrace: #0 0xffffffff806747d8 at kdb_backtrace+0x68 #1 0xffffffff8063f2ab at panic+0x21b #2 0xffffffff808c475d at trap_fatal+0x39d #3 0xffffffff808c48a0 at trap_pfault+0x120 #4 0xffffffff808c4f99 at trap+0x3d9 #5 0xffffffff808af70f at calltrap+0x8 #6 0xffffffff80649057 at sysctl_root+0x1e7 #7 0xffffffff80649337 at userland_sysctl+0x1f7 #8 0xffffffff80649773 at sys___sysctl+0x93 #9 0xffffffff808c3d6d at amd64_syscall+0x2fd #10 0xffffffff808af9f7 at Xfast_syscall+0xf7 Uptime: 44s Dumping 1538 out of 32708 MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% Dump complete Automatic reboot in 15 seconds - press a key on the console to abort Rebooting... cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 27 --------------040001040900060009020400 Content-Type: text/plain; charset=us-ascii; name="crash_1_kgdb.text" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="crash_1_kgdb.text" GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: <118>Starting bsnmpd. <118> <118>Wed Jun 26 18:45:52 CDT 2013 Fatal trap 12: page fault while in kernel mode cpuid = 30; apic id = 2e fault virtual address = 0xbff001019 fault code = supervisor write data, page not present instruction pointer = 0x20:0xffffffff8067ad6b stack pointer = 0x28:0xffffff88c41a9720 frame pointer = 0x28:0xffffff88c41a9800 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 60934 (bsnmpd) trap number = 12 panic: page fault cpuid = 27 KDB: stack backtrace: #0 0xffffffff806747d8 at kdb_backtrace+0x68 #1 0xffffffff8063f2ab at panic+0x21b #2 0xffffffff808c475d at trap_fatal+0x39d #3 0xffffffff808c48a0 at trap_pfault+0x120 #4 0xffffffff808c4f99 at trap+0x3d9 #5 0xffffffff808af70f at calltrap+0x8 #6 0xffffffff80649057 at sysctl_root+0x1e7 #7 0xffffffff80649337 at userland_sysctl+0x1f7 #8 0xffffffff80649773 at sys___sysctl+0x93 #9 0xffffffff808c3d6d at amd64_syscall+0x2fd #10 0xffffffff808af9f7 at Xfast_syscall+0xf7 Uptime: 44s Dumping 1538 out of 32708 MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% Reading symbols from /boot/kernel/aesni.ko...done. Loaded symbols for /boot/kernel/aesni.ko Reading symbols from /boot/kernel/cryptodev.ko...done. Loaded symbols for /boot/kernel/cryptodev.ko Reading symbols from /boot/kernel/zfs.ko...done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...done. Loaded symbols for /boot/kernel/opensolaris.ko #0 doadump (textdump=Variable "textdump" is not available. ) at pcpu.h:224 224 __asm("movq %%gs:0,%0" : "=r" (td)); (kgdb) list *0xffffffff8067ad6b 0xffffffff8067ad6b is in sysctl_rman (atomic.h:160). 155 static __inline int 156 atomic_cmpset_long(volatile u_long *dst, u_long expect, u_long src) 157 { 158 u_char res; 159 160 __asm __volatile( 161 " " MPLOCKED " " 162 " cmpxchgq %2,%1 ; " 163 " sete %0 ; " 164 "1: " (kgdb) bt #0 doadump (textdump=Variable "textdump" is not available. ) at pcpu.h:224 #1 0xffffffff8063ef67 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:448 #2 0xffffffff8063f2ef in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:636 #3 0xffffffff808c475d in trap_fatal (frame=0xffffff88c41a9670, eva=51522834457) at /usr/src/sys/amd64/amd64/trap.c:857 #4 0xffffffff808c48a0 in trap_pfault (frame=0xffffff88c41a9670, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:714 #5 0xffffffff808c4f99 in trap (frame=0xffffff88c41a9670) at /usr/src/sys/amd64/amd64/trap.c:456 #6 0xffffffff808af70f in calltrap () at /usr/src/sys/amd64/amd64/exception.S:228 #7 0xffffffff8067ad6b in sysctl_rman (oidp=Variable "oidp" is not available. ) at atomic.h:160 #8 0xffffffff80649057 in sysctl_root (oidp=Variable "oidp" is not available. ) at /usr/src/sys/kern/kern_sysctl.c:1513 #9 0xffffffff80649337 in userland_sysctl (td=Variable "td" is not available. ) at /usr/src/sys/kern/kern_sysctl.c:1623 #10 0xffffffff80649773 in sys___sysctl (td=0xfffffe002122c470, uap=0xffffff88c41a9a70) at /usr/src/sys/kern/kern_sysctl.c:1549 #11 0xffffffff808c3d6d in amd64_syscall (td=0xfffffe002122c470, traced=0) at subr_syscall.c:135 #12 0xffffffff808af9f7 in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:387 #13 0x00000007bd35646c in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) --------------040001040900060009020400 Content-Type: text/plain; charset=us-ascii; name="crash_2_console.text" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="crash_2_console.text" login: Jun 26 19:20:28 tumbrel sudo: amesbury : TTY=pts/0 ; PWD=/export/00/home/amesbury ; USER=root ; COMMAND=/etc/rc.d/bsnmpd onestar Fatal trap 12: page fault while in kernel mode cpuid = 11; apic id = 0b fault virtual address = 0xbff001019 fault code = supervisor write data, page not present instruction pointer = 0x20:0xffffffff8067ad6b stack pointer = 0x28:0xffffff88c494c720 frame pointer = 0x28:0xffffff88c494c800 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 98589 (bsnmpd) trap number = 12 panic: page fault cpuid = 14 KDB: stack backtrace: #0 0xffffffff806747d8 at kdb_backtrace+0x68 #1 0xffffffff8063f2ab at panic+0x21b #2 0xffffffff808c475d at trap_fatal+0x39d #3 0xffffffff808c48a0 at trap_pfault+0x120 #4 0xffffffff808c4f99 at trap+0x3d9 #5 0xffffffff808af70f at calltrap+0x8 #6 0xffffffff80649057 at sysctl_root+0x1e7 #7 0xffffffff80649337 at userland_sysctl+0x1f7 #8 0xffffffff80649773 at sys___sysctl+0x93 #9 0xffffffff808c3d6d at amd64_syscall+0x2fd #10 0xffffffff808af9f7 at Xfast_syscall+0xf7 Uptime: 29m42s Dumping 1940 out of 32708 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Dump complete Automatic reboot in 15 seconds - press a key on the console to abort Rebooting... cpu_reset: Restarting BSP cpu_reset_proxy: Stopped CPU 14 --------------040001040900060009020400 Content-Type: text/plain; charset=us-ascii; name="crash_2_kgdb.text" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="crash_2_kgdb.text" GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid = 11; apic id = 0b fault virtual address = 0xbff001019 fault code = supervisor write data, page not present instruction pointer = 0x20:0xffffffff8067ad6b stack pointer = 0x28:0xffffff88c494c720 frame pointer = 0x28:0xffffff88c494c800 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 98589 (bsnmpd) trap number = 12 panic: page fault cpuid = 14 KDB: stack backtrace: #0 0xffffffff806747d8 at kdb_backtrace+0x68 #1 0xffffffff8063f2ab at panic+0x21b #2 0xffffffff808c475d at trap_fatal+0x39d #3 0xffffffff808c48a0 at trap_pfault+0x120 #4 0xffffffff808c4f99 at trap+0x3d9 #5 0xffffffff808af70f at calltrap+0x8 #6 0xffffffff80649057 at sysctl_root+0x1e7 #7 0xffffffff80649337 at userland_sysctl+0x1f7 #8 0xffffffff80649773 at sys___sysctl+0x93 #9 0xffffffff808c3d6d at amd64_syscall+0x2fd #10 0xffffffff808af9f7 at Xfast_syscall+0xf7 Uptime: 29m42s Dumping 1940 out of 32708 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Reading symbols from /boot/kernel/aesni.ko...done. Loaded symbols for /boot/kernel/aesni.ko Reading symbols from /boot/kernel/cryptodev.ko...done. Loaded symbols for /boot/kernel/cryptodev.ko Reading symbols from /boot/kernel/zfs.ko...done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...done. Loaded symbols for /boot/kernel/opensolaris.ko #0 doadump (textdump=Variable "textdump" is not available. ) at pcpu.h:224 224 __asm("movq %%gs:0,%0" : "=r" (td)); (kgdb) list *0xffffffff8067ad6b 0xffffffff8067ad6b is in sysctl_rman (atomic.h:160). 155 static __inline int 156 atomic_cmpset_long(volatile u_long *dst, u_long expect, u_long src) 157 { 158 u_char res; 159 160 __asm __volatile( 161 " " MPLOCKED " " 162 " cmpxchgq %2,%1 ; " 163 " sete %0 ; " 164 "1: " (kgdb) bt #0 doadump (textdump=Variable "textdump" is not available. ) at pcpu.h:224 #1 0xffffffff8063ef67 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:448 #2 0xffffffff8063f2ef in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:636 #3 0xffffffff808c475d in trap_fatal (frame=0xffffff88c494c670, eva=51522834457) at /usr/src/sys/amd64/amd64/trap.c:857 #4 0xffffffff808c48a0 in trap_pfault (frame=0xffffff88c494c670, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:714 #5 0xffffffff808c4f99 in trap (frame=0xffffff88c494c670) at /usr/src/sys/amd64/amd64/trap.c:456 #6 0xffffffff808af70f in calltrap () at /usr/src/sys/amd64/amd64/exception.S:228 #7 0xffffffff8067ad6b in sysctl_rman (oidp=Variable "oidp" is not available. ) at atomic.h:160 #8 0xffffffff80649057 in sysctl_root (oidp=Variable "oidp" is not available. ) at /usr/src/sys/kern/kern_sysctl.c:1513 #9 0xffffffff80649337 in userland_sysctl (td=Variable "td" is not available. ) at /usr/src/sys/kern/kern_sysctl.c:1623 #10 0xffffffff80649773 in sys___sysctl (td=0xfffffe024242d470, uap=0xffffff88c494ca70) at /usr/src/sys/kern/kern_sysctl.c:1549 #11 0xffffffff808c3d6d in amd64_syscall (td=0xfffffe024242d470, traced=0) at subr_syscall.c:135 #12 0xffffffff808af9f7 in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:387 #13 0x00000007bd35646c in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) --------------040001040900060009020400--