From owner-freebsd-bugs@FreeBSD.ORG Sun Jul 29 00:30:12 2012 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 932F31065670 for ; Sun, 29 Jul 2012 00:30:12 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 670418FC1A for ; Sun, 29 Jul 2012 00:30:12 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q6T0U2st044073 for ; Sun, 29 Jul 2012 00:30:02 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q6T0U2s9044072; Sun, 29 Jul 2012 00:30:02 GMT (envelope-from gnats) Resent-Date: Sun, 29 Jul 2012 00:30:02 GMT Resent-Message-Id: <201207290030.q6T0U2s9044072@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Steve Wills Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C5BC3106564A for ; Sun, 29 Jul 2012 00:21:10 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22]) by mx1.freebsd.org (Postfix) with ESMTP id B118B8FC18 for ; Sun, 29 Jul 2012 00:21:10 +0000 (UTC) Received: from red.freebsd.org (localhost [127.0.0.1]) by red.freebsd.org (8.14.4/8.14.4) with ESMTP id q6T0LACQ003534 for ; Sun, 29 Jul 2012 00:21:10 GMT (envelope-from nobody@red.freebsd.org) Received: (from nobody@localhost) by red.freebsd.org (8.14.4/8.14.4/Submit) id q6T0LAhV003533; Sun, 29 Jul 2012 00:21:10 GMT (envelope-from nobody) Message-Id: <201207290021.q6T0LAhV003533@red.freebsd.org> Date: Sun, 29 Jul 2012 00:21:10 GMT From: Steve Wills To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.1 Cc: Subject: kern/170239: panic in rtsock.c X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jul 2012 00:30:12 -0000 >Number: 170239 >Category: kern >Synopsis: panic in rtsock.c >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Sun Jul 29 00:30:01 UTC 2012 >Closed-Date: >Last-Modified: >Originator: Steve Wills >Release: 9.0-RELEASE >Organization: >Environment: >Description: Seeing a panic on a box that has several gre tunnels setup and is moving approximately 80 to 100 mbit/sec. So far, this is all I've gotten out of kgdb: # kgdb kernel.debug /var/crash/vmcore.0 GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd"... Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid = 2; apic id = 06 fault virtual address = 0x44 fault code = supervisor read, page not present instruction pointer = 0x20:0xc0ae4eae stack pointer = 0x28:0xe0f00ab0 frame pointer = 0x28:0xe0f00b38 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 2372 (snmpd) trap number = 12 panic: page fault cpuid = 2 KDB: stack backtrace: #0 0xc0a50a47 at kdb_backtrace+0x47 #1 0xc0a1dfa7 at panic+0x117 #2 0xc0d5a243 at trap_fatal+0x323 #3 0xc0d5a2fd at trap_pfault+0xad #4 0xc0d5b085 at trap+0x465 #5 0xc0d43fdc at calltrap+0x6 #6 0xc0a27aca at sysctl_root+0x1fa #7 0xc0a27d83 at userland_sysctl+0x1d3 #8 0xc0a28144 at sys___sysctl+0x94 #9 0xc0d5a865 at syscall+0x355 #10 0xc0d44041 at Xint0x80_syscall+0x21 Uptime: 6d7h1m32s Physical memory: 3567 MB Dumping 334 MB: 319 303 287 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15 Reading symbols from /boot/kernel/pf.ko...Reading symbols from /boot/kernel/pf.ko.symbols...done. done. Loaded symbols for /boot/kernel/pf.ko Reading symbols from /boot/kernel/if_gre.ko...Reading symbols from /boot/kernel/if_gre.ko.symbols...done. done. Loaded symbols for /boot/kernel/if_gre.ko #0 doadump (textdump=1) at pcpu.h:244 244 __asm("movl %%fs:0,%0" : "=r" (td)); (kgdb) up #1 0xc0a1dd4a in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:442 442 doadump(TRUE); (kgdb) up #2 0xc0a1dfe1 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:607 607 kern_reboot(bootopt); (kgdb) up #3 0xc0d5a243 in trap_fatal (frame=0xe0f00a70, eva=68) at /usr/src/sys/i386/i386/trap.c:975 975 panic("%s", trap_msg[type]); (kgdb) up #4 0xc0d5a2fd in trap_pfault (frame=0xe0f00a70, usermode=0, eva=68) at /usr/src/sys/i386/i386/trap.c:839 839 trap_fatal(frame, eva); (kgdb) up #5 0xc0d5b085 in trap (frame=0xe0f00a70) at /usr/src/sys/i386/i386/trap.c:558 558 (void) trap_pfault(frame, FALSE, eva); (kgdb) up #6 0xc0d43fdc in calltrap () at /usr/src/sys/i386/i386/exception.s:168 168 call trap Current language: auto; currently asm (kgdb) up #7 0xc0ae4eae in sysctl_rtsock (oidp=0xc1031560, arg1=0xe0f00c08, arg2=4, req=0xe0f00b94) at /usr/src/sys/net/rtsock.c:1594 1594 ifam->ifam_index = ifa->ifa_ifp->if_index; Current language: auto; currently c (kgdb) i li 1594 Line 1594 of "/usr/src/sys/net/rtsock.c" starts at address 0xc0ae4eab and ends at 0xc0ae4eb6 . (kgdb) disas 0xc0ae4eab 0xc0ae4eb6 Dump of assembler code from 0xc0ae4eab to 0xc0ae4eb6: 0xc0ae4eab : mov 0x5c(%ebx),%eax 0xc0ae4eae : movzwl 0x44(%eax),%eax 0xc0ae4eb2 : mov %ax,0xc(%edx) End of assembler dump. (kgdb) p *(struct ifaddr *)$ebx $1 = {ifa_addr = 0xc827c7a8, ifa_dstaddr = 0xc827c7b8, ifa_netmask = 0xc77c8ca8, if_data = {ifi_type = 1 '\001', ifi_physical = 13 '\r', ifi_addrlen = 0 '\0', ifi_hdrlen = 0 '\0', ifi_link_state = 0 '\0', ifi_spare_char1 = 0 '\0', ifi_spare_char2 = 0 '\0', ifi_datalen = 0 '\0', ifi_mtu = 3426383120, ifi_metric = 0, ifi_baudrate = 0, ifi_ipackets = 3346381610, ifi_ierrors = 284187, ifi_opackets = 4294901815, ifi_oerrors = 0, ifi_collisions = 0, ifi_ibytes = 9385256, ifi_obytes = 4620, ifi_imcasts = 0, ifi_omcasts = 3358050108, ifi_iqdrops = 4294967295, ifi_noproto = 4294967295, ifi_hwassist = 4294967295, ifi_epoch = 0, ifi_lastchange = {tv_sec = 0, tv_usec = 0}}, ifa_ifp = 0x0, ifa_link = {tqe_next = 0x0, tqe_prev = 0xc827c760}, ifa_rtrequest = 0, ifa_flags = 51048, ifa_refcnt = 25499, ifa_metric = -936917136, ifa_claim_addr = 0, ifa_mtx = {lock_object = {lo_name = 0xc827c778 "", lo_flags = 0, lo_data = 3358050176, lo_witness = 0x0}, mtx_lock = 6}} (kgdb) For the record, net-snmpd is being polled every 60 seconds. >How-To-Repeat: Only seeing it on this one box that's moving a lot of traffic and calling snmpd often. >Fix: Fix is unknown at this time. >Release-Note: >Audit-Trail: >Unformatted: