From owner-freebsd-stable@freebsd.org Tue Sep 29 16:46:54 2015 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 202F3A0AAD1 for ; Tue, 29 Sep 2015 16:46:54 +0000 (UTC) (envelope-from thomas@FreeBSD.ORG) Received: from melamine.cuivre.fr.eu.org (houdart.cuivre.fr.eu.org [81.57.40.110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DE06F1A85 for ; Tue, 29 Sep 2015 16:46:52 +0000 (UTC) (envelope-from thomas@FreeBSD.ORG) Received: by melamine.cuivre.fr.eu.org (Postfix, from userid 1000) id 2EF113160B8; Tue, 29 Sep 2015 18:37:36 +0200 (CEST) Date: Tue, 29 Sep 2015 18:37:36 +0200 From: Thomas Quinot To: Zeus Panchenko Cc: freebsd-stable@FreeBSD.org Subject: Re: [panic] 10.1-RELEASE-p14 #4 r285499M sporadic reboot Message-ID: <20150929163736.GA74265@melamine.cuivre.fr.eu.org> References: <20150722135353.70269@smtp.new-ukraine.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150722135353.70269@smtp.new-ukraine.org> X-message-flag: WARNING! Using Outlook can damage your computer. User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2015 16:46:54 -0000 * Zeus Panchenko, 2015-07-22 : > on important production server I've started to experience weird regular > (once a day-two) sporadic reboots after last upgrade (2015-07-17), and > would appreciate if someone could help me to analyze further and fix the > issue. > > Collected crash information below > > full logs are available upon request. For the record, I observed a similar crash today: ceuta-ng.act-europe.fr dumped core - see /var/crash/vmcore.2 Tue Sep 29 18:25:59 CEST 2015 FreeBSD ceuta-ng.act-europe.fr 10.1-RELEASE-p8 FreeBSD 10.1-RELEASE-p8 #2 r280990M: Tue Jul 7 18:18:21 CEST 2015 root@lynchburg:/usr/obj/usr/src/releng/10.1/sys/ADACORE_FW amd64 panic: page fault GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid = 3; apic id = 03 fault virtual address = 0xffff804000000000 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80d4f81c stack pointer = 0x28:0xfffffe011bfaf810 frame pointer = 0x28:0xfffffe011bfaf820 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 12 (swi1: netisr 0) trap number = 12 panic: page fault cpuid = 3 KDB: stack backtrace: #0 0xffffffff80977830 at kdb_backtrace+0x60 #1 0xffffffff8093c955 at panic+0x155 #2 0xffffffff80d5dc3f at trap_fatal+0x38f #3 0xffffffff80d5df58 at trap_pfault+0x308 #4 0xffffffff80d5d5ba at trap+0x47a #5 0xffffffff80d434a2 at calltrap+0x8 #6 0xffffffff80e7000b at bounce_bus_dmamap_load_buffer+0x1bb #7 0xffffffff80972b42 at bus_dmamap_load_mbuf_sg+0x72 #8 0xffffffff819ca83e at igb_refresh_mbufs+0x19e #9 0xffffffff819ca642 at igb_rxeof+0x692 #10 0xffffffff819cb9a0 at igb_poll+0x110 #11 0xffffffff8092bd6c at netisr_poll+0xfc #12 0xffffffff80a09c3d at swi_net+0x1d #13 0xffffffff8090ea8b at intr_event_execute_handlers+0xab #14 0xffffffff8090eed6 at ithread_loop+0x96 #15 0xffffffff8090c6aa at fork_exit+0x9a #16 0xffffffff80d439de at fork_trampoline+0xe Uptime: 3d17h55m1s Dumping 448 out of 4069 MB:..4%..11%..22%..33%..43%..54%..61%..72%..83%..93% Reading symbols from /boot/kernel/carp.ko.symbols...done. Loaded symbols for /boot/kernel/carp.ko.symbols Reading symbols from /boot/kernel/if_igb.ko.symbols...done. Loaded symbols for /boot/kernel/if_igb.ko.symbols Reading symbols from /boot/kernel/pflog.ko.symbols...done. Loaded symbols for /boot/kernel/pflog.ko.symbols Reading symbols from /boot/kernel/pf.ko.symbols...done. Loaded symbols for /boot/kernel/pf.ko.symbols Reading symbols from /boot/kernel/pfsync.ko.symbols...done. Loaded symbols for /boot/kernel/pfsync.ko.symbols Reading symbols from /boot/kernel/ng_socket.ko.symbols...done. Loaded symbols for /boot/kernel/ng_socket.ko.symbols Reading symbols from /boot/kernel/netgraph.ko.symbols...done. Loaded symbols for /boot/kernel/netgraph.ko.symbols Reading symbols from /boot/kernel/ng_mppc.ko.symbols...done. Loaded symbols for /boot/kernel/ng_mppc.ko.symbols Reading symbols from /boot/kernel/rc4.ko.symbols...done. Loaded symbols for /boot/kernel/rc4.ko.symbols Reading symbols from /boot/kernel/if_tap.ko.symbols...done. Loaded symbols for /boot/kernel/if_tap.ko.symbols #0 doadump (textdump=) at pcpu.h:219 219 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump (textdump=) at pcpu.h:219 #1 0xffffffff8093c5d2 in kern_reboot (howto=260) at /usr/src/releng/10.1/sys/kern/kern_shutdown.c:452 #2 0xffffffff8093c994 in panic (fmt=) at /usr/src/releng/10.1/sys/kern/kern_shutdown.c:759 #3 0xffffffff80d5dc3f in trap_fatal (frame=, eva=) at /usr/src/releng/10.1/sys/amd64/amd64/trap.c:865 #4 0xffffffff80d5df58 in trap_pfault (frame=0xfffffe011bfaf760, usermode=) at /usr/src/releng/10.1/sys/amd64/amd64/trap.c:676 #5 0xffffffff80d5d5ba in trap (frame=0xfffffe011bfaf760) at /usr/src/releng/10.1/sys/amd64/amd64/trap.c:440 #6 0xffffffff80d434a2 in calltrap () at /usr/src/releng/10.1/sys/amd64/amd64/exception.S:232 #7 0xffffffff80d4f81c in pmap_kextract (va=0) at /usr/src/releng/10.1/sys/amd64/amd64/pmap.c:639 #8 0xffffffff80e7000b in bounce_bus_dmamap_load_buffer ( dmat=0xfffff80002b8ae00, map=0xffffffff81637498, buf=, buflen=, pmap=0xffffffff816beb60, flags=, segs=) at /usr/src/releng/10.1/sys/x86/x86/busdma_bounce.c:690 #9 0xffffffff80972b42 in bus_dmamap_load_mbuf_sg (dmat=0xfffff80002b8ae00, map=0x0, m0=, segs=0xfffffe011bfaf930, nsegs=0xfffffe011bfaf92c, flags=) at /usr/src/releng/10.1/sys/kern/subr_bus_dma.c:110 #10 0xffffffff819ca83e in igb_refresh_mbufs (rxr=0xfffff80002b71330, limit=633) at /usr/src/releng/10.1/sys/modules/igb/../../dev/e1000/if_igb.c:4144 #11 0xffffffff819ca642 in igb_rxeof (que=, count=, done=0xfffffe011bfafa44) at /usr/src/releng/10.1/sys/modules/igb/../../dev/e1000/if_igb.c:4998 #12 0xffffffff819cb9a0 in igb_poll (ifp=0xfffff80002b70800, cmd=, count=5) at /usr/src/releng/10.1/sys/modules/igb/../../dev/e1000/if_igb.c:1565 #13 0xffffffff8092bd6c in netisr_poll () at /usr/src/releng/10.1/sys/kern/kern_poll.c:447 #14 0xffffffff80a09c3d in swi_net (arg=0xffffffff81a8a880) at /usr/src/releng/10.1/sys/net/netisr.c:797 #15 0xffffffff8090ea8b in intr_event_execute_handlers ( p=, ie=0xfffff80002962c00) at /usr/src/releng/10.1/sys/kern/kern_intr.c:1263 #16 0xffffffff8090eed6 in ithread_loop (arg=0xfffff80002967f40) at /usr/src/releng/10.1/sys/kern/kern_intr.c:1276 #17 0xffffffff8090c6aa in fork_exit ( callout=0xffffffff8090ee40 , arg=0xfffff80002967f40, frame=0xfffffe011bfafc00) at /usr/src/releng/10.1/sys/kern/kern_fork.c:996 #18 0xffffffff80d439de in fork_trampoline () at /usr/src/releng/10.1/sys/amd64/amd64/exception.S:606 #19 0x0000000000000000 in ?? () Current language: auto; currently minimal (kgdb) [...] Full log and coredump have been preserved. Thomas.