From owner-freebsd-hackers@FreeBSD.ORG Tue Jul 19 03:42:16 2005 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4FA5116A41C for ; Tue, 19 Jul 2005 03:42:16 +0000 (GMT) (envelope-from edwin@verolan.com) Received: from ns11.webmasters.com (ns11.webmasters.com [66.118.156.2]) by mx1.FreeBSD.org (Postfix) with SMTP id E03B143D45 for ; Tue, 19 Jul 2005 03:42:15 +0000 (GMT) (envelope-from edwin@verolan.com) Received: (qmail 31320 invoked from network); 19 Jul 2005 03:39:07 -0000 Received: from unknown (HELO localhost.localdomain) (204.9.60.14) by ns11.webmasters.com with SMTP; 19 Jul 2005 03:39:07 -0000 Received: from localhost.localdomain (asx01 [127.0.0.1]) by localhost.localdomain (8.13.1/8.13.1) with ESMTP id j6J3gFIQ020768; Mon, 18 Jul 2005 23:42:15 -0400 Received: (from edwin@localhost) by localhost.localdomain (8.13.1/8.13.1/Submit) id j6J3gFXp020767; Mon, 18 Jul 2005 23:42:15 -0400 Date: Mon, 18 Jul 2005 23:42:15 -0400 From: Edwin To: freebsd-hackers@freebsd.org Message-ID: <20050719034215.GB20752@asx01.verolan.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.1i X-Operating-System: Linux/(i686) Cc: edwin@verolan.com Subject: help w/panic under heavy load - 5.4 X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 19 Jul 2005 03:42:16 -0000 Hi, I have a recurring (re-producible) panic on the 5.3/5.4 kernels and I would like to ask for some help in tracking it down. :) - it could be some misconfig on my part - but i have tried several different configs of the kernel - ultimately w/ polling on/off, ipfw on/off, ipfastforwarding on/off - although with ipff off - the box still crashes but in a different location - it will even crash w/ GENERIC kernel under heavy load. I'm not quite sure where to look past the below (ie. what variables/etc to present to the list). Thanks! /edwin details are: - Soekris net4801 platform (SBC w/ Geode processor - i586 class). - similar crash was seen (as regularly) under 5.3 as well - just never got around to getting crashdump/etc. - crash is re-producible under heavy load here is the crash-info: fb54c# Fatal trap 12: page fault while in kernel mode fault virtual address = 0xc fault code = supervisor read, page not present instruction pointer = 0x8:0xc0647527 stack pointer = 0x10:0xc7692b78 frame pointer = 0x10:0xc7692b9c code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 29 (swi1: net) trap number = 12 panic: page fault Uptime: 1m50s Dumping 128 MB 16 32 48 64 80 96 112 Dump complete Automatic reboot in 15 seconds - press a key on the console to abort ** and the kgdb output ** mbsd05# uname -a FreeBSD mbsd05.maplecreek.org 5.4-RELEASE FreeBSD 5.4-RELEASE #0: Fri Jul 15 01:34:05 EDT 2005 root@mbsd05.maplecreek.org:/usr/obj/usr/src/sys/SMP_OPT i386 mbsd05# kgdb ./kernel.debug /tmp/crash/vmcore.3 [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". #0 doadump () at pcpu.h:159 159 __asm __volatile("movl %%fs:0,%0" : "=r" (td)); (kgdb) (kgdb) l *0xc0647527 0xc0647527 is in m_copym (/usr/src/sys/kern/uipc_mbuf.c:386). 381 MBUF_CHECKSLEEP(wait); 382 if (off == 0 && m->m_flags & M_PKTHDR) 383 copyhdr = 1; 384 while (off > 0) { 385 KASSERT(m != NULL, ("m_copym, offset > size of mbuf chain")); 386 if (off < m->m_len) 387 break; 388 off -= m->m_len; 389 m = m->m_next; 390 } (kgdb) (kgdb) where #0 doadump () at pcpu.h:159 #1 0xc0615326 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:410 #2 0xc06155bc in panic (fmt=0xc0819052 "%s") at /usr/src/sys/kern/kern_shutdown.c:566 #3 0xc07d141c in trap_fatal (frame=0xc7692b38, eva=12) at /usr/src/sys/i386/i386/trap.c:817 #4 0xc07d1187 in trap_pfault (frame=0xc7692b38, usermode=0, eva=12) at /usr/src/sys/i386/i386/trap.c:735 #5 0xc07d0dc9 in trap (frame= {tf_fs = -1057423336, tf_es = 16, tf_ds = -1057226736, tf_edi = -1056791980, tf_esi = 1432, tf_ebp = -949408868, tf_isp = -949408924, tf_ebx = -1056792064, tf_edx = 0, tf_ecx = -1055088626, tf_eax = 0, tf_trapno = 12, tf_err = -1066926080, tf_eip = -1067158233, tf_cs = 8, tf_eflags = 66050, tf_esp = 0, tf_ss = -1065850424}) at /usr/src/sys/i386/i386/trap.c:425 #6 0xc07c12ca in calltrap () at /usr/src/sys/i386/i386/exception.s:140 #7 0xc0f90018 in ?? () #8 0x00000010 in ?? () #9 0xc0fc0010 in ?? () #10 0xc102a254 in ?? () #11 0x00000598 in ?? () #12 0xc7692b9c in ?? () #13 0xc7692b64 in ?? () #14 0xc102a200 in ?? () #15 0x00000000 in ?? () #16 0xc11ca00e in ?? () #17 0x00000000 in ?? () #18 0x0000000c in ?? () #19 0xc0680000 in pppdealloc (sc=0xc102a200) at /usr/src/sys/net/if_ppp.c:388 #20 0xc06ac168 in ip_fragment (ip=0xc11ca00e, m_frag=0xc7692c48, mtu=-1056791980, if_hwassist_flags=0, sw_csum=1) at /usr/src/sys/netinet/ip_output.c:967 #21 0xc06a39c8 in ip_fastforward (m=0xc11b8600) at /usr/src/sys/netinet/ip_fastfwd.c:572 #22 0xc067d65d in ether_demux (ifp=0xc0f91800, m=0xc11b8600) at /usr/src/sys/net/if_ethersubr.c:770 #23 0xc067d419 in ether_input (ifp=0xc0f91800, m=0xc11b8600) at /usr/src/sys/net/if_ethersubr.c:631 #24 0xc0729c2f in sis_rxeof (sc=0xc0f91800) at /usr/src/sys/pci/if_sis.c:1636 #25 0xc0729f6b in sis_poll (ifp=0xc0f91800, cmd=POLL_ONLY, count=0) at /usr/src/sys/pci/if_sis.c:1769 #26 0xc05f8398 in netisr_poll () at /usr/src/sys/kern/kern_poll.c:384 #27 0xc0684f7a in swi_net (dummy=0x0) at /usr/src/sys/net/netisr.c:338 #28 0xc0601c39 in ithread_loop (arg=0xc0ec6480) at /usr/src/sys/kern/kern_intr.c:547 #29 0xc0600ecc in fork_exit (callout=0xc0601ae8 , arg=0xc0ec6480, frame=0xc7692d48) at /usr/src/sys/kern/kern_fork.c:791 #30 0xc07c132c in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:209 (kgdb)