From owner-freebsd-bugs@FreeBSD.ORG Mon Sep 25 13:30:32 2006 Return-Path: X-Original-To: freebsd-bugs@hub.freebsd.org Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4771416A4AB for ; Mon, 25 Sep 2006 13:30:32 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6F12143D5F for ; Mon, 25 Sep 2006 13:30:31 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id k8PDUVTN007158 for ; Mon, 25 Sep 2006 13:30:31 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id k8PDUVpU007156; Mon, 25 Sep 2006 13:30:31 GMT (envelope-from gnats) Resent-Date: Mon, 25 Sep 2006 13:30:31 GMT Resent-Message-Id: <200609251330.k8PDUVpU007156@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Daniele Pilenga Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7BC7616A47E for ; Mon, 25 Sep 2006 13:25:47 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (www.freebsd.org [216.136.204.117]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4EDB643D6E for ; Mon, 25 Sep 2006 13:25:40 +0000 (GMT) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.13.1/8.13.1) with ESMTP id k8PDPdoK014745 for ; Mon, 25 Sep 2006 13:25:39 GMT (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.13.1/8.13.1/Submit) id k8PDPd8U014725; Mon, 25 Sep 2006 13:25:39 GMT (envelope-from nobody) Message-Id: <200609251325.k8PDPd8U014725@www.freebsd.org> Date: Mon, 25 Sep 2006 13:25:39 GMT From: Daniele Pilenga To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-2.3 Cc: Subject: kern/103619: Kernel panic (page fault) during normal operation X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Sep 2006 13:30:32 -0000 >Number: 103619 >Category: kern >Synopsis: Kernel panic (page fault) during normal operation >Confidential: no >Severity: serious >Priority: high >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Mon Sep 25 13:30:30 GMT 2006 >Closed-Date: >Last-Modified: >Originator: Daniele Pilenga >Release: FreeBSD 6.1-RELEASE-p7 >Organization: Atos Origin Italia s.p.a. >Environment: FreeBSD srvbsd01.gruppo.bipielle 6.1-RELEASE-p7 FreeBSD 6.1-RELEASE-p7 #0: Wed Sep 20 18:02:58 CEST 2006 root@srvbsd01.gruppo.bipielle:/usr/obj/usr/src/sys/SRVBSD i386 >Description: During normal operation, no particular stress or other activities involved, the kernel panics. It's an HP DL360G3, with 2 Xeon 3.2GHz and 2GB of ram acting as squid, bind, dhcp server, 4 carp interfaces defined as fail over with an other identical machine which exhibits the same problem. Kernel config follows: include GENERIC makeoptions DEBUG=-g options SMP ident SRVBSD nocpu I486_CPU nocpu I586_CPU device carp device pf device pflog kgdb follows: kgdb /boot/kernel/kernel.debug vmcore.0 [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 06 fault virtual address = 0x4 fault code = supervisor read, page not present instruction pointer = 0x20:0xc0719e1b stack pointer = 0x28:0xe8e608d8 frame pointer = 0x28:0xe8e608e8 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 64389 (sockstat) trap number = 12 panic: page fault cpuid = 1 Uptime: 1d18h59m4s Dumping 2047 MB (2 chunks) chunk 0: 1MB (159 pages) ... ok chunk 1: 2047MB (524026 pages) 2032 2016 2000 1984 1968 1952 1936 1920 1904 1888 1872 1856 1840 18 24 1808 1792 1776 1760 1744 1728 1712 1696 1680 1664 1648 1632 1616 1600 1584 1568 1552 1536 1520 15 04 1488 1472 1456 1440 1424 1408 1392 1376 1360 1344 1328 1312 1296 1280 1264 1248 1232 1216 1200 11 84 1168 1152 1136 1120 1104 1088 1072 1056 1040 1024 1008 992 976 960 944 928 912 896 880 864 848 83 2 816 800 784 768 752 736 720 704 688 672 656 640 624 608 592 576 560 544 528 512 496 480 464 448 43 2 416 400 384 368 352 336 320 304 288 272 256 240 224 208 192 176 160 144 128 112 96 80 64 48 32 16 #0 doadump () at pcpu.h:165 165 pcpu.h: No such file or directory. in pcpu.h (kgdb) where #0 doadump () at pcpu.h:165 #1 0xc06d10ed in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:402 #2 0xc06d149e in panic (fmt=0xc0969b52 "%s") at /usr/src/sys/kern/kern_shutdown.c:558 #3 0xc091a655 in trap_fatal (frame=0xe8e60898, eva=0) at /usr/src/sys/i386/i386/trap.c:836 #4 0xc091a349 in trap_pfault (frame=0xe8e60898, usermode=0, eva=4) at /usr/src/sys/i386/i386/trap.c:744 #5 0xc0919f2f in trap (frame= {tf_fs = -1064304632, tf_es = -969080792, tf_ds = -387579864, tf_edi = 1542, tf_esi = 0, tf_eb p = -387577624, tf_isp = -387577660, tf_ebx = -387577072, tf_edx = -387577396, tf_ecx = 0, tf_eax = 0, tf_trapno = 12, tf_err = 0, tf_eip = -1066295781, tf_cs = 32, tf_eflags = 66198, tf_esp = -387577 072, tf_ss = 0}) at /usr/src/sys/i386/i386/trap.c:434 #6 0xc090546a in calltrap () at /usr/src/sys/i386/i386/exception.s:139 #7 0xc0719e1b in sotoxsocket (so=0x0, xso=0xe8e60b10) at /usr/src/sys/kern/uipc_socket2.c:1454 #8 0xc0799eb4 in tcp_pcblist (oidp=0xc0a02f40, arg1=0x0, arg2=0, req=0xe8e60bf8) at /usr/src/sys/netinet/tcp_subr.c:964 #9 0xc06daba2 in sysctl_root (oidp=0x0, arg1=0x0, arg2=0, req=0xe8e60bf8) at /usr/src/sys/kern/kern_sysctl.c:1285 #10 0xc06dae3c in userland_sysctl (td=0x0, name=0xe8e60c68, namelen=4, old=0xe8e60bf8, oldlenp=0xbfbfedf8, inkernel=0, new=0x0, newlen=0, retval=0xe8e60c64, flags=0) at /usr/src/sys/kern/kern_sysctl.c:1384 #11 0xc06dac8d in __sysctl (td=0x0, uap=0xe8e60d04) at /usr/src/sys/kern/kern_sysctl.c:1319 #12 0xc091aa3c in syscall (frame= {tf_fs = 59, tf_es = 59, tf_ds = 59, tf_edi = -1077940744, tf_esi = -1077940928, tf_ebp = -107 7940984, tf_isp = -387576476, tf_ebx = 672416808, tf_edx = 0, tf_ecx = 134594560, tf_eax = 202, tf_t rapno = 12, tf_err = 2, tf_eip = 672265851, tf_cs = 51, tf_eflags = 663, tf_esp = -1077941028, tf_ss = 59}) at /usr/src/sys/i386/i386/trap.c:981 #13 0xc09054bf in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:200 #14 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) up 7 #7 0xc0719e1b in sotoxsocket (so=0x0, xso=0xe8e60b10) at /usr/src/sys/kern/uipc_socket2.c:1454 1454 xso->xso_so = so; (kgdb) p so $1 = (struct socket *) 0x0 It seems the problem is here, but my understanding of the kernel are too limited. I can do further tests if needed. >How-To-Repeat: The problem started after my first uses of the carp interface, but I have only recently upgraded from 5.4 and I can not be sure this is the only difference. >Fix: >Release-Note: >Audit-Trail: >Unformatted: