From owner-freebsd-i386@FreeBSD.ORG Tue Feb 24 12:59:36 2004 Return-Path: Delivered-To: freebsd-i386@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5C4BF16A4CE for ; Tue, 24 Feb 2004 12:59:36 -0800 (PST) Received: from spitfire.ecc.engr.uky.edu (spitfire.ecc.engr.uky.edu [128.163.144.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id C5B1143D1F for ; Tue, 24 Feb 2004 12:59:33 -0800 (PST) (envelope-from dhensley@engr.uky.edu) Received: from bort.ecc.engr.uky.edu ([128.163.144.93] helo=engr.uky.edu) by spitfire.ecc.engr.uky.edu with esmtp (Exim 4.30) id 1AvjeO-0008wX-48 for freebsd-i386@freebsd.org; Tue, 24 Feb 2004 15:59:32 -0500 Message-ID: <403BBB82.5030805@engr.uky.edu> Date: Tue, 24 Feb 2004 16:00:50 -0500 From: Dave Hensley User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.6) Gecko/20040218 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-i386@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 {/} Subject: Crashes (Fatal trap 12: page fault while in kernel mode) X-BeenThere: freebsd-i386@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: I386-specific issues for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2004 20:59:36 -0000 I have a 4.9-RELEASE-p1 system that crashes a few times per week. The invalid page fault seems to be triggered by nfs, as shown by the backtrace, though with my (very) limited kernel debugging experience I've been unable to diagnose further. Perhaps someone has had similar problems and knows how to fix it, or can at least give me some ideas to try? Any help would be very well appreciated. Thanks, Dave. Below is the output from gdb: SMP 4 cpus IdlePTD at phsyical address 0x00572000 initial pcb at physical address 0x0049bfe0 panicstr: page fault panic messages: --- Fatal trap 12: page fault while in kernel mode mp_lock = 00000002; cpuid = 0; lapic.id = 00000000 fault virtual address = 0x21 fault code = supervisor read, page not present instruction pointer = 0x8:0xc0253050 stack pointer = 0x10:0xf1e9bd34 frame pointer = 0x10:0xf1e9bd70 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 4274 (imapd) interrupt mask = bio <- SMP: XXX trap number = 12 panic: page fault mp_lock = 00000002; cpuid = 0; lapic.id = 00000000 boot() called on cpu#0 syncing disks... 2118 2101 2101 2100 2100 2100 2100 2100 2100 2100 2109 2098 2098 2098 2098 2098 2098 2098 2101 2098 2098 2098 2098 2098 2098 2098 2098 2098 2098 2098 2098 2098 2098 2098 2098 2098 2098 2098 2098 done Uptime: 1d0h8m27s dumping to dev #aacd/0x20001, offset 4194432 dump 2047 2046 2045 2044 2043 2042 2041 ------------snip!------------- 4 3 2 1 0 succeeded aac0: shutting down controller... --- #0 dumpsys () at /usr/src/sys/kern/kern_shutdown.c:487 487 if (dumping++) { (kgdb) l *0xc0253050 0xc0253050 is in brelse (/usr/src/sys/kern/vfs_bio.c:993). 988 989 KASSERT(!(bp->b_flags & (B_CLUSTER|B_PAGING)), ("brelse: inappropriate B_PAGING or B_CLUSTER bp %p", bp)); 990 991 s = splbio(); 992 993 if (bp->b_flags & B_LOCKED) 994 bp->b_flags &= ~B_ERROR; 995 996 if ((bp->b_flags & (B_READ | B_ERROR | B_INVAL)) == B_ERROR) { 997 /* (kgdb) bt #0 dumpsys () at /usr/src/sys/kern/kern_shutdown.c:487 #1 0xc022c71b in boot (howto=256) at /usr/src/sys/kern/kern_shutdown.c:316 #2 0xc022cb74 in poweroff_wait (junk=0xc042e4d9, howto=-1069359217) at /usr/src/sys/kern/kern_shutdown.c:595 #3 0xc03abda4 in trap_fatal (frame=0xf1e9bcf4, eva=33) at /usr/src/sys/i386/i386/trap.c:974 #4 0xc03aba35 in trap_pfault (frame=0xf1e9bcf4, usermode=0, eva=33) at /usr/src/sys/i386/i386/trap.c:867 #5 0xc03ab5a7 in trap (frame={tf_fs = 24, tf_es = 16, tf_ds = 16, tf_edi = 0, tf_esi = 0, tf_ebp = -236339856, tf_isp = -236339936, tf_ebx = 0, tf_edx = 1745141856, tf_ecx = 32, tf_eax = 0, tf_trapno = 12, tf_err = 0, tf_eip = -1071304624, tf_cs = 8, tf_eflags = 66054, tf_esp = 0, tf_ss = -227323200}) at /usr/src/sys/i386/i386/trap.c:466 #6 0xc0253050 in brelse (bp=0x0) at /usr/src/sys/kern/vfs_bio.c:991 #7 0xc02b1c29 in nfs_bioread (vp=0xf1a32dc0, uio=0xf1e9be78, ioflag=0, cred=0xc50b4800) at /usr/src/sys/nfs/nfs_bio.c:710 #8 0xc02da315 in nfs_readlink (ap=0xf1e9be68) at /usr/src/sys/nfs/nfs_vnops.c:1037 #9 0xc025931e in namei (ndp=0xf1e9bedc) at vnode_if.h:794 #10 0xc025e149 in change_dir (ndp=0xf1e9bedc, p=0xf1313ee0) at /usr/src/sys/kern/vfs_syscalls.c:969 #11 0xc025dffc in chdir (p=0xf1313ee0, uap=0xf1e9bf80) at /usr/src/sys/kern/vfs_syscalls.c:870 #12 0xc03ac0d5 in syscall2 (frame={tf_fs = 47, tf_es = 47, tf_ds = 47, tf_edi = 1, tf_esi = 674056736, tf_ebp = -1077938508, tf_isp = -236339244, tf_ebx = 0, tf_edx = 134886594, tf_ecx = 135323464, tf_eax = 12, tf_trapno = 12, tf_err = 2, tf_eip = 673714528, tf_cs = 31, tf_eflags = 518, tf_esp = -1077938552, tf_ss = 47}) at /usr/src/sys/i386/i386/trap.c:1175 #13 0xc03964eb in Xint0x80_syscall () #14 0x8056737 in ?? () #15 0x8055b8a in ?? () #16 0x8068693 in ?? () #17 0x804b2ef in ?? () #18 0x804a89a in ?? () ----------------------------------------------- From dmesg: Copyright (c) 1992-2003 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 4.9-RELEASE-p1 #0: Mon Feb 2 16:34:57 EST 2004 Timecounter "i8254" frequency 1193182 Hz CPU: Intel(R) Xeon(TM) CPU 1.80GHz (1789.10-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf27 Stepping = 7 Features=0xbfebfbff Hyperthreading: 2 logical CPUs real memory = 2147418112 (2097088K bytes) avail memory = 2086187008 (2037292K bytes) Changing APIC ID for IO APIC #0 from 0 to 4 on chip Changing APIC ID for IO APIC #1 from 0 to 5 on chip Changing APIC ID for IO APIC #2 from 0 to 6 on chip Programming 16 pins in IOAPIC #0 IOAPIC #0 intpin 2 -> irq 0 Programming 16 pins in IOAPIC #1 Programming 16 pins in IOAPIC #2 FreeBSD/SMP: Multiprocessor motherboard: 4 CPUs cpu0 (BSP): apic id: 0, version: 0x00050014, at 0xfee00000 cpu1 (AP): apic id: 1, version: 0x00050014, at 0xfee00000 cpu2 (AP): apic id: 2, version: 0x00050014, at 0xfee00000 cpu3 (AP): apic id: 3, version: 0x00050014, at 0xfee00000 io0 (APIC): apic id: 4, version: 0x000f0011, at 0xfec00000 io1 (APIC): apic id: 5, version: 0x000f0011, at 0xfec01000 io2 (APIC): apic id: 6, version: 0x000f0011, at 0xfec02000 Preloaded elf kernel "kernel" at 0xc0553000. Warning: Pentium 4 CPU: PSE disabled Pentium Pro MTRR support enabled md0: Malloc disk Using $PIR table, 9 entries at 0xc00fc490 npx0: on motherboard npx0: INT 16 interface pcib0: on motherboard IOAPIC #1 intpin 3 -> irq 2 IOAPIC #1 intpin 7 -> irq 7 IOAPIC #1 intpin 11 -> irq 10 pci0: on pcib0 pci0: (vendor=0x1028, dev=0x000c) at 4.0 irq 2 pci0: (vendor=0x1028, dev=0x0008) at 4.1 irq 7 pci0: (vendor=0x1028, dev=0x000d) at 4.2 irq 10 pci0: at 14.0 atapci0: port 0x8b0-0x8bf,0x8d8-0x8db,0x8d0-0x8d7,0x8c8-0x8cb,0x8c0-0x8c7 at device 15.1 on pci0 ata0: at 0x1f0 irq 14 on atapci0 ata1: at 0x170 irq 15 on atapci0 ohci0: mem 0xfe100000-0xfe100fff irq 5 at device 15.2 on pci0 usb0: OHCI version 1.0, legacy support usb0: SMM does not respond, resetting usb0: on ohci0 usb0: USB revision 1.0 uhub0: (0x1166) OHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 4 ports with 4 removable, self powered isab0: at device 15.3 on pci0 isa0: on isab0 pcib1: on motherboard pci1: on pcib1 pcib8: at device 8.0 on pci1 IOAPIC #1 intpin 4 -> irq 11 IOAPIC #1 intpin 5 -> irq 13 pci2: on pcib8 fxp0: port 0xdce0-0xdcff mem 0xfce00000-0xfcefffff,0xfcbff000-0xfcbfffff irq 11 at device 4.0 on pci2 fxp0: Ethernet address 00:02:b3:be:30:33 inphy0: on miibus0 inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto fxp1: port 0xdcc0-0xdcdf mem 0xfcd00000-0xfcdfffff,0xfcbfe000-0xfcbfefff irq 13 at device 5.0 on pci2 fxp1: Ethernet address 00:02:b3:be:30:34 inphy1: on miibus1 inphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto pcib2: on motherboard pci3: on pcib2 pcib3: on motherboard IOAPIC #1 intpin 12 -> irq 16 IOAPIC #1 intpin 13 -> irq 17 pci4: on pcib3 bge0: mem 0xfca10000-0xfca1ffff irq 16 at device 6.0 on pci4 bge0: Ethernet address: 00:06:5b:8f:f2:f7 miibus2: on bge0 brgphy0: on miibus2 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto bge1: mem 0xfca00000-0xfca0ffff irq 17 at device 8.0 on pci4 bge1: Ethernet address: 00:06:5b:8f:f2:f8 miibus3: on bge1 brgphy1: on miibus3 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto pcib4: on motherboard IOAPIC #1 intpin 14 -> irq 18 pci5: on pcib4 pcib9: at device 8.0 on pci5 IOAPIC #1 intpin 15 -> irq 19 pci6: on pcib9 pci6: (vendor=0x9005, dev=0x00c5) at 6.0 irq 18 pci6: (vendor=0x9005, dev=0x00c5) at 6.1 irq 19 aac0: mem 0xf0000000-0xf7ffffff irq 18 at device 8.1 on pci5 aac0: i960RX 100MHz, 118MB cache memory, optional battery present aac0: Kernel 2.7-1, Build 3170, S/N 441d3 aac0: Supported Options=75c pcib5: on motherboard pci7: on pcib5 pcib6: on motherboard pci8: on pcib6 pcib7: on motherboard pci9: on pcib7 orm0: