From owner-freebsd-net@FreeBSD.ORG Wed Apr 9 17:10:03 2008 Return-Path: Delivered-To: freebsd-net@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7B3BD1065675 for ; Wed, 9 Apr 2008 17:10:03 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 6011A8FC24 for ; Wed, 9 Apr 2008 17:10:03 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m39HA3fk055513 for ; Wed, 9 Apr 2008 17:10:03 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m39HA3Tj055511; Wed, 9 Apr 2008 17:10:03 GMT (envelope-from gnats) Date: Wed, 9 Apr 2008 17:10:03 GMT Message-Id: <200804091710.m39HA3Tj055511@freefall.freebsd.org> To: freebsd-net@FreeBSD.org From: Szymon Roczniak Cc: Subject: Re: kern/121298: [em] [panic] Fatal trap 12: page fault while in kernel mode (em0 taskq) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Szymon Roczniak List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2008 17:10:03 -0000 The following reply was made to PR kern/121298; it has been noted by GNATS. From: Szymon Roczniak To: bug-followup@FreeBSD.org Cc: Subject: Re: kern/121298: [em] [panic] Fatal trap 12: page fault while in kernel mode (em0 taskq) Date: Wed, 9 Apr 2008 17:49:00 +0100 A similar problem here (well, at least two out of three kernel dumps look a bit similar). The hardware is a dell PE2850, running GENERIC (only file name is different) FreeBSD 7-STABLE as of yesterday (8.04.2008). It's a web/mail server and it's not under heavy load. server# dmesg=0D Copyright (c) 1992-2008 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.0-STABLE #2: Tue Apr 8 12:04:29 BST 2008 szymon@server:/usr/obj/usr/src/sys/VALIUM Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(TM) CPU 2.80GHz (2793.01-MHz 686-class CPU) Origin =3D "GenuineIntel" Id =3D 0xf41 Stepping =3D 1 Features=3D0xbfebfbff Features2=3D0x641d AMD Features=3D0x20000000 real memory =3D 2147221504 (2047 MB) avail memory =3D 2091679744 (1994 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 6 ioapic0: Changing APIC ID to 7 ioapic1: Changing APIC ID to 8 ioapic2: Changing APIC ID to 9 ioapic3: Changing APIC ID to 10 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 32-55 on motherboard ioapic2 irqs 64-87 on motherboard ioapic3 irqs 96-119 on motherboard kbd1 at kbdmux0 ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 acpi_hpet0: iomem 0xfed00000-0xfed003ff on acp= i0 Timecounter "HPET" frequency 14318180 Hz quality 900 cpu0: on acpi0 p4tcc0: on cpu0 cpu1: on acpi0 p4tcc1: on cpu1 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 2.0 on pci0 pci1: on pcib1 pcib2: at device 0.0 on pci1 pci2: on pcib2 amr0: mem 0xd80f0000-0xd80fffff,0xdfdc0000-0xdfdff= fff irq 46 at device 14.0 on pci2 amr0: [ITHREAD] amr0: delete logical drives supported by controller amr0: Firmware 513O, BIOS H418, 256MB RAM pcib3: at device 0.2 on pci1 pci3: on pcib3 pcib4: at device 4.0 on pci0 pci4: on pcib4 pcib5: at device 5.0 on pci0 pci5: on pcib5 pcib6: at device 0.0 on pci5 pci6: on pcib6 em0: port 0xecc0-0xe= cff mem 0xdfae0000-0xdfafffff irq 64 at device 7.0 on pci6 em0: Ethernet address: 00:11:43:36:f3:3c em0: [FILTER] pcib7: at device 0.2 on pci5 pci7: on pcib7 em1: port 0xdcc0-0xd= cff mem 0xdf8e0000-0xdf8fffff irq 65 at device 8.0 on pci7 em1: Ethernet address: 00:11:43:36:f3:3d em1: [FILTER] pcib8: at device 6.0 on pci0 pci8: on pcib8 pcib9: at device 0.0 on pci8 pci9: on pcib9 pcib10: at device 0.2 on pci8 pci10: on pcib10 uhci0: port 0xbce0-0xbcff irq 1= 6 at device 29.0 on pci0 uhci0: [GIANT-LOCKED] uhci0: [ITHREAD] usb0: on uhci0 usb0: USB revision 1.0 uhub0: on usb0 uhub0: 2 ports with 2 removable, self powered uhci1: port 0xbcc0-0xbcdf irq 1= 9 at device 29.1 on pci0 uhci1: [GIANT-LOCKED] uhci1: [ITHREAD] usb1: on uhci1 usb1: USB revision 1.0 uhub1: on usb1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0xbca0-0xbcbf irq 1= 8 at device 29.2 on pci0 uhci2: [GIANT-LOCKED] uhci2: [ITHREAD] usb2: on uhci2 usb2: USB revision 1.0 uhub2: on usb2 uhub2: 2 ports with 2 removable, self powered ehci0: mem 0xdff00000-0xdff003f= f irq 23 at device 29.7 on pci0 ehci0: [GIANT-LOCKED] ehci0: [ITHREAD] usb3: EHCI version 1.0 usb3: companion controllers, 2 ports each: usb0 usb1 usb2 usb3: on ehci0 usb3: USB revision 2.0 uhub3: on usb3 uhub3: 6 ports with 6 removable, self powered uhub4: on = uhub3 uhub4: multiple transaction translators uhub4: 2 ports with 2 removable, self powered pcib11: at device 30.0 on pci0 pci11: on pcib11 vgapci0: port 0xcc00-0xccff mem 0xd0000000-0xd7fff= fff,0xdf4f0000-0xdf4fffff irq 18 at device 13.0 on pci11 isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177= ,0x376,0xfc00-0xfc0f at device 31.1 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: [FILTER] fd0: <1440-KB 3.5" drive> on fdc0 drive 0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acp= i0 sio0: type 16550A sio0: [FILTER] pmtimer0 on isa0 orm0: at iomem 0xc0000-0xcafff,0xcb000-0xcbfff,0xce800-0x= cf7ff,0xec000-0xeffff pnpid ORM0000 on isa0 ppc0: parallel port not found. sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=3D0x300> sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounters tick every 1.000 msec acd0: CDROM at ata0-master UDMA33 amr0: delete logical drives supported by controller amrd0: on amr0 amrd0: 34680MB (71024640 sectors) RAID 1 (optimal) SMP: AP CPU #1 Launched! Trying to mount root from ufs:/dev/amrd0s1a WARNING: / was not properly dismounted /: mount pending error: blocks 204 files 14 pid 3380 (httpd), uid 65534: exited on signal 11 server# kgdb /boot/kernel/kernel vm=07=0D vmcore.0 vmcore.1 vmcore.2 =0D server# kgdb /boot/kernel/kernel vmcore.0=0D [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so:= Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain condition= s. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". There is no member named pathname. Reading symbols from /boot/kernel/acpi.ko...Reading symbols from /boot/kern= el/acpi.ko.symbols...done. done. Loaded symbols for /boot/kernel/acpi.ko Reading symbols from /boot/kernel/fdescfs.ko...Reading symbols from /boot/k= ernel/fdescfs.ko.symbols...done. done. Loaded symbols for /boot/kernel/fdescfs.ko Reading symbols from /boot/kernel/pflog.ko...Reading symbols from /boot/ker= nel/pflog.ko.symbols...done. done. Loaded symbols for /boot/kernel/pflog.ko Reading symbols from /boot/kernel/pf.ko...Reading symbols from /boot/kernel= /pf.ko.symbols...done. done. Loaded symbols for /boot/kernel/pf.ko Unread portion of the kernel message buffer: <6>pid 49337 (megarc), uid 0: exited on signal 11 TPTE at 0xbfc20120 IS ZERO @ VA 08048000 panic: bad pte cpuid =3D 1 Uptime: 7h21m35s Physical memory: 2035 MB Dumping 275 MB: 260 244 228 212 196 180 164 148 132 116 100 84 68 52 36 20 4 #0 doadump () at pcpu.h:195 195 pcpu.h: No such file or directory. in pcpu.h (kgdb) bt #0 doadump () at pcpu.h:195 #1 0xc075b1a7 in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:4= 18 #2 0xc075b469 in panic (fmt=3DVariable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:572 #3 0xc0a509e6 in pmap_remove_pages (pmap=3D0xc6579a84) at /usr/src/sys/i38= 6/i386/pmap.c:3093 #4 0xc097e07c in vmspace_exit (td=3D0xc63dc000) at /usr/src/sys/vm/vm_map.= c:404 #5 0xc0739c3a in exit1 (td=3D0xc63dc000, rv=3D11) at /usr/src/sys/kern/ker= n_exit.c:294 #6 0xc075d633 in sigexit (td=3DVariable "td" is not available. ) at /usr/src/sys/kern/kern_sig.c:2877 #7 0xc075e8be in postsig (sig=3D11) at /usr/src/sys/kern/kern_sig.c:2749 #8 0xc078cd28 in ast (framep=3D0xe7d82d38) at /usr/src/sys/kern/subr_trap.= c:250 #9 0xc0a3c3fd in doreti_ast () at /usr/src/sys/i386/i386/exception.s:290 #10 0xe7d82d38 in ?? () #11 0x0000003b in ?? () #12 0x0000003b in ?? () #13 0x0000003b in ?? () #14 0xbfbfe834 in ?? () #15 0xbfbfe850 in ?? () #16 0xbfbfe2e8 in ?? () #17 0xe7d82d64 in ?? () #18 0x00000006 in ?? () #19 0x00000006 in ?? () #20 0x00000000 in ?? () #21 0x00000000 in ?? () #22 0x0000000c in ?? () #23 0x00000004 in ?? () #24 0x00000000 in ?? () #25 0x00000033 in ?? () #26 0x00010292 in ?? () #27 0xbfbfe280 in ?? () #28 0x0000003b in ?? () #29 0x00000000 in ?? () #30 0x00000004 in ?? () #31 0xbfbfedc8 in ?? () #32 0x282d6618 in ?? () #33 0x49700000 in ?? () #34 0xc0bf09c0 in tdq_cpu () #35 0xc63dc1e4 in ?? () #36 0xe7d8273c in ?? () #37 0xe7d82700 in ?? () #38 0x01944ce9 in ?? () #39 0xc07786bb in sched_switch (td=3DCannot access memory at address 0xbfbf= e2f0 ) at /usr/src/sys/kern/sched_ule.c:1898 Previous frame inner to this frame (corrupt stack?) (kgdb) server# kgdb /boot/kernel/kernel vmcore.0=08=1B[K1=0D [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so:= Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain condition= s. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". There is no member named pathname. Reading symbols from /boot/kernel/acpi.ko...Reading symbols from /boot/kern= el/acpi.ko.symbols...done. done. Loaded symbols for /boot/kernel/acpi.ko Reading symbols from /boot/kernel/fdescfs.ko...Reading symbols from /boot/k= ernel/fdescfs.ko.symbols...done. done. Loaded symbols for /boot/kernel/fdescfs.ko Reading symbols from /boot/kernel/pflog.ko...Reading symbols from /boot/ker= nel/pflog.ko.symbols...done. done. Loaded symbols for /boot/kernel/pflog.ko Reading symbols from /boot/kernel/pf.ko...Reading symbols from /boot/kernel= /pf.ko.symbols...done. done. Loaded symbols for /boot/kernel/pf.ko Unread portion of the kernel message buffer: <6>pid 8628 (megarc), uid 0: exited on signal 11 Fatal trap 12: page fault while in kernel mode cpuid =3D 1; apic id =3D 06 fault virtual address =3D 0xbfc00000 fault code =3D supervisor read, page not present instruction pointer =3D 0x20:0xc0a5096f stack pointer =3D 0x28:0xe7c74a74 frame pointer =3D 0x28:0xe7c74ab4 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, def32 1, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 8628 (megarc) trap number =3D 12 panic: page fault cpuid =3D 1 Uptime: 2h14m34s Physical memory: 2035 MB Dumping 253 MB: 238 222 206 190 174 158 142 126 110 94 78 62 46 30 14 #0 doadump () at pcpu.h:195 195 pcpu.h: No such file or directory. in pcpu.h (kgdb) bt #0 doadump () at pcpu.h:195 #1 0xc075b1a7 in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:4= 18 #2 0xc075b469 in panic (fmt=3DVariable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:572 #3 0xc0a54cdc in trap_fatal (frame=3D0xe7c74a34, eva=3D3217031168) at /usr= /src/sys/i386/i386/trap.c:899 #4 0xc0a54f60 in trap_pfault (frame=3D0xe7c74a34, usermode=3D0, eva=3D3217= 031168) at /usr/src/sys/i386/i386/trap.c:812 #5 0xc0a5590c in trap (frame=3D0xe7c74a34) at /usr/src/sys/i386/i386/trap.= c:490 #6 0xc0a3badb in calltrap () at /usr/src/sys/i386/i386/exception.s:139 #7 0xc0a5096f in pmap_remove_pages (pmap=3D0xc569db6c) at /usr/src/sys/i38= 6/i386/pmap.c:3086 #8 0xc097e07c in vmspace_exit (td=3D0xc62b6660) at /usr/src/sys/vm/vm_map.= c:404 #9 0xc0739c3a in exit1 (td=3D0xc62b6660, rv=3D11) at /usr/src/sys/kern/ker= n_exit.c:294 #10 0xc075d633 in sigexit (td=3DVariable "td" is not available. ) at /usr/src/sys/kern/kern_sig.c:2877 #11 0xc075e8be in postsig (sig=3D11) at /usr/src/sys/kern/kern_sig.c:2749 #12 0xc078cd28 in ast (framep=3D0xe7c74d38) at /usr/src/sys/kern/subr_trap.= c:250 #13 0xc0a3c3fd in doreti_ast () at /usr/src/sys/i386/i386/exception.s:290 #14 0xe7c74d38 in ?? () #15 0x0000003b in ?? () #16 0x0000003b in ?? () #17 0x0000003b in ?? () #18 0xbfbfee90 in ?? () #19 0x00000000 in ?? () #20 0xbfbfed58 in ?? () #21 0xe7c74d64 in ?? () #22 0x00000006 in ?? () #23 0x00000000 in ?? () #24 0x00000000 in ?? () #25 0x0809d094 in ?? () #26 0x0000000c in ?? () #27 0x00000004 in ?? () #28 0x0807b261 in ?? () #29 0x00000033 in ?? () #30 0x00010246 in ?? () #31 0xbfbfeb40 in ?? () #32 0x0000003b in ?? () #33 0x78617270 in ?? () #34 0x24007369 in ?? () #35 0x78502431 in ?? () #36 0x706d4959 in ?? () #37 0x17ed4000 in ?? () #38 0xc0bf09c0 in tdq_cpu () #39 0xc62b6844 in ?? () #40 0xe7c746a8 in ?? () #41 0xe7c7466c in ?? () #42 0x007b355a in ?? () #43 0xc07786bb in sched_switch (td=3DCannot access memory at address 0xbfbf= ed60 ) at /usr/src/sys/kern/sched_ule.c:1898 Previous frame inner to this frame (corrupt stack?) (kgdb) server# kgdb /boot/kernel/kernel vmcore.1=08=1B[K2=0D [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so:= Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain condition= s. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". There is no member named pathname. Reading symbols from /boot/kernel/acpi.ko...Reading symbols from /boot/kern= el/acpi.ko.symbols...done. done. Loaded symbols for /boot/kernel/acpi.ko Reading symbols from /boot/kernel/fdescfs.ko...Reading symbols from /boot/k= ernel/fdescfs.ko.symbols...done. done. Loaded symbols for /boot/kernel/fdescfs.ko Reading symbols from /boot/kernel/pflog.ko...Reading symbols from /boot/ker= nel/pflog.ko.symbols...done. done. Loaded symbols for /boot/kernel/pflog.ko Reading symbols from /boot/kernel/pf.ko...Reading symbols from /boot/kernel= /pf.ko.symbols...done. done. Loaded symbols for /boot/kernel/pf.ko Unread portion of the kernel message buffer: TPTE at 0xbfefeffc IS ZERO @ VA bfbff000 panic: bad pte cpuid =3D 1 Uptime: 7h43m42s Physical memory: 2035 MB Dumping 270 MB: 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15 #0 doadump () at pcpu.h:195 195 pcpu.h: No such file or directory. in pcpu.h (kgdb) bt #0 doadump () at pcpu.h:195 #1 0xc075b1a7 in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:4= 18 #2 0xc075b469 in panic (fmt=3DVariable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:572 #3 0xc0a509e6 in pmap_remove_pages (pmap=3D0xc62c3d3c) at /usr/src/sys/i38= 6/i386/pmap.c:3093 #4 0xc097e07c in vmspace_exit (td=3D0xc608e660) at /usr/src/sys/vm/vm_map.= c:404 #5 0xc0739c3a in exit1 (td=3D0xc608e660, rv=3D0) at /usr/src/sys/kern/kern= _exit.c:294 #6 0xc073af8d in sys_exit (td=3DCould not find the frame base for "sys_exi= t". ) at /usr/src/sys/kern/kern_exit.c:98 #7 0xc0a552b5 in syscall (frame=3D0xe7c65d38) at /usr/src/sys/i386/i386/tr= ap.c:1035 #8 0xc0a3bb40 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s= :196 #9 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) --=20 szymon roczniak=20 simon(a)dischaos.com=20