From owner-freebsd-bugs@FreeBSD.ORG Sat Mar 8 01:20:01 2008 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3E71B1065673 for ; Sat, 8 Mar 2008 01:20:01 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 1AED28FC1E for ; Sat, 8 Mar 2008 01:20:01 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m281K0O8067593 for ; Sat, 8 Mar 2008 01:20:00 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m281K0Fw067592; Sat, 8 Mar 2008 01:20:00 GMT (envelope-from gnats) Resent-Date: Sat, 8 Mar 2008 01:20:00 GMT Resent-Message-Id: <200803080120.m281K0Fw067592@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Rory Arms Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BF5581065671 for ; Sat, 8 Mar 2008 01:17:51 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21]) by mx1.freebsd.org (Postfix) with ESMTP id AD7CB8FC1D for ; Sat, 8 Mar 2008 01:17:51 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.14.2/8.14.2) with ESMTP id m281EiN9077641 for ; Sat, 8 Mar 2008 01:14:44 GMT (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.14.2/8.14.1/Submit) id m281EiQc077640; Sat, 8 Mar 2008 01:14:44 GMT (envelope-from nobody) Message-Id: <200803080114.m281EiQc077640@www.freebsd.org> Date: Sat, 8 Mar 2008 01:14:44 GMT From: Rory Arms To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.1 Cc: Subject: kern/121485: panic with 7.0-RELEASE X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 08 Mar 2008 01:20:01 -0000 >Number: 121485 >Category: kern >Synopsis: panic with 7.0-RELEASE >Confidential: no >Severity: critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Sat Mar 08 01:20:00 UTC 2008 >Closed-Date: >Last-Modified: >Originator: Rory Arms >Release: 7.0-RELEASE >Organization: >Environment: FreeBSD Tserver.TrueStep.com 7.0-RELEASE FreeBSD 7.0-RELEASE #1: Wed Mar 5 17:52:52 EST 2008 root@Tserver.TrueStep.com:/mnt/obj/usr/src/sys/TSERVER-70 i386 >Description: 7.0-RELEASE panic'd. I had the system configured to do a minidump. So, I used kgdb to get a backtrace. This is a dual processor Pentium II 375 Mhz. It is on a Tyan Thunder100 motherboard. It has 1 GiB of memory and ACPI is disabled both in the BIOS and in FreeBSD, as with it on, FreeBSD considers it to be blacklisted. This system has used FreeBSD since at least 4.1-RELEASE. Previous to 7.0, it was running 6.3 and seemed to be stable, there were no crashes. I think the last time it crashed was an early 6 or a 5.x release. The issue back them seemed to relate to a bug in networking when running out of GIANT, as the only way I could keep the system stable is to boot with debug.mpsafenet="0" in loader.conf(5). As long as it wasn't allowed to run as MP safe, it was stable. With this 7.0-RELEASE I've turned off that setting, in hopes that the bug is now resolved. Here's the backtrace from kgdb(1). Keep in mind this is the first time I've had a crash after turning on minidump support, so I'm assuming the procedures to analyze the core file are the same with this setting. The system was in the process of recompiling ports with portupgrade(1) > sudo kgdb /boot/kernel/kernel /usr/local/crash/vmcore.0 Password: [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". Unread portion of the kernel message buffer: pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 17859 (ruby18) trap number = 12 panic: page fault cpuid = 0 Uptime: 1d4h38m24s Physical memory: 1015 MB Dumping 206 MB: 191 175 159 143 127 111 95 79 63 47 31 15 #0 doadump () at pcpu.h:195 195 pcpu.h: No such file or directory. in pcpu.h (kgdb) bt #0 doadump () at pcpu.h:195 #1 0xc059f866 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 #2 0xc059fb3e in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:563 #3 0xc080bf9c in trap_fatal (frame=0xe66e8968, eva=0) at /usr/src/sys/i386/i386/trap.c:899 #4 0xc080c20b in trap_pfault (frame=0xe66e8968, usermode=0, eva=0) at /usr/src/sys/i386/i386/trap.c:812 #5 0xc080cc02 in trap (frame=0xe66e8968) at /usr/src/sys/i386/i386/trap.c:490 #6 0xc07f384b in calltrap () at /usr/src/sys/i386/i386/exception.s:139 #7 0xc0807689 in pmap_clear_modify (m=0xc1fa5ca8) at /usr/src/sys/i386/i386/pmap.c:3309 #8 0xc07ae99c in vm_page_set_validclean (m=0xc1fa5ca8, base=0, size=4096) at /usr/src/sys/vm/vm_page.c:1806 #9 0xc0600a77 in vfs_page_set_valid (bp=Variable "bp" is not available. ) at /usr/src/sys/kern/vfs_bio.c:3391 #10 0xc06072c7 in bdwrite (bp=0xd7d81074) at /usr/src/sys/kern/vfs_bio.c:3507 #11 0xc07852c9 in ffs_write (ap=0xe66e8bbc) at /usr/src/sys/ufs/ffs/ffs_vnops.c:780 #12 0xc08222e4 in VOP_WRITE_APV (vop=0xc08b8920, a=0xe66e8bbc) at vnode_if.c:691 #13 0xc06294c1 in vn_write (fp=0xc86e3a20, uio=0xe66e8c60, active_cred=0xc46e5500, flags=1, td=0xc469c630) at vnode_if.h:373 #14 0xc05d3657 in dofilewrite (td=0xc469c630, fd=3, fp=0xc86e3a20, auio=0xe66e8c60, offset=4189184, flags=1) at file.h:254 ---Type to continue, or q to quit--- #15 0xc05d3825 in kern_pwritev (td=0xc469c630, fd=3, auio=0xe66e8c60, offset=4189184) at /usr/src/sys/kern/sys_generic.c:449 #16 0xc05d38ad in pwrite (td=0xc469c630, uap=0xe66e8cfc) at /usr/src/sys/kern/sys_generic.c:350 #17 0xc080c595 in syscall (frame=0xe66e8d38) at /usr/src/sys/i386/i386/trap.c:1035 #18 0xc07f38b0 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:196 #19 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) Loaded KLDs: > kldstat Id Refs Address Size Name 1 8 0xc0400000 5b8030 kernel 2 1 0xc09b9000 5844 if_tap.ko 3 1 0xc3fcc000 9000 if_bridge.ko 4 1 0xc3fd5000 6000 bridgestp.ko 5 2 0xc4092000 d000 ipfw.ko 6 1 0xc40be000 4000 ipdivert.ko 7 1 0xc4514000 2000 green_saver.ko the dmesg(1): > dmesg Copyright (c) 1992-2008 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.0-RELEASE #1: Wed Mar 5 17:52:52 EST 2008 root@Tserver.TrueStep.com:/mnt/obj/usr/src/sys/TSERVER-70 Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Pentium II/Pentium II Xeon/Celeron (375.04-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x652 Stepping = 2 Features=0x183fbff real memory = 1073741824 (1024 MB) avail memory = 1041379328 (993 MB) MPTable: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 ioapic0: Assuming intbase of 0 ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) cpu0 on motherboard cpu1 on motherboard pcib0: pcibus 0 on motherboard pci0: on pcib0 agp0: on hostb0 pcib1: at device 1.0 on pci0 pci1: on pcib1 vgapci0: port 0xc800-0xc8ff mem 0xfd000000-0xfdffffff,0xfe1ff000-0xfe1fffff at device 0.0 on pci1 isab0: at device 7.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 7.1 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] uhci0: port 0xef80-0xef9f irq 19 at device 7.2 on pci0 uhci0: [GIANT-LOCKED] uhci0: [ITHREAD] usb0: on uhci0 usb0: USB revision 1.0 uhub0: on usb0 uhub0: 2 ports with 2 removable, self powered piix0: port 0x440-0x44f at device 7.3 on pci0 Timecounter "PIIX" frequency 3579545 Hz quality 0 pcib2: at device 16.0 on pci0 pci2: on pcib2 ath0: mem 0xfe6f0000-0xfe6fffff irq 16 at device 4.0 on pci2 ath0: [ITHREAD] ath0: using obsoleted if_watchdog interface ath0: Ethernet address: 00:0f:3d:ad:b9:f2 ath0: mac 5.9 phy 4.3 radio 4.6 fxp0: port 0xdf00-0xdf3f mem 0xfe6ef000-0xfe6effff,0xfe500000-0xfe5fffff irq 17 at device 5.0 on pci2 miibus0: on fxp0 inphy0: PHY 1 on miibus0 inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto fxp0: Ethernet address: 00:90:27:ee:02:97 fxp0: [ITHREAD] fxp1: port 0xef40-0xef5f mem 0xffaff000-0xffafffff,0xfea00000-0xfeafffff irq 19 at device 17.0 on pci0 miibus1: on fxp1 inphy1: PHY 1 on miibus1 inphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto fxp1: Ethernet address: 00:e0:81:10:22:27 fxp1: [ITHREAD] ahc0: port 0xe400-0xe4ff mem 0xfebfe000-0xfebfefff irq 16 at device 18.0 on pci0 ahc0: [ITHREAD] aic7895C: Ultra Wide Channel A, SCSI Id=7, 32/253 SCBs ahc1: port 0xe800-0xe8ff mem 0xfebff000-0xfebfffff irq 16 at device 18.1 on pci0 ahc1: [ITHREAD] aic7895C: Ultra Wide Channel B, SCSI Id=7, 32/253 SCBs ahc2: port 0xe000-0xe0ff mem 0xfebfd000-0xfebfdfff irq 16 at device 19.0 on pci0 ahc2: [ITHREAD] aic7850: Single Channel A, SCSI Id=7, 3/253 SCBs pmtimer0 on isa0 orm0: at iomem 0xc0000-0xcbfff,0xcc000-0xd07ff pnpid ORM0000 on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] fdc0: at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 fdc0: [FILTER] fd0: <1440-KB 3.5" drive> on fdc0 drive 0 ppc0: at port 0x378-0x37f irq 7 on isa0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/8 bytes threshold ppbus0: on ppc0 ppbus0: [ITHREAD] plip0: on ppbus0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 ppc0: [GIANT-LOCKED] ppc0: [ITHREAD] sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 sio0: type 16550A, console sio0: [FILTER] sio1 at port 0x2f8-0x2ff irq 3 on isa0 sio1: type 16550A sio1: [FILTER] vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 unknown: can't assign resources (memory) unknown: can't assign resources (port) unknown: can't assign resources (port) unknown: can't assign resources (port) unknown: can't assign resources (port) unknown: can't assign resources (port) unknown: can't assign resources (port) ugen0: on uhub0 Timecounters tick eahc2: Someone reset channel A very 1.000 msec Waiting 5 seconds for SCSI devices to settle ad0: 156334MB at ata0-master UDMA33 GEOM_LABEL: Label for provider ad0s1a is ufs/Network. da0 at ahc0 bus 0 target 2 lun 0 da0: Fixed Direct Access SCSI-2 device da0: 10.000MB/s transfers (10.000MHz, offset 15) da0: 8347MB (17096357 512 byte sectors: 255H 63S/T 1064C) da1 at ahc0 bus 0 target 6 lun 0 da1: Fixed Direct Access SCSI-2 device da1: 20.000MB/s transfers (20.000MHz, offset 15) da1: Command Queueing Enabled da1: 8191MB (16777215 512 byte sectors: 255H 63S/T 1044C) SMP: AP CPU #1 Launched! Trying to mount root from ufs:/dev/da0s1a WARNING: / was not properly dismounted WARNING: /usr was not properly dismounted /usr: mount pending error: blocks 200 files 4 WARNING: /var was not properly dismounted /var: mount pending error: blocks 968 files 1 GWARNING: /Network was not properly dismounted EOM_LABEL: Label ufs/Network removed. bridge0: Ethernet address: b2:4f:63:a9:da:42 ath0: ath_chan_set: unable to reset channel 6 (2437 Mhz, flags 0x490 hal flags 0x150) ipfw2 (+ipv6) initialized, divert loadable, rule-based forwarding disabled, default to deny, logging disabled pid 1444 (mu-conference), uid 1012: exited on signal 11 Lastly the KERNCONF that I used to compile TSERVER-70: cpu I686_CPU ident TSERVER-70 makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols options SCHED_4BSD # 4BSD scheduler options PREEMPTION # Enable kernel thread preemption options INET # InterNETworking options INET6 # IPv6 communications protocols options SCTP # Stream Control Transmission Protocol options FFS # Berkeley Fast Filesystem options SOFTUPDATES # Enable FFS soft updates support options UFS_ACL # Support for access control lists options UFS_DIRHASH # Improve performance on big directories options UFS_GJOURNAL # Enable gjournal-based UFS journaling options NFSCLIENT # Network Filesystem Client options NFSSERVER # Network Filesystem Server options MSDOSFS # MSDOS Filesystem options CD9660 # ISO 9660 Filesystem options PROCFS # Process filesystem (requires PSEUDOFS) options PSEUDOFS # Pseudo-filesystem framework options GEOM_PART_GPT # GUID Partition Tables. options GEOM_LABEL # Provides labelization options COMPAT_43TTY # BSD 4.3 TTY compat [KEEP THIS!] options COMPAT_FREEBSD6 # Compatible with FreeBSD6 options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI options KTRACE # ktrace(1) support options SYSVSHM # SYSV-style shared memory options SYSVMSG # SYSV-style message queues options SYSVSEM # SYSV-style semaphores options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions options KBD_INSTALL_CDEV # install a CDEV entry in /dev options ADAPTIVE_GIANT # Giant mutex is adaptive. options STOP_NMI # Stop CPUS using NMI instead of IPI options AUDIT # Security event auditing options SMP # Symmetric MultiProcessor Kernel device apic # I/O APIC device cpufreq device pci device fdc device ata device atadisk # ATA disk drives device ataraid # ATA RAID drives device atapicd # ATAPI CDROM drives options ATA_STATIC_ID # Static device numbering device ahc # AHA2940 and onboard AIC7xxx devices options AHC_REG_PRETTY_PRINT # Print register bitfields in debug # output. Adds ~128k to driver. # output. Adds ~215k to driver. device scbus # SCSI bus (required for SCSI) device ch # SCSI media changers device da # Direct Access (disks) device sa # Sequential Access (tape etc) device cd # CD device pass # Passthrough device (direct SCSI access) device ses # SCSI Environmental Services (and SAF-TE) device atkbdc # AT keyboard controller device atkbd # AT keyboard device psm # PS/2 mouse device kbdmux # keyboard multiplexer device vga # VGA video card driver device splash # Splash screen and screen saver support device sc device agp # support several AGP chipsets device pmtimer device cbb # cardbus (yenta) bridge device pccard # PC Card (16-bit) bus device cardbus # CardBus (32-bit) bus device sio # 8250, 16[45]50 based serial ports device uart # Generic UART driver device ppc device ppbus # Parallel port bus (required) device lpt # Printer device plip # TCP/IP over parallel device ppi # Parallel port interface device device miibus # MII bus support device fxp # Intel EtherExpress PRO/100B (82557, 82558) device wlan # 802.11 support device wlan_wep # 802.11 WEP support device wlan_ccmp # 802.11 CCMP support device wlan_tkip # 802.11 TKIP support device wlan_amrr # AMRR transmit rate control algorithm device wlan_scan_ap # 802.11 AP mode scanning device wlan_scan_sta # 802.11 STA mode scanning device ath # Atheros pci/cardbus NIC's device ath_hal # Atheros HAL (Hardware Access Layer) device ath_rate_sample # SampleRate tx rate control for ath device loop # Network loopback device random # Entropy device device ether # Ethernet support device sl # Kernel SLIP device ppp # Kernel PPP device tun # Packet tunnel. device pty # Pseudo-ttys (telnet etc) device md # Memory "disks" device gif # IPv6 and IPv4 tunneling device faith # IPv6-to-IPv4 relaying (translation) device firmware # firmware assist module device bpf # Berkeley packet filter device uhci # UHCI PCI->USB interface device ohci # OHCI PCI->USB interface device usb # USB Bus (required) device ugen # Generic device uhid # "Human Interface Devices" device ukbd # Keyboard device ulpt # Printer device umass # Disks/Mass storage - Requires scbus and da device ums # Mouse device ural # Ralink Technology RT2500USB wireless NICs device rum # Ralink Technology RT2501USB wireless NICs >How-To-Repeat: None, this has only happened once so far. The system has been running 7.0-RELEASE for about 2 days though. >Fix: >Release-Note: >Audit-Trail: >Unformatted: