From owner-freebsd-questions@FreeBSD.ORG Tue Sep 18 11:04:02 2007 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1AE1616A417 for ; Tue, 18 Sep 2007 11:04:01 +0000 (UTC) (envelope-from gert@lynge.org) Received: from smtp.emnet.dk (smtp.emnet.dk [217.198.208.67]) by mx1.freebsd.org (Postfix) with ESMTP id 440DF13C442 for ; Tue, 18 Sep 2007 11:04:01 +0000 (UTC) (envelope-from gert@lynge.org) Received: from bsd.lynge.org (unknown [217.198.221.101]) by smtp.emnet.dk (Postfix) with ESMTP id 8902949F1B2; Tue, 18 Sep 2007 13:03:59 +0200 (CEST) Received: from STATIONAER (0x535eb36a.slnxx4.adsl-dhcp.tele.dk [83.94.179.106]) (authenticated bits=0) by bsd.lynge.org (8.14.1/8.13.8) with ESMTP id l8IB3wA8003013; Tue, 18 Sep 2007 13:03:58 +0200 (CEST) (envelope-from gert@lynge.org) From: "Gert Lynge" To: , Date: Tue, 18 Sep 2007 13:04:08 +0200 Message-ID: <029d01c7f9e3$a5fd6cd0$0601a8c0@STATIONAER> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.6822 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.3138 Importance: Normal Thread-Index: Acf545qQ5a+CIJMRRd2OIe5NqO0/DQ== Cc: Subject: Rare / random hangs with FreeBSD 6.2-RELEASE-p4 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Sep 2007 11:04:02 -0000 Hi Running a FreeBSD 6.2 with mainly MySQL, Apache and PHP (kept up to date with manual freebsd-update / portupgrade). The hardware is SuperMicro SuperServer 5015MT+ with 2GB of ECC RAM and a Intel Core 2 Quad 6700. Running a BIOS assisted soft mirror with SATA disks... After aprox 2 weeks the server hangs. There is no dumps at /var/crash as = the server seems to lock up while dumping. Details below - unfortunately I = don't always get a picture of the console as I'm not always the rebooter. There is no hang-related notes i FreeBSD logs, and nothing at all in = IPMI and BIOS logs. Earlier I had the server running with HTTP Accept Filters on, but then = it reboots 4-5 times a day! I've also been running xcache (php accelrator), = but that was disabled during the latest hang. Do You have any suggestions? Does this seems to be hardware or software related? Anything I could try to figure out what is going on? Unfortunately the server is in production. Thank You for any help. Regards Gert Lynge **** Hang 1 panic: vm_fault: fault on nofault entry, addr: e92cf000 cpuid =3D 3 Uptime: 10d0h24m25s Dumping 2046 MB (2 chunks) chunk 0: 1MB (150 pages) ... Ok chunk 1: 2046MB (523744 pages)_ --- **** Hang 2: Fatal trap 12: page fault while in kernel mode Cupid =3D 2; apic id =3D 2 Fault virtual address =3D 0x5a Fault code =3D supervisor read, page not present Instruction pointer =3D 0x20:0xc07e40a3 Stack pointer =3D 0x28:0xe6a35b74 Frame pointer =3D 0x28:0xe6a35c40 Code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, def32 1, gran 1 Processor eflags =3D interrupt enabled, resume, IOPL =3D 0 Current process =3D 44 (pagezero) Trap number =3D 12 Panic: page fault Cupid =3D 2 Uptime 17d22h47m51s Dumping 2048 MB (2 chunks) Chunk 0: 1MB (150 pages)ipfw: xxxxxxxx ipfw: xxxxxxxxxx --- **** uname -a FreeBSD x.x.x 6.2-RELEASE-p4 FreeBSD 6.2-RELEASE-p4 #0: Thu Apr 26 = 17:55:55 UTC 2007 root@i386-builder.daemonology.net:/usr/obj/usr/src/sys/SMP i386 --- **** cat /etc/rc.conf defaultrouter=3D"x.x.x.x" font8x14=3D"cp865-8x14" font8x16=3D"cp865-8x16" font8x8=3D"cp865-8x8" hostname=3D"x.x.x" ifconfig_em1=3D"inet x.x.x.x netmask x.x.x.x" saver=3D"daemon" usbd_enable=3D"YES" keymap=3D"danish.cp865" keyrate=3D"fast" sshd_enable=3D"YES" firewall_enable=3D"YES" firewall_type=3D"x" firewall_logging=3D"YES" ntpd_enable=3D"YES" ntpd_sync_on_start=3D"YES" mysql_enable=3D"YES" apache22_enable=3D"YES" #apache22_http_accept_enable=3D"YES" inetd_enable=3D"YES" clear_tmp_enable=3D"YES" #log_in_vain=3D"1" sendmail_enable=3D"YES" rsyncd_enable=3D"YES" syslogd_flags=3D"-a x.x.x.x/x:*" clamav_freshclam_enable=3D"YES" local_startup=3D"/usr/local/etc/rc.d" dumpdev=3D"/dev/ar0s1b" --- **** cat /var/run/dmesg.boot Copyright (c) 1992-2007 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 6.2-RELEASE-p4 #0: Thu Apr 26 17:55:55 UTC 2007 root@i386-builder.daemonology.net:/usr/obj/usr/src/sys/SMP Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Core(TM)2 Quad CPU Q6700 @ 2.66GHz (2660.01-MHz = 686-class CPU) Origin =3D "GenuineIntel" Id =3D 0x6fb Stepping =3D 11 =20 Features=3D0xbfebfbff = Features2=3D0xe3bd,CX16,,> AMD Features=3D0x20000000 AMD Features2=3D0x1 Cores per package: 4 real memory =3D 2146304000 (2046 MB) avail memory =3D 2095165440 (1998 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard kbd1 at kbdmux0 ath_hal: 0.9.17.2 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, = RF5413) acpi0: on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: irq 16 at device 1.0 on pci0 pci1: on pcib1 pcib2: irq 17 at device 28.0 on pci0 pci9: on pcib2 pcib3: at device 0.0 on pci9 pci10: on pcib3 pci9: at device 0.1 (no driver attached) pcib4: irq 17 at device 28.4 on pci0 pci13: on pcib4 em0: port 0x4000-0x401f mem 0xe0200000-0xe021ffff irq 16 at device 0.0 on pci13 em0: Ethernet address: 00:30:48:8d:1f:5e pcib5: irq 16 at device 28.5 on pci0 pci14: on pcib5 em1: port 0x5000-0x501f mem 0xe0300000-0xe031ffff irq 17 at device 0.0 on pci14 em1: Ethernet address: 00:30:48:8d:1f:5f uhci0: port 0x3000-0x301f irq 23 at = device 29.0 on pci0 uhci0: [GIANT-LOCKED] usb0: on uhci0 usb0: USB revision 1.0 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered uhci1: port 0x3020-0x303f irq 19 at = device 29.1 on pci0 uhci1: [GIANT-LOCKED] usb1: on uhci1 usb1: USB revision 1.0 uhub1: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0x3040-0x305f irq 18 at = device 29.2 on pci0 uhci2: [GIANT-LOCKED] usb2: on uhci2 usb2: USB revision 1.0 uhub2: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub2: 2 ports with 2 removable, self powered uhci3: port 0x3060-0x307f irq 16 at = device 29.3 on pci0 uhci3: [GIANT-LOCKED] usb3: on uhci3 usb3: USB revision 1.0 uhub3: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub3: 2 ports with 2 removable, self powered ehci0: mem = 0xe0000000-0xe00003ff irq 23 at device 29.7 on pci0 ehci0: [GIANT-LOCKED] usb4: EHCI version 1.0 usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3 usb4: on ehci0 usb4: USB revision 2.0 uhub4: Intel EHCI root hub, class 9/0, rev 2.00/1.00, addr 1 uhub4: 8 ports with 8 removable, self powered pcib6: at device 30.0 on pci0 pci15: on pcib6 pci15: at device 0.0 (no driver attached) isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x30a0-0x30af at device 31.1 on pci0 ata0: on atapci0 ata1: on atapci0 atapci1: port 0x30e8-0x30ef,0x30dc-0x30df,0x30e0-0x30e7,0x30d8-0x30db,0x30b0-0x30bf = mem 0xe0000400-0xe00007ff irq 19 at device 31.2 on pci0 atapci1: AHCI Version 01.10 controller with 4 ports detected ata2: on atapci1 ata3: on atapci1 ata4: on atapci1 ata5: on atapci1 pci0: at device 31.3 (no driver attached) acpi_button0: on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: model IntelliMouse Explorer, device ID 4 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0 sio1: type 16550A fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on = acpi0 fdc0: [FAST] ppc0: port 0x378-0x37f,0x778-0x77f irq 7 drq = 3 on acpi0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/9 bytes threshold ppbus0: on ppc0 plip0: on ppbus0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 pmtimer0 on isa0 ipmi0: on isa0 ipmi0: KCS mode found at io 0xca8 alignment 0x4 on isa orm0: at iomem 0xc0000-0xcafff,0xcb000-0xcf7ff,0xcf800-0xd07ff,0xd0800-0xd17ff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=3D0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on = isa0 Timecounters tick every 1.000 msec acd0: CDROM at ata0-master UDMA33 ad4: 76319MB at ata2-master SATA300 ad6: 76319MB at ata3-master SATA300 ipmi0: IPMI device rev. 0, firmware rev. 2.2, version 2.0 ipmi0: Number of channels 4 ipmi0: Attached watchdog ar0: 76316MB status: READY ar0: disk0 READY (master) using ad4 at ata2-master ar0: disk1 READY (mirror) using ad6 at ata3-master SMP: AP CPU #1 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #3 Launched! Trying to mount root from ufs:/dev/ar0s1a WARNING: / was not properly dismounted WARNING: /tmp was not properly dismounted WARNING: /usr was not properly dismounted WARNING: /var was not properly dismounted ipfw2 (+ipv6) initialized, divert loadable, rule-based forwarding = disabled, default to deny, logging disabled