From owner-freebsd-bugs@FreeBSD.ORG Fri Nov 6 17:30:02 2009 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 336511065676 for ; Fri, 6 Nov 2009 17:30:02 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id E97288FC14 for ; Fri, 6 Nov 2009 17:30:01 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.3/8.14.3) with ESMTP id nA6HU1Fb070230 for ; Fri, 6 Nov 2009 17:30:01 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.3/8.14.3/Submit) id nA6HU1Ts070227; Fri, 6 Nov 2009 17:30:01 GMT (envelope-from gnats) Resent-Date: Fri, 6 Nov 2009 17:30:01 GMT Resent-Message-Id: <200911061730.nA6HU1Ts070227@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Kai Gallasch Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0FF8D106568D for ; Fri, 6 Nov 2009 17:28:41 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (www.freebsd.org [IPv6:2001:4f8:fff6::21]) by mx1.freebsd.org (Postfix) with ESMTP id F17048FC0C for ; Fri, 6 Nov 2009 17:28:40 +0000 (UTC) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.14.3/8.14.3) with ESMTP id nA6HSeZE044891 for ; Fri, 6 Nov 2009 17:28:40 GMT (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.14.3/8.14.3/Submit) id nA6HSeV4044890; Fri, 6 Nov 2009 17:28:40 GMT (envelope-from nobody) Message-Id: <200911061728.nA6HSeV4044890@www.freebsd.org> Date: Fri, 6 Nov 2009 17:28:40 GMT From: Kai Gallasch To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.1 Cc: Subject: kern/140338: FreeBSD 8.0 RC2 with vm.pmap.pg_ps_enabled=1 kernel panic with makeworld X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Nov 2009 17:30:02 -0000 >Number: 140338 >Category: kern >Synopsis: FreeBSD 8.0 RC2 with vm.pmap.pg_ps_enabled=1 kernel panic with makeworld >Confidential: no >Severity: serious >Priority: high >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Fri Nov 06 17:30:01 UTC 2009 >Closed-Date: >Last-Modified: >Originator: Kai Gallasch >Release: 8.0 RC2 amd64 >Organization: >Environment: Copyright (c) 1992-2009 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.0-RC2 #0: Tue Nov 3 20:24:06 CET 2009 root@sonnenkraft.free.de:/usr/obj/usr/src/sys/GENERIC amd64 WARNING: WITNESS option enabled, expect reduced performance. Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Quad-Core AMD Opteron(tm) Processor 2352 (2100.09-MHz K8-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Stepping = 3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee400800 AMD Features2=0x7ff TSC: P-state invariant real memory = 21474836480 (20480 MB) avail memory = 20701110272 (19742 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 ioapic0 irqs 0-15 on motherboard ioapic1 irqs 16-31 on motherboard ioapic2 irqs 32-47 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <32-bit timer at 3.579545MHz> port 0x920-0x923 on acpi0 acpi_hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 900 pcib0: on acpi0 pci0: on pcib0 vgapci0: port 0x1000-0x10ff mem 0xe8000000-0xefffffff,0xf7ff0000-0xf7ffffff irq 44 at device 3.0 on pci0 pci0: at device 4.0 (no driver attached) pci0: at device 4.2 (no driver attached) uhci0: port 0x1800-0x181f irq 45 at device 4.4 on pci0 uhci0: [ITHREAD] usbus0: on uhci0 pci0: at device 4.6 (no driver attached) pcib1: at device 5.0 on pci0 pci1: on pcib1 pcib2: at device 13.0 on pci1 pci2: on pcib2 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x500-0x50f at device 6.1 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] isab0: at device 6.2 on pci0 isa0: on isab0 ohci0: port 0x1c00-0x1cff mem 0xf7ee0000-0xf7ee0fff irq 5 at device 7.0 on pci0 ohci0: [ITHREAD] usbus1: on ohci0 ohci1: port 0x3000-0x30ff mem 0xf7ed0000-0xf7ed0fff irq 5 at device 7.1 on pci0 ohci1: [ITHREAD] usbus2: on ohci1 ehci0: port 0x3400-0x34ff mem 0xf7ec0000-0xf7ec0fff irq 5 at device 7.2 on pci0 ehci0: [ITHREAD] usbus3: EHCI version 1.0 usbus3: on ehci0 pcib3: irq 42 at device 15.0 on pci0 pci5: on pcib3 pcib4: irq 38 at device 16.0 on pci0 pci8: on pcib4 pcib5: irq 39 at device 17.0 on pci0 pci14: on pcib5 pcib6: irq 40 at device 18.0 on pci0 pci11: on pcib6 pcib7: irq 41 at device 19.0 on pci0 pci3: on pcib7 pcib8: at device 0.0 on pci3 pci4: on pcib8 bce0: mem 0xf8000000-0xf9ffffff irq 41 at device 0.0 on pci4 miibus0: on bce0 brgphy0: PHY 1 on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto bce0: Ethernet address: 00:1b:78:38:dd:02 bce0: [ITHREAD] bce0: ASIC (0x57081020); Rev (B2); Bus (PCI-X, 64-bit, 133MHz); B/C (1.9.6); Flags (MSI|MFW); MFW () pcib9: on acpi0 pci64: on pcib9 pcib10: irq 36 at device 15.0 on pci64 pci67: on pcib10 pcib11: irq 32 at device 16.0 on pci64 pci70: on pcib11 ciss0: port 0x4000-0x40ff mem 0xfdf00000-0xfdffffff,0xfdef0000-0xfdef0fff irq 32 at device 0.0 on pci70 ciss0: PERFORMANT Transport ciss0: [ITHREAD] pcib12: irq 33 at device 17.0 on pci64 pci73: on pcib12 pcib13: irq 34 at device 18.0 on pci64 pci65: on pcib13 pcib14: at device 0.0 on pci65 pci66: on pcib14 bce1: mem 0xfa000000-0xfbffffff irq 34 at device 0.0 on pci66 miibus1: on bce1 brgphy1: PHY 1 on miibus1 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto bce1: Ethernet address: 00:1b:78:38:dd:00 bce1: [ITHREAD] bce1: ASIC (0x57081020); Rev (B2); Bus (PCI-X, 64-bit, 133MHz); B/C (1.9.6); Flags (MSI|MFW); MFW () pcib15: irq 35 at device 19.0 on pci64 pci74: on pcib15 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model IntelliMouse, device ID 3 atrtc0: port 0x70-0x71 on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: [FILTER] uart0: console (9600,n,8,1) cpu0: on acpi0 hwpstate0: on cpu0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 orm0: at iomem 0xc0000-0xcafff,0xcb000-0xcefff,0xe5000-0xe6fff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 ppc0: cannot reserve I/O port range uart1: at port 0x2f8-0x2ff irq 3 on isa0 uart1: [FILTER] Timecounters tick every 1.000 msec usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 480Mbps High Speed USB v2.0 acd0: CDRW at ata0-master UDMA33 ugen0.1: <(0x103c)> at usbus0 uhub0: <(0x103c) UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0 ugen1.1: <(0x1166)> at usbus1 uhub1: <(0x1166) OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1 ugen2.1: <(0x1166)> at usbus2 uhub2: <(0x1166) OHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2 ugen3.1: <(0x1166)> at usbus3 uhub3: <(0x1166) EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus3 uhub1: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub0: 2 ports with 2 removable, self powered ugen0.2: at usbus0 ukbd0: on usbus0 kbd2 at ukbd0 ums0: on usbus0 ums0: 3 buttons and [XY] coordinates ID=0 uhub3: 4 ports with 4 removable, self powered ugen0.3: at usbus0 uhub4: on usbus0 ugen3.2: at usbus3 uhub5: on usbus3 uhub5: 4 ports with 4 removable, self powered uhub4: 7 ports with 7 removable, self powered da0 at ciss0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-5 device da0: 135.168MB/s transfers da0: Command Queueing enabled da0: 36863MB (75496320 512 byte sectors: 255H 32S/T 9252C) da1 at ciss0 bus 0 target 1 lun 0 da1: Fixed Direct Access SCSI-5 device da1: 135.168MB/s transfers da1: Command Queueing enabled da1: 243098MB (497866080 512 byte sectors: 255H 32S/T 61013C) da2 at ciss0 bus 0 target 2 lun 0 da2: Fixed Direct Access SCSI-5 device da2: 135.168MB/s transfers da2: Command Queueing enabled da2: 139979MB (286677120 512 byte sectors: 255H 32S/T 35132C) da3 at ciss0 bus 0 target 3 lun 0 da3: Fixed Direct Access SCSI-5 device da3: 135.168MB/s transfers da3: Command Queueing enabled da3: 139979MB (286677120 512 byte sectors: 255H 32S/T 35132C) da4 at ciss0 bus 0 target 4 lun 0 da4: Fixed Direct Access SCSI-5 device da4: 135.168MB/s transfers da4: Command Queueing enabled da4: 139979MB (286677120 512 byte sectors: 255H 32S/T 35132C) da5 at ciss0 bus 0 target 5 lun 0 da5: Fixed Direct Access SCSI-5 device da5: 135.168MB/s transfers da5: Command Queueing enabled da5: 139979MB (286677120 512 byte sectors: 255H 32S/T 35132C) SMP: AP CPU #1 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #2 Launched! WARNING: WITNESS option enabled, expect reduced performance.GEOM: da0: partition 3 does not start on a track boundary. GEOM: da0: partition 3 does not end on a track boundary. GEOM: da0: partition 2 does not start on a track boundary. GEOM: da0: partition 2 does not end on a track boundary. GEOM: da0: partition 1 does not start on a track boundary. GEOM: da0: partition 1 does not end on a track boundary. GEOM: da0s1: geometry does not match label (255h,63s != 255h,32s). GEOM: da0s2: geometry does not match label (255h,63s != 255h,32s). GEOM: da0s3: geometry does not match label (255h,63s != 255h,32s). Trying to mount root from ufs:/dev/da0s1a ZFS filesystem version 13 ZFS storage pool version 13 bce0: link state changed to UP >Description: I installed 8.0RC2-amd64 on an 8-core opteron server a few days ago when 8.0 RC2 came out. When I tried to do a make buildworld or make buildkernel the server rebooted without any message left in the logs. The same happened when building bigger ports (for example ruby18 or perl58) After this I installed 7.2-STABLE on this same server and did a "make buildworld" and "make buildkernel" which completed without any problem. Then I installed 8.0-BETA4 (crashes also when doing makeworld) Finally I reinstalled 8.0RC2-amd64 on the server again and build a 8.0RC2 debug kernel on another amd server for this crashing server. I also: - ran several passes with diagnostic software from the server manufacturer - reset BIOS settings to default - upgraded BIOS to newest release - booted server from 2 year old backup BIOS - took out the only pair of RAM modules that was different from the rest of the modules - ran memtest86 on the server (no problems found) The server kept on crashing under load, when running buildworld. Although dumpdev + dumpdir were correctly defined, the server just rebooted without writing a crashdump! - Running a makeworld in about 80% leads to a server crash without the server writing a crashdump to dumpdir. The server just reboots.. - In about 20% of the cases makeworld gets stuck in a not terminating process that eats up 100% cpu. This process cannot be killed. When restarting makeworld the server then reboots again - It makes no difference doing makeworld -j1 or -j8, result is the same Finally, I followed a hint I got on the freebsd-current list and set vm.pmap.pg_ps_enabled=0 in /boot/loader.conf an rebooted. The problem was gone! After successful buildworld and buildkernel I rebooted the server again with commented out vm.pmap.pg_ps_enabled=0 and the problem was there again. And then I set vm.pmap.pg_ps_enabled=0 again in loader.conf, rebooted + make buildworld .. no problem. Seems to be deterministic. With vm.pmap.pg_ps_enabled=1 the server crashes without being able to write crashdumps to dumpdev. (at least on this specific HP Proliant DL385G2 server with 20G RAM) >How-To-Repeat: Install FreeBSD 8.0 RC2 amd64 + Sources, do a makeworld. >Fix: Workaround: Setting vm.pmap.pg_ps_enabled=0 in loader.conf and reboot. >Release-Note: >Audit-Trail: >Unformatted: