From owner-freebsd-current@FreeBSD.ORG Fri Aug 14 16:21:10 2009 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5A1F1106568C for ; Fri, 14 Aug 2009 16:21:10 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-fx0-f205.google.com (mail-fx0-f205.google.com [209.85.220.205]) by mx1.freebsd.org (Postfix) with ESMTP id A28B58FC43 for ; Fri, 14 Aug 2009 16:21:09 +0000 (UTC) Received: by fxm1 with SMTP id 1so1190107fxm.7 for ; Fri, 14 Aug 2009 09:21:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=Ln3lV8+Y/01Ox05bhsjwwguZfs/h563M+HPSh1wA5xA=; b=byEgT4aFbZ2jrOhF+xG4IOngMQEEQ8xUjvXFnVLQ0E/yakD06kZVQSLxf7uZrHbQ/O c81FxulWXkZ/6rSa1mKSV7Ig7k07dl03Qe+W+z7yaUAXEtQiCekIVvuUnr07CiBugsRL IThN8lOFl1/o8tY+7aX456NgpzSHXnc8Lm0lQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=NPg6CoRJrOeqbjuR6fCOUMxB7nHrOkl76Tk/Gg5Fz7D2gHa+IUSCGCI24v5G3/N0og nPnP8UNLStfW5Kirtb5whzeyDJ20aBR1WwkoAzGkmwqb1k6slYrEtIlRjIK70KMd3FYC lyaBvoZmwyb5T+ehyV0sLyKLevJnhTriQHXzI= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.223.117.1 with SMTP id o1mr598497faq.20.1250266868531; Fri, 14 Aug 2009 09:21:08 -0700 (PDT) In-Reply-To: References: Date: Fri, 14 Aug 2009 18:21:08 +0200 X-Google-Sender-Auth: f39ff812399a0f08 Message-ID: <3bbf2fe10908140921v4ba9ad6drc5225a4f533206b1@mail.gmail.com> From: Attilio Rao To: Alexey Tarasov Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: current@freebsd.org Subject: Re: Kernel panic on all versions of FreeBSD X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Aug 2009 16:21:10 -0000 2009/8/14 Alexey Tarasov : > Hello. > > I have more than 30 Supermicro servers with same config and on all of them I > get kernel panic. > > Here is the screenshots of kgdb output: > http://lexasoft.ru/metamphetamine/Panic/Picture%2073.png > http://lexasoft.ru/metamphetamine/Panic/Picture%2074.png > http://lexasoft.ru/metamphetamine/Panic/Picture%2075.png > http://lexasoft.ru/metamphetamine/Panic/Picture%2076.png > http://lexasoft.ru/metamphetamine/Panic/Picture%2077.png > http://lexasoft.ru/metamphetamine/Panic/Picture%2078.png > http://lexasoft.ru/metamphetamine/Panic/Picture%2079.png > http://lexasoft.ru/metamphetamine/Panic/Picture%2080.png > http://lexasoft.ru/metamphetamine/Panic/Picture%2080.png > http://lexasoft.ru/metamphetamine/Panic/Picture%2082.png > http://lexasoft.ru/metamphetamine/Panic/Picture%2083.png > > Here is dmesg output: > > Copyright (c) 1992-2009 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 7.2-STABLE #0: Wed Aug 12 01:06:04 MSD 2009 > root@st2.srv.getthebit.com:/usr/obj/usr/src/sys/STORAGE > WARNING: WITNESS option enabled, expect reduced performance. > Timecounter "i8254" frequency 1193182 Hz quality 0 > CPU: Intel(R) Pentium(R) 4 CPU 3.00GHz (2992.52-MHz K8-class CPU) > Origin = "GenuineIntel" Id = 0xf65 Stepping = 5 > Features=0xbfebfbff > Features2=0xe59d > AMD Features=0x20100800 > AMD Features2=0x1 > TSC: P-state invariant > Logical CPUs per core: 2 > usable memory = 2132484096 (2033 MB) > avail memory = 2056134656 (1960 MB) > ACPI APIC Table: > FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs > cpu0 (BSP): APIC ID: 0 > cpu1 (AP/HT): APIC ID: 1 > ioapic0 irqs 0-23 on motherboard > ioapic1 irqs 24-47 on motherboard > kbd1 at kbdmux0 > acpi0: on motherboard > acpi0: [ITHREAD] > acpi0: Power Button (fixed) > Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 > acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 > pcib0: port 0xcf8-0xcff on acpi0 > pci0: on pcib0 > pcib1: irq 16 at device 1.0 on pci0 > pci1: on pcib1 > pcib2: irq 17 at device 28.0 on pci0 > pci9: on pcib2 > pcib3: at device 0.0 on pci9 > pci10: on pcib3 > pcib4: irq 17 at device 28.4 on pci0 > pci13: on pcib4 > em0: port 0x4000-0x401f mem > 0xe8000000-0xe801ffff irq 16 at device 0.0 on pci13 > em0: Using MSI interrupt > em0: [FILTER] > em0: Ethernet address: 00:30:48:91:93:40 > pcib5: irq 16 at device 28.5 on pci0 > pci14: on pcib5 > em1: port 0x5000-0x501f mem > 0xe8200000-0xe821ffff irq 17 at device 0.0 on pci14 > em1: Using MSI interrupt > em1: [FILTER] > em1: Ethernet address: 00:30:48:91:93:41 > uhci0: port 0x3000-0x301f irq 23 at device > 29.0 on pci0 > uhci0: [GIANT-LOCKED] > uhci0: [ITHREAD] > usb0: on uhci0 > usb0: USB revision 1.0 > uhub0: on usb0 > uhub0: 2 ports with 2 removable, self powered > uhci1: port 0x3020-0x303f irq 19 at device > 29.1 on pci0 > uhci1: [GIANT-LOCKED] > uhci1: [ITHREAD] > usb1: on uhci1 > usb1: USB revision 1.0 > uhub1: on usb1 > uhub1: 2 ports with 2 removable, self powered > uhci2: port 0x3040-0x305f irq 18 at device > 29.2 on pci0 > uhci2: [GIANT-LOCKED] > uhci2: [ITHREAD] > usb2: on uhci2 > usb2: USB revision 1.0 > uhub2: on usb2 > uhub2: 2 ports with 2 removable, self powered > uhci3: port 0x3060-0x307f irq 16 at device > 29.3 on pci0 > uhci3: [GIANT-LOCKED] > uhci3: [ITHREAD] > usb3: on uhci3 > usb3: USB revision 1.0 > uhub3: on usb3 > uhub3: 2 ports with 2 removable, self powered > ehci0: mem 0xe8600000-0xe86003ff > irq 23 at device 29.7 on pci0 > ehci0: [GIANT-LOCKED] > ehci0: [ITHREAD] > usb4: EHCI version 1.0 > usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3 > usb4: on ehci0 > usb4: USB revision 2.0 > uhub4: on usb4 > uhub4: 8 ports with 8 removable, self powered > pcib6: at device 30.0 on pci0 > pci15: on pcib6 > vgapci0: port 0x6000-0x60ff mem > 0xe0000000-0xe7ffffff,0xe8300000-0xe830ffff irq 16 at device 0.0 on pci15 > isab0: at device 31.0 on pci0 > isa0: on isab0 > atapci0: port > 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x30a0-0x30af at device 31.1 on pci0 > ata0: on atapci0 > ata0: [ITHREAD] > atapci1: port > 0x30e8-0x30ef,0x30dc-0x30df,0x30e0-0x30e7,0x30d8-0x30db,0x30b0-0x30bf mem > 0xe8600400-0xe86007ff irq 19 at device 31.2 on pci0 > atapci1: [ITHREAD] > ata2: on atapci1 > ata2: [ITHREAD] > ata3: on atapci1 > ata3: [ITHREAD] > pci0: at device 31.3 (no driver attached) > acpi_button0: on acpi0 > atkbdc0: port 0x60,0x64 irq 1 on acpi0 > atkbd0: irq 1 on atkbdc0 > kbd0 at atkbd0 > atkbd0: [GIANT-LOCKED] > atkbd0: [ITHREAD] > sio0: configured irq 4 not in bitmap of probed irqs 0 > sio0: port may not be enabled > sio0: configured irq 4 not in bitmap of probed irqs 0 > sio0: port may not be enabled > sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on > acpi0 > sio0: type 16550A > sio0: [FILTER] > sio1: configured irq 3 not in bitmap of probed irqs 0 > sio1: port may not be enabled > sio1: configured irq 3 not in bitmap of probed irqs 0 > sio1: port may not be enabled > sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0 > sio1: type 16550A > sio1: [FILTER] > fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 > fdc0: does not respond > device_attach: fdc0 attach returned 6 > ppc0: port 0x378-0x37f,0x778-0x77f irq 7 drq 3 on acpi0 > ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode > ppc0: FIFO with 16/16/9 bytes threshold > ppbus0: on ppc0 > ppbus0: [ITHREAD] > plip0: on ppbus0 > plip0: WARNING: using obsoleted IFF_NEEDSGIANT flag > lpt0: on ppbus0 > lpt0: Interrupt-driven port > ppi0: on ppbus0 > ppc0: [GIANT-LOCKED] > ppc0: [ITHREAD] > cpu0: on acpi0 > est0: on cpu0 > est: CPU supports Enhanced Speedstep, but is not recognized. > est: cpu_vendor GenuineIntel, msr f2700000f27 > device_attach: est0 attach returned 6 > p4tcc0: on cpu0 > cpu1: on acpi0 > est1: on cpu1 > est: CPU supports Enhanced Speedstep, but is not recognized. > est: cpu_vendor GenuineIntel, msr f2700000f27 > device_attach: est1 attach returned 6 > p4tcc1: on cpu1 > fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 > fdc0: does not respond > device_attach: fdc0 attach returned 6 > orm0: at iomem > 0xc0000-0xcafff,0xcb000-0xcbfff,0xcc000-0xccfff on isa0 > sc0: at flags 0x100 on isa0 > sc0: VGA <16 virtual consoles, flags=0x300> > vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 > Timecounters tick every 1.000 msec > acd0: CDROM at ata0-slave UDMA33 > ad4: 381554MB at ata2-master SATA150 > ad5: 381554MB at ata2-slave SATA150 > ad6: 381554MB at ata3-master SATA150 > ad7: 476940MB at ata3-slave SATA150 > SMP: AP CPU #1 Launched! > WARNING: WITNESS option enabled, expect reduced performance. > Trying to mount root from ufs:/dev/ad4s1a > WARNING: / was not properly dismounted > WARNING: /data1 was not properly dismounted > WARNING: /data2 was not properly dismounted > WARNING: /data3 was not properly dismounted > WARNING: /data4 was not properly dismounted > WARNING: /tmp was not properly dismounted > WARNING: /usr was not properly dismounted > /usr: mount pending error: blocks 176 files 26 > WARNING: /var was not properly dismounted > /var: mount pending error: blocks 44 files 11 > em0: link state changed to UP > > I have vmcore in /var/crash folder. What debug information is needed to > debug this problem? As you got a dump, could you please list with kgdb the precise point within witness_lock() where the offending spinlock unlocking happens? Can you show the state of the offending spinlock? Attilio -- Peace can only be achieved by understanding - A. Einstein