From owner-freebsd-current@FreeBSD.ORG Fri Dec 14 18:57:36 2007 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6EEA016A417 for ; Fri, 14 Dec 2007 18:57:36 +0000 (UTC) (envelope-from mikej@paymentallianceintl.com) Received: from mx2.confluenttech.com (mx2.confluentasp.com [216.26.153.14]) by mx1.freebsd.org (Postfix) with ESMTP id 8AF9713C465 for ; Fri, 14 Dec 2007 18:57:30 +0000 (UTC) (envelope-from mikej@paymentallianceintl.com) Received: from calvin.pai.local ([10.0.6.33]) by mx2.confluenttech.com (8.14.1/8.12.9) with ESMTP id lBEIMweM055607 for ; Fri, 14 Dec 2007 13:23:14 -0500 (EST) (envelope-from mikej@paymentallianceintl.com) X-MimeOLE: Produced By Microsoft MimeOLE V6.00.3790.4073 Content-class: urn:content-classes:message MIME-Version: 1.0 Importance: normal Priority: normal Date: Fri, 14 Dec 2007 13:13:54 -0500 Message-ID: In-Reply-To: X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: 7.0-BETA4 - witness_warn on boot Thread-Index: Acg+UgGpCL5rSjc7SyCq8MAtvXhE5QAA/vFgAAnDzTA= References: From: "Michael Jung" To: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: 7.0-BETA4 - witness_warn on boot X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Dec 2007 18:57:36 -0000 I was running 7.0-BETA4 and had problems with SEG_FAULT 12 in g_up so I rebuilt a stock GENERIC kernel with these additional options: options KDB option DDB options INVARIANTS options INVARIANT_SUPPORT options WITNESS options DEBUG_LOCKS options DEBUG_VFS_LOCKS options DIAGNOSTIC Here is the build date: FreeBSD charon.confluentasp.com 7.0-BETA4 FreeBSD 7.0-BETA4 #6: Tue Dec 11 09:05:50 EST 2007 mikej@charon.confluentasp.com:/usr/obj/usr/src/sys/CHARON i386 (I cvsup'd source on this day just prior to kernel build - tag=RELENG_7) Now when my SCSI array is plugged into the AHA-2944 controller the kernel panics on boot. If the array is not plugged into the controller the system boots and runs fine. I've read the handbook and tried to follow what posts I read as to what would be of use in trouble shooting this. Please don't shoot the messenger. I'll be glad to supply and additional information - this box is not in production so I'm willing to try anything. Just to note, the controller and array had been in production under 6.2-RELEASE and prior versions and has been running for years without any problems. Thanks. --mikej KDB: debugger backends: ddb KDB: current backend: ddb Copyright (c) 1992-2007 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.0-BETA4 #6: Tue Dec 11 09:05:50 EST 2007 mikej@charon.confluentasp.com:/usr/obj/usr/src/sys/CHARON WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Mobile AMD Sempron(tm) Processor 3000+ (1799.81-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x20fc2 Stepping = 2 Features=0x78bfbff Features2=0x1 AMD Features=0xc2500800 AMD Features2=0x1 real memory = 1005453312 (958 MB) avail memory = 969691136 (924 MB) ACPI APIC Table: WITNESS: spin lock intrcnt not in order list ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, 3bde0000 (3) failed Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0 cpu0: on acpi0 powernow0: on cpu0 acpi_button0: on acpi0 acpi_button1: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 agp0: on hostb0 pcib1: at device 1.0 on pci0 pci1: on pcib1 vgapci0: mem 0xf0000000-0xf3ffffff,0xf4000000-0xf4ffffff irq 16 at device 0.0 on pci1 pcib2: at device 8.0 on pci0 pci2: on pcib2 ahc0: port 0xc000-0xc0ff mem 0xf6013000-0xf6013fff irq 16 at device 4.0 on pci2 ahc0: [ITHREAD] aic7880: Ultra Wide Channel A, SCSI Id=7, 16/253 SCBs ahc1: port 0xc400-0xc4ff mem 0xf6010000-0xf6010fff irq 17 at device 5.0 on pci2 ahc1: [ITHREAD] aic7880: Ultra Wide Channel A, SCSI Id=7, 16/253 SCBs ahc2: port 0xc800-0xc8ff mem 0xf6011000-0xf6011fff irq 18 at device 6.0 on pci2 ahc2: [ITHREAD] aic7880: Ultra Wide Channel A, SCSI Id=7, 16/253 SCBs ahc3: port 0xcc00-0xccff mem 0xf6012000-0xf6012fff irq 19 at device 7.0 on pci2 ahc3: [ITHREAD] aic7880: Ultra Wide Channel A, SCSI Id=7, 16/253 SCBs atapci0: port 0xe100-0xe107,0xe700-0xe703,0xe800-0xe807,0xe900-0xe903,0xe000-0xe00f,0x d000-0xd0ff irq 20 at device 15.0 on pci0 atapci0: [ITHREAD] ata2: on atapci0 ata2: [ITHREAD] ata3: on atapci0 ata3: [ITHREAD] atapci1: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xe200-0xe20f at device 15.1 on pci0 ata0: on atapci1 ata0: [ITHREAD] ata1: on atapci1 ata1: [ITHREAD] uhci0: port 0xe300-0xe31f irq 21 at device 16.0 on pci0 uhci0: [GIANT-LOCKED] uhci0: [ITHREAD] usb0: on uhci0 usb0: USB revision 1.0 uhub0: on usb0 uhub0: 2 ports with 2 removable, self powered uhci1: port 0xe400-0xe41f irq 21 at device 16.1 on pci0 uhci1: [GIANT-LOCKED] uhci1: [ITHREAD] usb1: on uhci1 usb1: USB revision 1.0 uhub1: on usb1 uhub1: 2 ports with 2 removable, self powered uhci2: port 0xe500-0xe51f irq 21 at device 16.2 on pci0 uhci2: [GIANT-LOCKED] uhci2: [ITHREAD] usb2: on uhci2 usb2: USB revision 1.0 uhub2: on usb2 uhub2: 2 ports with 2 removable, self powered uhci3: port 0xe600-0xe61f irq 21 at device 16.3 on pci0 uhci3: [GIANT-LOCKED] uhci3: [ITHREAD] usb3: on uhci3 usb3: USB revision 1.0 uhub3: on usb3 uhub3: 2 ports with 2 removable, self powered ehci0: mem 0xf6100000-0xf61000ff irq 21 at device 16.4 on pci0 ehci0: [GIANT-LOCKED] ehci0: [ITHREAD] usb4: EHCI version 1.0 usb4: companion controllers, 2 ports each: usb0 usb1 usb2 usb3 usb4: on ehci0 usb4: USB revision 2.0 uhub4: on usb4 uhub4: 8 ports with 8 removable, self powered isab0: at device 17.0 on pci0 isa0: on isab0 pci0: at device 17.5 (no driver attached) vr0: port 0xdc00-0xdcff mem 0xf6101000-0xf61010ff irq 23 at device 18.0 on pci0 vr0: Quirks: 0x0 miibus0: on vr0 rlphy0: PHY 1 on miibus0 rlphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto vr0: using obsoleted if_watchdog interface vr0: Ethernet address: 00:e0:4d:0a:1b:e6 vr0: [ITHREAD] acpi_tz0: on acpi0 fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: [FILTER] fd0: <1440-KB 3.5" drive> on fdc0 drive 0 sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: configured irq 4 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A, console sio0: [FILTER] atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model IntelliMouse, device ID 3 pmtimer0 on isa0 orm0: at iomem 0xcc000-0xcc7ff pnpid ORM0000 on isa0 ppc0: at port 0x378-0x37f irq 7 on isa0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ppbus0: on ppc0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 plip0: on ppbus0 ppc0: [GIANT-LOCKED] ppc0: [ITHREAD] sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounter "TSC" frequency 1799808459 Hz quality 800 Timecounters tick every 1.000 msec Waiting 5 seconds for SCSI devices to settle ad4: 381554MB at ata2-master SATA150 suspending ithread with the following locks held: exclusive sleep mutex XPT lock r = 0 (0xc0b883a4) locked @ /usr/src/sys/cam/cam_xpt.c:3852 panic: witness_warn cpuid = 0 KDB: enter: panic [thread pid 18 tid 100016 ] Stopped at kdb_enter+0x32: leave db> ps pid ppid pgrp uid state wmesg wchan cmd 53 0 0 0 SL waiting_ 0xc0c084cc [sctp_iterator] 52 0 0 0 WL [irq7: ppc0] 51 0 0 0 WL [irq12: psm0] 50 0 0 0 WL [irq1: atkbd0] 49 0 0 0 WL [swi0: sio] 48 0 0 0 SL - 0xc401f23c [fdc0] 47 0 0 0 SL cooling 0xc3fedad4 [acpi_cooling0] 46 0 0 0 SL tzpoll 0xc0d85d40 [acpi_thermal] 45 0 0 0 WL [irq23: vr0] 44 0 0 0 SL usbevt 0xc3fe4210 [usb4] 43 0 0 0 SL usbevt 0xc3ffb210 [usb3] 42 0 0 0 SL usbevt 0xc3ff5210 [usb2] 41 0 0 0 SL usbevt 0xc3fee210 [usb1] 40 0 0 0 SL usbtsk 0xc0bb7174 [usbtask-dr] 39 0 0 0 SL usbtsk 0xc0bb7160 [usbtask-hc] 38 0 0 0 SL usbevt 0xc3fc8210 [usb0] 37 0 0 0 WL [irq21: uhci0 uhci*] 36 0 0 0 WL [irq15: ata1] 35 0 0 0 WL [irq14: ata0] --More-- 34 0 0 0 WL [irq20: atapci0] 33 0 0 0 SL idle 0xc3ef1e00 [aic_recovery3] 32 0 0 0 RL [irq19: ahc3] 31 0 0 0 SL idle 0xc3ef1e00 [aic_recovery3] 30 0 0 0 SL idle 0xc3ef1400 [aic_recovery2] 29 0 0 0 RL [irq18: ahc2] 28 0 0 0 SL idle 0xc3ef1400 [aic_recovery2] 27 0 0 0 SL idle 0xc3ef0a00 [aic_recovery1] 26 0 0 0 RL [irq17: ahc1] 25 0 0 0 SL idle 0xc3ef0a00 [aic_recovery1] 24 0 0 0 SL idle 0xc3ef0c00 [aic_recovery0] 23 0 0 0 WL [irq16: ahc0] 22 0 0 0 SL idle 0xc3ef0c00 [aic_recovery0] 21 0 0 0 WL [irq9: acpi0] 20 0 0 0 SL - 0xc3ed9580 [thread taskq] 19 0 0 0 WL [swi5: +] 18 0 0 0 RL CPU 0 [swi2: cambio] 9 0 0 0 SL ccb_scan 0xc0b88374 [xpt_thrd] 8 0 0 0 SL - 0xc3ed9900 [acpi_task_2] 7 0 0 0 SL - 0xc3ed9900 [acpi_task_1] --More-- 6 0 0 0 SL - 0xc3ed9900 [acpi_task_0] 5 0 0 0 SL - 0xc3ed9980 [kqueue taskq] 17 0 0 0 WL [swi6: task queue] 16 0 0 0 WL [swi6: Giant taskq] 15 0 0 0 SL - 0xc0bb94d4 [yarrow] 4 0 0 0 SL - 0xc0bb754c [g_down] 3 0 0 0 SL - 0xc0bb7548 [g_up] 2 0 0 0 SL - 0xc0bb7540 [g_event] 14 0 0 0 WL [swi3: vm] 13 0 0 0 RL [swi4: clock sio] 12 0 0 0 WL [swi1: net] 11 0 0 0 RL [idle: cpu0] 1 0 0 0 ?L [swapper] 10 0 0 0 SL audit_wo 0xc0c11ad4 [audit] 0 0 0 0 SLs conifhk 0xc0b599ec [swapper] db> show reg cs 0x20 ds 0x28 es 0x28 fs 0x8 ss 0x28 eax 0x12 ecx 0xc1434bde edx 0 ebx 0x100 esp 0xe175cc5c ebp 0xe175cc64 esi 0x1 edi 0xc3dea210 eip 0xc0777ca2 kdb_enter+0x32 efl 0x80282 kdb_enter+0x32: leave db> show locks exclusive sleep mutex XPT lock r = 0 (0xc0b883a4) locked @ /usr/src/sys/cam/cam_xpt.c:3852 db> show lockedvnods Locked vnodes db> bt Tracing pid 18 tid 100016 td 0xc3dea210 kdb_enter(c0aa2af2,0,c0aa7718,e175cc9c,0,...) at kdb_enter+0x32 panic(c0aa7718,e175ccd8,0,0,0,...) at panic+0x124 witness_warn(2,0,c0a9f7d0,471,c3ed97e4,...) at witness_warn+0x19c ithread_loop(c3ed80d0,e175cd38,c0a9f445,2ea,c3ee42ac,...) at ithread_loop+0x2ca fork_exit(c0735670,c3ed80d0,e175cd38) at fork_exit+0xb8 fork_trampoline() at fork_trampoline+0x8 --- trap 0, eip = 0, esp = 0xe175cd70, ebp = 0 --- db> CONFIDENTIALITY NOTE: This message is intended only for the use of the individual or entity to whom it is addressed and may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this transmission in error, please notify us by telephone at (502) 212-4001 or notify us at PAI , Dept. 99, 11857 Commonwealth Drive, Louisville, KY 40299. Thank you.