From owner-freebsd-bugs Tue Jun 4 18:30:25 2002 Delivered-To: freebsd-bugs@hub.freebsd.org Received: from freefall.freebsd.org (freefall.FreeBSD.org [216.136.204.21]) by hub.freebsd.org (Postfix) with ESMTP id 8F34937B408 for ; Tue, 4 Jun 2002 18:30:01 -0700 (PDT) Received: (from gnats@localhost) by freefall.freebsd.org (8.11.6/8.11.6) id g551U1047050; Tue, 4 Jun 2002 18:30:01 -0700 (PDT) (envelope-from gnats) Received: from woolridge.org (H10.C245.tor.velocet.net [216.138.245.10]) by hub.freebsd.org (Postfix) with SMTP id F1CE937B40A for ; Tue, 4 Jun 2002 18:28:54 -0700 (PDT) Received: (qmail 18115 invoked from network); 5 Jun 2002 01:42:31 -0000 Received: from unknown (192.168.0.4) by home.woolridge.org with QMQP; 5 Jun 2002 01:42:31 -0000 Message-Id: <20020605014231.GA494@woolridge.ca> Date: Tue, 4 Jun 2002 21:42:31 -0400 From: Dale Woolridge Reply-To: Dale Woolridge To: FreeBSD-gnats-submit@FreeBSD.org X-Send-Pr-Version: 3.113 Subject: kern/38909: kernel panic in lockmgr...with invalid pid/lockholder Sender: owner-freebsd-bugs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org >Number: 38909 >Category: kern >Synopsis: kernel panic in lockmgr...with invalid pid/lockholder >Confidential: no >Severity: critical >Priority: high >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Tue Jun 04 18:30:01 PDT 2002 >Closed-Date: >Last-Modified: >Originator: Dale Woolridge >Release: FreeBSD 4.6-RC i386 >Organization: >Environment: System: FreeBSD najla.woolridge.ca 4.6-RC FreeBSD 4.6-RC #10: Tue Jun 4 16:14:04 EDT 2002 root@najla.woolridge.ca:/usr/obj/usr/src/sys/NAJLA i386 Even though the problem occurs on 4.6-RC, it was also happening under 4.5-STABLE. It could be easily reproduced under 4.5-STABLE too. Here's some gdb output: --- gdb.session begins here --- Script started on Tue Jun 4 21:22:41 2002 najla# gdb -k kernel.debug.20020603 vmcore.1 GNU gdb 4.18 (FreeBSD) Copyright 1998 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-unknown-freebsd"... SMP 2 cpus IdlePTD at phsyical address 0x00361000 initial pcb at physical address 0x002cd3c0 panicstr: rslock: cpu: 1, addr: 0xc6099da8, lock: 0x01000001 panic messages: --- panic: lockmgr: pid -2, not exclusive lock holder -268435458 unlocking mp_lock = 01000001; cpuid = 1; lapic.id = 01000000 boot() called on cpu#1 syncing disks... panic: rslock: cpu: 1, addr: 0xc6099da8, lock: 0x01000001 mp_lock = 01000001; cpuid = 1; lapic.id = 01000000 boot() called on cpu#1 Uptime: 40m49s dumping to dev #ad/0x20021, offset 530560 dump ata2: resetting devices .. done 255 254 253 252 251 250 249 248 247 246 245 244 243 242 241 240 239 238 237 236 235 234 233 232 231 230 229 228 227 226 225 224 223 222 221 220 219 218 217 216 215 214 213 212 211 210 209 208 207 206 205 204 203 202 201 200 199 198 197 196 195 194 193 192 191 190 189 188 187 186 185 184 183 182 181 180 179 178 177 176 175 174 173 172 171 170 169 168 167 166 165 164 163 162 161 160 159 158 157 156 155 154 153 152 151 150 149 148 147 146 145 144 143 142 141 140 139 138 137 136 135 134 133 132 131 130 129 128 127 126 125 124 123 122 121 120 119 118 117 116 115 114 113 112 111 110 109 108 107 106 105 104 103 102 101 100 99 98 97 96 95 94 93 92 91 90 89 88 87 86 85 84 83 82 81 80 79 78 77 76 75 74 73 72 71 70 69 68 67 66 65 64 63 62 61 60 59 58 57 56 55 54 53 52 51 50 49 48 47 46 45 44 43 42 41 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 --- #0 dumpsys () at /usr/src/sys/kern/kern_shutdown.c:487 487 if (dumping++) { (kgdb) where #0 dumpsys () at /usr/src/sys/kern/kern_shutdown.c:487 #1 0xc0162b7b in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:316 #2 0xc0162fed in panic (fmt=0xc025aa82 "rslock: cpu: %d, addr: 0x%08x, lock: 0x%08x") at /usr/src/sys/kern/kern_shutdown.c:595 #3 0xc025aa82 in bsl1 () #4 0xc0162966 in boot (howto=256) at /usr/src/sys/sys/buf.h:356 #5 0xc0162fed in panic (fmt=0xc027e860 "lockmgr: pid %d, not %s %d unlocking") at /usr/src/sys/kern/kern_shutdown.c:595 #6 0xc015d468 in lockmgr (lkp=0xc6099da8, flags=6, interlkp=0x0, p=0x0) at /usr/src/sys/kern/kern_lock.c:383 #7 0xc01895a9 in bqrelse (bp=0xc6099d80) at /usr/src/sys/sys/buf.h:320 #8 0xc018b111 in biodone (bp=0xc6099d80) at /usr/src/sys/kern/vfs_bio.c:2834 #9 0xc018d08c in cluster_callback (bp=0xc60458c0) at /usr/src/sys/kern/vfs_cluster.c:549 #10 0xc018ae3c in biodone (bp=0xc60458c0) at /usr/src/sys/kern/vfs_bio.c:2698 #11 0xc0136cd7 in ad_interrupt (request=0xc10afd00) at /usr/src/sys/dev/ata/ata-disk.c:694 #12 0xc012e47c in ata_intr (data=0xc0d48000) at /usr/src/sys/dev/ata/ata-all.c:614 #13 0xc0264b69 in intr_mux (arg=0xc0a33720) at /usr/src/sys/i386/isa/intr_machdep.c:582 (kgdb) up 6 #6 0xc015d468 in lockmgr (lkp=0xc6099da8, flags=6, interlkp=0x0, p=0x0) at /usr/src/sys/kern/kern_lock.c:383 383 panic("lockmgr: pid %d, not %s %d unlocking", (kgdb) print *lkp $1 = {lk_interlock = {lock_data = 16777217}, lk_flags = 2098176, lk_sharecount = 0, lk_waitcount = 1, lk_exclusivecount = 1, lk_prio = 20, lk_wmesg = 0xc02829a1 "getblk", lk_timo = 0, lk_lockholder = -268435458} (kgdb) quit najla# exit exit Script done on Tue Jun 4 21:24:19 2002 --- gdb.session ends here --- >Description: I've only ever noticed this happening when I perform large/many file operations, but it has happened even when only operating on a handful of files (< 100). >How-To-Repeat: I can pretty much cause this to happen anytime I want by doing something like: # cd /var # grep -r xxx . I'd be more than happy to supply more info or make any other system modifications to help find the problem. >Fix: --- dmesg.boot begins here --- Copyright (c) 1992-2002 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 4.6-RC #10: Tue Jun 4 16:14:04 EDT 2002 root@najla.woolridge.ca:/usr/obj/usr/src/sys/NAJLA Timecounter "i8254" frequency 1193182 Hz CPU: Pentium III/Pentium III Xeon/Celeron (732.13-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x686 Stepping = 6 Features=0x387fbff real memory = 268369920 (262080K bytes) avail memory = 258007040 (251960K bytes) Programming 24 pins in IOAPIC #0 IOAPIC #0 intpin 2 -> irq 0 FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 0, version: 0x00040011, at 0xfee00000 cpu1 (AP): apic id: 1, version: 0x00040011, at 0xfee00000 io0 (APIC): apic id: 2, version: 0x00178011, at 0xfec00000 Preloaded elf kernel "kernel" at 0xc0342000. VESA: v3.0, 16384k memory, flags:0x1, mode table:0xc02e1457 (1000117) VESA: 3Dfx Interactive, Inc. netsmb_dev: loaded Pentium Pro MTRR support enabled md0: Malloc disk Using $PIR table, 8 entries at 0xc00fdbc0 npx0: on motherboard npx0: INT 16 interface pcib0: on motherboard pci0: on pcib0 pcib1: at device 1.0 on pci0 pci1: on pcib1 isab0: at device 7.0 on pci0 isa0: on isab0 atapci0: port 0xc000-0xc00f at device 7.1 on pci0 ata0: at 0x1f0 irq 14 on atapci0 ata1: at 0x170 irq 15 on atapci0 uhci0: port 0xc400-0xc41f irq 10 at device 7.2 on pci0 usb0: on uhci0 usb0: USB revision 1.0 uhub0: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered uhci1: port 0xc800-0xc81f irq 10 at device 7.3 on pci0 usb1: on uhci1 usb1: USB revision 1.0 uhub1: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub1: 2 ports with 2 removable, self powered pci0: (vendor=0x1106, dev=0x3057) at 7.4 pci0: <3Dfx Voodoo Banshee graphics accelerator> at 9.0 irq 5 dc0: port 0xd000-0xd0ff mem 0xd9000000-0xd90000ff irq 10 at device 12.0 on pci0 dc0: Ethernet address: 00:80:c6:e9:b4:80 miibus0: on dc0 dcphy0: on miibus0 dcphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto atapci1: port 0xe400-0xe4ff,0xe000-0xe003,0xdc00-0xdc07,0xd800-0xd803,0xd400-0xd407 irq 11 at device 14.0 on pci0 ata2: at 0xd400 on atapci1 ata3: at 0xdc00 on atapci1 orm0: