Date: Tue, 4 Jun 2002 21:42:31 -0400 From: Dale Woolridge <dale-freebsd-pr-submit@woolridge.org> To: FreeBSD-gnats-submit@FreeBSD.org Subject: kern/38909: kernel panic in lockmgr...with invalid pid/lockholder Message-ID: <20020605014231.GA494@woolridge.ca>
next in thread | raw e-mail | index | archive | help
>Number: 38909
>Category: kern
>Synopsis: kernel panic in lockmgr...with invalid pid/lockholder
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Tue Jun 04 18:30:01 PDT 2002
>Closed-Date:
>Last-Modified:
>Originator: Dale Woolridge
>Release: FreeBSD 4.6-RC i386
>Organization:
>Environment:
System: FreeBSD najla.woolridge.ca 4.6-RC FreeBSD 4.6-RC #10: Tue Jun 4 16:14:04 EDT 2002 root@najla.woolridge.ca:/usr/obj/usr/src/sys/NAJLA i386
Even though the problem occurs on 4.6-RC, it was also happening under 4.5-STABLE.
It could be easily reproduced under 4.5-STABLE too.
Here's some gdb output:
--- gdb.session begins here ---
Script started on Tue Jun 4 21:22:41 2002
najla# gdb -k kernel.debug.20020603 vmcore.1
GNU gdb 4.18 (FreeBSD)
Copyright 1998 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-unknown-freebsd"...
SMP 2 cpus
IdlePTD at phsyical address 0x00361000
initial pcb at physical address 0x002cd3c0
panicstr: rslock: cpu: 1, addr: 0xc6099da8, lock: 0x01000001
panic messages:
---
panic: lockmgr: pid -2, not exclusive lock holder -268435458 unlocking
mp_lock = 01000001; cpuid = 1; lapic.id = 01000000
boot() called on cpu#1
syncing disks... panic: rslock: cpu: 1, addr: 0xc6099da8, lock: 0x01000001
mp_lock = 01000001; cpuid = 1; lapic.id = 01000000
boot() called on cpu#1
Uptime: 40m49s
dumping to dev #ad/0x20021, offset 530560
dump ata2: resetting devices .. done
255 254 253 252 251 250 249 248 247 246 245 244 243 242 241 240 239 238 237 236 235 234 233 232 231 230 229 228 227 226 225 224 223 222 221 220 219 218 217 216 215 214 213 212 211 210 209 208 207 206 205 204 203 202 201 200 199 198 197 196 195 194 193 192 191 190 189 188 187 186 185 184 183 182 181 180 179 178 177 176 175 174 173 172 171 170 169 168 167 166 165 164 163 162 161 160 159 158 157 156 155 154 153 152 151 150 149 148 147 146 145 144 143 142 141 140 139 138 137 136 135 134 133 132 131 130 129 128 127 126 125 124 123 122 121 120 119 118 117 116 115 114 113 112 111 110 109 108 107 106 105 104 103 102 101 100 99 98 97 96 95 94 93 92 91 90 89 88 87 86 85 84 83 82 81 80 79 78 77 76 75 74 73 72 71 70 69 68 67 66 65 64 63 62 61 60 59 58 57 56 55 54 53 52 51 50 49 48 47 46 45 44 43 42 41 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0
---
#0 dumpsys () at /usr/src/sys/kern/kern_shutdown.c:487
487 if (dumping++) {
(kgdb) where
#0 dumpsys () at /usr/src/sys/kern/kern_shutdown.c:487
#1 0xc0162b7b in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:316
#2 0xc0162fed in panic (fmt=0xc025aa82 "rslock: cpu: %d, addr: 0x%08x, lock: 0x%08x") at /usr/src/sys/kern/kern_shutdown.c:595
#3 0xc025aa82 in bsl1 ()
#4 0xc0162966 in boot (howto=256) at /usr/src/sys/sys/buf.h:356
#5 0xc0162fed in panic (fmt=0xc027e860 "lockmgr: pid %d, not %s %d unlocking") at /usr/src/sys/kern/kern_shutdown.c:595
#6 0xc015d468 in lockmgr (lkp=0xc6099da8, flags=6, interlkp=0x0, p=0x0) at /usr/src/sys/kern/kern_lock.c:383
#7 0xc01895a9 in bqrelse (bp=0xc6099d80) at /usr/src/sys/sys/buf.h:320
#8 0xc018b111 in biodone (bp=0xc6099d80) at /usr/src/sys/kern/vfs_bio.c:2834
#9 0xc018d08c in cluster_callback (bp=0xc60458c0) at /usr/src/sys/kern/vfs_cluster.c:549
#10 0xc018ae3c in biodone (bp=0xc60458c0) at /usr/src/sys/kern/vfs_bio.c:2698
#11 0xc0136cd7 in ad_interrupt (request=0xc10afd00) at /usr/src/sys/dev/ata/ata-disk.c:694
#12 0xc012e47c in ata_intr (data=0xc0d48000) at /usr/src/sys/dev/ata/ata-all.c:614
#13 0xc0264b69 in intr_mux (arg=0xc0a33720) at /usr/src/sys/i386/isa/intr_machdep.c:582
(kgdb) up 6
#6 0xc015d468 in lockmgr (lkp=0xc6099da8, flags=6, interlkp=0x0, p=0x0) at /usr/src/sys/kern/kern_lock.c:383
383 panic("lockmgr: pid %d, not %s %d unlocking",
(kgdb) print *lkp
$1 = {lk_interlock = {lock_data = 16777217}, lk_flags = 2098176, lk_sharecount = 0, lk_waitcount = 1, lk_exclusivecount = 1,
lk_prio = 20, lk_wmesg = 0xc02829a1 "getblk", lk_timo = 0, lk_lockholder = -268435458}
(kgdb) quit
najla# exit
exit
Script done on Tue Jun 4 21:24:19 2002
--- gdb.session ends here ---
>Description:
I've only ever noticed this happening when I perform large/many
file operations, but it has happened even when only operating on a handful
of files (< 100).
>How-To-Repeat:
I can pretty much cause this to happen anytime I want by doing something like:
# cd /var
# grep -r xxx .
I'd be more than happy to supply more info or make any other system modifications
to help find the problem.
>Fix:
--- dmesg.boot begins here ---
Copyright (c) 1992-2002 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD 4.6-RC #10: Tue Jun 4 16:14:04 EDT 2002
root@najla.woolridge.ca:/usr/obj/usr/src/sys/NAJLA
Timecounter "i8254" frequency 1193182 Hz
CPU: Pentium III/Pentium III Xeon/Celeron (732.13-MHz 686-class CPU)
Origin = "GenuineIntel" Id = 0x686 Stepping = 6
Features=0x387fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,PN,MMX,FXSR,SSE>
real memory = 268369920 (262080K bytes)
avail memory = 258007040 (251960K bytes)
Programming 24 pins in IOAPIC #0
IOAPIC #0 intpin 2 -> irq 0
FreeBSD/SMP: Multiprocessor motherboard
cpu0 (BSP): apic id: 0, version: 0x00040011, at 0xfee00000
cpu1 (AP): apic id: 1, version: 0x00040011, at 0xfee00000
io0 (APIC): apic id: 2, version: 0x00178011, at 0xfec00000
Preloaded elf kernel "kernel" at 0xc0342000.
VESA: v3.0, 16384k memory, flags:0x1, mode table:0xc02e1457 (1000117)
VESA: 3Dfx Interactive, Inc.
netsmb_dev: loaded
Pentium Pro MTRR support enabled
md0: Malloc disk
Using $PIR table, 8 entries at 0xc00fdbc0
npx0: <math processor> on motherboard
npx0: INT 16 interface
pcib0: <Host to PCI bridge> on motherboard
pci0: <PCI bus> on pcib0
pcib1: <VIA 82C598MVP (Apollo MVP3) PCI-PCI (AGP) bridge> at device 1.0 on pci0
pci1: <PCI bus> on pcib1
isab0: <VIA 82C686 PCI-ISA bridge> at device 7.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <VIA 82C686 ATA100 controller> port 0xc000-0xc00f at device 7.1 on pci0
ata0: at 0x1f0 irq 14 on atapci0
ata1: at 0x170 irq 15 on atapci0
uhci0: <VIA 83C572 USB controller> port 0xc400-0xc41f irq 10 at device 7.2 on pci0
usb0: <VIA 83C572 USB controller> on uhci0
usb0: USB revision 1.0
uhub0: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
uhci1: <VIA 83C572 USB controller> port 0xc800-0xc81f irq 10 at device 7.3 on pci0
usb1: <VIA 83C572 USB controller> on uhci1
usb1: USB revision 1.0
uhub1: VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1: 2 ports with 2 removable, self powered
pci0: <unknown card> (vendor=0x1106, dev=0x3057) at 7.4
pci0: <3Dfx Voodoo Banshee graphics accelerator> at 9.0 irq 5
dc0: <Macronix 98715AEC-C 10/100BaseTX> port 0xd000-0xd0ff mem 0xd9000000-0xd90000ff irq 10 at device 12.0 on pci0
dc0: Ethernet address: 00:80:c6:e9:b4:80
miibus0: <MII bus> on dc0
dcphy0: <Intel 21143 NWAY media interface> on miibus0
dcphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
atapci1: <HighPoint HPT370 ATA100 controller> port 0xe400-0xe4ff,0xe000-0xe003,0xdc00-0xdc07,0xd800-0xd803,0xd400-0xd407 irq 11 at device 14.0 on pci0
ata2: at 0xd400 on atapci1
ata3: at 0xdc00 on atapci1
orm0: <Option ROMs> at iomem 0xc0000-0xc7fff,0xc8000-0xcb7ff on isa0
fdc0: ready for input in output
fdc0: cmd 3 failed at out byte 1 of 3
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: model MouseMan+, device ID 0
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
sc0: <System console> on isa0
sc0: VGA <16 virtual consoles, flags=0x200>
APIC_IO: Testing 8254 interrupt delivery
APIC_IO: routing 8254 via IOAPIC #0 intpin 2
SMP: AP CPU #1 Launched!
ad4: 19623MB <IC35L020AVER07-0> [39870/16/63] at ata2-master UDMA100
ad5: 29314MB <IC35L030AVER07-0> [59560/16/63] at ata2-slave UDMA100
Mounting root from ufs:/dev/ad4s1a
WARNING: / was not properly dismounted
--- dmesg.boot ends here ---
--- kernel config begins here ---
machine i386
cpu I686_CPU
ident NAJLA
maxusers 32
options INCLUDE_CONFIG_FILE
options DDB, DDB_UNATTENDED
makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols
options MATH_EMULATE #Support for x87 emulation
options INET #InterNETworking
options FFS #Berkeley Fast Filesystem
options FFS_ROOT #FFS usable as root device [keep this!]
options UFS_DIRHASH #Improve performance on big directories
options MFS #Memory Filesystem
options MD_ROOT #MD is a potential root device
options NFS #Network Filesystem
options NFS_ROOT #NFS usable as root device, NFS required
options MSDOSFS #MSDOS Filesystem
options PROCFS #Process filesystem
options COMPAT_43 #Compatible with BSD 4.3 [KEEP THIS!]
options UCONSOLE #Allow users to grab the console
options USERCONFIG #boot -c editor
options VISUAL_USERCONFIG #visual boot -c editor
options KTRACE #ktrace(1) support
options SYSVSHM #SYSV-style shared memory
options SYSVMSG #SYSV-style message queues
options SYSVSEM #SYSV-style semaphores
options P1003_1B #Posix P1003_1B real-time extentions
options _KPOSIX_PRIORITY_SCHEDULING
options ICMP_BANDLIM #Rate limit bad replies
# these required by NETSMB
options LIBMCHAIN #mbuf management library
options LIBICONV
options NETSMB #SMB/CIFS requester
options NETSMBCRYPTO #encrypted password support for SMB
options SMP
options APIC_IO
options EXT2FS #Add support for the EXT2FS filesystem of Linux fame. Be a bit
options SMBFS #SMB/CIFS filesystem
# Coda stuff:
options CODA #CODA filesystem.
pseudo-device vcoda 4 #coda minicache <-> venus comm.
device isa
device eisa
device pci
# Floppy drives
device fdc0 at isa? port IO_FD1 irq 6 drq 2
device fd0 at fdc0 drive 0
# ATA and ATAPI devices
device ata
device atadisk # ATA disk drives
device atapicd # ATAPI CDROM drives
device atapifd # ATAPI floppy drives
options ATA_STATIC_ID #Static device numbering
# atkbdc0 controls both the keyboard and the PS/2 mouse
device atkbdc0 at isa? port IO_KBD
device atkbd0 at atkbdc? irq 1
device psm0 at atkbdc? irq 12
options VESA
#options VESA_DEBUG=9
device vga0 at isa?
# splash screen/screen saver
pseudo-device splash
# syscons is the default console driver, resembling an SCO console
device sc0 at isa?
# Floating point support - do not disable.
device npx0 at nexus? port IO_NPX irq 13
# PCI Ethernet NICs that use the common MII bus controller code.
device miibus # MII bus support
device dc # DEC/Intel 21143 and various workalikes
# Pseudo devices - the number indicates how many units to allocated.
pseudo-device loop # Network loopback
pseudo-device ether # Ethernet support
pseudo-device pty # Pseudo-ttys (telnet etc)
pseudo-device md # Memory "disks"
pseudo-device snp 3 #Snoop device - to look at pty/vty/etc..
pseudo-device gzip #Exec gzipped a.out's
# The `bpf' pseudo-device enables the Berkeley Packet Filter.
# Be aware of the administrative consequences of enabling this!
pseudo-device bpf #Berkeley packet filter
#pseudo-device speaker #Play IBM BASIC-style noises out your speaker
#device sb0 at isa? port 0x220 irq 5 drq 1
#device awe0 at isa? port 0x620
#device pcm
#device sbc
# USB support
device uhci # UHCI controller
device usb # General USB code (mandatory for USB)
device ugen # Generic USB device driver
device uhid # Human Interface Device (anything with buttons and dials)
#device ukbd # USB keyboard
device ulpt # USB printer
#device umass # USB Iomega Zip 100 Drive
#device ums # USB mouse
device uscanner # USB scanners
#
# debugging options for the USB subsystem
#
#options UHCI_DEBUG
#options USB_DEBUG
#options UGEN_DEBUG
#options UHID_DEBUG
#options UKBD_DEBUG
#options ULPT_DEBUG
#options UMASS_DEBUG
#options UMS_DEBUG
# options for ukbd:
#options UKBD_DFLT_KEYMAP # specify the built-in keymap
#makeoptions UKBD_DFLT_KEYMAP=it.iso
--- kernel config ends here ---
>Release-Note:
>Audit-Trail:
>Unformatted:
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-bugs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020605014231.GA494>