From owner-freebsd-current Thu Apr 5 0:26:21 2001 Delivered-To: freebsd-current@freebsd.org Received: from kyle.tandemedia.com (kyle.tandemedia.com [216.29.169.3]) by hub.freebsd.org (Postfix) with ESMTP id AF25037B443 for ; Thu, 5 Apr 2001 00:26:07 -0700 (PDT) (envelope-from rmtodd@ichotolot.servalan.com) Received: by kyle.tandemedia.com (Postfix, from userid 66) id 1D21855406; Thu, 5 Apr 2001 03:26:07 -0400 (EDT) Received: from ichotolot.servalan.com([127.0.0.1]) (16468 bytes) by servalan.servalan.com via sendmail with P:esmtp/R:smart_host/T:hacked-uux (sender: ) id for ; Thu, 5 Apr 2001 01:39:35 -0500 (CDT) (Smail-3.2.0.111 2000-Feb-17 #1 built 2001-Jan-15) Message-Id: To: current@freebsd.org Subject: Panic I got: mutex sx backing lock recursed at ../../kern/kern_condvar.c:198 Date: Thu, 05 Apr 2001 01:39:35 -0500 From: Richard Todd Sender: owner-freebsd-current@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I'm running -CURRENT on a dual PII/400 box with 128M of RAM. The kernel I'm running was built from sources current as of last night (i.e. around 9PM CDT Apr 3). Just now, while listening to streaming audio with xmms, the machine crashed. It's done that a couple times before, with recent-ish kernels while doing streaming audio with xmms, but the other times didn't give core dumps with usable backtraces. *This* time I got a decent backtrace. If I'm reading this backtrace right, the thread handling the sound hardware called selwakeup() (frame #19). This called pfind() (frame #18), which tries to lock allproc. Somewhere in doing this, witness_sleep() (frame #15) decides it wants to printf() a message. printf() calls down into the tty code, which goes into ptsstart() (frame #9) and the pty code (I'm not entirely sure why). This code then tries to do a selwakeup() of its own (frame #7) which calls pfind() which tries (again) to lock allproc, leading to the "mutex recursed" panic. GDB output and (if it matters) kernel config file below. Script started on Thu Apr 5 01:12:28 2001 ichotolot# cd /usr/src/sys/compile/ICHOTOLOTSMP ichotolot# gdb -k kernel.debug /var/crash/vmcore.7 GNU gdb 4.18 Copyright 1998 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-unknown-freebsd"... IdlePTD 6356992 initial pcb at 513860 panicstr: mutex sx backing lock recursed at ../../kern/kern_condvar.c:198 panic messages: --- panic: mutex sx backing lock recursed at ../../kern/kern_condvar.c:198 cpuid = 0; lapic.id = 00000000 boot() called on cpu#0 syncing disks... 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 16 1: dev:da0s2e, flags:21021024, blkno:11469104, lblkno:11469104 2: dev:da0s2e, flags:21021024, blkno:11468864, lblkno:11468864 3: dev:da0s2e, flags:21000024, blkno:2048, lblkno:2048 4: dev:da0s2e, flags:21021024, blkno:2752848, lblkno:2752848 5: dev:da0s2e, flags:21021024, blkno:2752736, lblkno:2752736 6: dev:da0s2e, flags:21021024, blkno:11468976, lblkno:11468976 7: dev:da0s2a, flags:21021024, blkno:131152, lblkno:131152 8: dev:da0s2e, flags:21021024, blkno:2294176, lblkno:2294176 9: dev:da0s2e, flags:21021024, blkno:2425120, lblkno:2425120 10: dev:da0s2a, flags:21021024, blkno:131184, lblkno:131184 11: dev:da0s2e, flags:21000024, blkno:16, lblkno:16 12: dev:da0s2e, flags:21021024, blkno:2294160, lblkno:2294160 13: dev:da0s2e, flags:21021024, blkno:14221440, lblkno:14221440 14: dev:da0s2e, flags:21021024, blkno:2294192, lblkno:2294192 15: dev:da0s2e, flags:01011024, blkno:11474186, lblkno:0 16: dev:da0s2e, flags:01000024, blkno:11468848, lblkno:11468848 giving up on 16 buffers Uptime: 23h3m37s dumping to dev da0s2b, offset 270336 dump 128 127 126 125 124 123 122 121 120 119 118 117 116 115 114 113 112 111 110 109 108 107 106 105 104 103 102 101 100 99 98 97 96 95 94 93 92 91 90 89 88 87 86 85 84 83 82 81 80 79 78 77 76 75 74 73 72 71 70 69 68 67 66 65 64 63 62 61 60 59 58 57 56 55 54 53 52 51 50 49 48 47 46 45 44 43 42 41 40 39 38 37 36 35 34 33 32 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 --- #0 dumpsys () at ../../kern/kern_shutdown.c:478 478 if (dumping++) { (kgdb) bt #0 dumpsys () at ../../kern/kern_shutdown.c:478 #1 0xc0251547 in boot (howto=256) at ../../kern/kern_shutdown.c:321 #2 0xc0251a09 in panic (fmt=0xc0464a44 "mutex %s recursed at %s:%d") at ../../kern/kern_shutdown.c:592 #3 0xc024aec3 in _mtx_assert (m=0xc054765c, what=9, file=0xc0462932 "../../kern/kern_condvar.c", line=198) at ../../kern/kern_mutex.c:602 #4 0xc02369a2 in cv_wait (cvp=0xc0547698, mp=0xc054765c) at ../../kern/kern_condvar.c:198 #5 0xc0258caa in _sx_slock (sx=0xc0547640, file=0xc0464e23 "../../kern/kern_proc.c", line=143) at ../../kern/kern_sx.c:117 #6 0xc024bf48 in pfind (pid=606) at ../../kern/kern_proc.c:143 #7 0xc026ffe1 in selwakeup (sip=0xc10eea04) at ../../kern/sys_generic.c:1061 #8 0xc027accf in ptcwakeup (tp=0xc10eea20, flag=1) at ../../kern/tty_pty.c:318 #9 0xc027acaa in ptsstart (tp=0xc10eea20) at ../../kern/tty_pty.c:307 #10 0xc0278170 in ttstart (tp=0xc10eea20) at ../../kern/tty.c:1417 #11 0xc027978d in tputchar (c=46, tp=0xc10eea20) at ../../kern/tty.c:2484 #12 0xc0268813 in putchar (c=46, arg=0xc7f12e10) at ../../kern/subr_prf.c:304 #13 0xc0268fb8 in kvprintf ( fmt=0xc0468642 ":%d: %s with \"%s\" locked from %s:%d\n", func=0xc02687c4 , arg=0xc7f12e10, radix=10, ap=0xc7f12e2c "Ç") at ../../kern/subr_prf.c:637 #14 0xc0268740 in printf ( fmt=0xc0468640 "%s:%d: %s with \"%s\" locked from %s:%d\n") at ../../kern/subr_prf.c:260 #15 0xc026cff9 in witness_sleep (check_only=0, lock=0xc054765c, file=0xc0462932 "../../kern/kern_condvar.c", line=199) at ../../kern/subr_witness.c:725 #16 0xc02369b7 in cv_wait (cvp=0xc0547698, mp=0xc054765c) at ../../kern/kern_condvar.c:199 #17 0xc0258caa in _sx_slock (sx=0xc0547640, file=0xc0464e23 "../../kern/kern_proc.c", line=143) at ../../kern/kern_sx.c:117 #18 0xc024bf48 in pfind (pid=70414) at ../../kern/kern_proc.c:143 #19 0xc026ffe1 in selwakeup (sip=0xc0f59f54) at ../../kern/sys_generic.c:1061 #20 0xc01f5cea in chn_wakeup (c=0xc0f4d580) at ../../dev/sound/pcm/channel.c:97 #21 0xc01f5e4f in chn_wrfeed (c=0xc0f4d580) at ../../dev/sound/pcm/channel.c:188 #22 0xc01f5e75 in chn_wrintr (c=0xc0f4d580) at ../../dev/sound/pcm/channel.c:206 #23 0xc01f63dd in chn_intr (c=0xc0f4d580) at ../../dev/sound/pcm/channel.c:407 #24 0xc01df38e in sb_intr (arg=0xc0f4d700) at ../../dev/sound/isa/sb16.c:498 #25 0xc01e12b1 in sbc_intr (p=0xc0f4d9d8) at ../../dev/sound/isa/sbc.c:491 #26 0xc0243823 in ithread_loop (arg=0xc0f4d900) at ../../kern/kern_intr.c:517 #27 0xc0242018 in fork_exit (callout=0xc02433d0 , arg=0xc0f4d900, frame=0xc7f12fa8) at ../../kern/kern_fork.c:731 (kgdb) fr 14 #14 0xc0268740 in printf ( fmt=0xc0468640 "%s:%d: %s with \"%s\" locked from %s:%d\n") at ../../kern/subr_prf.c:260 260 retval = kvprintf(fmt, putchar, &pca, 10, ap); (kgdb) l 255 consintr = 0; 256 va_start(ap, fmt); 257 pca.tty = NULL; 258 pca.flags = TOCONS | TOLOG; 259 pca.pri = -1; 260 retval = kvprintf(fmt, putchar, &pca, 10, ap); 261 va_end(ap); 262 if (!panicstr) 263 msgbuftrigger = 1; 264 consintr = savintr; /* reenable interrupts */ (kgdb) fr 15 #15 0xc026cff9 in witness_sleep (check_only=0, lock=0xc054765c, file=0xc0462932 "../../kern/kern_condvar.c", line=199) at ../../kern/subr_witness.c:725 725 printf("%s:%d: %s with \"%s\" locked from %s:%d\n", (kgdb) l 720 lock1 = lle->ll_children[i]; 721 if (lock1 == lock || lock1 == &Giant.mtx_object || 722 (lock1->lo_flags & LO_SLEEPABLE)) 723 continue; 724 n++; 725 printf("%s:%d: %s with \"%s\" locked from %s:%d\n", 726 file, line, check_only ? "could sleep" : "sleeping", 727 lock1->lo_name, lock1->lo_file, lock1->lo_line); 728 } 729 if (lock_list == &p->p_sleeplocks) { (kgdb) p file $1 = 0xc0462932 "../../kern/kern_condvar.c" (kgdb) p line $2 = 199 (kgdb) p check_only $3 = 0 (kgdb) p lock1 $4 = (struct lock_object *) 0x0 [Is this supposed to be null? -- rmt] (kgdb) fr 16 #16 0xc02369b7 in cv_wait (cvp=0xc0547698, mp=0xc054765c) at ../../kern/kern_condvar.c:199 199 WITNESS_SLEEP(0, &mp->mtx_object); (kgdb) fr 17 #17 0xc0258caa in _sx_slock (sx=0xc0547640, file=0xc0464e23 "../../kern/kern_proc.c", line=143) at ../../kern/kern_sx.c:117 117 cv_wait(&sx->sx_shrd_cv, &sx->sx_lock); (kgdb) fr 18 #18 0xc024bf48 in pfind (pid=70414) at ../../kern/kern_proc.c:143 143 sx_slock(&allproc_lock); (kgdb) fr 19 #19 0xc026ffe1 in selwakeup (sip=0xc0f59f54) at ../../kern/sys_generic.c:1061 1061 p = pfind(sip->si_pid); (kgdb) q ichotolot# cat /usr/src/sys/i386/conf/ICHOTOLOTSMP # # GENERIC -- Generic kernel configuration file for FreeBSD/i386 # # For more information on this file, please read the handbook section on # Kernel Configuration Files: # # http://www.FreeBSD.org/handbook/kernelconfig-config.html # # The handbook is also available locally in /usr/share/doc/handbook # if you've installed the doc distribution, otherwise always see the # FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the # latest information. # # An exhaustive list of options and more detailed explanations of the # device lines is also present in the NOTES configuration file. If you are # in doubt as to the purpose or necessity of a line, check first in NOTES. # # $FreeBSD: src/sys/i386/conf/GENERIC,v 1.306 2001/03/16 11:52:25 sos Exp $ machine i386 cpu I686_CPU ident GENERIC maxusers 32 #To statically compile in device wiring instead of /boot/device.hints #hints "GENERIC.hints" #Default places to look for devices. makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols options MATH_EMULATE #Support for x87 emulation options INET #InterNETworking options INET6 #IPv6 communications protocols options FFS #Berkeley Fast Filesystem options SOFTUPDATES #Enable FFS soft updates support options MFS #Memory Filesystem options MD_ROOT #MD is a potential root device options NFS #Network Filesystem options NFS_ROOT #NFS usable as root device, NFS required options MSDOSFS #MSDOS Filesystem options CD9660 #ISO 9660 Filesystem options DEVFS #Device Filesystem options PROCFS #Process filesystem options COMPAT_43 #Compatible with BSD 4.3 [KEEP THIS!] options SCSI_DELAY=15000 #Delay (in ms) before probing SCSI options UCONSOLE #Allow users to grab the console options USERCONFIG #boot -c editor options VISUAL_USERCONFIG #visual boot -c editor options KTRACE #ktrace(1) support options SYSVSHM #SYSV-style shared memory options SYSVMSG #SYSV-style message queues options SYSVSEM #SYSV-style semaphores options P1003_1B #Posix P1003_1B real-time extensions options _KPOSIX_PRIORITY_SCHEDULING options KBD_INSTALL_CDEV # install a CDEV entry in /dev # To make an SMP kernel, the next two are needed options SMP # Symmetric MultiProcessor Kernel options APIC_IO # Symmetric (APIC) I/O device isa device eisa device pci # Floppy drives device fdc # ATA and ATAPI devices device ata device atadisk # ATA disk drives device atapicd # ATAPI CDROM drives device atapifd # ATAPI floppy drives device atapist # ATAPI tape drives options ATA_STATIC_ID #Static device numbering # SCSI Controllers device ahb # EISA AHA1742 family device ahc # AHA2940 and onboard AIC7xxx devices device amd # AMD 53C974 (Tekram DC-390(T)) device isp # Qlogic family #device ncr # NCR/Symbios Logic device sym # NCR/Symbios Logic (newer chipsets + those of `ncr') device adv # Advansys SCSI adapters device adw # Advansys wide SCSI adapters device aha # Adaptec 154x SCSI adapters device aic # Adaptec 15[012]x SCSI adapters, AIC-6[23]60. device bt # Buslogic/Mylex MultiMaster SCSI adapters device ncv # NCR 53C500 device nsp # Workbit Ninja SCSI-3 device stg # TMC 18C30/18C50 # RAID controllers interfaced to the SCSI subsystem device asr # DPT SmartRAID V, VI and Adaptec SCSI RAID device dpt # DPT Smartcache III, IV - See NOTES for options! device mly # Mylex AcceleRAID/eXtremeRAID # SCSI peripherals device scbus # SCSI bus (required) device da # Direct Access (disks) device sa # Sequential Access (tape etc) device cd # CD device pass # Passthrough device (direct SCSI access) # RAID controllers device aac # Adaptec FSA RAID device amr # AMI MegaRAID device ida # Compaq Smart RAID device mlx # Mylex DAC960 family device twe # 3ware ATA RAID # atkbdc0 controls both the keyboard and the PS/2 mouse device atkbdc 1 # At keyboard controller device atkbd # at keyboard device psm # psm mouse device vga # VGA screen # splash screen/screen saver device splash # syscons is the default console driver, resembling an SCO console device sc 1 # Enable this for the pcvt (VT220 compatible) console driver #device vt #options XSERVER # support for X server on a vt console #options FAT_CURSOR # start with block cursor # If you have a ThinkPAD, uncomment this along with the rest of the PCVT lines #options PCVT_SCANSET=2 # IBM keyboards are non-std # Floating point support - do not disable. device npx # Power management support (see NOTES for more options) device apm # Add suspend/resume support for the i8254. device pmtimer # Audio support device pcm # PCCARD (PCMCIA) support device card # pccard bus device pcic # PCMCIA bridge # Serial (COM) ports device sio # 8250, 16[45]50 based serial ports # Parallel port device ppc device ppbus # Parallel port bus (required) device lpt # Printer device plip # TCP/IP over parallel device ppi # Parallel port interface device #device vpo # Requires scbus and da # PCI Ethernet NICs. device de # DEC/Intel DC21x4x (``Tulip'') device vx # 3Com 3c590, 3c595 (``Vortex'') device wx # Intel Gigabit Ethernet Card (``Wiseman'') # PCI Ethernet NICs that use the common MII bus controller code. # NOTE: Be sure to keep the 'device miibus' line in order to use these NICs! device miibus # MII bus support device dc # DEC/Intel 21143 and various workalikes device fxp # Intel EtherExpress PRO/100B (82557, 82558) device pcn # AMD Am79C79x PCI 10/100 NICs device rl # RealTek 8129/8139 device sf # Adaptec AIC-6915 (``Starfire'') device sis # Silicon Integrated Systems SiS 900/SiS 7016 device ste # Sundance ST201 (D-Link DFE-550TX) device tl # Texas Instruments ThunderLAN device tx # SMC EtherPower II (83c170 ``EPIC'') device vr # VIA Rhine, Rhine II device wb # Winbond W89C840F device xl # 3Com 3c90x (``Boomerang'', ``Cyclone'') # ISA Ethernet NICs. pccard nics included. device cs # Crystal Semiconductor CS89x0 NIC # 'device ed' requires 'device miibus' device ed # NE[12]000, SMC Ultra, 3c503, DS8390 cards device ex # Intel EtherExpress Pro/10 and Pro/10+ device ep # Etherlink III based cards device fe # Fujitsu MB8696x based cards device sn # SMC's 9000 series of ethernet chips device xe # Xircom pccard ethernet # The probe order of these is presently determined by i386/isa/isa_compat.c. #device ie #device le device lnc # Wireless NIC cards device an # Aironet 4500/4800 802.11 wireless NICs. device awi # BayStack 660 and others device wi # WaveLAN/IEEE 802.11 wireless NICs. #device wl # Older non 802.11 Wavelan wireless NIC. # Pseudo devices - the number indicates how many units to allocate. device random # Entropy device device loop # Network loopback device ether # Ethernet support device sl # Kernel SLIP device ppp 1 # Kernel PPP device tun # Packet tunnel. device pty # Pseudo-ttys (telnet etc) device md # Memory "disks" device gif 4 # IPv6 and IPv4 tunneling device faith 1 # IPv6-to-IPv4 relaying (translation) # The `bpf' device enables the Berkeley Packet Filter. # Be aware of the administrative consequences of enabling this! device bpf # Berkeley packet filter # USB support device uhci # UHCI PCI->USB interface device ohci # OHCI PCI->USB interface device usb # USB Bus (required) #device udbp # USB Double Bulk Pipe devices device ugen # Generic device uhid # "Human Interface Devices" device ukbd # Keyboard device ulpt # Printer device umass # Disks/Mass storage - Requires scbus and da device ums # Mouse device urio # Diamond Rio 500 MP3 player device uscanner # Scanners # USB Ethernet, requires mii device aue # ADMtek USB ethernet device cue # CATC USB ethernet device kue # Kawasaki LSI USB ethernet device sbc options INVARIANTS options INVARIANT_SUPPORT options IPSEC options IPSEC_ESP options DDB options DDB_UNATTENDED options WITNESS options DIAGNOSTIC ichotolot# Script done on Thu Apr 5 01:17:50 2001 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message