Date: Mon, 2 Jul 2007 14:08:17 GMT From: Kirk Russell <kirk@ba23.org> To: freebsd-gnats-submit@FreeBSD.org Subject: kern/114216: aio when using dt panic: lock "aiomtx" 0xc473f1f0 already initialized Message-ID: <200707021408.l62E8HxC005308@www.freebsd.org> Resent-Message-ID: <200707021410.l62EA3uD004343@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 114216 >Category: kern >Synopsis: aio when using dt panic: lock "aiomtx" 0xc473f1f0 already initialized >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Mon Jul 02 14:10:02 GMT 2007 >Closed-Date: >Last-Modified: >Originator: Kirk Russell >Release: 7.0-CURRENT-200706 >Organization: http://www.ba23.org/ >Environment: FreeBSD amd.ott.ba23.org 7.0-CURRENT-200706 FreeBSD 7.0-CURRENT-200706 #0: Sat J un 30 18:48:33 EDT 2007 root@amd.ott.ba23.org:/usr/obj/usr/src/sys/MYKERNEL i386 >Description: I get a panic with a 7.0-CURRENT-200706 when using dt/AIO. amd# kgdb kernel.debug /var/crash/vmcore.5 [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". Unread portion of the kernel message buffer: panic: lock "aiomtx" 0xc473f1f0 already initialized cpuid = 0 Uptime: 7m38s Physical memory: 1139 MB Dumping 51 MB: 36 20 4 #0 doadump () at pcpu.h:172 172 __asm __volatile("movl %%fs:0,%0" : "=r" (td)); (kgdb) bt #0 doadump () at pcpu.h:172 #1 0xc0730a3e in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 #2 0xc0730d43 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:563 #3 0xc0756e24 in lock_init (lock=0x0, class=0xc0ab2ae4, name=0xc47046e7 "aiomtx", type=0x0, flags=131072) at /usr/src/sys/kern/subr_lock.c:201 #4 0xc07258c4 in mtx_init (m=0xc473f1f0, name=0xc47046e7 "aiomtx", type=0x0, opts=0) at /usr/src/sys/kern/kern_mutex.c:714 #5 0xc4701f2e in ?? () #6 0xc473f1f0 in ?? () #7 0xc47046e7 in ?? () #8 0x00000000 in ?? () #9 0x00000000 in ?? () #10 0x00000000 in ?? () #11 0x28202800 in ?? () #12 0x00000000 in ?? () #13 0xe6c49c64 in ?? () #14 0xc470237c in ?? () #15 0xc472a4b8 in ?? () #16 0x00000003 in ?? () #17 0x00000004 in ?? () #18 0x00000002 in ?? () #19 0xe6c49c28 in ?? () #20 0xe6c49c1e in ?? () #21 0x0000000a in ?? () #22 0x00009c24 in ?? () #23 0x02570000 in ?? () #24 0x98c0ee98 in ?? () #25 0x000001ca in ?? () #26 0xda7a0000 in ?? () #27 0x00006400 in ?? () #28 0xc472a4b8 in ?? () #29 0x000018d5 in ?? () #30 0x00000000 in ?? () #31 0xc0ab49c0 in th4 () #32 0xc0ab4980 in th3 () #33 0xe6c49c50 in ?? () #34 0xc09819fe in __udivdi3 (a=Unhandled dwarf expression opcode 0x93 ) at /usr/src/sys/libkern/udivdi3.c:47 Previous frame inner to this frame (corrupt stack?) (kgdb) frame 3 #3 0xc0756e24 in lock_init (lock=0x0, class=0xc0ab2ae4, name=0xc47046e7 "aiomtx", type=0x0, flags=131072) at /usr/src/sys/kern/subr_lock.c:201 201 KASSERT(!lock_initalized(lock), ("lock \"%s\" %p already initialized", (kgdb) print lock $1 = (struct lock_object *) 0x0 Copyright (c) 1992-2007 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.0-CURRENT-200706 #0: Sat Jun 30 18:48:33 EDT 2007 root@amd.ott.ba23.org:/usr/obj/usr/src/sys/MYKERNEL WARNING: WITNESS option enabled, expect reduced performance. Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: AMD Athlon(tm) Processor (1210.79-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x642 Stepping = 2 Features=0x183f9ff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR> AMD Features=0xc0440800<SYSCALL,<b18>,MMX+,3DNow!+,3DNow!> real memory = 1207877632 (1151 MB) avail memory = 1168920576 (1114 MB) kbd1 at kbdmux0 ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) acpi0: <ASUS A7V-133> on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, 47f00000 (3) failed Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0xe408-0xe40b on acpi0 cpu0: <ACPI CPU> on acpi0 acpi_throttle0: <ACPI CPU Throttling> on cpu0 acpi_button0: <Power Button> on acpi0 pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 pci0: <ACPI PCI bus> on pcib0 agp0: <VIA 82C8363 (Apollo KT133x/KM133) host to PCI bridge> on hostb0 pcib1: <PCI-PCI bridge> at device 1.0 on pci0 pci1: <PCI bus> on pcib1 vgapci0: <VGA-compatible display> port 0xd800-0xd8ff mem 0xe0000000-0xe3ffffff,0xd6000000-0xd600ffff irq 11 at device 0.0 on pci1 vgapci1: <VGA-compatible display> mem 0xd8000000-0xdbffffff,0xd5800000-0xd580ffff at device 0.1 on pci1 isab0: <PCI-ISA bridge> at device 4.0 on pci0 isa0: <ISA bus> on isab0 atapci0: <VIA 82C686B UDMA100 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xb800-0xb80f at device 4.1 on pci0 ata0: <ATA channel 0> on atapci0 ata0: [ITHREAD] ata1: <ATA channel 1> on atapci0 ata1: [ITHREAD] uhci0: <VIA 83C572 USB controller> port 0xb000-0xb01f irq 7 at device 4.3 on pci0 uhci0: [GIANT-LOCKED] uhci0: [ITHREAD] usb0: <VIA 83C572 USB controller> on uhci0 usb0: USB revision 1.0 uhub0: <VIA UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0 uhub0: 2 ports with 2 removable, self powered pci0: <bridge> at device 4.4 (no driver attached) pci0: <multimedia, video> at device 9.0 (no driver attached) pci0: <multimedia, audio> at device 10.0 (no driver attached) pci0: <input device> at device 10.1 (no driver attached) ahc0: <Adaptec 2940 Ultra SCSI adapter> port 0x8800-0x88ff mem 0xd5000000-0xd5000fff irq 10 at device 11.0 on pci0 ahc0: [ITHREAD] aic7880: Ultra Wide Channel A, SCSI Id=7, 16/253 SCBs fxp0: <Intel 82559 Pro/100 Ethernet> port 0x8400-0x843f mem 0xd4800000-0xd4800fff,0xd4000000-0xd40fffff irq 11 at device 12.0 on pci0 miibus0: <MII bus> on fxp0 inphy0: <i82555 10/100 media interface> PHY 1 on miibus0 inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto fxp0: Ethernet address: 00:04:ac:d3:7e:2f fxp0: [ITHREAD] ahc1: <Adaptec 2944 Ultra SCSI adapter> port 0x8000-0x80ff mem 0xd3800000-0xd3800fff irq 7 at device 13.0 on pci0 ahc1: [ITHREAD] aic7880: Ultra Wide Channel A, SCSI Id=7, 16/253 SCBs atapci1: <Promise PDC20265 UDMA100 controller> port 0x7800-0x7807,0x7400-0x7403,0x7000-0x7007,0x6800-0x6803,0x6400-0x643f mem 0xd3000000-0xd301ffff irq 10 at device 17.0 on pci0 atapci1: [ITHREAD] ata2: <ATA channel 0> on atapci1 ata2: [ITHREAD] ata3: <ATA channel 1> on atapci1 ata3: [ITHREAD] fdc0: <floppy drive controller> port 0x3f2-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: [FILTER] fd0: <1440-KB 3.5" drive> on fdc0 drive 0 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A sio0: [FILTER] sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0 sio1: type 16550A sio1: [FILTER] atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0 atkbd0: <AT Keyboard> irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] pmtimer0 on isa0 orm0: <ISA Option ROMs> at iomem 0xc0000-0xccfff,0xd0000-0xd47ff,0xd8000-0xd87ff pnpid ORM0000 on isa0 ppc0: parallel port not found. sc0: <System console> at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounter "TSC" frequency 1210794194 Hz quality 800 Timecounters tick every 1.000 msec Waiting 5 seconds for SCSI devices to settle ad0: 8063MB <Maxtor 90845D4 GAS54112> at ata0-master UDMA33 acd0: DVDR <PIONEER DVD-RW DVR-110D/1.17> at ata1-master UDMA66 acd1: DVDROM <MATSHITADVD-ROM SR-8585/1A28> at ata1-slave UDMA33 (probe0:ahc0:0:0:0): inquiry data fails comparison at DV1 step (probe0:ahc0:0:0:0): inquiry data fails comparison at DV1 step (probe0:ahc0:0:0:0): inquiry data fails comparison at DV1 step da0 at ahc0 bus 0 target 0 lun 0 da0: <COMPAQPC WDE9100W 1.01> Fixed Direct Access SCSI-2 device da0: 6.600MB/s transfers (16bit) da0: Command Queueing Enabled da0: 8678MB (17773500 512 byte sectors: 255H 63S/T 1106C) Trying to mount root from ufs:/dev/ad0s1a >How-To-Repeat: This script appears to work okay with 6.0-RELEASE: node116# kldload aio node116# logout $ uname -a FreeBSD node116.on.kr 6.0-RELEASE FreeBSD 6.0-RELEASE #0: Tue Dec 20 21:06:21 ES T 2005 root@node116.on.kr:/usr/src/sys/i386/compile/NODE116 i386 $ /usr/bin/time ./bstg0003.sh 26.38 real 7.99 user 0.89 sys $ echo $? 0 But with 7.0-CURRENT-200706, I get a panic instead. Sometimes, you have to run the script more than once: amd# kldload aio amd# logout $ uname -a FreeBSD amd.ott.ba23.org 7.0-CURRENT-200706 FreeBSD 7.0-CURRENT-200706 #0: Sat J un 30 18:48:33 EDT 2007 root@amd.ott.ba23.org:/usr/obj/usr/src/sys/MYKERNEL i386 $ /usr/bin/time ./bstg0003.sh 78.25 real 7.89 user 2.15 sys $ /usr/bin/time ./bstg0003.sh [... panic ...] #!/bin/sh DT=${DT:-/usr/local/sbin/dt} TMPDIR=${TMPDIR:-/tmp} TEMPFILE=${TMPDIR}/bstg0003.$$.$RANDOM # # See aio(4) about kernel POSIX AIO support. # I have to run "kldload aio" before this test. # # If dt gives similar error messages, then your kernel does not # appear to have AIO support: # dt: Child process NNNN, exiting because of signal 12 # Bad system call (core dumped) # ### kldload aio # # IMHO, dt is in the same league as fsx for finding interesting # OS bugs. # # There is a FreeBSD ports for dt: # http://www.freebsd.org/cgi/url.cgi?ports/sysutils/dt/pkg-descr # # Run dt (similar command line args to dd): # 64 processes will each create a 6m file # will delete the file when finished # output block size of 16k # avoid printing any runtime stats to stdout # using POSIX AIO with 16 queued IOs per process # rm -f $TEMPFILE $DT of=afile bs=16k limit=6m \ disable=stats enable=aio aios=16 procs=64 > $TEMPFILE 2>&1 # # I do not check the return code of dt. # If any text appears on stdout/stderr, then you need to investigate. # if test -s $TEMPFILE ; then # oops -- appears to be a FAIL cat $TEMPFILE rm -f $TEMPFILE exit 1 fi # done -- appears to be a PASS rm -f $TEMPFILE exit 0 >Fix: >Release-Note: >Audit-Trail: >Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200707021408.l62E8HxC005308>