From owner-freebsd-current@FreeBSD.ORG Tue Oct 23 19:28:25 2007 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BC2FB16A417 for ; Tue, 23 Oct 2007 19:28:25 +0000 (UTC) (envelope-from ohauer@gmx.de) Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by mx1.freebsd.org (Postfix) with SMTP id 24A4013C4BE for ; Tue, 23 Oct 2007 19:28:24 +0000 (UTC) (envelope-from ohauer@gmx.de) Received: (qmail invoked by alias); 23 Oct 2007 19:28:12 -0000 Received: from u18-124.dsl.vianetworks.de (EHLO [172.20.1.30]) [194.231.39.124] by mail.gmx.net (mp054) with SMTP; 23 Oct 2007 21:28:12 +0200 X-Authenticated: #1956535 X-Provags-ID: V01U2FsdGVkX1/Xsvo5mzTX8kSTUckO0xLdko1kVEdOp8Y6Io2H+G fgVg+ptQTLkHdX Message-ID: <471E4B3C.3060807@gmx.de> Date: Tue, 23 Oct 2007 21:27:56 +0200 From: Olli Hauer User-Agent: Thunderbird 2.0.0.6 (Windows/20070728) MIME-Version: 1.0 To: freebsd-current@freebsd.org References: <471BCF96.7000207@gmx.de> <20071022184033.GC1118@garage.freebsd.pl> In-Reply-To: <20071022184033.GC1118@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 Subject: Re: zfs kernel messages X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Oct 2007 19:28:25 -0000 Pawel Jakub Dawidek wrote: > On Mon, Oct 22, 2007 at 12:15:50AM +0200, Olli Hauer wrote: >> while playing a little with zfs i get the following >> kernel messages at ttyv0 (no panic/freeze happend) >> >> (captured vi ssh from other system) >> # vidcontrol -P < /dev/ttyv0 >> >> uma_zalloc_arg: zone "256" with the following non-sleepable locks held: >> exclusive sleep mutex struct mount mtx r = 0 (0xc64e7d3c) locked @ >> /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_vfs.c:206 >> KDB: stack backtrace: >> db_trace_self_wrapper(c0a9c175,e72bb6e4,c078428d,c0a9c538,e72bb6f8,...) at >> db_trace_self_wrapper+0x26 >> kdb_backtrace(c0a9c538,e72bb6f8,4,1,0,...) at kdb_backtrace+0x29 >> witness_warn(5,0,c0aba40d,c0ac3e77,1,...) at witness_warn+0x1cd >> uma_zalloc_arg(c146d1e0,0,102,2,c64e7d0c,...) at uma_zalloc_arg+0x34 >> malloc(94,c0b4c4e0,102,c64e7d0c,e72bb790,...) at malloc+0xd2 >> crget(c6985300,c0b4c4e0,c64e7d0c,e72bb7d4,c431b9dc,...) at crget+0x23 >> crdup(c3f07600,0,c439444c,ce,c2,...) at crdup+0xc >> domount(c450a420,c6ee4330,c43995db,cc152900,e72bb810,...) at domount+0x20c >> zfsctl_snapdir_lookup(e72bbaa0,e72bbaa0,c450a420,2,c456e660,...) at >> zfsctl_snapdir_lookup+0x362 >> VOP_LOOKUP_APV(c439d5c0,e72bbaa0,c450a420,c0aa409d,19b,...) at >> VOP_LOOKUP_APV+0xa5 >> lookup(e72bbb48,c0aa409d,c6,bf,c730102c,...) at lookup+0x58e >> namei(e72bbb48,e72bbb94,60,0,c450a420,...) at namei+0x34b >> kern_lstat(c450a420,28220318,0,e72bbc18,2d4738cb,...) at kern_lstat+0x4f >> lstat(c450a420,e72bbcfc,8,c0a9ed42,c0b469d0,...) at lstat+0x2f >> syscall(e72bbd38) at syscall+0x2b3 >> Xint0x80_syscall() at Xint0x80_syscall+0x20 >> --- syscall (190, FreeBSD ELF32, lstat), eip = 0x2815548b, esp = >> 0xbfbfe93c, ebp = 0xbfbfe9c8 --- > > That's easy to fix. Try this patch: > > http://people.freebsd.org/~pjd/patches/opensolaris_vfs.c.patch > > Thanks for the report. > Sorry for the late responce. Patch applied, rebuild/installed kernel, reboot same test output is below. Let me know If you need more information or if you have new instructions. ***** after this test I wait 5 min then try to reboot the system ***** Syncing disks, vnodes remaining...2 2 2 2 0 0 done All buffers synced. ***** system hang, wait 10 minutes than push the power button once ***** Shutting down demon process:. Shutting down local daemons:. Writing entropy file:. Oct 23.... init: some process would not die; ps axl advised Waiting (max60 seconds) for system process `vnlu` to stop... timed out Waiting (max60 seconds) for system process `bufdaemon` to stop... timed out Waiting (max60 seconds) for system process `syncer` to stop... timed out ... ***** system power off ***** ***** start system again (see dmesg) ***** ... ***** output of the ls test ***** # vidcontrol -P < /dev/ttyv0 lock order reversal: 1st 0xc4cea568 struct mount mtx (struct mount mtx) @ /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_vfs.c:209 2nd 0xc3ee9010 sleep mtxpool (sleep mtxpool) @ /usr/src/sys/kern/kern_resource.c:1266 KDB: stack backtrace: db_trace_self_wrapper(c0a9c175,e7318734,c078510e,c0a9e63c,c3ee9010,...) at db_trace_self_wrapper+0x26 kdb_backtrace(c0a9e63c,c3ee9010,c0a982df,c0a982df,c0a98e54,...) at kdb_backtrace+0x29 witness_checkorder(c3ee9010,9,c0a98e54,4f2,38,...) at witness_checkorder+0x6de _mtx_lock_flags(c3ee9010,0,c0a98e54,4f2,c4c32d00,...) at _mtx_lock_flags+0xbc uifree(c3f08c20,c4967220,c4cea538,e73187d4,c431b9df,...) at uifree+0x2d crfree(c4c32d00,0,c439446c,d1,c3,...) at crfree+0x54 domount(c48f8840,c4967220,c43995fb,c469e260,e7318810,...) at domount+0x20f zfsctl_snapdir_lookup(e7318aa0,e7318aa0,c48f8840,2,c4967330,...) at zfsctl_snapdir_lookup+0x362 VOP_LOOKUP_APV(c439d5e0,e7318aa0,c48f8840,c0aa409d,19b,...) at VOP_LOOKUP_APV+0xa5 lookup(e7318b48,c0aa409d,c6,bf,c4cd772c,...) at lookup+0x58e namei(e7318b48,c48f8840,c0bb51d4,c48f8840,e7318b4c,...) at namei+0x34b kern_lstat(c48f8840,282111b8,0,e7318c18,c0aa5520,...) at kern_lstat+0x4f lstat(c48f8840,e7318cfc,8,c0a9f13d,c0b469d0,...) at lstat+0x2f syscall(e7318d38) at syscall+0x2b3 Xint0x80_syscall() at Xint0x80_syscall+0x20 --- syscall (190, FreeBSD ELF32, lstat), eip = 0x2819c48b, esp = 0xbfbfe57c, ebp = 0xbfbfe608 --- ***** output of zpool test ***** # zpool status pool: tank state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror ONLINE 0 0 0 twed0 ONLINE 0 0 0 twed1 ONLINE 0 0 0 errors: No known data errors # zpool scrub tank # zpool status pool: tank state: ONLINE scrub: scrub completed with 0 errors on Tue Oct 23 21:15:42 2007 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 mirror ONLINE 0 0 0 twed0 ONLINE 0 0 0 twed1 ONLINE 0 0 0 errors: No known data errors # uname -a FreeBSD amd.uni.vrs 7.0-CURRENT-200710 FreeBSD 7.0-CURRENT-200710 #0: Tue Oct 23 20:12:59 CEST 2007 root@amd.uni.vrs:/usr/obj/usr/src/sys/GENERIC i386 # dmesg Copyright (c) 1992-2007 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.0-CURRENT-200710 #0: Tue Oct 23 20:12:59 CEST 2007 root@amd.uni.vrs:/usr/obj/usr/src/sys/GENERIC WARNING: WITNESS option enabled, expect reduced performance. Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: AMD Sempron(tm) 2300+ (1580.46-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x681 Stepping = 1 Features=0x383fbff AMD Features=0xc0480800 real memory = 1073676288 (1023 MB) avail memory = 1037058048 (989 MB) mptable_probe: MP Config Table has bad signature: \^O8\^O kbd1 at kbdmux0 ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, 3fef0000 (3) failed Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x4008-0x400b on acpi0 cpu0: on acpi0 acpi_button0: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 Correcting nForce2 C1 CPU disconnect hangs agp0: on hostb0 pci0: at device 0.1 (no driver attached) pci0: at device 0.2 (no driver attached) pci0: at device 0.3 (no driver attached) pci0: at device 0.4 (no driver attached) pci0: at device 0.5 (no driver attached) isab0: at device 1.0 on pci0 isa0: on isab0 pci0: at device 1.1 (no driver attached) ohci0: mem 0xe0001000-0xe0001fff irq 3 at device 2.0 on pci0 ohci0: [GIANT-LOCKED] ohci0: [ITHREAD] usb0: OHCI version 1.0, legacy support usb0: on ohci0 usb0: USB revision 1.0 uhub0: on usb0 uhub0: 3 ports with 3 removable, self powered ohci1: mem 0xe0002000-0xe0002fff irq 5 at device 2.1 on pci0 ohci1: [GIANT-LOCKED] ohci1: [ITHREAD] usb1: OHCI version 1.0, legacy support usb1: on ohci1 usb1: USB revision 1.0 uhub1: on usb1 uhub1: 3 ports with 3 removable, self powered ehci0: mem 0xe0003000-0xe00030ff irq 11 at device 2.2 on pci0 ehci0: [GIANT-LOCKED] ehci0: [ITHREAD] usb2: EHCI version 1.0 usb2: companion controllers, 4 ports each: usb0 usb1 usb2: on ehci0 usb2: USB revision 2.0 uhub2: on usb2 uhub2: 6 ports with 6 removable, self powered pci0: at device 6.0 (no driver attached) pcib1: at device 8.0 on pci0 pci1: on pcib1 twe0: <3ware Storage Controller. Driver version 1.50.01.002> port 0xd000-0xd00f mem 0xdf800000-0xdf80000f,0xdf000000-0xdf7fffff irq 11 at device 6.0 on pci1 twe0: [GIANT-LOCKED] twe0: [ITHREAD] twe0: 4 ports, Firmware FE7X 1.05.00.068, BIOS BE7X 1.08.00.048 xl0: <3Com 3c905C-TX Fast Etherlink XL> port 0xd400-0xd47f mem 0xdf801000-0xdf80107f irq 5 at device 7.0 on pci1 miibus0: on xl0 ukphy0: PHY 24 on miibus0 ukphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto xl0: Ethernet address: 00:04:75:bf:81:11 xl0: [ITHREAD] atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xf000-0xf00f at device 9.0 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] pcib2: at device 30.0 on pci0 pci2: on pcib2 vgapci0: mem 0xdc000000-0xdcffffff,0xd0000000-0xd7ffffff irq 5 at device 0.0 on pci2 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A sio0: [FILTER] atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model Generic PS/2 mouse, device ID 0 pmtimer0 on isa0 orm0: at iomem 0xc0000-0xca7ff,0xcc000-0xccfff,0xcd000-0xcd7ff pnpid ORM0000 on isa0 ppc0: at port 0x378-0x37f irq 7 on isa0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/16 bytes threshold ppbus0: on ppc0 ppi0: on ppbus0 plip0: on ppbus0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppc0: [GIANT-LOCKED] ppc0: [ITHREAD] sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sio1: configured irq 3 not in bitmap of probed irqs 0 sio1: port may not be enabled vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounter "TSC" frequency 1580462894 Hz quality 800 Timecounters tick every 1.000 msec ad0: 305245MB at ata0-master UDMA133 ad1: 78167MB at ata0-slave UDMA133 ad2: 238475MB at ata1-master UDMA133 acd0: DVDR at ata1-slave UDMA33 twed0: on twe0 twed0: 305245MB (625142448 sectors) twed1: on twe0 twed1: 305245MB (625142448 sectors) WARNING: WITNESS option enabled, expect reduced performance. Trying to mount root from ufs:/dev/ad0s1a WARNING: / was not properly dismounted WARNING: /tmp was not properly dismounted WARNING: /usr was not properly dismounted WARNING: /var was not properly dismounted WARNING: ZFS is considered to be an experimental feature in FreeBSD. ZFS filesystem version 6 ZFS storage pool version 6