From owner-freebsd-current@FreeBSD.ORG Mon May 18 12:56:14 2015 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7772058A for ; Mon, 18 May 2015 12:56:14 +0000 (UTC) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 520B31387 for ; Mon, 18 May 2015 12:56:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=In-Reply-To:Content-Type:MIME-Version:References:Message-ID:Subject:To:From:Date; bh=QjNhLSQQFqkWNYJ+nPMVw8n78puQx8blZiMljL3LrrU=; b=Ar53Go7Sm/9xhviq0KcWjzV0eDClndvpJvk08ldqs4IFPuzNYYIsE5Vzl3Ievm5OVQVpbc9p6rrmlIMX9NuEHd+5EN4Uskuq/1eK4n1LvC4jKJ0GI9Tr+EfJmyWkoaAjVQlgYrjyMZQERZF9tnYFECpF4fBCuyCfWF0rVopwjGM=; Received: from 104-54-221-134.lightspeed.austtx.sbcglobal.net ([104.54.221.134]:56534 helo=borg.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (Exim 4.85 (FreeBSD)) (envelope-from ) id 1YuKab-0009gF-9C for freebsd-current@freebsd.org; Mon, 18 May 2015 07:56:13 -0500 Date: Mon, 18 May 2015 07:56:01 -0500 From: Larry Rosenman To: freebsd-current@freebsd.org Subject: Re: use after free panic ZFS Message-ID: <20150518125600.GA1274@borg.lerctr.org> References: <20150518124247.GA6220@borg.lerctr.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150518124247.GA6220@borg.lerctr.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Score: -1.0 (-) X-LERCTR-Spam-Score: -1.0 (-) X-Spam-Report: SpamScore (-1.0/5.0) ALL_TRUSTED=-1,SHORTCIRCUIT=-0.0001 X-LERCTR-Spam-Report: SpamScore (-1.0/5.0) ALL_TRUSTED=-1, SHORTCIRCUIT=-0.0001 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 May 2015 12:56:14 -0000 On Mon, May 18, 2015 at 07:42:47AM -0500, Larry Rosenman wrote: > found the following panic this am: > > borg.lerctr.org dumped core - see /var/crash/vmcore.5 > > Sun May 17 23:47:48 CDT 2015 > > FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #40 r283007: Sat May 16 07:23:43 CDT 2015 root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 > > panic: Most recently used by solaris > > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"... > > Unread portion of the kernel message buffer: > Memory modified after free 0xfffff808535ea000(120) val=deadc0dd @ 0xfffff808535ea050 > panic: Most recently used by solaris > > cpuid = 5 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe100bfb7660 > vpanic() at vpanic+0x189/frame 0xfffffe100bfb76e0 > panic() at panic+0x43/frame 0xfffffe100bfb7740 > mtrash_dtor() at mtrash_dtor/frame 0xfffffe100bfb7760 > uma_zalloc_arg() at uma_zalloc_arg+0x4c2/frame 0xfffffe100bfb77d0 > malloc() at malloc+0x198/frame 0xfffffe100bfb7820 > zfs_range_lock() at zfs_range_lock+0x4a/frame 0xfffffe100bfb7880 > zfs_get_data() at zfs_get_data+0x14c/frame 0xfffffe100bfb78f0 > zil_commit() at zil_commit+0x94c/frame 0xfffffe100bfb7a10 > zfs_freebsd_fsync() at zfs_freebsd_fsync+0xc8/frame 0xfffffe100bfb7a40 > VOP_FSYNC_APV() at VOP_FSYNC_APV+0xf7/frame 0xfffffe100bfb7a70 > sys_fsync() at sys_fsync+0x173/frame 0xfffffe100bfb7ae0 > amd64_syscall() at amd64_syscall+0x25a/frame 0xfffffe100bfb7bf0 > Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe100bfb7bf0 > --- syscall (95, FreeBSD ELF64, sys_fsync), rip = 0x801eb5daa, rsp = 0x7fffffffd598, rbp = 0x7fffffffd5b0 --- > Uptime: 1d14h25m26s > Dumping 12469 out of 64457 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% > > Reading symbols from /boot/kernel/linux.ko.symbols...done. > Loaded symbols for /boot/kernel/linux.ko.symbols > Reading symbols from /boot/kernel/if_lagg.ko.symbols...done. > Loaded symbols for /boot/kernel/if_lagg.ko.symbols > Reading symbols from /boot/kernel/snd_envy24ht.ko.symbols...done. > Loaded symbols for /boot/kernel/snd_envy24ht.ko.symbols > Reading symbols from /boot/kernel/snd_spicds.ko.symbols...done. > Loaded symbols for /boot/kernel/snd_spicds.ko.symbols > Reading symbols from /boot/kernel/coretemp.ko.symbols...done. > Loaded symbols for /boot/kernel/coretemp.ko.symbols > Reading symbols from /boot/kernel/ichsmb.ko.symbols...done. > Loaded symbols for /boot/kernel/ichsmb.ko.symbols > Reading symbols from /boot/kernel/smbus.ko.symbols...done. > Loaded symbols for /boot/kernel/smbus.ko.symbols > Reading symbols from /boot/kernel/ichwd.ko.symbols...done. > Loaded symbols for /boot/kernel/ichwd.ko.symbols > Reading symbols from /boot/kernel/cpuctl.ko.symbols...done. > Loaded symbols for /boot/kernel/cpuctl.ko.symbols > Reading symbols from /boot/kernel/crypto.ko.symbols...done. > Loaded symbols for /boot/kernel/crypto.ko.symbols > Reading symbols from /boot/kernel/cryptodev.ko.symbols...done. > Loaded symbols for /boot/kernel/cryptodev.ko.symbols > Reading symbols from /boot/kernel/dtraceall.ko.symbols...done. > Loaded symbols for /boot/kernel/dtraceall.ko.symbols > Reading symbols from /boot/kernel/profile.ko.symbols...done. > Loaded symbols for /boot/kernel/profile.ko.symbols > Reading symbols from /boot/kernel/dtrace.ko.symbols...done. > Loaded symbols for /boot/kernel/dtrace.ko.symbols > Reading symbols from /boot/kernel/systrace_freebsd32.ko.symbols...done. > Loaded symbols for /boot/kernel/systrace_freebsd32.ko.symbols > Reading symbols from /boot/kernel/systrace.ko.symbols...done. > Loaded symbols for /boot/kernel/systrace.ko.symbols > Reading symbols from /boot/kernel/sdt.ko.symbols...done. > Loaded symbols for /boot/kernel/sdt.ko.symbols > Reading symbols from /boot/kernel/lockstat.ko.symbols...done. > Loaded symbols for /boot/kernel/lockstat.ko.symbols > Reading symbols from /boot/kernel/fasttrap.ko.symbols...done. > Loaded symbols for /boot/kernel/fasttrap.ko.symbols > Reading symbols from /boot/kernel/fbt.ko.symbols...done. > Loaded symbols for /boot/kernel/fbt.ko.symbols > Reading symbols from /boot/kernel/dtnfscl.ko.symbols...done. > Loaded symbols for /boot/kernel/dtnfscl.ko.symbols > Reading symbols from /boot/kernel/dtmalloc.ko.symbols...done. > Loaded symbols for /boot/kernel/dtmalloc.ko.symbols > Reading symbols from /boot/modules/vboxdrv.ko...done. > Loaded symbols for /boot/modules/vboxdrv.ko > Reading symbols from /boot/modules/nvidia.ko...done. > Loaded symbols for /boot/modules/nvidia.ko > Reading symbols from /boot/kernel/ipmi.ko.symbols...done. > Loaded symbols for /boot/kernel/ipmi.ko.symbols > Reading symbols from /boot/kernel/ipmi_linux.ko.symbols...done. > Loaded symbols for /boot/kernel/ipmi_linux.ko.symbols > Reading symbols from /boot/kernel/radeonkms.ko.symbols...done. > Loaded symbols for /boot/kernel/radeonkms.ko.symbols > Reading symbols from /boot/kernel/iicbb.ko.symbols...done. > Loaded symbols for /boot/kernel/iicbb.ko.symbols > Reading symbols from /boot/kernel/iicbus.ko.symbols...done. > Loaded symbols for /boot/kernel/iicbus.ko.symbols > Reading symbols from /boot/kernel/iic.ko.symbols...done. > Loaded symbols for /boot/kernel/iic.ko.symbols > Reading symbols from /boot/kernel/drm2.ko.symbols...done. > Loaded symbols for /boot/kernel/drm2.ko.symbols > Reading symbols from /boot/kernel/radeonkmsfw_R100_cp.ko.symbols...done. > Loaded symbols for /boot/kernel/radeonkmsfw_R100_cp.ko.symbols > Reading symbols from /boot/kernel/uhid.ko.symbols...done. > Loaded symbols for /boot/kernel/uhid.ko.symbols > Reading symbols from /boot/kernel/ums.ko.symbols...done. > Loaded symbols for /boot/kernel/ums.ko.symbols > Reading symbols from /boot/modules/vboxnetflt.ko...done. > Loaded symbols for /boot/modules/vboxnetflt.ko > Reading symbols from /boot/kernel/netgraph.ko.symbols...done. > Loaded symbols for /boot/kernel/netgraph.ko.symbols > Reading symbols from /boot/kernel/ng_ether.ko.symbols...done. > Loaded symbols for /boot/kernel/ng_ether.ko.symbols > Reading symbols from /boot/modules/vboxnetadp.ko...done. > Loaded symbols for /boot/modules/vboxnetadp.ko > #0 doadump (textdump=Unhandled dwarf expression opcode 0x93 > ) at pcpu.h:221 > 221 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) #0 doadump (textdump=Unhandled dwarf expression opcode 0x93 > ) at pcpu.h:221 > #1 0xffffffff80a839b5 in kern_reboot (howto=Unhandled dwarf expression opcode 0x93 > ) > at /usr/src/sys/kern/kern_shutdown.c:447 > #2 0xffffffff80a83fa8 in vpanic (fmt=, > ap=) at /usr/src/sys/kern/kern_shutdown.c:744 > #3 0xffffffff80a83ff3 in panic (fmt=0x0) > at /usr/src/sys/kern/kern_shutdown.c:675 > #4 0xffffffff80d13750 in mtrash_ctor (mem=, > size=, arg=, > flags=) at /usr/src/sys/vm/uma_dbg.c:138 > #5 0xffffffff80d0f6d2 in uma_zalloc_arg (zone=0xfffff80ffffc9680, udata=0x0, > flags=2) at /usr/src/sys/vm/uma_core.c:2197 > #6 0xffffffff80a64158 in malloc (size=, > mtp=0xffffffff815e16e0, flags=) at uma.h:336 > #7 0xffffffff80402b4a in zfs_range_lock (zp=0xfffff8075e835730, off=9158656, > len=8192, type=Unhandled dwarf expression opcode 0x93 > ) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_rlock.c:432 > #8 0xffffffff8040886c in zfs_get_data (arg=, > lr=, > buf=0xfffffe0662be8178
, > zio=0xfffff80d78b89ac8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1250 > #9 0xffffffff8041c71c in zil_commit (zilog=0xfffff800185c1400, > foid=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c:1108 > #10 0xffffffff80410168 in zfs_freebsd_fsync (ap=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:2747 > #11 0xffffffff80fdfcd7 in VOP_FSYNC_APV (vop=, > a=) at vnode_if.c:1328 > #12 0xffffffff80b40883 in sys_fsync (td=0xfffff8011b253940, > uap=) at vnode_if.h:549 > #13 0xffffffff80e968da in amd64_syscall (td=0xfffff8011b253940, traced=0) > at subr_syscall.c:133 > #14 0xffffffff80e767bb in Xfast_syscall () > at /usr/src/sys/amd64/amd64/exception.S:395 > #15 0x0000000801eb5daa in ?? () > Previous frame inner to this frame (corrupt stack?) > Current language: auto; currently minimal > (kgdb) > > I have the core. And, trying to re-compile to pick up the latest, got another one: borg.lerctr.org dumped core - see /var/crash/vmcore.6 Mon May 18 07:51:57 CDT 2015 FreeBSD borg.lerctr.org 11.0-CURRENT FreeBSD 11.0-CURRENT #40 r283007: Sat May 16 07:23:43 CDT 2015 root@borg.lerctr.org:/usr/obj/usr/src/sys/VT-LER amd64 panic: Most recently used by solaris GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: Memory modified after free 0xfffff80c88f1f980(120) val=deadc0dd @ 0xfffff80c88f1f9c0 panic: Most recently used by solaris cpuid = 0 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe100c459600 vpanic() at vpanic+0x189/frame 0xfffffe100c459680 panic() at panic+0x43/frame 0xfffffe100c4596e0 mtrash_dtor() at mtrash_dtor/frame 0xfffffe100c459700 uma_zalloc_arg() at uma_zalloc_arg+0x4c2/frame 0xfffffe100c459770 malloc() at malloc+0x198/frame 0xfffffe100c4597c0 zfs_range_lock() at zfs_range_lock+0x4a/frame 0xfffffe100c459820 zfs_freebsd_read() at zfs_freebsd_read+0x1c7/frame 0xfffffe100c4598c0 VOP_READ_APV() at VOP_READ_APV+0xf1/frame 0xfffffe100c4598f0 vn_read() at vn_read+0x237/frame 0xfffffe100c459970 vn_io_fault() at vn_io_fault+0x10a/frame 0xfffffe100c4599f0 dofileread() at dofileread+0x95/frame 0xfffffe100c459a40 kern_readv() at kern_readv+0x68/frame 0xfffffe100c459a90 sys_read() at sys_read+0x63/frame 0xfffffe100c459ae0 amd64_syscall() at amd64_syscall+0x25a/frame 0xfffffe100c459bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xfffffe100c459bf0 --- syscall (3, FreeBSD ELF64, sys_read), rip = 0x8009638fa, rsp = 0x7fffffffe968, rbp = 0x7fffffffe980 --- Uptime: 7h59m25s Dumping 14815 out of 64457 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Reading symbols from /boot/kernel/linux.ko.symbols...done. Loaded symbols for /boot/kernel/linux.ko.symbols Reading symbols from /boot/kernel/if_lagg.ko.symbols...done. Loaded symbols for /boot/kernel/if_lagg.ko.symbols Reading symbols from /boot/kernel/snd_envy24ht.ko.symbols...done. Loaded symbols for /boot/kernel/snd_envy24ht.ko.symbols Reading symbols from /boot/kernel/snd_spicds.ko.symbols...done. Loaded symbols for /boot/kernel/snd_spicds.ko.symbols Reading symbols from /boot/kernel/coretemp.ko.symbols...done. Loaded symbols for /boot/kernel/coretemp.ko.symbols Reading symbols from /boot/kernel/ichsmb.ko.symbols...done. Loaded symbols for /boot/kernel/ichsmb.ko.symbols Reading symbols from /boot/kernel/smbus.ko.symbols...done. Loaded symbols for /boot/kernel/smbus.ko.symbols Reading symbols from /boot/kernel/ichwd.ko.symbols...done. Loaded symbols for /boot/kernel/ichwd.ko.symbols Reading symbols from /boot/kernel/cpuctl.ko.symbols...done. Loaded symbols for /boot/kernel/cpuctl.ko.symbols Reading symbols from /boot/kernel/crypto.ko.symbols...done. Loaded symbols for /boot/kernel/crypto.ko.symbols Reading symbols from /boot/kernel/cryptodev.ko.symbols...done. Loaded symbols for /boot/kernel/cryptodev.ko.symbols Reading symbols from /boot/kernel/dtraceall.ko.symbols...done. Loaded symbols for /boot/kernel/dtraceall.ko.symbols Reading symbols from /boot/kernel/profile.ko.symbols...done. Loaded symbols for /boot/kernel/profile.ko.symbols Reading symbols from /boot/kernel/dtrace.ko.symbols...done. Loaded symbols for /boot/kernel/dtrace.ko.symbols Reading symbols from /boot/kernel/systrace_freebsd32.ko.symbols...done. Loaded symbols for /boot/kernel/systrace_freebsd32.ko.symbols Reading symbols from /boot/kernel/systrace.ko.symbols...done. Loaded symbols for /boot/kernel/systrace.ko.symbols Reading symbols from /boot/kernel/sdt.ko.symbols...done. Loaded symbols for /boot/kernel/sdt.ko.symbols Reading symbols from /boot/kernel/lockstat.ko.symbols...done. Loaded symbols for /boot/kernel/lockstat.ko.symbols Reading symbols from /boot/kernel/fasttrap.ko.symbols...done. Loaded symbols for /boot/kernel/fasttrap.ko.symbols Reading symbols from /boot/kernel/fbt.ko.symbols...done. Loaded symbols for /boot/kernel/fbt.ko.symbols Reading symbols from /boot/kernel/dtnfscl.ko.symbols...done. Loaded symbols for /boot/kernel/dtnfscl.ko.symbols Reading symbols from /boot/kernel/dtmalloc.ko.symbols...done. Loaded symbols for /boot/kernel/dtmalloc.ko.symbols Reading symbols from /boot/modules/vboxdrv.ko...done. Loaded symbols for /boot/modules/vboxdrv.ko Reading symbols from /boot/modules/nvidia.ko...done. Loaded symbols for /boot/modules/nvidia.ko Reading symbols from /boot/kernel/ipmi.ko.symbols...done. Loaded symbols for /boot/kernel/ipmi.ko.symbols Reading symbols from /boot/kernel/ipmi_linux.ko.symbols...done. Loaded symbols for /boot/kernel/ipmi_linux.ko.symbols Reading symbols from /boot/kernel/radeonkms.ko.symbols...done. Loaded symbols for /boot/kernel/radeonkms.ko.symbols Reading symbols from /boot/kernel/iicbb.ko.symbols...done. Loaded symbols for /boot/kernel/iicbb.ko.symbols Reading symbols from /boot/kernel/iicbus.ko.symbols...done. Loaded symbols for /boot/kernel/iicbus.ko.symbols Reading symbols from /boot/kernel/iic.ko.symbols...done. Loaded symbols for /boot/kernel/iic.ko.symbols Reading symbols from /boot/kernel/drm2.ko.symbols...done. Loaded symbols for /boot/kernel/drm2.ko.symbols Reading symbols from /boot/kernel/radeonkmsfw_R100_cp.ko.symbols...done. Loaded symbols for /boot/kernel/radeonkmsfw_R100_cp.ko.symbols Reading symbols from /boot/kernel/uhid.ko.symbols...done. Loaded symbols for /boot/kernel/uhid.ko.symbols Reading symbols from /boot/kernel/ums.ko.symbols...done. Loaded symbols for /boot/kernel/ums.ko.symbols Reading symbols from /boot/modules/vboxnetflt.ko...done. Loaded symbols for /boot/modules/vboxnetflt.ko Reading symbols from /boot/kernel/netgraph.ko.symbols...done. Loaded symbols for /boot/kernel/netgraph.ko.symbols Reading symbols from /boot/kernel/ng_ether.ko.symbols...done. Loaded symbols for /boot/kernel/ng_ether.ko.symbols Reading symbols from /boot/modules/vboxnetadp.ko...done. Loaded symbols for /boot/modules/vboxnetadp.ko #0 doadump (textdump=Unhandled dwarf expression opcode 0x93 ) at pcpu.h:221 221 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump (textdump=Unhandled dwarf expression opcode 0x93 ) at pcpu.h:221 #1 0xffffffff80a839b5 in kern_reboot (howto=Unhandled dwarf expression opcode 0x93 ) at /usr/src/sys/kern/kern_shutdown.c:447 #2 0xffffffff80a83fa8 in vpanic (fmt=, ap=) at /usr/src/sys/kern/kern_shutdown.c:744 #3 0xffffffff80a83ff3 in panic (fmt=0x0) at /usr/src/sys/kern/kern_shutdown.c:675 #4 0xffffffff80d13750 in mtrash_ctor (mem=, size=, arg=, flags=) at /usr/src/sys/vm/uma_dbg.c:138 #5 0xffffffff80d0f6d2 in uma_zalloc_arg (zone=0xfffff80ffffc9680, udata=0x0, flags=2) at /usr/src/sys/vm/uma_core.c:2197 #6 0xffffffff80a64158 in malloc (size=, mtp=0xffffffff815e16e0, flags=) at uma.h:336 #7 0xffffffff80402b4a in zfs_range_lock (zp=0xfffff806afc8d170, off=75316383, len=131072, type=Unhandled dwarf expression opcode 0x93 ) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_rlock.c:432 #8 0xffffffff8040e517 in zfs_freebsd_read (ap=) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:703 #9 0xffffffff80fdf3b1 in VOP_READ_APV (vop=, a=) at vnode_if.c:930 #10 0xffffffff80b461b7 in vn_read (fp=0xfffff8002e1389b0, uio=0xfffffe100c459ab0, active_cred=, flags=, td=0x0) at vnode_if.h:384 #11 0xffffffff80b425ea in vn_io_fault (fp=0xfffff8002e1389b0, uio=0xfffffe100c459ab0, active_cred=0x0, flags=0, td=0x0) at /usr/src/sys/kern/vfs_vnops.c:1167 #12 0xffffffff80ae1525 in dofileread (td=0xfffff804bb5f8940, fd=3, fp=0xfffff8002e1389b0, auio=0xfffffe100c459ab0, offset=, flags=Unhandled dwarf expression opcode 0x93 ) at file.h:296 #13 0xffffffff80ae1228 in kern_readv (td=0xfffff804bb5f8940, fd=Unhandled dwarf expression opcode 0x93 ) at /usr/src/sys/kern/sys_generic.c:272 #14 0xffffffff80ae11b3 in sys_read (td=0x0, uap=) at /usr/src/sys/kern/sys_generic.c:185 #15 0xffffffff80e968da in amd64_syscall (td=0xfffff804bb5f8940, traced=0) at subr_syscall.c:133 #16 0xffffffff80e767bb in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:395 #17 0x00000008009638fa in ?? () Previous frame inner to this frame (corrupt stack?) Current language: auto; currently minimal (kgdb) I have BOTH cores. -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 108 Turvey Cove, Hutto, TX 78634-5688