From owner-freebsd-current@FreeBSD.ORG Wed Nov 10 05:58:16 2010 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D05D81065672 for ; Wed, 10 Nov 2010 05:58:16 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 1E22A8FC14 for ; Wed, 10 Nov 2010 05:58:15 +0000 (UTC) Received: from porto.topspin.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id HAA10014 for ; Wed, 10 Nov 2010 07:58:14 +0200 (EET) (envelope-from avg@freebsd.org) Received: from localhost.topspin.kiev.ua ([127.0.0.1]) by porto.topspin.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1PG3hG-000AeV-9B for freebsd-current@FreeBSD.ORG; Wed, 10 Nov 2010 07:58:14 +0200 Message-ID: <4CDA3475.3060101@freebsd.org> Date: Wed, 10 Nov 2010 07:58:13 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.12) Gecko/20101029 Lightning/1.0b2 Thunderbird/3.1.6 MIME-Version: 1.0 To: freebsd-current@freebsd.org X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=X-VIET-VPS Content-Transfer-Encoding: 7bit Subject: panic: uma_zalloc_arg: Returning an empty bucket. X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Nov 2010 05:58:16 -0000 I've been doing some stress-testing for VM+ZFS on a kernel with WITNESS+INVARIANTS and I have already stumbled into some curious panics. Here's another one: Unread portion of the kernel message buffer: panic: uma_zalloc_arg: Returning an empty bucket. cpuid = 0 KDB: stack backtrace: db_trace_self_wrapper() at 0xffffffff801b8b3a = db_trace_self_wrapper+0x2a kdb_backtrace() at 0xffffffff803a7a8a = kdb_backtrace+0x3a panic() at 0xffffffff80370632 = panic+0x1d2 uma_zalloc_arg() at 0xffffffff80507850 = uma_zalloc_arg+0x300 kmem_cache_alloc() at 0xffffffff80dac385 = kmem_cache_alloc+0x15 arc_buf_alloc() at 0xffffffff80c016e4 = arc_buf_alloc+0x34 arc_read_nolock() at 0xffffffff80c0389d = arc_read_nolock+0x2bd arc_read() at 0xffffffff80c03dff = arc_read+0x7f dbuf_prefetch() at 0xffffffff80c0826a = dbuf_prefetch+0x15a dmu_zfetch_dofetch() at 0xffffffff80c21444 = dmu_zfetch_dofetch+0x104 dmu_zfetch() at 0xffffffff80c21c89 = dmu_zfetch+0x5e9 dbuf_read() at 0xffffffff80c06c04 = dbuf_read+0x454 dmu_buf_hold_array_by_dnode() at 0xffffffff80c0a58d = dmu_buf_hold_array_by_dnode+0x27d dmu_read() at 0xffffffff80c0a80f = dmu_read+0xbf zfs_freebsd_getpages() at 0xffffffff80c6c9b5 = zfs_freebsd_getpages+0x8c5 VOP_GETPAGES_APV() at 0xffffffff805a6618 = VOP_GETPAGES_APV+0xe8 vnode_pager_getpages() at 0xffffffff80523c09 = vnode_pager_getpages+0xb9 vm_fault() at 0xffffffff8050a67a = vm_fault+0xf5a trap_pfault() at 0xffffffff80545d6f = trap_pfault+0x11f trap() at 0xffffffff80546448 = trap+0x4a8 calltrap() at 0xffffffff80530628 = calltrap+0x8 (kgdb) bt #0 doadump () at pcpu.h:224 #1 0xffffffff8036fc9a in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:306 #2 0xffffffff80370668 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:609 #3 0xffffffff80507850 in uma_zalloc_arg (zone=0xffffff00071c6e00, udata=0xffffff00071c8380, flags=2) at /usr/src/sys/vm/uma_core.c:2132 #4 0xffffffff80dac385 in kmem_cache_alloc (cache=Variable "cache" is not available. ) at /usr/src/sys/modules/opensolaris/../../cddl/compat/opensolaris/kern/opensolaris_kmem.c:196 #5 0xffffffff80c016e4 in arc_buf_alloc (spa=0xffffff00071e5000, size=131072, tag=Variable "tag" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:1344 #6 0xffffffff80c0389d in arc_read_nolock (pio=0x0, spa=0xffffff00071e5000, bp=0xffffff8002c71e80, done=0, private=0x0, priority=6, zio_flags=3, arc_flags=0xffffff812673b2bc, zb=0xffffff812673b280) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2931 #7 0xffffffff80c03dff in arc_read (pio=0x0, spa=0xffffff00071e5000, bp=0xffffff8002c71e80, pbuf=0xffffff00abaa31b0, done=0, private=0x0, priority=6, zio_flags=3, arc_flags=0xffffff812673b2bc, zb=0xffffff812673b280) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c:2832 #8 0xffffffff80c0826a in dbuf_prefetch (dn=0xffffff00b440f000, blkid=221) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:1641 #9 0xffffffff80c21444 in dmu_zfetch_dofetch (zf=0xffffff00b440f2b8, zs=0xffffff0126bb6480) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_zfetch.c:308 #10 0xffffffff80c21c89 in dmu_zfetch (zf=0xffffff00b440f2b8, offset=Variable "offset" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_zfetch.c:526 #11 0xffffffff80c06c04 in dbuf_read (db=0xffffff0031970b60, zio=0xffffff0078bb2328, flags=54) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dbuf.c:568 #12 0xffffffff80c0a58d in dmu_buf_hold_array_by_dnode (dn=0xffffff00b440f000, offset=Variable "offset" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:235 #13 0xffffffff80c0a80f in dmu_read (os=Variable "os" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/dmu.c:572 #14 0xffffffff80c6c9b5 in zfs_freebsd_getpages (ap=Variable "ap" is not available. ) at /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:4348 #15 0xffffffff805a6618 in VOP_GETPAGES_APV (vop=0xffffffff80cd8680, a=0xffffff812673b740) at vnode_if.c:2666 #16 0xffffffff80523c09 in vnode_pager_getpages (object=Variable "object" is not available. ) at vnode_if.h:1154 #17 0xffffffff8050a67a in vm_fault (map=0xffffff00ab3d5c40, vaddr=35585523712, fault_type=1 '\001', fault_flags=Variable "fault_flags" is not available. ) at vm_pager.h:130 #18 0xffffffff80545d6f in trap_pfault (frame=0xffffff812673bc40, usermode=1) at /usr/src/sys/amd64/amd64/trap.c:729 #19 0xffffffff80546448 in trap (frame=0xffffff812673bc40) at /usr/src/sys/amd64/amd64/trap.c:403 #20 0xffffffff80530628 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:228 #21 0x0000000800858fe6 in ?? () -- Andriy Gapon