From owner-freebsd-bugs@FreeBSD.ORG Wed Nov 5 17:14:05 2014 Return-Path: Delivered-To: freebsd-bugs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D1282FA4 for ; Wed, 5 Nov 2014 17:14:05 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 95C80B16 for ; Wed, 5 Nov 2014 17:14:05 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id sA5HE5ce009707 for ; Wed, 5 Nov 2014 17:14:05 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-bugs@FreeBSD.org Subject: [Bug 194513] zfs recv hangs in state kmem arena Date: Wed, 05 Nov 2014 17:14:05 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: james-freebug@jrv.org X-Bugzilla-Status: Needs Triage X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-bugs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Nov 2014 17:14:05 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=194513 --- Comment #3 from James Van Artsdalen --- Created attachment 149075 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=149075&action=edit procstat -kk -a The same systems hang the same way on "zpool history", although the attached EATMEM does not seem tp help. Attached is the full "procstat -kk -a" output with a "zpool history" stuck. SUPERTEX:/root# uname -a FreeBSD SUPERTEX.housenet.jrv 10.1-PRERELEASE FreeBSD 10.1-PRERELEASE #3 r273476M: Wed Oct 22 15:05:29 CDT 2014 root@SUPERTEX.housenet.jrv:/usr/obj/usr/src/sys/GENERIC amd64 SUPERTEX:/root# procstat -kk -a > kk SUPERTEX:/root# grep zio_wait kk SUPERTEX:/root# grep zio_done kk SUPERTEX:/root# grep zio_int kk SUPERTEX:/root# grep zfs_ kk 0 100475 kernel zfs_vn_rele_task mi_switch+0xe1 sleepq_wait+0x3a _sleep+0x287 taskqueue_thread_loop+0xd5 fork_exit+0x9a fork_trampoline+0xe 0 101018 kernel zfs_vn_rele_task mi_switch+0xe1 sleepq_wait+0x3a _sleep+0x287 taskqueue_thread_loop+0xd5 fork_exit+0x9a fork_trampoline+0xe 0 101522 kernel zfs_vn_rele_task mi_switch+0xe1 sleepq_wait+0x3a _sleep+0x287 taskqueue_thread_loop+0xd5 fork_exit+0x9a fork_trampoline+0xe 12105 101599 zpool - mi_switch+0xe1 sleepq_wait+0x3a _cv_wait+0x16d vmem_xalloc+0x568 vmem_alloc+0x3d kmem_malloc+0x33 uma_large_malloc+0x49 malloc+0x43 zfs_ioc_pool_get_history+0xbd zfsdev_ioctl+0x5ca devfs_ioctl_f+0x139 kern_ioctl+0x255 sys_ioctl+0x13c amd64_syscall+0x351 Xfast_syscall+0xfb SUPERTEX:/root# grep dmu_ kk SUPERTEX:/root# grep arc_ kk 3 100125 zfskern arc_reclaim_thre mi_switch+0xe1 sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18b arc_reclaim_thread+0x288 fork_exit+0x9a fork_trampoline+0xe 3 100126 zfskern l2arc_feed_threa mi_switch+0xe1 sleepq_timedwait+0x3a _cv_timedwait_sbt+0x18b l2arc_feed_thread+0x19f fork_exit+0x9a fork_trampoline+0xe SUPERTEX:/root# grep buf_ kk 20 100139 bufdaemon - mi_switch+0xe1 sleepq_timedwait+0x3a _sleep+0x26e buf_daemon+0x78 fork_exit+0x9a fork_trampoline+0xe SUPERTEX:/root# ps lp 12105 UID PID PPID CPU PRI NI VSZ RSS MWCHAN STAT TT TIME COMMAND 0 12105 12104 0 20 0 105692 35984 kmem are D - 0:03.76 zpool history BI SUPERTEX:/root# -- You are receiving this mail because: You are the assignee for the bug.