From owner-freebsd-fs@FreeBSD.ORG Fri Sep 26 07:02:36 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C11391AA for ; Fri, 26 Sep 2014 07:02:36 +0000 (UTC) Received: from mail.jrv.org (rrcs-24-73-246-106.sw.biz.rr.com [24.73.246.106]) by mx1.freebsd.org (Postfix) with ESMTP id 906517A9 for ; Fri, 26 Sep 2014 07:02:36 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.jrv.org (Postfix) with ESMTP id DA33F1790C8 for ; Fri, 26 Sep 2014 01:42:57 -0500 (CDT) Received: from mail.jrv.org ([127.0.0.1]) by localhost (zimbra64.housenet.jrv [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id eNQKoWMu6GBG for ; Fri, 26 Sep 2014 01:42:48 -0500 (CDT) Received: from localhost (localhost.localdomain [127.0.0.1]) by mail.jrv.org (Postfix) with ESMTP id EC4E21790C3 for ; Fri, 26 Sep 2014 01:42:47 -0500 (CDT) X-Virus-Scanned: amavisd-new at zimbra64.housenet.jrv Received: from mail.jrv.org ([127.0.0.1]) by localhost (zimbra64.housenet.jrv [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id t4hO_jjT1JfY for ; Fri, 26 Sep 2014 01:42:47 -0500 (CDT) Received: from [192.168.138.128] (BMX.housenet.jrv [192.168.3.140]) by mail.jrv.org (Postfix) with ESMTPSA id C02AC1790C0 for ; Fri, 26 Sep 2014 01:42:47 -0500 (CDT) Message-ID: <54250AE9.6070609@jrv.org> Date: Fri, 26 Sep 2014 01:42:49 -0500 From: "James R. Van Artsdalen" User-Agent: Mozilla/5.0 (Windows NT 5.0; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: zfs recv hangs in kmem arena on STABLE10 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Sep 2014 07:02:36 -0000 FreeBSD BLACKIE.housenet.jrv 10.1-BETA2 FreeBSD 10.1-BETA2 #2 r272070M: Wed Sep 24 17:36:56 CDT 2014 james@BLACKIE.housenet.jrv:/usr/obj/usr/src/sys/GENERIC amd64 With current STABLE10 I am unable to replicate a ZFS pool using zfs send/recv without zfs hanging in state "kmem arena", within the first 4TB or so (of a 23TB Pool). The most recent attempt used this command line SUPERTEX:/root# zfs send -R BIGTEX/UNIX@syssnap | ssh BLACKIE zfs recv -duvF BIGTOX though local replications fail in kmem arena too. The two machines I've been attempting this on have 16BG and 32GB of RAM each and are otherwise idle. Any suggestions on how to get around, or investigate, "kmem arena"? # top last pid: 3272; load averages: 0.22, 0.22, 0.23 up 0+08:25:02 01:32:07 34 processes: 1 running, 33 sleeping CPU: 0.0% user, 0.0% nice, 0.1% system, 0.0% interrupt, 99.9% idle Mem: 21M Active, 82M Inact, 15G Wired, 28M Cache, 450M Free ARC: 12G Total, 24M MFU, 12G MRU, 23M Anon, 216M Header, 47M Other Swap: 16G Total, 16G Free PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 1173 root 1 52 0 86476K 7780K select 0 124:33 0.00% sshd 1176 root 1 46 0 87276K 47732K kmem a 3 48:36 0.00% zfs 968 root 32 20 0 12344K 1888K rpcsvc 0 0:13 0.00% nfsd 1009 root 1 20 0 25452K 2864K select 3 0:01 0.00% ntpd ...