From owner-freebsd-stable@FreeBSD.ORG Fri Oct 1 15:53:26 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CAFC6106566B for ; Fri, 1 Oct 2010 15:53:26 +0000 (UTC) (envelope-from dan@langille.org) Received: from nyi.unixathome.org (nyi.unixathome.org [64.147.113.42]) by mx1.freebsd.org (Postfix) with ESMTP id 99EA58FC17 for ; Fri, 1 Oct 2010 15:53:26 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by nyi.unixathome.org (Postfix) with ESMTP id CC225509A5; Fri, 1 Oct 2010 16:53:25 +0100 (BST) X-Virus-Scanned: amavisd-new at unixathome.org Received: from nyi.unixathome.org ([127.0.0.1]) by localhost (nyi.unixathome.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QC+dUZquPq+Z; Fri, 1 Oct 2010 16:53:25 +0100 (BST) Received: from nyi.unixathome.org (localhost [127.0.0.1]) by nyi.unixathome.org (Postfix) with ESMTP id EA8E4509A3; Fri, 1 Oct 2010 16:53:24 +0100 (BST) Received: from 68.64.144.221 (SquirrelMail authenticated user dan) by nyi.unixathome.org with HTTP; Fri, 1 Oct 2010 11:53:25 -0400 Message-ID: <55a0e58dd844285fbb50cb2904820943.squirrel@nyi.unixathome.org> In-Reply-To: References: Date: Fri, 1 Oct 2010 11:53:25 -0400 From: "Dan Langille" To: "Artem Belevich" User-Agent: SquirrelMail/1.4.20-RC2 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Cc: freebsd-stable@freebsd.org Subject: Re: zfs send/receive: is this slow? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Oct 2010 15:53:26 -0000 On Fri, October 1, 2010 11:45 am, Dan Langille wrote: > > On Wed, September 29, 2010 3:57 pm, Artem Belevich wrote: >> On Wed, Sep 29, 2010 at 11:04 AM, Dan Langille wrote: >>> It's taken about 15 hours to copy 800GB.  I'm sure there's some tuning >>> I >>> can do. >>> >>> The system is now running: >>> >>> # zfs send storage/bacula@transfer | zfs receive >>> storage/compressed/bacula >> >> Try piping zfs data through mbuffer (misc/mbuffer in ports). I've >> found that it does help a lot to smooth out data flow and increase >> send/receive throughput even when send/receive happens on the same >> host. Run it with a buffer large enough to accommodate few seconds >> worth of write throughput for your target disks. >> >> Here's an example: >> http://blogs.everycity.co.uk/alasdair/2010/07/using-mbuffer-to-speed-up-slow-zfs-send-zfs-receive/ > > I'm failing. In one session: > > # mbuffer -s 128k -m 1G -I 9090 | zfs receive > storage/compressed/bacula-mbuffer > Assertion failed: ((err == 0) && (bsize == sizeof(rcvsize))), function > openNetworkInput, file mbuffer.c, line 1358. > cannot receive: failed to read from stream > > > In the other session: > > # time zfs send storage/bacula@transfer | mbuffer -s 128k -m 1G -O > 10.55.0.44:9090 > Assertion failed: ((err == 0) && (bsize == sizeof(sndsize))), function > openNetworkOutput, file mbuffer.c, line 897. > warning: cannot send 'storage/bacula@transfer': Broken pipe > Abort trap: 6 (core dumped) > > real 0m17.709s > user 0m0.000s > sys 0m2.502s My installed mbuffer was out of date. After an upgrade: # mbuffer -s 128k -m 1G -I 9090 | zfs receive storage/compressed/bacula-mbuffer mbuffer: warning: unable to set socket buffer size: No buffer space available in @ 0.0 kB/s, out @ 0.0 kB/s, 1897 MB total, buffer 100% full # time zfs send storage/bacula@transfer | mbuffer -s 128k -m 1G -O ::1:9090 mbuffer: warning: unable to set socket buffer size: No buffer space available in @ 4343 kB/s, out @ 2299 kB/s, 3104 MB total, buffer 85% full -- Dan Langille -- http://langille.org/