From owner-freebsd-stable@FreeBSD.ORG Thu Sep 30 01:20:50 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AADD0106566C for ; Thu, 30 Sep 2010 01:20:50 +0000 (UTC) (envelope-from dan@langille.org) Received: from nyi.unixathome.org (nyi.unixathome.org [64.147.113.42]) by mx1.freebsd.org (Postfix) with ESMTP id 7B3158FC18 for ; Thu, 30 Sep 2010 01:20:50 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by nyi.unixathome.org (Postfix) with ESMTP id 77868509B2; Thu, 30 Sep 2010 02:20:49 +0100 (BST) X-Virus-Scanned: amavisd-new at unixathome.org Received: from nyi.unixathome.org ([127.0.0.1]) by localhost (nyi.unixathome.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LQjYsoOaaDRP; Thu, 30 Sep 2010 02:20:49 +0100 (BST) Received: from smtp-auth.unixathome.org (smtp-auth.unixathome.org [10.4.7.7]) (Authenticated sender: hidden) by nyi.unixathome.org (Postfix) with ESMTPSA id 3C285509A8 ; Thu, 30 Sep 2010 02:20:49 +0100 (BST) Message-ID: <4CA3E5F0.7000603@langille.org> Date: Wed, 29 Sep 2010 21:20:48 -0400 From: Dan Langille Organization: The FreeBSD Diary User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.9) Gecko/20100915 Thunderbird/3.1.4 MIME-Version: 1.0 To: Artem Belevich References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org Subject: Re: zfs send/receive: is this slow? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Sep 2010 01:20:50 -0000 On 9/29/2010 3:57 PM, Artem Belevich wrote: > On Wed, Sep 29, 2010 at 11:04 AM, Dan Langille wrote: >> It's taken about 15 hours to copy 800GB. I'm sure there's some tuning I >> can do. >> >> The system is now running: >> >> # zfs send storage/bacula@transfer | zfs receive storage/compressed/bacula > > Try piping zfs data through mbuffer (misc/mbuffer in ports). I've > found that it does help a lot to smooth out data flow and increase > send/receive throughput even when send/receive happens on the same > host. Run it with a buffer large enough to accommodate few seconds > worth of write throughput for your target disks. Thanks. I just installed it. I'll use it next time. I don't want to interrupt this one. I'd like to see how long it takes. Then compare. > Here's an example: > http://blogs.everycity.co.uk/alasdair/2010/07/using-mbuffer-to-speed-up-slow-zfs-send-zfs-receive/ That looks really good. Thank you. -- Dan Langille - http://langille.org/