From owner-freebsd-questions@FreeBSD.ORG Tue Sep 23 21:17:24 2008 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D21C3106566B for ; Tue, 23 Sep 2008 21:17:24 +0000 (UTC) (envelope-from fbsd06+2B=235abae4@mlists.homeunix.com) Received: from mxout-03.mxes.net (mxout-03.mxes.net [216.86.168.178]) by mx1.freebsd.org (Postfix) with ESMTP id A98AE8FC14 for ; Tue, 23 Sep 2008 21:17:24 +0000 (UTC) (envelope-from fbsd06+2B=235abae4@mlists.homeunix.com) Received: from gumby.homeunix.com. (unknown [87.81.140.128]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.mxes.net (Postfix) with ESMTP id 5091923E3E7 for ; Tue, 23 Sep 2008 17:17:22 -0400 (EDT) Date: Tue, 23 Sep 2008 22:17:20 +0100 From: RW To: freebsd-questions@freebsd.org Message-ID: <20080923221720.184885a7@gumby.homeunix.com.> In-Reply-To: <20080923153700.GA85529@gizmo.acns.msu.edu> References: <6e5cf6a70809230804g1d2f0359g566c756ebfe9a038@mail.gmail.com> <20080923153700.GA85529@gizmo.acns.msu.edu> X-Mailer: Claws Mail 3.5.0 (GTK+ 2.12.11; i386-portbld-freebsd7.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: blocksize when using dd to copy disks? bigger = better? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Sep 2008 21:17:24 -0000 On Tue, 23 Sep 2008 11:37:00 -0400 Jerry McAllister wrote: > On Tue, Sep 23, 2008 at 11:04:13AM -0400, Joachim Rosenfeld wrote: > > > When mirroring a disk with dd, I notice that a blocksize of 512 runs > > awfully slow, but with bs=1MB (2^10bytes), it runs fairly quickly. > > > > Can someone explain the implications of this? Did all the data not > > copy properly with the larger blocksize? > > If you are on a beach moving sand and you pick up one grain at a > time and move it, it will take a very long time because the overhead > of moving yourself is much higher than the amount of sand moved. > If you use the largest bucket or scoop that you can handle, then > it goes much faster because the same body motions result in much > more being moved. Moving data has a similar dynamic. I tried playing around with this once, and I found that the speed rose rapidly up to a certain blocksize, then levelled-out for a decade or so and then dropped to half of the peak speed. IIRC in that particular case the optimum range was something like 20k-200k. I presume what happens is that you can make the blocksize too big for the other buffering, and end-up alternating reads and writes rather than doing them in parallel.