From owner-freebsd-questions@FreeBSD.ORG Thu Mar 28 13:01:20 2013 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 99CB314A for ; Thu, 28 Mar 2013 13:01:20 +0000 (UTC) (envelope-from freebsd@qeng-ho.org) Received: from blue.qeng-ho.org (blue.qeng-ho.org [217.155.128.241]) by mx1.freebsd.org (Postfix) with ESMTP id 2099B6C7 for ; Thu, 28 Mar 2013 13:01:19 +0000 (UTC) Received: from fileserver.home.qeng-ho.org (localhost [127.0.0.1]) by fileserver.home.qeng-ho.org (8.14.5/8.14.5) with ESMTP id r2SCjkmd011743; Thu, 28 Mar 2013 12:45:49 GMT (envelope-from freebsd@qeng-ho.org) Message-ID: <51543B7A.4030300@qeng-ho.org> Date: Thu, 28 Mar 2013 12:45:46 +0000 From: Arthur Chance User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130309 Thunderbird/17.0.4 MIME-Version: 1.0 To: "Ronald F. Guilmette" Subject: Re: Copying memstick image to a USB (flash/thumb) drive References: <19222.1364466721@server1.tristatelogic.com> In-Reply-To: <19222.1364466721@server1.tristatelogic.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Mar 2013 13:01:20 -0000 On 03/28/13 10:32, Ronald F. Guilmette wrote: > > In message <5153FEFF.4090305@sneakertech.com>, you wrote: > >> >>> I have filed the following PR: >>> >>> http://www.freebsd.org/cgi/query-pr.cgi?pr=177431 >> >> Er, don't take my word for law: > > I didn't. I won't. > >> I have *no* idea if 1M is a good idea > > Any size which is an exact multiple of the physical block size for > the target device should provide performance which is as good as > it gets. > > I googled around and read various comments. Some of these kinds > of devices have a physical block size of 64KiB. Some have 128KiB. > Some have 256KiB. Some have 1MiB. > > For all of these devices, seting blocksize to 1MiB will provide optimal > performance with at worst only a _relatively_ tiny waste of space. > >> As for the conv=sync option, I'm not convinced it's necessary either >> way. > > Neither am I, but I would rather have it there, than not and take > chances. > > It won't hurt anything, and it appears that it _may_ perhaps help. > >> I don't think modern systems really care what the end is padded with > > That wasn't my concern. My concern is that I personally do not know > what the officially defined semantics are in cases where dd is asked > to copy data in a specific input block size _and_ the actual data > available from the input device doesn't perfectly fill up that last > block. > > It is possible, I would guess, that dd may notice the EOF occuring > before it has filled up an entire input buffer, and then just quit > at that point, _without_ writing the partial last block to the > output device. It will attempt to write a short block. E.g. arthur@fileserver[2]> ls -l t.pdf -rw-rw-r-- 1 arthur arthur 233812 Feb 18 12:26 t.pdf arthur@fileserver[2]> dd if=t.pdf of=/dev/null bs=1k 228+1 records in 228+1 records out 233812 bytes transferred in 0.036731 secs (6365521 bytes/sec) Those +1 records are the final short block. > It seems to me that conv=sync is cheap insurance against this > possibility. It's used as an insurance against output devices which have a fixed (or in the case of tape, a minimum) block size. If the short block is not an exact integer multiple of the device block size then the final write will fail. conv=sync and a bs (or obs) that's a strict multiple of block size prevents the problem. That's exactly your use case. > I have always been a belt and suspenders kind of guy. Show me the experienced sysadmin who isn't. Invariably learnt the hard way. :-}