From owner-freebsd-stable@freebsd.org Thu Jun 8 10:02:25 2017 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4C8FEBFAFD0 for ; Thu, 8 Jun 2017 10:02:25 +0000 (UTC) (envelope-from ardovm@yahoo.it) Received: from nm6-vm9.bullet.mail.ir2.yahoo.com (nm6-vm9.bullet.mail.ir2.yahoo.com [212.82.96.125]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D858C7BBE0 for ; Thu, 8 Jun 2017 10:02:24 +0000 (UTC) (envelope-from ardovm@yahoo.it) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.it; s=s2048; t=1496915993; bh=Gg9OcjgVb4VCc1/5+gRbxyn9Htqg5ySU2+zPzObuFaE=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From:Subject; b=L23hphgdwmc7yuf208Kv/CnW1YRggr1qevBDRMyafNXKf3LhToyhg45F8h/xa52UxkKVDulIDlp+bbofoAaJWfajCNvaC9a3yfYBBxx+QNtY6Fa6OpV6QGJtCoWjHfmiTnKtjAxJI80UyRrj1ouyVTEpeurK/j4uY9lrJEtnBt+IDXUMuHATLFY9S8BlLxxbVsHCF79JxU6FfvRLW5Ab98PUw+omo4V1aufo1MmtZt+TtoKYSQ1l21cdCDLJ/li+ZleMbsBjlJhJ6Ff1xZHRa8aSU3ycY1YSrtWzyHfNf3W/efUmmhLHAtff9hQUTf8voG9NRhvh0Jzl6Unp4R9ILA== Received: from [212.82.98.49] by nm6.bullet.mail.ir2.yahoo.com with NNFMP; 08 Jun 2017 09:59:53 -0000 Received: from [46.228.39.79] by tm2.bullet.mail.ir2.yahoo.com with NNFMP; 08 Jun 2017 09:59:53 -0000 Received: from [127.0.0.1] by smtp116.mail.ir2.yahoo.com with NNFMP; 08 Jun 2017 09:59:53 -0000 X-Yahoo-Newman-Id: 773498.9713.bm@smtp116.mail.ir2.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: E0g0aGcVM1mvaNiRSZQuzjfcHFULgaahF5TfSXy_7MCLFwn ecQeCaNF_JRtLLrwlIK4kgBINtDv.ZqJ5QeYanjziRKMZtTXS_HbP8e7c0Ht fGZoQVq8yQlkE5xQ_2H7QUQb6T0ZlYcVhNsla2MQLA5t5kxj7ypzgg3u1keV l7h6jLbPnyVgBvbU0WcFIQO.U1.ndbPt9cYgE_BL6HSjKv5jwHDBSabXISVM ui_8ZnHVzzaoHhk1d2pWNW0Fjx20fQ8xsDnevUAsF1sgs87_WsYLWTx8zey1 YLZBRzOWP8xzS0vra8bkI.6HGFOIabwqgfU7F4ELsgY5ju8hxAy8_4w1_RXb ltfVuHQB9FscJqMMs_5LWPmg4BXRhdub6.n7LH0awyrV3vG6bjdn_ePaIrAh kZGrUrkxMXaaPoHP7frGfWZ75EyPpoG7ComFKjSSjlx.I7dBLGgvMtQ2vdWI _kmBuZAnOR6Yg_dLQermhhBlcfCFMmCMEcbXJSvCiyO.P6leTUoF73GoD462 Y2LGbGNpuVJUC9yO75LG1G0xbKsWTT6wZf_dG4K4BjrndWob1 X-Yahoo-SMTP: WU.IBxeswBAAnLcBZV3tEZIK0A-- Received: by nuvolo.localdomain (Postfix, from userid 1001) id 7E5221127EF; Thu, 8 Jun 2017 11:59:52 +0200 (CEST) Date: Thu, 8 Jun 2017 11:59:52 +0200 From: Arrigo Marchiori To: Christopher Hall Cc: Dewayne Geraghty , freebsd-stable stable Subject: Re: dd is returning different content using same inputs Message-ID: <20170608095952.GA19586@nuvolo> References: <20170608171515.72fa7844@arria.bitmark.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20170608171515.72fa7844@arria.bitmark.lan> User-Agent: Mutt/1.8.2 (2017-04-18) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 Jun 2017 10:02:25 -0000 Hello, On Thu, Jun 08, 2017 at 05:15:15PM +0800, Christopher Hall wrote: > Hello Dewayne, > > On Thu, 8 Jun 2017 18:10:11 +1000, Dewayne Geraghty > wrote: > > > I'm attempting to skip the first three bytes of a file and then pull > > the header out of message. Please note the file size varies between > > 105 and 54 bytes, even though the input file and everything else > > remains constant, on > > > > FreeBSD hathor 11.1-PRERELEASE FreeBSD 11.1-PRERELEASE #0 r319562M: > > Mon Jun 5 03:13:08 AEST 2017 > > root@hathor:/110007/D/K8/hqdev-amd64-smp-vga amd64 1100514 1100514 > > > > # rm encheader;dd if=/tmp/tLbdoJ/encmessage bs=1 skip=3 | dd bs=384 > > count=1 of=encheader ; ls -lrth encheader > > 0+1 records in > > 0+1 records out > > 105 bytes transferred in 0.000026 secs (4071661 bytes/sec) > > -rw-r--r-- 1 root wheel 105B Jun 8 16:35 encheader > > # rm encheader;dd if=/tmp/tLbdoJ/encmessage bs=1 skip=3 | dd bs=384 > > count=1 of=encheader ; ls -lrth encheader > > 0+1 records in > > 0+1 records out > > 67 bytes transferred in 0.000026 secs (2554912 bytes/sec) > > -rw-r--r-- 1 root wheel 67B Jun 8 16:35 encheader > > # rm encheader;dd if=/tmp/tLbdoJ/encmessage bs=1 skip=3 | dd bs=384 > > count=1 of=encheader ; ls -lrth encheader > > 0+1 records in > > 0+1 records out > > 68 bytes transferred in 0.000024 secs (2867263 bytes/sec) > > -rw-r--r-- 1 root wheel 68B Jun 8 16:35 encheader > > # rm encheader;dd if=/tmp/tLbdoJ/encmessage bs=1 skip=3 | dd bs=384 > > count=1 of=encheader ; ls -lrth encheader > > 0+1 records in > > 0+1 records out > > 54 bytes transferred in 0.000025 secs (2167717 bytes/sec) > > -rw-r--r-- 1 root wheel 54B Jun 8 16:35 encheader > > # rm encheader;dd if=/tmp/tLbdoJ/encmessage bs=1 skip=3 | dd bs=384 > > count=1 of=encheader ; ls -lrth encheader > > 0+1 records in > > 0+1 records out > > 91 bytes transferred in 0.000024 secs (3751494 bytes/sec) > > -rw-r--r-- 1 root wheel 91B Jun 8 16:35 encheader > > > > I used this around May 8th (from a OS and ports rebuild on 4th May) > > and was performing some final testing on an 11.1 Prerelease built on > > Jun 5. > > > It works similarly on TrueOS (FreeBSD 12) I suspect it is because you > block sizes are different on the input and output dd commands. > I would think that the output block size (obs) on 1st dd needs to be > specified the same input block size on receiving dd to make it work > correctly. I also agree; the ``funny'' behavior is probably given by the pipe and the different block sizes. The manual page of dd(1) says about the bs parameter: [...] > bs=n Set both input and output block size to n bytes, superseding the > ibs and obs operands. If no conversion values other than > noerror, notrunc or sync are specified, then each input block is > copied to the output as a single block without any aggregation > of short blocks. The second dd seems to receive non-deterministic amounts of data from the first, and due to the bs parameter, it does not wait until 384 bytes are received, but it outputs its partial block immediately and then quits. > > This works correctly so perhaps I'm not using dd correctly? > > dd if=/tmp/tLbdoJ/encmessage skip=1 bs=3 | dd bs=1 count=384 > > of=encheader ; hd encheader|head -n1; ls -l encheader If I understood correctly, you could issue a single command: $ dd if=inputfile bs=1 skip=3 count=384 of=outputfile I hope this helps. -- rigo http://rigo.altervista.org