From owner-freebsd-hackers@freebsd.org Fri Mar 3 08:36:02 2017 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 183A0CF6B3D; Fri, 3 Mar 2017 08:36:02 +0000 (UTC) (envelope-from alfred@freebsd.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 076DA177A; Fri, 3 Mar 2017 08:36:01 +0000 (UTC) (envelope-from alfred@freebsd.org) Received: from Alfreds-MacBook-Pro-2.local (unknown [IPv6:2601:645:8003:a4d6:f496:718a:b775:a369]) by elvis.mu.org (Postfix) with ESMTPSA id E0F79346DDDC; Fri, 3 Mar 2017 00:36:00 -0800 (PST) Subject: Re: effect of strip(1) on du(1) To: Alan Somers , Ngie Cooper References: <201703030031.v230VvIl066398@pdx.rh.CN85.dnsmgr.net> Cc: "Rodney W. Grimes" , Subbsd , Peter Jeremy , freebsd-hackers , freebsd-current Current From: Alfred Perlstein Organization: FreeBSD Message-ID: <9fb69e17-cb7f-5f04-aeb3-914ba58ddab4@freebsd.org> Date: Fri, 3 Mar 2017 00:36:00 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0) Gecko/20100101 Thunderbird/45.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Mar 2017 08:36:02 -0000 On 3/2/17 5:30 PM, Alan Somers wrote: > On Thu, Mar 2, 2017 at 6:12 PM, Ngie Cooper wrote: >> On Thu, Mar 2, 2017 at 4:31 PM, Rodney W. Grimes >> wrote: >> ... >>> Even if that is the case file system cache effects should NOT be >>> visible to a userland process. This is NOT as if your running >>> 2 different processing beating on a file. Your test cases are >>> serialially syncronous shell invoked commands seperated with >>> && the results should be exact and predictable. >>> >>> When strip returns the operation from the userland perspecive >>> is completed and any and all processeses started after that >>> should have the view of the completed strip command. >>> >>> This IS a bug. >> Would the same statement necessarily apply if the filesystem was >> writing things asynchronously to the backing storage? >> Thanks, >> -Ngie > du(1) is using fts_read(3), which is based on the stat(2) information. > The OpenGroup defines st_blocksize as "Number of blocks allocated for > this object." In the case of ZFS, a write(2) may return before any > blocks are actually allocated. And thanks to compression, gang > blocks, and deduplication, at this point it's not even possible for > ZFS to know how many blocks it will need to allocate. I think > st_blocksize should be interpreted as a "best effort" output. Just > like df(1), you can't rely on du's output to be mathematically precise > in any way. I certainly don't see any way to fix it besides doing > something like an fsync(2) before getting stat information, and we > certainly don't want to do that. > > Try adding an fsync(1) to the file before running du(1) on it. -Alfred