From owner-freebsd-current@freebsd.org Fri Mar 3 01:30:33 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F221FCF689A; Fri, 3 Mar 2017 01:30:33 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-yw0-x243.google.com (mail-yw0-x243.google.com [IPv6:2607:f8b0:4002:c05::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B01121CFA; Fri, 3 Mar 2017 01:30:33 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-yw0-x243.google.com with SMTP id l16so8609366ywb.2; Thu, 02 Mar 2017 17:30:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=EdM/8axQuC0OFffM0mnX9s3KpyE5JWDfqjhjtwaao2A=; b=sob9MzDHh9vbtuAErAUinxPll3KpMWhXylkbyj4Imuf/HwHRVNRWCOMQ74Gl64J7T2 QzcgcqNVZdXCA5qDo6JGpXNs/NwlmGHCspygRa0fXQU9g+gZ2HUWTOwmPSWTQWovnA0M bOfH7uRJSvTs2hYq/NQsSpHzqzsUpU+wuBl6mIzE8e15zL9fUW+HTtJEDO/NEljqlSbJ F/EA8BxyAt/IWHxASIczhO58ELeS+4jxw22Pq0/V47H5t/w2f1r5qEtatW1tjuCayLtE 1DcVWNrv3joJJS5VYE//1oscajjNLFwvV+5xKHGVPoQ7NshfrNRv5KbUfElcmDqdpYkg ZmOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=EdM/8axQuC0OFffM0mnX9s3KpyE5JWDfqjhjtwaao2A=; b=d737FR6oiETO2ts4A89VFbEwF3W1isqD32yP9Da/FPpNfjmVZkmQOP9JhlWAm3wuxl 33uiWWNAAE4GHwAuPSRiWJtlkVusdZLwSG/sSi7qt7z1YMu8quAkkTTq7FirVwYPFGEN GSG5N3fit7GZkNEkXQ/m2W9NNORCe7mXRFbT3bV6JS/bfgpOZxh3EqyihAdlKpMfaxAK 60LKYwtf6ybGUeYCcBd8A48f220wnzwzzSxQA1QnWKt3pvJmObMxJWaFxwcJaS67IK3L Xn/LeBPHOAMMWVq+f9QkEAtmb78PbUYFjjlO/2jELHjWVy8tJyA60IpcWHY7lR/bYz0Y 7F+g== X-Gm-Message-State: AMke39nbyTdwcLAH8+N3U8QsxxrKcQST0Wo/zyQXY1reKtWz23jFUPzwdFb00LqOS5rNQuFeDT63t2+3L53V0Q== X-Received: by 10.37.215.23 with SMTP id o23mr121955ybg.113.1488504632767; Thu, 02 Mar 2017 17:30:32 -0800 (PST) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.129.38.133 with HTTP; Thu, 2 Mar 2017 17:30:32 -0800 (PST) In-Reply-To: References: <201703030031.v230VvIl066398@pdx.rh.CN85.dnsmgr.net> From: Alan Somers Date: Thu, 2 Mar 2017 18:30:32 -0700 X-Google-Sender-Auth: Qeu-d44pK0cgbWCyD6YpAsttZdk Message-ID: Subject: Re: effect of strip(1) on du(1) To: Ngie Cooper Cc: "Rodney W. Grimes" , Subbsd , Peter Jeremy , freebsd-hackers , freebsd-current Current Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Mar 2017 01:30:34 -0000 On Thu, Mar 2, 2017 at 6:12 PM, Ngie Cooper wrote: > On Thu, Mar 2, 2017 at 4:31 PM, Rodney W. Grimes > wrote: > ... >> Even if that is the case file system cache effects should NOT be >> visible to a userland process. This is NOT as if your running >> 2 different processing beating on a file. Your test cases are >> serialially syncronous shell invoked commands seperated with >> && the results should be exact and predictable. >> >> When strip returns the operation from the userland perspecive >> is completed and any and all processeses started after that >> should have the view of the completed strip command. >> >> This IS a bug. > > Would the same statement necessarily apply if the filesystem was > writing things asynchronously to the backing storage? > Thanks, > -Ngie du(1) is using fts_read(3), which is based on the stat(2) information. The OpenGroup defines st_blocksize as "Number of blocks allocated for this object." In the case of ZFS, a write(2) may return before any blocks are actually allocated. And thanks to compression, gang blocks, and deduplication, at this point it's not even possible for ZFS to know how many blocks it will need to allocate. I think st_blocksize should be interpreted as a "best effort" output. Just like df(1), you can't rely on du's output to be mathematically precise in any way. I certainly don't see any way to fix it besides doing something like an fsync(2) before getting stat information, and we certainly don't want to do that. -Alan