From owner-freebsd-fs@freebsd.org Fri Nov 16 02:58:14 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D76001123E2D for ; Fri, 16 Nov 2018 02:58:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 537306D96E for ; Fri, 16 Nov 2018 02:58:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 154DD1123E2C; Fri, 16 Nov 2018 02:58:13 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E778A1123E2B for ; Fri, 16 Nov 2018 02:58:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 56E0F6D96A for ; Fri, 16 Nov 2018 02:58:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 6F9C42C8B for ; Fri, 16 Nov 2018 02:58:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id wAG2wBZk033253 for ; Fri, 16 Nov 2018 02:58:11 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id wAG2wBdJ033251 for fs@FreeBSD.org; Fri, 16 Nov 2018 02:58:11 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 233245] [UFS] Softupdates fails to track dependency between appended data and i_size Date: Fri, 16 Nov 2018 02:58:11 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: cem@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-Rspamd-Queue-Id: 537306D96E X-Spamd-Result: default: False [-105.92 / 40.00]; FORGED_RECIPIENTS_FORWARDING(0.00)[]; ALLOW_DOMAIN_WHITELIST(-100.00)[freebsd.org]; FORWARDED(0.00)[fs@mailman.ysv.freebsd.org]; SPF_FAIL_FORWARDING(0.00)[]; TO_DN_NONE(0.00)[]; HAS_XAW(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all]; XAW_SERVICE_ACCT(1.00)[]; RCVD_IN_DNSWL_MED(-0.20)[5.0.0.0.0.5.0.0.0.0.0.0.0.0.0.0.a.6.0.2.4.5.2.2.0.0.9.1.1.0.0.2.list.dnswl.org : 127.0.9.2]; MX_GOOD(-0.01)[cached: mx66.freebsd.org]; NEURAL_HAM_SHORT(-1.00)[-1.000,0]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; IP_SCORE(-3.71)[ip: (-9.87), ipnet: 2001:1900:2254::/48(-4.82), asn: 10310(-3.75), country: US(-0.10)]; ASN(0.00)[asn:10310, ipnet:2001:1900:2254::/48, country:US]; FORGED_RECIPIENTS(0.00)[fs@FreeBSD.org,freebsd-fs@freebsd.org]; TO_DOM_EQ_FROM_DOM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; FROM_NO_DN(0.00)[]; RCVD_COUNT_SEVEN(0.00)[7] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Nov 2018 02:58:14 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D233245 --- Comment #2 from Conrad Meyer --- Hi Kirk, Thanks for your prompt reply. (In reply to Kirk McKusick from comment #1) > As you note, soft updates does not currently consider it necessary to ens= ure > that the new contents of the block be written before increasing the file = size > if it knows that the exposed contents will be zero (as opposed to the ran= dom > data that was in a previously used block where it does ensure that the bl= ock is > written before it can be accessed). Yep, understood. > It would be possible to add a requirement that the new data be written be= fore > the size could be updated, That is the proposal. :-) > but it is not clear to me that adding this extra overhead is worthwhile. Overhead in which sense? I can imagine a few objections (it makes the code more complicated; there is some minor additional memory burden; SU dependen= cy graphs get a little deeper) but maybe you have others in mind I hadn't thou= ght of. I don't think there is any additional IO cost (perhaps longer fsync ti= me on appended files due to the additional ordering barrier?), but I might wel= l be missing something. As far as whether any runtime overhead is worthwhile, I think there are two relevant dimensions. One, what is the measurable overhead, if any? (We ca= n't really answer this one until we have a proof of concept implementation.) A= nd two, what is the user willing to accept, both in terms of overhead and appe= nd data fidelity? I think it's likely a trade-off we might not want to make unilaterally, but instead leave to the end user. It should be pretty easy to add it as a mount option or something like that= ; in the same vein as "noatime" allows users to disable a feature with too much overhead. I don't see any significant barriers to enabling or disabling it on-the-fly for existing mounts at runtime, either. > Also, this case includes overwriting data in the middle of an earlier blo= ck in the file for which there is nothing that we can do. Understood =E2=80=94 append is /the/ special case of write where this is po= ssible. But it is also a fairly common case, and given that we can do it safely, I thin= k we should aim to do it safely. (For middle-of-file partial overwrites, we could order data block flushing = with inode mtime update, if we do not already. That would be a pretty minimal protection and would not save us from torn writes. I think we might also be able to allocate data blocks out-of-place and order full block writes with their corresponding block pointer or indirect block pointer updates, but th= at has significant downsides, such as file fragmentation, and isn't something = I'm interested in right now.) > Note that you cannot put the test into ffs_write() in the way that you ha= ve > done (which makes the call for any growth in size). It can only be done w= hen > the growth is such that it does not go out of an existing block allocatio= n or > is overwriting an earlier part of the file. The elided portion of ffs_write() between lines 756 and 781 invokes UFS_BALLOC() to allocate any blocks needed -- so I believe the suggested location in ffs_write() would be within an existing block allocation. But maybe I am mistaken. I am not attached to the location, it just seemed lik= e a plausible spot to me. (There is a smaller separate concern, which is that any full block append is already fully tracked with allocdirect or allocindirect =E2=80=94 we only w= ant the proposed partial append dependency for the last iteration of the loop in ffs_write.) --=20 You are receiving this mail because: You are the assignee for the bug.=