From nobody Sat Dec 25 00:38:21 2021 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id F0AB818F4778; Sat, 25 Dec 2021 00:38:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4JLQ5T4Xlfz4cZR; Sat, 25 Dec 2021 00:38:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6E0845C85; Sat, 25 Dec 2021 00:38:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1BP0cLoU098442; Sat, 25 Dec 2021 00:38:21 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1BP0cL1j098441; Sat, 25 Dec 2021 00:38:21 GMT (envelope-from git) Date: Sat, 25 Dec 2021 00:38:21 GMT Message-Id: <202112250038.1BP0cL1j098441@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Rick Macklem Subject: git: 6cccfeaafc29 - stable/13 - nfscl: Fix must_commit/writeverf handling for Direct I/O List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rmacklem X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 6cccfeaafc298e2536bac9ef53a1ea0cbc1956dd Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1640392701; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=JqO+I7FR24JRWgcPN8Vmx5uG4yKhE7i6ksVkLo2oFdY=; b=obt00r45FHQHZdcykOihWF6JO/R3UpqvytgzrrJJjCahBWUJnJHyvGU02wdCzwkILLdcb7 7zUQGHZ7gJudYqii2J4gOSU+/IhwNMbnHHgoJ4GZAiCun2RYlnGC+0Yyc3B/ZI0ci0NQMt 0H6qQ3tO9xSveVHouQSNwhILdwqFT3YrIyTJTlrWoj+0JTqcOKmDp3DSlzra08UD/yIB3s n2q90wUhMgq3iSuZy+SlfrEl7cQuO84eeBN7XwvAaiMXeJIHWTJh6UfqQzOxAzRpMtRDwj 7yob6I3aLExoHnmWl9yAhRsbXDSV3lok8mNN7tHkxzDidPYTkN7gJvl0+jgXug== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1640392701; a=rsa-sha256; cv=none; b=QjRy0GkH+q3fKAx6M+DbPp6yuru22M54CRKZ+ANy1bBYGOaaal0iqWftAUal6l28edKpXv Wy1ZZWt5qKCLI6Ul9IEwDuyuDMC+Is9YfZPUd5oLE1JomlE+0gQ7HxLkPqgz2Sr22av2pA 2FCIw64/+PhYZqDaj0wINHQIbN0y/iz+HEie4MUd3TvJZFyJfIq+t+mOOAQdHc+k23TZNl 9zCGg1y4sT81j/bqEayQjiYGffrglj7zJAlHgFRrMJyxhpjJcBujDNAbrbhj9hzieTBR0n Hb/MvBNtWGARkF5t/p4SV2djJEDxpstSAs9VGi+7o7nQ8O+hA59GTsTv/1jdTA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch stable/13 has been updated by rmacklem: URL: https://cgit.FreeBSD.org/src/commit/?id=6cccfeaafc298e2536bac9ef53a1ea0cbc1956dd commit 6cccfeaafc298e2536bac9ef53a1ea0cbc1956dd Author: Rick Macklem AuthorDate: 2021-12-11 23:00:30 +0000 Commit: Rick Macklem CommitDate: 2021-12-25 00:33:57 +0000 nfscl: Fix must_commit/writeverf handling for Direct I/O Without this patch, the KASSERT(must_commit == 0,..) can be triggered by the writeverf in the Direct I/O write reply changing. This is not a situation that should cause a panic(). Correct handling is to ignore the change in "writeverf" for Direct I/O, since it is done with NFSWRITE_FILESYNC. This patch modifies the semantics of the "must_commit" argument slightly, allowing an initial value of 2 to indicate that a change in "writeverf" should be ignored. It also fixes the KASSERT()s. This bug would affect few, since Direct I/O is not enabled by default and "writeverf" rarely changes. Normally "writeverf" only changes when a NFS server reboots, however I found the bug when testing against a Linux 5.15.1 kernel nfsd, which replied to a NFSWRITE_FILESYNC write with a "writeverf" of all 0x0 bytes. (cherry picked from commit ead50c94cb604594987e6512289268891a427725) --- sys/fs/nfsclient/nfs_clbio.c | 37 +++++++++++++++++++++++++++++++++---- sys/fs/nfsclient/nfs_clrpcops.c | 14 +++++++++----- 2 files changed, 42 insertions(+), 9 deletions(-) diff --git a/sys/fs/nfsclient/nfs_clbio.c b/sys/fs/nfsclient/nfs_clbio.c index 73f559ad82f8..29bc66669dfb 100644 --- a/sys/fs/nfsclient/nfs_clbio.c +++ b/sys/fs/nfsclient/nfs_clbio.c @@ -784,12 +784,26 @@ do_sync: uio.uio_rw = UIO_WRITE; uio.uio_td = td; iomode = NFSWRITE_FILESYNC; + /* + * When doing direct I/O we do not care if the + * server's write verifier has changed, but we + * do not want to update the verifier if it has + * changed, since that hides the change from + * writes being done through the buffer cache. + * By passing must_commit in set to two, the code + * in nfsrpc_writerpc() will not update the + * verifier on the mount point. + */ + must_commit = 2; error = ncl_writerpc(vp, &uio, cred, &iomode, &must_commit, 0); - KASSERT((must_commit == 0), - ("ncl_directio_write: Did not commit write")); + KASSERT((must_commit == 2), + ("ncl_directio_write: Updated write verifier")); if (error) return (error); + if (iomode != NFSWRITE_FILESYNC) + printf("nfs_directio_write: Broken server " + "did not reply FILE_SYNC\n"); uiop->uio_offset += size; uiop->uio_resid -= size; if (uiop->uio_iov->iov_len <= size) { @@ -1592,8 +1606,23 @@ ncl_doio_directwrite(struct buf *bp) iomode = NFSWRITE_FILESYNC; uiop->uio_td = NULL; /* NULL since we're in nfsiod */ + /* + * When doing direct I/O we do not care if the + * server's write verifier has changed, but we + * do not want to update the verifier if it has + * changed, since that hides the change from + * writes being done through the buffer cache. + * By passing must_commit in set to two, the code + * in nfsrpc_writerpc() will not update the + * verifier on the mount point. + */ + must_commit = 2; ncl_writerpc(bp->b_vp, uiop, bp->b_wcred, &iomode, &must_commit, 0); - KASSERT((must_commit == 0), ("ncl_doio_directwrite: Did not commit write")); + KASSERT((must_commit == 2), ("ncl_doio_directwrite: Updated write" + " verifier")); + if (iomode != NFSWRITE_FILESYNC) + printf("ncl_doio_directwrite: Broken server " + "did not reply FILE_SYNC\n"); free(iov_base, M_NFSDIRECTIO); free(uiop->uio_iov, M_NFSDIRECTIO); free(uiop, M_NFSDIRECTIO); @@ -1864,7 +1893,7 @@ ncl_doio(struct vnode *vp, struct buf *bp, struct ucred *cr, struct thread *td, } } bp->b_resid = uiop->uio_resid; - if (must_commit) + if (must_commit == 1) ncl_clearcommit(vp->v_mount); bufdone(bp); return (error); diff --git a/sys/fs/nfsclient/nfs_clrpcops.c b/sys/fs/nfsclient/nfs_clrpcops.c index bd2e1f24eef2..2e1d3e86c1d2 100644 --- a/sys/fs/nfsclient/nfs_clrpcops.c +++ b/sys/fs/nfsclient/nfs_clrpcops.c @@ -1721,7 +1721,8 @@ nfsrpc_write(vnode_t vp, struct uio *uiop, int *iomode, int *must_commit, nfsv4stateid_t stateid; void *lckp; - *must_commit = 0; + KASSERT(*must_commit >= 0 && *must_commit <= 2, + ("nfsrpc_write: must_commit out of range=%d", *must_commit)); if (nmp->nm_clp != NULL) clidrev = nmp->nm_clp->nfsc_clientidrev; newcred = cred; @@ -1934,7 +1935,7 @@ nfsrpc_writerpc(vnode_t vp, struct uio *uiop, int *iomode, NFSX_VERF); NFSSETWRITEVERF(nmp); } else if (NFSBCMP(tl, nmp->nm_verf, - NFSX_VERF)) { + NFSX_VERF) && *must_commit != 2) { *must_commit = 1; NFSBCOPY(tl, nmp->nm_verf, NFSX_VERF); } @@ -6568,7 +6569,8 @@ nfsrpc_writeds(vnode_t vp, struct uio *uiop, int *iomode, int *must_commit, if (!NFSHASWRITEVERF(nmp)) { NFSBCOPY(tl, nmp->nm_verf, NFSX_VERF); NFSSETWRITEVERF(nmp); - } else if (NFSBCMP(tl, nmp->nm_verf, NFSX_VERF)) { + } else if (NFSBCMP(tl, nmp->nm_verf, NFSX_VERF) && + *must_commit != 2) { *must_commit = 1; NFSBCOPY(tl, nmp->nm_verf, NFSX_VERF); } @@ -6578,7 +6580,8 @@ nfsrpc_writeds(vnode_t vp, struct uio *uiop, int *iomode, int *must_commit, if ((dsp->nfsclds_flags & NFSCLDS_HASWRITEVERF) == 0) { NFSBCOPY(tl, dsp->nfsclds_verf, NFSX_VERF); dsp->nfsclds_flags |= NFSCLDS_HASWRITEVERF; - } else if (NFSBCMP(tl, dsp->nfsclds_verf, NFSX_VERF)) { + } else if (NFSBCMP(tl, dsp->nfsclds_verf, NFSX_VERF) && + *must_commit != 2) { *must_commit = 1; NFSBCOPY(tl, dsp->nfsclds_verf, NFSX_VERF); } @@ -6686,7 +6689,8 @@ nfsrpc_writedsmir(vnode_t vp, int *iomode, int *must_commit, if ((dsp->nfsclds_flags & NFSCLDS_HASWRITEVERF) == 0) { NFSBCOPY(tl, dsp->nfsclds_verf, NFSX_VERF); dsp->nfsclds_flags |= NFSCLDS_HASWRITEVERF; - } else if (NFSBCMP(tl, dsp->nfsclds_verf, NFSX_VERF)) { + } else if (NFSBCMP(tl, dsp->nfsclds_verf, NFSX_VERF) && + *must_commit != 2) { *must_commit = 1; NFSBCOPY(tl, dsp->nfsclds_verf, NFSX_VERF); }