From owner-svn-src-all@freebsd.org Fri Nov 13 09:31:58 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B9E1A2D3CB2; Fri, 13 Nov 2020 09:31:58 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CXYC24qXTz3Q6H; Fri, 13 Nov 2020 09:31:58 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9872516319; Fri, 13 Nov 2020 09:31:58 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0AD9Vwb3082845; Fri, 13 Nov 2020 09:31:58 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0AD9VwBL082843; Fri, 13 Nov 2020 09:31:58 GMT (envelope-from kib@FreeBSD.org) Message-Id: <202011130931.0AD9VwBL082843@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Fri, 13 Nov 2020 09:31:58 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r367631 - in head/sys: kern sys X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: in head/sys: kern sys X-SVN-Commit-Revision: 367631 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2020 09:31:58 -0000 Author: kib Date: Fri Nov 13 09:31:57 2020 New Revision: 367631 URL: https://svnweb.freebsd.org/changeset/base/367631 Log: Implement vn_lock_pair(). In collaboration with: pho Reviewed by: mckusick (previous version), markj (previous version) Tested by: markj (syzkaller), pho Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D26136 Modified: head/sys/kern/vfs_vnops.c head/sys/sys/vnode.h Modified: head/sys/kern/vfs_vnops.c ============================================================================== --- head/sys/kern/vfs_vnops.c Fri Nov 13 02:05:45 2020 (r367630) +++ head/sys/kern/vfs_vnops.c Fri Nov 13 09:31:57 2020 (r367631) @@ -70,6 +70,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -275,6 +276,10 @@ restart: vn_finished_write(mp); if (error) { NDFREE(ndp, NDF_ONLY_PNBUF); + if (error == ERELOOKUP) { + NDREINIT(ndp); + goto restart; + } return (error); } fmode &= ~O_TRUNC; @@ -1524,6 +1529,7 @@ vn_truncate(struct file *fp, off_t length, struct ucre vp = fp->f_vnode; +retry: /* * Lock the whole range for truncation. Otherwise split i/o * might happen partly before and partly after the truncation. @@ -1550,6 +1556,8 @@ out: vn_finished_write(mp); out1: vn_rangelock_unlock(vp, rl_cookie); + if (error == ERELOOKUP) + goto retry; return (error); } @@ -3317,4 +3325,92 @@ vn_fallocate(struct file *fp, off_t offset, off_t len, } return (error); +} + +static u_long vn_lock_pair_pause_cnt; +SYSCTL_ULONG(_debug, OID_AUTO, vn_lock_pair_pause, CTLFLAG_RD, + &vn_lock_pair_pause_cnt, 0, + "Count of vn_lock_pair deadlocks"); + +static void +vn_lock_pair_pause(const char *wmesg) +{ + atomic_add_long(&vn_lock_pair_pause_cnt, 1); + pause(wmesg, prng32_bounded(hz / 10)); +} + +/* + * Lock pair of vnodes vp1, vp2, avoiding lock order reversal. + * vp1_locked indicates whether vp1 is exclusively locked; if not, vp1 + * must be unlocked. Same for vp2 and vp2_locked. One of the vnodes + * can be NULL. + * + * The function returns with both vnodes exclusively locked, and + * guarantees that it does not create lock order reversal with other + * threads during its execution. Both vnodes could be unlocked + * temporary (and reclaimed). + */ +void +vn_lock_pair(struct vnode *vp1, bool vp1_locked, struct vnode *vp2, + bool vp2_locked) +{ + int error; + + if (vp1 == NULL && vp2 == NULL) + return; + if (vp1 != NULL) { + if (vp1_locked) + ASSERT_VOP_ELOCKED(vp1, "vp1"); + else + ASSERT_VOP_UNLOCKED(vp1, "vp1"); + } else { + vp1_locked = true; + } + if (vp2 != NULL) { + if (vp2_locked) + ASSERT_VOP_ELOCKED(vp2, "vp2"); + else + ASSERT_VOP_UNLOCKED(vp2, "vp2"); + } else { + vp2_locked = true; + } + if (!vp1_locked && !vp2_locked) { + vn_lock(vp1, LK_EXCLUSIVE | LK_RETRY); + vp1_locked = true; + } + + for (;;) { + if (vp1_locked && vp2_locked) + break; + if (vp1_locked && vp2 != NULL) { + if (vp1 != NULL) { + error = VOP_LOCK1(vp2, LK_EXCLUSIVE | LK_NOWAIT, + __FILE__, __LINE__); + if (error == 0) + break; + VOP_UNLOCK(vp1); + vp1_locked = false; + vn_lock_pair_pause("vlp1"); + } + vn_lock(vp2, LK_EXCLUSIVE | LK_RETRY); + vp2_locked = true; + } + if (vp2_locked && vp1 != NULL) { + if (vp2 != NULL) { + error = VOP_LOCK1(vp1, LK_EXCLUSIVE | LK_NOWAIT, + __FILE__, __LINE__); + if (error == 0) + break; + VOP_UNLOCK(vp2); + vp2_locked = false; + vn_lock_pair_pause("vlp2"); + } + vn_lock(vp1, LK_EXCLUSIVE | LK_RETRY); + vp1_locked = true; + } + } + if (vp1 != NULL) + ASSERT_VOP_ELOCKED(vp1, "vp1 ret"); + if (vp2 != NULL) + ASSERT_VOP_ELOCKED(vp2, "vp2 ret"); } Modified: head/sys/sys/vnode.h ============================================================================== --- head/sys/sys/vnode.h Fri Nov 13 02:05:45 2020 (r367630) +++ head/sys/sys/vnode.h Fri Nov 13 09:31:57 2020 (r367631) @@ -729,6 +729,8 @@ bool vn_isdisk_error(struct vnode *vp, int *errp); bool vn_isdisk(struct vnode *vp); int _vn_lock(struct vnode *vp, int flags, const char *file, int line); #define vn_lock(vp, flags) _vn_lock(vp, flags, __FILE__, __LINE__) +void vn_lock_pair(struct vnode *vp1, bool vp1_locked, struct vnode *vp2, + bool vp2_locked); int vn_open(struct nameidata *ndp, int *flagp, int cmode, struct file *fp); int vn_open_cred(struct nameidata *ndp, int *flagp, int cmode, u_int vn_open_flags, struct ucred *cred, struct file *fp);