From owner-svn-src-head@freebsd.org Fri Nov 13 16:24:54 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 385D42E6D5D; Fri, 13 Nov 2020 16:24:54 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-ot1-f51.google.com (mail-ot1-f51.google.com [209.85.210.51]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CXkMV0M6vz4ZVx; Fri, 13 Nov 2020 16:24:53 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-ot1-f51.google.com with SMTP id n15so9411449otl.8; Fri, 13 Nov 2020 08:24:53 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:reply-to :from:date:message-id:subject:to:cc; bh=L9Wey0GtLOLu2tJ4CH/c0ALzCICtUfoImGYBuE/sRXA=; b=JoboaOPvzSJ4urvtRk6GsZnRBOJLuO9zeMu2U0CUJrqWKxURKfnb4g/ZZGZ1TdNE0Z k8b3wsAnySJvojEW4FttyyCiNPHwBleOfLFbAcs1PYxS8r7jr/RI8/yxF02w7bAPNM/0 vT/LuHFxUGTTF9PC2zcrXTX+hxS+g5Jv2EKE3jfN/09Jfo27S5JddCWGB4WTSaRfPtzs jFaKopSVkzleMKW1Sk+apGwS6AaC8xNiAj9rdfbRXaNmlYePRSlPEiOGesty8snoQQcW LXX4LwfM1JHFC7C3GDQs6mqwhXVGhLCiEo0vkB/KPiCKjdRg0Nwt8Tpg4WT5Y/Ie3tMo gDOA== X-Gm-Message-State: AOAM532ei33u7kXnt6FfnZlT2NWQhbuTGYkoleXi/ldRrp6NKlsqh8L4 qluQQ0lgFhqvnH3geUoWhhrJ5jOcFDc= X-Google-Smtp-Source: ABdhPJyeeTy3tgZpfrncTEpvz0LQ07o/CNEcArh4q85Sgvm7CEAwgFe9kkiJmuoQhpQ7cfxFosoOiQ== X-Received: by 2002:a05:6830:789:: with SMTP id w9mr2066052ots.243.1605284687507; Fri, 13 Nov 2020 08:24:47 -0800 (PST) Received: from mail-oo1-f52.google.com (mail-oo1-f52.google.com. [209.85.161.52]) by smtp.gmail.com with ESMTPSA id 64sm2043659otq.26.2020.11.13.08.24.47 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 13 Nov 2020 08:24:47 -0800 (PST) Received: by mail-oo1-f52.google.com with SMTP id c25so2276081ooe.13; Fri, 13 Nov 2020 08:24:47 -0800 (PST) X-Received: by 2002:a4a:9486:: with SMTP id k6mr2090362ooi.85.1605284682065; Fri, 13 Nov 2020 08:24:42 -0800 (PST) MIME-Version: 1.0 References: <202011130931.0AD9VwBL082843@repo.freebsd.org> In-Reply-To: <202011130931.0AD9VwBL082843@repo.freebsd.org> Reply-To: cem@freebsd.org From: Conrad Meyer Date: Fri, 13 Nov 2020 08:24:30 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: svn commit: r367631 - in head/sys: kern sys To: Konstantin Belousov Cc: src-committers , svn-src-all , svn-src-head Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4CXkMV0M6vz4ZVx X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[] X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 13 Nov 2020 16:24:54 -0000 Hi Konstantin, On Fri, Nov 13, 2020 at 1:32 AM Konstantin Belousov wrote: > > Author: kib > Date: Fri Nov 13 09:31:57 2020 > New Revision: 367631 > URL: https://svnweb.freebsd.org/changeset/base/367631 > > Log: > Implement vn_lock_pair(). > > Modified: head/sys/kern/vfs_vnops.c > ============================================================================== > --- head/sys/kern/vfs_vnops.c Fri Nov 13 02:05:45 2020 (r367630) > +++ head/sys/kern/vfs_vnops.c Fri Nov 13 09:31:57 2020 (r367631) > @@ -3317,4 +3325,92 @@ vn_fallocate(struct file *fp, off_t offset, off_t len, > ... > + > +static void > +vn_lock_pair_pause(const char *wmesg) > +{ > + atomic_add_long(&vn_lock_pair_pause_cnt, 1); > + pause(wmesg, prng32_bounded(hz / 10)); > +} This function is called when the try-lock of the second lock in the pair (either order) fails. The back-off period is up to 100ms, expected average 50ms. That seems really high? Separately: prng32_bounded() may return 0, which is transparently converted to the equivalent of 1 by pause_sbt(9). This means a 1 tick pause is marginally more likely than any other possible duration. It probably doesn't matter. Thanks, Conrad > + > +/* > + * Lock pair of vnodes vp1, vp2, avoiding lock order reversal. > + * vp1_locked indicates whether vp1 is exclusively locked; if not, vp1 > + * must be unlocked. Same for vp2 and vp2_locked. One of the vnodes > + * can be NULL. > + * > + * The function returns with both vnodes exclusively locked, and > + * guarantees that it does not create lock order reversal with other > + * threads during its execution. Both vnodes could be unlocked > + * temporary (and reclaimed). > + */ > +void > +vn_lock_pair(struct vnode *vp1, bool vp1_locked, struct vnode *vp2, > + bool vp2_locked) > +{ > + int error; > + > + if (vp1 == NULL && vp2 == NULL) > + return; > + if (vp1 != NULL) { > + if (vp1_locked) > + ASSERT_VOP_ELOCKED(vp1, "vp1"); > + else > + ASSERT_VOP_UNLOCKED(vp1, "vp1"); > + } else { > + vp1_locked = true; > + } > + if (vp2 != NULL) { > + if (vp2_locked) > + ASSERT_VOP_ELOCKED(vp2, "vp2"); > + else > + ASSERT_VOP_UNLOCKED(vp2, "vp2"); > + } else { > + vp2_locked = true; > + } > + if (!vp1_locked && !vp2_locked) { > + vn_lock(vp1, LK_EXCLUSIVE | LK_RETRY); > + vp1_locked = true; > + } > + > + for (;;) { > + if (vp1_locked && vp2_locked) > + break; > + if (vp1_locked && vp2 != NULL) { > + if (vp1 != NULL) { > + error = VOP_LOCK1(vp2, LK_EXCLUSIVE | LK_NOWAIT, > + __FILE__, __LINE__); > + if (error == 0) > + break; > + VOP_UNLOCK(vp1); > + vp1_locked = false; > + vn_lock_pair_pause("vlp1"); (Pause called here and in similar elided case for vp2 -> vp1 below.) > + } > + vn_lock(vp2, LK_EXCLUSIVE | LK_RETRY); > + vp2_locked = true; > + }