From owner-freebsd-fs@FreeBSD.ORG Sat Nov 22 09:25:33 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4698E9E0 for ; Sat, 22 Nov 2014 09:25:33 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AF02F975 for ; Sat, 22 Nov 2014 09:25:32 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id sAM9PRrq046793 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 22 Nov 2014 11:25:27 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua sAM9PRrq046793 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id sAM9PRT1046791; Sat, 22 Nov 2014 11:25:27 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 22 Nov 2014 11:25:27 +0200 From: Konstantin Belousov To: Mateusz Guzik Subject: Re: atomic v_usecount and v_holdcnt Message-ID: <20141122092527.GT17068@kib.kiev.ua> References: <20141122002812.GA32289@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20141122002812.GA32289@dft-labs.eu> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Nov 2014 09:25:33 -0000 On Sat, Nov 22, 2014 at 01:28:12AM +0100, Mateusz Guzik wrote: > The idea is that we don't need an interlock as long as we don't > transition either counter 1->0 or 0->1. > > Patch itself is more of a PoC, so I didn't rename vholdl & friends just > yet. > > It helps during lookups with same vnodes since the interlock which was > taken twice served as a serializatin point and this effect is now > reduced. > > There are other places which can avoid VI_LOCK + vget scheme. > > Patch below survived make -j 40 buildworld, poudriere with 40 workers > etc on a 2 package(s) x 10 core(s) x 2 SMT threads machine with and > without debugs (including DEBUG_VFS_LOCKS). > > Perf difference: > > in a crap microbenchmark of 40 threads doing a stat on > /foo/bar/baz/quux${N}, where each thread stats a separate file I got > over 4 times speed up on tmpfs. > > Comments? I already said that something along the lines of the patch should work. In fact, you need vnode lock when hold count changes between 0 and 1, and probably the same for use count. Some notes about the patch. mtx_owned() braces are untolerable ugliness. You should either pass a boolean flag (preferred), or create locked/unlocked versions of the functions. Similarly, I dislike vget_held(). Add a flag to vget(), see LK_EATTR_MASK in sys/lockmgr.h. Could there be consequences of not taking vnode interlock and passing LK_INTERLOCK to vn_lock() in vget() ? Taking interlock when vnode lock is already owned is probably fine and does not add to contention. I mean that making VI_OWEINACT so loose breaks the VOP_INACTIVE() contract.