From owner-freebsd-current@freebsd.org  Sun Jul 10 12:22:53 2016
Return-Path: <owner-freebsd-current@freebsd.org>
Delivered-To: freebsd-current@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id A4044B83FA9
 for <freebsd-current@mailman.ysv.freebsd.org>;
 Sun, 10 Jul 2016 12:22:53 +0000 (UTC)
 (envelope-from kostikbel@gmail.com)
Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1])
 (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 1E6F61BCC
 for <freebsd-current@freebsd.org>; Sun, 10 Jul 2016 12:22:52 +0000 (UTC)
 (envelope-from kostikbel@gmail.com)
Received: from tom.home (kib@localhost [127.0.0.1])
 by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u6ACMlmT027699
 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO);
 Sun, 10 Jul 2016 15:22:47 +0300 (EEST)
 (envelope-from kostikbel@gmail.com)
DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u6ACMlmT027699
Received: (from kostik@localhost)
 by tom.home (8.15.2/8.15.2/Submit) id u6ACMluY027698;
 Sun, 10 Jul 2016 15:22:47 +0300 (EEST)
 (envelope-from kostikbel@gmail.com)
X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com
 using -f
Date: Sun, 10 Jul 2016 15:22:47 +0300
From: Konstantin Belousov <kostikbel@gmail.com>
To: Mateusz Guzik <mjguzik@gmail.com>
Cc: freebsd-current@freebsd.org
Subject: Re: [PATCH] microoptimize locking primitives by introducing
 randomized delay between atomic ops
Message-ID: <20160710122247.GS38613@kib.kiev.ua>
References: <20160710111326.GA7853@dft-labs.eu>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20160710111326.GA7853@dft-labs.eu>
User-Agent: Mutt/1.6.1 (2016-04-27)
X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00,
 DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no
 autolearn_force=no version=3.4.1
X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 10 Jul 2016 12:22:53 -0000

On Sun, Jul 10, 2016 at 01:13:26PM +0200, Mateusz Guzik wrote:
> If the lock is contended, primitives like __mtx_lock_sleep will spin
> checking if the owner is running or the lock was freed. The problem is
> that once it is discovered that the lock is free, multiple CPUs are
> likely to try to do the atomic op which will make it more costly for
> everyone and throughput suffers.
> 
> The standard thing to do is to have some sort of a randomized delay so
> that this kind of behaviour is reduced.
> 
> As such, below is a trivial hack which takes cpu_ticks() into account
> and performs % 2048, which in my testing gives reasonbly good results.
> 
> Please note there is definitely way more room for improvement in general.
> 
> In terms of results, there was no statistically significant change in
> -j 40 buildworld nor buildkernel.
> 
> However, a 40-way find on a ports tree placed on tmpfs yielded the following:

I am curious why did you added randomizer to sx adaptive loop but not to
lockmgr loop, and probably most important, to the spinlocks (unless I
misread the patch).