From owner-freebsd-smp@FreeBSD.ORG Mon Apr 17 07:54:09 2006 Return-Path: X-Original-To: smp@freebsd.org Delivered-To: freebsd-smp@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9EFA116A403 for ; Mon, 17 Apr 2006 07:54:09 +0000 (UTC) (envelope-from surerlistmail@gmail.com) Received: from nproxy.gmail.com (nproxy.gmail.com [64.233.182.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id B498C43D4C for ; Mon, 17 Apr 2006 07:54:08 +0000 (GMT) (envelope-from surerlistmail@gmail.com) Received: by nproxy.gmail.com with SMTP id h2so366832nfe for ; Mon, 17 Apr 2006 00:54:07 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=l2+B6J6+/sWaobY7CpT6pBC2gPA5a4rVNJ2zF7Tn8vduzqkxXNuOvqEa/LhDqHq5v1OWRncyDCpgZhXZy0+Od9lPSkKmK1SWhFzfnaPMvdjuhBmh9pA9YMNQ0EXv8tvvWhrib7+DrWayA+UYFnhvZeJR3twUZKNT2lC08z6xlKk= Received: by 10.49.9.9 with SMTP id m9mr2350669nfi; Mon, 17 Apr 2006 00:54:07 -0700 (PDT) Received: by 10.49.61.12 with HTTP; Mon, 17 Apr 2006 00:54:07 -0700 (PDT) Message-ID: Date: Mon, 17 Apr 2006 03:54:07 -0400 From: "Surer Dink" To: "Kris Kennaway" , smp@freebsd.org, current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Cc: Subject: Re: Anomalous performance increase from mutex profiling X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Apr 2006 07:54:09 -0000 On Mon, 17 Apr 2006, Kris Kennaway wrote: > On Mon, Apr 17, 2006 at 01:14:40AM -0400, Kris Kennaway wrote: > >> * Our best guess is that mutex profiling is doing something that >> reduces contention on this very heavily contended mutex (unp), but I'd >> like to know what is happening precisely so I can maybe make use of >> it. >> >> Can anyone think of what may be happening that I've missed? > > I think it is just doing effectively the same thing as my exponential > spin backoff patch, namely introducing delays with the effect of > reducing common memory accesses. When I turn the maximum spin backoff > limit *way* up (from 1600 to 51200) I get performance that slightly > exceeds what I see from mutex profiling alone (adding mutex profiling > again on top of this gives a small further increase, but only a few % > and so probably achievable by further increasing the backoff limit). > > A limit of 51200 is not an appropriate default since it penalizes the > common case of light to moderate contention. The point is that here > basically all 12 CPUs are spinning on a single lock > (kern/uipc_usrreq.c:308), so it's massively over-contended and all we > can do is mitigate the effects of this. > > On this system, the maximum supersmack performance (3700 queries/sec) > comes when there are only 6 clients, so (as jasone eloquently put it) > with 10 clients the difference between 2300 queries/sec (with absurdly > high backoff limits or mutex profiling) and 1450/sec (with reasonable > backoff limits) is the difference between "slow" and "ass slow". Please excuse if this is a stupid question - but might using MCS or QOLB locks in this situation be useful?