From owner-freebsd-current@FreeBSD.ORG Sun May 14 19:21:39 2006 Return-Path: X-Original-To: freebsd-current@freebsd.org Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8033216A40A; Sun, 14 May 2006 19:21:39 +0000 (UTC) (envelope-from hadara@bsd.ee) Received: from mx1.starman.ee (smtp-out1.starman.ee [85.253.0.3]) by mx1.FreeBSD.org (Postfix) with ESMTP id DAF3043D46; Sun, 14 May 2006 19:21:38 +0000 (GMT) (envelope-from hadara@bsd.ee) Received: from [192.168.12.235] (depression.softematic.com [62.65.205.81]) by mx1.starman.ee (Postfix) with ESMTP id C4A2123C14C; Sun, 14 May 2006 22:21:35 +0300 (EEST) From: Sven Petai To: freebsd-current@freebsd.org Date: Sun, 14 May 2006 22:21:45 +0300 User-Agent: KMail/1.8.2 References: <20060506150622.C17611@fledge.watson.org> <20060507230430.GA6872@xor.obsecurity.org> <20060508065207.GA20386@xor.obsecurity.org> In-Reply-To: <20060508065207.GA20386@xor.obsecurity.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200605142221.46093.hadara@bsd.ee> X-Virus-Scanned: by Amavisd-New at mx1.starman.ee Cc: Robert Watson , Kris Kennaway Subject: Re: Fine-grained locking for POSIX local sockets (UNIX domain sockets) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 14 May 2006 19:21:44 -0000 On Monday 08 May 2006 09:52, Kris Kennaway wrote: > The other big gain is to sleep > mtxpool contention, which roughly doubled: > > /* > * Change the total socket buffer size a user has used. > */ > int > chgsbsize(uip, hiwat, to, max) > struct uidinfo *uip; > u_int *hiwat; > u_int to; > rlim_t max; > { > rlim_t new; > > UIDINFO_LOCK(uip); > > So the next question is how can that be optimized? > > Kris hi on the 8 core machine this lock was the top contended one with rwatsons patch, with over 8 million failed acquire attempts. Originally the unp lock had only ~3 million of those, so this explains the sharp drop with larger number of threads I suppose. I feel like I'm missing some very obvious reason, but wouldn't the simplest workaround be just to return 1 right away if limit is set to infinity, which is almost always the case since it's the default, and document on the login.conf manpage that you might take performance hit with this type of workloads when you set sbsize limits. --- /usr/src/sys_clean/kern/kern_resource.c Sat Mar 11 12:48:19 2006 +++ /usr/src/sys/kern/kern_resource.c Sun May 14 05:34:02 2006 @@ -1169,6 +1169,10 @@ { rlim_t new; + if (max == RLIM_INFINITY) { + *hiwat = to; + return (1); + } UIDINFO_LOCK(uip); new = uip->ui_sbsize + to - *hiwat; /* Don't allow them to exceed max, but allow subtraction. */ 8 core machine that I originally used for benchmarking was shipped out to client, so I couldn't test how it would have performed with uidinfo contention out of the way, but results from a 1 * dualcore machine look good: http://bsd.ee/~hadara/debug/mysql4/dualcore/stats.html Several interesting things can be noticed from this data * on dualcore rwatsons patch gives consistent performance boost with all the thread settings tested, no sharp drop after 20 that I had on 8 core * with threadcount in range [3;10] even number of threads performs usually ~4-5% better than odd number * with uidinfo + rwatson patch there were some significant outliers where one result was more than 30% better than others with same settings, these were removed before calculating mean values for graphs after I had finished benchmarking I discovered that new malloc library has debug turned on. After turning it off I see large (20-25%) performance boosts across the range, so I started new round of testing with NO_MALLOC_EXTRAS defined, I'll update the results ASAP. I wonder if I should set up automatic&periodic performance testing system, that would run all the tests for example once a week, with latest current and stable, so that it would be easier for developers to see how changes affect different workloads. If you guys think it would be worthwile, what would be the bechmarks you would like to see in addition to mysql+supersmack ?