From owner-cvs-all@FreeBSD.ORG  Tue Apr  1 02:17:28 2008
Return-Path: <owner-cvs-all@FreeBSD.ORG>
Delivered-To: cvs-all@hub.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id A0915106566C;
	Tue,  1 Apr 2008 02:17:28 +0000 (UTC)
	(envelope-from davidxu@FreeBSD.org)
Received: from freefall.freebsd.org (freefall.freebsd.org
	[IPv6:2001:4f8:fff6::28])
	by mx1.freebsd.org (Postfix) with ESMTP id 8E4918FC23;
	Tue,  1 Apr 2008 02:17:28 +0000 (UTC)
	(envelope-from davidxu@FreeBSD.org)
Received: from apple.my.domain (root@localhost [127.0.0.1])
	by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m312HOXp062786;
	Tue, 1 Apr 2008 02:17:25 GMT (envelope-from davidxu@freebsd.org)
Message-ID: <47F19B89.20601@freebsd.org>
Date: Tue, 01 Apr 2008 10:18:49 +0800
From: David Xu <davidxu@FreeBSD.org>
User-Agent: Thunderbird 2.0.0.9 (X11/20071211)
MIME-Version: 1.0
To: Jeff Roberson <jroberson@chesapeake.net>
References: <200803310255.m2V2toJl053104@repoman.freebsd.org>
	<20080331150346.R72156@desktop>
In-Reply-To: <20080331150346.R72156@desktop>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: cvs-src@FreeBSD.org, src-committers@FreeBSD.org, cvs-all@FreeBSD.org
Subject: Re: cvs commit: src/lib/libthr/thread thr_cond.c thr_private.h  
 thr_rwlock.c
X-BeenThere: cvs-all@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: CVS commit messages for the entire tree <cvs-all.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/cvs-all>,
	<mailto:cvs-all-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/cvs-all>
List-Post: <mailto:cvs-all@freebsd.org>
List-Help: <mailto:cvs-all-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/cvs-all>,
	<mailto:cvs-all-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 01 Apr 2008 02:17:28 -0000

Jeff Roberson wrote:
> On Mon, 31 Mar 2008, David Xu wrote:
> 
>> davidxu     2008-03-31 02:55:50 UTC
>>
>>  FreeBSD src repository
>>
>>  Modified files:
>>    lib/libthr/thread    thr_cond.c thr_private.h thr_rwlock.c
>>  Log:
>>  Rewrite rwlock to user atomic operations to change rwlock state, this
>>  eliminates internal mutex lock contention when most rwlock operations
>>  are read.
>>
>>  Orignal patch provided by: jeff
> 
> Thanks David!
> 
> Do you still plan to do kernel side support so we can totally eliminate 
> the pthread mutex and condvars from rwlock?  I have evidence that this 
> would produce further improvements.
> 
> Jeff

I have pure kernel version working, sometimes locking kernel sleep
chain is more expensive than a pure userland mutex locking, since
kernel sleep chain lock is global. I am hestiate adding more code
to kernel since I have tested mysql super-smack and sysbench
read-only and seen no visible performance differences.
all patches are here:

http://people.freebsd.org/~davidxu/patch/libthr_umtx_rwlock.diff
http://people.freebsd.org/~davidxu/patch/umtx_rwlock.diff

Te main bottleneck is in mysql itself, there are lots of lock
contention, the following is a list of the main contention points when
testing super-smack read benchmark.

http://people.freebsd.org/~davidxu/mysql/mysql_lock.txt

I don't think it scales beyond 4 cores ( I have tested it on a 4-core
machine).

Regards,
David Xu