From owner-freebsd-current@FreeBSD.ORG  Fri Mar  9 03:18:15 2007
Return-Path: <owner-freebsd-current@FreeBSD.ORG>
X-Original-To: current@freebsd.org
Delivered-To: freebsd-current@FreeBSD.ORG
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
	by hub.freebsd.org (Postfix) with ESMTP id BE32816A400
	for <current@freebsd.org>; Fri,  9 Mar 2007 03:18:15 +0000 (UTC)
	(envelope-from julian@elischer.org)
Received: from outY.internet-mail-service.net (outY.internet-mail-service.net
	[216.240.47.248])
	by mx1.freebsd.org (Postfix) with ESMTP id A88DC13C471
	for <current@freebsd.org>; Fri,  9 Mar 2007 03:18:15 +0000 (UTC)
	(envelope-from julian@elischer.org)
Received: from mx0.idiom.com (HELO idiom.com) (216.240.32.160)
	by out.internet-mail-service.net (qpsmtpd/0.32) with ESMTP;
	Thu, 08 Mar 2007 18:51:53 -0800
Received: from [10.251.22.38] (nat.ironport.com [63.251.108.100])
	by idiom.com (Postfix) with ESMTP id 9A93E125B4C
	for <current@freebsd.org>; Thu,  8 Mar 2007 19:18:14 -0800 (PST)
Message-ID: <45F0D1F5.9010200@elischer.org>
Date: Thu, 08 Mar 2007 19:18:13 -0800
From: Julian Elischer <julian@elischer.org>
User-Agent: Thunderbird 1.5.0.10 (Macintosh/20070221)
MIME-Version: 1.0
To: FreeBSD Current <current@freebsd.org>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: 
Subject: proc lock might become an sx lock?
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
	<freebsd-current.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>, 
	<mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>,
	<mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 09 Mar 2007 03:18:15 -0000

currently the thread list in the process is protected by the sched lock.
for a process with a lot of threads this is probably not a good idea.
I experimented with making it protected by the proc loc, but the following 
sort of thing happens a lot:

        sx_slock(&allproc_lock);
        FOREACH_PROC_IN_SYSTEM(p) {
                mtx_lock_spin(&sched_lock);
                FOREACH_THREAD_IN_PROC(p, td) {
                        ...
                }
                mtx_unlock_spin(&sched_lock);

Changing the protection of the thread list to use the proc lock would
replace the sched_lock with the proc lock, but.....
this has a problem.. the proc lock is a mutex and can therefore not be inside the
allproc_lock.

and in fact you get:

 Trying to mount root from ufs:/dev/aacd0s1d
 panic: blockable sleep lock (sleep mutex) process lock @ kern/sched_4bsd.c:383
 cpuid = 2
 KDB: enter: panic
 [thread pid 48 tid 100054 ]
 Stopped at      kdb_enter+0x2b: nop
 db> bt
 Tracing pid 48 tid 100054 td 0xc5ff4a20
 kdb_enter(c06ce300) at kdb_enter+0x2b
 panic(c06d3506,c06e7061,c06cd73b,c06cff47,17f,...) at panic+0x11c
 witness_checkorder(c60a12c8,9,c06cff47,17f) at witness_checkorder+0xb8
 _mtx_lock_flags(c60a12c8,0,c06cff3e,17f,85,...) at _mtx_lock_flags+0x87
 schedcpu(e65a9d24,c0516f50,0,e65a9d38,c6259000,...) at schedcpu+0x80
 schedcpu_thread(0,e65a9d38) at schedcpu_thread+0x9

My reading of the man page is that making it an sx lock and locking it in
shared mode would be sufficient for this sort of thing (assuming we are not changing
the thread list) and would be just fine.

I'm not very familiar with the implementation of sx locks in freeBSD so I'm just learning
about them. 

am I reading this right? and does anyone else have any thoughts on this?