From owner-freebsd-current@FreeBSD.ORG Wed Apr 2 01:19:17 2003 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EA4D337B401 for ; Wed, 2 Apr 2003 01:19:16 -0800 (PST) Received: from puffin.mail.pas.earthlink.net (puffin.mail.pas.earthlink.net [207.217.120.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5019F43F3F for ; Wed, 2 Apr 2003 01:19:16 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0093.cvx40-bradley.dialup.earthlink.net ([216.244.42.93] helo=mindspring.com) by puffin.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 190eOl-0001Eq-00; Wed, 02 Apr 2003 01:19:12 -0800 Message-ID: <3E8AAAA1.CB4B51B0@mindspring.com> Date: Wed, 02 Apr 2003 01:17:21 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Stijn Hoop References: <20030331225124.W64602-100000@mail.chesapeake.net> <20030402083026.GB83512@pcwin002.win.tue.nl> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4db8b4a983a519484b56f88a96dcfe43e350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c cc: Jeff Roberson cc: current@freebsd.org Subject: Re: libthr and 1:1 threading. X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 Apr 2003 09:19:17 -0000 Stijn Hoop wrote: > On Mon, Mar 31, 2003 at 10:54:45PM -0500, Jeff Roberson wrote: > > I have commited libthr. To try this out you'll need to do the following > > I know very very little about threads, but I'm interested as to what the > purpose is of this library. Is there a document available somewhere that > describes the relationships between this, KSE, libc_r, pthreads, the > Giant-unwinding-make-SMP-work-better project and some of the other > threads and SMP related libraries and terminology? No, not really: the new libthr was pretty much a "Skunk Works" project, and was not managed through the KSE project; it's really orthogonal, but builds on some of the KSE work already done in the kernel so far... most of KSE lives there. Here's a thumbnail sketch, though (forgive me, KSE folks, if I mung it too badly): pthreads: POSIX Threads is a threads API, which is specified by the standard ISO/IEC 9945-1:1996 (``POSIX.1''). libc_r: A user space implementation of the pthreads API; this implementation uses a "call conversion" scheduler, in user space, to convert blocking calls into non-blocking calls, plus a threads context switch via the user space scheduler. Like all interactive timesharing schedulers, it gives the illusion of concurrency. However, the kernel is not thread-reentrant in this model, so it does not scale to more than one CPU on an SMP system, since there is only a single scheduler context. KSE: "Kernel Schedulable Entitites" is the name of the modified scheduler activations framework, as well as the user space components, and kernel modifications, for an N:M model threading system. It has the advantage over the "libc_r" in that it causes the kernel to be thread reentrant, and so it provides SMP scalability. Because it's N:M, it also has the advantage over the 1:1 approach of causing full quantum utilization, and providing for CPU affinity for individual threads, and CPU negaffinity for threads within the same process, thereby providing for theoretically optimal CPU and other resource utilization. It also includes a user-space library component, which is incomplete at present. libthr: This is the recently committed 1:1 model threading library. It provides a simpler user space library component, which provides the same SMP scalability, as far as kernel thread reentrancy is concerned, but fails to provide for full quantum utilization, and, at present, does not directly address the CPU affinity issues itself (no sophisticated use of KSEGRP). It builds on the kernel modifications for KSE, and adds a couple of system call API's in order to manage creation, etc., of threads. The major intent is to provide for SMP scalability; as a side effect, it provides a proof-of-concept for the KSE code already in the kernel, and as such, has been very welcome. SMPng: "The Giant-unwinding-make-SMP-work-better project", to quote you. 8-). SMPng has it's own project page and documentation. To give another thumbnail drawing, it's about improving the granularity of locking and the logical seperation between kernel subsystems, so that stall barriers are eliminated. By doing this, inter-CPU contention is reduced, and you get better SMP scaling. Traditionally, when you added another CPU, you maybe got a 20% performance improvement for a 100% increase in cost. The idea is to get that 20% up to as close to 100% as possible (impossible, but you can approach that). SVR4, for example, scales well up to 4 processors on Intel. It scales higher than that, but the incremental improvement is about 80%, and so at about 4 processors, you hit a point where the cost of additional processors is higher than the value of the additional compute cycles. You may also find these resources useful: http://people.freebsd.org/~julian/threads/ http://www.freebsd.org/smp/index.html http://www.freebsd.org/projects/busdma/index.html http://www.freebsd.org/projects/projects.html Most of the documentation lives in mailing list archives, and is not terribly formal (Software Engineers, not English Majors, and all that...). -- Terry