From owner-freebsd-hackers@FreeBSD.ORG Fri May 23 07:43:38 2003 Return-Path: <owner-freebsd-hackers@FreeBSD.ORG> Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 16EB237B401 for <freebsd-hackers@freebsd.org>; Fri, 23 May 2003 07:43:38 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4D08F43F85 for <freebsd-hackers@freebsd.org>; Fri, 23 May 2003 07:43:37 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h4NEhNwQ008104; Fri, 23 May 2003 10:43:23 -0400 (EDT) Received: from localhost (eischen@localhost)h4NEhMT0008097; Fri, 23 May 2003 10:43:22 -0400 (EDT) Date: Fri, 23 May 2003 10:43:22 -0400 (EDT) From: Daniel Eischen <eischen@pcnet1.pcnet.com> To: Terry Lambert <tlambert2@mindspring.com> In-Reply-To: <3ECDACF2.CF7A637A@mindspring.com> Message-ID: <Pine.GSO.4.10.10305231034040.6389-100000@pcnet1.pcnet.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-hackers@freebsd.org cc: Dan Nelson <dnelson@allantgroup.com> cc: Julian Elischer <julian@elischer.org> Subject: Re: libkse and SMP (was Re: USB bulk read & pthreads) X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD <freebsd-hackers.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>, <mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers> List-Post: <mailto:freebsd-hackers@freebsd.org> List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>, <mailto:freebsd-hackers-request@freebsd.org?subject=subscribe> X-List-Received-Date: Fri, 23 May 2003 14:43:38 -0000 On Thu, 22 May 2003, Terry Lambert wrote: > Dan Nelson wrote: > > In the last episode (May 22), Terry Lambert said: > > > Make sure you use SCHED_4BSD, rather than SCHED_ULE, if you are using > > > one of the kernel threads libraries, for now. > > > > > > You really should read the -current archives before attempting any of > > > this, if you don't follow -current closely enough to have caught > > > Jeff's message on this, or which kernel threading libraries are > > > available. > > > > I do read -current, and I'm definitely not going anywhere near > > SCHED_ULE. I just tested both libraries with today's kernel and > > libraries, and was able to get a hard lockup with both libthr and > > libkse. Mysql seems to run okay. Starting a threaded pike process > > seems to be the killer. Unfortunately, pike's a pretty large app so > > it's not easy to get a stripped-down testcase. > > The philosophies behing libthr and libkse are different. For > libthr, you basically have a FreeBSD version of Linux threads; > for libkse, there are some issues you have to deal with. > > The first of these issues is that you have to add Daniel's patch > to the libkse scheduler code. This is necessary because there is > a lot of threads code that is not completely compliant with the > POSIX standard: it makes assumptions which POSIX does not permit, > about what thread gets scheduled to run after an involuntary > context switch. The original Netscape (e.g. 4.7) has this same > assumption, and Java interfaces that use image maps lock it up if > you move the mouse over the map while the GIF is loading. If you > wait for it to load, there's no problem. Mozilla has similar > assumptions. This is just because rtld-elf is not thread safe (or written so that locks are not needed as you claim). > The second of these is that the libkse model is M:N, with N being > defaulted to 1. If you want more kernel threads, you have to ask N is defaulted to the number of CPUs that you have. Libkse will create as many KSEs as there are CPUs (or whatever kern.threads.virtual_cpu is set to); these will be KSEs that run scope process threads. Scope system threads get their own KSE/KSEG pair without regard to number of CPUs or kern.threads.virtual_cpu. Trying to exceed the number of CPUs with kern.threads.virtual_cpu will not work unless you also set kern.threads.debug=1. You really shouldn't need to do this, but we use it for testing and debugging. > for them (and you aren't doing that). In general, a lot of code > is going to assume for M:N that M==N; to get this, you will need > to create threads with an attribute other than the default of > "NULL", with the scope set to PTHREAD_SCOPE_SYSTEM. You should > also look at the actual scheduler code in libkse to see if there > are any other requirements for creating KSEG's for KSE's for > user space threads (I haven't looked at it for several weeks now, > so I don't know if anything has changed, unless it was sent to > one of the mailing lists). > > With both of these out of the way, libkse should work for you. > > The libthr may not work for you due to scheduling order, or it > may not work for you because signals are not masked per process, > or it may not work if you have threads that depend on the > PTHREAD_SCOPE_PROCESS attribute (kernel threads like the Linux > model are inherently incapable of supporting this scope). > > Basically, you'll have to play around a bit with it. > > -- Terry > -- Dan Eischen