Date: Sat, 31 Oct 98 13:56:47 -0600 From: "Richard Seaman, Jr." <lists@tar.com> To: "Mike Smith" <mike@smith.net.au> Cc: "info@highwind.com" <info@highwind.com>, "eischen@vigrid.com" <eischen@vigrid.com>, "current@freebsd.org" <current@FreeBSD.ORG> Subject: Re: Kernel threading (was Re: Thread Scheduler bug) Message-ID: <199810311956.NAA23144@ns.tar.com>
next in thread | raw e-mail | index | archive | help
On Thu, 29 Oct 1998 14:46:14 -0800, Mike Smith wrote: >> Of course, it we had kernel threads, the pthreads code would be >> a *lot* simpler and maybe less prone to bugs, the kernel would >> do the preemption for us, and context switches would be much >> faster than the current user thread implementation. :) >> >> I've been poking around in the code, and I'd guess that a >> uniprocessor kernel threads implementation wouldn't involve >> all that much work. However, I understand there's are fair >> amount of kernel work that needs to be done for SMP kernel >> threads. > >My suggestion to you: get started. Once you run into the SMP-related >issues, you'll find that there are people that can help you. But if >you wait for an SMP-kernel-thread-guru to materialise from nowhere, >we're never going to get anywhere. Here's what my "poking around" amounts to at this point. I'd be willing to carry it further, if there's interest. But, I also think that others want to see a somewhat different direction (ie. mixed user/kernel threads). I've grafted some of John Dyson's kernel threading code into the existing pthread code: 1) pthread_create will create a kernel thread via a call to Dyson's thrfork call, which in turn calls rfork. 2) pthread wait calls (eg. mutex wait, cond wait, etc) use the syscall thr_sleep 3) pthread wakeup calls (eg. in mutex and cond unlocks) use the syscall thr_wakeup 4) sched_yield and pthread_yield use the syscall yield, though when _POSIX_PRIORITY_SCHEDULING is enabled in the kernel, it would be easy enough to use the sched_yield syscall. I've only tested a handful of programs, and I know there's more work to be done to put this into any kind of shape for general testing. However, I can at least verify the following: 1) It runs my benchmark programs correctly that I was using when I found a few of the user thread scheduler bugs. It appears that context switches are 2-3 times faster than the current implemenation of user threads, maybe more in some circumstances. (This is contrary to conventional wisdom that user threads should be faster, but the reasons have been discussed on this list recently). 2) Highwinds schedBug.C test code runs as I think Highwinds intends it. 3) Highwinds condBug.C test code runs correctly. Things the current code doesn't do: 1) Lots, probably. It hasn't been tested, really. 2) Mixed user/kernel threads. I haven't even attempted to code for this. (AFAIK you can compile one app against the user thread library and another against the kernel thread library and run them both at the same time. You just can't mix user and kernel threads in the same app). 3) SMP kernel threads. I haven't actually tried this, but my reading of the rfork code is that the RFMEM option needed for kernel threads is not currently implemented for SMP kernels. John Dyson has also listed other SMP kernel issues that need to be addressed for SMP kernel threads. However, I'm not aware of much, if any SMP specific coding that needs to be done in the pthreads library itself. The only possible areas I know of might be enhanced spinlock handling, and the question of whether some kind of memory barrier instruction is needed in the mutex locking code. 4) sigwait. Also signal handling is untested and no doubt needs work. 5) I think more work needs to be done for proper exit and termination handling. This is largely unexplored, though pthread_join and pthread_exit at least "work", ie. threads get joined and/or terminated. Whether all the proper cleanup gets done is untested. My general strategy was this. I took the existing libc_r code and copied it to a new directory. The "uthread" code was split into 2 directories: 1) code that is user thread specific. I have modified this code only very slightly so that the interfaces to the "generic" code is the same for user and kernel threads. Also, I've added code for a bug fix or two, and added code for deferred cancelation points in some of the wrapped syscalls. 2) code that is "generic" or could easily be made "generic", where what I mean by generic is that it could be used for either kernel or user thread libraries. I've made some changes to this code in addition to making the interfaces to the kernel/user schedulers consistent. a) I've coded, but not tested, pthread_mutexattr type interfaces and altered the mutex code to handle the 4 mutex types in the XSH5 spec. I've also tried to bring the mutex error checking up to spec, and made the default mutex type handling conform to the spec. b) Added added error checks to the pthread_cond code as indicated in the spec. c) I've coded, but not tested, pthread_canel interfaces and added deferred cancellation points to syscalls as indicated in the specs. Asynchronous cancellation coding still needs to be done. d) The _POSIX_THREAD_PRIORITY_SCHEDULING interfaces are coded but untested. Then, I've created a third directory which contains the kernel thread specific code. I've preserved separate make files so that both user thread and kernel thread libraries can be generated useing the same "generic" pthread code. If this general approach is of any interest, let me know. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199810311956.NAA23144>