From owner-freebsd-threads@FreeBSD.ORG Mon Apr 26 05:42:05 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 93C2E16A4CE; Mon, 26 Apr 2004 05:42:05 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7496F43D5A; Mon, 26 Apr 2004 05:42:05 -0700 (PDT) (envelope-from davidxu@freebsd.org) Received: from freebsd.org (davidxu@localhost [127.0.0.1]) i3QCg1BO024129; Mon, 26 Apr 2004 05:42:03 -0700 (PDT) (envelope-from davidxu@freebsd.org) Message-ID: <408D0373.8050006@freebsd.org> Date: Mon, 26 Apr 2004 20:41:23 +0800 From: David Xu User-Agent: Mozilla Thunderbird 0.5 (X11/20040417) X-Accept-Language: en-us, en MIME-Version: 1.0 To: John Baldwin References: <200404231357.23096.jhb@FreeBSD.org> In-Reply-To: <200404231357.23096.jhb@FreeBSD.org> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: threads@freebsd.org Subject: Re: kse_release and kse_wakeup problem (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Apr 2004 12:42:05 -0000 John Baldwin wrote: > On Thursday 22 April 2004 06:02 pm, Daniel Eischen wrote: > >>Sorry, I should have included threads@. >> >>On Thu, 22 Apr 2004, John Baldwin wrote: >> >>>On Thursday 22 April 2004 09:01 am, Daniel Eischen wrote: >>> >>>>What do you guys think of this patch? >>> >>>I think that the thread code should check for the upcall case the same >>>way >> >>The thread code where? Everywhere msleep() is called? > > > No, in sleepq_catch_signals() just as I quoted below: > > >>>that we check for signals by calliing cursig() in sleepq_catch_signals(), >>>that is: >>> >>> /* Mark thread as being in an interruptible sleep. */ >>> mtx_lock_spin(&sched_lock); >>> MPASS(TD_ON_SLEEPQ(td)); >>> td->td_flags |= TDF_SINTR; >>> mtx_unlock_spin(&sched_lock); >>> sleepq_release(wchan); >>> >>> /* See if there are any pending signals for this thread. */ >>> PROC_LOCK(p); >>> mtx_lock(&p->p_sigacts->ps_mtx); >>> sig = cursig(td); >>> mtx_unlock(&p->p_sigacts->ps_mtx); >>> if (sig == 0 && thread_suspend_check(1)) >>> sig = SIGSTOP; >>> PROC_UNLOCK(p); >>> >>> /* >>> * If there were pending signals and this thread is still on >>> * the sleep queue, remove it from the sleep queue. >>> */ >>> >>>The thread_suspend_check() code should also be checking for the UPCALL >>>case and as well. Maybe something like: >>> >>> if (sig == 0 && thread_suspend_check(1)) >>> sig == SIGSTOP; >>> if (sig == 0 && thread_upcall_check()) >>> doing_upcall = 1; >>> >>>and then dequeue if we are doing an upcall just like we do if there is >>>already a signal. This mirrors the way we handle signals since UPCALLs >>>are a kind of special cased signal. The patch below has incorrect >>>locking (td_flags could get trashed) by the way. > > > I.e. do the upcall check in sleepq_catch_signals() right where you already do > thread_suspend_check(1). The only reason you have to do this, btw, is > because the kse_release() code is trying to mess with thread state internals > using sleepq_abort(), etc. The other in-kernel code that does that (signals) > already does the check in sleepq_catch_signals() and has done the same type > of check in msleep()/tsleep() for quite a while. > > If the kse_release() stuff was just using sleep/wakeup() rather than trying to > manually abort sleeps it wouldn't have to be so intimate with the sleep > interface. > > Note that thr's thr_wakeup() and thr_sleep() manage to simulate > synchronization w/o having to abort sleeps, but it is probably also easier to > do that than for the M:N case. > I think libthr will encounters same problem as libpthread with new sleep queue code, because mtx is released too early in msleep before thread markes itself as ON_SLEEPQ, thr_suspend and thr_wakeup have same race window as kse_release and kse_wakeup. Any code wants to put synchronous bit in td_flags like these codes will be broken. David Xu From owner-freebsd-threads@FreeBSD.ORG Mon Apr 26 07:55:44 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 16B1816A4CE; Mon, 26 Apr 2004 07:55:44 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id A003E43D5E; Mon, 26 Apr 2004 07:55:43 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.10/8.12.1) with ESMTP id i3QEtdQk023465; Mon, 26 Apr 2004 10:55:39 -0400 (EDT) Date: Mon, 26 Apr 2004 10:55:39 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: David Xu In-Reply-To: <408D0373.8050006@freebsd.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org cc: John Baldwin Subject: Re: kse_release and kse_wakeup problem (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Apr 2004 14:55:44 -0000 On Mon, 26 Apr 2004, David Xu wrote: > John Baldwin wrote: > > > > > > I.e. do the upcall check in sleepq_catch_signals() right where you already do > > thread_suspend_check(1). The only reason you have to do this, btw, is > > because the kse_release() code is trying to mess with thread state internals > > using sleepq_abort(), etc. The other in-kernel code that does that (signals) > > already does the check in sleepq_catch_signals() and has done the same type > > of check in msleep()/tsleep() for quite a while. > > > > If the kse_release() stuff was just using sleep/wakeup() rather than trying to > > manually abort sleeps it wouldn't have to be so intimate with the sleep > > interface. > > > > Note that thr's thr_wakeup() and thr_sleep() manage to simulate > > synchronization w/o having to abort sleeps, but it is probably also easier to > > do that than for the M:N case. > > > > I think libthr will encounters same problem as libpthread with new sleep > queue code, because mtx is released too early in msleep before thread > markes itself as ON_SLEEPQ, thr_suspend and thr_wakeup have same race > window as kse_release and kse_wakeup. Any code wants to put synchronous > bit in td_flags like these codes will be broken. I'm experimenting with adding an wakeup_thread() to kern_thread.c (to complement wakeup() and wakeup_one()). If we shouldn't be using sleepq's directly, the thread code either needs to a) queue msleep()'ing upcalls/threads itself having them all block on on their own unique wchan's; or b) use a wakeup_thread() that wakes up a specific thread. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Mon Apr 26 10:38:11 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7865E16A4CE; Mon, 26 Apr 2004 10:38:11 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 289B043D5E; Mon, 26 Apr 2004 10:38:11 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.10/8.12.1) with ESMTP id i3QHc9Qk010299; Mon, 26 Apr 2004 13:38:09 -0400 (EDT) Date: Mon, 26 Apr 2004 13:38:09 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: David Xu In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org cc: John Baldwin Subject: Re: kse_release and kse_wakeup problem (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Apr 2004 17:38:11 -0000 On Mon, 26 Apr 2004, Daniel Eischen wrote: > On Mon, 26 Apr 2004, David Xu wrote: > > > John Baldwin wrote: > > > > > > > > > I.e. do the upcall check in sleepq_catch_signals() right where you already do > > > thread_suspend_check(1). The only reason you have to do this, btw, is > > > because the kse_release() code is trying to mess with thread state internals > > > using sleepq_abort(), etc. The other in-kernel code that does that (signals) > > > already does the check in sleepq_catch_signals() and has done the same type > > > of check in msleep()/tsleep() for quite a while. > > > > > > If the kse_release() stuff was just using sleep/wakeup() rather than trying to > > > manually abort sleeps it wouldn't have to be so intimate with the sleep > > > interface. > > > > > > Note that thr's thr_wakeup() and thr_sleep() manage to simulate > > > synchronization w/o having to abort sleeps, but it is probably also easier to > > > do that than for the M:N case. > > > > > > > I think libthr will encounters same problem as libpthread with new sleep > > queue code, because mtx is released too early in msleep before thread > > markes itself as ON_SLEEPQ, thr_suspend and thr_wakeup have same race > > window as kse_release and kse_wakeup. Any code wants to put synchronous > > bit in td_flags like these codes will be broken. > > I'm experimenting with adding an wakeup_thread() to kern_thread.c > (to complement wakeup() and wakeup_one()). If we shouldn't be > using sleepq's directly, the thread code either needs to > > a) queue msleep()'ing upcalls/threads itself having them > all block on on their own unique wchan's; or > > b) use a wakeup_thread() that wakes up a specific thread. Sorry, patch for b) is at: http://people.freebsd.org/~deischen/sys.diffs I don't like using upcall flags, though. It seems to require taking the scheduler lock to fiddle with them and that adds some overhead. Perhaps add another field so we only need to use the proc lock. There was also a bug in kse_release() where wakeup_one() is called with the sched_lock held (fixed in above patch). -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Mon Apr 26 11:01:34 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D6A9116A4D0 for ; Mon, 26 Apr 2004 11:01:34 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id B884F43D5C for ; Mon, 26 Apr 2004 11:01:34 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) i3QI1Ybc045487 for ; Mon, 26 Apr 2004 11:01:34 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.12.11/8.12.11/Submit) id i3QI1XKr045481 for freebsd-threads@freebsd.org; Mon, 26 Apr 2004 11:01:33 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Date: Mon, 26 Apr 2004 11:01:33 -0700 (PDT) Message-Id: <200404261801.i3QI1XKr045481@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: freebsd-threads@FreeBSD.org Subject: Current problem reports assigned to you X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Apr 2004 18:01:35 -0000 Current FreeBSD problem reports Critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/06/13] kern/19247 threads uthread_sigaction.c does not do anything s [2004/03/15] kern/64313 threads FreeBSD (OpenBSD) pthread implicit set/un o [2004/04/22] threads/65883threads libkse's sigwait does not work after fork 3 problems total. Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/07/18] kern/20016 threads pthreads: Cannot set scheduling timer/Can o [2000/08/26] misc/20861 threads libc_r does not honor socket timeouts o [2001/01/19] bin/24472 threads libc_r does not honor SO_SNDTIMEO/SO_RCVT o [2001/01/25] bin/24632 threads libc_r delicate deviation from libc in ha o [2001/01/25] misc/24641 threads pthread_rwlock_rdlock can deadlock o [2001/11/26] bin/32295 threads pthread dont dequeue signals o [2002/02/01] i386/34536 threads accept() blocks other threads o [2002/05/25] kern/38549 threads the procces compiled whith pthread stoppe o [2002/06/27] bin/39922 threads [PATCH?] Threaded applications executed w o [2002/08/04] misc/41331 threads Pthread library open sets O_NONBLOCK flag o [2003/03/02] bin/48856 threads Setting SIGCHLD to SIG_IGN still leaves z o [2003/03/10] bin/49087 threads Signals lost in programs linked with libc a [2003/04/08] bin/50733 threads buildworld won't build, because of linkin o [2003/05/07] bin/51949 threads thread in accept cannot be cancelled 14 problems total. Non-critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/05/25] misc/18824 threads gethostbyname is not thread safe o [2000/10/21] misc/22190 threads A threaded read(2) from a socketpair(2) f o [2001/09/09] bin/30464 threads pthread mutex attributes -- pshared o [2002/05/02] bin/37676 threads libc_r: msgsnd(), msgrcv(), pread(), pwri s [2002/07/16] misc/40671 threads pthread_cancel doesn't remove thread from 5 problems total. From owner-freebsd-threads@FreeBSD.ORG Mon Apr 26 16:30:45 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EFAF116A4CE; Mon, 26 Apr 2004 16:30:45 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id CEC9D43D2D; Mon, 26 Apr 2004 16:30:45 -0700 (PDT) (envelope-from davidxu@freebsd.org) Received: from freebsd.org (davidxu@localhost [127.0.0.1]) i3QNUiYF087972; Mon, 26 Apr 2004 16:30:44 -0700 (PDT) (envelope-from davidxu@freebsd.org) Message-ID: <408D9B7E.2080804@freebsd.org> Date: Tue, 27 Apr 2004 07:30:06 +0800 From: David Xu User-Agent: Mozilla Thunderbird 0.5 (X11/20040417) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Daniel Eischen References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: threads@freebsd.org cc: John Baldwin Subject: Re: kse_release and kse_wakeup problem (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Apr 2004 23:30:46 -0000 Daniel Eischen wrote: > On Mon, 26 Apr 2004, Daniel Eischen wrote: > > >>On Mon, 26 Apr 2004, David Xu wrote: >> >> >>>John Baldwin wrote: >>> >>>> >>>>I.e. do the upcall check in sleepq_catch_signals() right where you already do >>>>thread_suspend_check(1). The only reason you have to do this, btw, is >>>>because the kse_release() code is trying to mess with thread state internals >>>>using sleepq_abort(), etc. The other in-kernel code that does that (signals) >>>>already does the check in sleepq_catch_signals() and has done the same type >>>>of check in msleep()/tsleep() for quite a while. >>>> >>>>If the kse_release() stuff was just using sleep/wakeup() rather than trying to >>>>manually abort sleeps it wouldn't have to be so intimate with the sleep >>>>interface. >>>> >>>>Note that thr's thr_wakeup() and thr_sleep() manage to simulate >>>>synchronization w/o having to abort sleeps, but it is probably also easier to >>>>do that than for the M:N case. >>>> >>> >>>I think libthr will encounters same problem as libpthread with new sleep >>>queue code, because mtx is released too early in msleep before thread >>>markes itself as ON_SLEEPQ, thr_suspend and thr_wakeup have same race >>>window as kse_release and kse_wakeup. Any code wants to put synchronous >>>bit in td_flags like these codes will be broken. >> >>I'm experimenting with adding an wakeup_thread() to kern_thread.c >>(to complement wakeup() and wakeup_one()). If we shouldn't be >>using sleepq's directly, the thread code either needs to >> >> a) queue msleep()'ing upcalls/threads itself having them >> all block on on their own unique wchan's; or >> >> b) use a wakeup_thread() that wakes up a specific thread. > > > Sorry, patch for b) is at: > > http://people.freebsd.org/~deischen/sys.diffs > > I don't like using upcall flags, though. It seems to require taking > the scheduler lock to fiddle with them and that adds some overhead. > Perhaps add another field so we only need to use the proc lock. > > There was also a bug in kse_release() where wakeup_one() is called > with the sched_lock held (fixed in above patch). > The patch is fine for me. if you don't like sched_lock, you can create another field and use proc lock. David Xu From owner-freebsd-threads@FreeBSD.ORG Tue Apr 27 06:47:10 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E095A16A4CE for ; Tue, 27 Apr 2004 06:47:10 -0700 (PDT) Received: from mail5.speakeasy.net (mail5.speakeasy.net [216.254.0.205]) by mx1.FreeBSD.org (Postfix) with ESMTP id B515643D64 for ; Tue, 27 Apr 2004 06:47:10 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 32559 invoked from network); 27 Apr 2004 13:47:10 -0000 Received: from dsl027-160-063.atl1.dsl.speakeasy.net (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) encrypted SMTP for ; 27 Apr 2004 13:47:10 -0000 Received: from slimer.baldwin.cx (slimer.baldwin.cx [192.168.0.16]) by server.baldwin.cx (8.12.11/8.12.11) with ESMTP id i3RDknLF007021; Tue, 27 Apr 2004 09:46:52 -0400 (EDT) (envelope-from jhb@FreeBSD.org) From: John Baldwin To: David Xu Date: Tue, 27 Apr 2004 09:43:56 -0400 User-Agent: KMail/1.6 References: <200404231357.23096.jhb@FreeBSD.org> <408D0373.8050006@freebsd.org> In-Reply-To: <408D0373.8050006@freebsd.org> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200404270943.56789.jhb@FreeBSD.org> X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on server.baldwin.cx cc: threads@FreeBSD.org Subject: Re: kse_release and kse_wakeup problem (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Apr 2004 13:47:11 -0000 On Monday 26 April 2004 08:41 am, David Xu wrote: > I think libthr will encounters same problem as libpthread with new sleep > queue code, because mtx is released too early in msleep before thread > markes itself as ON_SLEEPQ, thr_suspend and thr_wakeup have same race > window as kse_release and kse_wakeup. Any code wants to put synchronous > bit in td_flags like these codes will be broken. Nope. libthr msleep's on PROC_LOCK. It can do this because it holds both sched_lock and proc_lock when writing to its thread flag, so it can hold just the proc_lock for reading the flag, including msleep/wakeup. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve" = http://www.FreeBSD.org From owner-freebsd-threads@FreeBSD.ORG Tue Apr 27 06:47:11 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6FD9216A4CE for ; Tue, 27 Apr 2004 06:47:11 -0700 (PDT) Received: from mail6.speakeasy.net (mail6.speakeasy.net [216.254.0.206]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3A2C943D39 for ; Tue, 27 Apr 2004 06:47:11 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 21803 invoked from network); 27 Apr 2004 13:47:10 -0000 Received: from dsl027-160-063.atl1.dsl.speakeasy.net (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) encrypted SMTP for ; 27 Apr 2004 13:47:10 -0000 Received: from slimer.baldwin.cx (slimer.baldwin.cx [192.168.0.16]) by server.baldwin.cx (8.12.11/8.12.11) with ESMTP id i3RDknLG007021; Tue, 27 Apr 2004 09:47:08 -0400 (EDT) (envelope-from jhb@FreeBSD.org) From: John Baldwin To: Daniel Eischen Date: Tue, 27 Apr 2004 09:47:08 -0400 User-Agent: KMail/1.6 References: In-Reply-To: MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200404270947.08523.jhb@FreeBSD.org> X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on server.baldwin.cx cc: threads@FreeBSD.org cc: David Xu Subject: Re: kse_release and kse_wakeup problem (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Apr 2004 13:47:11 -0000 On Monday 26 April 2004 01:38 pm, Daniel Eischen wrote: > On Mon, 26 Apr 2004, Daniel Eischen wrote: > > On Mon, 26 Apr 2004, David Xu wrote: > > > John Baldwin wrote: > > > > I.e. do the upcall check in sleepq_catch_signals() right where you > > > > already do thread_suspend_check(1). The only reason you have to do > > > > this, btw, is because the kse_release() code is trying to mess with > > > > thread state internals using sleepq_abort(), etc. The other > > > > in-kernel code that does that (signals) already does the check in > > > > sleepq_catch_signals() and has done the same type of check in > > > > msleep()/tsleep() for quite a while. > > > > > > > > If the kse_release() stuff was just using sleep/wakeup() rather than > > > > trying to manually abort sleeps it wouldn't have to be so intimate > > > > with the sleep interface. > > > > > > > > Note that thr's thr_wakeup() and thr_sleep() manage to simulate > > > > synchronization w/o having to abort sleeps, but it is probably also > > > > easier to do that than for the M:N case. > > > > > > I think libthr will encounters same problem as libpthread with new > > > sleep queue code, because mtx is released too early in msleep before > > > thread markes itself as ON_SLEEPQ, thr_suspend and thr_wakeup have same > > > race window as kse_release and kse_wakeup. Any code wants to put > > > synchronous bit in td_flags like these codes will be broken. > > > > I'm experimenting with adding an wakeup_thread() to kern_thread.c > > (to complement wakeup() and wakeup_one()). If we shouldn't be > > using sleepq's directly, the thread code either needs to > > > > a) queue msleep()'ing upcalls/threads itself having them > > all block on on their own unique wchan's; or > > > > b) use a wakeup_thread() that wakes up a specific thread. > > Sorry, patch for b) is at: > > http://people.freebsd.org/~deischen/sys.diffs Erm, does sleepq_signal_thread() do anything different than sleepq_remove() (removes a thread from a specified wait channel if and only if the thread is sleeping on that wait channel)? -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve" = http://www.FreeBSD.org From owner-freebsd-threads@FreeBSD.ORG Tue Apr 27 07:47:25 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A832516A4CF; Tue, 27 Apr 2004 07:47:25 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3982943D41; Tue, 27 Apr 2004 07:47:25 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.10/8.12.1) with ESMTP id i3RElLQk025312; Tue, 27 Apr 2004 10:47:21 -0400 (EDT) Date: Tue, 27 Apr 2004 10:47:21 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: John Baldwin In-Reply-To: <200404270947.08523.jhb@FreeBSD.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@FreeBSD.org cc: David Xu Subject: Re: kse_release and kse_wakeup problem (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Apr 2004 14:47:25 -0000 On Tue, 27 Apr 2004, John Baldwin wrote: > On Monday 26 April 2004 01:38 pm, Daniel Eischen wrote: > > On Mon, 26 Apr 2004, Daniel Eischen wrote: > > > > > > I'm experimenting with adding an wakeup_thread() to kern_thread.c > > > (to complement wakeup() and wakeup_one()). If we shouldn't be > > > using sleepq's directly, the thread code either needs to > > > > > > a) queue msleep()'ing upcalls/threads itself having them > > > all block on on their own unique wchan's; or > > > > > > b) use a wakeup_thread() that wakes up a specific thread. > > > > Sorry, patch for b) is at: > > > > http://people.freebsd.org/~deischen/sys.diffs > > Erm, does sleepq_signal_thread() do anything different than sleepq_remove() > (removes a thread from a specified wait channel if and only if the thread is > sleeping on that wait channel)? I guess not. I thought we would have to search the list of threads to ensure it was queued. I've updated the patch slightly -- added thread_upcall_check() and changed where the new thread flags are stored: http://people.freebsd.org/~deischen/sys.diffs If I remove sleepq_signal_thread() and use sleepq_remove() instead, does the patch look OK to you? Thanks, -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Tue Apr 27 10:37:15 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EC99A16A4CF for ; Tue, 27 Apr 2004 10:37:15 -0700 (PDT) Received: from mail4.speakeasy.net (mail4.speakeasy.net [216.254.0.204]) by mx1.FreeBSD.org (Postfix) with ESMTP id A8CAE43D77 for ; Tue, 27 Apr 2004 10:37:15 -0700 (PDT) (envelope-from jhb@FreeBSD.org) Received: (qmail 18492 invoked from network); 27 Apr 2004 17:37:14 -0000 Received: from dsl027-160-063.atl1.dsl.speakeasy.net (HELO server.baldwin.cx) ([216.27.160.63]) (envelope-sender ) encrypted SMTP for ; 27 Apr 2004 17:37:14 -0000 Received: from 10.50.40.205 (gw1.twc.weather.com [216.133.140.1]) by server.baldwin.cx (8.12.11/8.12.11) with ESMTP id i3RHb8R3008406; Tue, 27 Apr 2004 13:37:11 -0400 (EDT) (envelope-from jhb@FreeBSD.org) From: John Baldwin To: Daniel Eischen Date: Tue, 27 Apr 2004 13:34:39 -0400 User-Agent: KMail/1.6 References: In-Reply-To: MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200404271334.39709.jhb@FreeBSD.org> X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on server.baldwin.cx cc: threads@FreeBSD.org cc: David Xu Subject: Re: kse_release and kse_wakeup problem (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Apr 2004 17:37:16 -0000 On Tuesday 27 April 2004 10:47 am, Daniel Eischen wrote: > On Tue, 27 Apr 2004, John Baldwin wrote: > > On Monday 26 April 2004 01:38 pm, Daniel Eischen wrote: > > > On Mon, 26 Apr 2004, Daniel Eischen wrote: > > > > I'm experimenting with adding an wakeup_thread() to kern_thread.c > > > > (to complement wakeup() and wakeup_one()). If we shouldn't be > > > > using sleepq's directly, the thread code either needs to > > > > > > > > a) queue msleep()'ing upcalls/threads itself having them > > > > all block on on their own unique wchan's; or > > > > > > > > b) use a wakeup_thread() that wakes up a specific thread. > > > > > > Sorry, patch for b) is at: > > > > > > http://people.freebsd.org/~deischen/sys.diffs > > > > Erm, does sleepq_signal_thread() do anything different than > > sleepq_remove() (removes a thread from a specified wait channel if and > > only if the thread is sleeping on that wait channel)? > > I guess not. I thought we would have to search the list of threads > to ensure it was queued. I've updated the patch slightly -- added > thread_upcall_check() and changed where the new thread flags are > stored: > > http://people.freebsd.org/~deischen/sys.diffs > > If I remove sleepq_signal_thread() and use sleepq_remove() instead, > does the patch look OK to you? Sure (looks like you already did). FWIW, I would just call sleepq_remove() directly rather than adding a wakeup_thread() function. sleepq_remove() is already used for a similar purpose in at least one other place (where we speed up the syncer). Thanks for working on this. -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve" = http://www.FreeBSD.org From owner-freebsd-threads@FreeBSD.ORG Thu Apr 29 10:39:29 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 51EDC16A4CE for ; Thu, 29 Apr 2004 10:39:29 -0700 (PDT) Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by mx1.FreeBSD.org (Postfix) with ESMTP id CFCDD43D46 for ; Thu, 29 Apr 2004 10:39:28 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([24.7.73.28]) by comcast.net (rwcrmhc13) with ESMTP id <2004042917392801500i0q48e>; Thu, 29 Apr 2004 17:39:28 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id KAA87566 for ; Thu, 29 Apr 2004 10:39:27 -0700 (PDT) X-Received: from mx2.freebsd.org (mx2.freebsd.org [216.136.204.119]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id IAA86487 for ; Thu, 29 Apr 2004 08:50:40 -0700 (PDT) X-Received: from hub.freebsd.org (hub.freebsd.org [216.136.204.18]) by mx2.freebsd.org (Postfix) with ESMTP id CE4D955672 for ; Thu, 29 Apr 2004 08:50:39 -0700 (PDT) (envelope-from owner-freebsd-current@freebsd.org) X-Received: by hub.freebsd.org (Postfix) id CFD1D16A50B; Thu, 29 Apr 2004 08:50:08 -0700 (PDT) Delivered-To: julian@freebsd.org X-Received: from hub.freebsd.org (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id B296B16A4FF; Thu, 29 Apr 2004 08:50:08 -0700 (PDT) Delivered-To: freebsd-current@freebsd.org X-Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5E56E16A4CE for ; Thu, 29 Apr 2004 08:49:55 -0700 (PDT) X-Received: from mail.a-quadrat.at (mail.a-quadrat.at [81.223.141.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id E2C6F43D5C for ; Thu, 29 Apr 2004 08:49:54 -0700 (PDT) (envelope-from mbretter@a-quadrat.at) X-Received: from localhost (localhost.a-quadrat.at [127.0.0.1]) by mail.a-quadrat.at (Postfix) with ESMTP id 0E11C5D025 for ; Thu, 29 Apr 2004 17:49:54 +0200 (CEST) X-Received: from mail.a-quadrat.at ([127.0.0.1]) by localhost (files.a-quadrat.at [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 37379-09 for ; Thu, 29 Apr 2004 17:49:52 +0200 (CEST) X-Received: from a-quadrat.at (files.a-quadrat.at [192.168.90.9]) by mail.a-quadrat.at (Postfix) with ESMTP id 909C75C3CA for ; Thu, 29 Apr 2004 17:49:52 +0200 (CEST) Message-ID: <4091241D.7020501@a-quadrat.at> Date: Thu, 29 Apr 2004 17:49:49 +0200 From: Michael Bretterklieber User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; de-AT; rv:1.6) Gecko/20040113 X-Accept-Language: de-at, de, en-us, en MIME-Version: 1.0 To: current@FreeBSD.org Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at a-quadrat.at X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Sender: owner-freebsd-current@freebsd.org Errors-To: owner-freebsd-current@freebsd.org ReSent-Date: Thu, 29 Apr 2004 10:39:18 -0700 (PDT) Resent-From: Julian Elischer Resent-To: threads@freebsd.org ReSent-Subject: libthr, libpthread problems ReSent-Message-ID: Subject: libthr, libpthread problems X-BeenThere: freebsd-threads@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Apr 2004 17:39:29 -0000 Hi, currently I'm testing Mpd-4 on -current and It ran into deadlock, therefore I changed the lib mapping via libmap.conf. When using libc_r everything is ok, when using libpthread a deadlock occurs under some circumstances, when using libthr, then: elvis:/home/mbretter/mpd/src# ./mpd Multi-link PPP for FreeBSD, by Archie L. Cobbs. Based on iij-ppp, by Toshiharu OHNO. The mutex recurse count cannot be less than zero Abnormal termination, file: /usr/src/lib/libthr/thread/thr_mutex.c, line: 677 Abort trap (core dumped) #0 0x282dd48b in thr_kill () from /lib/libc.so.5 (gdb) bt #0 0x282dd48b in thr_kill () from /lib/libc.so.5 #1 0x28287e9f in _pthread_kill (pthread=0x808e000, sig=6) at /usr/src/lib/libthr/thread/thr_sig.c:94 #2 0x28287aab in _raise (sig=6) at /usr/src/lib/libthr/thread/thr_syscalls.c:282 #3 0x2833ff4f in abort () from /lib/libc.so.5 #4 0x2828a9bf in mutex_unlock_common (mutex=0x281460b0, add_reference=0) at /usr/src/lib/libthr/thread/thr_mutex.c:712 #5 0x2828a664 in __pthread_mutex_unlock (mutex=0x281460b0) at /usr/src/lib/libthr/thread/thr_mutex.c:587 #6 0x28127251 in typed_mem_realloc () from /usr/local/lib/libpdel.so.0 #7 0x281275b2 in typed_mem_strdup () from /usr/local/lib/libpdel.so.0 #8 0x28127672 in typed_mem_vasprintf () from /usr/local/lib/libpdel.so.0 #9 0x2812b4b1 in valog () from /usr/local/lib/libpdel.so.0 #10 0x0806b610 in vlogprintf (fmt=0x0, ap=0x0) at log.c:516 #11 0x0806af9e in LogPrintf (fmt=0x80824c0 "mpd: pid %lu, version %s") at log.c:273 #12 0x0806ba56 in Greetings () at main.c:249 #13 0x0806b71a in main (ac=-1077941372, av=0xbfbfecf0) at main.c:159 #14 0x0804b23e in _start () any ideas, is this a bug in the application itself or libthr specific? If I'm using libpthread, then it works, but sometimes Mpd gets deadlocked (after terminating a thread and acquiring Mpd's internal "giant mutex"). elvis:/home/mbretter/mpd/src# ps -p 7558 -H PID TT STAT TIME COMMAND 7558 p0 SL 0:00.12 ./mpd 7558 p0 SL 0:00.12 ./mpd 7558 p0 SL 0:00.12 ./mpd and state of the process is "kserel". When Using libc_r everything works fine. I also tried to attach gdb to the running process (when Mpd was deadlocked), but this didn't worked, gdb als gets locked then. The machine has installed a -current dated of yesterday, using a non-smp kernel. bye, -- ------------------------------- ---------------------------------- Michael Bretterklieber - http://www.bretterklieber.com A-Quadrat Automation GmbH - http://www.a-quadrat.at Tel: ++43-(0)3172-41679 - GSM: ++43-(0)699 12861847 ------------------------------- ---------------------------------- "...the number of UNIX installations has grown to 10, with more expected..." - Dennis Ritchie and Ken Thompson, June 1972 _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-threads@FreeBSD.ORG Thu Apr 29 12:54:04 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C150F16A4CF; Thu, 29 Apr 2004 12:54:04 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 353B043D39; Thu, 29 Apr 2004 12:54:04 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.10/8.12.1) with ESMTP id i3TJroQk027209; Thu, 29 Apr 2004 15:53:51 -0400 (EDT) Date: Thu, 29 Apr 2004 15:53:50 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Michael Bretterklieber In-Reply-To: <4091241D.7020501@a-quadrat.at> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org cc: current@freebsd.org cc: Julian Elischer Subject: Re: libthr, libpthread problems X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Apr 2004 19:54:04 -0000 On Thu, 29 Apr 2004, Michael Bretterklieber wrote: > Hi, > > currently I'm testing Mpd-4 on -current and It ran into deadlock, > therefore I changed the lib mapping via libmap.conf. When using libc_r > everything is ok, when using libpthread a deadlock occurs under some > circumstances, when using libthr, then: > > elvis:/home/mbretter/mpd/src# ./mpd > Multi-link PPP for FreeBSD, by Archie L. Cobbs. > Based on iij-ppp, by Toshiharu OHNO. > The mutex recurse count cannot be less than zero Sounds like a bug in the application (recurse count shouldn't be less than zero). -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Fri Apr 30 08:48:14 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1ED0616A4CF for ; Fri, 30 Apr 2004 08:48:14 -0700 (PDT) Received: from rms04.rommon.net (rms04.rommon.net [212.54.2.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id C6E6743D5A for ; Fri, 30 Apr 2004 08:48:12 -0700 (PDT) (envelope-from pete@he.iki.fi) Received: from he.iki.fi (h91.vuokselantie10.fi [193.64.42.145]) by rms04.rommon.net (8.12.10/8.12.9) with ESMTP id i3UFmAmo016062 for ; Fri, 30 Apr 2004 18:48:10 +0300 (EEST) (envelope-from pete@he.iki.fi) Message-ID: <40927537.6070405@he.iki.fi> Date: Fri, 30 Apr 2004 18:48:07 +0300 From: Petri Helenius User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-threads@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Subject: system priorities X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Apr 2004 15:48:14 -0000 Am I correct in the observation that threads inherit their scheduling priority (nice and rtprio) from the first "main" thread in the process instead of the parent thread? I´m creating a few system scope threads from a rtprio main process and then dropping rtprio in the created threads and then creating more threads from them which seem to get rtprio from somewhere. Is there a specification reference on this and if not, would it make more sense to inherit them from the parent thread? Pete From owner-freebsd-threads@FreeBSD.ORG Fri Apr 30 14:04:04 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 932DD16A4CE for ; Fri, 30 Apr 2004 14:04:04 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3877043D58 for ; Fri, 30 Apr 2004 14:04:04 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.10/8.12.1) with ESMTP id i3UL42S8029132; Fri, 30 Apr 2004 17:04:03 -0400 (EDT) Date: Fri, 30 Apr 2004 17:04:02 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Petri Helenius In-Reply-To: <40927537.6070405@he.iki.fi> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=X-UNKNOWN Content-Transfer-Encoding: QUOTED-PRINTABLE cc: freebsd-threads@freebsd.org Subject: Re: system priorities X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Apr 2004 21:04:04 -0000 On Fri, 30 Apr 2004, Petri Helenius wrote: >=20 > Am I correct in the observation that threads inherit their scheduling=20 > priority (nice and rtprio) > from the first "main" thread in the process instead of the parent thread? As it stands now, newly created KSEGs inherit from the creating KSEG. The nice and rtprio priorites are stored in the KSEG. > I=B4m creating a few system scope threads from a rtprio main process and= =20 > then dropping rtprio in the created threads and then creating more=20 > threads from them which seem to get rtprio from somewhere. It looks like rtprio will set the priority of the first KSEG in the proc's list of KSEGs. This isn't necessarily the first KSEG created (main). In fact, newly created KSEGs get inserted onto the head of the list, so rtprio() may act on different KSEGs depending on if it is called before or after any KSEG creations. [ System scope threads =3D=3D new KSEG creation ] > Is there a specification reference on this and if not, would it make=20 > more sense to inherit them from the parent thread? I think they do inherit from the parent KSEG, but rtprio() may not be setting the priority for the current KSEG. I would really like a Solaris-like priocntl() so that you can specify which KSEG (or whatever thing that will hold the priority in the future) to act on. --=20 Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Fri Apr 30 14:49:11 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2580F16A4D0 for ; Fri, 30 Apr 2004 14:49:11 -0700 (PDT) Received: from hotmail.com (bay17-f36.bay17.hotmail.com [64.4.43.86]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5874E43D49 for ; Fri, 30 Apr 2004 14:49:10 -0700 (PDT) (envelope-from ikazdek@hotmail.com) Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC; Fri, 30 Apr 2004 14:25:44 -0700 Received: from 24.10.96.33 by by17fd.bay17.hotmail.msn.com with HTTP; Fri, 30 Apr 2004 21:25:44 GMT X-Originating-IP: [24.10.96.33] X-Originating-Email: [ikazdek@hotmail.com] X-Sender: ikazdek@hotmail.com From: "K A Z" To: freebsd-threads@freebsd.org Date: Fri, 30 Apr 2004 21:25:44 +0000 Mime-Version: 1.0 Content-Type: text/plain; format=flowed Message-ID: X-OriginalArrivalTime: 30 Apr 2004 21:25:44.0275 (UTC) FILETIME=[B2163230:01C42EF9] Subject: FreeBSD AMD64 + high performance MySQL - Possible? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Apr 2004 21:49:11 -0000 A person I know runs a very large forum using a dual AMD Opteron box running Linux for the MySQL database backend. I think he does several thousand conncurrent connections or something. I intend to do the same, and have the hardware sitting here to do it... But alas, I am a FreeBSD user and would like to stay that way for this database server if possible. I read a lot on Jeremy Zawodny's BLOG site (Yahoo's server god I guess) where he initially slammed MySQL on FreeBSD because of FreeBSD's poor thread support vs linux and then made several follow ups.... The first stating that if you compile MySQL on FreeBSD with inuxThreads support it will eliminate the poor threading issue. (Which, afaik, is not an option for me on AMD64) The second follow up he made was that FreeBSD 5.2 and on (for i386) use a new threading method (called?) that helps resolve FreeBSD's threading issues with MySQL. But I noticed in just last weeks archives for this mailing list that people were still discussing FreeBSD + MySQL problems... (presumably on intel architecture). Several weeks ago the MySQL port maintainer was given access to our Opteron machine in order to make optimizations the MySQL port for the AMD64 branch. But I think at the time he said the new threading was still being worked on. Actually here is his quote regarding benchmark results... "I tested libc_r and libpthread in different manners. I couldn't test linuxthreads since I cannot connect to the server anymore. Anyway, the fastest configuration on your machine is mysql compiled statically with libc_r. This may change if you upgrade your system to -current." So I guess my question is... does anyone out there know the best way to compile MySQL on an AMD64 box without too much (or any) performance loss over an identical linux machine? OR, If anyone from the FreeBSD/AMD64 dev team would like to use our hardware to optimize threading for MySQL on AMD64, and compare the results to a linux install (I have a linux install on identical drive we can swap out)... I would be more than willing to donate access to interested and qualified parties. The hardware is: Dual Opteron 241 CPUs, 2 Gigs ram (can purchase more ram) Tyan S2880UGNR w/ LSI Zero Channel Raid card 4x 15k rpm SCSI drives running 2 raid 0 channels. Swapable OS/system drive. I would love to be able to help compare and narrow the performance diferrences for MySQL on FreeBSD and Linux for the AMD64 platform. Thanks for any information or interest! _________________________________________________________________ Is your PC infected? Get a FREE online computer virus scan from McAfeeŽ Security. http://clinic.mcafee.com/clinic/ibuy/campaign.asp?cid=3963 From owner-freebsd-threads@FreeBSD.ORG Sat May 1 07:27:30 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 62B1116A4CF for ; Sat, 1 May 2004 07:27:30 -0700 (PDT) Received: from rms04.rommon.net (rms04.rommon.net [212.54.2.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id 56B4243D1F for ; Sat, 1 May 2004 07:27:29 -0700 (PDT) (envelope-from pete@he.iki.fi) Received: from he.iki.fi (h91.vuokselantie10.fi [193.64.42.145]) by rms04.rommon.net (8.12.10/8.12.9) with ESMTP id i41ERRmo020212; Sat, 1 May 2004 17:27:27 +0300 (EEST) (envelope-from pete@he.iki.fi) Message-ID: <4093B3CC.90502@he.iki.fi> Date: Sat, 01 May 2004 17:27:24 +0300 From: Petri Helenius User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Daniel Eischen References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-threads@freebsd.org Subject: Re: system priorities X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 May 2004 14:27:30 -0000 Daniel Eischen wrote: > >It looks like rtprio will set the priority of the first >KSEG in the proc's list of KSEGs. This isn't necessarily >the first KSEG created (main). In fact, newly created >KSEGs get inserted onto the head of the list, so rtprio() >may act on different KSEGs depending on if it is called >before or after any KSEG creations. > > > This sounds like a bug. Could it change the KSEG which called the rtprio so there would be some deterministic behaviour to the rtprio if used in threaded program? So I take it from this that rtprio is also broken for libthr? >I would really like a Solaris-like priocntl() so that you >can specify which KSEG (or whatever thing that will hold >the priority in the future) to act on. > > > I fail to understand the value of being able to point to other threads for priority setting? Or are you talking about a facility which would exist below the rtprio call? Pete From owner-freebsd-threads@FreeBSD.ORG Sat May 1 07:52:20 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A231716A4CE for ; Sat, 1 May 2004 07:52:20 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1773A43D2F for ; Sat, 1 May 2004 07:52:20 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.10/8.12.1) with ESMTP id i41EqJS8027345; Sat, 1 May 2004 10:52:19 -0400 (EDT) Date: Sat, 1 May 2004 10:52:19 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Petri Helenius In-Reply-To: <4093B3CC.90502@he.iki.fi> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: system priorities X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 May 2004 14:52:20 -0000 On Sat, 1 May 2004, Petri Helenius wrote: > Daniel Eischen wrote: > > > > >It looks like rtprio will set the priority of the first > >KSEG in the proc's list of KSEGs. This isn't necessarily > >the first KSEG created (main). In fact, newly created > >KSEGs get inserted onto the head of the list, so rtprio() > >may act on different KSEGs depending on if it is called > >before or after any KSEG creations. > > > > > > > This sounds like a bug. Could it change the KSEG which called the rtprio > so there would be some deterministic behaviour to the rtprio if used in > threaded program? So I take it from this that rtprio is also broken for > libthr? Yes. > >I would really like a Solaris-like priocntl() so that you > >can specify which KSEG (or whatever thing that will hold > >the priority in the future) to act on. > > > > > > > I fail to understand the value of being able to point to other threads > for priority setting? Or are you talking about a facility which would > exist below the rtprio call? The Solaris priocntl() lets you set/get: P_LWPID The id argument is an LWP ID. The priocntl function applies to the LWP with the specified ID within the calling process. P_PID The id argument is a process ID specifying a single process. The priocntl() function applies to all LWPs currently associated with the specified process. P_PPID The id argument is a parent process ID. The priocntl() function applies to all LWPs currently associated with processes with the specified parent process ID. P_PGID The id argument is a process group ID. The priocntl() function applies to all LWPs currently associated with processes in the specified process group. P_SID The id argument is a session ID. The priocntl() func- tion applies to all LWPs currently associated with processes in the specified session. P_TASKID The id argument is a task ID. The priocntl() function applies to all LWPs currently associated with processes in the specified task. P_CID The id argument is a class ID (returned by the priocntl() PC_GETCID command as explained below). The priocntl() function applies to all LWPs in the speci- fied class. P_UID The id argument is a user ID. The priocntl() function applies to all LWPs with this effective user ID. P_GID The id argument is a group ID. The priocntl() function applies to all LWPs with this effective group ID. P_PROJID The id argument is a project ID. The priocntl() func- tion applies to all LWPs with this project ID. P_ALL The priocntl() function applies to all existing LWPs. The value of id is ignored. The permission restric- tions described below still apply. P_MYID The calling LWP's LWP_ID, PPID, GID, SID, TASKID, CID, UID, GID, or PROJID... We probably don't need all these, but perhaps some of them could be useful. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Sat May 1 08:45:28 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9721816A4ED for ; Sat, 1 May 2004 08:45:28 -0700 (PDT) Received: from rms04.rommon.net (rms04.rommon.net [212.54.2.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8A85043D4C for ; Sat, 1 May 2004 08:45:27 -0700 (PDT) (envelope-from pete@he.iki.fi) Received: from he.iki.fi (h91.vuokselantie10.fi [193.64.42.145]) by rms04.rommon.net (8.12.10/8.12.9) with ESMTP id i41FjPmo020418; Sat, 1 May 2004 18:45:25 +0300 (EEST) (envelope-from pete@he.iki.fi) Message-ID: <4093C611.40600@he.iki.fi> Date: Sat, 01 May 2004 18:45:21 +0300 From: Petri Helenius User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Daniel Eischen References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit cc: freebsd-threads@freebsd.org Subject: Re: system priorities X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 May 2004 15:45:28 -0000 Daniel Eischen wrote: >>This sounds like a bug. Could it change the KSEG which called the rtprio >>so there would be some deterministic behaviour to the rtprio if used in >>threaded program? So I take it from this that rtprio is also broken for >>libthr? >> >> > >Yes. > > > Can and should it be fixed? > >The Solaris priocntl() lets you set/get: > > P_LWPID > The id argument is an LWP ID. The priocntl function > applies to the LWP with the specified ID within the > calling process. > > > Where would I get the "P_LWPID"? I´m not sure it's a good idea to expose the kse structure to an application using pthreads. Pete From owner-freebsd-threads@FreeBSD.ORG Sat May 1 09:09:20 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 64E3F16A4CE for ; Sat, 1 May 2004 09:09:20 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id E88B843D49 for ; Sat, 1 May 2004 09:09:19 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.10/8.12.1) with ESMTP id i41G9JS8017036; Sat, 1 May 2004 12:09:19 -0400 (EDT) Date: Sat, 1 May 2004 12:09:19 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Petri Helenius In-Reply-To: <4093C611.40600@he.iki.fi> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=X-UNKNOWN Content-Transfer-Encoding: QUOTED-PRINTABLE cc: freebsd-threads@freebsd.org Subject: Re: system priorities X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 May 2004 16:09:20 -0000 On Sat, 1 May 2004, Petri Helenius wrote: > Daniel Eischen wrote: >=20 > >>This sounds like a bug. Could it change the KSEG which called the rtpri= o=20 > >>so there would be some deterministic behaviour to the rtprio if used in= =20 > >>threaded program? So I take it from this that rtprio is also broken for= =20 > >>libthr? > >> =20 > >> > > > >Yes. > > > > =20 > > > Can and should it be fixed? Index: kern/kern_resource.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /opt/FreeBSD/cvs/src/sys/kern/kern_resource.c,v retrieving revision 1.136 diff -u -r1.136 kern_resource.c --- kern/kern_resource.c=0910 Apr 2004 11:08:16 -0000=091.136 +++ kern/kern_resource.c=091 May 2004 14:00:37 -0000 @@ -318,6 +318,7 @@ { =09struct proc *curp; =09register struct proc *p; +=09struct ksegrp *kg; =09struct rtprio rtp; =09int cierror, error; =20 @@ -328,6 +329,7 @@ =09=09cierror =3D 0; =20 =09curp =3D td->td_proc; +=09kg =3D td->td_ksegrp; =09if (uap->pid =3D=3D 0) { =09=09p =3D curp; =09=09PROC_LOCK(p); @@ -342,7 +344,7 @@ =09=09if ((error =3D p_cansee(td, p))) =09=09=09break; =09=09mtx_lock_spin(&sched_lock); -=09=09pri_to_rtp(FIRST_KSEGRP_IN_PROC(p), &rtp); +=09=09pri_to_rtp(kg, &rtp); =09=09mtx_unlock_spin(&sched_lock); =09=09PROC_UNLOCK(p); =09=09return (copyout(&rtp, uap->rtp, sizeof(struct rtprio))); @@ -373,7 +375,7 @@ =09=09=09} =09=09} =09=09mtx_lock_spin(&sched_lock); -=09=09error =3D rtp_to_pri(&rtp, FIRST_KSEGRP_IN_PROC(p)); +=09=09error =3D rtp_to_pri(&rtp, kg); =09=09mtx_unlock_spin(&sched_lock); =09=09break; =09default: Index: kern/kern_thread.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D RCS file: /opt/FreeBSD/cvs/src/sys/kern/kern_thread.c,v retrieving revision 1.176 diff -u -r1.176 kern_thread.c --- kern/kern_thread.c=0928 Apr 2004 20:36:53 -0000=091.176 +++ kern/kern_thread.c=091 May 2004 14:06:50 -0000 @@ -337,7 +337,7 @@ =09kg->kg_numupcalls =3D 0; =09/* link it in now that it's consistent */ =09p->p_numksegrps++; -=09TAILQ_INSERT_HEAD(&p->p_ksegrps, kg, kg_ksegrp); +=09TAILQ_INSERT_TAIL(&p->p_ksegrps, kg, kg_ksegrp); } =20 void > > > >The Solaris priocntl() lets you set/get: > > > > P_LWPID > > The id argument is an LWP ID. The priocntl function > > applies to the LWP with the specified ID within the > > calling process. > > > > =20 > > > Where would I get the "P_LWPID"? I=B4m not sure it's a good idea to expos= e=20 > the kse structure to an application using pthreads. There would be a way to get it, but you needn't if you use P_MYID. Solaris has _lwp_self(). One of the other things on my wish list is to be able to bind (soft affinity) KSEs to processors. --=20 Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Sat May 1 12:45:59 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EBD5816A4CE for ; Sat, 1 May 2004 12:45:59 -0700 (PDT) Received: from rms04.rommon.net (rms04.rommon.net [212.54.2.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id CC67043D1D for ; Sat, 1 May 2004 12:45:58 -0700 (PDT) (envelope-from pete@he.iki.fi) Received: from he.iki.fi (h91.vuokselantie10.fi [193.64.42.145]) by rms04.rommon.net (8.12.10/8.12.9) with ESMTP id i41Jjvmo021064; Sat, 1 May 2004 22:45:57 +0300 (EEST) (envelope-from pete@he.iki.fi) Message-ID: <4093FE70.7040305@he.iki.fi> Date: Sat, 01 May 2004 22:45:52 +0300 From: Petri Helenius User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.6) Gecko/20040113 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Daniel Eischen References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit cc: freebsd-threads@freebsd.org Subject: Re: system priorities X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 May 2004 19:46:00 -0000 Daniel Eischen wrote: Thanks, I´ll try the patch. >There would be a way to get it, but you needn't if you >use P_MYID. Solaris has _lwp_self(). One of the other >things on my wish list is to be able to bind (soft >affinity) KSEs to processors. > > > I would like to have KSE or thread processor affinity. Specially in HTT environments where I could assign the threads I know do the bulk of the work to separate physical CPU's and let the small ones run whereever they like ... Pete From owner-freebsd-threads@FreeBSD.ORG Sat May 1 13:02:37 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 884FF16A4CE for ; Sat, 1 May 2004 13:02:37 -0700 (PDT) Received: from mxsf20.cluster1.charter.net (mxsf20.cluster1.charter.net [209.225.28.220]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3321643D2D for ; Sat, 1 May 2004 13:02:36 -0700 (PDT) (envelope-from archie@dellroad.org) Received: from InterJet.dellroad.org (public.cpe.mvllo.al.charter.com [24.196.20.72])i41K05X3090758 for ; Sat, 1 May 2004 16:00:05 -0400 (EDT) Received: from arch20m.dellroad.org (arch20m.dellroad.org [10.2.2.20]) by InterJet.dellroad.org (8.9.1a/8.9.1) with ESMTP id OAA51039 for ; Sat, 1 May 2004 14:51:02 -0500 (CDT) Received: from arch20m.dellroad.org (localhost [127.0.0.1]) i41Jp08l002437 for ; Sat, 1 May 2004 14:51:00 -0500 (CDT) (envelope-from archie@arch20m.dellroad.org) Received: (from archie@localhost) by arch20m.dellroad.org (8.12.9p2/8.12.9/Submit) id i41Jp0I1002436 for freebsd-threads@freebsd.org; Sat, 1 May 2004 14:51:00 -0500 (CDT) (envelope-from archie) From: Archie Cobbs Message-ID: <200405011757.i41HvHrM001972@arch20m.dellroad.org> To: eischen@vigrid.com, julian@elischer.org Date: Sat, 1 May 2004 12:57:17 -0500 (CDT) X-Mailer: ELM [version 2.4ME+ PL99b (25)] MIME-Version: 1.0 Sender: archie@dellroad.org cc: mbretter@a-quadrat.at cc: freebsd-threads@freebsd.org Subject: Re: libthr, libpthread problems X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 May 2004 20:02:37 -0000 Dan Eischen wrote: > On Thu, 29 Apr 2004, Michael Bretterklieber wrote: > > currently I'm testing Mpd-4 on -current and It ran into deadlock, > > therefore I changed the lib mapping via libmap.conf. When using libc_r > > everything is ok, when using libpthread a deadlock occurs under some > > circumstances, when using libthr, then: > > > > elvis:/home/mbretter/mpd/src# ./mpd > > Multi-link PPP for FreeBSD, by Archie L. Cobbs. > > Based on iij-ppp, by Toshiharu OHNO. > > The mutex recurse count cannot be less than zero > > Sounds like a bug in the application (recurse count shouldn't > be less than zero). Nope.. here's a simpler test case that demonstrates the problem: $ cat xx.c #include #include #include static pthread_mutex_t mutex; int main(int argc, char **argv) { pthread_mutexattr_t mattr; pthread_mutexattr_init(&mattr); pthread_mutexattr_settype(&mattr, PTHREAD_MUTEX_RECURSIVE); pthread_mutex_init(&mutex, &mattr); pthread_mutexattr_destroy(&mattr); pthread_mutex_lock(&mutex); pthread_mutex_unlock(&mutex); return 0; } $ cc -g -Wall -o xx xx.o -pthread $ ./xx The mutex recurse count cannot be less than zero Abnormal termination, file: /usr/src/lib/libthr/thread/thr_mutex.c, line: 677 Abort $ uname -r 5.2-CURRENT $ ldd ./xx ./xx: libpthread.so.1 => /usr/lib/libthr.so.1 (0x2807b000) libc.so.5 => /lib/libc.so.5 (0x28094000) Seems to be PTHREAD_MUTEX_RECURSIVE-specific. -Archie __________________________________________________________________________ Archie Cobbs * CTO, Awarix * http://www.awarix.com From owner-freebsd-threads@FreeBSD.ORG Sat May 1 20:07:17 2004 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AE9F916A4CE for ; Sat, 1 May 2004 20:07:17 -0700 (PDT) Received: from ojoink.com (center.ojoink.com [216.65.123.180]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5D94E43D39 for ; Sat, 1 May 2004 20:07:17 -0700 (PDT) (envelope-from amd64list@jpgsworld.com) Received: (qmail 14299 invoked by uid 89); 29 Apr 2004 00:02:56 -0000 Received: from unknown (HELO MAINBX.jpgsworld.com) (amd64list@jpgsworld.com@67.172.167.223) by host180.ojoink.com with SMTP; 29 Apr 2004 00:02:56 -0000 Message-Id: <5.2.0.9.2.20040428163742.015072a0@mail.ojoink.com> X-Sender: amd64list@jpgsworld.com@mail.ojoink.com X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Wed, 28 Apr 2004 17:00:34 -0700 To: freebsd-threads@freebsd.org From: JG Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Subject: FreeBSD AMD64 + high performance MySQL - Possible? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 02 May 2004 03:07:17 -0000 A guy I know runs a very large forum using a dual AMD Opteron box running Linux for the MySQL database backend. I think he does several thousand conncurrent connections or something. I intend to do the same, and have the hardware sitting here to do it... But alas, I am a FreeBSD user and would like to stay that way for this database server if possible. I read a lot on Jeremy Zawodny's BLOG site (Yahoo's server god I guess) where he initially slammed MySQL on FreeBSD because of FreeBSD's poor thread support vs linux and then made several follow ups.... The first stating that if you compile MySQL on FreeBSD with inuxThreads support it will eliminate the poor threading issue. (Which, afaik, is not an option for me on AMD64) The second follow up he made was that FreeBSD 5.2 and on (for i386) use a new threading method (called?) that helps resolve FreeBSD's threading issues with MySQL. But I noticed in just last weeks archives that people were still discussing FreeBSD + MySQL problems... presumably on intel architecture. Several weeks ago the MySQL port maintainer Alex Dupre was given access to our Opteron machine in order to make optimizations the port for AMD64. But I think at the time he said the new threading was still being worked on or something. So I guess my question is... does anyone know the best way to compile MySQL on an AMD64 box without too much (or any) performance loss over an identical linux machine? If anyone from the FreeBSD/AMD64 dev team would like to use our hardware to optimize threading for MySQL on AMD64, and compare the results to a linux install (I have a linux install on identical drive we can swap out)... I would be more than willing to donate access to interested and qualified parties. Dual Opteron 241 CPUs, 2 Gigs ram. Tyan S2880UGNR w/ LSI Zero Channel Raid card 4x 15k rpm SCSI drives running 2 raid 0 channels. Swapable OS/system drive. Thanks for any information!