From owner-freebsd-smp Mon Jun 19 3:10:40 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 4834C37B7D7 for ; Mon, 19 Jun 2000 03:10:36 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id DAA76740; Mon, 19 Jun 2000 03:10:35 -0700 (PDT) (envelope-from dillon) Date: Mon, 19 Jun 2000 03:10:35 -0700 (PDT) From: Matthew Dillon Message-Id: <200006191010.DAA76740@apollo.backplane.com> To: freebsd-smp@freebsd.org Subject: Mutex/BSDI work update Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Ok, here's an update: * I got the mutex implementation mostly working * I got idle processes working (one per cpu) (page-zeroing is disabled at the moment). * All the mainline code is now using mutexes * The interrupt code still needs a lot of work, and probably hardclock too. (The interrupt code is the hardest part). * I figured out how to support spl*()'s. * I'm getting the stuff working with Single-processor builds first, and then will get it working on MP builds. The breakage in -current lost me some time, so I didn't get started until Sunday. The patch set isn't working well enough yet to give out, but I don't expect it to take long. I'll put together a synopsis tomorrow. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 9:13:56 2000 Delivered-To: freebsd-smp@freebsd.org Received: from Ilsa.StevesCafe.com (Ilsa.StevesCafe.com [206.168.13.65]) by hub.freebsd.org (Postfix) with ESMTP id BAF0F37B740 for ; Tue, 20 Jun 2000 09:13:53 -0700 (PDT) (envelope-from fbsd@Ilsa.StevesCafe.com) Received: from Ilsa.StevesCafe.com (localhost [127.0.0.1]) by Ilsa.StevesCafe.com (8.9.3/8.9.3) with ESMTP id KAA16723 for ; Tue, 20 Jun 2000 10:13:47 -0600 (MDT) (envelope-from fbsd@Ilsa.StevesCafe.com) Message-Id: <200006201613.KAA16723@Ilsa.StevesCafe.com> X-Mailer: exmh version 2.0.2 2/24/98 From: Steve Passe To: freebsd-smp@FreeBSD.ORG Subject: SMPng Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 20 Jun 2000 10:13:47 -0600 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi, For those of you not subscribing to current, check out: http://people.FreeBSD.org/~fsmp/SMP/ngannounce.html I will try to track all the ongoing activity from my main SMP page: http://people.FreeBSD.org/~fsmp/SMP/SMP.html -- Steve Passe | powered by smp@timing.com | Symmetric MultiProcessor FreeBSD To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 10: 8:37 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 512FD37BC2D; Tue, 20 Jun 2000 10:08:31 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id KAA87060; Tue, 20 Jun 2000 10:08:30 -0700 (PDT) (envelope-from dillon) Date: Tue, 20 Jun 2000 10:08:30 -0700 (PDT) From: Matthew Dillon Message-Id: <200006201708.KAA87060@apollo.backplane.com> To: freebsd-smp@freebsd.org Cc: Greg Lehey Subject: SMP discussion moving to freebsd-smp Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org All SMP/FreeBSD/BSDI work should move to the freebsd-smp list. (I've BCC'd this to -current). -- I now have mutexes 99% working in an SP build. I will be making SP (single processor) patch sets available tonight or tomorrow morning. (99% == I can boot the machine normally but after a while it locks up due to an interrupt nesting issue which I haven't dealt with yet). In order to make an MP build work, we need interrupt threads (either light or heavy weight). Greg, you expressed an interest in doing the light-weight interrupt threads. Do you want me to do the heavy-weight interrupt threads first? It's a lot of nasty assembly and it is a prerequisit for being able to do the light weight interrupt threads. I think once the heavy weight interrupt threads are done that the light weight interrupts can be implemented entirely in C. Right now the SP build works because I am allowing (unthreaded) interrupts to steal the idleproc's context, and that only works because they can get the giant mutex without blocking (remember, the BSDI giant mutex is a blocking mutex, not a spin mutex). In the MP build the interrupts need to be able to block getting the giant mutex which means we need to implement heavy-weight interrupt threads at the very least before we can get anything working, because we are not allowed to block in the idleproc. I believe I have done almost everything necessary to implement heavy weight interrupt threads, including moving the kthreads initialization code to an earlier point in the boot process (so much so that I think the device init stuff can run with working kernel processes rather then with all the delayed-thread stuff). I also think that people can start hacking on the code with the SP build once I fix this last little problem, then I can work on the MP build in parallel. The MP build requires a significant amount of additional work including having to rewrite most of the APIC assembly code (just as I had to rewrite most of the ICU assembly code for the SP build). It's another weekend at least for that. Using the SP build does not exercise all the MP characteristics of the mutexes but it exercises enough that it can be used to vett for dumb mistakes. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 10:16:29 2000 Delivered-To: freebsd-smp@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id DC5C137BF57 for ; Tue, 20 Jun 2000 10:16:26 -0700 (PDT) (envelope-from mjacob@feral.com) Received: from semuta.feral.com (semuta [192.67.166.70]) by feral.com (8.9.3/8.9.3) with ESMTP id KAA17217; Tue, 20 Jun 2000 10:16:24 -0700 Date: Tue, 20 Jun 2000 10:15:59 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: Matthew Dillon Cc: freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp In-Reply-To: <200006201708.KAA87060@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Matt- is this all on i386 now? I can start throwing some cycles into trying to clean up the alpha side- that is, if Doug isn't doing it already. -matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 10:56:35 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 8C84F37BFFD for ; Tue, 20 Jun 2000 10:54:36 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id KAA87277; Tue, 20 Jun 2000 10:54:34 -0700 (PDT) (envelope-from dillon) Date: Tue, 20 Jun 2000 10:54:34 -0700 (PDT) From: Matthew Dillon Message-Id: <200006201754.KAA87277@apollo.backplane.com> To: Matthew Jacob Cc: freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp References: Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org : :Matt- is this all on i386 now? I can start throwing some cycles into trying to :clean up the alpha side- that is, if Doug isn't doing it already. : :-matt Yes, all i386. At the meeting we decided to break the alpha port temporarily. I think alpha work can begin when I make the SP (single processor) patches available. I would recommend sticking to SP alpha builds probably for a few months until our i386 MP work (which is mostly machine independant) is reasonably stable. (does alpha even have an MP build yet?). -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 11:16:28 2000 Delivered-To: freebsd-smp@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id 5B74C37BBE7 for ; Tue, 20 Jun 2000 11:16:23 -0700 (PDT) (envelope-from mjacob@feral.com) Received: from semuta.feral.com (semuta [192.67.166.70]) by feral.com (8.9.3/8.9.3) with ESMTP id LAA17411; Tue, 20 Jun 2000 11:16:20 -0700 Date: Tue, 20 Jun 2000 11:15:55 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: Matthew Dillon Cc: freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp In-Reply-To: <200006201754.KAA87277@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Yes, all i386. At the meeting we decided to break the alpha port > temporarily. I guess it burns me a bit that this meeting excluded some of us who have a strong interest in seeing FreeBSD succeed. > I think alpha work can begin when I make the SP (single processor) > patches available. I would recommend sticking to SP alpha builds > probably for a few months until our i386 MP work (which is mostly > machine independant) is reasonably stable. (does alpha even have an MP > build yet?). Doug was 'real close' last I heard. Okay, I'll drop working on FreeBSD until next year then. 'so long. -matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 11:43:48 2000 Delivered-To: freebsd-smp@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.40.131]) by hub.freebsd.org (Postfix) with ESMTP id 005FB37BC9E for ; Tue, 20 Jun 2000 11:43:44 -0700 (PDT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.9.3/8.9.3) with ESMTP id UAA54707; Tue, 20 Jun 2000 20:43:19 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: mjacob@feral.com Cc: Matthew Dillon , freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp In-reply-to: Your message of "Tue, 20 Jun 2000 11:15:55 PDT." Date: Tue, 20 Jun 2000 20:43:19 +0200 Message-ID: <54705.961526599@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org >I guess it burns me a bit that this meeting excluded some of us who have a >strong interest in seeing FreeBSD succeed. It would be rather costly to invite several thousand people to the meeting, wouldn't it ? >Okay, I'll drop working on FreeBSD until next year then. 'so long. Thanks! [*] Poul-Henning [*] I take the chance of presuming this also means we will not see any of your harrassing ravings in our email during that time. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD coreteam member | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 11:45:48 2000 Delivered-To: freebsd-smp@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id 6155C37BF8B for ; Tue, 20 Jun 2000 11:45:44 -0700 (PDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.9.3/8.9.3) with ESMTP id MAA41132; Tue, 20 Jun 2000 12:45:30 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id MAA70842; Tue, 20 Jun 2000 12:44:01 -0600 (MDT) Message-Id: <200006201844.MAA70842@harmony.village.org> To: Matthew Dillon Subject: Re: SMP discussion moving to freebsd-smp Cc: freebsd-smp@FreeBSD.ORG, Greg Lehey In-reply-to: Your message of "Tue, 20 Jun 2000 10:08:30 PDT." <200006201708.KAA87060@apollo.backplane.com> References: <200006201708.KAA87060@apollo.backplane.com> Date: Tue, 20 Jun 2000 12:44:00 -0600 From: Warner Losh Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message <200006201708.KAA87060@apollo.backplane.com> Matthew Dillon writes: : I now have mutexes 99% working in an SP build. I will be making : SP (single processor) patch sets available tonight or tomorrow : morning. (99% == I can boot the machine normally but after a : while it locks up due to an interrupt nesting issue which I haven't : dealt with yet). May I request that you don't check in the new MP core until the SP issues have been resolved? I don't care if the MP boxes have major pain after the commit date (that's rather the point of all the new MP work), but have some serious issues if the SP boxes have major pain. It wasn't clear from the orignial message if the major pain and bleeding is for all users, or just MP users. It would be extremely hard to make forward progress on anything while the pain is there. For a few days or even a couple of weeks this would be OK, but for months that is not acceptible to me. I'm trying to get NEWCARD finished, as well as start the cardbus work and those two projects are being done in -current. There's no pain to non-laptop users (in fact, there's currently no pain to laptop users that use the old system). It is extremely desirable to have the current tree buildable and stable for SP machines during this transition. Please note, this isn't an attempt to disparage your work, or to try to get overly in the way of it. I'm merely trying to keep the SP world relatively stable while the MP work is going on. Ther'es a 6 year tradition of having -current be basically usable for developers to develop in that I'd like to continue. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 11:47:16 2000 Delivered-To: freebsd-smp@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id 9550937BF93 for ; Tue, 20 Jun 2000 11:47:13 -0700 (PDT) (envelope-from mjacob@feral.com) Received: from semuta.feral.com (semuta [192.67.166.70]) by feral.com (8.9.3/8.9.3) with ESMTP id LAA17535; Tue, 20 Jun 2000 11:46:44 -0700 Date: Tue, 20 Jun 2000 11:46:19 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: Poul-Henning Kamp Cc: Matthew Dillon , freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp In-Reply-To: <54705.961526599@critter.freebsd.dk> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > >I guess it burns me a bit that this meeting excluded some of us who have a > >strong interest in seeing FreeBSD succeed. > > It would be rather costly to invite several thousand people to the > meeting, wouldn't it ? It would have been nice to invite those of us who've had some experience in this area the opportunity to contribute. > > >Okay, I'll drop working on FreeBSD until next year then. 'so long. > > Thanks! [*] > > Poul-Henning > > [*] I take the chance of presuming this also means we will not see > any of your harrassing ravings in our email during that time. Did you see them as that? I'm sorry if they bothered you, then. I had no idea you felt that way. -matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 11:52:57 2000 Delivered-To: freebsd-smp@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.40.131]) by hub.freebsd.org (Postfix) with ESMTP id 3D53E37BFB6 for ; Tue, 20 Jun 2000 11:52:50 -0700 (PDT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.9.3/8.9.3) with ESMTP id UAA54801; Tue, 20 Jun 2000 20:52:26 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: mjacob@feral.com Cc: Matthew Dillon , freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp In-reply-to: Your message of "Tue, 20 Jun 2000 11:46:19 PDT." Date: Tue, 20 Jun 2000 20:52:26 +0200 Message-ID: <54799.961527146@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message , Matthe w Jacob writes: >> >> >I guess it burns me a bit that this meeting excluded some of us who have a >> >strong interest in seeing FreeBSD succeed. >> >> It would be rather costly to invite several thousand people to the >> meeting, wouldn't it ? > >It would have been nice to invite those of us who've had some experience in >this area the opportunity to contribute. Well, everybody can contribute, I hope we'll see the roadmap published RSN. >> >Okay, I'll drop working on FreeBSD until next year then. 'so long. >> >> Thanks! [*] >> >> Poul-Henning >> >> [*] I take the chance of presuming this also means we will not see >> any of your harrassing ravings in our email during that time. > >Did you see them as that? I'm sorry if they bothered you, then. I had no idea >you felt that way. Well, you have a certain way of saying things which does not impress mee too much. Like the above "OK I wasn't invited so I'll take my ball and go home and sulk!" statement. Have you even briefly considered that we're all trying to do this as best we can, and have no specific intent to make your or anybody elses life difficult ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD coreteam member | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 11:58:43 2000 Delivered-To: freebsd-smp@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id 9053637BF75 for ; Tue, 20 Jun 2000 11:58:40 -0700 (PDT) (envelope-from mjacob@feral.com) Received: from semuta.feral.com (semuta [192.67.166.70]) by feral.com (8.9.3/8.9.3) with ESMTP id LAA17570; Tue, 20 Jun 2000 11:58:06 -0700 Date: Tue, 20 Jun 2000 11:57:41 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: Poul-Henning Kamp Cc: Matthew Dillon , freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp In-Reply-To: <54799.961527146@critter.freebsd.dk> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org [ taking this offline - it's not an SMP issue ] To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 12: 4:24 2000 Delivered-To: freebsd-smp@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id D6C2137BFA5 for ; Tue, 20 Jun 2000 12:04:17 -0700 (PDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.9.3/8.9.3) with ESMTP id NAA41281; Tue, 20 Jun 2000 13:04:12 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id NAA71105; Tue, 20 Jun 2000 13:02:43 -0600 (MDT) Message-Id: <200006201902.NAA71105@harmony.village.org> To: mjacob@feral.com Subject: Re: SMP discussion moving to freebsd-smp Cc: Poul-Henning Kamp , Matthew Dillon , freebsd-smp@FreeBSD.ORG In-reply-to: Your message of "Tue, 20 Jun 2000 11:46:19 PDT." References: Date: Tue, 20 Jun 2000 13:02:43 -0600 From: Warner Losh Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org : > >Okay, I'll drop working on FreeBSD until next year then. 'so long. : > : > Thanks! [*] So what if I had said it PHK? I think this "we'll ram it down your throats for the good of the project" attitude is extremely bad. I want to continue to do NEWCARD work, and that work is done in - current. if -current is so unstable that I cannot do the work for a long period of time, the only other real choice I'd have is to do it in -stable and MFS -> current once the dust has sttled, and that's a big no no. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 12:24:27 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 10A7137BBAB for ; Tue, 20 Jun 2000 12:24:19 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id MAA88045; Tue, 20 Jun 2000 12:24:17 -0700 (PDT) (envelope-from dillon) Date: Tue, 20 Jun 2000 12:24:17 -0700 (PDT) From: Matthew Dillon Message-Id: <200006201924.MAA88045@apollo.backplane.com> To: Matthew Jacob Cc: freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp References: Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org :> I think alpha work can begin when I make the SP (single processor) :> patches available. I would recommend sticking to SP alpha builds :> probably for a few months until our i386 MP work (which is mostly :> machine independant) is reasonably stable. (does alpha even have an MP :> build yet?). : :Doug was 'real close' last I heard. : :Okay, I'll drop working on FreeBSD until next year then. 'so long. : :-matt All I am saying here is that the BSDI SMP merge work will turn the MP build into a moving target for the next month at least, and it wouldn't be a very good use of time trying to keep the alpha build in sync for the MP build until after the API's have settled down. The SP build is another story... I expect those API's to settle down within a week. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 12:25:50 2000 Delivered-To: freebsd-smp@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id CDD4B37B5CC for ; Tue, 20 Jun 2000 12:25:46 -0700 (PDT) (envelope-from mjacob@feral.com) Received: from semuta.feral.com (semuta [192.67.166.70]) by feral.com (8.9.3/8.9.3) with ESMTP id MAA17728; Tue, 20 Jun 2000 12:25:44 -0700 Date: Tue, 20 Jun 2000 12:25:19 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: Matthew Dillon Cc: freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp In-Reply-To: <200006201924.MAA88045@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > :> I think alpha work can begin when I make the SP (single processor) > :> patches available. I would recommend sticking to SP alpha builds > :> probably for a few months until our i386 MP work (which is mostly > :> machine independant) is reasonably stable. (does alpha even have an MP > :> build yet?). > : > :Doug was 'real close' last I heard. > : > :Okay, I'll drop working on FreeBSD until next year then. 'so long. > : > :-matt > > All I am saying here is that the BSDI SMP merge work will turn the > MP build into a moving target for the next month at least, and it > wouldn't be a very good use of time trying to keep the alpha build > in sync for the MP build until after the API's have settled down. The > SP build is another story... I expect those API's to settle down within > a week. I understood that, thank you. Bonne journee... To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 12:30:38 2000 Delivered-To: freebsd-smp@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.40.131]) by hub.freebsd.org (Postfix) with ESMTP id 518D237BC78 for ; Tue, 20 Jun 2000 12:30:34 -0700 (PDT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.9.3/8.9.3) with ESMTP id VAA55144; Tue, 20 Jun 2000 21:30:04 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Warner Losh Cc: mjacob@feral.com, Matthew Dillon , freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp In-reply-to: Your message of "Tue, 20 Jun 2000 13:02:43 MDT." <200006201902.NAA71105@harmony.village.org> Date: Tue, 20 Jun 2000 21:30:04 +0200 Message-ID: <55142.961529404@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message <200006201902.NAA71105@harmony.village.org>, Warner Losh writes: >: > >Okay, I'll drop working on FreeBSD until next year then. 'so long. >: > >: > Thanks! [*] > >So what if I had said it PHK? I think this "we'll ram it down your >throats for the good of the project" attitude is extremely bad. I would have said the same thing to you, Matt, David or even Jordan. No single person is so important that we cannot do without them, no person carries a Veto in the FreeBSD project. I fully agree with you, that the planning which we have seen so far is very inadequate and I fully agree that -current should not be broken with Matts wizardry until the rest of the committers have had a chance to see the big plan and comment on it. I don't agree with your rather rabid "current must always work" attitude, having been one of the primary people through the 1.0 to 2.0 transition I know how many developers that cost us. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD coreteam member | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 12:36:12 2000 Delivered-To: freebsd-smp@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.40.131]) by hub.freebsd.org (Postfix) with ESMTP id C707437BC13 for ; Tue, 20 Jun 2000 12:36:06 -0700 (PDT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.9.3/8.9.3) with ESMTP id VAA55238; Tue, 20 Jun 2000 21:34:51 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) Cc: Warner Losh , mjacob@feral.com, Matthew Dillon , freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp In-reply-to: Your message of "Tue, 20 Jun 2000 21:30:04 +0200." <55142.961529404@critter.freebsd.dk> Date: Tue, 20 Jun 2000 21:34:51 +0200 Message-ID: <55236.961529691@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message <55142.961529404@critter.freebsd.dk>, Poul-Henning Kamp writes: >I don't agree with your rather rabid "current must always work" >attitude, having been one of the primary people through the >1.0 to 2.0 transition I know how many developers that cost us. Damn, sign bug! Should have read: I don't agree with your rather rabid "current must always work" attitude, but having been one of the primary people through the 1.0 to 2.0 transition I know how many developers the opposite cost us. The meaning here is: No, it can't be basically useless for months on end, but yes, it may be dead for days or maybe even a week. I think I will agree that requiring UP to compile and work (for the regular -current definition of "work") for at leat 80-90% of the days is not an unreasonable requirement. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD coreteam member | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 12:36:26 2000 Delivered-To: freebsd-smp@freebsd.org Received: from lor.watermarkgroup.com (lor.watermarkgroup.com [207.202.73.33]) by hub.freebsd.org (Postfix) with ESMTP id E717137B7C4 for ; Tue, 20 Jun 2000 12:36:22 -0700 (PDT) (envelope-from luoqi@watermarkgroup.com) Received: (from luoqi@localhost) by lor.watermarkgroup.com (8.10.1/8.10.1) id e5KJZhE04172; Tue, 20 Jun 2000 15:35:43 -0400 (EDT) Date: Tue, 20 Jun 2000 15:35:43 -0400 (EDT) From: Luoqi Chen Message-Id: <200006201935.e5KJZhE04172@lor.watermarkgroup.com> To: dillon@apollo.backplane.com, freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp Cc: grog@lemis.com Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Right now the SP build works because I am allowing (unthreaded) interrupts > to steal the idleproc's context, and that only works because they can > get the giant mutex without blocking (remember, the BSDI giant mutex > is a blocking mutex, not a spin mutex). In the MP build the interrupts > need to be able to block getting the giant mutex which means we need > to implement heavy-weight interrupt threads at the very least before > we can get anything working, because we are not allowed to block in > the idleproc. > May I suggest we first make the giant mutex a spin mutex (as our good old giant kernel lock) and focus on the SP first? In the mean time, MP should continue to work, as only one processor is allowed in the kernel. Once the SP is stablized, we start to tackle the MP build. -lq To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 12:36:41 2000 Delivered-To: freebsd-smp@freebsd.org Received: from ns.yogotech.com (ns.yogotech.com [206.127.123.66]) by hub.freebsd.org (Postfix) with ESMTP id 9856637BC13 for ; Tue, 20 Jun 2000 12:36:37 -0700 (PDT) (envelope-from nate@yogotech.com) Received: from nomad.yogotech.com (nomad.yogotech.com [206.127.123.131]) by ns.yogotech.com (8.9.3/8.9.3) with ESMTP id NAA24751; Tue, 20 Jun 2000 13:36:23 -0600 (MDT) (envelope-from nate@nomad.yogotech.com) Received: (from nate@localhost) by nomad.yogotech.com (8.8.8/8.8.8) id NAA20788; Tue, 20 Jun 2000 13:36:19 -0600 (MDT) (envelope-from nate) Date: Tue, 20 Jun 2000 13:36:19 -0600 (MDT) Message-Id: <200006201936.NAA20788@nomad.yogotech.com> From: Nate Williams MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: Poul-Henning Kamp Cc: Warner Losh , mjacob@feral.com, Matthew Dillon , freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp In-Reply-To: <55142.961529404@critter.freebsd.dk> References: <200006201902.NAA71105@harmony.village.org> <55142.961529404@critter.freebsd.dk> X-Mailer: VM 6.34 under 19.16 "Lille" XEmacs Lucid Reply-To: nate@yogotech.com (Nate Williams) Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > I don't agree with your rather rabid "current must always work" ^^^^^^^^^^^^ I disagree that's his attitude is rabid. This is an attempt by you to make this is a personal attack, rather than a techincal argument. > attitude, having been one of the primary people through the > 1.0 to 2.0 transition I know how many developers that cost us. And how many would that be? I don't remember the masses leaving us because of the 1.0 -> 2.0 transition, but because of the lawsuit. Nate To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 12:37:49 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 8D49C37BFB6 for ; Tue, 20 Jun 2000 12:37:44 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id MAA88247; Tue, 20 Jun 2000 12:36:47 -0700 (PDT) (envelope-from dillon) Date: Tue, 20 Jun 2000 12:36:47 -0700 (PDT) From: Matthew Dillon Message-Id: <200006201936.MAA88247@apollo.backplane.com> To: Warner Losh Cc: freebsd-smp@FreeBSD.ORG, Greg Lehey Subject: Re: SMP discussion moving to freebsd-smp References: <200006201708.KAA87060@apollo.backplane.com> <200006201844.MAA70842@harmony.village.org> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org : :In message <200006201708.KAA87060@apollo.backplane.com> Matthew Dillon writes: :: I now have mutexes 99% working in an SP build. I will be making :: SP (single processor) patch sets available tonight or tomorrow :: morning. (99% == I can boot the machine normally but after a :: while it locks up due to an interrupt nesting issue which I haven't :: dealt with yet). : :May I request that you don't check in the new MP core until the SP :issues have been resolved? I don't care if the MP boxes have major :pain after the commit date (that's rather the point of all the new MP :work), but have some serious issues if the SP boxes have major pain. :It wasn't clear from the orignial message if the major pain and :bleeding is for all users, or just MP users. : :It would be extremely hard to make forward progress on anything while :the pain is there. For a few days or even a couple of weeks this :would be OK, but for months that is not acceptible to me. I'm trying :to get NEWCARD finished, as well as start the cardbus work and those :... The BSDI SMP merge work is not just an MP issue. It changes the way the kernel operates at a fundamental level. What it means, in short, is that the MP mechanisms are going to become 'native' to the kernel, even for SP builds. I'll give you an example: The SPL mechanism is inherently a single-processor algorithm. It just doesn't work on an MP system. Rather then hack SPL's for MP builds we are going to (eventually) remove them entirely from the system ... SP or MP build. So, for example, the Scheduler will no longer be protected by splhigh()/spl0(). It will instead be protected by SchedMutex... the same protection mechanism will be used for both the SP and MP builds. Another example: Interrupts are going to become interrupt threads natively (on both the SP and MP builds). This means that the kernel must be preemptable while sitting in kernel mode. On both the SP and MP builds. So, as you can see, most of the mechanisms we need to implement apply to both SP and MP builds. This means that SP builds are going to be unstable for a while, not just MP builds, as we work out the issues. I am fairly confident that I can produce a stable set of changes in the first patch set, but the fact of the matter is that much of the work is going to entail ripping out the SPL compatibility mechanisms one at a time and replacing them with mutexes. This will be an ongoing process over the next 6 months and that means that -current is going to be less stable for the entire time -- for both the SP and MP builds. :Please note, this isn't an attempt to disparage your work, or to try :to get overly in the way of it. I'm merely trying to keep the SP :world relatively stable while the MP work is going on. Ther'es a 6 :year tradition of having -current be basically usable for developers :to develop in that I'd like to continue. : :Warner -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 12:38: 8 2000 Delivered-To: freebsd-smp@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id E8F8A37BBFB for ; Tue, 20 Jun 2000 12:38:03 -0700 (PDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.9.3/8.9.3) with ESMTP id NAA41514; Tue, 20 Jun 2000 13:38:02 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id NAA71564; Tue, 20 Jun 2000 13:36:33 -0600 (MDT) Message-Id: <200006201936.NAA71564@harmony.village.org> To: Poul-Henning Kamp Subject: Re: SMP discussion moving to freebsd-smp Cc: mjacob@feral.com, Matthew Dillon , freebsd-smp@FreeBSD.ORG In-reply-to: Your message of "Tue, 20 Jun 2000 21:30:04 +0200." <55142.961529404@critter.freebsd.dk> References: <55142.961529404@critter.freebsd.dk> Date: Tue, 20 Jun 2000 13:36:33 -0600 From: Warner Losh Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message <55142.961529404@critter.freebsd.dk> Poul-Henning Kamp writes: : I don't agree with your rather rabid "current must always work" : attitude, having been one of the primary people through the : 1.0 to 2.0 transition I know how many developers that cost us. -current needs to always be buildable. It should always be bootable, but the periods of time that it isn't should be measured in weeks, not months. I'm not insisting that it be rock solid all the time, so if that's what people are reading, I'm doing a poor job at comunicating. I'm objecting to a vague "it will be painful for months, cope" plan, for which I must assume that it will be so bad for months that I won't be able to get any work with it done. If it is stable enough for developers after a few days/weeks and I can get work done without unrelated bugs crashing me all the time, that's cool. If it crashes, on the average, after an hour or two, or under heavy load, that's also OK, so long as it is fixed by a release (which is the plan, as I read it). If it is so unstable that it won't even boot and run for 5 minutes for the next several months, I don't think that's acceptible. I think that code going into the tree has to have at least some minor level of stability for most users. I think we disagree on what that level of stability is. The whole 1.0 -> 2.0 transition is a blur to me now... I think that much of my objections will be answered if I can get a clearer picture of the level of pain, what the likely problems would be and just what the plan is. If these things were clearer, I would be much more confortable. I think that's the real issue I have with all of this. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 12:39:36 2000 Delivered-To: freebsd-smp@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.40.131]) by hub.freebsd.org (Postfix) with ESMTP id E762837BCD1 for ; Tue, 20 Jun 2000 12:39:32 -0700 (PDT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.9.3/8.9.3) with ESMTP id VAA55297; Tue, 20 Jun 2000 21:39:02 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: nate@yogotech.com (Nate Williams) Cc: Warner Losh , mjacob@feral.com, Matthew Dillon , freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp In-reply-to: Your message of "Tue, 20 Jun 2000 13:36:19 MDT." <200006201936.NAA20788@nomad.yogotech.com> Date: Tue, 20 Jun 2000 21:39:02 +0200 Message-ID: <55295.961529942@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message <200006201936.NAA20788@nomad.yogotech.com>, Nate Williams writes: >> attitude, having been one of the primary people through the >> 1.0 to 2.0 transition I know how many developers that cost us. > >And how many would that be? I don't remember the masses leaving us >because of the 1.0 -> 2.0 transition, but because of the lawsuit. I recently had reason to go through all the news archives and mail archives such as they exist, and we lost *a lot* of people between 1.1.5.1 and 2.0. Most seem to have gravitated to Linux. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD coreteam member | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 12:40:50 2000 Delivered-To: freebsd-smp@freebsd.org Received: from ns.yogotech.com (ns.yogotech.com [206.127.123.66]) by hub.freebsd.org (Postfix) with ESMTP id D827837B5BC for ; Tue, 20 Jun 2000 12:40:47 -0700 (PDT) (envelope-from nate@yogotech.com) Received: from nomad.yogotech.com (nomad.yogotech.com [206.127.123.131]) by ns.yogotech.com (8.9.3/8.9.3) with ESMTP id NAA24806; Tue, 20 Jun 2000 13:40:35 -0600 (MDT) (envelope-from nate@nomad.yogotech.com) Received: (from nate@localhost) by nomad.yogotech.com (8.8.8/8.8.8) id NAA20840; Tue, 20 Jun 2000 13:40:35 -0600 (MDT) (envelope-from nate) Date: Tue, 20 Jun 2000 13:40:35 -0600 (MDT) Message-Id: <200006201940.NAA20840@nomad.yogotech.com> From: Nate Williams MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: Poul-Henning Kamp Cc: nate@yogotech.com (Nate Williams), Warner Losh , mjacob@feral.com, Matthew Dillon , freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp In-Reply-To: <55295.961529942@critter.freebsd.dk> References: <200006201936.NAA20788@nomad.yogotech.com> <55295.961529942@critter.freebsd.dk> X-Mailer: VM 6.34 under 19.16 "Lille" XEmacs Lucid Reply-To: nate@yogotech.com (Nate Williams) Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > >> attitude, having been one of the primary people through the > >> 1.0 to 2.0 transition I know how many developers that cost us. > > > >And how many would that be? I don't remember the masses leaving us > >because of the 1.0 -> 2.0 transition, but because of the lawsuit. > > I recently had reason to go through all the news archives and mail > archives such as they exist, and we lost *a lot* of people between > 1.1.5.1 and 2.0. Most seem to have gravitated to Linux. Again, was that because of the 'current does build' problem, or because of the lawsuit? I'd argue that the lawsuit had most to do with it, which is the reason for the 1.x -> 2.0 move. Nate To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 12:42:40 2000 Delivered-To: freebsd-smp@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id B2BC437B7C4 for ; Tue, 20 Jun 2000 12:42:36 -0700 (PDT) (envelope-from mjacob@feral.com) Received: from semuta.feral.com (semuta [192.67.166.70]) by feral.com (8.9.3/8.9.3) with ESMTP id MAA17797; Tue, 20 Jun 2000 12:42:01 -0700 Date: Tue, 20 Jun 2000 12:41:36 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: Warner Losh Cc: Poul-Henning Kamp , Matthew Dillon , freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp In-Reply-To: <200006201936.NAA71564@harmony.village.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I don't think I entirely agree that "-current must always be buildable and runnable" in that I believe we all *did* understand that there would be breakage for the BSD/OS merge- at least I did. I also would have suggested a branch myself (despite the pain to those working on the branch), or the adoption of a better merge/toolset technology (like bitkeeper, for example). The problem here is that we cannot plan for such breakage, either to assist or get out of the way, when all we get are edicts that "this is happening" with no clue as to what the architectural or procedural choices are. But, frankly, the matter now all speaks for itself. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 12:44: 2 2000 Delivered-To: freebsd-smp@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id EC5E037B5BC for ; Tue, 20 Jun 2000 12:43:56 -0700 (PDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.9.3/8.9.3) with ESMTP id NAA41581; Tue, 20 Jun 2000 13:43:46 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id NAA71681; Tue, 20 Jun 2000 13:42:17 -0600 (MDT) Message-Id: <200006201942.NAA71681@harmony.village.org> To: Poul-Henning Kamp Subject: Re: SMP discussion moving to freebsd-smp Cc: mjacob@feral.com, Matthew Dillon , freebsd-smp@FreeBSD.ORG In-reply-to: Your message of "Tue, 20 Jun 2000 21:34:51 +0200." <55236.961529691@critter.freebsd.dk> References: <55236.961529691@critter.freebsd.dk> Date: Tue, 20 Jun 2000 13:42:17 -0600 From: Warner Losh Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message <55236.961529691@critter.freebsd.dk> Poul-Henning Kamp writes: : The meaning here is: No, it can't be basically useless for months : on end, but yes, it may be dead for days or maybe even a week. Yes. I agree. I have no problems with it being bolluxed up for 1 day or 1 week or even if it will gain us a lot, 2 weeks. Beyond that, it becomes too painful for me to use the tree. I susepct others would agree with threasholds in this general area, some tigheter some looser. : I think I will agree that requiring UP to compile and work (for : the regular -current definition of "work") for at leat 80-90% : of the days is not an unreasonable requirement. Yes. I object to having the tree be unusable for development work for "on the order of months" with no clear plan. It has taken me too many messages to reach this one sentence summary. Sorry about that. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 12:47: 5 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 85CA737BBFB for ; Tue, 20 Jun 2000 12:47:02 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id MAA88405; Tue, 20 Jun 2000 12:46:30 -0700 (PDT) (envelope-from dillon) Date: Tue, 20 Jun 2000 12:46:30 -0700 (PDT) From: Matthew Dillon Message-Id: <200006201946.MAA88405@apollo.backplane.com> To: Luoqi Chen Cc: freebsd-smp@FreeBSD.ORG, grog@lemis.com Subject: Re: SMP discussion moving to freebsd-smp References: <200006201935.e5KJZhE04172@lor.watermarkgroup.com> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org : :> Right now the SP build works because I am allowing (unthreaded) interrupts :> to steal the idleproc's context, and that only works because they can :> get the giant mutex without blocking (remember, the BSDI giant mutex :> is a blocking mutex, not a spin mutex). In the MP build the interrupts :> need to be able to block getting the giant mutex which means we need :> to implement heavy-weight interrupt threads at the very least before :> we can get anything working, because we are not allowed to block in :> the idleproc. :> :May I suggest we first make the giant mutex a spin mutex (as our good old :giant kernel lock) and focus on the SP first? In the mean time, MP should :continue to work, as only one processor is allowed in the kernel. Once the :SP is stablized, we start to tackle the MP build. : :-lq Without interrupt threads it doesn't matter whether the GiantMutex is a spin or a blocking mutex because it will never get that far... it will always succeed the first try in the SP build. But we have to implement interrupt threads in order to be able to migrate away from the SPL mechanism, and we have to do that for both builds (rip SPL's out of the kernel entirely). This, in fact, represents *most* of the work that is going to occur in the next few months (ripping out SPL's and replacing them with Mutexes). The whole thing is interrelated. The GiantMutex must be held in order to be able to use the 'old' SPL mechanisms during the transition period. Interrupts must be threaded in order to be able to start replacing SPL's with mutexes. GiantMutex must be a blocking mutex to be able to thread interrupts. Therefore the GiantMutex must be a blocking mutex. Interrupt threads are going to be 'in' in less then a week. To delay it would delay the entire project for the reasons outlined in the previous paragraph. The Schedular mutex is a spinning lock in the new design and replaces the scheduler-aspects that the *old* spinning MP lock handled before. However, the aspects of the old MP lock that protected entry into the kernel (verses just the scheduler), migrates to the new blocking GiantMutex. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 12:48:46 2000 Delivered-To: freebsd-smp@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.40.131]) by hub.freebsd.org (Postfix) with ESMTP id 3300937BFD7 for ; Tue, 20 Jun 2000 12:48:32 -0700 (PDT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.9.3/8.9.3) with ESMTP id VAA55398; Tue, 20 Jun 2000 21:48:16 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Matthew Dillon Cc: Warner Losh , freebsd-smp@FreeBSD.ORG, Greg Lehey Subject: Re: SMP discussion moving to freebsd-smp In-reply-to: Your message of "Tue, 20 Jun 2000 12:36:47 PDT." <200006201936.MAA88247@apollo.backplane.com> Date: Tue, 20 Jun 2000 21:48:16 +0200 Message-ID: <55396.961530496@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > I can produce a stable set of changes in the first patch set, but > the fact of the matter is that much of the work is going to entail > ripping out the SPL compatibility mechanisms one at a time and replacing > them with mutexes. This will be an ongoing process over the next 6 > months and that means that -current is going to be less stable for > the entire time -- for both the SP and MP builds. Right, and here I think we have the crux of the matter: There is a lot of water between: "doesn't build, doesn't run" and "always compiles, rock stable" And I think we can pressume that we'll end somewhere in the middle. I am sure that all members of the SMPng crowd understand that leaving the rest of the FreeBSD developers stranded for more than a few days (worst case a week) at a time is simply not a good idea for anybody, right ? I mean, the SMPng people will want people to beat the snot out of their chances and if people cannot built and run -current, how are they supposed to test it ? So, Warner and everybody else: Calm down, nobody is closing the development in FreeBSD project down for half a year. Matt and the rest of the SMPng crowd: Give us some more concrete details. Don't commit anything until people have had a chance to look at it and test it (normal procedure for large-scale changes applies 100% here) In particular a lot of people are at USENIX, nothing should be committed until they have caught up with their email. And promise that you'll make sure that -current is usable as much of the time as possible for the rest of the developers. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD coreteam member | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 12:51:22 2000 Delivered-To: freebsd-smp@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.40.131]) by hub.freebsd.org (Postfix) with ESMTP id 5E59637BF77 for ; Tue, 20 Jun 2000 12:51:17 -0700 (PDT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.9.3/8.9.3) with ESMTP id VAA55432; Tue, 20 Jun 2000 21:50:42 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: nate@yogotech.com (Nate Williams) Cc: Warner Losh , mjacob@feral.com, Matthew Dillon , freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp In-reply-to: Your message of "Tue, 20 Jun 2000 13:40:35 MDT." <200006201940.NAA20840@nomad.yogotech.com> Date: Tue, 20 Jun 2000 21:50:42 +0200 Message-ID: <55430.961530642@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message <200006201940.NAA20840@nomad.yogotech.com>, Nate Williams writes: >> >> attitude, having been one of the primary people through the >> >> 1.0 to 2.0 transition I know how many developers that cost us. >> > >> >And how many would that be? I don't remember the masses leaving us >> >because of the 1.0 -> 2.0 transition, but because of the lawsuit. >> >> I recently had reason to go through all the news archives and mail >> archives such as they exist, and we lost *a lot* of people between >> 1.1.5.1 and 2.0. Most seem to have gravitated to Linux. > >Again, was that because of the 'current does build' problem, or because >of the lawsuit? Basically because we couldn't provide them with a set of boot floppies so they could do any development on their machines. I don't know if you remember the clamour there were when we had the first "this almost installs a v2 machine" floppies out through the door, otherwise read your archives :-) >I'd argue that the lawsuit had most to do with it, which is the reason >for the 1.x -> 2.0 move. No, monthlong disruption of peoples ability to install and develop was the main reason. Basically as a developer FreeBSD didn't really exist in that period. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD coreteam member | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 12:52: 4 2000 Delivered-To: freebsd-smp@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id 976B837BBFB for ; Tue, 20 Jun 2000 12:52:00 -0700 (PDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.9.3/8.9.3) with ESMTP id NAA41651; Tue, 20 Jun 2000 13:51:44 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id NAA71778; Tue, 20 Jun 2000 13:50:14 -0600 (MDT) Message-Id: <200006201950.NAA71778@harmony.village.org> To: Matthew Dillon Subject: Re: SMP discussion moving to freebsd-smp Cc: freebsd-smp@FreeBSD.ORG, Greg Lehey In-reply-to: Your message of "Tue, 20 Jun 2000 12:36:47 PDT." <200006201936.MAA88247@apollo.backplane.com> References: <200006201936.MAA88247@apollo.backplane.com> <200006201708.KAA87060@apollo.backplane.com> <200006201844.MAA70842@harmony.village.org> Date: Tue, 20 Jun 2000 13:50:14 -0600 From: Warner Losh Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message <200006201936.MAA88247@apollo.backplane.com> Matthew Dillon writes: : This means that SP builds are going to be unstable for a while, not : just MP builds, as we work out the issues. I am fairly confident that : I can produce a stable set of changes in the first patch set, but : the fact of the matter is that much of the work is going to entail : ripping out the SPL compatibility mechanisms one at a time and replacing : them with mutexes. This will be an ongoing process over the next 6 : months and that means that -current is going to be less stable for : the entire time -- for both the SP and MP builds. It sounds like there will be a large step function of instability that will be introduced right away. Then, over time, thigns will become more or less stable as the work progresses. Some days will be good tree days, others bad. So long as I can boot my laptop with -current to do pccard work (which means it must last 15 minutes of light usage, ideally more), then it meets my lower bounds of acceptible. It would be nice if I could also build kernels and such on my laptop during that time as well, but that isn't as critical to me since I can reboot with the older kernel. There's some pain, but on the whole it would be workable if I can do these things. I know that it is hard, but can you characterize the level of instability that I'd see on a regular basis? Where in the continum of "we talking uptimes of 1 minute or less" to "don't put this into production" do you see things most of the time? Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 12:54:13 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 5F58837B5BC for ; Tue, 20 Jun 2000 12:54:09 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id MAA88510; Tue, 20 Jun 2000 12:54:06 -0700 (PDT) (envelope-from dillon) Date: Tue, 20 Jun 2000 12:54:06 -0700 (PDT) From: Matthew Dillon Message-Id: <200006201954.MAA88510@apollo.backplane.com> To: Warner Losh Cc: Poul-Henning Kamp , mjacob@feral.com, freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp References: <55142.961529404@critter.freebsd.dk> <200006201936.NAA71564@harmony.village.org> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org The kernel will always be buildable. It should be stable enough to last for 5 minutes. But there are going to be a lot of legacy issues that need to be fixed -- drivers that get broken. What will probably happen is that a base system with only core features enabled (disk, serial, console, network) will always be more stable during the mergework then a system with the kitchen sink in. A system with the kitchen sink in may not even compile! I don't expect the work required to make drivers compile and run again to be all that difficult, that's the whole reason for keeping the SPL compatibility code in place during the merge work. But it will also be non-zero, and we will want the SPL compatibility code *gone* by release time. I'll give you one example of this: Drivers that create kernel threads (there are two or three) have to use the new mutex code on entry now, and the kthread_create() has been renamed to mp_kthread_create() to guarentee that. Fixing them will not be difficult, but is also non-zero. Another example: cpu_switch() has been renamed to mp_cpu_switch() and mi_switch() has been renamed to mp_switch(). These interfaces may now only be called from mutex-aware code. I've adjusted all the core code to deal with it but there are probably a few drivers that try to poke too deep into the kernel that need to be adjusted as well. The adjustments required are straightforward. For example, in order to be able to call mp_switch() the caller must hold the scheduler mutex and release Giant (there are two helper routines to do all the dirty work here). -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 12:58:55 2000 Delivered-To: freebsd-smp@freebsd.org Received: from sydney.worldwide.lemis.com (term33.conference.usenix.org [209.179.127.33]) by hub.freebsd.org (Postfix) with ESMTP id 672C837BFBA for ; Tue, 20 Jun 2000 12:58:51 -0700 (PDT) (envelope-from grog@sydney.worldwide.lemis.com) Received: (from grog@localhost) by sydney.worldwide.lemis.com (8.9.3/8.9.3) id MAA05534; Tue, 20 Jun 2000 12:58:40 -0700 (PDT) (envelope-from grog) Date: Tue, 20 Jun 2000 12:58:40 -0700 From: Greg Lehey To: Matthew Dillon Cc: freebsd-smp@freebsd.org Subject: Re: SMP discussion moving to freebsd-smp Message-ID: <20000620125840.E502@sydney.worldwide.lemis.com> References: <200006201708.KAA87060@apollo.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0pre2i In-Reply-To: <200006201708.KAA87060@apollo.backplane.com> Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tuesday, 20 June 2000 at 10:08:30 -0700, Matthew Dillon wrote: > All SMP/FreeBSD/BSDI work should move to the freebsd-smp list. > > (I've BCC'd this to -current). > > -- > > I now have mutexes 99% working in an SP build. I will be making > SP (single processor) patch sets available tonight or tomorrow > morning. (99% == I can boot the machine normally but after a > while it locks up due to an interrupt nesting issue which I haven't > dealt with yet). Great. > In order to make an MP build work, we need interrupt threads (either > light or heavy weight). > > Greg, you expressed an interest in doing the light-weight interrupt > threads. Yes, I've committed to both tasks, first the heavyweight processes and then the light-weight threads. > Do you want me to do the heavy-weight interrupt threads first? > It's a lot of nasty assembly and it is a prerequisit for being > able to do the light weight interrupt threads. Thanks, I don't have any problem with assembler. > I think once the heavy weight interrupt threads are done that > the light weight interrupts can be implemented entirely in C. Interestingly, I was expecting more assembler in the light-weight threads. But in both cases I would expect the assembler component to be minimal in either case. > Right now the SP build works because I am allowing (unthreaded) > interrupts to steal the idleproc's context, and that only works > because they can get the giant mutex without blocking (remember, > the BSDI giant mutex is a blocking mutex, not a spin mutex). In > the MP build the interrupts need to be able to block getting the > giant mutex which means we need to implement heavy-weight > interrupt threads at the very least before we can get anything > working, because we are not allowed to block in the idleproc. Right, that was my understanding last week. > I believe I have done almost everything necessary to implement > heavy weight interrupt threads, including moving the kthreads > initialization code to an earlier point in the boot process (so > much so that I think the device init stuff can run with working > kernel processes rather then with all the delayed-thread stuff). Why don't we take this one off line and discuss just how it should be done? I'm at USENIX at the moment and don't have much time to invest on this topic (as promised last week), but we also agreed that it didn't have to be ready this week. Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 13:10:57 2000 Delivered-To: freebsd-smp@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id F17ED37BFA6 for ; Tue, 20 Jun 2000 13:10:48 -0700 (PDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.9.3/8.9.3) with ESMTP id OAA41798; Tue, 20 Jun 2000 14:10:35 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id OAA72038; Tue, 20 Jun 2000 14:09:05 -0600 (MDT) Message-Id: <200006202009.OAA72038@harmony.village.org> To: Matthew Dillon Subject: Re: SMP discussion moving to freebsd-smp Cc: Poul-Henning Kamp , mjacob@feral.com, freebsd-smp@FreeBSD.ORG In-reply-to: Your message of "Tue, 20 Jun 2000 12:54:06 PDT." <200006201954.MAA88510@apollo.backplane.com> References: <200006201954.MAA88510@apollo.backplane.com> <55142.961529404@critter.freebsd.dk> <200006201936.NAA71564@harmony.village.org> Date: Tue, 20 Jun 2000 14:09:05 -0600 From: Warner Losh Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message <200006201954.MAA88510@apollo.backplane.com> Matthew Dillon writes: : The kernel will always be buildable. It should be stable enough to : last for 5 minutes. But there are going to be a lot of legacy issues : that need to be fixed -- drivers that get broken. What will probably : happen is that a base system with only core features enabled (disk, : serial, console, network) will always be more stable during the mergework : then a system with the kitchen sink in. A system with the kitchen : sink in may not even compile! OK. That's useful to know. I just need that much of the system to work for my work on NEWCARD. Well, it would be really nice if loadable modules don't get broken in all of that, but I know that part of the system will go through its unstable phase like everything else. : I'll give you one example of this: Drivers that create kernel threads : (there are two or three) have to use the new mutex code on entry now, : and the kthread_create() has been renamed to mp_kthread_create() to : guarentee that. Fixing them will not be difficult, but is also non-zero. That will hit NEWCARD, as an example :-). However, I view it as my responsibilty to cope with such changes and to assist as much as I can in moving forward. It may have some minorly negative implications, but nothing to get upset about. OK. So long as there's a strong commitment to having a "-current-ly" stable core part of the kernel, I can live with the pain of instability around the edges. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 13:39: 3 2000 Delivered-To: freebsd-smp@freebsd.org Received: from ixori.demon.nl (ixori.demon.nl [195.11.248.5]) by hub.freebsd.org (Postfix) with ESMTP id 7855F37C08F for ; Tue, 20 Jun 2000 13:38:50 -0700 (PDT) (envelope-from bart@ixori.demon.nl) Received: from smtp-relay by ixori.demon.nl (8.9.3/8.9.2) with ESMTP id WAA00502 for ; Tue, 20 Jun 2000 22:42:21 +0200 (CEST) (envelope-from bart@ixori.demon.nl) Received: from network (intranet) by smtp-relay (Bart's intranet smtp server) Message-ID: <394FD759.181CD936@ixori.demon.nl> Date: Tue, 20 Jun 2000 22:43:05 +0200 From: Bart van Leeuwen Reply-To: bart@ixori.demon.nl Organization: IxorI X-Mailer: Mozilla 4.7 [en] (X11; I; FreeBSD 4.0-STABLE i386) X-Accept-Language: en MIME-Version: 1.0 To: freebsd-smp@freebsd.org Subject: Re: SMP discussion moving to freebsd-smp References: <55142.961529404@critter.freebsd.dk> <200006201936.NAA71564@harmony.village.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org hmm.. I'm just an observer (and tracker of the smp stuff in -current from a quite early time) The whole discussion about -current makes me think. Its funny to see how the actual result of the work in -current usually is more stable and usable then many commercial products, even if thats not the purpose of -current For the rest, if the kind of instability discussed here is not allowed you are simply not able to implement certain big changes. Oh well... after a couple of years I'm still a bit amazed by a development source that is usually stable enough to actually use it as a development and even as a working platform (well.. if you have some experience in recovering systems for the cases -current breaks which does happen... but hey... thats just a usefull experience which doesn't have to cost a lot of time once you figure a nice way to do it) -- Bart van Leeuwen ----------------------------------------------------------- mailto:bart@ixori.demon.nl - http://www.ixori.demon.nl/ ----------------------------------------------------------- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 13:39:37 2000 Delivered-To: freebsd-smp@freebsd.org Received: from web3503.mail.yahoo.com (web3503.mail.yahoo.com [204.71.203.70]) by hub.freebsd.org (Postfix) with SMTP id 480C837C2DD for ; Tue, 20 Jun 2000 13:39:21 -0700 (PDT) (envelope-from songuiuc@yahoo.com) Message-ID: <20000620203917.23092.qmail@web3503.mail.yahoo.com> Received: from [136.182.2.221] by web3503.mail.yahoo.com; Tue, 20 Jun 2000 13:39:17 PDT Date: Tue, 20 Jun 2000 13:39:17 -0700 (PDT) From: Yuan Song Subject: volunteer To: freebsd-smp@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi, I am interested in doing some volunteer work. The following is my resume. Let me know if I am qualified. Thanks, Yuan Song ----------------------------------------------------------------------------------------- 400 Manda Lane, Apt. 524 Wheeling, IL 60090 Phone: (847) 495 3376 Email: songuiuc@yahoo.edu OBJECTIVE: To seek a volunteer job as freeBSD software engineer. EDUCATION: Master of Computer Science University of Illinois at Urbana-Champaign (major GPA 3.87/4.0) EXPERIENCE: Software Engineer (Jan. 2000 -- present) Motorola GSM division. Work on General Packet Radio Service system which connects mobile phone to the Internet. Research Assistant (Jan. 1999 -- Dec. 1999) National Center of Supercomputing Applications at the University of Illinois Perform financial data analysis and visualization with various algorithms. Teaching Assistant of Operating System(Sept. 1998 -- Jan 1999) University of Illinois at Urbana-Champaign Design and implement machine problems that simulate various parts of operating system, such as virtual memory, file system, scheduler, threads, and remote procedure calls. Graduate Assistant (May 1997-Aug. 1998) University Library at University of Illinois at Urbana-Champaign Create and maintain databases. Design and implement database applications. SKILLS: C/C++(highly proficient, about 4 years experience), Java(about 1 year experience), Intel assembly language, MIPS, SmallTalk, Perl, UNIX/X Windows, Windows 95/NT, SUN Sparc, HP, IBM PC, Macintosh, TCP/IP, UNIX/Windows Sockets, SQL, Informix, and Relational DBMS COURSE WORK Database Systems, Object-Oriented Programming and Design, Embedded Systems Architectures, Mobile Computing and Communications, Communication Networks for Computers, Operating System, Multimedia System, Advanced Topics in Distributed Systems, Computer System Organization, Artificial Intelligence, Programming Language and Compilers, and Combinatorial Algorithms PROJECTS * Implemented a real-time controller for water see-saw. * Designed and implemented a mobile computing simulator which simulates the communication algorithms for mobile hosts. * Designed and implemented many operating system algorithms such as thread scheduling and synchronization, supervisor system calls, virtual memory, file system, scheduler, threads, networking, and remote system calls, using C++. (Jan. 1998--May 1998. and Sept. 1999--Dec. 1999 as a teaching assistant) * Implemented a reliable link protocol and a routing protocol with multicast support in a group of two. Worked as the leader and designer. * Designed and implemented individually a brand-new web search engine with neural networks. The project applied a modified version of Hopfield neural network that can store a large amount of patterns in small amount of memory space and retrieve with very high accuracy. * Designed and implemented a web-based multimedia database system with a team of four. Worked as leader and primary designer in the team. * Implemented a distributed multimedia authoring and presentation system that can record multimedia information, store it in a server, and then download the information and present it at a client. * Pattern Recognition: Implement various algorithms such as k-means, SOM, etc. with Java, C/C++, and Perl to classify financial data into different clusters and find out the relations among different vector spaces automatically. __________________________________________________ Do You Yahoo!? Send instant messages with Yahoo! Messenger. http://im.yahoo.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 13:39:43 2000 Delivered-To: freebsd-smp@freebsd.org Received: from web3503.mail.yahoo.com (web3503.mail.yahoo.com [204.71.203.70]) by hub.freebsd.org (Postfix) with SMTP id D50BB37C282 for ; Tue, 20 Jun 2000 13:39:36 -0700 (PDT) (envelope-from songuiuc@yahoo.com) Message-ID: <20000620203934.23131.qmail@web3503.mail.yahoo.com> Received: from [136.182.2.221] by web3503.mail.yahoo.com; Tue, 20 Jun 2000 13:39:34 PDT Date: Tue, 20 Jun 2000 13:39:34 -0700 (PDT) From: Yuan Song Subject: volunteer To: freebsd-smp@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi, I am interested in doing some volunteer work. The following is my resume. Let me know if I am qualified. Thanks, Yuan Song ----------------------------------------------------------------------------------------- 400 Manda Lane, Apt. 524 Wheeling, IL 60090 Phone: (847) 495 3376 Email: songuiuc@yahoo.edu OBJECTIVE: To seek a volunteer job as freeBSD software engineer. EDUCATION: Master of Computer Science University of Illinois at Urbana-Champaign (major GPA 3.87/4.0) EXPERIENCE: Software Engineer (Jan. 2000 -- present) Motorola GSM division. Work on General Packet Radio Service system which connects mobile phone to the Internet. Research Assistant (Jan. 1999 -- Dec. 1999) National Center of Supercomputing Applications at the University of Illinois Perform financial data analysis and visualization with various algorithms. Teaching Assistant of Operating System(Sept. 1998 -- Jan 1999) University of Illinois at Urbana-Champaign Design and implement machine problems that simulate various parts of operating system, such as virtual memory, file system, scheduler, threads, and remote procedure calls. Graduate Assistant (May 1997-Aug. 1998) University Library at University of Illinois at Urbana-Champaign Create and maintain databases. Design and implement database applications. SKILLS: C/C++(highly proficient, about 4 years experience), Java(about 1 year experience), Intel assembly language, MIPS, SmallTalk, Perl, UNIX/X Windows, Windows 95/NT, SUN Sparc, HP, IBM PC, Macintosh, TCP/IP, UNIX/Windows Sockets, SQL, Informix, and Relational DBMS COURSE WORK Database Systems, Object-Oriented Programming and Design, Embedded Systems Architectures, Mobile Computing and Communications, Communication Networks for Computers, Operating System, Multimedia System, Advanced Topics in Distributed Systems, Computer System Organization, Artificial Intelligence, Programming Language and Compilers, and Combinatorial Algorithms PROJECTS * Implemented a real-time controller for water see-saw. * Designed and implemented a mobile computing simulator which simulates the communication algorithms for mobile hosts. * Designed and implemented many operating system algorithms such as thread scheduling and synchronization, supervisor system calls, virtual memory, file system, scheduler, threads, networking, and remote system calls, using C++. (Jan. 1998--May 1998. and Sept. 1999--Dec. 1999 as a teaching assistant) * Implemented a reliable link protocol and a routing protocol with multicast support in a group of two. Worked as the leader and designer. * Designed and implemented individually a brand-new web search engine with neural networks. The project applied a modified version of Hopfield neural network that can store a large amount of patterns in small amount of memory space and retrieve with very high accuracy. * Designed and implemented a web-based multimedia database system with a team of four. Worked as leader and primary designer in the team. * Implemented a distributed multimedia authoring and presentation system that can record multimedia information, store it in a server, and then download the information and present it at a client. * Pattern Recognition: Implement various algorithms such as k-means, SOM, etc. with Java, C/C++, and Perl to classify financial data into different clusters and find out the relations among different vector spaces automatically. __________________________________________________ Do You Yahoo!? Send instant messages with Yahoo! Messenger. http://im.yahoo.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 13:41:59 2000 Delivered-To: freebsd-smp@freebsd.org Received: from web3503.mail.yahoo.com (web3503.mail.yahoo.com [204.71.203.70]) by hub.freebsd.org (Postfix) with SMTP id BE13637C0F3 for ; Tue, 20 Jun 2000 13:41:52 -0700 (PDT) (envelope-from songuiuc@yahoo.com) Message-ID: <20000620204150.23360.qmail@web3503.mail.yahoo.com> Received: from [136.182.2.221] by web3503.mail.yahoo.com; Tue, 20 Jun 2000 13:41:50 PDT Date: Tue, 20 Jun 2000 13:41:50 -0700 (PDT) From: Yuan Song Subject: volunteer To: freebsd-smp@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi, I am interested in doing some volunteer work. The following is my resume. Let me know if I am qualified. Thanks, Yuan Song ----------------------------------------------------------------------------------------- 400 Manda Lane, Apt. 524 Wheeling, IL 60090 Phone: (847) 495 3376 Email: songuiuc@yahoo.edu OBJECTIVE: To seek a volunteer job as freeBSD software engineer. EDUCATION: Master of Computer Science University of Illinois at Urbana-Champaign (major GPA 3.87/4.0) EXPERIENCE: Software Engineer (Jan. 2000 -- present) Motorola GSM division. Work on General Packet Radio Service system which connects mobile phone to the Internet. Research Assistant (Jan. 1999 -- Dec. 1999) National Center of Supercomputing Applications at the University of Illinois Perform financial data analysis and visualization with various algorithms. Teaching Assistant of Operating System(Sept. 1998 -- Jan 1999) University of Illinois at Urbana-Champaign Design and implement machine problems that simulate various parts of operating system, such as virtual memory, file system, scheduler, threads, and remote procedure calls. Graduate Assistant (May 1997-Aug. 1998) University Library at University of Illinois at Urbana-Champaign Create and maintain databases. Design and implement database applications. SKILLS: C/C++(highly proficient, about 4 years experience), Java(about 1 year experience), Intel assembly language, MIPS, SmallTalk, Perl, UNIX/X Windows, Windows 95/NT, SUN Sparc, HP, IBM PC, Macintosh, TCP/IP, UNIX/Windows Sockets, SQL, Informix, and Relational DBMS COURSE WORK Database Systems, Object-Oriented Programming and Design, Embedded Systems Architectures, Mobile Computing and Communications, Communication Networks for Computers, Operating System, Multimedia System, Advanced Topics in Distributed Systems, Computer System Organization, Artificial Intelligence, Programming Language and Compilers, and Combinatorial Algorithms PROJECTS * Implemented a real-time controller for water see-saw. * Designed and implemented a mobile computing simulator which simulates the communication algorithms for mobile hosts. * Designed and implemented many operating system algorithms such as thread scheduling and synchronization, supervisor system calls, virtual memory, file system, scheduler, threads, networking, and remote system calls, using C++. (Jan. 1998--May 1998. and Sept. 1999--Dec. 1999 as a teaching assistant) * Implemented a reliable link protocol and a routing protocol with multicast support in a group of two. Worked as the leader and designer. * Designed and implemented individually a brand-new web search engine with neural networks. The project applied a modified version of Hopfield neural network that can store a large amount of patterns in small amount of memory space and retrieve with very high accuracy. * Designed and implemented a web-based multimedia database system with a team of four. Worked as leader and primary designer in the team. * Implemented a distributed multimedia authoring and presentation system that can record multimedia information, store it in a server, and then download the information and present it at a client. * Pattern Recognition: Implement various algorithms such as k-means, SOM, etc. with Java, C/C++, and Perl to classify financial data into different clusters and find out the relations among different vector spaces automatically. __________________________________________________ Do You Yahoo!? Send instant messages with Yahoo! Messenger. http://im.yahoo.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 13:43:35 2000 Delivered-To: freebsd-smp@freebsd.org Received: from web3503.mail.yahoo.com (web3503.mail.yahoo.com [204.71.203.70]) by hub.freebsd.org (Postfix) with SMTP id 25B3637C00E for ; Tue, 20 Jun 2000 13:43:30 -0700 (PDT) (envelope-from songuiuc@yahoo.com) Message-ID: <20000620204322.23530.qmail@web3503.mail.yahoo.com> Received: from [136.182.2.221] by web3503.mail.yahoo.com; Tue, 20 Jun 2000 13:43:22 PDT Date: Tue, 20 Jun 2000 13:43:22 -0700 (PDT) From: Yuan Song Subject: volunteer To: freebsd-smp@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi, I am interested in doing some volunteer work. The following is my resume. Let me know if I am qualified. Thanks, Yuan Song ----------------------------------------------------------------------------------------- 400 Manda Lane, Apt. 524 Wheeling, IL 60090 Phone: (847) 495 3376 Email: songuiuc@yahoo.edu OBJECTIVE: To seek a volunteer job as freeBSD software engineer. EDUCATION: Master of Computer Science University of Illinois at Urbana-Champaign (major GPA 3.87/4.0) EXPERIENCE: Software Engineer (Jan. 2000 -- present) Motorola GSM division. Work on General Packet Radio Service system which connects mobile phone to the Internet. Research Assistant (Jan. 1999 -- Dec. 1999) National Center of Supercomputing Applications at the University of Illinois Perform financial data analysis and visualization with various algorithms. Teaching Assistant of Operating System(Sept. 1998 -- Jan 1999) University of Illinois at Urbana-Champaign Design and implement machine problems that simulate various parts of operating system, such as virtual memory, file system, scheduler, threads, and remote procedure calls. Graduate Assistant (May 1997-Aug. 1998) University Library at University of Illinois at Urbana-Champaign Create and maintain databases. Design and implement database applications. SKILLS: C/C++(highly proficient, about 4 years experience), Java(about 1 year experience), Intel assembly language, MIPS, SmallTalk, Perl, UNIX/X Windows, Windows 95/NT, SUN Sparc, HP, IBM PC, Macintosh, TCP/IP, UNIX/Windows Sockets, SQL, Informix, and Relational DBMS COURSE WORK Database Systems, Object-Oriented Programming and Design, Embedded Systems Architectures, Mobile Computing and Communications, Communication Networks for Computers, Operating System, Multimedia System, Advanced Topics in Distributed Systems, Computer System Organization, Artificial Intelligence, Programming Language and Compilers, and Combinatorial Algorithms PROJECTS * Implemented a real-time controller for water see-saw. * Designed and implemented a mobile computing simulator which simulates the communication algorithms for mobile hosts. * Designed and implemented many operating system algorithms such as thread scheduling and synchronization, supervisor system calls, virtual memory, file system, scheduler, threads, networking, and remote system calls, using C++. (Jan. 1998--May 1998. and Sept. 1999--Dec. 1999 as a teaching assistant) * Implemented a reliable link protocol and a routing protocol with multicast support in a group of two. Worked as the leader and designer. * Designed and implemented individually a brand-new web search engine with neural networks. The project applied a modified version of Hopfield neural network that can store a large amount of patterns in small amount of memory space and retrieve with very high accuracy. * Designed and implemented a web-based multimedia database system with a team of four. Worked as leader and primary designer in the team. * Implemented a distributed multimedia authoring and presentation system that can record multimedia information, store it in a server, and then download the information and present it at a client. * Pattern Recognition: Implement various algorithms such as k-means, SOM, etc. with Java, C/C++, and Perl to classify financial data into different clusters and find out the relations among different vector spaces automatically. __________________________________________________ Do You Yahoo!? Send instant messages with Yahoo! Messenger. http://im.yahoo.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 16: 7: 9 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 8E72E37B811 for ; Tue, 20 Jun 2000 16:07:00 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id QAA89331; Tue, 20 Jun 2000 16:05:25 -0700 (PDT) (envelope-from dillon) Date: Tue, 20 Jun 2000 16:05:25 -0700 (PDT) From: Matthew Dillon Message-Id: <200006202305.QAA89331@apollo.backplane.com> To: Brian Somers Cc: Warner Losh , Jason Evans , freebsd-smp@FreeBSD.ORG, brian@hak.lan.Awfulhak.org Subject: Re: HEADS UP: Destabilization due to SMP development References: <200006202015.VAA66308@hak.lan.Awfulhak.org> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org (note: redirected to freebsd-smp) : :What about doing the changes on a branch with the understanding that :the branch will *replace* HEAD when it stabilises ? : :This sounds odd at first glance, but it means that others are forced :to MFC into the smp branch - if they don't they lose. : :Anybody that's not confident to be able to merge into the smp branch :will simply be in the same position - merge or hold off. They'd also :be just as likely to break the smp work with their commits as if the :smp work was done in HEAD. Won't work. Too many files are being messed with. Just to give people an idea of the number of files being modified, these are the Index: lines from my diff set so far: Index: conf/Makefile.i386 Index: conf/files Index: conf/files.i386 Index: dev/kbd/atkbd.c Index: dev/pcic/i82365.c Index: dev/syscons/scmouse.c Index: dev/syscons/scterm-sc.c Index: dev/syscons/syscons.c Index: dev/usb/usb.c Index: dev/usb/usb_port.h Index: i386/i386/exception.s Index: i386/i386/genassym.c Index: i386/i386/globals.s Index: i386/i386/identcpu.c Index: i386/i386/locore.s Index: i386/i386/machdep.c Index: i386/i386/mem.c Index: i386/i386/mp_machdep.c Index: i386/i386/mpboot.s Index: i386/i386/mplock.s Index: i386/i386/swtch.s Index: i386/i386/trap.c Index: i386/i386/vm86bios.s Index: i386/i386/vm_machdep.c Index: i386/ibcs2/ibcs2_signal.c Index: i386/include/asnames.h Index: i386/include/atomic.h Index: i386/include/globaldata.h Index: i386/include/globals.h Index: i386/include/ipl.h Index: i386/include/lock.h Index: i386/include/pcb.h Index: i386/isa/apic_ipl.s Index: i386/isa/cx.c Index: i386/isa/icu_ipl.s Index: i386/isa/icu_vector.s Index: i386/isa/intr_machdep.h Index: i386/isa/ipl.s Index: i386/isa/ipl_funcs.c Index: isa/sio.c Index: kern/imgact_aout.c Index: kern/imgact_elf.c Index: kern/init_main.c Index: kern/kern_clock.c Index: kern/kern_descrip.c Index: kern/kern_event.c Index: kern/kern_exec.c Index: kern/kern_exit.c Index: kern/kern_fork.c Index: kern/kern_intr.c Index: kern/kern_kthread.c Index: kern/kern_lock.c Index: kern/kern_proc.c Index: kern/kern_random.c Index: kern/kern_shutdown.c Index: kern/kern_sig.c Index: kern/kern_subr.c Index: kern/kern_switch.c Index: kern/kern_synch.c Index: kern/kern_threads.c Index: kern/kern_time.c Index: kern/kern_timeout.c Index: kern/subr_log.c Index: kern/subr_prf.c Index: kern/subr_rman.c Index: kern/subr_taskqueue.c Index: kern/sys_generic.c Index: kern/sys_pipe.c Index: kern/sys_process.c Index: kern/tty.c Index: kern/tty_pty.c Index: kern/uipc_socket.c Index: kern/uipc_socket2.c Index: kern/uipc_syscalls.c Index: kern/vfs_aio.c Index: kern/vfs_bio.c Index: kern/vfs_subr.c Index: miscfs/procfs/procfs_mem.c Index: msdosfs/msdosfs_vnops.c Index: net/bpf.c Index: netgraph/ng_ksocket.c Index: netinet6/ip6_input.c Index: netkey/key.c Index: pci/pccbb.c Index: svr4/svr4_signal.c Index: sys/buf.h Index: sys/kernel.h Index: sys/kthread.h Index: sys/proc.h Index: sys/signalvar.h Index: sys/systm.h Index: sys/unistd.h Index: vm/vm_fault.c Index: vm/vm_glue.c Index: vm/vm_pageout.c apollo:/FreeBSD/FreeBSD-current/src/sys# wc /tmp/x.diff 7522 26191 196377 /tmp/x The changes to i386/ are substantial. The changes to kern are mid-sized. The changes to most other places are minor. This is with the legacy spl code still left intact in most places. Every file in the entire sys tree is going to change when we start ripping spl's out for real. At the moment I only have to rip them out of the sysconsole path and in i386. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 16: 7:39 2000 Delivered-To: freebsd-smp@freebsd.org Received: from thelab.hub.org (nat204.182.mpoweredpc.net [142.177.204.182]) by hub.freebsd.org (Postfix) with ESMTP id 8632237B760 for ; Tue, 20 Jun 2000 16:07:35 -0700 (PDT) (envelope-from scrappy@hub.org) Received: from localhost (scrappy@localhost) by thelab.hub.org (8.9.3/8.9.3) with ESMTP id UAA04499; Tue, 20 Jun 2000 20:03:10 -0300 (ADT) (envelope-from scrappy@hub.org) X-Authentication-Warning: thelab.hub.org: scrappy owned process doing -bs Date: Tue, 20 Jun 2000 20:03:10 -0300 (ADT) From: The Hermit Hacker To: Warner Losh Cc: mjacob@feral.com, Poul-Henning Kamp , Matthew Dillon , freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp In-Reply-To: <200006201902.NAA71105@harmony.village.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, 20 Jun 2000, Warner Losh wrote: > : > >Okay, I'll drop working on FreeBSD until next year then. 'so long. > : > > : > Thanks! [*] > > So what if I had said it PHK? I think this "we'll ram it down your > throats for the good of the project" attitude is extremely bad. > > I want to continue to do NEWCARD work, and that work is done in - > current. if -current is so unstable that I cannot do the work for a > long period of time, the only other real choice I'd have is to do it > in -stable and MFS -> current once the dust has sttled, and that's a > big no no. Which one would be harder to merge back into -current from a branch? Your NEWCARD work or the SMP work? Why not make a branch for NEWCARD and when the SMP work is declared stable, *you* spend the effort of merging it back in? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 16: 7:54 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 6581637C01B for ; Tue, 20 Jun 2000 16:07:45 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id QAA89367; Tue, 20 Jun 2000 16:07:34 -0700 (PDT) (envelope-from dillon) Date: Tue, 20 Jun 2000 16:07:34 -0700 (PDT) From: Matthew Dillon Message-Id: <200006202307.QAA89367@apollo.backplane.com> To: Warner Losh Cc: freebsd-smp@FreeBSD.ORG, Greg Lehey Subject: Re: SMP discussion moving to freebsd-smp References: <200006201936.MAA88247@apollo.backplane.com> <200006201708.KAA87060@apollo.backplane.com> <200006201844.MAA70842@harmony.village.org> <200006201950.NAA71778@harmony.village.org> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org :It sounds like there will be a large step function of instability that :will be introduced right away. Then, over time, thigns will become :more or less stable as the work progresses. Some days will be good :tree days, others bad. So long as I can boot my laptop with -current :to do pccard work (which means it must last 15 minutes of light usage, :ideally more), then it meets my lower bounds of acceptible. It would :be nice if I could also build kernels and such on my laptop during :that time as well, but that isn't as critical to me since I can reboot :with the older kernel. There's some pain, but on the whole it would :be workable if I can do these things. : :I know that it is hard, but can you characterize the level of :instability that I'd see on a regular basis? Where in the continum of :"we talking uptimes of 1 minute or less" to "don't put this into :production" do you see things most of the time? : :Warner Hmm. I think from the point of view of pccard development you will be in good shape. I can't really guess what the level of instability will be, but I think if we test things enough before committing (i.e. the machine has to survive a 'make' of the kernel) that it should be good enough to not intefere too much with your pccard work. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 16:13:13 2000 Delivered-To: freebsd-smp@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id E0BD537B570 for ; Tue, 20 Jun 2000 16:13:09 -0700 (PDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.9.3/8.9.3) with ESMTP id RAA42826; Tue, 20 Jun 2000 17:13:08 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id RAA75680; Tue, 20 Jun 2000 17:11:38 -0600 (MDT) Message-Id: <200006202311.RAA75680@harmony.village.org> To: The Hermit Hacker Subject: Re: SMP discussion moving to freebsd-smp Cc: freebsd-smp@FreeBSD.ORG In-reply-to: Your message of "Tue, 20 Jun 2000 20:03:10 -0300." References: Date: Tue, 20 Jun 2000 17:11:37 -0600 From: Warner Losh Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message The Hermit Hacker writes: : Which one would be harder to merge back into -current from a branch? Your : NEWCARD work or the SMP work? Why not make a branch for NEWCARD and when : the SMP work is declared stable, *you* spend the effort of merging it back : in? I would do that if the SMP stuff is really bad. I think that Matt's assuances indicate this won't be the case. However, it isn't just NEWCARD. It is every other project that is going on in -current right now. There's work on the usb stack, in the sound system, in the config system, etc. Should all of them run on a branch? No. They shouldn't. If the SMP folks were to have done what was implied in the original message, I think we'd all get together and create a branch for real work. Having said that, I think that the SMP folks now understand the importance of keeping the really bad breakage down to a few days and keeping the base of the system somewhat stable. I don't think a branch will be necessary unless the SMP folks aren't able to do what they say they are going to do. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 16:13:37 2000 Delivered-To: freebsd-smp@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id 5EAEE37B570 for ; Tue, 20 Jun 2000 16:13:34 -0700 (PDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.9.3/8.9.3) with ESMTP id RAA42832; Tue, 20 Jun 2000 17:13:33 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id RAA75692; Tue, 20 Jun 2000 17:12:03 -0600 (MDT) Message-Id: <200006202312.RAA75692@harmony.village.org> To: The Hermit Hacker Subject: Re: SMP discussion moving to freebsd-smp Cc: freebsd-smp@FreeBSD.ORG In-reply-to: Your message of "Tue, 20 Jun 2000 20:03:10 -0300." References: Date: Tue, 20 Jun 2000 17:12:02 -0600 From: Warner Losh Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message The Hermit Hacker writes: : Which one would be harder to merge back into -current from a branch? Your : NEWCARD work or the SMP work? Why not make a branch for NEWCARD and when : the SMP work is declared stable, *you* spend the effort of merging it back : in? I would do that if the SMP stuff is really bad. I think that Matt's assuances indicate this won't be the case. However, it isn't just NEWCARD. It is every other project that is going on in -current right now. There's work on the usb stack, in the sound system, in the config system, etc. Should all of them run on a branch? No. They shouldn't. If the SMP folks were to have done what was implied in the original message, I think we'd all get together and create a branch for real work. Having said that, I think that the SMP folks now understand the importance of keeping the really bad breakage down to a few days and keeping the base of the system somewhat stable. I don't think a branch will be necessary unless the SMP folks aren't able to do what they say they are going to do. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 16:20:47 2000 Delivered-To: freebsd-smp@freebsd.org Received: from rover.village.org (rover.village.org [204.144.255.49]) by hub.freebsd.org (Postfix) with ESMTP id 91AF437BCCE for ; Tue, 20 Jun 2000 16:20:43 -0700 (PDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (harmony.village.org [10.0.0.6]) by rover.village.org (8.9.3/8.9.3) with ESMTP id RAA42870; Tue, 20 Jun 2000 17:20:35 -0600 (MDT) (envelope-from imp@harmony.village.org) Received: from harmony.village.org (localhost.village.org [127.0.0.1]) by harmony.village.org (8.9.3/8.8.3) with ESMTP id RAA75806; Tue, 20 Jun 2000 17:19:04 -0600 (MDT) Message-Id: <200006202319.RAA75806@harmony.village.org> To: Matthew Dillon Subject: Re: SMP discussion moving to freebsd-smp Cc: freebsd-smp@FreeBSD.ORG, Greg Lehey In-reply-to: Your message of "Tue, 20 Jun 2000 16:07:34 PDT." <200006202307.QAA89367@apollo.backplane.com> References: <200006202307.QAA89367@apollo.backplane.com> <200006201936.MAA88247@apollo.backplane.com> <200006201708.KAA87060@apollo.backplane.com> <200006201844.MAA70842@harmony.village.org> <200006201950.NAA71778@harmony.village.org> Date: Tue, 20 Jun 2000 17:19:03 -0600 From: Warner Losh Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message <200006202307.QAA89367@apollo.backplane.com> Matthew Dillon writes: : Hmm. I think from the point of view of pccard development you : will be in good shape. I can't really guess what the level of : instability will be, but I think if we test things enough before : committing (i.e. the machine has to survive a 'make' of the kernel) : that it should be good enough to not intefere too much with your : pccard work. I can accept that level of testing. You build the kernel, reboot and rebuild the kernel and if that succeeds, you can commit it. For most of the changes that the SMP work will be doing, this will be acceptible, to my way of seeing it. This is certainly much less severe than the original message suggested. Warner To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 20 23:53: 1 2000 Delivered-To: freebsd-smp@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 3E9F437BDCA for ; Tue, 20 Jun 2000 23:52:58 -0700 (PDT) (envelope-from jasone@magnesium.net) Received: (qmail 91425 invoked by uid 1142); 21 Jun 2000 06:52:57 -0000 Date: 20 Jun 2000 23:52:57 -0700 Date: Tue, 20 Jun 2000 23:48:39 -0700 From: Jason Evans To: Matthew Dillon Cc: Matthew Jacob , freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp Message-ID: <20000620234839.C233@blitz.canonware.com> References: <200006201754.KAA87277@apollo.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <200006201754.KAA87277@apollo.backplane.com>; from dillon@apollo.backplane.com on Tue, Jun 20, 2000 at 10:54:34AM -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, Jun 20, 2000 at 10:54:34AM -0700, Matthew Dillon wrote: > Yes, all i386. At the meeting we decided to break the alpha port > temporarily. I have no recollection of this even being discussed, let alone decided on. > I think alpha work can begin when I make the SP (single processor) > patches available. I would recommend sticking to SP alpha builds > probably for a few months until our i386 MP work (which is mostly > machine independant) is reasonably stable. (does alpha even have an MP > build yet?). I for one have an alpha that I really want this SMP work to run on in a timely fashion. Granted, alpha may reasonably lag behind i386 by some amount (on the order of days), but FreeBSD is no longer just i386, and we need to give the alpha port enough attention to keep it more or less in sync with the i386 port. Matt, we've threatened to provide you with an alpha, and we may force one on you if the need arises. =) Seriously though, alpha is not going to be intentionally left in the dust on this project. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 21 0:17:23 2000 Delivered-To: freebsd-smp@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 3B4C237BDC6 for ; Wed, 21 Jun 2000 00:17:21 -0700 (PDT) (envelope-from jasone@magnesium.net) Received: (qmail 91947 invoked by uid 1142); 21 Jun 2000 07:17:21 -0000 Date: 21 Jun 2000 00:17:21 -0700 Date: Wed, 21 Jun 2000 00:10:50 -0700 From: Jason Evans To: Warner Losh Cc: The Hermit Hacker , freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp Message-ID: <20000621001050.E233@blitz.canonware.com> References: <200006202311.RAA75680@harmony.village.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <200006202311.RAA75680@harmony.village.org>; from imp@village.org on Tue, Jun 20, 2000 at 05:11:37PM -0600 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tue, Jun 20, 2000 at 05:11:37PM -0600, Warner Losh wrote: > In message The Hermit Hacker writes: > : Which one would be harder to merge back into -current from a branch? Your > : NEWCARD work or the SMP work? Why not make a branch for NEWCARD and when > : the SMP work is declared stable, *you* spend the effort of merging it back > : in? > > I would do that if the SMP stuff is really bad. I think that Matt's > assuances indicate this won't be the case. > > However, it isn't just NEWCARD. It is every other project that is > going on in -current right now. There's work on the usb stack, in the > sound system, in the config system, etc. Should all of them run on a > branch? No. They shouldn't. If the SMP folks were to have done what > was implied in the original message, I think we'd all get together and > create a branch for real work. > > Having said that, I think that the SMP folks now understand the > importance of keeping the really bad breakage down to a few days and > keeping the base of the system somewhat stable. I don't think a > branch will be necessary unless the SMP folks aren't able to do what > they say they are going to do. This is in my opinion a bit revisionist, but part of the blame falls on me for not clarifying "destabilization" adequately. -current will continue to compile for the most part, and may even work okay most of the time. However, there are a couple of transitions we will have to make that will cause massive instability for periods of as much as a week or two as we smash all the major problems. This is not anything too far out of the ordinary with regard to -current, which is meant for development. The difference is that we expect this periods of instability to be unavoidable. We'll push through them as quickly as possible, but things will be a bit iffy at times. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 21 0:22: 1 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 914E937BDB5 for ; Wed, 21 Jun 2000 00:21:58 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id AAA91176; Wed, 21 Jun 2000 00:21:57 -0700 (PDT) (envelope-from dillon) Date: Wed, 21 Jun 2000 00:21:57 -0700 (PDT) From: Matthew Dillon Message-Id: <200006210721.AAA91176@apollo.backplane.com> To: Jason Evans Cc: Matthew Jacob , freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp References: <200006201754.KAA87277@apollo.backplane.com> <20000620234839.C233@blitz.canonware.com> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org : :On Tue, Jun 20, 2000 at 10:54:34AM -0700, Matthew Dillon wrote: :> Yes, all i386. At the meeting we decided to break the alpha port :> temporarily. : :I have no recollection of this even being discussed, let alone decided on. : :> I think alpha work can begin when I make the SP (single processor) :> patches available. I would recommend sticking to SP alpha builds :> probably for a few months until our i386 MP work (which is mostly :> machine independant) is reasonably stable. (does alpha even have an MP :> build yet?). : :I for one have an alpha that I really want this SMP work to run on in a :timely fashion. Granted, alpha may reasonably lag behind i386 by some :amount (on the order of days), but FreeBSD is no longer just i386, and we :need to give the alpha port enough attention to keep it more or less in :sync with the i386 port. : :Matt, we've threatened to provide you with an alpha, and we may force one :on you if the need arises. =) Seriously though, alpha is not going to be :intentionally left in the dust on this project. :.. :Jason I definitely remember it being brought up... I forget who brought the point out but I'm sure it was one of the alpha guys. My impression was that someone on the Alpha team would pick up the ball. There is no way I have time to do Alpha assembly, even if I had experience with it (which I don't). I'm already taking a chunk out of work just to do the i386 code. One of the alpha guys is going to have to do that piece of it. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 21 0:22:24 2000 Delivered-To: freebsd-smp@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.40.131]) by hub.freebsd.org (Postfix) with ESMTP id 5D0CB37BEDA for ; Wed, 21 Jun 2000 00:22:21 -0700 (PDT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.9.3/8.9.3) with ESMTP id JAA58203; Wed, 21 Jun 2000 09:22:02 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Jason Evans Cc: Warner Losh , The Hermit Hacker , freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp In-reply-to: Your message of "Wed, 21 Jun 2000 00:10:50 PDT." <20000621001050.E233@blitz.canonware.com> Date: Wed, 21 Jun 2000 09:22:02 +0200 Message-ID: <58201.961572122@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message <20000621001050.E233@blitz.canonware.com>, Jason Evans writes: >This is in my opinion a bit revisionist, but part of the blame falls on me >for not clarifying "destabilization" adequately. -current will continue to >compile for the most part, and may even work okay most of the time. I think it is this "may" that people are not comfortable with. It sounds like "Hey, it's broken but - what me worry ?" rather than "It's broken we know, we're doing our best to fix it." If we had heard you say "Unfortunately this will mean that -current will be broken at times, but we will do what we can to minimize this so we don't impact the rest of the FreeBSD developers more than absolutely necessary." I'm sure people would not be up in arms the way they are. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD coreteam member | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 21 0:27:12 2000 Delivered-To: freebsd-smp@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id 2538037B9FB for ; Wed, 21 Jun 2000 00:27:10 -0700 (PDT) (envelope-from mjacob@feral.com) Received: from beppo.feral.com (beppo [192.67.166.79]) by feral.com (8.9.3/8.9.3) with ESMTP id AAA19395; Wed, 21 Jun 2000 00:27:06 -0700 Date: Wed, 21 Jun 2000 00:27:07 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: Matthew Dillon Cc: Jason Evans , freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp In-Reply-To: <200006210721.AAA91176@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > I definitely remember it being brought up... I forget who brought > the point out but I'm sure it was one of the alpha guys. My impression > was that someone on the Alpha team would pick up the ball. I offered you an alpha last week, remember? Actually, I originally wrote the mail I wrote to try and find out what was needing doing with alpha and try and find out what the plans were so I could schedule time for it. > There is no way I have time to do Alpha assembly, even if I had > experience with it (which I don't). I'm already taking a chunk out of > work just to do the i386 code. One of the alpha guys is going to have > to do that piece of it. Wish you'd said this paragraph first instead of "alpha's gonna break". Doug, I or Andrew can and will pick up the work and fix the alpha. That is, we all have alphas, interest in SMP, and so on. Doug will possibly get to it first. I'm gone from the effort now, but if it's broken next month, I'm sure I'll get back to it. I first have to untangle the config hint breakage for Alpha 8200s. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 21 0:29:47 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id CC4EE37B9BA for ; Wed, 21 Jun 2000 00:29:44 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id AAA91264; Wed, 21 Jun 2000 00:29:43 -0700 (PDT) (envelope-from dillon) Date: Wed, 21 Jun 2000 00:29:43 -0700 (PDT) From: Matthew Dillon Message-Id: <200006210729.AAA91264@apollo.backplane.com> To: Matthew Jacob Cc: Jason Evans , freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp References: Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org : : :> :> I definitely remember it being brought up... I forget who brought :> the point out but I'm sure it was one of the alpha guys. My impression :> was that someone on the Alpha team would pick up the ball. : :I offered you an alpha last week, remember? : :Actually, I originally wrote the mail I wrote to try and find out what was :needing doing with alpha and try and find out what the plans were so I could :schedule time for it. Yes, and I said that I *DIDN'T HAVE TIME*. Guys, I'm working 60 hour weeks in my new startup. I don't have the time to learn Alpha assembly, and I don't have time to double-code all the machine dependant pieces even if I *did* know Alpha assembly. It's that simple. My problem is not hardware... I could buy as much hardware as I wanted from what I got out of BEST. It's time that's the problem. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 21 0:33:51 2000 Delivered-To: freebsd-smp@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id 8FA6B37B9BA for ; Wed, 21 Jun 2000 00:33:45 -0700 (PDT) (envelope-from mjacob@feral.com) Received: from beppo.feral.com (beppo [192.67.166.79]) by feral.com (8.9.3/8.9.3) with ESMTP id AAA19422; Wed, 21 Jun 2000 00:33:41 -0700 Date: Wed, 21 Jun 2000 00:33:42 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: Matthew Dillon Cc: Jason Evans , freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp In-Reply-To: <200006210729.AAA91264@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > : > : > :> > :> I definitely remember it being brought up... I forget who brought > :> the point out but I'm sure it was one of the alpha guys. My impression > :> was that someone on the Alpha team would pick up the ball. > : > :I offered you an alpha last week, remember? > : > :Actually, I originally wrote the mail I wrote to try and find out what was > :needing doing with alpha and try and find out what the plans were so I could > :schedule time for it. > > Yes, and I said that I *DIDN'T HAVE TIME*. > > Guys, I'm working 60 hour weeks in my new startup. I don't have the > time to learn Alpha assembly, and I don't have time to double-code > all the machine dependant pieces even if I *did* know Alpha assembly. Matt- I didn't expect you to do this. Really- I'm not blaming you for that portion of it. Do what you can. I'm ticked at the overall attitude of 'you lose' for alpha, but that's not your fault. A statement like "Yeah- we'll break alpha, likely. Sorry. Can you pick it up?" would have made all the difference. > It's that simple. My problem is not hardware... I could buy as much > hardware as I wanted from what I got out of BEST. It's time that's > the problem. Right. Let's stop this thread already, then. -matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 21 0:41:27 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 5A2A637B787 for ; Wed, 21 Jun 2000 00:41:24 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id AAA91331; Wed, 21 Jun 2000 00:41:03 -0700 (PDT) (envelope-from dillon) Date: Wed, 21 Jun 2000 00:41:03 -0700 (PDT) From: Matthew Dillon Message-Id: <200006210741.AAA91331@apollo.backplane.com> To: Jason Evans , Greg Lehey Cc: Warner Losh , The Hermit Hacker , freebsd-smp@FreeBSD.ORG Subject: SP Patchset #1 up Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org The single-processor patchset is now up on my web page: http://apollo.backplane.com/FreeBSDSmp/ This patch set will break MP builds, and it will break the alpha (since interrupt requirements have changed and machine-dependant code needs new functions). We will not be able to make MP builds work until Greg has got the heavy-weight interrupt threading in place and I rewrite the MP interrupt code (which is set of assembly files independant from the SP interrupt code). My part of it will take another weekend. People using this patchset should do minimalist SP builds. I have tested it on an IDE and a SCSI box and it appears to work (read: not crash if I build a kernel on it). There are lots of pieces missing, I probably missed some of the SPL code (that is, SPL code that runs outside of the GiantMutex. Stuff inside GiantMutex is just fine). But it's a good base to start with... it should be good enough for Greg to be able to start working on the interrupt threads. I hacked the syscons spl's to pieces, they are almost certainly not completely correct but probably not any worse then they were before (all those spltty's were useless for the most part since SIO uses a fast interrupt which ignores cpl protection already). I will put a document together on the web page describing the new calls. This is a very rough patchset, it's going to take a lot of hands to smooth out. Remember that the goal of this patchset is to get most of the infrastructure in place so that others can start working on the system in parallel. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 21 0:47: 1 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id D774337B9BA for ; Wed, 21 Jun 2000 00:46:58 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id AAA91380; Wed, 21 Jun 2000 00:46:57 -0700 (PDT) (envelope-from dillon) Date: Wed, 21 Jun 2000 00:46:57 -0700 (PDT) From: Matthew Dillon Message-Id: <200006210746.AAA91380@apollo.backplane.com> To: Jason Evans , Greg Lehey , Warner Losh , The Hermit Hacker , freebsd-smp@FreeBSD.ORG Subject: Re: SP Patchset #1 up References: <200006210741.AAA91331@apollo.backplane.com> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Oh, I forgot to mention... the hardclock/statclock is probably not working completely properly yet. They work, but there are some special-cases in there that used spl*()'s in a special way that I ripped out. I'm not sure it's a factor any more (I don't see anything wrong in my tests) but once we get things stabilized I'd like someone to help me review that whole codepath. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 21 1:11:19 2000 Delivered-To: freebsd-smp@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id F204C37BD04 for ; Wed, 21 Jun 2000 01:11:14 -0700 (PDT) (envelope-from jasone@magnesium.net) Received: (qmail 93476 invoked by uid 1142); 21 Jun 2000 08:11:14 -0000 Date: 21 Jun 2000 01:11:14 -0700 Date: Wed, 21 Jun 2000 01:11:10 -0700 From: Jason Evans To: Matthew Dillon Cc: Matthew Jacob , freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp Message-ID: <20000621011110.H233@blitz.canonware.com> References: <200006210729.AAA91264@apollo.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <200006210729.AAA91264@apollo.backplane.com>; from dillon@apollo.backplane.com on Wed, Jun 21, 2000 at 12:29:43AM -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Wed, Jun 21, 2000 at 12:29:43AM -0700, Matthew Dillon wrote: > :I offered you an alpha last week, remember? > > Yes, and I said that I *DIDN'T HAVE TIME*. > > Guys, I'm working 60 hour weeks in my new startup. I don't have the > time to learn Alpha assembly, and I don't have time to double-code > all the machine dependant pieces even if I *did* know Alpha assembly. > > It's that simple. My problem is not hardware... I could buy as much > hardware as I wanted from what I got out of BEST. It's time that's > the problem. There are a couple of issues that need to be resolved here. Matt (Dillon), you've made it clear that you don't have the time/background/desire to do the alpha bits. That is totally acceptable; your work is valuable even without the alpha bits. The other issue has to do with how we are going to approach the SMP work with regard to the alpha port. We must keep the i386 and alpha ports more or less in sync, because this is such a wide-sweeping change to the kernel. Since we have different people doing the machine-dependent work for i386 and alpha, this is going to require some communication to get the bits ready for committing in approximately the same timeframe. Working on both i386 and alpha will probably mean that our first couple of stages will take a bit longer, but we cannot afford to leave alpha in the dust right off the bat. This might cost us a couple of weeks up front, but I think it is well worth the effort, and may more than pay off in the long run if it avoids having to do re-work to factor out machine-dependent portions of the work. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 21 3:54:59 2000 Delivered-To: freebsd-smp@freebsd.org Received: from web1902.mail.yahoo.com (web1902.mail.yahoo.com [128.11.23.51]) by hub.freebsd.org (Postfix) with SMTP id ECD6037BCEA for ; Wed, 21 Jun 2000 03:54:56 -0700 (PDT) (envelope-from p_k_nandan@yahoo.com) Received: (qmail 23000 invoked by uid 60001); 21 Jun 2000 10:54:56 -0000 Message-ID: <20000621105456.22999.qmail@web1902.mail.yahoo.com> Received: from [203.197.190.137] by web1902.mail.yahoo.com; Wed, 21 Jun 2000 03:54:56 PDT Date: Wed, 21 Jun 2000 03:54:56 -0700 (PDT) From: "NandaKumar P.K." Subject: SMP kernel crash To: freebsd-smp@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi, I am trying to configure a SMP kernel. The Motherboard has two CPUs. The following options are set in the Kernel configuration file. options SMP options APIC_IO options NCPU=2 options NBUS=4 options NAPIC=1 options NINTR=61 mptable gave the same values as above. But the kernel crashes saying APIC RTC!=8 mp_lock=00000001 cpu_id=0 Can anyone help ? Regards, Nandan __________________________________________________ Do You Yahoo!? Send instant messages with Yahoo! Messenger. http://im.yahoo.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 21 5:10:58 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id E713037BE4A for ; Wed, 21 Jun 2000 05:10:43 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.8.7/8.8.7) with ESMTP id WAA19645; Wed, 21 Jun 2000 22:09:52 +1000 Date: Wed, 21 Jun 2000 22:09:50 +1000 (EST) From: Bruce Evans X-Sender: bde@besplex.bde.org To: Matthew Dillon Cc: Jason Evans , Greg Lehey , Warner Losh , The Hermit Hacker , freebsd-smp@FreeBSD.ORG Subject: Re: SP Patchset #1 up In-Reply-To: <200006210741.AAA91331@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Wed, 21 Jun 2000, Matthew Dillon wrote: > I hacked the syscons spl's to pieces, they are almost certainly > not completely correct but probably not any worse then they were > before (all those spltty's were useless for the most part since > SIO uses a fast interrupt which ignores cpl protection already). These were more or less correct, and unrelated to sio. Note that the low level console drivers must not use any locks or switch context, since they need to work in debugger traps which may occur with locks in any state. Old reentrancy bugs in syscons will probably be more obvious now. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 21 8:28:59 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id CC1BE37B7B4 for ; Wed, 21 Jun 2000 08:28:56 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id IAA94804; Wed, 21 Jun 2000 08:28:49 -0700 (PDT) (envelope-from dillon) Date: Wed, 21 Jun 2000 08:28:49 -0700 (PDT) From: Matthew Dillon Message-Id: <200006211528.IAA94804@apollo.backplane.com> To: Bruce Evans Cc: Jason Evans , Greg Lehey , Warner Losh , The Hermit Hacker , freebsd-smp@FreeBSD.ORG Subject: Re: SP Patchset #1 up References: Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org :These were more or less correct, and unrelated to sio. : :Note that the low level console drivers must not use any locks or switch :context, since they need to work in debugger traps which may occur with :locks in any state. Old reentrancy bugs in syscons will probably be :more obvious now. : :Bruce Yah. That's why I replaced them all with mtx_enter_sched_quick(), which is the closest equivalent to those requirements. The spl*() calls only work if you are holding GiantMutex, and that obviously is not necessarily the case for the kernel printf(). Believe me, I hit those problems early--- when I could ctl-alt-esc into DDB or when a panic would occur and not even print the panic message. I think I have those cases fixed in patch set #1. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 21 8:39:20 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 5047A37C134; Wed, 21 Jun 2000 08:39:13 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id IAA94918; Wed, 21 Jun 2000 08:39:02 -0700 (PDT) (envelope-from dillon) Date: Wed, 21 Jun 2000 08:39:02 -0700 (PDT) From: Matthew Dillon Message-Id: <200006211539.IAA94918@apollo.backplane.com> To: Nick Hibma Cc: freebsd-smp@freebsd.org Subject: Re: irunning, width in bits. References: Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org (Moving this to freebsd-smp, Bcc'ing current) :What about shared interrupts? How are they going to be treated? With the :spl leaving the arena it somehow looks feasible to run one interrupt :source on two different threads if there are two pieces of hardware :attached to the same interrupt line. : :>From what I understood from dfr, when switching away from an interrupt :handler it is converted into a full thread. When the second piece of :hardware fires an interrupt it could then run at the same time. : :Nick :-- :n_hibma@webweaving.org This came up at the meeting and the conclusion was that shared interrupts would run serially. That is, each 'bit' in the cpl (spl*(), also represented by ipending, the vector table dispatch, and so forth) would be treated as a single interrupt thread. If there are N interrupts hanging off that IRQ, then each of the N would be run serially from a single interrupt thread. This also came up a few months ago, you can probably find the discussion in the archives. What it comes down to is complexity and convenience. In almost all systems it is possible to rearrange the PCI boards such that you do not have two critical interrupts on the same IRQ. We are not precluding being able to schedule shared interrupts separately, but if we are going to do it it will probably be much later when things are more stable and not now. The KISS principle applies here. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 21 8:40:40 2000 Delivered-To: freebsd-smp@freebsd.org Received: from gilgamesch.bik-gmbh.de (T1-Hansenet.BIK-GmbH.de [192.76.134.246]) by hub.freebsd.org (Postfix) with ESMTP id D6D7B37C05D for ; Wed, 21 Jun 2000 08:40:34 -0700 (PDT) (envelope-from cracauer@gilgamesch.bik-gmbh.de) Received: (from cracauer@localhost) by gilgamesch.bik-gmbh.de (8.9.3/8.7.3) id RAA57170; Wed, 21 Jun 2000 17:37:32 +0200 (MET DST) Date: Wed, 21 Jun 2000 17:37:32 +0200 From: Martin Cracauer To: Matthew Dillon Cc: Warner Losh , Poul-Henning Kamp , mjacob@feral.com, freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp Message-ID: <20000621173732.A56214@cons.org> References: <55142.961529404@critter.freebsd.dk> <200006201936.NAA71564@harmony.village.org> <200006201954.MAA88510@apollo.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <200006201954.MAA88510@apollo.backplane.com>; from dillon@apollo.backplane.com on Tue, Jun 20, 2000 at 12:54:06PM -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In <200006201954.MAA88510@apollo.backplane.com>, Matthew Dillon wrote: > The kernel will always be buildable. It should be stable enough to > last for 5 minutes. But there are going to be a lot of legacy issues > that need to be fixed -- drivers that get broken. What will probably > happen is that a base system with only core features enabled (disk, > serial, console, network) will always be more stable during the mergework > then a system with the kitchen sink in. A system with the kitchen > sink in may not even compile! Please give us a list of devices in your development machine. I'm certainly willing to build a similar machine to walk in your footsteps. Another suggestion: It would be very useful is to keep stability on the kernel->userland API for the transition period. That means, if -current becomes unstable from Jul, 1th until the end of August, I'd like to be able to live with an userland from Jun, 30th for all the then-current kernels. Modulo ps/top, which I can rebuild on demand, but - if possible - no problems with essential system-near tools like mount and fsck. Martin -- %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Martin Cracauer http://www.cons.org/cracauer/ BSD User Group Hamburg, Germany http://www.bsdhh.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 21 9: 3:44 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id AED6E37BEA8 for ; Wed, 21 Jun 2000 09:03:35 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id JAA95167; Wed, 21 Jun 2000 09:03:28 -0700 (PDT) (envelope-from dillon) Date: Wed, 21 Jun 2000 09:03:28 -0700 (PDT) From: Matthew Dillon Message-Id: <200006211603.JAA95167@apollo.backplane.com> To: Martin Cracauer Cc: Warner Losh , Poul-Henning Kamp , mjacob@feral.com, freebsd-smp@FreeBSD.ORG Subject: Re: SMP discussion moving to freebsd-smp References: <55142.961529404@critter.freebsd.dk> <200006201936.NAA71564@harmony.village.org> <200006201954.MAA88510@apollo.backplane.com> <20000621173732.A56214@cons.org> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org :.. :> happen is that a base system with only core features enabled (disk, :> serial, console, network) will always be more stable during the mergework :> then a system with the kitchen sink in. A system with the kitchen :> sink in may not even compile! : :Please give us a list of devices in your development machine. I'm :certainly willing to build a similar machine to walk in your :footsteps. Probably not useful, but I've appended my TEST3 kernel config file. :Another suggestion: It would be very useful is to keep stability on :the kernel->userland API for the transition period. That means, if :-current becomes unstable from Jul, 1th until the end of August, I'd :like to be able to live with an userland from Jun, 30th for all the :then-current kernels. : :Modulo ps/top, which I can rebuild on demand, but - if possible - no :problems with essential system-near tools like mount and fsck. : :Martin I don't forsee any major changes to the userland. There will probably be a lot of recompiling of libkvm and related programs, though. -Matt machine i386 #cpu I386_CPU #cpu I486_CPU cpu I586_CPU cpu I686_CPU ident TEST3 maxusers 128 hints "GENERIC.hints" #Default places to look for devices. options IDE_DELAY=5000 options SCSI_DELAY=5000 options MAXMEM="(256*1024)" options DDB options INVARIANTS options INVARIANT_SUPPORT options SOFTUPDATES #options SHMALL=65537 #options SHMMAXPGS=65537 # -Os broken # makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols makeoptions COPTFLAGS="-O -pipe" #options MATH_EMULATE #Support for x87 emulation options INET #InterNETworking #options INET6 #IPv6 communications protocols options FFS #Berkeley Fast Filesystem options FFS_ROOT #FFS usable as root device [keep this!] options MFS #Memory Filesystem options MD_ROOT #MD is a potential root device options NFS #Network Filesystem options NFS_ROOT #NFS usable as root device, NFS required options MSDOSFS #MSDOS Filesystem options CD9660 #ISO 9660 Filesystem options CD9660_ROOT #CD-ROM usable as root, CD9660 required options PROCFS #Process filesystem options COMPAT_43 #Compatible with BSD 4.3 [KEEP THIS!] #options SCSI_DELAY=15000 #Delay (in ms) before probing SCSI options UCONSOLE #Allow users to grab the console options USERCONFIG #boot -c editor options VISUAL_USERCONFIG #visual boot -c editor options KTRACE #ktrace(1) support options SYSVSHM #SYSV-style shared memory options SYSVMSG #SYSV-style message queues options SYSVSEM #SYSV-style semaphores #options P1003_1B #Posix P1003_1B real-time extensions #options _KPOSIX_PRIORITY_SCHEDULING #options KBD_INSTALL_CDEV # install a CDEV entry in /dev # To make an SMP kernel, the next two are needed # XXX #options SMP # Symmetric MultiProcessor Kernel #options APIC_IO # Symmetric (APIC) I/O # Optionally these may need tweaked, (defaults shown): #options NCPU=2 # number of CPUs #options NBUS=8 # number of busses #options NAPIC=1 # number of IO APICs #options NINTR=24 # number of INTs device isa #device eisa device pci #options COMPAT_OLDISA # compatability shims for lnc, fe, le #options COMPAT_OLDPCI # compatability shims for lnc, vx # Floppy drives device fdc # ATA and ATAPI devices device ata device atadisk # ATA disk drives device atapicd # ATAPI CDROM drives device atapifd # ATAPI floppy drives device atapist # ATAPI tape drives options ATA_STATIC_ID #Static device numbering #options ATA_ENABLE_ATAPI_DMA #Enable DMA on ATAPI devices # SCSI Controllers #device ahb # EISA AHA1742 family device ahc # AHA2940 and onboard AIC7xxx devices #device amd # AMD 53C974 (Teckram DC-390(T)) #device dpt # DPT Smartcache - See LINT for options! #device isp # Qlogic family #device ncr # NCR/Symbios Logic #device sym # NCR/Symbios Logic (newer chipsets) #device adv #device adw #device bt #device aha 1 #device aic # SCSI peripherals device scbus # SCSI bus (required) device da # Direct Access (disks) device sa # Sequential Access (tape etc) device cd # CD device pass # Passthrough device (direct SCSI access) # RAID controllers #device ida # Compaq Smart RAID #device amr # AMI MegaRAID #device mlx # Mylex DAC960 family # atkbdc0 controls both the keyboard and the PS/2 mouse device atkbdc 1 device atkbd device psm device vga # splash screen/screen saver device splash # syscons is the default console driver, resembling an SCO console device sc 1 # Enable this for the pcvt (VT220 compatible) console driver #device vt #options XSERVER # support for X server on a vt console #options FAT_CURSOR # start with block cursor # If you have a ThinkPAD, uncomment this along with the rest of the PCVT lines #options PCVT_SCANSET=2 # IBM keyboards are non-std # Floating point support - do not disable. device npx ## Power management support (see LINT for more options) #device apm # ## PCCARD (PCMCIA) support #device card #device pcic # Serial (COM) ports device sio ## Parallel port #device ppc #device ppbus # Parallel port bus (required) #device lpt # Printer #device plip # TCP/IP over parallel #device ppi # Parallel port interface device ##device vpo # Requires scbus and da # PCI Ethernet NICs. device de # DEC/Intel DC21x4x (``Tulip'') device fxp # Intel EtherExpress PRO/100B (82557, 82558) #device tx # SMC 9432TX (83c170 ``EPIC'') #device vx # 3Com 3c590, 3c595 (``Vortex'') #device wx # Intel Gigabit Ethernet Card (``Wiseman'') # PCI Ethernet NICs that use the common MII bus controller code. device miibus # MII bus support device dc # DEC/Intel 21143 and various workalikes #device rl # RealTek 8129/8139 #device sf # Adaptec AIC-6915 (``Starfire'') #device sis # Silicon Integrated Systems SiS 900/SiS 7016 #device ste # Sundance ST201 (D-Link DFE-550TX) #device tl # Texas Instruments ThunderLAN #device vr # VIA Rhine, Rhine II #device wb # Winbond W89C840F #device xl # 3Com 3c90x (``Boomerang'', ``Cyclone'') ## ISA Ethernet NICs. #device ed #device ex #device ep #device cs #device sn ## WaveLAN/IEEE 802.11 wireless NICs. Note: the WaveLAN/IEEE really ## exists only as a PCMCIA device, so there is no ISA attatement needed ## and resources will always be dynamically assigned by the pccard code. #device wi ## Aironet 4500/4800 802.11 wireless NICs. Note: the declaration below will ## work for PCMCIA and PCI cards, as well as ISA cards set to ISA PnP ## mode (the factory default). If you set the switches on your ISA ## card for a manually chosen I/O address and IRQ, you must specify ## those paremeters here. #device an ## BayStack 660 and others #device awi ## Xircom pccard ethernet #device xe ## The probe order of these is presently determined by i386/isa/isa_compat.c. #device ie #device fe #device le #device lnc # Pseudo devices - the number indicates how many units to allocated. device loop # Network loopback device ether # Ethernet support device sl 4 # Kernel SLIP device ppp 4 # Kernel PPP device tun 4 # Packet tunnel. device pty # Pseudo-ttys (telnet etc) device ccd 4 device md # Memory "disks" device gif 4 # IPv6 and IPv4 tunneling device faith 1 # IPv6-to-IPv4 relaying (translation) # The `bpf' device enables the Berkeley Packet Filter. # Be aware of the administrative consequences of enabling this! device bpf 4 # Berkeley packet filter ## USB support #device uhci # UHCI PCI->USB interface #device ohci # OHCI PCI->USB interface #device usb # USB Bus (required) ##device udbp # USB Double Bulk Pipe devices #device ugen # Generic #device uhid # "Human Interface Devices" #device ukbd # Keyboard #device ulpt # Printer #device umass # Disks/Mass storage - Requires scbus and da #device ums # Mouse #device urio # Diamond Rio 500 MP3 player ## USB Ethernet, requires mii #device aue # ADMtek USB ethernet #device cue # CATC USB ethernet #device kue # Kawasaki LSI USB ethernet To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 21 9:54:51 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp03.primenet.com (smtp03.primenet.com [206.165.6.133]) by hub.freebsd.org (Postfix) with ESMTP id 1156D37BE9B for ; Wed, 21 Jun 2000 09:54:49 -0700 (PDT) (envelope-from tlambert@usr08.primenet.com) Received: (from daemon@localhost) by smtp03.primenet.com (8.9.3/8.9.3) id JAA24691; Wed, 21 Jun 2000 09:54:24 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp03.primenet.com, id smtpdAAA..aylW; Wed Jun 21 09:54:19 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id JAA28545; Wed, 21 Jun 2000 09:54:29 -0700 (MST) From: Terry Lambert Message-Id: <200006211654.JAA28545@usr08.primenet.com> Subject: Re: SMP discussion moving to freebsd-smp To: imp@village.org (Warner Losh) Date: Wed, 21 Jun 2000 16:54:28 +0000 (GMT) Cc: phk@critter.freebsd.dk (Poul-Henning Kamp), mjacob@feral.com, dillon@apollo.backplane.com (Matthew Dillon), freebsd-smp@FreeBSD.ORG In-Reply-To: <200006201942.NAA71681@harmony.village.org> from "Warner Losh" at Jun 20, 2000 01:42:17 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > : The meaning here is: No, it can't be basically useless for months > : on end, but yes, it may be dead for days or maybe even a week. > > Yes. I agree. I have no problems with it being bolluxed up for 1 day > or 1 week or even if it will gain us a lot, 2 weeks. Beyond that, it > becomes too painful for me to use the tree. I susepct others would > agree with threasholds in this general area, some tigheter some > looser. Too bad the source code control tool that FreeBSD uses doesn't support multiple lines of developement. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 21 22:12:14 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 656FE37B5AD for ; Wed, 21 Jun 2000 22:12:12 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id WAA99006; Wed, 21 Jun 2000 22:12:04 -0700 (PDT) (envelope-from dillon) Date: Wed, 21 Jun 2000 22:12:04 -0700 (PDT) From: Matthew Dillon Message-Id: <200006220512.WAA99006@apollo.backplane.com> To: Matthew Dillon Cc: Jason Evans , Greg Lehey , Warner Losh , The Hermit Hacker , freebsd-smp@FreeBSD.ORG Subject: SP Patchset #2 up (one minor bug fix) Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I fixed a minor race condition in the Mutex code in tsleep(). It was releasing the interrupt mask (spl0()) too early and allowing an interrupt in which might attempt to wake the id up before the process could actually be queued to it, resulting in a forever-blocked process. Apart from that, the patch set appears to be reasonably stable. http://www.backplane.com/FreeBSDSmp/ -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 21 22:26: 7 2000 Delivered-To: freebsd-smp@freebsd.org Received: from sharmas.dhs.org (c62443-a.frmt1.sfba.home.com [24.0.69.165]) by hub.freebsd.org (Postfix) with ESMTP id 88F0037B94E for ; Wed, 21 Jun 2000 22:26:05 -0700 (PDT) (envelope-from adsharma@sharmas.dhs.org) Received: (from adsharma@localhost) by sharmas.dhs.org (8.9.3/8.9.3) id WAA29645; Wed, 21 Jun 2000 22:25:15 -0700 Date: Wed, 21 Jun 2000 22:25:15 -0700 From: Arun Sharma Message-Id: <200006220525.WAA29645@sharmas.dhs.org> To: tlambert@primenet.com Cc: smp@freebsd.org Subject: source control (Was Re: SMP discussion moving to freebsd-smp) In-Reply-To: <200006211654.JAA28545@usr08.primenet.com> References: <200006211654.JAA28545@usr08.primenet.com> Reply-To: adsharma@sharmas.dhs.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Wed, 21 Jun 2000 16:54:28 +0000 (GMT), Terry Lambert wrote: > > Yes. I agree. I have no problems with it being bolluxed up for 1 day > > or 1 week or even if it will gain us a lot, 2 weeks. Beyond that, it > > becomes too painful for me to use the tree. I susepct others would > > agree with threasholds in this general area, some tigheter some > > looser. > > Too bad the source code control tool that FreeBSD uses doesn't > support multiple lines of developement. AFAIK, you can have multiple branches in cvs. You probably meant, it doesn't support multiple lines of developement _well_ ? Or am I overlooking some obvious shortcoming of cvs ? At work, we use perforce. The nice thing (which cvs doesn't have) about it is that it has a concept of a changeset. Any extended discussion is off topic here, I think. I don't mind offline email though :) -Arun To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Jun 22 0:57: 7 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 2416D37C1D0 for ; Thu, 22 Jun 2000 00:57:02 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.8.7/8.8.7) with ESMTP id RAA15045; Thu, 22 Jun 2000 17:55:50 +1000 Date: Thu, 22 Jun 2000 17:55:48 +1000 (EST) From: Bruce Evans X-Sender: bde@besplex.bde.org To: Matthew Dillon Cc: Jason Evans , Greg Lehey , Warner Losh , The Hermit Hacker , freebsd-smp@FreeBSD.ORG Subject: Re: SP Patchset #1 up In-Reply-To: <200006211528.IAA94804@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Wed, 21 Jun 2000, Matthew Dillon wrote: > :Note that the low level console drivers must not use any locks or switch > :context, since they need to work in debugger traps which may occur with > :locks in any state. Old reentrancy bugs in syscons will probably be > :more obvious now. > > Yah. That's why I replaced them all with mtx_enter_sched_quick(), > which is the closest equivalent to those requirements. The spl*() > calls only work if you are holding GiantMutex, and that obviously > is not necessarily the case for the kernel printf(). You now have deadlock when the low level console i/o routines are reentered for ddb i/o, since task switching is either impossible or undesireable while ddb is active (it's essentially impossible if ddb was entered via a breakpoint in the middle of task switching code). When task switching is possible to avoid deadlock on console i/o resources, it may still be undesirable. A printf from a high priority interrupt task shouldn't have to wait for a printf from a low priority task to complete. > Believe me, I hit those problems early--- when I could ctl-alt-esc > into DDB or when a panic would occur and not even print the panic > message. I think I have those cases fixed in patch set #1. Those are slightly different problems. spltty() was used for quasi-locking different things. It gives deadlock when translated to real locking. The things are different so there is no inherent problem. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Jun 22 8:58:50 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id C595137C326 for ; Thu, 22 Jun 2000 08:58:46 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id IAA03062; Thu, 22 Jun 2000 08:58:40 -0700 (PDT) (envelope-from dillon) Date: Thu, 22 Jun 2000 08:58:40 -0700 (PDT) From: Matthew Dillon Message-Id: <200006221558.IAA03062@apollo.backplane.com> To: Bruce Evans Cc: Jason Evans , Greg Lehey , Warner Losh , The Hermit Hacker , freebsd-smp@FreeBSD.ORG Subject: Re: SP Patchset #1 up References: Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org :> :Note that the low level console drivers must not use any locks or switch :> :context, since they need to work in debugger traps which may occur with :> :locks in any state. Old reentrancy bugs in syscons will probably be :> :more obvious now. :> :> Yah. That's why I replaced them all with mtx_enter_sched_quick(), :> which is the closest equivalent to those requirements. The spl*() :> calls only work if you are holding GiantMutex, and that obviously :> is not necessarily the case for the kernel printf(). : :You now have deadlock when the low level console i/o routines are :reentered for ddb i/o, since task switching is either impossible or :undesireable while ddb is active (it's essentially impossible if ddb :was entered via a breakpoint in the middle of task switching code). : :When task switching is possible to avoid deadlock on console i/o :resources, it may still be undesirable. A printf from a high priority :interrupt task shouldn't have to wait for a printf from a low priority :task to complete. I don't follow. Why do I have a deadlock? This is SchedMutex I'm talking about here, not GiantMutex. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Jun 22 11:17:39 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp01.primenet.com (smtp01.primenet.com [206.165.6.131]) by hub.freebsd.org (Postfix) with ESMTP id C852737BE95 for ; Thu, 22 Jun 2000 11:17:35 -0700 (PDT) (envelope-from tlambert@usr08.primenet.com) Received: (from daemon@localhost) by smtp01.primenet.com (8.9.3/8.9.3) id LAA01938; Thu, 22 Jun 2000 11:17:25 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp01.primenet.com, id smtpdAAALkayac; Thu Jun 22 11:15:27 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id LAA06872; Thu, 22 Jun 2000 11:15:23 -0700 (MST) From: Terry Lambert Message-Id: <200006221815.LAA06872@usr08.primenet.com> Subject: Re: source control (Was Re: SMP discussion moving to freebsd-smp) To: adsharma@sharmas.dhs.org Date: Thu, 22 Jun 2000 18:15:23 +0000 (GMT) Cc: tlambert@primenet.com, smp@freebsd.org In-Reply-To: <200006220525.WAA29645@sharmas.dhs.org> from "Arun Sharma" at Jun 21, 2000 10:25:15 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > Too bad the source code control tool that FreeBSD uses doesn't > > support multiple lines of developement. > > AFAIK, you can have multiple branches in cvs. You probably meant, it > doesn't support multiple lines of developement _well_ ? Or am I > overlooking some obvious shortcoming of cvs ? > > At work, we use perforce. The nice thing (which cvs doesn't have) > about it is that it has a concept of a changeset. This is equivalent to what Larry McVoy calls "an LOD" or "Line Of Developement". The problem with CVS is reconciliation of lines of developement with the HEAD branch. To make this work, you have to consider each LOD as a container for historical LODs, and the winning code for any given problem would propagate up to the "HEAD" -- the top level in a hierarchy. The CVS model is HEAD-centric, which means that there is always "One True FreeBSD". From a Physics standpoint, you could see multiple LODs as an uncertainty wave that only collapses to particular code when an observer named "-RELEASE" looks at it. 8-). > Any extended discussion is off topic here, I think. I don't mind > offline email though :) It's actually appropriate for -arch, I think. Our tools constrain what we can implement more than our thinking does. For example, the linker set code in FreeBSD wasn't possible until it was supported by our tools; now we could not imagine going through the contortions not having them would require. I would even posit that some of the problems that are solved today using linker sets would not have been permitted to be solved, if it meant adding convoluted and difficult for a layman to understand code to accomplish the same ends. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Jun 22 14: 1:38 2000 Delivered-To: freebsd-smp@freebsd.org Received: from sydney.worldwide.lemis.com (pm236.conference.usenix.org [209.179.127.236]) by hub.freebsd.org (Postfix) with ESMTP id 38C1337B6BF for ; Thu, 22 Jun 2000 14:01:34 -0700 (PDT) (envelope-from grog@sydney.worldwide.lemis.com) Received: (from grog@localhost) by sydney.worldwide.lemis.com (8.9.3/8.9.3) id OAA01038; Thu, 22 Jun 2000 14:01:07 -0700 (PDT) (envelope-from grog) Date: Thu, 22 Jun 2000 14:01:07 -0700 From: Greg Lehey To: Matthew Dillon Cc: Nick Hibma , freebsd-smp@FreeBSD.ORG Subject: Re: irunning, width in bits. Message-ID: <20000622140107.E845@sydney.worldwide.lemis.com> References: <200006211539.IAA94918@apollo.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0pre2i In-Reply-To: <200006211539.IAA94918@apollo.backplane.com> Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Wednesday, 21 June 2000 at 8:39:02 -0700, Matthew Dillon wrote: > (Moving this to freebsd-smp, Bcc'ing current) > >> What about shared interrupts? How are they going to be treated? With the >> spl leaving the arena it somehow looks feasible to run one interrupt >> source on two different threads if there are two pieces of hardware >> attached to the same interrupt line. >> >>> From what I understood from dfr, when switching away from an interrupt >> handler it is converted into a full thread. When the second piece of >> hardware fires an interrupt it could then run at the same time. > > This came up at the meeting and the conclusion was that shared > interrupts would run serially. That is, each 'bit' in the cpl > (spl*(), also represented by ipending, the vector table > dispatch, and so forth) would be treated as a single interrupt > thread. If there are N interrupts hanging off that IRQ, then > each of the N would be run serially from a single interrupt > thread. I think, however, that Nick's suggestion is a thing we should follow up on--*after* we have got the simple case working. Once we have the ability to block in interrupt context, I'm sure we'll find lots of applications for our new-found freedom, some of them good. But first we need to understand the environment. Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Jun 22 23:11:48 2000 Delivered-To: freebsd-smp@freebsd.org Received: from io.yi.org (24.67.218.186.bc.wave.home.com [24.67.218.186]) by hub.freebsd.org (Postfix) with ESMTP id E703B37B628 for ; Thu, 22 Jun 2000 23:11:42 -0700 (PDT) (envelope-from jburkhol@home.com) Received: from io.yi.org (localhost.gvcl1.bc.wave.home.com [127.0.0.1]) by io.yi.org (Postfix) with ESMTP id 1A4A7BA4E; Thu, 22 Jun 2000 23:11:58 -0700 (PDT) X-Mailer: exmh version 2.1.1 10/15/1999 To: Greg Lehey Cc: Matthew Dillon , Jason Evans , smp@freebsd.org Subject: Re: HEADS UP: Destabilization due to SMP development In-Reply-To: Message from Greg Lehey of "Thu, 22 Jun 2000 13:56:56 PDT." <20000622135656.D845@sydney.worldwide.lemis.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Thu, 22 Jun 2000 23:11:58 -0700 From: Jake Burkholder Message-Id: <20000623061158.1A4A7BA4E@io.yi.org> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org -current --> -smp > On Thursday, 22 June 2000 at 10:07:38 -0700, Matthew Dillon wrote: > > > >> On Mon, Jun 19, 2000 at 05:34:47PM -0700, Matthew Dillon wrote: > >>> > >>> Ok, I have put up a web page that will track my efforts. > >>> > >>> http://apollo.backplane.com/FreeBSDSmp/ > >> > >> Your first patchset contains only i386 code. > >> What is the timeframe for alpha relative to i386? > >> Will each i386 code step converted to it a short time later or finally > >> after the i386 code completely has been stabilized? > > > > The alpha code is going to be dealt with by the alpha guys. I am > > not an alpha assembly programmer. There is going to be considerably > > more breakage for the alpha port in the next month then the i386 port, > > but hopefully it will get worked out. > > Hmm. This adds another dependency. We will really need to get the > Alpha code in place before we can commit anything. Is there anybody > out there who can do this? I don't have an alpha, but I have access to beast and I know some alpha assembly. I have time in the next few days to work on making the 2nd patch set at least compile there if that would be useful. Do the mutex routines really need to be machine dependent? The i386 assembly is all in the inline function atomic_cmpexg_int, which can be implemented in alpha assembly[1]. mtx_lock could be changed to something guaranteed large enough to hold curproc. Would that work? There's a decent alpha instruction set reference here: ftp://ftp.compaq.com/pub/products/alphaCPUdocs/alpha_arch_hbk.pdf Jake [1] http://people.freebsd.org/~jake/cmpexg.diff I believe this is correct, only trivially tested. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Jun 23 1:46:31 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id ED0A137C0EB for ; Fri, 23 Jun 2000 01:46:23 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.8.7/8.8.7) with ESMTP id SAA15077; Fri, 23 Jun 2000 18:45:34 +1000 Date: Fri, 23 Jun 2000 18:45:32 +1000 (EST) From: Bruce Evans X-Sender: bde@besplex.bde.org To: Matthew Dillon Cc: Jason Evans , Greg Lehey , Warner Losh , The Hermit Hacker , freebsd-smp@FreeBSD.ORG Subject: Re: SP Patchset #1 up In-Reply-To: <200006221558.IAA03062@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Thu, 22 Jun 2000, Matthew Dillon wrote: > :You now have deadlock when the low level console i/o routines are > :reentered for ddb i/o, since task switching is either impossible or > :undesireable while ddb is active (it's essentially impossible if ddb > :was entered via a breakpoint in the middle of task switching code). > : > :When task switching is possible to avoid deadlock on console i/o > :resources, it may still be undesirable. A printf from a high priority > :interrupt task shouldn't have to wait for a printf from a low priority > :task to complete. > > I don't follow. Why do I have a deadlock? This is SchedMutex I'm > talking about here, not GiantMutex. mtx_enter_sched_quick() is mtx_enter(&SchedMutex, MTX_SPIN). | /* | * mtx_enter() | * ... | * WARNING: DO NOT PUT KERNEL PRINTFS IN THE SPIN CODE!, you may | * create a recursion. | */ This warning gives another reason why ddb can't call locking functions. Putting a breakpoint at a locking functions or wandering into a locking function while single stepping will cause console i/o. | static __inline void | mtx_enter(struct mtx *mtp, int type) | { | u_int32_t newv; | | newv = (u_int32_t)curproc | MTF_HELD; | if ((mtp->mtx_lock & ~MTF_CONTESTED) == newv) { Here is another reason why ddb can't call locking functions. curproc may be in the middle of being switched when a debugger trap occurs, so ddb can't call anything that depends on it being valid. SchedMutex may be invalid for the same reason. | ++mtp->mtx_recurse; /* bump recursion */ I'm not sure if this code is reached. If it is, then we don't have deadlock but we have the same bugs as for spltty() -- recursion is not prevented recursion although some console i/o routines don't support it. Many levels of recursion are possible: printf in process context --> interrupt ... printf in interrupt context --> higher priority interrupt ... printf in interrupt context ... even higher priority interrupts --> debugger trap ... debugger printf --> fatal trap for bug in printf ... bogus printfs (should longjmp) --> double fault for bug in fatal trap handler ... final printfs If switching from a high priority interrupt task to a low priority one is allowed, then the first few levels don't need to be supported. | } else { | if (atomic_cmpexg_int(&mtp->mtx_lock, 0, newv) != 0) { | if (type & MTX_SPIN) { | _mtx_enter_spin(mtp, newv); If this code is reached, then it should spin for ever, since other tasks should not run, even if there is another CPU for it to run on. | } else { | _mtx_enter_blocking(mtp, newv); | } | } | mtp->mtx_type = type; /* set the type after obtaining */ | } | } Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Jun 23 4:25:13 2000 Delivered-To: freebsd-smp@freebsd.org Received: from anchor-post-30.mail.demon.net (anchor-post-30.mail.demon.net [194.217.242.88]) by hub.freebsd.org (Postfix) with ESMTP id 346DA37B5E2 for ; Fri, 23 Jun 2000 04:25:10 -0700 (PDT) (envelope-from n_hibma@qubesoft.com) Received: from calcaphon.demon.co.uk ([193.237.19.5] helo=bluebottle.qubesoft.com) by anchor-post-30.mail.demon.net with esmtp (Exim 2.12 #1) id 135Ra8-0008l7-0U; Fri, 23 Jun 2000 12:25:09 +0100 Received: from henny.webweaving.org (henny.qubesoft.com [192.168.1.5]) by bluebottle.qubesoft.com (8.9.3/8.9.1) with ESMTP id MAA21485; Fri, 23 Jun 2000 12:26:05 +0100 (BST) (envelope-from n_hibma@qubesoft.com) Received: from localhost (localhost [127.0.0.1]) by henny.webweaving.org (8.9.3/8.9.3) with ESMTP id KAA01580; Fri, 23 Jun 2000 10:55:15 +0100 (BST) (envelope-from n_hibma@qubesoft.com) Date: Fri, 23 Jun 2000 10:55:15 +0100 (BST) From: Nick Hibma X-Sender: n_hibma@localhost Reply-To: Nick Hibma To: Greg Lehey Cc: Matthew Dillon , freebsd-smp@FreeBSD.ORG Subject: Re: irunning, width in bits. In-Reply-To: <20000622140107.E845@sydney.worldwide.lemis.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > I think, however, that Nick's suggestion is a thing we should follow > up on--*after* we have got the simple case working. Once we have the > ability to block in interrupt context, I'm sure we'll find lots of > applications for our new-found freedom, some of them good. But first > we need to understand the environment. For one we can tell everyone that interrupts can be shared at a very low cost (which means that people can stop moving around hardware to get rid of shared interrupts) Just bung in your PCI cards, make sure all the ISA stuff sits on its own and off you go. Nick -- n_hibma@webweaving.org n_hibma@freebsd.org USB project http://www.etla.net/~n_hibma/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Jun 23 8:56:45 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 2FCBD37BA73 for ; Fri, 23 Jun 2000 08:56:41 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id IAA10465; Fri, 23 Jun 2000 08:56:35 -0700 (PDT) (envelope-from dillon) Date: Fri, 23 Jun 2000 08:56:35 -0700 (PDT) From: Matthew Dillon Message-Id: <200006231556.IAA10465@apollo.backplane.com> To: Bruce Evans Cc: Jason Evans , Greg Lehey , Warner Losh , The Hermit Hacker , freebsd-smp@FreeBSD.ORG Subject: Re: SP Patchset #1 up References: Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org :> :> I don't follow. Why do I have a deadlock? This is SchedMutex I'm :> talking about here, not GiantMutex. : :mtx_enter_sched_quick() is mtx_enter(&SchedMutex, MTX_SPIN). : :| /* :| * mtx_enter() :| * ... :| * WARNING: DO NOT PUT KERNEL PRINTFS IN THE SPIN CODE!, you may :| * create a recursion. :| */ : :This warning gives another reason why ddb can't call locking functions. :Putting a breakpoint at a locking functions or wandering into a locking :function while single stepping will cause console i/o. A break point is a one-time deal. That'l work just fine. If the SchedMutex is already held then getting it again isn't going to hurt. If it isn't held then getting it the first time isn't going to hurt. The warning is there to avoid this situation: printf -> mtx_enter -> causes printf -> mtx_enter -> causes printf -> mtx_enter If we clear the break point prior to entering SchedMutex, then we don't have a recursion problem. :| static __inline void :| mtx_enter(struct mtx *mtp, int type) :| { :| u_int32_t newv; :| :| newv = (u_int32_t)curproc | MTF_HELD; :| if ((mtp->mtx_lock & ~MTF_CONTESTED) == newv) { : :Here is another reason why ddb can't call locking functions. curproc :may be in the middle of being switched when a debugger trap occurs, so :ddb can't call anything that depends on it being valid. SchedMutex may :be invalid for the same reason. SchedMutex is never invalid. No mutex is ever 'invalid'... they are locked in an atomic cmpexg. Either SchedMutex is held when the break point occurs, or it isn't. : :| ++mtp->mtx_recurse; /* bump recursion */ : :I'm not sure if this code is reached. If it is, then we don't have :deadlock but we have the same bugs as for spltty() -- recursion is not :prevented recursion although some console i/o routines don't support it. :Many levels of recursion are possible: : : printf in process context : --> interrupt ... printf in interrupt context : --> higher priority interrupt ... printf in interrupt context : ... even higher priority interrupts : --> debugger trap ... debugger printf : --> fatal trap for bug in printf ... bogus printfs (should longjmp) : --> double fault for bug in fatal trap handler ... final printfs I don't see the issue here. interrupts cannot nest SchedMutex - look at the interrupt code. At the moment interrupt can nest other interrupts only insofar as the new interrupt occuring before the old interrupt has obtained the giant mutex. :If switching from a high priority interrupt task to a low priority one is :allowed, then the first few levels don't need to be supported. I don't follow. The SchedMutex recursion count is saved and restored when a switch occurs. : :| } else { :| if (atomic_cmpexg_int(&mtp->mtx_lock, 0, newv) != 0) { :| if (type & MTX_SPIN) { :| _mtx_enter_spin(mtp, newv); : :If this code is reached, then it should spin for ever, since other tasks :should not run, even if there is another CPU for it to run on. If you place a debugger trap at a point where SchedMutex is held, guess which cpu the debugger takes the trap on? The one holding SchedMutex. Thus no problem... the debug trap will simply bump the recursion count of SchedMutex temporarily. :| } else { :| _mtx_enter_blocking(mtp, newv); :| } :| } :| mtp->mtx_type = type; /* set the type after obtaining */ :| } :| } : :Bruce The only thing we have to worry about insofar as debugger traps go is to be sure to clear the debug point prior to entering the SchedMutex within the debugger trap. I think this is trivial. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Jun 23 10:15: 2 2000 Delivered-To: freebsd-smp@freebsd.org Received: from gidora.zeta.org.au (gidora.zeta.org.au [203.26.10.25]) by hub.freebsd.org (Postfix) with SMTP id 24F9137C3E3 for ; Fri, 23 Jun 2000 10:14:51 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: (qmail 6652 invoked from network); 23 Jun 2000 17:14:43 -0000 Received: from unknown (HELO bde.zeta.org.au) (203.2.228.102) by gidora.zeta.org.au with SMTP; 23 Jun 2000 17:14:43 -0000 Date: Sat, 24 Jun 2000 03:14:40 +1000 (EST) From: Bruce Evans X-Sender: bde@besplex.bde.org To: Matthew Dillon Cc: Jason Evans , Greg Lehey , Warner Losh , The Hermit Hacker , freebsd-smp@FreeBSD.ORG Subject: Re: SP Patchset #1 up In-Reply-To: <200006231556.IAA10465@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Fri, 23 Jun 2000, Matthew Dillon wrote: > :> I don't follow. Why do I have a deadlock? This is SchedMutex I'm > :> talking about here, not GiantMutex. > : > :mtx_enter_sched_quick() is mtx_enter(&SchedMutex, MTX_SPIN). > : > :| /* > :| * mtx_enter() > :| * ... > :| * WARNING: DO NOT PUT KERNEL PRINTFS IN THE SPIN CODE!, you may > :| * create a recursion. > :| */ > : > :This warning gives another reason why ddb can't call locking functions. > :Putting a breakpoint at a locking functions or wandering into a locking > :function while single stepping will cause console i/o. > > A break point is a one-time deal. That'l work just fine. If the Except for a breakpoint in the debugger's breakpoint handling code or in the debugger's i/o functions. Setting a breakpoint in the debugger's breakpoint handling code is user error and very difficult to render harmless. Setting a breakpoint in debugger's i/o functions is not necessarily a user error, since parts of the functions are used by the kernel generally. E.g., setting a breakpoint at cnputc() currently causes a recursive panic in some cases, because trap_fatal() calls printf() before ddb can remove the breakpoint and kdb_trap() calls db_printf() before ddb removes the breakpoint. > SchedMutex is already held then getting it again isn't going to hurt. > If it isn't held then getting it the first time isn't going to hurt. In that case, it doesn't protect against unsupported recursion in the syscons console i/o routines any better than spltty(). > :| static __inline void > :| mtx_enter(struct mtx *mtp, int type) > :| { > :| u_int32_t newv; > :| > :| newv = (u_int32_t)curproc | MTF_HELD; > :| if ((mtp->mtx_lock & ~MTF_CONTESTED) == newv) { > : > :Here is another reason why ddb can't call locking functions. curproc > :may be in the middle of being switched when a debugger trap occurs, so > :ddb can't call anything that depends on it being valid. SchedMutex may > :be invalid for the same reason. > > SchedMutex is never invalid. No mutex is ever 'invalid'... they are > locked in an atomic cmpexg. Either SchedMutex is held when the break > point occurs, or it isn't. From your patches for exception.s: | + /* | + * We have to protect the SchedMutex and curproc fixup from | + * interrupts on this cpu (which check for SchedMutex being | + * held on the 'current' cpu). | + */ | + cli | + | #ifdef SMP | #ifdef GRAB_LOPRIO /* hold LOPRIO for INTs */ | #ifdef CHEAP_TPR | @@ -455,12 +297,22 @@ | movl %edx, _curpcb | movl %ecx, _curproc /* into next process */ | | -#ifdef SMP | - movl _cpu_lockid, %eax | - orl PCB_MPNEST(%edx), %eax /* add next count from PROC */ | - movl %eax, _mp_lock /* load the mp_lock */ | + /* | + * Restore SchedMutex, be careful not to loose MTF_CONTESTED. | + * Note that MTF_HELD *MUST* be set here (XXX add assertion/panic if | + * not). | + * | + * The curproc portion of SchedMutex will temporarily cycle through | + * NULL, but since interrupts are disabled and curproc can never | + * be NULL, this will not result in improper operation. | + */ | + movl PCB_SCHEDNEST(%edx), %eax /* add next count from PROC */ | + MPLOCKED | + andl $(MTF_HELD|MTF_CONTESTED),_SchedMutex+MTX_LOCK | + MPLOCKED | + orl %ecx, _SchedMutex+MTX_LOCK | + movl %eax, _SchedMutex+MTX_RECURSE /* load the sched mutex */ | /* XXX FIXME: we should be restoring the local APIC TPR */ | -#endif /* SMP */ "cli" and other forms of disabling interrupts provide no protection against exceptions which aren't interrupts, in particular against debugger traps. > :Many levels of recursion are possible: > : > : printf in process context > : --> interrupt ... printf in interrupt context > : --> higher priority interrupt ... printf in interrupt context > : ... even higher priority interrupts > : --> debugger trap ... debugger printf > : --> fatal trap for bug in printf ... bogus printfs (should longjmp) > : --> double fault for bug in fatal trap handler ... final printfs > > I don't see the issue here. interrupts cannot nest SchedMutex - look > at the interrupt code. At the moment interrupt can nest other interrupts > only insofar as the new interrupt occuring before the old interrupt has > obtained the giant mutex. Traps are not interrupts. Debugger traps are the main problem. NMIs would be a bigger problem if they occurred in normal operation (note that our NMI handling is broken; it does dangerous things like calling printf). > :If switching from a high priority interrupt task to a low priority one is > :allowed, then the first few levels don't need to be supported. > > I don't follow. The SchedMutex recursion count is saved and restored > when a switch occurs. This will only matter when (if) prioritized interrupts are supported again. > :| } else { > :| if (atomic_cmpexg_int(&mtp->mtx_lock, 0, newv) != 0) { > :| if (type & MTX_SPIN) { > :| _mtx_enter_spin(mtp, newv); > : > :If this code is reached, then it should spin for ever, since other tasks > :should not run, even if there is another CPU for it to run on. > > If you place a debugger trap at a point where SchedMutex is held, > guess which cpu the debugger takes the trap on? The one holding > SchedMutex. Thus no problem... the debug trap will simply bump the > recursion count of SchedMutex temporarily. Are you saying that this code is never reached for debugger traps? I'm not even thinking about complications for SMP. The problems I'm talking about affect for UP. The SMP case shouldn't be significantly different for debugger traps, since the first thing the debugger trap handler should do is stop the other CPUs. > The only thing we have to worry about insofar as debugger traps go > is to be sure to clear the debug point prior to entering the SchedMutex > within the debugger trap. I think this is trivial. This is the problem that I mentioned above. It is not completely trivial to fix, because the debugger is entered for all types of fatal traps. Fatal traps aren't necessarily fatal for people who can use ddb. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Jun 23 10:43:37 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 6F9F937C3CE for ; Fri, 23 Jun 2000 10:43:07 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id KAA11163; Fri, 23 Jun 2000 10:43:01 -0700 (PDT) (envelope-from dillon) Date: Fri, 23 Jun 2000 10:43:01 -0700 (PDT) From: Matthew Dillon Message-Id: <200006231743.KAA11163@apollo.backplane.com> To: Bruce Evans Cc: Jason Evans , Greg Lehey , Warner Losh , The Hermit Hacker , freebsd-smp@FreeBSD.ORG Subject: Re: SP Patchset #1 up References: Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org :> SchedMutex is already held then getting it again isn't going to hurt. :> If it isn't held then getting it the first time isn't going to hurt. : :In that case, it doesn't protect against unsupported recursion in the syscons :console i/o routines any better than spltty(). It protects against: * an interrupt occuring during the critical section on the same cpu * entry into those routines from another cpu Which is as good as you can get, really, since the only other option is to lockup. If you get a trap in the middle of the debugging section, with SchedMutex held, I'd wager you would rather see another debug prompt then see a complete lockup. -Matt :> SchedMutex is never invalid. No mutex is ever 'invalid'... they are :> locked in an atomic cmpexg. Either SchedMutex is held when the break :> point occurs, or it isn't. : :>From your patches for exception.s: : :| + /* :| + * We have to protect the SchedMutex and curproc fixup from :| + * interrupts on this cpu (which check for SchedMutex being :| + * held on the 'current' cpu). :| + */ :| + cli :| + You mean i386/i386/swtch.s The issue here is that if an interrupt occurs on the cpu holding SchedMutex, the interrupt must be defered. When the switch code switches to a new process it must change curproc, which breaks the detection in the interrupt code that determines that the current cpu is holding SchedMutex. Thus interrupts (on the current cpu only) must be disabled to allow the switch code to update curproc AND SchedMutex. The SchedMutex is held throughout this period, but there is a small window where it is held by the 'wrong' process (curproc has been changed, but SchedMutex's mtx_lock has not), which I am currently using cli to protect. If a debugger trap were to occur on the current cpu at just point, it would indeed lead to a lockup. You are absolutely correct. So maybe the comment should read: don't create any break points in between the setting of curproc and the fixup of SchedMutex! We can solve this without the CLI/STI by releasing the SchedMutex entirely, changing curproc, and then regaining SchedMutex. I personally do not like that idea but it would be more 'correct', and it would not have the lockup problem if a DDB trap or NMI occured on the current cpu at just that point. :"cli" and other forms of disabling interrupts provide no protection against :exceptions which aren't interrupts, in particular against debugger traps. Very true. On the otherhand, I don't think it's possible for anything but a set break point to trap the code at that point so I don't consider it a big problem. Oh, ok.. NMI could do it. :> I don't see the issue here. interrupts cannot nest SchedMutex - look :> at the interrupt code. At the moment interrupt can nest other interrupts :> only insofar as the new interrupt occuring before the old interrupt has :> obtained the giant mutex. : :Traps are not interrupts. Debugger traps are the main problem. NMIs :would be a bigger problem if they occurred in normal operation (note :that our NMI handling is broken; it does dangerous things like calling :printf). We could support NMIs by releasing the SchedMutex in that critical swtch.s section and then regaining it after adjusting curproc. The NMI would be able to support the kernel printf() in that case. It is possible to release SchedMutex simply with a non-locked andl $MTF_CONTESTED,_SchedMutex+MTX_LOCK (because MTF_CONTESTED cannot be set by the 'other' cpu for SchedMutex), but regaining it would require a call to mtx_enter_sched_quick. On the otherhand, we could get rid of the cli/sti pair and that might make up enough time to be close to break-even on the performance. :> :If switching from a high priority interrupt task to a low priority one is :> :allowed, then the first few levels don't need to be supported. :> :> I don't follow. The SchedMutex recursion count is saved and restored :> when a switch occurs. : :This will only matter when (if) prioritized interrupts are supported again. Prioritized interrupts will follow naturally when Greg puts in the heavy-weight interrupt threads. The interrupt threads will be prioritized just like normal processes, and the highest priority thread is the one that is going to get to run first (even interrupting and being switched into from a lower priority interrupt thread). :> :> If you place a debugger trap at a point where SchedMutex is held, :> guess which cpu the debugger takes the trap on? The one holding :> SchedMutex. Thus no problem... the debug trap will simply bump the :> recursion count of SchedMutex temporarily. : :Are you saying that this code is never reached for debugger traps? : :I'm not even thinking about complications for SMP. The problems I'm :talking about affect for UP. The SMP case shouldn't be significantly :different for debugger traps, since the first thing the debugger trap :handler should do is stop the other CPUs. I think if we fix the one case you pointed out in swtch.s, that we could take a debugger trap anywhere except in the debugger trap dispatch code itself. SP or MP. :> The only thing we have to worry about insofar as debugger traps go :> is to be sure to clear the debug point prior to entering the SchedMutex :> within the debugger trap. I think this is trivial. : :This is the problem that I mentioned above. It is not completely trivial :to fix, because the debugger is entered for all types of fatal traps. :Fatal traps aren't necessarily fatal for people who can use ddb. : :Bruce I meant 'clear the break point'. Assuming a set break point. (break point trap) (make sure no printf's in the code path) debugger entry (make sure no printf's in the code path) clear break point ... now printf's are safe. ... now scheduler mutex calls are safe I have not audited the debugger breakpoint code. It's probably 'wrong', but it wouldn't take much to fix it if it is. There are a few recursive situations that must be carefully protected for debugger access. The spl*() code in isa/ipl_funcs.c for example has code to issue certain panics one-time in order to prevent panic recursion when the system drops into the debugger. I will point out that there are several cases *already* (without my patch) where an attempt to enter the debugger leads to a recursive panic. We aren't going to be making things any worse then they already are. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Jun 23 11:37:10 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 41AE937C412 for ; Fri, 23 Jun 2000 11:37:06 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id LAA11456; Fri, 23 Jun 2000 11:37:03 -0700 (PDT) (envelope-from dillon) Date: Fri, 23 Jun 2000 11:37:03 -0700 (PDT) From: Matthew Dillon Message-Id: <200006231837.LAA11456@apollo.backplane.com> To: freebsd-smp@freebsd.org Subject: Stepping on Toes Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hmm. Well, apparently I've stepped on a few toes by not using the BSDI mutex code verbatim. My position is this: * I don't think we can use the BSDI mutex code verbatim because we intend to keep legacy SPL support and it would take as much work addressing that in the BSDI mutex code as it would doing a new core-mutex piece from scratch. * I think we CAN use the *WITNESS* code pretty much verbatim. The reason it isn't in patch set #1 is because the witness code is not going to be useful until we start unwinding the SPL code in the system, and that is not going to happen until *after* heavy weight interrupt threads are implemented. Until that time (maybe a month or two from now?) it's just extra baggage. It's the next logical step to take after the heavy weight interrupt code is made to work, not before. The witness code isn't going to help us for just the Sched and Giant mutexes... simple KASSERT's deal with those situations. In regards to the general project. If people want to try to port the BSDI mutex code directly then be my guest. I've got a patch set that works now ... I can successfully make buildworld on an SP machine with it. You can try porting the BSDI mutex code with or without the patchset I've already got, but if after a month or two if you still don't have something that works I would recommend going back to the piecemeal approach I have taken, starting with my patchset. I won't be angry or anything if people decide to take this course, even though I think it's the wrong course to take. But you guys have to decide what to do ASAP. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Jun 23 13: 7:40 2000 Delivered-To: freebsd-smp@freebsd.org Received: from gidora.zeta.org.au (gidora.zeta.org.au [203.26.10.25]) by hub.freebsd.org (Postfix) with SMTP id AA53F37B639 for ; Fri, 23 Jun 2000 13:07:33 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: (qmail 14603 invoked from network); 23 Jun 2000 20:07:29 -0000 Received: from unknown (HELO bde.zeta.org.au) (203.2.228.102) by gidora.zeta.org.au with SMTP; 23 Jun 2000 20:07:29 -0000 Date: Sat, 24 Jun 2000 06:07:26 +1000 (EST) From: Bruce Evans X-Sender: bde@besplex.bde.org To: Matthew Dillon Cc: Jason Evans , Greg Lehey , Warner Losh , The Hermit Hacker , freebsd-smp@FreeBSD.ORG Subject: Re: SP Patchset #1 up In-Reply-To: <200006231743.KAA11163@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Fri, 23 Jun 2000, Matthew Dillon wrote: > :> SchedMutex is never invalid. No mutex is ever 'invalid'... they are > :> ... > :>From your patches for exception.s: > :>... > You mean i386/i386/swtch.s > ... > If a debugger trap were to occur on the current cpu at just point, it > would indeed lead to a lockup. You are absolutely correct. So maybe > the comment should read: don't create any break points in between the > setting of curproc and the fixup of SchedMutex! Also: don't trace into the context switching code. This is very inconvenient. I've often typed 'n' in the debugger and then #$&* when the system crashes because ddb traced into nonreentrant code. (This happened yesterday when I tried 'n' to get a count of the number of instructions taken by a vmware ioctl.) > We can solve this without the CLI/STI by releasing the SchedMutex > entirely, changing curproc, and then regaining SchedMutex. I personally > do not like that idea but it would be more 'correct', and it would not > have the lockup problem if a DDB trap or NMI occured on the current cpu > at just that point. What should happen is for the low level console driver to be reentrant, like the sio driver attempts to be. Then locking would only be cosmetic. It might prevent garbling of single characters, but since it is per-char it won't prevent interleaving of output. I'm not sure what exactly what reentrancy means for the SMP case. For debugger i/o, it is the same as for the UP case since all cpus except the one running the debugger must be stopped or spinning. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Jun 23 14: 7:57 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 7F03037B777 for ; Fri, 23 Jun 2000 14:07:53 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id OAA11962; Fri, 23 Jun 2000 14:07:47 -0700 (PDT) (envelope-from dillon) Date: Fri, 23 Jun 2000 14:07:47 -0700 (PDT) From: Matthew Dillon Message-Id: <200006232107.OAA11962@apollo.backplane.com> To: Bruce Evans Cc: Jason Evans , Greg Lehey , Warner Losh , The Hermit Hacker , freebsd-smp@FreeBSD.ORG Subject: Re: SP Patchset #1 up References: Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org :> ... :> If a debugger trap were to occur on the current cpu at just point, it :> would indeed lead to a lockup. You are absolutely correct. So maybe :> the comment should read: don't create any break points in between the :> setting of curproc and the fixup of SchedMutex! : :Also: don't trace into the context switching code. This is very inconvenient. :I've often typed 'n' in the debugger and then #$&* when the system crashes :because ddb traced into nonreentrant code. (This happened yesterday when :I tried 'n' to get a count of the number of instructions taken by a vmware :ioctl.) We can make tracing through the context switch code work. Hmm. We have two choices: either make the context switch code trace-safe (by actually unlocking SchedMutex when changing curproc), or make the DDB trap save/override and restore the SchedMutex state, which is what it did before. Sorta. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Fri Jun 23 22:51:32 2000 Delivered-To: freebsd-smp@freebsd.org Received: from web1904.mail.yahoo.com (web1904.mail.yahoo.com [128.11.23.53]) by hub.freebsd.org (Postfix) with SMTP id D24C037BB37 for ; Fri, 23 Jun 2000 22:51:28 -0700 (PDT) (envelope-from p_k_nandan@yahoo.com) Received: (qmail 23941 invoked by uid 60001); 24 Jun 2000 05:51:28 -0000 Message-ID: <20000624055128.23940.qmail@web1904.mail.yahoo.com> Received: from [203.197.180.197] by web1904.mail.yahoo.com; Fri, 23 Jun 2000 22:51:28 PDT Date: Fri, 23 Jun 2000 22:51:28 -0700 (PDT) From: "NandaKumar P.K." Subject: SMP kernel problem in compaq proliant 6000 To: freebsd-smp@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi, I am trying to configure a SMP kernel (FreeBSD 3.4) in a Compaq proliant 6000 model machine with 2 Pentium II Xeon Processor.The kernel crashes while booting saying APIC RTC !=8. The options which are enabled in the configuration file as follows. options SMP options APIC_IO options NCPU=2 options NBUS=4 options NAPIC=1 options NINTR=61 mptable gives the same values as above. I tried with NAPIC=8,but it doesn't work. I am not able to get any motherboard model no. Any help will be deeply appreciated Regards, Nandan __________________________________________________ Do You Yahoo!? Get Yahoo! Mail - Free email you can access from anywhere! http://mail.yahoo.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Jun 24 5:48:48 2000 Delivered-To: freebsd-smp@freebsd.org Received: from paperboy.sixforty.co.uk (paperboy.sixforty.co.uk [195.10.242.11]) by hub.freebsd.org (Postfix) with ESMTP id 58B1A37B6B1 for ; Sat, 24 Jun 2000 05:48:30 -0700 (PDT) (envelope-from lawrence@epcdirect.co.uk) Received: from mail.shorewood-epc.co.uk (mail.shorewood-epc.co.uk [195.10.240.17]) by paperboy.sixforty.co.uk (8.9.3/8.9.3) with ESMTP id NAA31287 for ; Sat, 24 Jun 2000 13:48:07 +0100 (BST) (envelope-from lawrence@epcdirect.co.uk) Received: from lfarr (host-191.shorewood-epc.co.uk [192.168.15.191]) by mail.shorewood-epc.co.uk (8.9.3/8.9.3) with SMTP id NAA49280 for ; Sat, 24 Jun 2000 13:48:01 +0100 (BST) (envelope-from lawrence@epcdirect.co.uk) Reply-To: From: "Lawrence Farr" To: Subject: Revisited: HP Vectra XU Date: Sat, 24 Jun 2000 13:47:52 +0100 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2919.6600 Importance: Normal Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I posted this originally on the 7th June, and since then have had a bit of a play around with it. Upgrading to 4-STABLE a few days after this did not work, same error, but booting the same machine on the 4-CURRENT SMP kernel, it worked fine. Going back to 4-RELEASE again, I realised the machine was on Plug and Play. Turned it off, built a SMP kernel, and it was fine. Took it up to stable again, made world, made kernel, same result as below. I am assuming there is no hardware problem as it works OK on older code. Does anyone have any suggestions/patches I can try? -----Original Message----- From: Lawrence Farr [mailto:l.farr@epcdirect.co.uk] Sent: 07 June 2000 18:58 To: freebsd-smp@freebsd.org Subject: HP Vectra XU I have a HP Vectra XU, Dual PPro 200, 256Mb RAM, Internal Adaptec, that ran quite happily on a 4.0-CURRENT from mid January if I remember rightly. Upgraded to STABLE and it died on boot. Boots single processor OK, dual it wont. Blanked it, and put 4.0-RELEASE on there, same again. Single OK, Dual I get: (snip) avail memory = 257159168 (251132K bytes) Programming 16 pins in IOAPIC #0 IOAPIC #0 intpin 2 -> irq 0 Then stops. Any ideas anyone? Lawrence Farr EPC Direct Limited mailto:l.farr@epcdirect.co.uk T:01179666123 F:01179666111 M:07970780901 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Jun 24 8:39:44 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mta5.snfc21.pbi.net (mta5.snfc21.pbi.net [206.13.28.241]) by hub.freebsd.org (Postfix) with ESMTP id 74D7837BC35 for ; Sat, 24 Jun 2000 08:39:35 -0700 (PDT) (envelope-from adagio_v@pacbell.net) Received: from pacbell.net ([63.194.212.226]) by mta5.snfc21.pbi.net (Sun Internet Mail Server sims.3.5.2000.01.05.12.18.p9) with ESMTP id <0FWO00ICW0S8BE@mta5.snfc21.pbi.net> for freebsd-smp@freebsd.org; Sat, 24 Jun 2000 08:38:33 -0700 (PDT) Date: Sat, 24 Jun 2000 08:34:33 -0700 From: Adagio Vangogh Subject: Re: SP Patchset #2 up (one minor bug fix) To: freebsd-smp@freebsd.org Cc: adagio_v@pacbell.net Reply-To: adagio_v@pacbell.net Message-id: <3954D509.748BD536@pacbell.net> MIME-version: 1.0 X-Mailer: Mozilla 4.73 [en] (Windows NT 5.0; I) Content-type: text/plain; charset=us-ascii Content-transfer-encoding: 7bit X-Accept-Language: en Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi Matt, can you also post the modified files in full for those -current challenged. /av To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Jun 24 11:26:30 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 9907B37B595 for ; Sat, 24 Jun 2000 11:26:28 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id LAA17019; Sat, 24 Jun 2000 11:26:20 -0700 (PDT) (envelope-from dillon) Date: Sat, 24 Jun 2000 11:26:20 -0700 (PDT) From: Matthew Dillon Message-Id: <200006241826.LAA17019@apollo.backplane.com> To: Adagio Vangogh Cc: freebsd-smp@FreeBSD.ORG Subject: Re: SP Patchset #2 up (one minor bug fix) References: <3954D509.748BD536@pacbell.net> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org :Hi Matt, : can you also post the modified files in full for those -current : challenged. : : /av I'll do at least one, but I'm waiting to see what the methodology is going to be. I can't spend the next year managing patchsets... I'd prefer to see the stuff committed. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Jun 24 12:16:55 2000 Delivered-To: freebsd-smp@freebsd.org Received: from timingpdc.timing.com (timingpdc.timing.com [206.168.13.194]) by hub.freebsd.org (Postfix) with ESMTP id 3AB2B37B640; Sat, 24 Jun 2000 12:16:44 -0700 (PDT) (envelope-from smp@timing.com) Received: from RoadRunner.timing.com ([206.168.13.190]) by timingpdc.timing.com (Post.Office MTA v3.1.2 release (PO205-101c) ID# 103-49575U100L2S100) with ESMTP id AAA990; Sat, 24 Jun 2000 13:18:37 -0600 Received: from RoadRunner.timing.com (localhost [127.0.0.1]) by RoadRunner.timing.com (8.9.3/8.9.3) with ESMTP id NAA58694; Sat, 24 Jun 2000 13:15:51 -0600 (MDT) (envelope-from smp@RoadRunner.timing.com) Message-Id: <200006241915.NAA58694@RoadRunner.timing.com> X-Mailer: exmh version 2.0.2 2/24/98 From: Steve Passe To: msmith@FreeBSD.org Cc: smp@FreeBSD.org Subject: breakage to intr_machdep.c Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sat, 24 Jun 2000 13:15:51 -0600 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi, Modified files: sys/i386/isa clock.c intr_machdep.c isa_dma.c pcibus.c spkr.c Revision Changes Path 1.32 +68 -2 src/sys/i386/isa/intr_machdep.c this breaks MP compiles: cc -c -O -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -fformat-extensions -ansi -nostdinc -I- -I. -I../.. -I../../../include -D_KERNEL -include opt_global.h -elf -mpreferred-stack-boundary=2 ../../i386/isa/intr_machdep.c ../../i386/isa/intr_machdep.c: In function `atpic_attach': ../../i386/isa/intr_machdep.c:191: syntax error before `res' ../../i386/isa/intr_machdep.c:195: `res' undeclared (first use in this function) ../../i386/isa/intr_machdep.c:195: (Each undeclared identifier is reported only once ../../i386/isa/intr_machdep.c:195: for each function it appears in.) ../../i386/isa/intr_machdep.c:195: `SYS_RES_IRQ' undeclared (first use in this function) *** Error code 1 Stop in /usr/src/sys/compile/LOCAL_MP. --- The fix is: *** intr_machdep.c.orig Sat Jun 24 12:49:04 2000 --- intr_machdep.c Sat Jun 24 12:56:31 2000 *************** *** 66,71 **** --- 66,72 ---- #if defined(APIC_IO) #include #include /** FAST_HI */ + #include #endif /* APIC_IO */ #ifdef PC98 #include *************** *** 188,194 **** { #ifdef APIC_IO int rid; ! bus_resource_t res; /* try to allocate our IRQ and then free it */ rid = 0; --- 189,195 ---- { #ifdef APIC_IO int rid; ! struct resource *res; /* try to allocate our IRQ and then free it */ rid = 0; -- Steve Passe | powered by smp@timing.com | Symmetric MultiProcessor FreeBSD To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Jun 24 13: 6:34 2000 Delivered-To: freebsd-smp@freebsd.org Received: from feral.com (feral.com [192.67.166.1]) by hub.freebsd.org (Postfix) with ESMTP id AE35A37B840; Sat, 24 Jun 2000 13:06:08 -0700 (PDT) (envelope-from mjacob@feral.com) Received: from semuta.feral.com (semuta [192.67.166.70]) by feral.com (8.9.3/8.9.3) with ESMTP id NAA29540; Sat, 24 Jun 2000 13:05:59 -0700 Date: Sat, 24 Jun 2000 13:05:19 -0700 (PDT) From: Matthew Jacob Reply-To: mjacob@feral.com To: freebsd-smp@freebsd.org, alpha@freebsd.org Cc: Jake Burkholder Subject: Re: HEADS UP: Destabilization due to SMP development (fwd) Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Just to clarify for all and sundry. ---------- Forwarded message ---------- Date: Sat, 24 Jun 2000 20:56:22 +0100 (BST) From: Doug Rabson To: Matthew Jacob Cc: Andrew Gallatin Subject: Re: HEADS UP: Destabilization due to SMP development (fwd) On Fri, 23 Jun 2000, Matthew Jacob wrote: > > > Seriously- if neither of you are signed up for this, I'll take a run at it. I'm sorry about any confusion that Matt Dillon has caused. I told him in person that he shouldn't worry about the mutex implementation since I already had some working mutex code which could be reused to implement the BSD/OS 5.0 style mutex. This somehow mutated into "I'll break the alpha port until further notice". I will be working on the alpha SMP code, both old-style and new-style. -- Doug Rabson Mail: dfr@nlsystems.com Nonlinear Systems Ltd. Phone: +44 20 8442 9037 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Jun 24 13:15:11 2000 Delivered-To: freebsd-smp@freebsd.org Received: from gti.net (apollo.gti.net [199.171.27.7]) by hub.freebsd.org (Postfix) with ESMTP id 2751F37B705 for ; Sat, 24 Jun 2000 13:15:07 -0700 (PDT) (envelope-from shadow@apollo.gti.net) Received: from localhost (shadow@localhost) by gti.net (8.9.3/8.8.8) with ESMTP id QAA02158 for ; Sat, 24 Jun 2000 16:15:06 -0400 (EDT) Date: Sat, 24 Jun 2000 16:15:06 -0400 (EDT) From: Shadow To: freebsd-smp@FreeBSD.ORG Subject: Re: SMP kernel problem in compaq proliant 6000 In-Reply-To: <20000624055128.23940.qmail@web1904.mail.yahoo.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I have a similar problem on a 6500 with 4 Xeons. Everything runs fine with a single processor kernel, but mounting / hangs with SMP enabled. It's also running the ida raid driver. I will give more details on monday if anyone is intrested, not anywhere near the computer right now. -Nick Sr. Systems Administrator, Global Telecom Inc. shadow@gti.net On Fri, 23 Jun 2000, NandaKumar P.K. wrote: > Hi, > > I am trying to configure a SMP kernel (FreeBSD 3.4) in > a Compaq proliant 6000 model machine with 2 Pentium II > Xeon Processor.The kernel crashes while booting saying > APIC RTC !=8. The options which are enabled in the > configuration file as follows. > > options SMP > options APIC_IO > > options NCPU=2 > options NBUS=4 > options NAPIC=1 > options NINTR=61 > > mptable gives the same values as above. > I tried with NAPIC=8,but it doesn't work. I am not > able to get any motherboard model no. > > Any help will be deeply appreciated > > Regards, > Nandan > > > __________________________________________________ > Do You Yahoo!? > Get Yahoo! Mail - Free email you can access from anywhere! > http://mail.yahoo.com/ > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Jun 24 21:21:36 2000 Delivered-To: freebsd-smp@freebsd.org Received: from implode.root.com (root.com [209.102.106.178]) by hub.freebsd.org (Postfix) with ESMTP id 8818E37B76C for ; Sat, 24 Jun 2000 21:21:29 -0700 (PDT) (envelope-from dg@implode.root.com) Received: from implode.root.com (localhost [127.0.0.1]) by implode.root.com (8.8.8/8.8.5) with ESMTP id VAA08876; Sat, 24 Jun 2000 21:11:18 -0700 (PDT) Message-Id: <200006250411.VAA08876@implode.root.com> To: Matthew Dillon Cc: freebsd-smp@FreeBSD.ORG Subject: Re: Stepping on Toes In-reply-to: Your message of "Fri, 23 Jun 2000 11:37:03 PDT." <200006231837.LAA11456@apollo.backplane.com> From: David Greenman Reply-To: dg@root.com Date: Sat, 24 Jun 2000 21:11:18 -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Hmm. Well, apparently I've stepped on a few toes by not using the > BSDI mutex code verbatim. > > My position is this: > > * I don't think we can use the BSDI mutex code verbatim > because we intend to keep legacy SPL support and it would take > as much work addressing that in the BSDI mutex code as it would > doing a new core-mutex piece from scratch. > > * I think we CAN use the *WITNESS* code pretty much verbatim. The > reason it isn't in patch set #1 is because the witness code is > not going to be useful until we start unwinding the SPL code in > the system, and that is not going to happen until *after* heavy > weight interrupt threads are implemented. Until that time > (maybe a month or two from now?) it's just extra baggage. > It's the next logical step to take after the heavy weight > interrupt code is made to work, not before. > > The witness code isn't going to help us for just the Sched and > Giant mutexes... simple KASSERT's deal with those situations. > > In regards to the general project. If people want to try to port > the BSDI mutex code directly then be my guest. I've got a patch set > that works now ... I can successfully make buildworld on an SP machine > with it. You can try porting the BSDI mutex code with or without the > patchset I've already got, but if after a month or two if you still don't > have something that works I would recommend going back to the piecemeal > approach I have taken, starting with my patchset. > > I won't be angry or anything if people decide to take this course, > even though I think it's the wrong course to take. But you guys have > to decide what to do ASAP. Of course we need to wait a few days for people to return from Usenix. I just got back myself a few hours ago and have over 3000 emails to deal with. My general view is that we should probably go directly to the threaded interrupt model and dump spls in the first cut. Given this, I think it would be prudent to use as much of the BSDI mutex code as possible, at least for the medium term, since it has been well tested and debugged. Not using it reaks of NIH to folks on the outside, even though the SMPng group (at least) knows that that wasn't at all your motivation in rolling your own. -DG David Greenman Co-founder, The FreeBSD Project - http://www.freebsd.org Manufacturer of high-performance Internet servers - http://www.terasolutions.com Pave the road of life with opportunities. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sat Jun 24 23:56:38 2000 Delivered-To: freebsd-smp@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 4484A37B5D2 for ; Sat, 24 Jun 2000 23:56:31 -0700 (PDT) (envelope-from jasone@magnesium.net) Received: (qmail 15321 invoked by uid 1142); 25 Jun 2000 06:56:30 -0000 Date: 24 Jun 2000 23:56:30 -0700 Date: Sat, 24 Jun 2000 23:56:05 -0700 From: Jason Evans To: smp@freebsd.org Subject: SMP meeting summary Message-ID: <20000624235605.D8965@blitz.canonware.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org [The following meeting summary was originally written by Greg Lehey, and he later revised it to include various points from the notes that I took during the meeting. Finally, I edited (added some, changed some, removed some) Greg's summary. Thanks go to Greg for doing the majority of the writing work!] On the 15th and 16th of June we had a seminar at Yahoo! in Sunnyvale about the recent changes to the BSD/OS kernel designed to improve SMP performance. Participants were, in seating order: Don Brady Apple Computer File systems Ramesh ? Apple Computer Ted Walker Apple Computer network drivers Jeffrey Hsu FreeBSD project Chuck Paterson BSDi Chief developer Jonathan Lemon Cisco, FreeBSD project Matt Dillon FreeBSD project VM, NFS Paul Saab Yahoo! Kirk McKusick Peter Wemm Yahoo! Jayanth ? Yahoo! Doug Rabson FreeBSD project Alpha port Jason Evans FreeBSD project kernel threads David Greenman FreeBSD project chief architect Justin Gibbs Adaptec, FreeBSD project SCSI, 0 copy TCP Greg Lehey Linuxcare, FreeBSD project storage management Mike Smith BSDi, FreeBSD project hardware, iA64 port Alfred Perlstein Wintel, FreeBSD project David O'Brien BSDi, FreeBSD project compilers, binutils Ceren Ercen Linuxcare Daemon babe We met for approximately 8 hours on Thursday and 4 hours on Friday. Chuck Patterson spent Thursday presenting how BSDi implemented SMP in BSD/OS 5.0 (as of yet unreleased). Chuck also briefly explained BSD/OS 4.x's SMP implementation. The BSD/OS 4.x SMP implementation is mainly comprised of a giant lock, but with a twist. Whenever a processor tries to acquire the giant lock it can either succeed or fail. If it succeeds, then it's business as usual. However, if the acquisition fails, the processor does not spin on the giant lock. Instead, it acquires the schedlock (which protects scheduler-related portions of the kernel) and schedules another runnable process, if any. This technique works extremely well for heavy work loads that have less than one CPU worth of system (kernel processing) load. It is very simple, and it achieves optimal throughput. The BSD/OS 5.0 SMP implementation is more complex, and is what most of the meeting time was spent discussing. From here on, all discussion of BSD/OS is with regard to 5.0. 1. Source code access. BSD/OS is a proprietary operating system, for which binary-only and source code licenses are available. BSD/OS is based on the same free sources (4.4BSD) as the free BSD operating systems. It is similar to FreeBSD, though the two have diverged significantly enough to cause serious pains when moving kernel code between them. A few weeks back, BSDi made the source code of BSD/OS available to all FreeBSD committers. During the meeting we discussed what this really means, and Kirk McKusick (amongst other things chairman of the board of BSDi) said, "Well, we're quite happy for you to take generous chunks, but if you ended up taking it all, people might get a little uneasy". Basically, anything short of simple repackaging of BSD/OS is not an issue. 2. The current problems. UNIX was written for single processor machines, and many of the design choices are not just suboptimal for SMP, they're just plain ugly. In particular the synchronization mechanisms don't work well with more than one processor. Briefly: - The process context, including the upper half of device drivers, doesn't need to protect itself. The kernel is non-preemptive, so as long as a process is executing in the kernel, no other process can execute in the kernel. If another process, even with higher priority, becomes runnable while a process is executing kernel code, it will have to wait until the active process leaves the kernel or sleeps. - Processes protect themselves against the interrupt context, primarily the bottom half of device drivers, by masking interrupts. The original PDP-11 UNIX used the hardware priority levels (numbered 4 to 7), and even today you'll find function calls like spl4() and spl7() in System V code. BSD changed the names to more descriptive terms like splbio(), splnet() and splhigh(), and also replaced the fixed priorities by interrupt masks, but the principle remains the same. It's not always easy to solve the question of which interrupts need to be masked in which context, and one of the interesting observations at this meeting was that as time goes on, the interrupt masks are getting blacker. In other words each spl() is masking off more and more bits in the interrupt mask register. This is not good for performance. - Processes synchronize with each other using the sleep() or tsleep() calls. Traditional UNIX, including System V, uses sleep(), and BSD prefers tsleep(), which provides nice strings which ps(1) displays to show what the process is waiting for. FreeBSD no longer has a sleep() call, while BSD/OS has both, but sleep() is deprecated. tsleep() is used both for voluntary process synchronization (e.g. send a request to another process and wait until it is finished), and for involuntary synchronization (e.g. wait for a shared resource to become available). Processes sleep on a specific address. In many cases, the address in itself has no meaning, and it's probably easier to think of it as a number. When a process sleeps, it is put on a sleep queue. The wakeup() function takes the sleep address, walks through the sleep queue, and wakes *every* process which is sleeping on this address. This can cause massive problems even on single processor machines; UNIX was never really intended to have hundreds of processes waiting on the same resource, and a number of Apache performance problems center around this behavior. As a partial solution, FreeBSD also has an additional function, wakeup_one(), which only wakes one process. There are a number of reasons why this concept is not a good solution for SMP. Firstly, the simplistic assumption "nothing else can be executing in the kernel while I am" falls flat. We currently haven't implemented a solution for this. Instead, we found a way of enforcing this illogical state, the Big Giant Lock (BGL). Any process entering the kernel must first obtain the BGL; if a process executing on another processor has the lock, then the current processor spins; it can't even schedule another process to run, because that requires entering the kernel. The other issue is with masking interrupts. This is also quite a problem for SMP machines, since it requires masking the interrupts on all processors, again requiring an expensive synchronization. 3. The BSD/OS solution. - The BGL remains, but becomes increasingly meaningless. In particular, it is not always necessary to obtain it in order to enter the kernel. - Instead the system protects shared data structures with mutexes. These mutexes replace calls to tsleep() when waiting on shared resources (the involuntary process synchronization mentioned above). In contrast to traditional UNIX, mutexes will be used much more frequently in order to protect data structures which were previously implicitly protected by the non-preemptive nature of the kernel. This mechanism will replace calls to tsleep() for involuntary context switches. Compared with the use of tsleep(), mutexes have a number of advantages: - Each mutex has its own wait (sleep) queue. When a process releases a mutex, it automatically schedules the next process waiting on the queue. This is more efficient than searching a possibly very long, linear sleep queue. It also avoids the flooding when multiple processes get scheduled, and most of them have to go back to sleep again. - Mutexes can be a combination of spin and sleep mutexes: for a resource which may be held only for a very short period of time, even the overhead of sleeping and rescheduling may be higher than waiting in a tight loop. A spin/sleep lock might first wait in a tight loop for 2 microseconds and then sleep if the lock is still not available at that time. This is an issue which Sun has investigated in great detail with Solaris. BSDi has not pursued this yet, though the BSD/OS threading primitives make this an easy extention to add. It's possibly an area for us to investigate once the system is up and limping again. - Interrupt lockout (spl()s) go away completely. Instead, interrupt functions use mutexes for synchronization. This means that an interrupt function must be capable of blocking, which is currently impossible. In order to block, the function must have a "process" context (a stack and a process control block). In particular, this could include kernel threads. BSD/OS on Intel currently uses light-weight interrupt threads to process interrupts, while on SPARC uses normal ("heavyweight") processes. Chuck indicated that the decision to implement light-weight threads initially was probably the wrong one, since it gave rise to a large number of problems, and although the heavyweight process model would give lousy performance, it would probably make it easier to develop the kernel while the light-weight processes were being debugged. There is also the possibility of building a kernel with one or the other support, so that in case of problems during development it would be possible to revert to the heavy-weight processes while searching for the bug. Other details we discussed included: - BSD/OS has not implemented condition variables. We didn't go into details. The opinion was expressed that they would be very useful for synchronization, but that they require coding discipline somewhat different than the tsleep() mechanism. Converting all use of tsleep() is a lot of work, and of dubious value. However, condition variables can live along with tsleep(), so a full changeover is not necessary. - BSD/OS also does not implement read/write locks, a thing that Linux has recently introduced. We didn't discuss this further, but the concepts are well understood, and it should be relatively simple to implement them if we find a need. - Netgraph poses locking performance problems, since locks have to be released at multiple potential transfer points, regardless of whether Netgraph is in use. This problem also exists with System V STREAMS. During the meeting we didn't come to a clear consensus on how much of a problem this really is. - Interrupts can have priority inversion problems on MP machines in combination with lazy context switching (aka context stealing). However, it's a temporary inversion that just causes latency. The reason that deadlock never occurs is that as soon as a lock is missed, the interrupt stack stealing is unwound, so there is never a situation where a lock is held that can cause deadlock. When a high-priority process waits for a lower priority process, the blocking process temporarily lends its priority to the running process in order to ensure that it finishes quickly. This technique is interchangeably called priority inheritance, priority lending, deadlock avoidance, and probably other names, just to make things confusing. - NFS leasing causes big problems. Samba will have similar problems, potentially. - Message queues are probably worthwhile, but they're currently not a high priority. - There are a number of global variable updates that are not locked, and can thus result in partially updated variables (i.e. reads can get corrupt values). This requires either using a locked instruction, or using a mutex. A mutex isn't much more expensive, and is probably easier. - We should split part of struct proc out into a fixed-size kproc for ps use. This isn't really related to the SMP work, but it would be nice to get rid of the dreaded "proc size mismatch" error message that people get when their kernel is out of sync with userland. - We spoke about naming conventions. Some people weren't too happy with BSD/OS's macro names. Chuck agreed and said that he would adopt our naming convention if we chose a better one. - Per-CPU variables need GET_*() and SET_*() routines to lock. 4. Things we need to do. There are a number of things we need to do. During the meeting we didn't get beyond deciding the first couple of things: - First remove the BGL (currently a spinlock) and replace it with two, maybe three mutexes, one for the scheduler (schedlock), and a blocking mutex for the kernel in place of the BGL. BSD/OS also has an ipending lock for posting interrupts, which we should probably implement in the short term, though it's possible that it might go again. - In addition, implement the heavy-weight interrupt processes. These would remain in place while the light-weight threads were being debugged. 5. Who does what? A number of people will work on the SMP project. During the first stage: - Matt Dillon will put in locking primitives and schedlock. This includes resurrecting our long-dead idle process to scan the run queue for interrupt threads. He won't have time for NFS. - Doug Rabson will work on the alpha bits, so that it doesn't get left in the dust. - Greg Lehey will implement the heavyweight interrupt processes and lightweight interrupt threads. - Jason Evans will be the project manager. 6. Timing. We have a general agreement that it's better to do it right than do it quickly. Thus far, Matt has implemented much of his part and is now waiting on Greg to do the interrupt processes. When they've done that, they'll do their own tests, and others will do additional testing. All commits will be dependent on approval from Jason, and the first can be expected within two months (probably sooner). The SMP changes will be maintained as patches against -current until the following milestones have been met: - Port the BSD/OS locking primitives to the i386 port (Matt) and the alpha port (Doug Rabson). - Convert the BGL to a blocking lock, add the schedlock, add per-CPU idle processes (Matt). - Implement heavy-weight interrupt threads (Greg). Light-weight interrupt thread context switching may be working by the time the first commit is made, but this is not a requirement. - Stub out (basically disable) spl()s. - Demonstrated successful compilation and reasonable stability (self-hosted kernel build) on both i386 (UP and SMP) and alpha. The maintenance of the patches is expected to be a bit of pain, but we have decided not to branch due to technical issues with maintaining branches in CVS. The patches are expected to exist only until the first commit is made. At that point, all further development will be done directly on HEAD in cvs. On the light side, we had a rather amusing experience on Friday. We wanted to order some sandwiches, but something went wrong with the order, so Paul ordered pizza instead. A bit later, the pizza boy came in and deposited the pizzas on the conference table and was about to leave when Paul introduced him. His name is David Filo. Thanks for the pizza! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message