From owner-freebsd-smp Sun Jun 25 6:58:52 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pcnet1.pcnet.com (pcnet1.pcnet.com [204.213.232.3]) by hub.freebsd.org (Postfix) with ESMTP id 1C42337B618 for ; Sun, 25 Jun 2000 06:58:48 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: (from eischen@localhost) by pcnet1.pcnet.com (8.8.7/PCNet) id JAA07263; Sun, 25 Jun 2000 09:58:27 -0400 (EDT) Date: Sun, 25 Jun 2000 09:58:27 -0400 (EDT) From: Daniel Eischen To: Jason Evans Cc: smp@FreeBSD.ORG Subject: Re: SMP meeting summary In-Reply-To: <20000624235605.D8965@blitz.canonware.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 24 Jun 2000, Jason Evans wrote: > 3. The BSD/OS solution. > > - The BGL remains, but becomes increasingly meaningless. In particular, > it is not always necessary to obtain it in order to enter the kernel. > > - Instead the system protects shared data structures with mutexes. > These mutexes replace calls to tsleep() when waiting on shared > resources (the involuntary process synchronization mentioned above). > In contrast to traditional UNIX, mutexes will be used much more > frequently in order to protect data structures which were previously > implicitly protected by the non-preemptive nature of the kernel. This > mechanism will replace calls to tsleep() for involuntary context > switches. > > Compared with the use of tsleep(), mutexes have a number of > advantages: > > - Each mutex has its own wait (sleep) queue. When a process releases > a mutex, it automatically schedules the next process waiting on the > queue. This is more efficient than searching a possibly very long, > linear sleep queue. It also avoids the flooding when multiple > processes get scheduled, and most of them have to go back to sleep > again. > > - Mutexes can be a combination of spin and sleep mutexes: for a > resource which may be held only for a very short period of time, > even the overhead of sleeping and rescheduling may be higher than > waiting in a tight loop. A spin/sleep lock might first wait in a > tight loop for 2 microseconds and then sleep if the lock is still > not available at that time. This is an issue which Sun has > investigated in great detail with Solaris. BSDi has not pursued > this yet, though the BSD/OS threading primitives make this an easy > extention to add. It's possibly an area for us to investigate once > the system is up and limping again. If anyone is interested... All high-level interrupts (levels 11-15, mostly PIO serial interrupts) in Solaris use spin mutexes and don't use an interrupt thread. They execute in the context of the thread that was currently running. All other interrupts below level 11 (clock, network, disk, etc) use interrupt threads. A Solaris (non-spinning) mutex will only spin while the owning thread is running. Since BSDi mutexes have owners (correct me if I'm wrong), this seems to be better than arbitrarily spinning. > > - Interrupt lockout (spl()s) go away completely. Instead, interrupt > functions use mutexes for synchronization. This means that an > interrupt function must be capable of blocking, which is currently > impossible. In order to block, the function must have a "process" > context (a stack and a process control block). In particular, this > could include kernel threads. > > BSD/OS on Intel currently uses light-weight interrupt threads to > process interrupts, while on SPARC uses normal ("heavyweight") > processes. Chuck indicated that the decision to implement > light-weight threads initially was probably the wrong one, since it > gave rise to a large number of problems, and although the heavyweight > process model would give lousy performance, it would probably make it > easier to develop the kernel while the light-weight processes were > being debugged. There is also the possibility of building a kernel > with one or the other support, so that in case of problems during > development it would be possible to revert to the heavy-weight > processes while searching for the bug. > > Other details we discussed included: > > - BSD/OS has not implemented condition variables. We didn't go into > details. The opinion was expressed that they would be very useful for > synchronization, but that they require coding discipline somewhat > different than the tsleep() mechanism. Converting all use of tsleep() > is a lot of work, and of dubious value. However, condition variables > can live along with tsleep(), so a full changeover is not necessary. For a lot of drivers, it seems pretty straight forward to convert splXXX() ... tsleep() ... splx() to mtx_enter() ... cv_wait()/cv_wait_sig() ... mtx_exit(). > > - BSD/OS also does not implement read/write locks, a thing that Linux > has recently introduced. We didn't discuss this further, but the > concepts are well understood, and it should be relatively simple to > implement them if we find a need. Mutexes are only used in Solaris when they will be held for very small amounts of time. Read/write locks and semaphores are used for all other instances. While we are modifying the kernel to add mutexes, it would probably be worthwhile to comment those sections of code that could hold mutexes for something other than a very short period of time. Or even use a different naming convention for those mutexes. -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Jun 25 9:19:50 2000 Delivered-To: freebsd-smp@freebsd.org Received: from zippy.osd.bsdi.com (zippy.cdrom.com [204.216.27.228]) by hub.freebsd.org (Postfix) with ESMTP id 3AEA537BBA5; Sun, 25 Jun 2000 09:19:37 -0700 (PDT) (envelope-from jkh@zippy.osd.bsdi.com) Received: from localhost (jkh@localhost [127.0.0.1]) by zippy.osd.bsdi.com (8.9.3/8.9.3) with ESMTP id JAA02985; Sun, 25 Jun 2000 09:20:14 -0700 (PDT) (envelope-from jkh@zippy.osd.bsdi.com) To: committers@freebsd.org Cc: smp@freebsd.org Subject: SMPng effort with BSD/OS - the overall story Date: Sun, 25 Jun 2000 09:20:14 -0700 Message-ID: <2982.961950014@localhost> From: "Jordan K. Hubbard" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org First, as the guy who set up the recent "SMP seminar", let me just apologise right now for dropping the ball on one very significant item: I didn't have someone appointed "scribe" from the list of full-time employee types who went there and this resulted in an unfortunate delay in briefing those people who weren't in attendance. Greg Lehey did do some video taping of the event, but that was just something he did on his own and it will take him some time to figure out how to get it into a format which everyone else can use. Again, in retrospect, we should have arranged this more formally and I'll take full responsibility for blowing it there. Sorry. Jason Evans has just posted a fuller synopsis of the event to the freebsd-smp mailing list and will be issuing periodic updates as things progress. Speaking of Jason, I should also point out that the "SMP working group" who convened for this meeting did have the wisdom to appoint an overall manager for this effort (sorry for clapping the pointy-hair helmet on you, Jason) since it's a BIG JOB we're talking about here. Having a full-time person of Jason's caliber driving and organizing things for this effort is exactly what we need if we actually want to finish it and NOT do things like break -current for extended periods of time, something there has been a lot of fear expressed over in the last couple of days. Please rest assured that nobody has been issued a flame thrower and told to direct it at -current. The situation is quite the contrary and by appointing Jason "ringmaster" of the SMP effort, we hope to avoid a lot of that kind of pain. What he says goes with regard to this integration effort and NO SMP RELATED CODE ENTERS -CURRENT WITHOUT HIS APPROVAL! This will not be an ad-hack effort, it will be an organized project with some very big goals on its plate and Jason Evans has core's full support as its ring master. As far as we're concerned, it's up to Jason to make sure things proceed smoothly with the SMP merger process and up to him to take any corrective actions which may be necessary along the way. It's also up to him to make sure timely bulletins are issued to the rest of committers so that everyone is kept well up-to-date on its progress and given full opportunity to contribute to the effort. The group which met at Yahoo last week were not "the chosen few" who will dictate all aspects of how things go, they were simply the people who put their hands up when I called for volunteers to attend the event in the freebsd-smp list and SOME of the people who will be driving the effort. The only person to come out of that meeting with a funny hat was Jason, and he kind of had that forced on him anyway. :-) - Jordan [Speaking for core] To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Jun 25 10:13:27 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp02.primenet.com (smtp02.primenet.com [206.165.6.132]) by hub.freebsd.org (Postfix) with ESMTP id 25DC837BC5A for ; Sun, 25 Jun 2000 10:13:12 -0700 (PDT) (envelope-from tlambert@usr02.primenet.com) Received: (from daemon@localhost) by smtp02.primenet.com (8.9.3/8.9.3) id KAA12048; Sun, 25 Jun 2000 10:12:22 -0700 (MST) Received: from usr02.primenet.com(206.165.6.202) via SMTP by smtp02.primenet.com, id smtpdAAAmfaOGx; Sun Jun 25 10:12:21 2000 Received: (from tlambert@localhost) by usr02.primenet.com (8.8.5/8.8.5) id KAA09348; Sun, 25 Jun 2000 10:13:07 -0700 (MST) From: Terry Lambert Message-Id: <200006251713.KAA09348@usr02.primenet.com> Subject: Re: SMP meeting summary To: jasone@canonware.com (Jason Evans) Date: Sun, 25 Jun 2000 17:13:07 +0000 (GMT) Cc: smp@FreeBSD.ORG In-Reply-To: <20000624235605.D8965@blitz.canonware.com> from "Jason Evans" at Jun 24, 2000 11:56:05 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > - Netgraph poses locking performance problems, since locks have to be > released at multiple potential transfer points, regardless of whether > Netgraph is in use. This problem also exists with System V STREAMS. > During the meeting we didn't come to a clear consensus on how much of > a problem this really is. FWIW, Soft Updates will have similar issues. In Streams, this was addressed by removing two primitives, and creating what are called "priority bands" to deal with the issue (I worked for Novell USG when UnixWare was going to 2.0 and integrating the ES/MP code into the SVR4 base, and saw much of this happen). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Jun 25 10:24:45 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 2C48437BAEA for ; Sun, 25 Jun 2000 10:24:42 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id KAA22733; Sun, 25 Jun 2000 10:24:39 -0700 (PDT) (envelope-from dillon) Date: Sun, 25 Jun 2000 10:24:39 -0700 (PDT) From: Matthew Dillon Message-Id: <200006251724.KAA22733@apollo.backplane.com> To: David Greenman Cc: freebsd-smp@FreeBSD.ORG Subject: Re: Stepping on Toes References: <200006250411.VAA08876@implode.root.com> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org A new SMP patchset is available, bringing it up to date with changes committed -current in recent days (that caused conflicts). This patchset does not yet integrate the alpha patch supplied by Jake Burkholder. (p.s. Jake, was that patch just to make things compile or to make the mutex stuff actually *run* on an alpha?). : Of course we need to wait a few days for people to return from Usenix. I :just got back myself a few hours ago and have over 3000 emails to deal with. :My general view is that we should probably go directly to the threaded :interrupt model and dump spls in the first cut. Given this, I think it would :be prudent to use as much of the BSDI mutex code as possible, at least :for the medium term, since it has been well tested and debugged. Not using :it reaks of NIH to folks on the outside, even though the SMPng group (at :least) knows that that wasn't at all your motivation in rolling your own. : :-DG : :David Greenman Even with interrupt threads we have the GiantMutex issue... the same issue that we have with our current MP implementation. We cannot remove SPL's until we remove the GiantMutex, and we cannot remove GiantMutex without major modifications to just about every single source file in sys/ This is not a trivial job. If we attempt to remove SPLs prior to making the first commit and in tandem with implementing interrupt threads, the result is likely going to be a nightmware disaster (read: months of extra work debugging problems). As much as I like digging into the kernel, I would balk at that (even if I had the time available). The core mutexes (Sched and Giant) and Interrupt threads (both heavy and light weight) are compartmentalizable problems. They can be done piecemeal and thus debugged quite easily alone. If we throw in a thousand other requirements the mess we are going to be left with will be much, much harder to debug. I think the project will go a whole lot faster if we start committing to -current earlier and go through a three-stage process to get to where BSDI is now: * SchedMutex, GiantMutex, spl* compatibility (my patch) ... and commit * Interrupt Threads (Greg) ... and commit * Remove GiantMutex, spl*() replacement (one subsystem at a time) ... and commit, commit, commit Even if you do not wind up using my mutex code (I presume the rest of my patch would be used since I doubt anyone else wants to slog through the interrupt code. It should be easy to separate out the mutex implementation), I still recommend the above three-step process. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Jun 25 10:37:16 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp02.primenet.com (smtp02.primenet.com [206.165.6.132]) by hub.freebsd.org (Postfix) with ESMTP id 5194837B737 for ; Sun, 25 Jun 2000 10:37:10 -0700 (PDT) (envelope-from tlambert@usr02.primenet.com) Received: (from daemon@localhost) by smtp02.primenet.com (8.9.3/8.9.3) id KAA16602; Sun, 25 Jun 2000 10:36:21 -0700 (MST) Received: from usr02.primenet.com(206.165.6.202) via SMTP by smtp02.primenet.com, id smtpdAAA2GaaAG; Sun Jun 25 10:36:16 2000 Received: (from tlambert@localhost) by usr02.primenet.com (8.8.5/8.8.5) id KAA09884; Sun, 25 Jun 2000 10:36:58 -0700 (MST) From: Terry Lambert Message-Id: <200006251736.KAA09884@usr02.primenet.com> Subject: Re: SMP meeting summary To: eischen@vigrid.com (Daniel Eischen) Date: Sun, 25 Jun 2000 17:36:57 +0000 (GMT) Cc: jasone@canonware.com (Jason Evans), smp@FreeBSD.ORG In-Reply-To: from "Daniel Eischen" at Jun 25, 2000 09:58:27 AM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > All high-level interrupts (levels 11-15, mostly PIO serial interrupts) > in Solaris use spin mutexes and don't use an interrupt thread. They > execute in the context of the thread that was currently running. All > other interrupts below level 11 (clock, network, disk, etc) use interrupt > threads. > > A Solaris (non-spinning) mutex will only spin while the owning thread is > running. Since BSDi mutexes have owners (correct me if I'm wrong), this > seems to be better than arbitrarily spinning. We need to learn from Dynix (Sequent's UNIX). The main issue that block concurrency is access to shared resources. Critical sectioning is actually better than mutex protection of structures for maximizing concurrency, but few people appear to be willing to go down this road, since it requires flatening the call graph for much of the kernel to ensure that locks are held and released at the same call level, so that stack unwinding is not needed to permit preemption. Dynix had no problem with 32 processors. Most SVR4 variants, and I will include Solaris in this, use mutex protection of structures, and start to fall down drastically over 4 processors. The main reason Dynix did not have this scaling issue is that it dealt with the shared resource issue by placing most objects into per-processor allocation/deallocation pools. These pools were filled/drained from/to system pools. Lock contention was only necessary when the pools needed filling/draining, or when an object was being migrated between CPUs. Similarly, one can consider that the idea of CPU reentrancy into the kernel is identical in all but inter-CPU synchronization to the idea of kernel preemption. It would perhaps be a good idea from this standpoint to adopt the realtime code recently donated to the OpenBSD project, since the issues involved in making a kernel RT are similar to those of ensuring SMP kernel reentrancy without blocking on resource contention. > Mutexes are only used in Solaris when they will be held for very small > amounts of time. Read/write locks and semaphores are used for all > other instances. While we are modifying the kernel to add mutexes, > it would probably be worthwhile to comment those sections of code > that could hold mutexes for something other than a very short period > of time. Or even use a different naming convention for those mutexes. Anything that can hold a mutex for other than a very short time will need to go away. This is one of the problems with data protection rather than critical sectioning. Reader/writer locks are an obvious optimization, if one is to use mutex protection of data. Another similar optimization is intention mode locking. The Soft Updates dependency flooding problem that is associated with an update being commited to the update clock list, and someone else needing to access it (the poor ZD Labs benchmark results were in part traced to this), is one place where intention mode locks would be useful in increasing concurrency. Search altavista for "+intention +lock +SIX" to find the relevent literature. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Jun 25 10:43: 7 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mail.interware.hu (mail.interware.hu [195.70.32.130]) by hub.freebsd.org (Postfix) with ESMTP id 8991437B737 for ; Sun, 25 Jun 2000 10:42:44 -0700 (PDT) (envelope-from julian@elischer.org) Received: from dakar-05.budapest.interware.hu ([195.70.51.69] helo=jules.elischer.org) by mail.interware.hu with smtp (Exim 3.12 #1 (Debian)) id 136GfV-0006XD-00; Sun, 25 Jun 2000 19:58:06 +0200 Message-ID: <39564465.794BDF32@elischer.org> Date: Sun, 25 Jun 2000 10:41:57 -0700 From: Julian Elischer X-Mailer: Mozilla 3.04Gold (X11; I; FreeBSD 5.0-CURRENT i386) MIME-Version: 1.0 To: Jason Evans Cc: smp@freebsd.org Subject: Re: SMP meeting summary References: <20000624235605.D8965@blitz.canonware.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Jason Evans wrote: > - Netgraph poses locking performance problems, since locks have to be > released at multiple potential transfer points, regardless of whether > Netgraph is in use. This problem also exists with System V STREAMS. > During the meeting we didn't come to a clear consensus on how much of > a problem this really is. > I think this is a mis-stated issue. There is no REQUIREMENT that locks be released or held at every transfer between modules. While it is the case that some modules may require locking operations, it cannot be said that there is a 1:1 correspondence between locks and modules. A packet could wuite easily travel thought quite a number of modules without any additional locking work. Many modules would have no resources worth protecting.. (at least in the common case) -- __--_|\ Julian Elischer / \ julian@elischer.org ( OZ ) World tour 2000 )_.---._/ presently in: Budapest v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Jun 25 11: 8:14 2000 Delivered-To: freebsd-smp@freebsd.org Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.40.131]) by hub.freebsd.org (Postfix) with ESMTP id 3AC9337BB4F for ; Sun, 25 Jun 2000 11:07:57 -0700 (PDT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.9.3/8.9.3) with ESMTP id UAA13300; Sun, 25 Jun 2000 20:07:44 +0200 (CEST) (envelope-from phk@critter.freebsd.dk) To: Julian Elischer Cc: Jason Evans , smp@FreeBSD.ORG Subject: Re: SMP meeting summary In-reply-to: Your message of "Sun, 25 Jun 2000 10:41:57 PDT." <39564465.794BDF32@elischer.org> Date: Sun, 25 Jun 2000 20:07:44 +0200 Message-ID: <13298.961956464@critter.freebsd.dk> From: Poul-Henning Kamp Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org In message <39564465.794BDF32@elischer.org>, Julian Elischer writes: >Jason Evans wrote: > >> - Netgraph poses locking performance problems, since locks have to be >> released at multiple potential transfer points, regardless of whether >> Netgraph is in use. This problem also exists with System V STREAMS. >> During the meeting we didn't come to a clear consensus on how much of >> a problem this really is. >> > > >I think this is a mis-stated issue. >There is no REQUIREMENT that locks be released >or held at every transfer between modules. > >While it is the case that some modules may require >locking operations, it cannot be said that there is a 1:1 >correspondence between locks and modules. >A packet could wuite easily travel thought quite >a number of modules without any additional locking work. >Many modules would have no resources worth protecting.. >(at least in the common case) The "cisco" module being a very good example. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD coreteam member | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Jun 25 11:20:28 2000 Delivered-To: freebsd-smp@freebsd.org Received: from io.yi.org (24.67.218.186.bc.wave.home.com [24.67.218.186]) by hub.freebsd.org (Postfix) with ESMTP id 8E7B537BC2D for ; Sun, 25 Jun 2000 11:20:15 -0700 (PDT) (envelope-from jburkhol@home.com) Received: from io.yi.org (localhost.gvcl1.bc.wave.home.com [127.0.0.1]) by io.yi.org (Postfix) with ESMTP id B0259BA4E; Sun, 25 Jun 2000 11:20:18 -0700 (PDT) X-Mailer: exmh version 2.1.1 10/15/1999 To: Matthew Dillon Cc: David Greenman , freebsd-smp@FreeBSD.ORG, jake@io.yi.org Subject: Re: Stepping on Toes In-Reply-To: Message from Matthew Dillon of "Sun, 25 Jun 2000 10:24:39 PDT." <200006251724.KAA22733@apollo.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Sun, 25 Jun 2000 11:20:18 -0700 From: Jake Burkholder Message-Id: <20000625182018.B0259BA4E@io.yi.org> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > A new SMP patchset is available, bringing it up to date with changes > committed -current in recent days (that caused conflicts). > > This patchset does not yet integrate the alpha patch supplied by > Jake Burkholder. (p.s. Jake, was that patch just to make things > compile or to make the mutex stuff actually *run* on an alpha?). > It was to get a feel for what you were doing, and for what the changes would be like. Since then I've done more studying, and I feel like I have a good understanding of what your patch does. The initial patch did nothing but add the mutexes, since then I've added grabbing the locks in what seems to be the right places, and changed the assembly to rely on idleproc and to save/restore the sched_mutex nesting count in the pcb. This was mostly a mental exercise. I understand that Doug Rabson will be working on the alpha port; I'd like to help but I don't want to step on any toes either. If you want I can post that patch too; it builds and might work, but will certainly have problems with interrupts. As an aside, the assembly in sys/alpha/alpha is minimal, most of the code is C and easy to understand. Jake To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Jun 25 21:42:31 2000 Delivered-To: freebsd-smp@freebsd.org Received: from ns.yogotech.com (ns.yogotech.com [206.127.123.66]) by hub.freebsd.org (Postfix) with ESMTP id 8A16A37BA78 for ; Sun, 25 Jun 2000 21:42:25 -0700 (PDT) (envelope-from nate@yogotech.com) Received: from nomad.yogotech.com (nomad.yogotech.com [206.127.123.131]) by ns.yogotech.com (8.9.3/8.9.3) with ESMTP id WAA04205; Sun, 25 Jun 2000 22:42:09 -0600 (MDT) (envelope-from nate@nomad.yogotech.com) Received: (from nate@localhost) by nomad.yogotech.com (8.8.8/8.8.8) id WAA15731; Sun, 25 Jun 2000 22:42:02 -0600 (MDT) (envelope-from nate) Date: Sun, 25 Jun 2000 22:42:02 -0600 (MDT) Message-Id: <200006260442.WAA15731@nomad.yogotech.com> From: Nate Williams MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: Terry Lambert Cc: eischen@vigrid.com (Daniel Eischen), jasone@canonware.com (Jason Evans), smp@FreeBSD.ORG Subject: Re: SMP meeting summary In-Reply-To: <200006251736.KAA09884@usr02.primenet.com> References: <200006251736.KAA09884@usr02.primenet.com> X-Mailer: VM 6.34 under 19.16 "Lille" XEmacs Lucid Reply-To: nate@yogotech.com (Nate Williams) Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Dynix had no problem with 32 processors. Most SVR4 variants, and > I will include Solaris in this, use mutex protection of structures, > and start to fall down drastically over 4 processors. Amazing that you say this, yet I see extremely good results on Solaris boxes up to 64 processors. Suffice it to say that I'm not convinced, nor am I convinced that mutex's around data structures is any different than critical sectioning. They are essentially the same thing, in that the critical section is almost always the code that deals with a particular (shared) data structure. Nate To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Jun 25 23:31:59 2000 Delivered-To: freebsd-smp@freebsd.org Received: from tinker.exit.com (exit-gw.power.net [207.151.46.196]) by hub.freebsd.org (Postfix) with ESMTP id D62FA37B9AC for ; Sun, 25 Jun 2000 23:31:54 -0700 (PDT) (envelope-from frank@exit.com) Received: from realtime.exit.com (realtime.exit.com [206.223.0.5]) by tinker.exit.com (8.9.3/8.9.3) with ESMTP id XAA05675; Sun, 25 Jun 2000 23:31:52 -0700 (PDT) (envelope-from frank@exit.com) Received: (from frank@localhost) by realtime.exit.com (8.9.3/8.9.3) id XAA43949; Sun, 25 Jun 2000 23:31:51 -0700 (PDT) (envelope-from frank) From: Frank Mayhar Message-Id: <200006260631.XAA43949@realtime.exit.com> Subject: Re: SMP meeting summary In-Reply-To: <200006251713.KAA09348@usr02.primenet.com> from Terry Lambert at "Jun 25, 2000 05:13:07 pm" To: Terry Lambert Date: Sun, 25 Jun 2000 23:31:51 -0700 (PDT) Cc: Jason Evans , smp@FreeBSD.ORG Reply-To: frank@exit.com Organization: Exit Consulting X-Copyright0: Copyright 2000 Frank Mayhar. All Rights Reserved. X-Copyright1: Permission granted for electronic reproduction as Usenet News or email only. X-Mailer: ELM [version 2.4ME+ PL68 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Terry Lambert wrote: > In Streams, this was addressed by removing two primitives, and creating > what are called "priority bands" to deal with the issue (I worked for > Novell USG when UnixWare was going to 2.0 and integrating the ES/MP code > into the SVR4 base, and saw much of this happen). Speaking of which, there has _got_ to be a better way. It's possible when dealing with Streams to get into nested-lock situations where you need a lock with a lower priority and it's not possible (due to constraints elsewhere) to raise the priority of the lock. And, of course, it's always dangerous to raise the priority of an existing lock, since you may miss a case. I'm not sure that interrupt threads are the right way to go either, though. If there even _is_ a "right" way. -- Frank Mayhar frank@exit.com http://www.exit.com/ Exit Consulting http://store.exit.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Sun Jun 25 23:32:44 2000 Delivered-To: freebsd-smp@freebsd.org Received: from tinker.exit.com (exit-gw.power.net [207.151.46.196]) by hub.freebsd.org (Postfix) with ESMTP id E18C737B9AC for ; Sun, 25 Jun 2000 23:32:39 -0700 (PDT) (envelope-from frank@exit.com) Received: from realtime.exit.com (realtime.exit.com [206.223.0.5]) by tinker.exit.com (8.9.3/8.9.3) with ESMTP id XAA05686 for ; Sun, 25 Jun 2000 23:32:27 -0700 (PDT) (envelope-from frank@exit.com) Received: (from frank@localhost) by realtime.exit.com (8.9.3/8.9.3) id XAA43962 for smp@freebsd.org; Sun, 25 Jun 2000 23:32:27 -0700 (PDT) (envelope-from frank) From: Frank Mayhar Message-Id: <200006260632.XAA43962@realtime.exit.com> Subject: Re: SMP meeting summary In-Reply-To: <200006260442.WAA15731@nomad.yogotech.com> from Nate Williams at "Jun 25, 2000 10:42:02 pm" To: Nate Williams Date: Sun, 25 Jun 2000 23:27:25 -0700 (PDT) Cc: Terry Lambert , Daniel Eischen , Jason Evans , smp@FreeBSD.ORGG Reply-To: frank@exit.com Organization: Exit Consulting X-Copyright0: Copyright 2000 Frank Mayhar. All Rights Reserved. X-Copyright1: Permission granted for electronic reproduction as Usenet News or email only. X-Mailer: ELM [version 2.4ME+ PL68 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Nate Williams wrote: > > Dynix had no problem with 32 processors. Most SVR4 variants, and > > I will include Solaris in this, use mutex protection of structures, > > and start to fall down drastically over 4 processors. > Amazing that you say this, yet I see extremely good results on Solaris > boxes up to 64 processors. Hmm. Do you have numbers? > Suffice it to say that I'm not convinced, nor am I convinced that > mutex's around data structures is any different than critical > sectioning. > > They are essentially the same thing, in that the critical section is > almost always the code that deals with a particular (shared) data > structure. I agree with this, but I can state that Unixware doesn't scale well (i.e. linearly) over roughly four (or possibly eight, its been a while since I looked at the numbers) processors. This has been clearly shown by various and sundry benchmarks at Compaq and elsewhere. I do like the "per-cpu pool" idea, though (although I haven't yet thought it completely through). This would have to, I think, go along with a much stronger CPU affinity for threads and interrupts. I clearly see that, under 4.0, processes pretty much freely migrate from CPU to CPU. This is bad for SMP performance (kills the cache) and would also mean that if a process using a particular structure were on a different CPU, it would have to either move to the proper CPU or move the structure into the per-CPU pool of the CPU it's using. Or use a non-CPU-pool structure. I'll have to think about this some more. It's an interesting idea, though. -- Frank Mayhar frank@exit.com http://www.exit.com/ Exit Consulting http://store.exit.com/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jun 26 9:47: 9 2000 Delivered-To: freebsd-smp@freebsd.org Received: from lor.watermarkgroup.com (lor.watermarkgroup.com [207.202.73.33]) by hub.freebsd.org (Postfix) with ESMTP id 1453537B9F7 for ; Mon, 26 Jun 2000 09:47:07 -0700 (PDT) (envelope-from luoqi@watermarkgroup.com) Received: (from luoqi@localhost) by lor.watermarkgroup.com (8.10.1/8.10.1) id e5QGkUS06290; Mon, 26 Jun 2000 12:46:30 -0400 (EDT) Date: Mon, 26 Jun 2000 12:46:30 -0400 (EDT) From: Luoqi Chen Message-Id: <200006261646.e5QGkUS06290@lor.watermarkgroup.com> To: jasone@canonware.com, smp@FreeBSD.ORG Subject: Re: SMP meeting summary Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Compared with the use of tsleep(), mutexes have a number of > advantages: > > - Each mutex has its own wait (sleep) queue. When a process releases > a mutex, it automatically schedules the next process waiting on the > queue. This is more efficient than searching a possibly very long, > linear sleep queue. It also avoids the flooding when multiple > processes get scheduled, and most of them have to go back to sleep > again. > What about processes of different priorities blocking for the same mutex? Would you do a linear search on the queue? or have the queue sorted by priority? or a FIFO queue is good enough? -lq To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jun 26 9:48:27 2000 Delivered-To: freebsd-smp@freebsd.org Received: from sharmas.dhs.org (c62443-a.frmt1.sfba.home.com [24.0.69.165]) by hub.freebsd.org (Postfix) with ESMTP id 4C8E037BBA9 for ; Mon, 26 Jun 2000 09:48:18 -0700 (PDT) (envelope-from adsharma@sharmas.dhs.org) Received: (from adsharma@localhost) by sharmas.dhs.org (8.9.3/8.9.3) id JAA07306; Mon, 26 Jun 2000 09:47:31 -0700 Date: Mon, 26 Jun 2000 09:47:31 -0700 From: Arun Sharma Message-Id: <200006261647.JAA07306@sharmas.dhs.org> To: smp@freebsd.org Subject: Re: SMP meeting summary In-Reply-To: <200006260442.WAA15731@nomad.yogotech.com> References: <200006251736.KAA09884@usr02.primenet.com> <200006260442.WAA15731@nomad.yogotech.com> Reply-To: adsharma@sharmas.dhs.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Sun, 25 Jun 2000 22:42:02 -0600 (MDT), Nate Williams wrote: > Suffice it to say that I'm not convinced, nor am I convinced that > mutex's around data structures is any different than critical > sectioning. > > They are essentially the same thing, in that the critical section is > almost always the code that deals with a particular (shared) data > structure. I'd argue that mutexes around data structures allow more concurrency than critical sections. It's the "lock the data - not code" principle. Think of the case where there are a thousand instances of the data structure and one critical section. -Arun To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jun 26 9:51: 8 2000 Delivered-To: freebsd-smp@freebsd.org Received: from berserker.bsdi.com (berserker.twistedbit.com [199.79.183.1]) by hub.freebsd.org (Postfix) with ESMTP id 9635C37B60E for ; Mon, 26 Jun 2000 09:51:00 -0700 (PDT) (envelope-from cp@berserker.bsdi.com) Received: from berserker.bsdi.com (cp@LOCALHOST [127.0.0.1]) by berserker.bsdi.com (8.9.3/8.9.3) with ESMTP id KAA17801; Mon, 26 Jun 2000 10:50:44 -0600 (MDT) Message-Id: <200006261650.KAA17801@berserker.bsdi.com> To: Matthew Dillon Cc: David Greenman , freebsd-smp@freebsd.org Subject: Re: Stepping on Toes From: Chuck Paterson Date: Mon, 26 Jun 2000 10:50:44 -0600 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org } } Even with interrupt threads we have the GiantMutex issue... the same } issue that we have with our current MP implementation. We cannot remove } SPL's until we remove the GiantMutex, and we cannot remove GiantMutex } without major modifications to just about every single source file in sys/ } In general this isn't true. If you get to the point where 1) All entrance to unsafe code is proteced by Giant. 2) Tsleep and friend if any release Giant when they a process is suspended and re-acquire it on exit 3) Interrupts have a context to run in. 4) You have one or more scheduling locks. Then you can just turn spls into a nop. There is lots of hand waving in regards to details at this point. BSD/OS SMPng is the existance proof. It seems like one of the major problems in retaining spls during the change over period is that they don't much useful, and effectively push everything under Giant. Grabbing a spl will only block interrupts, it will not give any protection against an interrupt thread which has already started. This means that any device which might be blocked by splbio() can not be brought out from under the Giant lock until all instances of splbio have been removed. Chuck To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jun 26 9:57:36 2000 Delivered-To: freebsd-smp@freebsd.org Received: from sharmas.dhs.org (c62443-a.frmt1.sfba.home.com [24.0.69.165]) by hub.freebsd.org (Postfix) with ESMTP id 3A9D537BA1B for ; Mon, 26 Jun 2000 09:57:34 -0700 (PDT) (envelope-from adsharma@sharmas.dhs.org) Received: (from adsharma@localhost) by sharmas.dhs.org (8.9.3/8.9.3) id JAA07384 for smp@freebsd.org; Mon, 26 Jun 2000 09:56:48 -0700 Date: Mon, 26 Jun 2000 09:56:48 -0700 From: Arun Sharma To: smp@freebsd.org Subject: kern/18524 Message-ID: <20000626095648.A7369@sharmas.dhs.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Now that SMP and per CPU pools are hot again, I'm hoping that someone on this list will consider my patch: http://www.FreeBSD.org/cgi/query-pr.cgi?pr=18524 Related patch: http://www.FreeBSD.org/cgi/query-pr.cgi?pr=16928 -Arun To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jun 26 11: 7:24 2000 Delivered-To: freebsd-smp@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 2F0AF37BC3E for ; Mon, 26 Jun 2000 11:07:20 -0700 (PDT) (envelope-from jasone@magnesium.net) Received: (qmail 65785 invoked by uid 1142); 26 Jun 2000 18:07:16 -0000 Date: 26 Jun 2000 11:07:16 -0700 Date: Mon, 26 Jun 2000 11:06:33 -0700 From: Jason Evans To: Luoqi Chen Cc: smp@FreeBSD.ORG Subject: Re: SMP meeting summary Message-ID: <20000626110633.F8965@blitz.canonware.com> References: <200006261646.e5QGkUS06290@lor.watermarkgroup.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <200006261646.e5QGkUS06290@lor.watermarkgroup.com>; from luoqi@watermarkgroup.com on Mon, Jun 26, 2000 at 12:46:30PM -0400 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Mon, Jun 26, 2000 at 12:46:30PM -0400, Luoqi Chen wrote: > > Compared with the use of tsleep(), mutexes have a number of > > advantages: > > > > - Each mutex has its own wait (sleep) queue. When a process releases > > a mutex, it automatically schedules the next process waiting on the > > queue. This is more efficient than searching a possibly very long, > > linear sleep queue. It also avoids the flooding when multiple > > processes get scheduled, and most of them have to go back to sleep > > again. > > > What about processes of different priorities blocking for the same mutex? > Would you do a linear search on the queue? or have the queue sorted by > priority? or a FIFO queue is good enough? Processes that block on a mutex are granted the lock in FIFO order, rather than priority order. In order to avoid priority inversion, the mutex wait queue implements priority lending. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jun 26 12:26:58 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id B242237B942 for ; Mon, 26 Jun 2000 12:26:53 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id MAA29301; Mon, 26 Jun 2000 12:26:50 -0700 (PDT) (envelope-from dillon) Date: Mon, 26 Jun 2000 12:26:50 -0700 (PDT) From: Matthew Dillon Message-Id: <200006261926.MAA29301@apollo.backplane.com> To: Luoqi Chen Cc: jasone@canonware.com, smp@FreeBSD.ORG Subject: Re: SMP meeting summary References: <200006261646.e5QGkUS06290@lor.watermarkgroup.com> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org :> Compared with the use of tsleep(), mutexes have a number of :> advantages: :> :> - Each mutex has its own wait (sleep) queue. When a process releases :> a mutex, it automatically schedules the next process waiting on the :> queue. This is more efficient than searching a possibly very long, :> linear sleep queue. It also avoids the flooding when multiple :> processes get scheduled, and most of them have to go back to sleep :> again. :> :What about processes of different priorities blocking for the same mutex? :Would you do a linear search on the queue? or have the queue sorted by :priority? or a FIFO queue is good enough? : :-lq I've found the a sorted queue is best for this sort of thing. It is possible to do a low-overhead sorted queue by having the insertion code scan from the beginning forward AND the end backwards until it finds the insertion point. Usually it finds the insertion point in only one or two iterations. The wakeup code then simply pops the first proc from the queue and wakes it up. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jun 26 12:49: 2 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 6EC4737B691 for ; Mon, 26 Jun 2000 12:49:00 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id MAA29502; Mon, 26 Jun 2000 12:49:00 -0700 (PDT) (envelope-from dillon) Date: Mon, 26 Jun 2000 12:49:00 -0700 (PDT) From: Matthew Dillon Message-Id: <200006261949.MAA29502@apollo.backplane.com> To: Matthew Dillon Cc: Luoqi Chen , jasone@canonware.com, smp@FreeBSD.ORG Subject: Re: SMP meeting summary References: <200006261646.e5QGkUS06290@lor.watermarkgroup.com> <200006261926.MAA29301@apollo.backplane.com> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org ::priority? or a FIFO queue is good enough? :: ::-lq : : I've found the a sorted queue is best for this sort of thing. It : is possible to do a low-overhead sorted queue by having the insertion : code scan from the beginning forward AND the end backwards until it : finds the insertion point. Usually it finds the insertion point in only : one or two iterations. : : The wakeup code then simply pops the first proc from the queue and wakes : it up. Scrap my last comment, I must be high. Jason's explanation is the right one (priority lending). -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jun 26 12:57: 5 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mail.enteract.com (mail.enteract.com [207.229.143.33]) by hub.freebsd.org (Postfix) with ESMTP id 6EA1A37B691 for ; Mon, 26 Jun 2000 12:56:51 -0700 (PDT) (envelope-from jrs@enteract.com) Received: from shell-2.enteract.com (jrs@shell-2.enteract.com [207.229.143.41]) by mail.enteract.com (8.9.3/8.9.3) with SMTP id OAA11900; Mon, 26 Jun 2000 14:56:30 -0500 (CDT) (envelope-from jrs@enteract.com) Date: Mon, 26 Jun 2000 14:56:30 -0500 (CDT) From: John Sconiers To: Matthew Dillon Cc: Luoqi Chen , jasone@canonware.com, smp@FreeBSD.ORG Subject: Re: SMP meeting summary In-Reply-To: <200006261949.MAA29502@apollo.backplane.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Quick question. I've been reading this as well as past threads and understand the merit and functionality of the new SMP model. However, how much (if any) will Performance on both single and multi cpu machines improve after this change?? Will the new systme use up more resources(ie memory, swap, cpu) in single and multi CPU configurations?? I apologize if this has already been brought up. JRS > : I've found the a sorted queue is best for this sort of thing. It > : is possible to do a low-overhead sorted queue by having the insertion > : code scan from the beginning forward AND the end backwards until it > : finds the insertion point. Usually it finds the insertion point in only > : one or two iterations. > : The wakeup code then simply pops the first proc from the queue and wakes > : it up. > Scrap my last comment, I must be high. > Jason's explanation is the right one (priority lending). To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jun 26 13: 8:25 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id E7D5237BC26 for ; Mon, 26 Jun 2000 13:08:14 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id NAA29627; Mon, 26 Jun 2000 13:08:00 -0700 (PDT) (envelope-from dillon) Date: Mon, 26 Jun 2000 13:08:00 -0700 (PDT) From: Matthew Dillon Message-Id: <200006262008.NAA29627@apollo.backplane.com> To: John Sconiers Cc: Luoqi Chen , jasone@canonware.com, smp@FreeBSD.ORG Subject: Re: SMP meeting summary References: Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org : : :Quick question. I've been reading this as well as past threads and :understand the merit and functionality of the new SMP model. However, how :much (if any) will Performance on both single and multi cpu machines :improve after this change?? Will the new systme use up more resources(ie :memory, swap, cpu) in single and multi CPU configurations?? I apologize :if this has already been brought up. : :JRS Resource utilization will not change much. Single cpu machines will be slightly slower (probably unnoticeably). MP machines should be considerably faster under heavy system loads. Under light system loads you will probably not notice any difference. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jun 26 13:14: 8 2000 Delivered-To: freebsd-smp@freebsd.org Received: from lor.watermarkgroup.com (lor.watermarkgroup.com [207.202.73.33]) by hub.freebsd.org (Postfix) with ESMTP id 57A5E37BD95 for ; Mon, 26 Jun 2000 13:13:56 -0700 (PDT) (envelope-from luoqi@watermarkgroup.com) Received: (from luoqi@localhost) by lor.watermarkgroup.com (8.10.1/8.10.1) id e5QKDOP09679; Mon, 26 Jun 2000 16:13:24 -0400 (EDT) Date: Mon, 26 Jun 2000 16:13:24 -0400 (EDT) From: Luoqi Chen Message-Id: <200006262013.e5QKDOP09679@lor.watermarkgroup.com> To: jasone@canonware.com Subject: Re: SMP meeting summary Cc: smp@FreeBSD.ORG Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > On Mon, Jun 26, 2000 at 12:46:30PM -0400, Luoqi Chen wrote: > > > Compared with the use of tsleep(), mutexes have a number of > > > advantages: > > > > > > - Each mutex has its own wait (sleep) queue. When a process releases > > > a mutex, it automatically schedules the next process waiting on the > > > queue. This is more efficient than searching a possibly very long, > > > linear sleep queue. It also avoids the flooding when multiple > > > processes get scheduled, and most of them have to go back to sleep > > > again. > > > > > What about processes of different priorities blocking for the same mutex? > > Would you do a linear search on the queue? or have the queue sorted by > > priority? or a FIFO queue is good enough? > > Processes that block on a mutex are granted the lock in FIFO order, rather > than priority order. In order to avoid priority inversion, the mutex wait > queue implements priority lending. > > Jason > Ok. I remember I have read somewhere that solaris 7 has given up the behavior of waking up only one thread after a mutex is released, now it wakes up all the blocking threads. It seems that the "thundering herd" problem is not serious after all if the lock granuity is high enough. -lq To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jun 26 13:22:49 2000 Delivered-To: freebsd-smp@freebsd.org Received: from anchor-post-30.mail.demon.net (anchor-post-30.mail.demon.net [194.217.242.88]) by hub.freebsd.org (Postfix) with ESMTP id 6691937BC4B for ; Mon, 26 Jun 2000 13:22:41 -0700 (PDT) (envelope-from dfr@nlsystems.com) Received: from nlsys.demon.co.uk ([158.152.125.33] helo=herring.nlsystems.com) by anchor-post-30.mail.demon.net with esmtp (Exim 2.12 #1) id 136fOk-000CjP-0U; Mon, 26 Jun 2000 21:22:26 +0100 Received: from salmon.nlsystems.com (salmon.nlsystems.com [10.0.0.3]) by herring.nlsystems.com (8.9.3/8.8.8) with ESMTP id VAA97005; Mon, 26 Jun 2000 21:22:37 +0100 (BST) (envelope-from dfr@nlsystems.com) Date: Mon, 26 Jun 2000 21:26:33 +0100 (BST) From: Doug Rabson To: Luoqi Chen Cc: jasone@canonware.com, smp@freebsd.org Subject: Re: SMP meeting summary In-Reply-To: <200006261646.e5QGkUS06290@lor.watermarkgroup.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Mon, 26 Jun 2000, Luoqi Chen wrote: > > Compared with the use of tsleep(), mutexes have a number of > > advantages: > > > > - Each mutex has its own wait (sleep) queue. When a process releases > > a mutex, it automatically schedules the next process waiting on the > > queue. This is more efficient than searching a possibly very long, > > linear sleep queue. It also avoids the flooding when multiple > > processes get scheduled, and most of them have to go back to sleep > > again. > > > What about processes of different priorities blocking for the same mutex? > Would you do a linear search on the queue? or have the queue sorted by > priority? or a FIFO queue is good enough? The BSD/OS mutex has the queue sorted by priority. -- Doug Rabson Mail: dfr@nlsystems.com Nonlinear Systems Ltd. Phone: +44 20 8442 9037 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jun 26 14:56:58 2000 Delivered-To: freebsd-smp@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id A8A1C37BBDE for ; Mon, 26 Jun 2000 14:56:51 -0700 (PDT) (envelope-from jasone@magnesium.net) Received: (qmail 73643 invoked by uid 1142); 26 Jun 2000 21:56:49 -0000 Date: 26 Jun 2000 14:56:49 -0700 Date: Mon, 26 Jun 2000 14:49:57 -0700 From: Jason Evans To: Luoqi Chen Cc: smp@FreeBSD.ORG Subject: Re: SMP meeting summary Message-ID: <20000626144957.J8965@blitz.canonware.com> References: <200006262013.e5QKDOP09679@lor.watermarkgroup.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <200006262013.e5QKDOP09679@lor.watermarkgroup.com>; from luoqi@watermarkgroup.com on Mon, Jun 26, 2000 at 04:13:24PM -0400 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Mon, Jun 26, 2000 at 04:13:24PM -0400, Luoqi Chen wrote: > > Processes that block on a mutex are granted the lock in FIFO order, rather > > than priority order. In order to avoid priority inversion, the mutex wait > > queue implements priority lending. > > > Ok. I remember I have read somewhere that solaris 7 has given up the behavior > of waking up only one thread after a mutex is released, now it wakes up all > the blocking threads. It seems that the "thundering herd" problem is not > serious after all if the lock granuity is high enough. I don't think this is the case. Solaris uses what are called turnstiles to implement priority lending. For a reasonably detailed explanation, see: http://www.sunworld.com/sunworldonline/swol-08-1999/swol-08-insidesolaris.html My reading of this article is that turnstiles use priority lending to boost the current owner(s) of a lock, but that subsequent lock granting is done in priority order. This lock granting behavior isn't strictly necessary, but it may have desireable characteristics. I haven't looked at the BSD/OS code in detail yet, but according to Doug Rabson, it behaves in basically the same way. Also, there is a book due out within the next several weeks that contains a lot of good information about the Solaris kernel: Solaris Internals: Architecture and Techniques Vol. 1 Core Kernel Components by Jim Mauro, Richard McDougall ISBN: 0-13-022496-0 Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jun 26 15:14:56 2000 Delivered-To: freebsd-smp@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 2AAE537B906 for ; Mon, 26 Jun 2000 15:14:52 -0700 (PDT) (envelope-from jasone@magnesium.net) Received: (qmail 74138 invoked by uid 1142); 26 Jun 2000 22:14:51 -0000 Date: 26 Jun 2000 15:14:51 -0700 Date: Mon, 26 Jun 2000 15:14:41 -0700 From: Jason Evans To: Luoqi Chen Cc: smp@FreeBSD.ORG Subject: Re: SMP meeting summary Message-ID: <20000626151441.L8965@blitz.canonware.com> References: <200006262013.e5QKDOP09679@lor.watermarkgroup.com> <20000626144957.J8965@blitz.canonware.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <20000626144957.J8965@blitz.canonware.com>; from jasone@canonware.com on Mon, Jun 26, 2000 at 02:49:57PM -0700 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Mon, Jun 26, 2000 at 02:49:57PM -0700, Jason Evans wrote: > On Mon, Jun 26, 2000 at 04:13:24PM -0400, Luoqi Chen wrote: > > > Processes that block on a mutex are granted the lock in FIFO order, rather > > > than priority order. In order to avoid priority inversion, the mutex wait > > > queue implements priority lending. > > > > > Ok. I remember I have read somewhere that solaris 7 has given up the behavior > > of waking up only one thread after a mutex is released, now it wakes up all > > the blocking threads. It seems that the "thundering herd" problem is not > > serious after all if the lock granuity is high enough. > > I don't think this is the case. Whoops. The article is broken into two web pages, and the second page states exactly what you said: as of Solaris 7, all waiting threads are woken up. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jun 26 17: 0:30 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pcnet1.pcnet.com (pcnet1.pcnet.com [204.213.232.3]) by hub.freebsd.org (Postfix) with ESMTP id 961F637BA93 for ; Mon, 26 Jun 2000 17:00:27 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: (from eischen@localhost) by pcnet1.pcnet.com (8.8.7/PCNet) id UAA17781; Mon, 26 Jun 2000 20:00:09 -0400 (EDT) Date: Mon, 26 Jun 2000 20:00:09 -0400 (EDT) From: Daniel Eischen To: Jason Evans Cc: Luoqi Chen , smp@FreeBSD.ORG Subject: Re: SMP meeting summary In-Reply-To: <20000626151441.L8965@blitz.canonware.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On 26 Jun 2000, Jason Evans wrote: > On Mon, Jun 26, 2000 at 02:49:57PM -0700, Jason Evans wrote: > > On Mon, Jun 26, 2000 at 04:13:24PM -0400, Luoqi Chen wrote: > > > > Processes that block on a mutex are granted the lock in FIFO order, rather > > > > than priority order. In order to avoid priority inversion, the mutex wait > > > > queue implements priority lending. > > > > > > > Ok. I remember I have read somewhere that solaris 7 has given up the behavior > > > of waking up only one thread after a mutex is released, now it wakes up all > > > the blocking threads. It seems that the "thundering herd" problem is not > > > serious after all if the lock granuity is high enough. > > > > I don't think this is the case. > > Whoops. The article is broken into two web pages, and the second page > states exactly what you said: as of Solaris 7, all waiting threads are > woken up. Yes, this confirms what Jim Mauro said in the Solaris Internals course at USENIX. Since mutexes are held only for very small amounts of time and the kernel is sufficiently fine-grained, their was no advantage to calling wake_one() as opposed to wake_all(). Obviously with these semantics, the waiter with the highest priority should obtain the mutex. At least that was my recollection... In regards to turnstiles, each kernel thread is born with its own turnstile. When it blocks on a mutex that doesn't have any waiters (no turnstile allocated to it), it uses the threads turnstile. If the mutex already has a turnstile (there are other waiters), then the threads turnstile is added to the system (per-CPU?) pool of turnstiles. When the thread wakes up and acquires the mutex, it takes a turnstile back from the turnstile pool. Turnstiles are also used for read/write locks. -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jun 26 19:22:44 2000 Delivered-To: freebsd-smp@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 3A35737BDC8 for ; Mon, 26 Jun 2000 19:22:41 -0700 (PDT) (envelope-from jasone@magnesium.net) Received: (qmail 80746 invoked by uid 1142); 27 Jun 2000 02:22:36 -0000 Date: 26 Jun 2000 19:22:36 -0700 Date: Mon, 26 Jun 2000 19:21:56 -0700 From: Jason Evans To: smp@freebsd.org Subject: SMP project web page Message-ID: <20000626192155.C15267@blitz.canonware.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org I've created a web page that will serve as the central location for SMP project status and news. Project discussion will remain in the -smp mailing list, but important milestones, decisions, links, etc. will also be reflected on the web page. Throughout the project, please feel free to contact me with regard to changes that need to be made to the web page. http://people.freebsd.org/~jasone/smp/ Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jun 26 19:41:41 2000 Delivered-To: freebsd-smp@freebsd.org Received: from lor.watermarkgroup.com (lor.watermarkgroup.com [207.202.73.33]) by hub.freebsd.org (Postfix) with ESMTP id 989D737BDB8 for ; Mon, 26 Jun 2000 19:41:37 -0700 (PDT) (envelope-from luoqi@watermarkgroup.com) Received: (from luoqi@localhost) by lor.watermarkgroup.com (8.10.1/8.10.1) id e5R2Exu13573; Mon, 26 Jun 2000 22:14:59 -0400 (EDT) Date: Mon, 26 Jun 2000 22:14:59 -0400 (EDT) From: Luoqi Chen Message-Id: <200006270214.e5R2Exu13573@lor.watermarkgroup.com> To: eischen@vigrid.com Subject: Re: SMP meeting summary Cc: smp@FreeBSD.ORG Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > In regards to turnstiles, each kernel thread is born with its own > turnstile. When it blocks on a mutex that doesn't have any waiters > (no turnstile allocated to it), it uses the threads turnstile. If > the mutex already has a turnstile (there are other waiters), then > the threads turnstile is added to the system (per-CPU?) pool of > turnstiles. When the thread wakes up and acquires the mutex, it > takes a turnstile back from the turnstile pool. Turnstiles are > also used for read/write locks. > > -- > Dan Eischen > Does anyone know why a turnstile structure is used, instead of a sleep queue embedded in the mutex structure? With cache line size of 16/32 bytes, the latter seems to be more advantageous. -lq To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jun 26 19:55:41 2000 Delivered-To: freebsd-smp@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 78DF437BDC7 for ; Mon, 26 Jun 2000 19:55:36 -0700 (PDT) (envelope-from jasone@magnesium.net) Received: (qmail 81511 invoked by uid 1142); 27 Jun 2000 02:55:34 -0000 Date: 26 Jun 2000 19:55:34 -0700 Date: Mon, 26 Jun 2000 19:55:26 -0700 From: Jason Evans To: Luoqi Chen Cc: eischen@vigrid.com, smp@FreeBSD.ORG Subject: Re: SMP meeting summary Message-ID: <20000626195526.D15267@blitz.canonware.com> References: <200006270214.e5R2Exu13573@lor.watermarkgroup.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <200006270214.e5R2Exu13573@lor.watermarkgroup.com>; from luoqi@watermarkgroup.com on Mon, Jun 26, 2000 at 10:14:59PM -0400 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Mon, Jun 26, 2000 at 10:14:59PM -0400, Luoqi Chen wrote: > > In regards to turnstiles, each kernel thread is born with its own > > turnstile. When it blocks on a mutex that doesn't have any waiters > > (no turnstile allocated to it), it uses the threads turnstile. If > > the mutex already has a turnstile (there are other waiters), then > > the threads turnstile is added to the system (per-CPU?) pool of > > turnstiles. When the thread wakes up and acquires the mutex, it > > takes a turnstile back from the turnstile pool. Turnstiles are > > also used for read/write locks. > > > > -- > > Dan Eischen > > > Does anyone know why a turnstile structure is used, instead of a sleep > queue embedded in the mutex structure? With cache line size of 16/32 > bytes, the latter seems to be more advantageous. There only needs to be one turnstile per entity (process/thread) that can block. Putting one in every mutex would actually require much more space. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Mon Jun 26 20: 9:44 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mailhost.iprg.nokia.com (mailhost.iprg.nokia.com [205.226.5.12]) by hub.freebsd.org (Postfix) with ESMTP id 0902237BDF4 for ; Mon, 26 Jun 2000 20:09:40 -0700 (PDT) (envelope-from jre@iprg.nokia.com) Received: from darkstar.iprg.nokia.com (darkstar.iprg.nokia.com [205.226.5.69]) by mailhost.iprg.nokia.com (8.9.3/8.9.3-GLGS) with ESMTP id UAA20838; Mon, 26 Jun 2000 20:09:31 -0700 (PDT) Received: (from root@localhost) by darkstar.iprg.nokia.com (8.9.3/8.9.3-VIRSCAN) id UAA09929; Mon, 26 Jun 2000 20:09:29 -0700 X-Virus-Scanned: Mon, 26 Jun 2000 20:09:29 -0700 Nokia Silicon Valley Email Exploit Scanner Received: from (radio.iprg.nokia.com [205.226.1.150]) by darkstar.iprg.nokia.com SMTP/WTS (12.69) xma009794; Mon, 26 Jun 00 20:09:24 -0700 Message-ID: <39581AE5.C157F592@iprg.nokia.com> Date: Mon, 26 Jun 2000 20:09:25 -0700 From: Joe Eykholt Organization: Nokia IPRG X-Mailer: Mozilla 4.7 [en] (X11; I; FreeBSD 2.2.6-RELEASE i386) X-Accept-Language: en MIME-Version: 1.0 To: Luoqi Chen Cc: eischen@vigrid.com, smp@FreeBSD.ORG Subject: Re: SMP meeting summary References: <200006270214.e5R2Exu13573@lor.watermarkgroup.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Luoqi Chen wrote: > > > In regards to turnstiles, each kernel thread is born with its own > > turnstile. When it blocks on a mutex that doesn't have any waiters > > (no turnstile allocated to it), it uses the threads turnstile. If > > the mutex already has a turnstile (there are other waiters), then > > the threads turnstile is added to the system (per-CPU?) pool of > > turnstiles. When the thread wakes up and acquires the mutex, it > > takes a turnstile back from the turnstile pool. Turnstiles are > > also used for read/write locks. > > > > -- > > Dan Eischen > > > Does anyone know why a turnstile structure is used, instead of a sleep > queue embedded in the mutex structure? With cache line size of 16/32 > bytes, the latter seems to be more advantageous. Its basically for lock size, to keep the locks really small so each lock inside other structures wouldn't get too large. The cache line size doesn't matter so much as the size of the other structures that contain locks. But, if you make structures large (by including large locks in them) they won't fit in cache lines so well. In a fine-grained implementation, lots of small structures need locking. You only need turnstiles for the locks that are being waited on, but the turnstile can't be in the thread (or process) that's doing the waiting. There are a lot more mutexes and r/w locks than there are potential waiters, so its a big waste of space to have all the data per lock. The main thing is to not block on mutexes very frequently. The cost of blocking is secondary if the frequency is low enough. There are a lot of considerations and a lot of trial-and-error involved in getting a balanced (not perfect) implementation. Joe Eykholt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 27 6:35:27 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mirage.nlink.com.br (mirage.nlink.com.br [200.249.195.3]) by hub.freebsd.org (Postfix) with SMTP id 1BD6A37C1BD for ; Tue, 27 Jun 2000 06:34:53 -0700 (PDT) (envelope-from rmariz@nlink.com.br) Received: (qmail 87228 invoked by uid 65534); 27 Jun 2000 13:34:48 -0000 To: Subject: Compaq Proliant ML350 SMP problem Message-ID: <962112888.3958ad781b4ba@www.nlink.com.br> Date: Tue, 27 Jun 2000 10:34:48 -0300 From: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit User-Agent: IMP/PHP IMAP webmail program 2.2.0-pre12 X-Originating-IP: 200.249.140.164 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi all, I am in trouble with a Compaq Proliant ML 350 dual processor machine. It didn't work in SMP mode with FreeBSD. After recompiling the FreeBSD 4.0 kernel to support SMP, and rebooting, it simply stop (halt). I tried installing Linux RedHat 6.2 and it works well with SMP. But when I inspected the DMESG output comand, I saw two lines which could be the problem to FreeBSD (I don't know): >mtrr: your CPUs had inconsistent fixed MTRR settings >mtrr: probably your BIOS does not setup all CPUs The BIOS setup of the machine has not mention regard to MTRR. This machine has a bridged PCI card. I transcript bellow the entire DMESG output of the linux an the output console from the FreeBSD boot. Any suggestions are welcome. Thanks in advance, Renato Mariz. -------------------------------------------------------------------- Linux DMESG output: .4376 MHz. Booting processor 0 eip 2000 Calibrating delay loop... 727.45 BogoMIPS OK. CPU0: Intel Pentium III (Coppermine) stepping 01 Total of 2 processors activated (1454.90 BogoMIPS). enabling symmetric IO mode... ...done. ENABLING IO-APIC IRQs ...changing IO-APIC physical APIC ID to 8 ...changing IO-APIC physical APIC ID to 3 init IO_APIC IRQs IO-APIC (apicid-pin) 8-0WARNING: ASSIGN_IRQ_VECTOR wrapped back to 52 not connected. ..MP-BIOS bug: 8254 timer not connected to IO-APIC ...trying to set up timer as ExtINT... .. (found pin 0) ... works. number of MP IRQ sources: 40. number of IO-APIC #8 registers: 16. number of IO-APIC #3 registers: 16. testing the IO APIC....................... IO APIC #8...... .... register #00: 08000000 ....... : physical APIC id: 08 .... register #01: 000F0011 ....... : max redirection entries: 000F ....... : IO APIC version: 0011 .... register #02: 00000000 ....... : arbitration: 00 .... IRQ redirection table: NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: 00 001 01 0 0 0 0 0 0 7 51 01 000 00 0 0 0 0 0 1 1 59 02 000 00 0 0 0 0 0 1 1 51 03 000 00 0 0 0 0 0 1 1 61 04 000 00 0 0 0 0 0 1 1 69 05 000 00 0 0 0 0 0 1 1 71 06 000 00 0 0 0 0 0 1 1 79 07 000 00 0 0 0 0 0 1 1 81 08 000 00 0 0 0 0 0 1 1 89 09 0FF 0F 1 1 0 1 0 1 1 91 0a 0FF 0F 1 1 0 1 0 1 1 99 0b 0FF 0F 1 1 0 1 0 1 1 A1 0c 000 00 0 0 0 0 0 1 1 A9 0d 000 00 1 0 0 0 0 0 0 00 0e 000 00 0 0 0 0 0 1 1 B1 0f 0FF 0F 1 1 0 1 0 1 1 B9 IO APIC #3...... .... register #00: 03000000 ....... : physical APIC id: 03 .... register #01: 000F0011 ....... : max redirection entries: 000F ....... : IO APIC version: 0011 .... register #02: 0D000000 ....... : arbitration: 0D .... IRQ redirection table: NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: 00 0FF 0F 1 1 0 1 0 1 1 C1 01 0FF 0F 1 1 0 1 0 1 1 C9 02 0FF 0F 1 1 0 1 0 1 1 D1 03 0FF 0F 1 1 0 1 0 1 1 D9 04 0FF 0F 1 1 0 1 0 1 1 E1 05 0FF 0F 1 1 0 1 0 1 1 E9 06 0FF 0F 1 1 0 1 0 1 1 F1 07 0FF 0F 1 1 0 1 0 1 1 F9 08 0FF 0F 1 1 0 1 0 1 1 52 09 0FF 0F 1 1 0 1 0 1 1 5A 0a 0FF 0F 1 1 0 1 0 1 1 62 0b 0FF 0F 1 1 0 1 0 1 1 6A 0c 0FF 0F 1 1 0 1 0 1 1 72 0d 0FF 0F 1 1 0 1 0 1 1 7A 0e 0FF 0F 1 1 0 1 0 1 1 82 0f 0FF 0F 1 1 0 1 0 1 1 8A IRQ to pin mappings: IRQ0 -> 2 IRQ1 -> 1 IRQ3 -> 3 IRQ4 -> 4 IRQ5 -> 5 IRQ6 -> 6 IRQ7 -> 7 IRQ8 -> 8 IRQ9 -> 9 IRQ10 -> 10 IRQ11 -> 11 IRQ12 -> 12 IRQ13 -> 13 IRQ14 -> 14 IRQ15 -> 15 IRQ16 -> 0 IRQ17 -> 1 IRQ18 -> 2 IRQ19 -> 3 IRQ20 -> 4 IRQ21 -> 5 IRQ22 -> 6 IRQ23 -> 7 IRQ24 -> 8 IRQ25 -> 9 IRQ26 -> 10 IRQ27 -> 11 IRQ28 -> 12 IRQ29 -> 13 IRQ30 -> 14 IRQ31 -> 15 .................................... done. checking TSC synchronization across CPUs: passed. mtrr: your CPUs had inconsistent fixed MTRR settings mtrr: probably your BIOS does not setup all CPUs PCI: PCI BIOS revision 2.10 entry at 0xeda3c PCI: Using configuration type 1 PCI: Probing PCI hardware PCI: 00:00 [1166/0009]: Scanning peer host bridges PCI: Scanning RCC HE/LE Peer Bus Bridge 00/00 PCI: 00:01 [1166/0009]: Scanning peer host bridges PCI: Scanning RCC HE/LE Peer Bus Bridge 00/01 PCI->APIC IRQ transform: (B1,I4,P0) -> 16 PCI->APIC IRQ transform: (B1,I4,P0) -> 16 PCI->APIC IRQ transform: (B1,I5,P0) -> 17 PCI->APIC IRQ transform: (B1,I7,P0) -> 19 Linux NET4.0 for Linux 2.2 Based upon Swansea University Computer Society NET3.039 NET4: Unix domain sockets 1.0 for Linux NET4.0. NET4: Linux TCP/IP 1.0 for NET4.0 IP Protocols: ICMP, UDP, TCP, IGMP TCP: Hash tables configured (ehash 262144 bhash 65536) Initializing RT netlink socket Starting kswapd v 1.5 Detected PS/2 Mouse Port. Serial driver version 4.27 with MANY_PORTS MULTIPORT SHARE_IRQ enabled ttyS00 at 0x03f8 (irq = 4) is a 16550A ttyS01 at 0x02f8 (irq = 3) is a 16550A pty: 256 Unix98 ptys configured Real Time Clock Driver v1.09 RAM disk driver initialized: 16 RAM disks of 4096K size PCI_IDE: unknown IDE controller on PCI bus 00 device 79, VID=1166, DID=0211 PCI_IDE: not 100% native mode: will probe irqs later ide0: BM-DMA at 0x3000-0x3007, BIOS settings: hda:DMA, hdb:pio ide1: BM-DMA at 0x3008-0x300f, BIOS settings: hdc:pio, hdd:pio hda: COMPAQ CDR-8435, ATAPI CDROM drive ide0 at 0x1f0-0x1f7,0x3f6 on irq 14 hda: ATAPI 32X CD-ROM drive, 128kB Cache Uniform CDROM driver Revision: 2.56 Floppy drive(s): fd0 is 1.44M FDC 0 is a National Semiconductor PC87306 md driver 0.90.0 MAX_MD_DEVS=256, MAX_REAL=12 raid5: measuring checksumming speed raid5: KNI detected, trying cache-avoiding KNI checksum routine pIII_kni : 1310.259 MB/sec raid5: MMX detected, trying high-speed MMX checksum routines pII_mmx : 1588.770 MB/sec p5_mmx : 1680.972 MB/sec 8regs : 1253.490 MB/sec 32regs : 686.562 MB/sec using fastest function: pIII_kni (1310.259 MB/sec) scsi : 0 hosts. scsi : detected total. md.c: sizeof(mdp_super_t) = 4096 Partition check: RAMDISK: Compressed image found at block 0 autodetecting RAID arrays autorun ... ... autorun DONE. VFS: Mounted root (ext2 filesystem). ncr53c8xx: at PCI bus 1, device 4, function 0 ncr53c8xx: 53c896 detected ncr53c8xx: at PCI bus 1, device 4, function 1 ncr53c8xx: 53c896 detected ncr53c896-0: rev=0x05, base=0xb0200000, io_port=0x1000, irq=16 ncr53c896-0: ID 7, Fast-40, Parity Checking ncr53c896-0: on-chip RAM at 0xb0000000 ncr53c896-0: restart (scsi reset). ncr53c896-0: Downloading SCSI SCRIPTS. ncr53c896-1: rev=0x05, base=0xb0300000, io_port=0x1400, irq=16 ncr53c896-1: ID 7, Fast-40, Parity Checking ncr53c896-1: on-chip RAM at 0xb0100000 ncr53c896-1: restart (scsi reset). ncr53c896-1: Downloading SCSI SCRIPTS. scsi0 : ncr53c8xx - version 3.2a-2 scsi1 : ncr53c8xx - version 3.2a-2 scsi : 2 hosts. Vendor: COMPAQ Model: BB00921B91 Rev: 3B07 Type: Direct-Access ANSI SCSI revision: 02 Detected scsi disk sda at scsi0, channel 0, id 0, lun 0 ncr53c896-0-<0,0>: tagged command queue depth set to 8 ncr53c896-0-<0,*>: FAST-20 WIDE SCSI 40.0 MB/s (50 ns, offset 15) SCSI device sda: hdwr sector= 512 bytes. Sectors= 17773524 [8678 MB] [8.7 GB] sda: sda1 sda2 < sda5 sda6 sda7 > sda3 autodetecting RAID arrays autorun ... ... autorun DONE. VFS: Mounted root (ext2 filesystem) readonly. change_root: old root has d_count=1 Trying to unmount old root ... okay Freeing unused kernel memory: 72k freed Adding Swap: 530104k swap-space (priority -1) eepro100.c:v1.09j-t 9/29/99 Donald Becker http://cesdis.gsfc.nasa.gov/linux/drivers/eepro100.html eepro100.c: $Revision: 1.18 $ 1999/12/29 Modified by Andrey V. Savochkin eth0: OEM i82557/i82558 10/100 Ethernet at 0xd0868000, 00:50:8B:9A:42:12, IRQ 17. Receiver lock-up bug exists -- enabling work-around. Board assembly 010101-034, Physical connectors present: RJ45 Primary interface chip i82555 PHY #1. General self-test: passed. Serial sub-system self-test: passed. Internal registers self-test: passed. ROM checksum self-test: passed (0x04f4518b). ------------------------------------------------------------------------------------- Output from console FreeBSD boot: Booting [kernel]... Copyright (c) 1992-2000 The FreeBSD Project. Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. FreeBSD 4.0-RELEASE #38: Wed Jun 21 15:32:12 GMT 2000 root@:/usr/src/sys/compile/CACHE Timecounter "i8254" frequency 1193182 Hz CPU: Pentium III/Pentium III Xeon (728.44-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x681 Stepping = 1 Features=0x383fbff real memory = 134217728 (131072K bytes) avail memory = 126832640 (123860K bytes) Programming 16 pins in IOAPIC #0 IOAPIC #0 intpin 2 -> irq 0 Programming 16 pins in IOAPIC #1 IOAPIC #1 intpin 0 -> irq 16 IOAPIC #1 intpin 1 -> irq 2 IOAPIC #1 intpin 2 -> irq 18 IOAPIC #1 intpin 3 -> irq 17 IOAPIC #1 intpin 4 -> irq 19 IOAPIC #1 intpin 5 -> irq 20 IOAPIC #1 intpin 6 -> irq 21 IOAPIC #1 intpin 7 -> irq 22 IOAPIC #1 intpin 8 -> irq 23 FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 1, version: 0x00040011, at 0xfee00000 cpu1 (AP): apic id: 0, version: 0x00040011, at 0xfee00000 io0 (APIC): apic id: 8, version: 0x000f0011, at 0xfec00000 io1 (APIC): apic id: 3, version: 0x000f0011, at 0xfec01000 Preloaded elf kernel "kernel" at 0xc0335000. Preloaded userconfig_script "/boot/kernel.conf" at 0xc033509c. Pentium Pro MTRR support enabled md0: Malloc disk npx0: on motherboard npx0: INT 16 interface pcib0: on motherboard pci0: on pcib0 pcib6: at device 1.0 on pci0 pci1: on pcib6 sym0: <896> port 0x1000-0x10ff mem 0xb0000000-0xb0001fff,0xb0200000-0xb02003ff i rq 16 at device 4.0 on pci1 sym0: No NVRAM, ID 7, Fast-40, LVD, parity checking sym1: <896> port 0x1400-0x14ff mem 0xb0100000-0xb0101fff,0xb0300000-0xb03003ff i rq 16 at device 4.1 on pci1 sym1: No NVRAM, ID 7, Fast-40, SE, parity checking pci1: (vendor=0x8086, dev=0x1229) at 5.0 irq 2 pci1: at 6.0 pci1: (vendor=0x0e11, dev=0xa0f0) at 7.0 irq 17 isab0: at device 15.0 on pci0 isa0: on isab0 atapci0: port 0x3000-0x300f at devic e 15.1 on pci0 ata0: at 0x1f0 irq 14 on atapci0 pcib5: on motherboard pci5: on pcib5 fdc0: at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 fdc0: FIFO enabled, 8 bytes threshold fd0: <1440-KB 3.5" drive> on fdc0 drive 0 sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 sio0: type 16550A, console sio1 at port 0x2f8-0x2ff irq 3 on isa0 sio1: type 16550A To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 27 6:42:48 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mirage.nlink.com.br (mirage.nlink.com.br [200.249.195.3]) by hub.freebsd.org (Postfix) with SMTP id D0E0A37B89A for ; Tue, 27 Jun 2000 06:42:38 -0700 (PDT) (envelope-from rmariz@nlink.com.br) Received: (qmail 87945 invoked by uid 65534); 27 Jun 2000 13:42:32 -0000 To: Subject: Re: SMP kernel problem in compaq proliant 6000 Message-ID: <962113352.3958af480e5dd@www.nlink.com.br> Date: Tue, 27 Jun 2000 10:42:32 -0300 From: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit User-Agent: IMP/PHP IMAP webmail program 2.2.0-pre12 X-Originating-IP: 200.249.140.164 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi, I have the same problem with a proliant ML350 with 2 Pentium III. I have just sent an e-mail to this list explaining my problem. Can anyone help us? Renato Mariz. Shadow wrote: > > I have a similar problem on a 6500 with 4 Xeons. Everything runs fine > with a single processor kernel, but mounting / hangs with SMP enabled. > It's also running the ida raid driver. I will give more details on monday > if anyone is intrested, not anywhere near the computer right now. > > -Nick > Sr. Systems Administrator, Global Telecom Inc. > shadow@gti.net > > On Fri, 23 Jun 2000, NandaKumar P.K. wrote: > > > Hi, > > > > I am trying to configure a SMP kernel (FreeBSD 3.4) in > > a Compaq proliant 6000 model machine with 2 Pentium II > > Xeon Processor.The kernel crashes while booting saying > > APIC RTC !=8. The options which are enabled in the > > configuration file as follows. > > > > options SMP > > options APIC_IO > > > > options NCPU=2 > > options NBUS=4 > > options NAPIC=1 > > options NINTR=61 > > > > mptable gives the same values as above. > > I tried with NAPIC=8,but it doesn't work. I am not > > able to get any motherboard model no. > > > > Any help will be deeply appreciated > > > > Regards, > > Nandan > > > > > > __________________________________________________ > > Do You Yahoo!? > > Get Yahoo! Mail - Free email you can access from anywhere! > > http://mail.yahoo.com/ > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > with "unsubscribe freebsd-smp" in the body of the message > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message -- "To reboot type win" To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 27 10:42:40 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id BC8C337C26B for ; Tue, 27 Jun 2000 10:42:34 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id KAA35851; Tue, 27 Jun 2000 10:42:20 -0700 (PDT) (envelope-from dillon) Date: Tue, 27 Jun 2000 10:42:20 -0700 (PDT) From: Matthew Dillon Message-Id: <200006271742.KAA35851@apollo.backplane.com> To: Chuck Paterson Cc: David Greenman , freebsd-smp@freebsd.org Subject: Re: Stepping on Toes References: <200006261650.KAA17801@berserker.bsdi.com> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org :} Even with interrupt threads we have the GiantMutex issue... the same :} issue that we have with our current MP implementation. We cannot remove :} SPL's until we remove the GiantMutex, and we cannot remove GiantMutex :} without major modifications to just about every single source file in sys/ : :In general this isn't true. If you get to the point where : :1) All entrance to unsafe code is proteced by Giant. :2) Tsleep and friend if any release Giant when they : a process is suspended and re-acquire it on exit :3) Interrupts have a context to run in. :4) You have one or more scheduling locks. : :Then you can just turn spls into a nop. There is lots of hand waving :in regards to details at this point. BSD/OS SMPng is the existance :proof. : : :It seems like one of the major problems in retaining spls during :the change over period is that they don't much useful, and effectively :push everything under Giant. : : Grabbing a spl will only block interrupts, it will not give : any protection against an interrupt thread which has already : started. : :This means that any device which might be blocked by splbio() can :not be brought out from under the Giant lock until all instances :of splbio have been removed. : :Chuck Yes, I see it. I agree. You don't even need to hold a scheduling lock... all you need to hold is Giant. #1 - done #2 - done #3 - (Greg) #4 - not required Right this moment the requirement is that only someone holding Giant is allowed to mess with spl*()'s (the cpl variable can only be messed with by people holding Giant). At this moment, without interrupt threads, interrupts can share Giant with the curproc they interrupted. This is how our existing MP stuff worked already. When Greg moves interrupts to their own threads, and obtains Giant to run those interrupts, no more sharing will occur and just the fact that the interrupt is holding Giant guarentees that nobody else will be messing with SPLs, thus the SPLs can be removed entirely. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 27 11:20:25 2000 Delivered-To: freebsd-smp@freebsd.org Received: from berserker.bsdi.com (berserker.twistedbit.com [199.79.183.1]) by hub.freebsd.org (Postfix) with ESMTP id 701EB37B514 for ; Tue, 27 Jun 2000 11:20:21 -0700 (PDT) (envelope-from cp@berserker.bsdi.com) Received: from berserker.bsdi.com (cp@LOCALHOST [127.0.0.1]) by berserker.bsdi.com (8.9.3/8.9.3) with ESMTP id MAA01315; Tue, 27 Jun 2000 12:19:38 -0600 (MDT) Message-Id: <200006271819.MAA01315@berserker.bsdi.com> To: Matthew Dillon Cc: David Greenman , freebsd-smp@freebsd.org Subject: Re: Stepping on Toes From: Chuck Paterson Date: Tue, 27 Jun 2000 12:19:38 -0600 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org } } Yes, I see it. I agree. You don't even need to hold a scheduling } lock... all you need to hold is Giant. Sorry, I wasn't real clear. I didn't mean to imply that it was necessary to hold the scheduling lock in unsafe code. It is necessary to have a scheduling lock to protect the run queues and friends. Chuck To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 27 16:37:43 2000 Delivered-To: freebsd-smp@freebsd.org Received: from exchange.telenordeste.com.br (mail.telenordeste.com.br [200.249.140.251]) by hub.freebsd.org (Postfix) with ESMTP id 0D34737C38A for ; Tue, 27 Jun 2000 16:35:50 -0700 (PDT) (envelope-from rmariz@timnordeste.com.br) Received: from intrape01.tim ([10.80.0.5]) by exchange.telenordeste.com.br with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) id NW3PJ2KN; Mon, 26 Jun 2000 19:46:08 -0300 Received: from timnordeste.com.br ([10.80.72.249]) by intrape01.tim (Lotus Domino Release 5.0.3 (Intl)) with ESMTP id 2000062617373385:140 ; Mon, 26 Jun 2000 17:37:33 -0300 Message-ID: <39579612.2B0D2EDD@timnordeste.com.br> Date: Mon, 26 Jun 2000 17:42:42 +0000 From: Renato Mariz de Moraes X-Mailer: Mozilla 4.7 [en] (X11; I; Linux 2.0.36 i386) X-Accept-Language: en MIME-Version: 1.0 To: freebsd-smp@FreeBSD.ORG, "NandaKumar P.K." , Shadow Subject: Re: SMP kernel problem in compaq proliant 6000 X-MIMETrack: Itemize by SMTP Server on intrape01/Timnordeste(Release 5.0.3 (Intl)|21 March 2000) at 06/26/2000 05:37:33 PM, Serialize by Router on intrape01/Timnordeste(Release 5.0.3 (Intl)|21 March 2000) at 06/26/2000 07:38:17 PM, Serialize complete at 06/26/2000 07:38:17 PM Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi, I have the same problem with a proliant ML350 with 2 Pentium III. I have just sent an e-mail to this list explaining my problem. Can anyone help us? Renato Mariz. Shadow wrote: > > I have a similar problem on a 6500 with 4 Xeons. Everything runs fine > with a single processor kernel, but mounting / hangs with SMP enabled. > It's also running the ida raid driver. I will give more details on monday > if anyone is intrested, not anywhere near the computer right now. > > -Nick > Sr. Systems Administrator, Global Telecom Inc. > shadow@gti.net > > On Fri, 23 Jun 2000, NandaKumar P.K. wrote: > > > Hi, > > > > I am trying to configure a SMP kernel (FreeBSD 3.4) in > > a Compaq proliant 6000 model machine with 2 Pentium II > > Xeon Processor.The kernel crashes while booting saying > > APIC RTC !=8. The options which are enabled in the > > configuration file as follows. > > > > options SMP > > options APIC_IO > > > > options NCPU=2 > > options NBUS=4 > > options NAPIC=1 > > options NINTR=61 > > > > mptable gives the same values as above. > > I tried with NAPIC=8,but it doesn't work. I am not > > able to get any motherboard model no. > > > > Any help will be deeply appreciated > > > > Regards, > > Nandan > > > > > > __________________________________________________ > > Do You Yahoo!? > > Get Yahoo! Mail - Free email you can access from anywhere! > > http://mail.yahoo.com/ > > > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > > with "unsubscribe freebsd-smp" in the body of the message > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 27 16:37:54 2000 Delivered-To: freebsd-smp@freebsd.org Received: from exchange.telenordeste.com.br (mail.telenordeste.com.br [200.249.140.251]) by hub.freebsd.org (Postfix) with ESMTP id 9C7EF37BF48 for ; Tue, 27 Jun 2000 16:35:40 -0700 (PDT) (envelope-from rmariz@timnordeste.com.br) Received: from intrape01.tim ([10.80.0.5]) by exchange.telenordeste.com.br with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) id NW3PJ2KL; Mon, 26 Jun 2000 19:46:08 -0300 Received: from timnordeste.com.br ([10.80.72.249]) by intrape01.tim (Lotus Domino Release 5.0.3 (Intl)) with ESMTP id 2000062617243234:77 ; Mon, 26 Jun 2000 17:24:32 -0300 Message-ID: <39579303.6610050C@timnordeste.com.br> Date: Mon, 26 Jun 2000 17:29:39 +0000 From: Renato Mariz de Moraes X-Mailer: Mozilla 4.7 [en] (X11; I; Linux 2.0.36 i386) X-Accept-Language: en MIME-Version: 1.0 To: freebsd-smp@freebsd.org Subject: Compaq Proliant ML350 SMP problem X-MIMETrack: Itemize by SMTP Server on intrape01/Timnordeste(Release 5.0.3 (Intl)|21 March 2000) at 06/26/2000 05:24:32 PM, Serialize by Router on intrape01/Timnordeste(Release 5.0.3 (Intl)|21 March 2000) at 06/26/2000 07:38:17 PM, Serialize complete at 06/26/2000 07:38:17 PM Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi all, I am in trouble with a Compaq Proliant ML 350 dual processor machine. It didn't work in SMP mode with FreeBSD. After recompiling the FreeBSD 4.0 kernel to support SMP, and rebooting, it simply stop (halt). I tried installing Linux RedHat 6.2 and it works well with SMP. But when I inspected the DMESG output comand, I saw two lines which could be the problem to FreeBSD (I don't know): >mtrr: your CPUs had inconsistent fixed MTRR settings >mtrr: probably your BIOS does not setup all CPUs The BIOS setup of the machine has not mention regard to MTRR. This machine has a bridged PCI card. I transcript bellow the entire DMESG output of the linux an the output console from the FreeBSD boot. Any suggestions are welcome. Thanks in advance, Renato Mariz. -------------------------------------------------------------------- Linux DMESG output: .4376 MHz. Booting processor 0 eip 2000 Calibrating delay loop... 727.45 BogoMIPS OK. CPU0: Intel Pentium III (Coppermine) stepping 01 Total of 2 processors activated (1454.90 BogoMIPS). enabling symmetric IO mode... ...done. ENABLING IO-APIC IRQs ...changing IO-APIC physical APIC ID to 8 ...changing IO-APIC physical APIC ID to 3 init IO_APIC IRQs IO-APIC (apicid-pin) 8-0WARNING: ASSIGN_IRQ_VECTOR wrapped back to 52 not connected. ..MP-BIOS bug: 8254 timer not connected to IO-APIC ...trying to set up timer as ExtINT... .. (found pin 0) ... works. number of MP IRQ sources: 40. number of IO-APIC #8 registers: 16. number of IO-APIC #3 registers: 16. testing the IO APIC....................... IO APIC #8...... .... register #00: 08000000 ....... : physical APIC id: 08 .... register #01: 000F0011 ....... : max redirection entries: 000F ....... : IO APIC version: 0011 .... register #02: 00000000 ....... : arbitration: 00 .... IRQ redirection table: NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: 00 001 01 0 0 0 0 0 0 7 51 01 000 00 0 0 0 0 0 1 1 59 02 000 00 0 0 0 0 0 1 1 51 03 000 00 0 0 0 0 0 1 1 61 04 000 00 0 0 0 0 0 1 1 69 05 000 00 0 0 0 0 0 1 1 71 06 000 00 0 0 0 0 0 1 1 79 07 000 00 0 0 0 0 0 1 1 81 08 000 00 0 0 0 0 0 1 1 89 09 0FF 0F 1 1 0 1 0 1 1 91 0a 0FF 0F 1 1 0 1 0 1 1 99 0b 0FF 0F 1 1 0 1 0 1 1 A1 0c 000 00 0 0 0 0 0 1 1 A9 0d 000 00 1 0 0 0 0 0 0 00 0e 000 00 0 0 0 0 0 1 1 B1 0f 0FF 0F 1 1 0 1 0 1 1 B9 IO APIC #3...... .... register #00: 03000000 ....... : physical APIC id: 03 .... register #01: 000F0011 ....... : max redirection entries: 000F ....... : IO APIC version: 0011 .... register #02: 0D000000 ....... : arbitration: 0D .... IRQ redirection table: NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect: 00 0FF 0F 1 1 0 1 0 1 1 C1 01 0FF 0F 1 1 0 1 0 1 1 C9 02 0FF 0F 1 1 0 1 0 1 1 D1 03 0FF 0F 1 1 0 1 0 1 1 D9 04 0FF 0F 1 1 0 1 0 1 1 E1 05 0FF 0F 1 1 0 1 0 1 1 E9 06 0FF 0F 1 1 0 1 0 1 1 F1 07 0FF 0F 1 1 0 1 0 1 1 F9 08 0FF 0F 1 1 0 1 0 1 1 52 09 0FF 0F 1 1 0 1 0 1 1 5A 0a 0FF 0F 1 1 0 1 0 1 1 62 0b 0FF 0F 1 1 0 1 0 1 1 6A 0c 0FF 0F 1 1 0 1 0 1 1 72 0d 0FF 0F 1 1 0 1 0 1 1 7A 0e 0FF 0F 1 1 0 1 0 1 1 82 0f 0FF 0F 1 1 0 1 0 1 1 8A IRQ to pin mappings: IRQ0 -> 2 IRQ1 -> 1 IRQ3 -> 3 IRQ4 -> 4 IRQ5 -> 5 IRQ6 -> 6 IRQ7 -> 7 IRQ8 -> 8 IRQ9 -> 9 IRQ10 -> 10 IRQ11 -> 11 IRQ12 -> 12 IRQ13 -> 13 IRQ14 -> 14 IRQ15 -> 15 IRQ16 -> 0 IRQ17 -> 1 IRQ18 -> 2 IRQ19 -> 3 IRQ20 -> 4 IRQ21 -> 5 IRQ22 -> 6 IRQ23 -> 7 IRQ24 -> 8 IRQ25 -> 9 IRQ26 -> 10 IRQ27 -> 11 IRQ28 -> 12 IRQ29 -> 13 IRQ30 -> 14 IRQ31 -> 15 .................................... done. checking TSC synchronization across CPUs: passed. mtrr: your CPUs had inconsistent fixed MTRR settings mtrr: probably your BIOS does not setup all CPUs PCI: PCI BIOS revision 2.10 entry at 0xeda3c PCI: Using configuration type 1 PCI: Probing PCI hardware PCI: 00:00 [1166/0009]: Scanning peer host bridges PCI: Scanning RCC HE/LE Peer Bus Bridge 00/00 PCI: 00:01 [1166/0009]: Scanning peer host bridges PCI: Scanning RCC HE/LE Peer Bus Bridge 00/01 PCI->APIC IRQ transform: (B1,I4,P0) -> 16 PCI->APIC IRQ transform: (B1,I4,P0) -> 16 PCI->APIC IRQ transform: (B1,I5,P0) -> 17 PCI->APIC IRQ transform: (B1,I7,P0) -> 19 Linux NET4.0 for Linux 2.2 Based upon Swansea University Computer Society NET3.039 NET4: Unix domain sockets 1.0 for Linux NET4.0. NET4: Linux TCP/IP 1.0 for NET4.0 IP Protocols: ICMP, UDP, TCP, IGMP TCP: Hash tables configured (ehash 262144 bhash 65536) Initializing RT netlink socket Starting kswapd v 1.5 Detected PS/2 Mouse Port. Serial driver version 4.27 with MANY_PORTS MULTIPORT SHARE_IRQ enabled ttyS00 at 0x03f8 (irq = 4) is a 16550A ttyS01 at 0x02f8 (irq = 3) is a 16550A pty: 256 Unix98 ptys configured Real Time Clock Driver v1.09 RAM disk driver initialized: 16 RAM disks of 4096K size PCI_IDE: unknown IDE controller on PCI bus 00 device 79, VID=1166, DID=0211 PCI_IDE: not 100% native mode: will probe irqs later ide0: BM-DMA at 0x3000-0x3007, BIOS settings: hda:DMA, hdb:pio ide1: BM-DMA at 0x3008-0x300f, BIOS settings: hdc:pio, hdd:pio hda: COMPAQ CDR-8435, ATAPI CDROM drive ide0 at 0x1f0-0x1f7,0x3f6 on irq 14 hda: ATAPI 32X CD-ROM drive, 128kB Cache Uniform CDROM driver Revision: 2.56 Floppy drive(s): fd0 is 1.44M FDC 0 is a National Semiconductor PC87306 md driver 0.90.0 MAX_MD_DEVS=256, MAX_REAL=12 raid5: measuring checksumming speed raid5: KNI detected, trying cache-avoiding KNI checksum routine pIII_kni : 1310.259 MB/sec raid5: MMX detected, trying high-speed MMX checksum routines pII_mmx : 1588.770 MB/sec p5_mmx : 1680.972 MB/sec 8regs : 1253.490 MB/sec 32regs : 686.562 MB/sec using fastest function: pIII_kni (1310.259 MB/sec) scsi : 0 hosts. scsi : detected total. md.c: sizeof(mdp_super_t) = 4096 Partition check: RAMDISK: Compressed image found at block 0 autodetecting RAID arrays autorun ... ... autorun DONE. VFS: Mounted root (ext2 filesystem). ncr53c8xx: at PCI bus 1, device 4, function 0 ncr53c8xx: 53c896 detected ncr53c8xx: at PCI bus 1, device 4, function 1 ncr53c8xx: 53c896 detected ncr53c896-0: rev=0x05, base=0xb0200000, io_port=0x1000, irq=16 ncr53c896-0: ID 7, Fast-40, Parity Checking ncr53c896-0: on-chip RAM at 0xb0000000 ncr53c896-0: restart (scsi reset). ncr53c896-0: Downloading SCSI SCRIPTS. ncr53c896-1: rev=0x05, base=0xb0300000, io_port=0x1400, irq=16 ncr53c896-1: ID 7, Fast-40, Parity Checking ncr53c896-1: on-chip RAM at 0xb0100000 ncr53c896-1: restart (scsi reset). ncr53c896-1: Downloading SCSI SCRIPTS. scsi0 : ncr53c8xx - version 3.2a-2 scsi1 : ncr53c8xx - version 3.2a-2 scsi : 2 hosts. Vendor: COMPAQ Model: BB00921B91 Rev: 3B07 Type: Direct-Access ANSI SCSI revision: 02 Detected scsi disk sda at scsi0, channel 0, id 0, lun 0 ncr53c896-0-<0,0>: tagged command queue depth set to 8 ncr53c896-0-<0,*>: FAST-20 WIDE SCSI 40.0 MB/s (50 ns, offset 15) SCSI device sda: hdwr sector= 512 bytes. Sectors= 17773524 [8678 MB] [8.7 GB] sda: sda1 sda2 < sda5 sda6 sda7 > sda3 autodetecting RAID arrays autorun ... ... autorun DONE. VFS: Mounted root (ext2 filesystem) readonly. change_root: old root has d_count=1 Trying to unmount old root ... okay Freeing unused kernel memory: 72k freed Adding Swap: 530104k swap-space (priority -1) eepro100.c:v1.09j-t 9/29/99 Donald Becker http://cesdis.gsfc.nasa.gov/linux/drivers/eepro100.html eepro100.c: $Revision: 1.18 $ 1999/12/29 Modified by Andrey V. Savochkin eth0: OEM i82557/i82558 10/100 Ethernet at 0xd0868000, 00:50:8B:9A:42:12, IRQ 17. Receiver lock-up bug exists -- enabling work-around. Board assembly 010101-034, Physical connectors present: RJ45 Primary interface chip i82555 PHY #1. General self-test: passed. Serial sub-system self-test: passed. Internal registers self-test: passed. ROM checksum self-test: passed (0x04f4518b). ------------------------------------------------------------------------------------- Output from console FreeBSD boot: Booting [kernel]... Copyright (c) 1992-2000 The FreeBSD Project. Copyright (c) 1982, 1986, 1989, 1991, 1993 The Regents of the University of California. All rights reserved. FreeBSD 4.0-RELEASE #38: Wed Jun 21 15:32:12 GMT 2000 root@:/usr/src/sys/compile/CACHE Timecounter "i8254" frequency 1193182 Hz CPU: Pentium III/Pentium III Xeon (728.44-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x681 Stepping = 1 Features=0x383fbff real memory = 134217728 (131072K bytes) avail memory = 126832640 (123860K bytes) Programming 16 pins in IOAPIC #0 IOAPIC #0 intpin 2 -> irq 0 Programming 16 pins in IOAPIC #1 IOAPIC #1 intpin 0 -> irq 16 IOAPIC #1 intpin 1 -> irq 2 IOAPIC #1 intpin 2 -> irq 18 IOAPIC #1 intpin 3 -> irq 17 IOAPIC #1 intpin 4 -> irq 19 IOAPIC #1 intpin 5 -> irq 20 IOAPIC #1 intpin 6 -> irq 21 IOAPIC #1 intpin 7 -> irq 22 IOAPIC #1 intpin 8 -> irq 23 FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 1, version: 0x00040011, at 0xfee00000 cpu1 (AP): apic id: 0, version: 0x00040011, at 0xfee00000 io0 (APIC): apic id: 8, version: 0x000f0011, at 0xfec00000 io1 (APIC): apic id: 3, version: 0x000f0011, at 0xfec01000 Preloaded elf kernel "kernel" at 0xc0335000. Preloaded userconfig_script "/boot/kernel.conf" at 0xc033509c. Pentium Pro MTRR support enabled md0: Malloc disk npx0: on motherboard npx0: INT 16 interface pcib0: on motherboard pci0: on pcib0 pcib6: at device 1.0 on pci0 pci1: on pcib6 sym0: <896> port 0x1000-0x10ff mem 0xb0000000-0xb0001fff,0xb0200000-0xb02003ff i rq 16 at device 4.0 on pci1 sym0: No NVRAM, ID 7, Fast-40, LVD, parity checking sym1: <896> port 0x1400-0x14ff mem 0xb0100000-0xb0101fff,0xb0300000-0xb03003ff i rq 16 at device 4.1 on pci1 sym1: No NVRAM, ID 7, Fast-40, SE, parity checking pci1: (vendor=0x8086, dev=0x1229) at 5.0 irq 2 pci1: at 6.0 pci1: (vendor=0x0e11, dev=0xa0f0) at 7.0 irq 17 isab0: at device 15.0 on pci0 isa0: on isab0 atapci0: port 0x3000-0x300f at devic e 15.1 on pci0 ata0: at 0x1f0 irq 14 on atapci0 pcib5: on motherboard pci5: on pcib5 fdc0: at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0 fdc0: FIFO enabled, 8 bytes threshold fd0: <1440-KB 3.5" drive> on fdc0 drive 0 sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0 sio0: type 16550A, console sio1 at port 0x2f8-0x2ff irq 3 on isa0 sio1: type 16550A To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 27 18:27:12 2000 Delivered-To: freebsd-smp@freebsd.org Received: from kithrup.com (kithrup.com [205.179.156.40]) by hub.freebsd.org (Postfix) with ESMTP id 976C537C505 for ; Tue, 27 Jun 2000 18:26:47 -0700 (PDT) (envelope-from sef@kithrup.com) Received: (from sef@localhost) by kithrup.com (8.8.8/8.8.8) id SAA29061 for smp@freebsd.org; Tue, 27 Jun 2000 18:26:47 -0700 (PDT) (envelope-from sef) Date: Tue, 27 Jun 2000 18:26:47 -0700 (PDT) From: Sean Eric Fagan Message-Id: <200006280126.SAA29061@kithrup.com> To: smp@freebsd.org Subject: 3.4-RELEASE, SMP, and serial consoles Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org A while back I asked about a very serious problem I was having with getting serial consoles to work on my SMP box. Well... the machine was hooked up to another unix box that was acting as a console server. The console server was rebooted recently... and the problem I was having went away. Serial ports really are not supposed to be black magic, you know. Especially with the number of ex-DEC people I have in my building :). To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 27 19:16:56 2000 Delivered-To: freebsd-smp@freebsd.org Received: from lor.watermarkgroup.com (lor.watermarkgroup.com [207.202.73.33]) by hub.freebsd.org (Postfix) with ESMTP id A57AC37B755 for ; Tue, 27 Jun 2000 19:16:51 -0700 (PDT) (envelope-from luoqi@watermarkgroup.com) Received: (from luoqi@localhost) by lor.watermarkgroup.com (8.10.1/8.10.1) id e5S2GoR00710; Tue, 27 Jun 2000 22:16:50 -0400 (EDT) Date: Tue, 27 Jun 2000 22:16:50 -0400 (EDT) From: Luoqi Chen Message-Id: <200006280216.e5S2GoR00710@lor.watermarkgroup.com> To: freebsd-smp@FreeBSD.ORG, rmariz@timnordeste.com.br Subject: Re: Compaq Proliant ML350 SMP problem Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > FreeBSD 4.0-RELEASE #38: Wed Jun 21 15:32:12 GMT 2000 4.0-RELEASE has a bug preventing it from working on machines with multiple IOAPICs, get the latest 4.0-STABLE code and try again. -lq To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 27 20: 0:48 2000 Delivered-To: freebsd-smp@freebsd.org Received: from front.linuxcare.com.au (linuxcare.com.au [203.29.91.49]) by hub.freebsd.org (Postfix) with ESMTP id 8682737B71D for ; Tue, 27 Jun 2000 20:00:36 -0700 (PDT) (envelope-from grog@sydney.worldwide.lemis.com) Received: from sydney.worldwide.lemis.com ([203.17.0.42]) by front.linuxcare.com.au (8.9.3/8.9.3/Debian 8.9.3-21) with ESMTP id NAA07171; Wed, 28 Jun 2000 13:00:32 +1000 X-Authentication-Warning: front.linuxcare.com.au: Host [203.17.0.42] claimed to be sydney.worldwide.lemis.com Received: (from grog@localhost) by sydney.worldwide.lemis.com (8.9.3/8.9.3) id NAA01876; Wed, 28 Jun 2000 13:00:31 +1000 (EST) (envelope-from grog) Date: Wed, 28 Jun 2000 13:00:31 +1000 From: Greg Lehey To: Daniel Eischen Cc: Jason Evans , smp@FreeBSD.ORG Subject: Re: SMP meeting summary Message-ID: <20000628130031.B1760@sydney.worldwide.lemis.com> References: <20000624235605.D8965@blitz.canonware.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: ; from eischen@vigrid.com on Sun, Jun 25, 2000 at 09:58:27AM -0400 Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Sunday, 25 June 2000 at 9:58:27 -0400, Daniel Eischen wrote: > On 24 Jun 2000, Jason Evans wrote: >> 3. The BSD/OS solution. >> >> - The BGL remains, but becomes increasingly meaningless. In particular, >> it is not always necessary to obtain it in order to enter the kernel. >> >> - Instead the system protects shared data structures with mutexes. >> These mutexes replace calls to tsleep() when waiting on shared >> resources (the involuntary process synchronization mentioned above). >> In contrast to traditional UNIX, mutexes will be used much more >> frequently in order to protect data structures which were previously >> implicitly protected by the non-preemptive nature of the kernel. This >> mechanism will replace calls to tsleep() for involuntary context >> switches. >> >> Compared with the use of tsleep(), mutexes have a number of >> advantages: >> >> - Each mutex has its own wait (sleep) queue. When a process releases >> a mutex, it automatically schedules the next process waiting on the >> queue. This is more efficient than searching a possibly very long, >> linear sleep queue. It also avoids the flooding when multiple >> processes get scheduled, and most of them have to go back to sleep >> again. >> >> - Mutexes can be a combination of spin and sleep mutexes: for a >> resource which may be held only for a very short period of time, >> even the overhead of sleeping and rescheduling may be higher than >> waiting in a tight loop. A spin/sleep lock might first wait in a >> tight loop for 2 microseconds and then sleep if the lock is still >> not available at that time. This is an issue which Sun has >> investigated in great detail with Solaris. BSDi has not pursued >> this yet, though the BSD/OS threading primitives make this an easy >> extention to add. It's possibly an area for us to investigate once >> the system is up and limping again. > > If anyone is interested... > > All high-level interrupts (levels 11-15, mostly PIO serial interrupts) > in Solaris use spin mutexes and don't use an interrupt thread. They > execute in the context of the thread that was currently running. All > other interrupts below level 11 (clock, network, disk, etc) use interrupt > threads. > > A Solaris (non-spinning) mutex will only spin while the owning thread is > running. Since BSDi mutexes have owners (correct me if I'm wrong), this > seems to be better than arbitrarily spinning. Mutexes only have owners when they're being held. But we won't spin for any length of time on a mutex; that's why we have a thread context for the interrupts. >> - BSD/OS also does not implement read/write locks, a thing that Linux >> has recently introduced. We didn't discuss this further, but the >> concepts are well understood, and it should be relatively simple to >> implement them if we find a need. > > Mutexes are only used in Solaris when they will be held for very small > amounts of time. Read/write locks and semaphores are used for all > other instances. While we are modifying the kernel to add mutexes, > it would probably be worthwhile to comment those sections of code > that could hold mutexes for something other than a very short period > of time. Or even use a different naming convention for those mutexes. Agreed, I don't like the terminology we seem to have settled on. From my way of thinking, a mutex is a spin lock, and a semaphore is a blocking lock. What we're talking about here are really semaphores, though it makes sense to spin a bit first before blocking in the case that the lock may be released quickly: it takes a fair amount of overhead to schedule, and if there's a good chance the lock will be available by the time we've scheduled, there's no point in blocking immediately. One of the things I want to do further down the line is to instrument some statistics on the semaphores^H^H^Hnmutexes so we can decide what kind we need where (and when). Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 27 20:26:23 2000 Delivered-To: freebsd-smp@freebsd.org Received: from magnesium.net (toxic.magnesium.net [207.154.84.15]) by hub.freebsd.org (Postfix) with SMTP id 6D35B37C491 for ; Tue, 27 Jun 2000 20:26:10 -0700 (PDT) (envelope-from jasone@magnesium.net) Received: (qmail 20706 invoked by uid 1142); 28 Jun 2000 03:26:09 -0000 Date: 27 Jun 2000 20:26:09 -0700 Date: Tue, 27 Jun 2000 20:25:57 -0700 From: Jason Evans To: Greg Lehey Cc: Daniel Eischen , smp@FreeBSD.ORG Subject: Re: SMP meeting summary Message-ID: <20000627202557.S15267@blitz.canonware.com> References: <20000624235605.D8965@blitz.canonware.com> <20000628130031.B1760@sydney.worldwide.lemis.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1i In-Reply-To: <20000628130031.B1760@sydney.worldwide.lemis.com>; from grog@lemis.com on Wed, Jun 28, 2000 at 01:00:31PM +1000 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Wed, Jun 28, 2000 at 01:00:31PM +1000, Greg Lehey wrote: > On Sunday, 25 June 2000 at 9:58:27 -0400, Daniel Eischen wrote: > > On 24 Jun 2000, Jason Evans wrote: > >> - BSD/OS also does not implement read/write locks, a thing that Linux > >> has recently introduced. We didn't discuss this further, but the > >> concepts are well understood, and it should be relatively simple to > >> implement them if we find a need. > > > > Mutexes are only used in Solaris when they will be held for very small > > amounts of time. Read/write locks and semaphores are used for all > > other instances. While we are modifying the kernel to add mutexes, > > it would probably be worthwhile to comment those sections of code > > that could hold mutexes for something other than a very short period > > of time. Or even use a different naming convention for those mutexes. > > Agreed, I don't like the terminology we seem to have settled on. From > my way of thinking, a mutex is a spin lock, and a semaphore is a > blocking lock. What we're talking about here are really semaphores, > though it makes sense to spin a bit first before blocking in the case > that the lock may be released quickly: it takes a fair amount of > overhead to schedule, and if there's a good chance the lock will be > available by the time we've scheduled, there's no point in blocking > immediately. One of the things I want to do further down the line is > to instrument some statistics on the semaphores^H^H^Hnmutexes so we > can decide what kind we need where (and when). Mutexes come in different flavors. From an API perspective, whether the mutex spins, blocks, or is adaptive isn't visible. A semaphore is significantly different. It can be used in place of a mutex, but it has additional functionality. A POSIX semaphore has a count associated with it. Other definitions of semaphores generally have a count associated with the semaphore as well, though there may be more functionality than POSIX semaphores provide. Posting (incrementing) the semaphore always succeeds, but waiting on (decrementing) the semaphore will spin/block until the decrement operation can be completed without the semaphore value becoming negative. So, both mutexes and semaphores can be implemented as spinning/blocking/adaptive. Jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 27 20:27:12 2000 Delivered-To: freebsd-smp@freebsd.org Received: from pcnet1.pcnet.com (pcnet1.pcnet.com [204.213.232.3]) by hub.freebsd.org (Postfix) with ESMTP id 30ACA37B64D for ; Tue, 27 Jun 2000 20:27:10 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: (from eischen@localhost) by pcnet1.pcnet.com (8.8.7/PCNet) id XAA20978; Tue, 27 Jun 2000 23:26:43 -0400 (EDT) Date: Tue, 27 Jun 2000 23:26:43 -0400 (EDT) From: Daniel Eischen To: Greg Lehey Cc: Jason Evans , smp@FreeBSD.ORG Subject: Re: SMP meeting summary In-Reply-To: <20000628130031.B1760@sydney.worldwide.lemis.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Wed, 28 Jun 2000, Greg Lehey wrote: > On Sunday, 25 June 2000 at 9:58:27 -0400, Daniel Eischen wrote: > > A Solaris (non-spinning) mutex will only spin while the owning thread is > > running. Since BSDi mutexes have owners (correct me if I'm wrong), this > > seems to be better than arbitrarily spinning. > > Mutexes only have owners when they're being held. But we won't spin > for any length of time on a mutex; that's why we have a thread context > for the interrupts. Right, I didn't think mutexes would have owners if they were not locked ;-) > > >> - BSD/OS also does not implement read/write locks, a thing that Linux > >> has recently introduced. We didn't discuss this further, but the > >> concepts are well understood, and it should be relatively simple to > >> implement them if we find a need. > > > > Mutexes are only used in Solaris when they will be held for very small > > amounts of time. Read/write locks and semaphores are used for all > > other instances. While we are modifying the kernel to add mutexes, > > it would probably be worthwhile to comment those sections of code > > that could hold mutexes for something other than a very short period > > of time. Or even use a different naming convention for those mutexes. > > Agreed, I don't like the terminology we seem to have settled on. From > my way of thinking, a mutex is a spin lock, and a semaphore is a > blocking lock. What we're talking about here are really semaphores, > though it makes sense to spin a bit first before blocking in the case > that the lock may be released quickly: it takes a fair amount of > overhead to schedule, and if there's a good chance the lock will be > available by the time we've scheduled, there's no point in blocking > immediately. It doesn't make sense to spin if the lock holder is not runnable, especially on a single CPU system. In order to make the owning thread runnable, you've got to take the scheduling queue lock and there has to be a context switch anyways. You might as well get ready to place the blocking thread on the sleep queue. If after acquiring (or while spinning on) the sleep queue lock, the owning thread becomes runnable, you can back out of the sleep queue insertion. > One of the things I want to do further down the line is > to instrument some statistics on the semaphores^H^H^Hnmutexes so we > can decide what kind we need where (and when). Great! Sounds like Solaris lockstat(1). > Greg > -- > Finger grog@lemis.com for PGP public key > See complete headers for address and phone numbers -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 27 21:21: 7 2000 Delivered-To: freebsd-smp@freebsd.org Received: from front.linuxcare.com.au (linuxcare.com.au [203.29.91.49]) by hub.freebsd.org (Postfix) with ESMTP id 4266137B818; Tue, 27 Jun 2000 21:20:59 -0700 (PDT) (envelope-from grog@sydney.worldwide.lemis.com) Received: from sydney.worldwide.lemis.com ([203.17.0.42]) by front.linuxcare.com.au (8.9.3/8.9.3/Debian 8.9.3-21) with ESMTP id OAA11365; Wed, 28 Jun 2000 14:20:56 +1000 X-Authentication-Warning: front.linuxcare.com.au: Host [203.17.0.42] claimed to be sydney.worldwide.lemis.com Received: (from grog@localhost) by sydney.worldwide.lemis.com (8.9.3/8.9.3) id OAA01942; Wed, 28 Jun 2000 14:20:56 +1000 (EST) (envelope-from grog) Date: Wed, 28 Jun 2000 14:20:56 +1000 From: Greg Lehey To: "Jordan K. Hubbard" Cc: committers@freebsd.org, smp@freebsd.org Subject: Re: SMPng effort with BSD/OS - the overall story Message-ID: <20000628142056.C1760@sydney.worldwide.lemis.com> References: <2982.961950014@localhost> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <2982.961950014@localhost>; from jkh@zippy.osd.bsdi.com on Sun, Jun 25, 2000 at 09:20:14AM -0700 Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Sunday, 25 June 2000 at 9:20:14 -0700, Jordan K. Hubbard wrote: > First, as the guy who set up the recent "SMP seminar", let me just > apologise right now for dropping the ball on one very significant > item: I didn't have someone appointed "scribe" from the list of > full-time employee types who went there and this resulted in an > unfortunate delay in briefing those people who weren't in attendance. > Greg Lehey did do some video taping of the event, but that was just > something he did on his own and it will take him some time to figure > out how to get it into a format which everyone else can use. Note also that, as Jordan says, I "just did this". The results are *very* unprofessional, since I didn't have time to play cameraman. Still, there may be something of value in there. Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 27 22: 0: 6 2000 Delivered-To: freebsd-smp@freebsd.org Received: from front.linuxcare.com.au (linuxcare.com.au [203.29.91.49]) by hub.freebsd.org (Postfix) with ESMTP id 0750737B58D for ; Tue, 27 Jun 2000 22:00:00 -0700 (PDT) (envelope-from grog@sydney.worldwide.lemis.com) Received: from sydney.worldwide.lemis.com ([203.17.0.42]) by front.linuxcare.com.au (8.9.3/8.9.3/Debian 8.9.3-21) with ESMTP id OAA13183; Wed, 28 Jun 2000 14:59:56 +1000 X-Authentication-Warning: front.linuxcare.com.au: Host [203.17.0.42] claimed to be sydney.worldwide.lemis.com Received: (from grog@localhost) by sydney.worldwide.lemis.com (8.9.3/8.9.3) id OAA02338; Wed, 28 Jun 2000 14:59:55 +1000 (EST) (envelope-from grog) Date: Wed, 28 Jun 2000 14:59:55 +1000 From: Greg Lehey To: Jason Evans Cc: Daniel Eischen , smp@FreeBSD.ORG Subject: Re: SMP meeting summary Message-ID: <20000628145955.A2209@sydney.worldwide.lemis.com> References: <20000624235605.D8965@blitz.canonware.com> <20000628130031.B1760@sydney.worldwide.lemis.com> <20000627202557.S15267@blitz.canonware.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0pre2i In-Reply-To: <20000627202557.S15267@blitz.canonware.com> Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Tuesday, 27 June 2000 at 20:25:57 -0700, Jason Evans wrote: > On Wed, Jun 28, 2000 at 01:00:31PM +1000, Greg Lehey wrote: >> On Sunday, 25 June 2000 at 9:58:27 -0400, Daniel Eischen wrote: >>> On 24 Jun 2000, Jason Evans wrote: >>>> - BSD/OS also does not implement read/write locks, a thing that Linux >>>> has recently introduced. We didn't discuss this further, but the >>>> concepts are well understood, and it should be relatively simple to >>>> implement them if we find a need. >>> >>> Mutexes are only used in Solaris when they will be held for very small >>> amounts of time. Read/write locks and semaphores are used for all >>> other instances. While we are modifying the kernel to add mutexes, >>> it would probably be worthwhile to comment those sections of code >>> that could hold mutexes for something other than a very short period >>> of time. Or even use a different naming convention for those mutexes. >> >> Agreed, I don't like the terminology we seem to have settled on. From >> my way of thinking, a mutex is a spin lock, and a semaphore is a >> blocking lock. What we're talking about here are really semaphores, >> though it makes sense to spin a bit first before blocking in the case >> that the lock may be released quickly: it takes a fair amount of >> overhead to schedule, and if there's a good chance the lock will be >> available by the time we've scheduled, there's no point in blocking >> immediately. One of the things I want to do further down the line is >> to instrument some statistics on the semaphores^H^H^Hnmutexes so we >> can decide what kind we need where (and when). > > Mutexes come in different flavors. From an API perspective, whether the > mutex spins, blocks, or is adaptive isn't visible. > > A semaphore is significantly different. It can be used in place of a > mutex, but it has additional functionality. A POSIX semaphore has a count > associated with it. Other definitions of semaphores generally have a count > associated with the semaphore as well, though there may be more > functionality than POSIX semaphores provide. Posting (incrementing) the > semaphore always succeeds, but waiting on (decrementing) the semaphore will > spin/block until the decrement operation can be completed without the > semaphore value becoming negative. Hmm. I haven't seen the POSIX definition, but Dijkstra's semaphores had a count and two main operations, P and V. P decrements, and if the semaphore counter goes negative, the process is placed on the semaphore sleep queue. If V increments to 0, the first process on the sleep queue is scheduled. The mutexes we're looking at here are a degenerate case of semaphores: instead of a count, we have a flag (in fact, a predicate) that says whether the semaphore is held or not. That's effectively a semaphore with an initial counter value of 0. > So, both mutexes and semaphores can be implemented as > spinning/blocking/adaptive. I was pretty sure that a semaphore was always blocking. Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Tue Jun 27 22:11:56 2000 Delivered-To: freebsd-smp@freebsd.org Received: from front.linuxcare.com.au (linuxcare.com.au [203.29.91.49]) by hub.freebsd.org (Postfix) with ESMTP id C532737B924 for ; Tue, 27 Jun 2000 22:11:51 -0700 (PDT) (envelope-from grog@sydney.worldwide.lemis.com) Received: from sydney.worldwide.lemis.com ([203.17.0.42]) by front.linuxcare.com.au (8.9.3/8.9.3/Debian 8.9.3-21) with ESMTP id PAA13798; Wed, 28 Jun 2000 15:11:50 +1000 X-Authentication-Warning: front.linuxcare.com.au: Host [203.17.0.42] claimed to be sydney.worldwide.lemis.com Received: (from grog@localhost) by sydney.worldwide.lemis.com (8.9.3/8.9.3) id PAA02396; Wed, 28 Jun 2000 15:11:49 +1000 (EST) (envelope-from grog) Date: Wed, 28 Jun 2000 15:11:49 +1000 From: Greg Lehey To: Nate Williams Cc: Terry Lambert , Daniel Eischen , Jason Evans , smp@FreeBSD.ORG Subject: Re: SMP meeting summary Message-ID: <20000628151149.B2209@sydney.worldwide.lemis.com> References: <200006251736.KAA09884@usr02.primenet.com> <200006260442.WAA15731@nomad.yogotech.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0pre2i In-Reply-To: <200006260442.WAA15731@nomad.yogotech.com> Organization: LEMIS, PO Box 460, Echunga SA 5153, Australia Phone: +61-8-8388-8286 Fax: +61-8-8388-8725 Mobile: +61-418-838-708 WWW-Home-Page: http://www.lemis.com/~grog X-PGP-Fingerprint: 6B 7B C3 8C 61 CD 54 AF 13 24 52 F8 6D A4 95 EF Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Sunday, 25 June 2000 at 22:42:02 -0600, Nate Williams wrote: >> Dynix had no problem with 32 processors. Most SVR4 variants, and >> I will include Solaris in this, use mutex protection of structures, >> and start to fall down drastically over 4 processors. > > Amazing that you say this, yet I see extremely good results on > Solaris boxes up to 64 processors. Yes, I was wondering about this statement too. As usual, it probably depends on what you're doing. Terry seems to know Dynix pretty well, so I wouldn't be surprised to hear that this statement originated there. > Suffice it to say that I'm not convinced, nor am I convinced that > mutex's around data structures is any different than critical > sectioning. I'm convinced that they're different. The real issue is which is better, and I tend towards locking data structures. But Terry, go ahead and prove us wrong if you want. I won't mind. > They are essentially the same thing, in that the critical section is > almost always the code that deals with a particular (shared) data > structure. That's a degenerate case, of course. Greg -- Finger grog@lemis.com for PGP public key See complete headers for address and phone numbers To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 28 16:15:51 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp05.primenet.com (smtp05.primenet.com [206.165.6.135]) by hub.freebsd.org (Postfix) with ESMTP id 9FA2137BA9D for ; Wed, 28 Jun 2000 16:15:43 -0700 (PDT) (envelope-from tlambert@usr08.primenet.com) Received: (from daemon@localhost) by smtp05.primenet.com (8.9.3/8.9.3) id QAA20000; Wed, 28 Jun 2000 16:15:58 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp05.primenet.com, id smtpdAAA6AaGaN; Wed Jun 28 16:15:48 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id QAA03731; Wed, 28 Jun 2000 16:15:22 -0700 (MST) From: Terry Lambert Message-Id: <200006282315.QAA03731@usr08.primenet.com> Subject: Re: SMP meeting summary To: nate@yogotech.com Date: Wed, 28 Jun 2000 23:15:22 +0000 (GMT) Cc: tlambert@primenet.com (Terry Lambert), eischen@vigrid.com (Daniel Eischen), jasone@canonware.com (Jason Evans), smp@FreeBSD.ORG In-Reply-To: <200006260442.WAA15731@nomad.yogotech.com> from "Nate Williams" at Jun 25, 2000 10:42:02 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > Dynix had no problem with 32 processors. Most SVR4 variants, and > > I will include Solaris in this, use mutex protection of structures, > > and start to fall down drastically over 4 processors. > > Amazing that you say this, yet I see extremely good results on Solaris > boxes up to 64 processors. Boxes or clusters? NUMA or non-NUMA? MESI or MEI cache coherency? > Suffice it to say that I'm not convinced, nor am I convinced that > mutex's around data structures is any different than critical > sectioning. > > They are essentially the same thing, in that the critical section is > almost always the code that deals with a particular (shared) data > structure. I can put you in touch with Sabsovitch or Leventhal if you need the people who actually wrote the code, rather than someone who has only read and modified the code, if you think that authority lends credulity. A SPARCCenter is not the same thing as an Intel box running SMP, and is not even the same as a SPARC-20 running multiple processors or a Sparc 5 with a Weitek processor add-in. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 28 16:19:14 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp01.primenet.com (smtp01.primenet.com [206.165.6.131]) by hub.freebsd.org (Postfix) with ESMTP id 420B637C24F for ; Wed, 28 Jun 2000 16:18:55 -0700 (PDT) (envelope-from tlambert@usr08.primenet.com) Received: (from daemon@localhost) by smtp01.primenet.com (8.9.3/8.9.3) id QAA26134; Wed, 28 Jun 2000 16:18:41 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp01.primenet.com, id smtpdAAAboaq5Y; Wed Jun 28 16:18:32 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id QAA03832; Wed, 28 Jun 2000 16:18:36 -0700 (MST) From: Terry Lambert Message-Id: <200006282318.QAA03832@usr08.primenet.com> Subject: Re: SMP meeting summary To: frank@exit.com Date: Wed, 28 Jun 2000 23:18:36 +0000 (GMT) Cc: tlambert@primenet.com (Terry Lambert), jasone@canonware.com (Jason Evans), smp@FreeBSD.ORG In-Reply-To: <200006260631.XAA43949@realtime.exit.com> from "Frank Mayhar" at Jun 25, 2000 11:31:51 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Terry Lambert wrote: > > In Streams, this was addressed by removing two primitives, and creating > > what are called "priority bands" to deal with the issue (I worked for > > Novell USG when UnixWare was going to 2.0 and integrating the ES/MP code > > into the SVR4 base, and saw much of this happen). > > Speaking of which, there has _got_ to be a better way. It's possible > when dealing with Streams to get into nested-lock situations where you > need a lock with a lower priority and it's not possible (due to constraints > elsewhere) to raise the priority of the lock. And, of course, it's always > dangerous to raise the priority of an existing lock, since you may miss > a case. > > I'm not sure that interrupt threads are the right way to go either, though. > > If there even _is_ a "right" way. One way that was "accidently" discovered at Novell was to run into the streams stack at interrupt context, until one reached a synchronization point. Obviously, this doesn't work the same in all stacks, and can only be run to completion unidirectionally, in the best case, but it was enough to justify the changeover to ODI drivers. 8-). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 28 16:38: 7 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp03.primenet.com (smtp03.primenet.com [206.165.6.133]) by hub.freebsd.org (Postfix) with ESMTP id CBEF737B554 for ; Wed, 28 Jun 2000 16:38:01 -0700 (PDT) (envelope-from tlambert@usr08.primenet.com) Received: (from daemon@localhost) by smtp03.primenet.com (8.9.3/8.9.3) id QAA24402; Wed, 28 Jun 2000 16:37:30 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp03.primenet.com, id smtpdAAA5BaytV; Wed Jun 28 16:37:14 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id QAA07749; Wed, 28 Jun 2000 16:37:41 -0700 (MST) From: Terry Lambert Message-Id: <200006282337.QAA07749@usr08.primenet.com> Subject: Re: SMP meeting summary To: adsharma@sharmas.dhs.org Date: Wed, 28 Jun 2000 23:37:41 +0000 (GMT) Cc: smp@FreeBSD.ORG In-Reply-To: <200006261647.JAA07306@sharmas.dhs.org> from "Arun Sharma" at Jun 26, 2000 09:47:31 AM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > > Suffice it to say that I'm not convinced, nor am I convinced that > > mutex's around data structures is any different than critical > > sectioning. > > > > They are essentially the same thing, in that the critical section is > > almost always the code that deals with a particular (shared) data > > structure. > > I'd argue that mutexes around data structures allow more concurrency > than critical sections. It's the "lock the data - not code" principle. > Think of the case where there are a thousand instances of the > data structure and one critical section. That would be on allocating, deleting, and enumerating lists of structures. In your case, you'd lock the data to protect it from being manipulated; in mine, I'd lock the section of code that did the allocation, deletion, or iteration. I'd also be able to split a list of 60 locked entries into 6 list heads of 10 locked entries each. Now I'm more concurrent than you are, since I can have any number of readers without protecting the data, and you need to lock the list heads, the list structures, and the list entries. If I implement shared, single reader, multiple writer locks with intention modes, and then arrange them in a hierarchical arrangement, I can have variable granularity that changes as, for example, the number of entries in the proclist goes up and down. The price would be the same if the data were migrating willy-nilly between processors, but you wouldn't allow that. Instead, you would strive to seperate the contention domains, per processor. It is the inter-processor contention that is the problem. Think of it as trying for a virtual NUMA system. This approach will have to be supported for better than 8-way processing on Xeon and similar machines with "glueless" SMP implementations. See: http://www.intel.com/PentiumII/xeon/home.htm Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 28 16:52:39 2000 Delivered-To: freebsd-smp@freebsd.org Received: from smtp05.primenet.com (smtp05.primenet.com [206.165.6.135]) by hub.freebsd.org (Postfix) with ESMTP id 5B17437B69C for ; Wed, 28 Jun 2000 16:52:35 -0700 (PDT) (envelope-from tlambert@usr08.primenet.com) Received: (from daemon@localhost) by smtp05.primenet.com (8.9.3/8.9.3) id QAA02380; Wed, 28 Jun 2000 16:52:49 -0700 (MST) Received: from usr08.primenet.com(206.165.6.208) via SMTP by smtp05.primenet.com, id smtpdAAAlmaOue; Wed Jun 28 16:52:31 2000 Received: (from tlambert@localhost) by usr08.primenet.com (8.8.5/8.8.5) id QAA08247; Wed, 28 Jun 2000 16:52:06 -0700 (MST) From: Terry Lambert Message-Id: <200006282352.QAA08247@usr08.primenet.com> Subject: Re: SMP meeting summary To: jrs@enteract.com (John Sconiers) Date: Wed, 28 Jun 2000 23:52:06 +0000 (GMT) Cc: dillon@apollo.backplane.com (Matthew Dillon), luoqi@watermarkgroup.com (Luoqi Chen), jasone@canonware.com, smp@FreeBSD.ORG In-Reply-To: from "John Sconiers" at Jun 26, 2000 02:56:30 PM X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > Quick question. I've been reading this as well as past threads and > understand the merit and functionality of the new SMP model. However, how > much (if any) will Performance on both single and multi cpu machines > improve after this change?? Will the new systme use up more resources(ie > memory, swap, cpu) in single and multi CPU configurations?? I apologize > if this has already been brought up. Here's my lab experience with UnixWare from Novell: Making the kernel preemptable and the UFS (SVR4's FFS) reentrant sped up UnixWare 2.0 SMP over Unixware 1.0 UP ~150%. That's measured, in a lab. UnixWare uses a method called DOW or Delayed Ordered Writes (the ReiserFS is in violation of this patent, if SCO wanted to push it); it is inferior to Soft Updates. It introduces a synchronization point, rather than an update, when a modification would require an update occur and a dependency entered into the dependency tree. In addition, UnixWare did (does) not have a unified VM and buffer cache. The per vnode working set quota code I had proposed to resolve the "move mouse, wiggle cursor" problem with page stealing by a single hungry process was not adopted, and a fixed scheduling class was adopted instead (giving X the cycles to page itself back in -- stealing the pages back -- in the process), so the results were poorer than they would have otherwise been. Combined, these mean that this lab test was biased against the speed improvement. On a reasonable non-benchmark workload where there were no stalling points being introduced by all processes whacking the same directory, and where no single process was allowed to force another out of core via page steal DOS attacks, and where the buffer cache and VM system didn't result in extra copies and "useless" reserve memory for one system or another, I'd expect this to increase to 250% or more (I was able to get ~190% overall by introducing my working set quota changes on a per process, rather than a per vnode, basis for experimental purposes). Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 28 20:47:22 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mail-out1.apple.com (mail-out1.apple.com [17.254.0.52]) by hub.freebsd.org (Postfix) with ESMTP id 7FAD737B5AC for ; Wed, 28 Jun 2000 20:47:19 -0700 (PDT) (envelope-from aramesh@scv2.apple.com) Received: from mailgate1.apple.com (A17-128-100-225.apple.com [17.128.100.225]) by mail-out1.apple.com (8.9.3/8.9.3) with ESMTP id UAA03638 for ; Wed, 28 Jun 2000 20:47:18 -0700 (PDT) Received: from scv2.apple.com (scv2.apple.com) by mailgate1.apple.com (Content Technologies SMTPRS 4.1.5) with ESMTP id for ; Wed, 28 Jun 2000 20:47:18 -0700 Received: from narmada (narmada.apple.com [17.202.42.187]) by scv2.apple.com (8.9.3/8.9.3) with SMTP id UAA29575 for ; Wed, 28 Jun 2000 20:47:22 -0700 (PDT) Message-Id: <200006290347.UAA29575@scv2.apple.com> To: freebsd-smp@freebsd.org Subject: Date: Wed, 28 Jun 2000 20:47:17 -0700 From: Ananthakrishna Ramesh Reply-To: aramesh@apple.com x-mailer: Apple Mail (2.318) Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org auth 2580da35 subscribe freebsd-smp aramesh@apple.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Wed Jun 28 21:44:18 2000 Delivered-To: freebsd-smp@freebsd.org Received: from sharmas.dhs.org (c62443-a.frmt1.sfba.home.com [24.0.69.165]) by hub.freebsd.org (Postfix) with ESMTP id E451E37C301 for ; Wed, 28 Jun 2000 21:44:15 -0700 (PDT) (envelope-from adsharma@sharmas.dhs.org) Received: (from adsharma@localhost) by sharmas.dhs.org (8.9.3/8.9.3) id VAA14300; Wed, 28 Jun 2000 21:43:22 -0700 Date: Wed, 28 Jun 2000 21:43:22 -0700 From: Arun Sharma Message-Id: <200006290443.VAA14300@sharmas.dhs.org> To: tlambert@primenet.com Cc: smp@freebsd.org Subject: Re: SMP meeting summary In-Reply-To: <200006282337.QAA07749@usr08.primenet.com> References: <200006282337.QAA07749@usr08.primenet.com> Reply-To: adsharma@sharmas.dhs.org Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org On Wed, 28 Jun 2000 23:37:41 +0000 (GMT), Terry Lambert wrote: > > That would be on allocating, deleting, and enumerating lists of > structures. Sure, when the groups of data structures are manipulated together, it makes sense to just use a single lock. But do you have any statistics to say that this is the common case ? The granularity of the lock certainly needs to match up with the common use case. -Arun To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Jun 29 10:27:31 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mirage.nlink.com.br (mirage.nlink.com.br [200.249.195.3]) by hub.freebsd.org (Postfix) with SMTP id 705C137BF01 for ; Thu, 29 Jun 2000 10:27:20 -0700 (PDT) (envelope-from rmariz@nlink.com.br) Received: (qmail 15458 invoked by uid 65534); 29 Jun 2000 17:27:10 -0000 To: Luoqi Chen Subject: Re: Compaq Proliant ML350 SMP problem Message-ID: <962299630.395b86ee72671@www.nlink.com.br> Date: Thu, 29 Jun 2000 14:27:10 -0300 From: Cc: References: <200006280216.e5S2GoR00710@lor.watermarkgroup.com> In-Reply-To: <200006280216.e5S2GoR00710@lor.watermarkgroup.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit User-Agent: IMP/PHP IMAP webmail program 2.2.0-pre12 X-Originating-IP: 200.249.140.216 Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Hi Chen, Thank you. Good tip. You were absolutely correct. Now I am happily running SMP on a two processor Compaq Proliant ML 350. Bye, Renato Mariz. Luoqi Chen wrote: > > FreeBSD 4.0-RELEASE #38: Wed Jun 21 15:32:12 GMT 2000 > > 4.0-RELEASE has a bug preventing it from working on machines with > multiple > IOAPICs, get the latest 4.0-STABLE code and try again. > > -lq > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-smp" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Jun 29 13:53:19 2000 Delivered-To: freebsd-smp@freebsd.org Received: from fast.cs.utah.edu (fast.cs.utah.edu [155.99.212.1]) by hub.freebsd.org (Postfix) with ESMTP id E1A5637C29C for ; Thu, 29 Jun 2000 13:53:14 -0700 (PDT) (envelope-from vanmaren@fast.cs.utah.edu) Received: (from vanmaren@localhost) by fast.cs.utah.edu (8.9.1/8.9.1) id OAA14920; Thu, 29 Jun 2000 14:52:59 -0600 (MDT) Date: Thu, 29 Jun 2000 14:52:59 -0600 (MDT) From: Kevin Van Maren Message-Id: <200006292052.OAA14920@fast.cs.utah.edu> To: cp@bsdi.com, dillon@apollo.backplane.com Subject: Re: Stepping on Toes Cc: dg@root.com, freebsd-smp@FreeBSD.ORG Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org > :} Even with interrupt threads we have the GiantMutex issue... the same > :} issue that we have with our current MP implementation. We cannot remove > :} SPL's until we remove the GiantMutex, and we cannot remove GiantMutex > :} without major modifications to just about every single source file in sys/ > :In general this isn't true. If you get to the point where > : > :1) All entrance to unsafe code is proteced by Giant. > :2) Tsleep and friend if any release Giant when they > : a process is suspended and re-acquire it on exit > :3) Interrupts have a context to run in. > :4) You have one or more scheduling locks. > : > :Then you can just turn spls into a nop. There is lots of hand waving > :in regards to details at this point. BSD/OS SMPng is the existance > :proof. > :It seems like one of the major problems in retaining spls during > :the change over period is that they don't much useful, and effectively > :push everything under Giant. > : > : Grabbing a spl will only block interrupts, it will not give > : any protection against an interrupt thread which has already > : started. > Right this moment the requirement is that only someone holding Giant > is allowed to mess with spl*()'s (the cpl variable can only be messed > with by people holding Giant). > > At this moment, without interrupt threads, interrupts can share Giant > with the curproc they interrupted. This is how our existing MP stuff > worked already. > > When Greg moves interrupts to their own threads, and obtains Giant to > run those interrupts, no more sharing will occur and just the fact > that the interrupt is holding Giant guarentees that nobody else will > be messing with SPLs, thus the SPLs can be removed entirely. > > -Matt Hi all, I'm really glad to see FreeBSD/SMP taking off again. It will be a great day when FreeBSD scales well on the Unisys ES7000 (which is what I work on for my day job these days). Forget those puny 8X servers... I do run FreeBSD on my dual P-II, so I'm excited about what will be in FreeBSD 5.0 Anyway, for my MS Thesis, I ran Linux/BSD device dirvers on top of a preemptive, multi-threaded kernel. The approach I took is similar to what is proposed here, so I thought it might be of general interest. Basically, I reduced the synchronization problem down to two mutexes: a process lock (giant), and an interrupt lock. Interrupts executed in their own threads, and acquired and released the interrupt lock. Process-level requests acquired the process lock (giant), and disabled interrupts (spl disables/enables all interrupts in this simple mapping) by acquiring the interrupt lock; this either caused the interrupt thread to block, or the process thread blocked until the interrupt handler completed. To preserve the atomicity of interrupts (which was assumed in *many* places, at least in Linux), I required the interrupt lock to be held to acquire or release "giant", and had the interrupt thread "stop" the process-level thread. sleep is handled as mentioned above, by releasing/acquiring the process lock. The solution here uses only a single lock for interrupts and process- level activity. This was not a solution for me because in Linux drivers would spin (on jiffies...) at process level until the interrupt handler executed. The biggest problem with my approach was that if the interrupt thread had to stop a process-level thread, that was pretty expensive. Also, a process-level thread could not be started until the interrupt handler completed -- I should have used a separate lock to protect it for the period of time the interrupt handler needed the info to be "valid". Removing the assumtion that interrupts are processed atomically is difficult (since the code doesn't state that it makes the assumption) and may be the cause of some hairy race conditions, but is critical for better performance (eliminating the extra synchronization to prevent process and interrupt threads from executing in parallel). A copy of my MS thesis may be downloaded from: http://www.cs.utah.edu/flux/papers/vanmaren-thesis-base.html I also have a bit of advice: cache lines are more important under SMP than on a uni-processor. Highly-contended spin-locks should be in their own cache line, but less-contended locks should be in the same cache line as the data they protect. It can be expensive to shuffle dirty cache lines around, especially in larger systems (8X up through the ES7000) where the processor buss is not shared by all the CPUs. Watching the data layout in structures would be a good idea, but can be tuned after everything "works".. Kevin Van Maren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Jun 29 17:43:20 2000 Delivered-To: freebsd-smp@freebsd.org Received: from quebec.procergs.com.br (quebec.via-rs.com.br [200.198.128.236]) by hub.freebsd.org (Postfix) with ESMTP id 13AD937B55F for ; Thu, 29 Jun 2000 17:42:54 -0700 (PDT) (envelope-from lauro@via-rs.net) Received: from via-rs.net (p16.sa06.pae.procergs.com.br [200.198.133.76]) by quebec.procergs.com.br (Postfix) with ESMTP id 530DBF0582; Thu, 29 Jun 2000 21:44:20 -0300 (EST) Message-ID: <395BECC4.69C9B15E@via-rs.net> Date: Thu, 29 Jun 2000 21:41:41 -0300 From: Lauro Fernando Costa Barbosa X-Mailer: Mozilla 4.72 [en] (Win95; I) X-Accept-Language: en MIME-Version: 1.0 To: Jonathan Chen Cc: freebsd-smp@FreeBSD.ORG Subject: Re: HELP WITH DELL MACHINES DUAL PENTIUM III !!!!!!!!! References: <395A0888.AC959A3E@via-rs.net> <20000629084516.B3160@jonc.ntdns.wilsonandhorton.co.n> Content-Type: multipart/mixed; boundary="------------A99383EF1F6AB8832A8BEA86" Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org This is a multi-part message in MIME format. --------------A99383EF1F6AB8832A8BEA86 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit My original message : Hi All! Please, I need help with SMP in DELL Machine PowerEdge 4400 with two processors PIII 677 MHz. I have configureted my Freebsd with SMP in config file and compiled, but when I reboot my system,my Freebsd can detect 2 cpu ,but after it detect SIO ,my Freebsd is deadlock, stoped forever. Anybody can help me ?? Thanks in advance, Lauro. My configfile : machine i386 #cpu I386_CPU #cpu I486_CPU #cpu I586_CPU cpu I686_CPU ident DELL1 maxusers 64 #makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols options MATH_EMULATE #Support for x87 emulation options INET #InterNETworking options INET6 #IPv6 communications protocols options FFS #Berkeley Fast Filesystem options FFS_ROOT #FFS usable as root device [keep this!] options MFS #Memory Filesystem options MD_ROOT #MD is a potential root device options NFS #Network Filesystem options NFS_ROOT #NFS usable as root device, NFS required options MSDOSFS #MSDOS Filesystem options CD9660 #ISO 9660 Filesystem options CD9660_ROOT #CD-ROM usable as root, CD9660 required options PROCFS #Process filesystem options COMPAT_43 #Compatible with BSD 4.3 [KEEP THIS!] options SCSI_DELAY=15000 #Delay (in ms) before probing SCSI options UCONSOLE #Allow users to grab the console options USERCONFIG #boot -c editor options VISUAL_USERCONFIG #visual boot -c editor options KTRACE #ktrace(1) support options SYSVSHM #SYSV-style shared memory options SYSVMSG #SYSV-style message queues options SYSVSEM #SYSV-style semaphores options P1003_1B #Posix P1003_1B real-time extentions options _KPOSIX_PRIORITY_SCHEDULING options ICMP_BANDLIM #Rate limit bad replies # To make an SMP kernel, the next two are needed options SMP # Symmetric MultiProcessor Kernel options APIC_IO # Symmetric (APIC) I/O # Optionally these may need tweaked, (defaults shown): options NCPU=2 # number of CPUs #options NBUS=4 # number of busses options NBUS=14 # number of busses #options NAPIC=1 # number of IO APICs options NAPIC=2 # number of IO APICs #options NINTR=24 # number of INTs options NINTR=46 # number of INTs device isa device eisa device pci # Floppy drives device fdc0 at isa? port IO_FD1 irq 6 drq 2 device fd0 at fdc0 drive 0 device fd1 at fdc0 drive 1 # ATA and ATAPI devices device ata0 at isa? port IO_WD1 irq 14 device ata1 at isa? port IO_WD2 irq 15 device ata device atadisk # ATA disk drives device atapicd # ATAPI CDROM drives device atapifd # ATAPI floppy drives device atapist # ATAPI tape drives options ATA_STATIC_ID #Static device numbering #options ATA_ENABLE_ATAPI_DMA #Enable DMA on ATAPI devices # SCSI Controllers #device ahb # EISA AHA1742 family device ahc # AHA2940 and onboard AIC7xxx devices #device amd # AMD 53C974 (Teckram DC-390(T)) #device dpt # DPT Smartcache - See LINT for options! #device isp # Qlogic family #device ncr # NCR/Symbios Logic #device sym # NCR/Symbios Logic (newer chipsets) #device adv0 at isa? #device adw #device bt0 at isa? #device aha0 at isa? #device aic0 at isa? # SCSI peripherals device scbus # SCSI bus (required) device da # Direct Access (disks) device sa # Sequential Access (tape etc) device cd # CD device pass # Passthrough device (direct SCSI access) # RAID controllers #device ida # Compaq Smart RAID #device amr # AMI MegaRAID #device mlx # Mylex DAC960 family # atkbdc0 controls both the keyboard and the PS/2 mouse device atkbdc0 at isa? port IO_KBD device atkbd0 at atkbdc? irq 1 device psm0 at atkbdc? irq 12 device vga0 at isa? # splash screen/screen saver pseudo-device splash # syscons is the default console driver, resembling an SCO console device sc0 at isa? # Enable this and PCVT_FREEBSD for pcvt vt220 compatible console driver #device vt0 at isa? #options XSERVER # support for X server on a vt console #options FAT_CURSOR # start with block cursor # If you have a ThinkPAD, uncomment this along with the rest of the PCVT lines #options PCVT_SCANSET=2 # IBM keyboards are non-std # Floating point support - do not disable. device npx0 at nexus? port IO_NPX irq 13 # Power management support (see LINT for more options) device apm0 at nexus? disable flags 0x20 # Advanced Power Management # PCCARD (PCMCIA) support device card device pcic0 at isa? irq 10 port 0x3e0 iomem 0xd0000 device pcic1 at isa? irq 11 port 0x3e2 iomem 0xd4000 disable # Serial (COM) ports device sio0 at isa? port IO_COM1 flags 0x10 irq 4 device sio1 at isa? port IO_COM2 irq 3 #device sio2 at isa? disable port IO_COM3 irq 5 #device sio3 at isa? disable port IO_COM4 irq 9 # Parallel port device ppc0 at isa? irq 7 device ppbus # Parallel port bus (required) device lpt # Printer device plip # TCP/IP over parallel device ppi # Parallel port interface device #device vpo # Requires scbus and da # PCI Ethernet NICs. #device de # DEC/Intel DC21x4x (``Tulip'') device fxp # Intel EtherExpress PRO/100B (82557, 82558) #device tx # SMC 9432TX (83c170 ``EPIC'') #device vx # 3Com 3c590, 3c595 (``Vortex'') #device wx # Intel Gigabit Ethernet Card (``Wiseman'') # PCI Ethernet NICs that use the common MII bus controller code. #device miibus # MII bus support #device dc # DEC/Intel 21143 and various workalikes #device rl # RealTek 8129/8139 #device sf # Adaptec AIC-6915 (``Starfire'') #device sis # Silicon Integrated Systems SiS 900/SiS 7016 #device ste # Sundance ST201 (D-Link DFE-550TX) #device tl # Texas Instruments ThunderLAN #device vr # VIA Rhine, Rhine II #device wb # Winbond W89C840F #device xl # 3Com 3c90x (``Boomerang'', ``Cyclone'') # ISA Ethernet NICs. #device ed0 at isa? port 0x280 irq 10 iomem 0xd8000 #device ex #device ep # WaveLAN/IEEE 802.11 wireless NICs. Note: the WaveLAN/IEEE really # exists only as a PCMCIA device, so there is no ISA attatement needed # and resources will always be dynamically assigned by the pccard code. #device wi # Aironet 4500/4800 802.11 wireless NICs. Note: the declaration below will # work for PCMCIA and PCI cards, as well as ISA cards set to ISA PnP # mode (the factory default). If you set the switches on your ISA # card for a manually chosen I/O address and IRQ, you must specify # those paremeters here. #device an # The probe order of these is presently determined by i386/isa/isa_compat.c. device ie0 at isa? port 0x300 irq 10 iomem 0xd0000 device fe0 at isa? port 0x300 device le0 at isa? port 0x300 irq 5 iomem 0xd0000 device lnc0 at isa? port 0x280 irq 10 drq 0 device cs0 at isa? port 0x300 device sn0 at isa? port 0x300 irq 10 # requires PCCARD (PCMCIA) support to be activated #device xe0 at isa? # Pseudo devices - the number indicates how many units to allocated. pseudo-device loop # Network loopback pseudo-device ether # Ethernet support pseudo-device sl 1 # Kernel SLIP pseudo-device ppp 1 # Kernel PPP pseudo-device tun # Packet tunnel. pseudo-device pty # Pseudo-ttys (telnet etc) pseudo-device md # Memory "disks" pseudo-device gif 4 # IPv6 and IPv4 tunneling pseudo-device faith 1 # IPv6-to-IPv4 relaying (translation) # The `bpf' pseudo-device enables the Berkeley Packet Filter. # Be aware of the administrative consequences of enabling this! pseudo-device bpf #Berkeley packet filter # USB support #device uhci # UHCI PCI->USB interface #device ohci # OHCI PCI->USB interface #device usb # USB Bus (required) #device ugen # Generic #device uhid # "Human Interface Devices" #device ukbd # Keyboard #device ulpt # Printer #device umass # Disks/Mass storage - Requires scbus and da #device ums # Mouse # USB Ethernet, requires mii #device aue # ADMtek USB ethernet #device cue # CATC USB ethernet #device kue # Kawasaki LSI USB ethernet The boot process stop immedatily after it detect SIO , sio0 and sio1. Jonathan Chen wrote: > On Wed, Jun 28, 2000 at 11:15:36AM -0300, Lauro Barbosa wrote: > > > > Hi All! > > Please, I need help with SMP in DELL Machine PowerEdge 4400 with two > > processors PIII 677 MHz. > > I have configureted my Freebsd with SMP in config file and compiled, but > > when I reboot my system,my Freebsd can detect 2 cpu ,but after it detect > > SIO ,my Freebsd is deadlock, stoped forever. > > Anybody can help me ?? > > Attach your config file, and if possible the output of the console > just before it hangs; you may need to Cc: freebsd-smp with the > details. > -- > Jonathan Chen > ---------------------------------------------------------------------- > "If you wish your merit to be known, acknowledge that of other people" --------------A99383EF1F6AB8832A8BEA86 Content-Type: text/x-vcard; charset=us-ascii; name="lauro.vcf" Content-Transfer-Encoding: 7bit Content-Description: Card for Lauro Fernando Costa Barbosa Content-Disposition: attachment; filename="lauro.vcf" begin:vcard n:Barbosa;Lauro tel;work:Net Sul x-mozilla-html:FALSE adr:;;;;;; version:2.1 email;internet:lauro@pro.via-rs.com.br note;quoted-printable:"O casamento =E9 o pre=E7o que os homens pagam pelo sexo=3B o sexo =E9 o pre=E7o=0D=0A que as mulheres pagam pelo casamento."=0D=0A AN=D4NIMO fn:Lauro Barbosa end:vcard --------------A99383EF1F6AB8832A8BEA86-- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message From owner-freebsd-smp Thu Jun 29 18:42:19 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mailhost.iprg.nokia.com (mailhost.iprg.nokia.com [205.226.5.12]) by hub.freebsd.org (Postfix) with ESMTP id 212CA37C2AF for ; Thu, 29 Jun 2000 18:42:15 -0700 (PDT) (envelope-from michaelw@iprg.nokia.com) Received: from darkstar.iprg.nokia.com (darkstar.iprg.nokia.com [205.226.5.69]) by mailhost.iprg.nokia.com (8.9.3/8.9.3-GLGS) with ESMTP id SAA06225 for ; Thu, 29 Jun 2000 18:42:14 -0700 (PDT) Received: (from root@localhost) by darkstar.iprg.nokia.com (8.9.3/8.9.3-VIRSCAN) id SAA26993 for ; Thu, 29 Jun 2000 18:42:10 -0700 X-Virus-Scanned: Thu, 29 Jun 2000 18:42:10 -0700 Nokia Silicon Valley Email Exploit Scanner Received: from (brakescr.iprg.nokia.com [205.226.1.186]) by darkstar.iprg.nokia.com SMTP/WTS (12.69) xma020227; Thu, 29 Jun 00 18:39:21 -0700 Message-ID: <395BFA4C.783551DF@iprg.nokia.com> Date: Thu, 29 Jun 2000 18:39:24 -0700 From: Michael Glenn Williams Organization: NOKIA X-Mailer: Mozilla 4.72 [en] (X11; U; FreeBSD 3.4-RELEASE i386) X-Accept-Language: en MIME-Version: 1.0 To: "freebsd-smp@FreeBSD.ORG" Subject: benchmarks Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Do benchmarks of Linux, FreeBSD and Solaris on IA32 1cpu through 8cpu exist? All suggestions welcome. Tx! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message