From owner-freebsd-smp Fri Jul 7 8:57:58 2000 Delivered-To: freebsd-smp@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 15F9737BF9A for ; Fri, 7 Jul 2000 08:57:26 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id IAA00648; Fri, 7 Jul 2000 08:54:37 -0700 (PDT) (envelope-from dillon) Date: Fri, 7 Jul 2000 08:54:37 -0700 (PDT) From: Matthew Dillon Message-Id: <200007071554.IAA00648@apollo.backplane.com> To: Chuck Paterson Cc: Doug Rabson , Greg Lehey , David Greenman , freebsd-smp@FreeBSD.ORG Subject: References: <200007071516.JAA12975@berserker.bsdi.com> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org :One disadvantage is that the actual mechanism to implement :the lazy interrupts is going to be a bitch to actually make work :right/well on multiple processors in parallel, further complicated :by the light weight context switches. Also I'm pretty sure interrupts :don't want to get pended to by the pic level, but rather on individual :apic pins. : :A second disadvantage is that interrupts will get randomly delivered to :a processor holding a spin mutex rather than run immediately on a :processor not holding a spin mutex. This is a problem with the :current scheme also, and not fixed with masks and ipending. Well, the first isn't a disadvantage, since I implemented it a few weeks ago. It was trivial. The second one doesn't apply to cli/sti verses ipending. I don't see any relationship. Interrupt delivery is not controlled by cli/sti, it's controlled by the APIC. :Having said this, it may be that Matt is correct and in the long :term we will want to ditch the cli/sti's in mutices. I really hope :we don't have to go down the lazy path, my mind goes numb just :thinking about the corner cases. : :Some random data points. : : The assumption that the scheduler lock will be the only : spin mutex is wrong. BSD/OS currently has about 5 which : are pretty much architecture independent. Most/all of these : are at one time or another acquired while the scheduler : lock is held. A per-process spin-held counter would address this both for the schedular mutex and any other spin mutex. : On Sparc BSD/OS already has the notion of a priority : level associated with a spin lock. Interrupts are blocked : by changing the interrupting block level, which on Sparc : is very cheap. : : When a processor is cli'd IPIs are not delivered. In general : this made life lots easier in BSD/OS. It is possible that : a condition will arise where is it required to deliver : an IPI while another processor holds a spin lock. This : did occur on Sparc with some low cache code (stuff done : in hardware on Intel). This particular IPI is delivered : at a level higher than just acquiring a mutex will not block. IPI's could be an issue since they don't have equivalent interrupt bits in irunning or ipending, and so can't be defered. I don't know of any FreeBSD IPIs that can't 'just run', with or without the scheduler mutex. In this case IPIs would not have to be defered even if the scheduler mutex were held by the current cpu. The only thing we use IPIs for seriously are VM page operations, to invalidate pte's on other cpu's, and for interrupt forwarding (which never worked quite right anyway). The former occurs only while Giant is held. :... lots of good stuff removed :Chuck : -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message