From owner-freebsd-arch Wed Feb 27 15:49:21 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 29FD337B402 for ; Wed, 27 Feb 2002 15:49:15 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1RNnDF31841; Wed, 27 Feb 2002 15:49:13 -0800 (PST) (envelope-from dillon) Date: Wed, 27 Feb 2002 15:49:13 -0800 (PST) From: Matthew Dillon Message-Id: <200202272349.g1RNnDF31841@apollo.backplane.com> To: Bosko Milekic Cc: Jeff Roberson , arch@FreeBSD.ORG Subject: Re: Slab allocator References: <20020227172755.W59764-100000@mail.chesapeake.net> <200202272307.g1RN75T31581@apollo.backplane.com> <20020227183124.B62898@unixdaemons.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :.. :> FAST interrupts do not usually call into other subsystems, to :> prevent unexpected alterations of the per-cpu data. They should :> not be thought of as a mechanism that blocks interrupts. : : Well, we're going to be doing context-stealing soon, don't forget :about that. That means that if we're in a critical section, we won't be :able to take advantage of stealing by taking the context and running the :handler immediately. So, to schedule an ithread we will need to get some :sort of sched_lock. Sure, perhaps it won't be _the_ sched_lock, but it :will have to be some sort of lock for the run queue we will be placing :our ithread on when not performing context stealing. Personally, I don't :believe that having the fast-path allocation be a critical section. :We're headed in the pre-emption path and disabling pre-emption every :time I want to allocate something is pretty unreasonable [*]. : :[*] Yes, I know that you don't like pre-emption. : :-- :Bosko Milekic If an i-scheduled interrupt is blocked by a critical section which only lasts 1 uS, there will be no detrimental effect on system performance. If our codebase is pockmarked with thousands of little 1uS-long critical sections it will have no cumulative effect whatsoever on interrupt latencies. None. This may seem counter-intuitive but it's true, and anyone who has ever written embedded software will tell you the same thing. It isn't the number of places you disable preemption that kills you, it's the LONGEST place you disable preemption that kills you. It's true on everything from a little 8 bit microcomputer to an IBM mainframe. Or, to put it another way, every time the system takes an interrupt it disrupts code flow and pollutes the L1 and L2 caches of the cpu if the cpu is otherwise doing something useful (like running a system call). If an interrupt does not require real time response, there is no point disrupting the system in order to give it real time response. The system will actually operate more efficiently if you allow it to continue whatever it was already doing, at least for a little while. This doesn't mean that preemption is bad, just that it is bad if you go overboard trying to do it. Being rabid about critical sections classifies as "going overboard". Critical sections can save far, far more clock cycles then non-conflicting mutexes without any effect on interrupt performance if used properly. Also keep in mind that a critical section != sched_lock. Just using a critical section in something like the SLAB allocator does not have the same detrimental effect that holding the sched_lock (which obtains a critical section) has. Bruce has demonstrated this and it should be an obvious truism to anyone in SMP land. We need to work on reducing the time we hold sched_lock, but that is a very different issue then using a critical section to protect a per-process cache. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message