From owner-freebsd-smp Wed May 1 21: 7:20 2002 Delivered-To: freebsd-smp@freebsd.org Received: from fw.wemm.org (12-232-135-171.client.attbi.com [12.232.135.171]) by hub.freebsd.org (Postfix) with ESMTP id 7A37B37B416; Wed, 1 May 2002 21:07:17 -0700 (PDT) Received: from overcee.wemm.org (overcee.wemm.org [10.0.0.3]) by fw.wemm.org (8.11.6/8.11.6) with ESMTP id g4247H403899; Wed, 1 May 2002 21:07:17 -0700 (PDT) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by overcee.wemm.org (Postfix) with ESMTP id CE4BA38CC; Wed, 1 May 2002 21:07:16 -0700 (PDT) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: Andrew Gallatin Cc: John Baldwin , freebsd-smp@FreeBSD.ORG Subject: Re: hlt when idle? In-Reply-To: <15568.20086.979721.992191@grasshopper.cs.duke.edu> Date: Wed, 01 May 2002 21:07:16 -0700 From: Peter Wemm Message-Id: <20020502040716.CE4BA38CC@overcee.wemm.org> Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Andrew Gallatin wrote: > > John Baldwin writes: > > > > On 01-May-2002 Andrew Gallatin wrote: > > > > > > > > > Can somebody remind me why we do not hlt in the idle loop on MP x86s? > > > Is this because a HLTed CPU is not going to notice a new runnable job > > > (possibly migrating from another CPU) until it gets an interrupt to > > > wake it up? > > > > Yes. > > This seems to be an acceptable "loss" in performance in environments > where cooling is a concern. Is there a deadlock danger? Or is it > just a performance tweak to not HLT SMPs? Would you object to making > it a sysctl (machdep.smp_idle_hlt)? There is no deadlock danger. The only "problem" is that if you have one cpu doing a wakeup() on another process and putting it on the run queue, nothing happens to jolt the other cpu out of the halt state so that it notices that there is work to do. So, on one hand you have the problem where you have a runnable process but the other cpu is asleep till the next interrupt. And on the other hand, right now we have the cpus hammering the cache line that holds the run queue status bits. Since you reported a 15 second speedup in your test, perhaps the spinning cpu is causing more contention than we save by responding immediately to a runnable process being added. IMHO, it is probably worth just putting the halts in and be done with it. Firing off an IPI to wake up cpus is the correct fix though. The catch is that is expensive so you have to know when to do it and when not to. ie: keep some sort of idle/busy state and have the add-run-queue checks notice that they added a new runnable process (vs previously idle) and notice that there is an idle cpu that could pick it up and fire an IPI in its direction. Only experimentation will tell us if this is worth it. Cheers, -Peter -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message