From owner-freebsd-smp Thu May 2 0:50:33 2002 Delivered-To: freebsd-smp@freebsd.org Received: from harrier.prod.itd.earthlink.net (harrier.mail.pas.earthlink.net [207.217.120.12]) by hub.freebsd.org (Postfix) with ESMTP id 0CD3137B41B; Thu, 2 May 2002 00:50:04 -0700 (PDT) Received: from pool0052.cvx21-bradley.dialup.earthlink.net ([209.179.192.52] helo=mindspring.com) by harrier.prod.itd.earthlink.net with esmtp (Exim 3.33 #2) id 173BLZ-0001zO-00; Thu, 02 May 2002 00:49:49 -0700 Message-ID: <3CD0EF79.FC1DA1A8@mindspring.com> Date: Thu, 02 May 2002 00:49:13 -0700 From: Terry Lambert X-Mailer: Mozilla 4.7 [en]C-CCK-MCD {Sony} (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Peter Wemm Cc: Andrew Gallatin , John Baldwin , freebsd-smp@FreeBSD.ORG Subject: Re: hlt when idle? References: <20020502040716.CE4BA38CC@overcee.wemm.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Peter Wemm wrote: > IMHO, it is probably worth just putting the halts in and be done with it. > Firing off an IPI to wake up cpus is the correct fix though. The catch is > that is expensive so you have to know when to do it and when not to. ie: > keep some sort of idle/busy state and have the add-run-queue checks notice > that they added a new runnable process (vs previously idle) and notice that > there is an idle cpu that could pick it up and fire an IPI in its > direction. Only experimentation will tell us if this is worth it. Yeah. I looked at doing the IPI thing back in 1996 with Jack Vogel's original FreeBSD SMP code from October of 1995, and it turned into an incredible mess really quickly. I know a lot more now than I did then, but I'd still hesitate to do it, unless I was being paid to bash my head against the wall until the wall broke. 8-(. The one comment I'd make about "just putting the halts in" is that it becomes a progressively worse trade-off the more CPUs there are in the machine. Of course, it's not like anyone is building a lot of large CPU count systems, like back in the Sequent 32 CPU machine days, but I know of at least two 8 CPU boxes that might lose based on adding in the HLT. The real problem is that when you have a crapload of CPUs, you generally have them for a reason, which involves shared state, which means multiple sleepers on shared resources. So you may be missing the wakeup on 6 processes on a shared wait. That can really add up, and it's not going to show up in a heterogenous load, like you get from a "make world". For all we know, a homogeneous load could show negative effects at 4 or even 2 CPUs. 8-(. Maybe it could be made into a compile time option? My preference would be to default it "off", to encourage someone to bash their head on "the right fix" (with a comment to that effect, so that someone doesn't "fix it"; have to have learned something from the ATA write caching default state toggle war...). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message