From owner-freebsd-stable@FreeBSD.ORG Mon Feb 23 10:08:38 2009 Return-Path: Delivered-To: stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A2D4E106564A; Mon, 23 Feb 2009 10:08:38 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 60C858FC15; Mon, 23 Feb 2009 10:08:38 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id 057CE46B09; Mon, 23 Feb 2009 05:08:38 -0500 (EST) Date: Mon, 23 Feb 2009 10:08:37 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Maxim Sobolev In-Reply-To: <49A23C1A.3070403@FreeBSD.org> Message-ID: References: <49A23C1A.3070403@FreeBSD.org> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Jeff Roberson , "current@freebsd.org" , stable@FreeBSD.org Subject: Re: The machdep.hyperthreading_allowed & ULE weirdness in 7.1 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Feb 2009 10:08:39 -0000 On Sun, 22 Feb 2009, Maxim Sobolev wrote: > Hi Jeff, > > I have a single-CPU system with P4 HTT-enabled processor (7.1-RELEASE-p3), > kernel compiled with SCHED_ULE. This is because machdep.hlt_logical_cpus doesn't do what you think it does. It causes HTT cores to invoke the hlt instruction in their idle loop, causing them to sleep until they receive an interrupt. For 4BSD, which uses a "pull" model (on the whole) to bring work from work queues, this means that CPUs will go to sleep and remain that way unless they're actively receiving interrupts. For ULE, which uses "push" as well as "pull", threads will constantly be being shed from too-busy CPUs to apparently idle ones under load. The only reliable way to disable hyperthreading is to do so using your BIOS setting, or use loader.conf to disable probing of the pics on unwanted cores, which will cause the CPUs not to be enumerated and hence not used. We don't support taking CPUs fully offline with any scheduler, although you can use the cpuset facility to prevent threads from running on specific cores. You could imagine teaching ULE (and presumably 4BSD) about policies such as "Don't use HTT cores", or perhaps just cpuset about those policies. Robert N M Watson Computer Laboratory University of Cambridge > > Copyright (c) 1992-2009 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 7.1-RELEASE-p3 #0: Fri Feb 20 09:53:32 UTC 2009 > root@foo.com:/usr/obj/usr/src/sys/SSP-PRODUCTION7 > Timecounter "i8254" frequency 1193182 Hz quality 0 > CPU: Intel(R) Pentium(R) 4 CPU 3.00GHz (2992.51-MHz 686-class CPU) > Origin = "GenuineIntel" Id = 0xf34 Stepping = 4 > > Features=0xbfebfbff > Features2=0x441d > Logical CPUs per core: 2 > real memory = 3758030848 (3583 MB) > avail memory = 3674083328 (3503 MB) > ACPI APIC Table: > FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs > cpu0 (BSP): APIC ID: 0 > cpu1 (AP): APIC ID: 1 > ioapic0: Changing APIC ID to 2 > ioapic0 irqs 0-23 on motherboard > ioapic1 irqs 24-47 on motherboard > > machdep.cpu_idle_hlt: 1 > machdep.hlt_cpus: 0 > machdep.hlt_logical_cpus: 0 > machdep.logical_cpus_mask: 2 > > If I flip machdep.hyperthreading_allowed sysctl, I still can see processes > scheduled to both logical CPUs, yet, if I check IRQ status in the systat -vm, > the CPU1 is not getting any timer interrupts, while CPU0 gets twice the > normal amount. I wonder if something is broken - I would expect no processes > to be scheduled to the CPU1. > > machdep.hyperthreading_allowed=1: > top: http://sobomax.sippysoft.com/~sobomax/ScreenShot458.png > systat -vm: http://sobomax.sippysoft.com/~sobomax/ScreenShot459.png > > machdep.hyperthreading_allowed=0: > top: http://sobomax.sippysoft.com/~sobomax/ScreenShot460.png > systat -vm: http://sobomax.sippysoft.com/~sobomax/ScreenShot461.png > > Please let me know if any other debug information is needed. > > -Maxim > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" >