From owner-freebsd-bugs@FreeBSD.ORG Fri Nov 19 00:10:12 2010 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CF3BD106564A for ; Fri, 19 Nov 2010 00:10:12 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id A4E1E8FC14 for ; Fri, 19 Nov 2010 00:10:12 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id oAJ0ACvn095955 for ; Fri, 19 Nov 2010 00:10:12 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id oAJ0ACwk095952; Fri, 19 Nov 2010 00:10:12 GMT (envelope-from gnats) Date: Fri, 19 Nov 2010 00:10:12 GMT Message-Id: <201011190010.oAJ0ACwk095952@freefall.freebsd.org> To: freebsd-bugs@FreeBSD.org From: Garrett Cooper Cc: Subject: Re: kern/145385: [cpu] Logical processor cannot be disabled for some SMT-enabled Intel procs X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Garrett Cooper List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Nov 2010 00:10:13 -0000 The following reply was made to PR kern/145385; it has been noted by GNATS. From: Garrett Cooper To: Andriy Gapon Cc: bug-followup@freebsd.org Subject: Re: kern/145385: [cpu] Logical processor cannot be disabled for some SMT-enabled Intel procs Date: Thu, 18 Nov 2010 16:09:46 -0800 On Wed, Nov 17, 2010 at 4:34 PM, Garrett Cooper wrote= : > On Thu, Oct 7, 2010 at 10:03 AM, Andriy Gapon wrote: >> >> Here's a patch to remove halted logical processors from root cpu set (cp= uset >> number zero) and consequently all child sets: >> http://people.freebsd.org/~avg/cpu-halt.diff >> >> Please note that unhalting CPUs will add them to cpuset zero, but will n= ot >> (re-)add them cpusets of the running processes. =A0So additional action = will be >> required from system administrator if e would like existing processes to= make use >> of unhalted CPUs. >> >> Also, interrupts that are bound to halted CPUs are not rebound on halt, = and so >> will be delivered to halted CPUs. =A0This should not be a problem for ty= pical >> environments, but would be nice to fix anyway. > > =A0 =A0Sorry for taking so long to get back on this item. The patch looks > ok as far as setting the CPUSET is concerned (setting > machdep.hlt_logical_cpus=3D1 works on an r710 with 8 logical procs as > SCHED_ULE schedules all of the tasks on the non-logical SMT cores, not > the logical ones, and the patch properly detects that there aren't any > logical processors on a Core2 Quad I have at home), but it's missing a > key case where I go from... > > machdep.hlt_logical_cpus: 1 -> 0 > > =A0 =A0... it should reset CPUSETZERO. > =A0 =A0The overall CPU topology should probably be cached and restored > when reset, or at least CPUSETZERO should be reset (probably the > simpler and cleaner route to go). > =A0 =A0After that all that's required is work for i386 and I'd vote for a > commit of the patch. > Thanks for the hard work Andriy :)! Just to illustrate, here's how I reproduced the successful behavior (and the problem): 1. Execute the following script: #!/bin/sh i=3D0 ncpus=3D`sysctl -n kern.smp.cpus` while [ $i -lt $ncpus ] ; do while : ; do : ; done & : $(( i +=3D 1 )) done 2. Execute sysctl machdep.hlt_logical_cpus=3D1 . 3. Look at cpuset info; example: Before: # for i in `ps aux | grep gcooper | grep -v grep | awk '{ print $2 }'` ; do cpuset -g -p $i ; done pid 2179 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 pid 2180 mask: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 After: # for i in `ps aux | grep gcooper | grep -v grep | awk '{ print $2 }'` ; do cpuset -g -p $i ; done pid 2179 mask: 0, 2, 4, 6, 8, 10, 12, 14 pid 2180 mask: 0, 2, 4, 6, 8, 10, 12, 14 # sysctl machdep.hlt_logical_cpus=3D0 machdep.hlt_logical_cpus: 1 -> 0 # for i in `ps aux | grep gcooper | grep -v grep | awk '{ print $2 }'` ; do cpuset -g -p $i ; done pid 2179 mask: 0, 2, 4, 6, 8, 10, 12, 14 pid 2180 mask: 0, 2, 4, 6, 8, 10, 12, 14 I'm taking the patch and running with it now to try and get it working 100%= .