From owner-freebsd-current@FreeBSD.ORG Mon Jul 21 16:27:27 2003 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7F3D637B401 for ; Mon, 21 Jul 2003 16:27:27 -0700 (PDT) Received: from franky.speednet.com.au (franky.speednet.com.au [203.57.65.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5903D43F85 for ; Mon, 21 Jul 2003 16:27:26 -0700 (PDT) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h6LNRHDs068406; Tue, 22 Jul 2003 09:27:18 +1000 (EST) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [172.22.2.17])h6LNREPd078658; Tue, 22 Jul 2003 09:27:15 +1000 (EST) (envelope-from andyf@speednet.com.au) Date: Tue, 22 Jul 2003 09:27:14 +1000 (EST) From: Andy Farkas X-X-Sender: andyf@hewey.af.speednet.com.au To: Terry Lambert In-Reply-To: <3F0BC75B.12BF42CF@mindspring.com> Message-ID: <20030722084135.F74931-100000@hewey.af.speednet.com.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Julian Elischer cc: FreeBSD current users Subject: Re: SMP and setrunnable()- scheduler 4bsd X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2003 23:27:27 -0000 On Wed, 9 Jul 2003, Terry Lambert wrote: > Andy Farkas wrote: > > On Tue, 8 Jul 2003, Julian Elischer wrote: > > > It looks tp me that if we make a thread runnable > > > and there is a processor in the idle loop, the idle processor should be > > > kicked in some way to make it go get the newly runnable thread. > > > > Is this what's happenning to me an my setiathomes? > > The command: > > sysctl machdep.cpu_idle_hlt > > will tell you. If it says 1, then it may be the problem. If it > says 0, then it's not the problem. You can explicitly set it to > zero to disable halting in the idle loop. If you do this, your > machine is likely to run ~20% hotter (depending on the CPU type), > since HLT'ing an idle CPU tends to cool it off. > > If this fixes your problem, then it's likely that what's happening > is that one or more of your CPU's are being idled until the clock > or some other interrupt fires, at which point in time your setiathome > processes are probably not the highest priority, as other things with > higher priority have gotten in the run queue ahead of them. > > If this is the case, then Julian's suggested fix of an IPI to one or > more idle CPU's when a process becomes ready-to-run will most likely > be necessary to avoid this situation. Well, looks like machdep.cpu_idle_hlt affects the 4bsd scheduler. Julian, have your patches at http://www.freebsd.org/~julian/it.diff been incorporated yet? SHould I try them? hummer# sysctl -a | grep kern.ver kern.version: FreeBSD 5.1-RELEASE #0: Thu Jun 26 01:57:42 EST 2003 hummer# sysctl -a | grep cpu kern.threads.virtual_cpu: 4 kern.ccpu: 1948 <--- whats this? kern.smp.cpus: 4 hw.ncpu: 4 machdep.cpu_idle_hlt: 0 machdep.hlt_cpus: 0 hummer# hummer# vmstat -c 12 -w 10 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad0 am0 in sy cs us sy id 1 1 3 124804 20792 99 0 0 0 131 20 0 0 362 0 544 33 3 64 2 1 2 125240 32460 197 0 0 0 358 258 0 1 365 0 2161 67 26 7 2 1 2 124728 32204 214 0 0 0 249 0 0 1 382 0 1280 72 23 5 1 1 3 124112 29652 257 0 0 0 247 0 3 1 414 0 1571 70 25 5 3 1 1 124624 26020 236 0 0 0 297 0 0 1 373 0 1367 70 24 7 2 1 2 124792 25464 222 0 0 0 266 0 2 1 408 0 1308 72 24 4 3 1 1 123764 25172 203 0 0 0 227 0 0 1 380 0 1053 72 23 5 3 1 2 124164 22004 229 0 0 0 264 0 0 1 371 0 1164 72 25 3 3 1 2 124796 20232 208 0 0 0 262 0 1 1 383 0 1269 72 23 5 3 1 1 123768 20448 225 0 0 0 252 0 0 1 374 0 1306 72 25 3 3 1 0 122736 33736 214 0 0 0 458 194 0 1 363 0 1341 72 24 4 3 1 1 123380 32008 236 0 0 0 222 0 2 1 405 0 1430 72 25 4 hummer# hummer# sysctl machdep.cpu_idle_hlt=1 machdep.cpu_idle_hlt: 0 -> 1 hummer# vmstat -c 12 -w 10 procs memory page disks faults cpu r b w avm fre flt re pi po fr sr ad0 am0 in sy cs us sy id 2 1 1 124760 20848 99 0 0 0 131 20 0 0 362 0 544 33 3 64 1 1 3 123948 20844 157 0 0 0 191 0 0 1 385 0 1142 51 21 28 2 1 2 124408 20472 134 0 0 0 176 0 0 1 372 0 1118 48 22 30 2 1 1 124992 18800 179 0 0 0 181 0 1 2 394 0 1147 49 20 31 2 2 3 123792 31228 162 0 0 0 367 196 0 1 378 0 1111 48 21 30 1 1 4 124348 30048 175 0 0 0 169 0 0 1 368 0 1084 51 22 27 2 0 4 122228 29140 142 0 0 0 193 0 1 2 399 0 1056 53 21 26 2 1 2 124688 25324 176 0 0 0 135 0 0 1 370 0 1138 47 21 31 2 1 1 122632 26572 136 0 0 0 209 0 0 1 374 0 1110 50 21 30 2 1 2 122288 26036 162 0 0 0 193 0 1 2 384 0 1152 49 22 29 2 1 2 122288 25204 136 0 0 0 157 0 0 1 369 0 1101 50 21 29 2 1 2 122700 23720 172 0 0 0 183 0 0 1 376 0 1123 48 21 31 Currently (cpu_idle_hlt=1) the load is fluctuating between 2.20 and 3.60 every few minutes! (xload looks like a graph of a sinewave) If I set cpu_idle_hlt back to 0 the load goes back to a steady 3.80 where it should be. -- :{ andyf@speednet.com.au Andy Farkas System Administrator Speednet Communications http://www.speednet.com.au/