From owner-freebsd-smp@FreeBSD.ORG Mon Jul 7 16:33:19 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0B3D037B404; Mon, 7 Jul 2003 16:33:19 -0700 (PDT) Received: from franky.speednet.com.au (franky.speednet.com.au [203.57.65.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id F226743FAF; Mon, 7 Jul 2003 16:33:17 -0700 (PDT) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h67NXGsw078454; Tue, 8 Jul 2003 09:33:16 +1000 (EST) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h67NXF2b006826; Tue, 8 Jul 2003 09:33:16 +1000 (EST) (envelope-from andyf@speednet.com.au) Date: Tue, 8 Jul 2003 09:33:15 +1000 (EST) From: Andy Farkas X-X-Sender: andyf@hewey.af.speednet.com.au To: freebsd-current@FreeBSD.ORG Message-ID: <20030708090530.T6312-100000@hewey.af.speednet.com.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-smp@FreeBSD.ORG Subject: whats going on with the scheduler? X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2003 23:33:19 -0000 FreeBSD 5.1-RELEASE with SCHED_4BSD on a quad ppro 200 (dell 6100/200). Last night I started 3 setiathome's then went to bed. The system was otherwise idle and had a load of 3.00, 3.00, 3.00. This morning, I wanted to copy a (large) file from a remote server, so I did a: scp -c blowfish -p -l 100 remote.host:filename . which is running in another window (and will run for 3 more hours). And now, on my otherwise idle system, the load is varying from less than 2.00 (!) to just over 3.00, with an average average of about 2.50. Here is some output from top: PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND 42946 setiathome 139 15 15524K 14952K *Giant 0 39.9H 89.26% 89.26% setiathome 49332 andyf 130 0 3084K 2176K *Giant 2 81:49 67.68% 67.68% ssh 12 root -16 0 0K 12K CPU2 2 152.1H 49.12% 49.12% idle: cpu2 13 root -16 0 0K 12K CPU1 1 148.7H 44.58% 44.58% idle: cpu1 11 root -16 0 0K 12K RUN 3 152.1H 44.14% 44.14% idle: cpu3 14 root -16 0 0K 12K CPU0 0 143.3H 41.65% 41.65% idle: cpu0 42945 setiathome 129 15 15916K 14700K *Giant 2 39.0H 25.20% 25.20% setiathome 42947 setiathome 129 15 15524K 14956K *Giant 1 40.3H 22.61% 22.61% setiathome So, can someone explain why the seti procs are not getting 100% cpu like they were before the scp(ssh) started and why there is so much idle time? I bet those *Giants have something to do with it... -- :{ andyf@speednet.com.au Andy Farkas System Administrator Speednet Communications http://www.speednet.com.au/ From owner-freebsd-smp@FreeBSD.ORG Mon Jul 7 16:41:34 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 388E737B401 for ; Mon, 7 Jul 2003 16:41:34 -0700 (PDT) Received: from knight.ixsystems.net (iweb2.ixsystems.net [206.40.55.73]) by mx1.FreeBSD.org (Postfix) with ESMTP id A973643F3F for ; Mon, 7 Jul 2003 16:41:33 -0700 (PDT) (envelope-from matto@knight.ixsystems.net) Received: from knight.ixsystems.net (localhost [127.0.0.1]) by knight.ixsystems.net (8.12.9/8.11.6) with ESMTP id h67NSkQs025780 for ; Mon, 7 Jul 2003 16:28:46 -0700 (PDT) (envelope-from matto@knight.ixsystems.net) Received: (from matto@localhost) by knight.ixsystems.net (8.12.9/8.12.9/Submit) id h67NSkHC025779 for freebsd-smp@freebsd.org; Mon, 7 Jul 2003 16:28:46 -0700 (PDT) (envelope-from matto) Date: Mon, 7 Jul 2003 16:28:46 -0700 From: Matt Olander To: freebsd-smp@freebsd.org Message-ID: <20030707162846.D25165@knight.ixsystems.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i Subject: problem with new gigabyte dual xeon board X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jul 2003 23:41:34 -0000 hi, we've got a 1U system with a new Gigabyte GA-8EGXDR-HT motherboard. works fine with 4.8 and a single CPU. however, once we add SMP support to the kernel and compile, the system hangs on reboot after launching CPU1. this is with hyperthreading disabled. with HT enabled, the system halts once CPU3 is launched. any ideas? -- Matt Olander (408)943-4100 Phone (408)943-4101 Fax www.offmyserver.com -- "Those who don't read have no advantage over those who can't" -Mark Twain From owner-freebsd-smp@FreeBSD.ORG Mon Jul 7 17:43:09 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0456037B404 for ; Mon, 7 Jul 2003 17:43:09 -0700 (PDT) Received: from mx1.freebsdsystems.com (mx1.FreeBSDsystems.COM [216.138.197.66]) by mx1.FreeBSD.org (Postfix) with ESMTP id A52B943FB1 for ; Mon, 7 Jul 2003 17:43:07 -0700 (PDT) (envelope-from lnb@FreeBSDsystems.COM) Received: (qmail 43896 invoked by uid 0); 8 Jul 2003 00:43:03 -0000 Received: from unknown (HELO ?192.168.0.69?) (lnb@216.138.218.34) by mx1.freebsdsystems.com with AES256-SHA encrypted SMTP; 8 Jul 2003 00:43:03 -0000 From: Lanny Baron To: Andy Farkas In-Reply-To: <20030708090530.T6312-100000@hewey.af.speednet.com.au> References: <20030708090530.T6312-100000@hewey.af.speednet.com.au> Content-Type: text/plain Organization: FreeBSD Systems, Inc. Message-Id: <1057625020.18842.25.camel@panda.freebsdsystems.com> Mime-Version: 1.0 X-Mailer: Ximian Evolution 1.4.0 Date: 07 Jul 2003 20:43:40 -0400 Content-Transfer-Encoding: 7bit cc: freebsd-current@FreeBSD.ORG cc: freebsd-smp@FreeBSD.ORG Subject: Re: whats going on with the scheduler? X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2003 00:43:09 -0000 A load of 3 is pretty high. I think you have more going on. On one of our iNET Servers in Texas that does mail for several thousand people along with shells, radius etc. ...... last pid: 97534; load averages: 0.07, 0.03, 0.01 up 55+21:34:34 19:40:48 200 processes: 2 running, 198 sleeping CPU states: 0.0% user, 0.0% nice, 0.2% system, 0.2% interrupt, 99.6% idle Mem: 149M Active, 1513M Inact, 257M Wired, 76M Cache, 199M Buf, 8236K Free Swap: 750M Total, 750M Free PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND 97525 lnb 28 0 2180K 1476K CPU1 0 0:00 0.69% 0.29% top 329 root 2 0 2904K 1508K select 0 31:21 0.00% 0.00% smbd 313 root 2 0 10420K 9660K select 1 26:39 0.00% 0.00% radiusd 314 root 2 0 10412K 9624K select 1 24:25 0.00% 0.00% radiusd 305 qmails 2 0 1056K 632K select 0 19:09 0.00% 0.00% qmail-s 1497 smbd 2 0 3308K 2732K select 0 16:39 0.00% 0.00% eggdrop Lanny On Mon, 2003-07-07 at 19:33, Andy Farkas wrote: > FreeBSD 5.1-RELEASE with SCHED_4BSD on a quad ppro 200 (dell 6100/200). > > Last night I started 3 setiathome's then went to bed. The system was > otherwise idle and had a load of 3.00, 3.00, 3.00. > > This morning, I wanted to copy a (large) file from a remote server, so I > did a: > > scp -c blowfish -p -l 100 remote.host:filename . > > which is running in another window (and will run for 3 more hours). > > And now, on my otherwise idle system, the load is varying from less than > 2.00 (!) to just over 3.00, with an average average of about 2.50. > > Here is some output from top: > > PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND > 42946 setiathome 139 15 15524K 14952K *Giant 0 39.9H 89.26% 89.26% setiathome > 49332 andyf 130 0 3084K 2176K *Giant 2 81:49 67.68% 67.68% ssh > 12 root -16 0 0K 12K CPU2 2 152.1H 49.12% 49.12% idle: cpu2 > 13 root -16 0 0K 12K CPU1 1 148.7H 44.58% 44.58% idle: cpu1 > 11 root -16 0 0K 12K RUN 3 152.1H 44.14% 44.14% idle: cpu3 > 14 root -16 0 0K 12K CPU0 0 143.3H 41.65% 41.65% idle: cpu0 > 42945 setiathome 129 15 15916K 14700K *Giant 2 39.0H 25.20% 25.20% setiathome > 42947 setiathome 129 15 15524K 14956K *Giant 1 40.3H 22.61% 22.61% setiathome > > So, can someone explain why the seti procs are not getting 100% cpu like > they were before the scp(ssh) started and why there is so much idle time? > I bet those *Giants have something to do with it... > > -- > > :{ andyf@speednet.com.au > > Andy Farkas > System Administrator > Speednet Communications > http://www.speednet.com.au/ > > > > _______________________________________________ > freebsd-smp@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-smp > To unsubscribe, send any mail to "freebsd-smp-unsubscribe@freebsd.org" -- =+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+= Lanny Baron Proud to be 100% FreeBSD http://www.FreeBSDsystems.COM =+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+= From owner-freebsd-smp@FreeBSD.ORG Mon Jul 7 19:16:24 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 94B0C37B409; Mon, 7 Jul 2003 19:16:24 -0700 (PDT) Received: from franky.speednet.com.au (franky.speednet.com.au [203.57.65.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 125D543FA3; Mon, 7 Jul 2003 19:16:23 -0700 (PDT) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h682GKsw087872; Tue, 8 Jul 2003 12:16:20 +1000 (EST) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [172.22.2.17])h682GI2b007288; Tue, 8 Jul 2003 12:16:19 +1000 (EST) (envelope-from andyf@speednet.com.au) Date: Tue, 8 Jul 2003 12:16:18 +1000 (EST) From: Andy Farkas X-X-Sender: andyf@hewey.af.speednet.com.au To: Lanny Baron In-Reply-To: <1057625020.18842.25.camel@panda.freebsdsystems.com> Message-ID: <20030708120950.S6312-100000@hewey.af.speednet.com.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-current@freebsd.org cc: freebsd-smp@freebsd.org Subject: Re: whats going on with the scheduler? X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2003 02:16:24 -0000 On 7 Jul 2003, Lanny Baron wrote: > A load of 3 is pretty high. I think you have more going on. Not for my box. Its only running at 75% cpu power. Its got four processors, so a load of 4 is when its running flat out. And I also said that the box is idle other than the 3 setiathomes. What I'm trying to find out is why at load 3 when adding another semi-cpu intensive process, the load starts fluctuating between 2 and 3, when it should go to a steady 3.4 or something. So the load actually goes down when I run another process! -- :{ andyf@speednet.com.au Andy Farkas System Administrator Speednet Communications http://www.speednet.com.au/ From owner-freebsd-smp@FreeBSD.ORG Mon Jul 7 20:53:12 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1C2BC37B401; Mon, 7 Jul 2003 20:53:12 -0700 (PDT) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5B06F43F75; Mon, 7 Jul 2003 20:53:11 -0700 (PDT) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.12.9/8.12.9) id h683rAuW005411; Mon, 7 Jul 2003 22:53:10 -0500 (CDT) (envelope-from dan) Date: Mon, 7 Jul 2003 22:53:10 -0500 From: Dan Nelson To: Andy Farkas Message-ID: <20030708035309.GE87950@dan.emsphone.com> References: <20030708090530.T6312-100000@hewey.af.speednet.com.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030708090530.T6312-100000@hewey.af.speednet.com.au> X-OS: FreeBSD 5.1-CURRENT X-message-flag: Outlook Error User-Agent: Mutt/1.5.4i cc: freebsd-current@freebsd.org cc: freebsd-smp@freebsd.org Subject: Re: whats going on with the scheduler? X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2003 03:53:12 -0000 In the last episode (Jul 08), Andy Farkas said: > FreeBSD 5.1-RELEASE with SCHED_4BSD on a quad ppro 200 (dell 6100/200). > > Last night I started 3 setiathome's then went to bed. The system was > otherwise idle and had a load of 3.00, 3.00, 3.00. > > This morning, I wanted to copy a (large) file from a remote server, so I > did a: > > scp -c blowfish -p -l 100 remote.host:filename . > > which is running in another window (and will run for 3 more hours). > > And now, on my otherwise idle system, the load is varying from less > than 2.00 (!) to just over 3.00, with an average average of about > 2.50. > > Here is some output from top: > > PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND > 42946 setiathome 139 15 15524K 14952K *Giant 0 39.9H 89.26% 89.26% setiathome > 49332 andyf 130 0 3084K 2176K *Giant 2 81:49 67.68% 67.68% ssh > 12 root -16 0 0K 12K CPU2 2 152.1H 49.12% 49.12% idle: cpu2 > 13 root -16 0 0K 12K CPU1 1 148.7H 44.58% 44.58% idle: cpu1 > 11 root -16 0 0K 12K RUN 3 152.1H 44.14% 44.14% idle: cpu3 > 14 root -16 0 0K 12K CPU0 0 143.3H 41.65% 41.65% idle: cpu0 > 42945 setiathome 129 15 15916K 14700K *Giant 2 39.0H 25.20% 25.20% setiathome > 42947 setiathome 129 15 15524K 14956K *Giant 1 40.3H 22.61% 22.61% setiathome > > So, can someone explain why the seti procs are not getting 100% cpu like > they were before the scp(ssh) started and why there is so much idle time? > I bet those *Giants have something to do with it... Most likely. That means they're waiting for some other process to release the big Giant kernel lock. Paste in top's header so we can see how many processes are locked, and what the system cpu percentage is. A truss of one of the seti processes may be useful too. setiathome really shouldn't be doing many syscalls at all. -- Dan Nelson dnelson@allantgroup.com From owner-freebsd-smp@FreeBSD.ORG Mon Jul 7 21:33:06 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0F6DA37B401; Mon, 7 Jul 2003 21:33:06 -0700 (PDT) Received: from franky.speednet.com.au (franky.speednet.com.au [203.57.65.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id B424943FDD; Mon, 7 Jul 2003 21:33:02 -0700 (PDT) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h684Wssw096024; Tue, 8 Jul 2003 14:32:54 +1000 (EST) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [172.22.2.17])h684Wq2b007713; Tue, 8 Jul 2003 14:32:53 +1000 (EST) (envelope-from andyf@speednet.com.au) Date: Tue, 8 Jul 2003 14:32:52 +1000 (EST) From: Andy Farkas X-X-Sender: andyf@hewey.af.speednet.com.au To: Dan Nelson In-Reply-To: <20030708035309.GE87950@dan.emsphone.com> Message-ID: <20030708135908.I6312-100000@hewey.af.speednet.com.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-current@freebsd.org cc: freebsd-smp@freebsd.org Subject: Re: whats going on with the scheduler? X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2003 04:33:06 -0000 On Mon, 7 Jul 2003, Dan Nelson wrote: > > I bet those *Giants have something to do with it... > > Most likely. That means they're waiting for some other process to > release the big Giant kernel lock. Paste in top's header so we can see > how many processes are locked, and what the system cpu percentage is. This is what top looks like (up to the 1st 0.00% process) when sitting idle* with 3 setiathomes: last pid: 50290; load averages: 3.02, 3.07, 3.06 up 8+23:24:11 14:00:47 97 processes: 9 running, 71 sleeping, 4 zombie, 12 waiting, 1 lock CPU states: 4.0% user, 72.0% nice, 4.6% system, 0.7% interrupt, 18.8% idle Mem: 142M Active, 220M Inact, 116M Wired, 19M Cache, 61M Buf, 1916K Free Swap: 64M Total, 128K Used, 64M Free PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND 42946 setiathome 139 15 16552K 15984K RUN 0 43.8H 98.00% 98.00% setiathome 42945 setiathome 139 15 16944K 15732K CPU1 1 43.0H 97.56% 97.56% setiathome 42947 setiathome 139 15 15524K 14956K CPU0 2 42.9H 94.14% 94.14% setiathome 14 root -16 0 0K 12K RUN 0 144.7H 21.97% 21.97% idle: cpu0 12 root -16 0 0K 12K RUN 2 153.5H 19.87% 19.87% idle: cpu2 11 root -16 0 0K 12K RUN 3 153.6H 18.60% 18.60% idle: cpu3 13 root -16 0 0K 12K RUN 1 150.2H 17.29% 17.29% idle: cpu1 50090 root 111 0 11884K 11084K CPU3 3 4:22 11.57% 11.57% cdparanoia 12571 andyf 100 0 20488K 19308K select 1 509:56 4.00% 4.00% XFree86 17629 andyf 97 0 2676K 1624K select 3 244:57 1.03% 1.03% xdaliclock 16 root -48 -167 0K 12K *Giant 1 122:57 0.39% 0.39% swi7: tty:sio clock 38 root 20 0 0K 12K syncer 0 101:47 0.00% 0.00% syncer *I'm running an X desktop and right now I'm ripping a cd but as you can see its not doing much else.. Note how the seti procs are getting 94-98% cpu time. When I do my scp thing, top looks like this: last pid: 50322; load averages: 1.99, 2.82, 2.98 up 8+23:39:09 14:15:45 98 processes: 8 running, 71 sleeping, 4 zombie, 12 waiting, 3 lock CPU states: 1.7% user, 33.7% nice, 20.1% system, 0.6% interrupt, 43.9% idle Mem: 135M Active, 224M Inact, 120M Wired, 19M Cache, 61M Buf, 1424K Free Swap: 64M Total, 128K Used, 64M Free PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND 42946 setiathome 139 15 16552K 15984K CPU3 2 44.0H 68.41% 68.41% setiathome 50296 andyf 125 0 3084K 2176K RUN 2 7:55 64.21% 64.21% ssh 12 root -16 0 0K 12K CPU2 2 153.6H 48.78% 48.78% idle: cpu2 11 root -16 0 0K 12K CPU3 3 153.6H 48.63% 48.63% idle: cpu3 13 root -16 0 0K 12K RUN 1 150.2H 48.44% 48.44% idle: cpu1 14 root -16 0 0K 12K RUN 0 144.8H 45.31% 45.31% idle: cpu0 42947 setiathome 130 15 15524K 14956K RUN 2 43.1H 28.56% 28.56% setiathome 42945 setiathome 125 15 15916K 14700K RUN 0 43.2H 25.05% 25.05% setiathome 50090 root -8 0 5636K 4832K cbwait 3 5:21 2.69% 2.69% cdparanoia 12571 andyf 97 0 20488K 19308K select 1 510:43 2.39% 2.39% XFree86 16 root -48 -167 0K 12K *Giant 0 123:11 0.98% 0.98% swi7: tty:sio clock 17629 andyf 97 0 2676K 1624K *Giant 0 245:18 0.93% 0.93% xdaliclock 38 root 20 0 0K 12K syncer 1 101:54 0.20% 0.20% syncer 50295 andyf 8 0 2528K 1256K nanslp 0 0:03 0.05% 0.05% scp 28905 root 8 0 0K 12K nfsidl 0 93:02 0.00% 0.00% nfsiod 0 Notice how 'nice' has gone to 33.7% and 'idle' to 43.9%, and the seti procs have dropped well below 94%. > A truss of one of the seti processes may be useful too. setiathome > really shouldn't be doing many syscalls at all. If setiathome is making lots of syscalls, then running the 3 instanses should already show a problem, no? -- :{ andyf@speednet.com.au Andy Farkas System Administrator Speednet Communications http://www.speednet.com.au/ From owner-freebsd-smp@FreeBSD.ORG Mon Jul 7 21:49:14 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 47E4337B401; Mon, 7 Jul 2003 21:49:14 -0700 (PDT) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8FA0A43FD7; Mon, 7 Jul 2003 21:49:13 -0700 (PDT) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.12.9/8.12.9) id h684nDBJ084013; Mon, 7 Jul 2003 23:49:13 -0500 (CDT) (envelope-from dan) Date: Mon, 7 Jul 2003 23:49:13 -0500 From: Dan Nelson To: Andy Farkas Message-ID: <20030708044912.GF87950@dan.emsphone.com> References: <20030708035309.GE87950@dan.emsphone.com> <20030708135908.I6312-100000@hewey.af.speednet.com.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030708135908.I6312-100000@hewey.af.speednet.com.au> X-OS: FreeBSD 5.1-CURRENT X-message-flag: Outlook Error User-Agent: Mutt/1.5.4i cc: freebsd-current@freebsd.org cc: freebsd-smp@freebsd.org Subject: Re: whats going on with the scheduler? X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2003 04:49:14 -0000 In the last episode (Jul 08), Andy Farkas said: > On Mon, 7 Jul 2003, Dan Nelson wrote: > > > I bet those *Giants have something to do with it... > > > > Most likely. That means they're waiting for some other process to > > release the big Giant kernel lock. Paste in top's header so we can see > > how many processes are locked, and what the system cpu percentage is. > > This is what top looks like (up to the 1st 0.00% process) when sitting > idle* with 3 setiathomes: > > 97 processes: 9 running, 71 sleeping, 4 zombie, 12 waiting, 1 lock > CPU states: 4.0% user, 72.0% nice, 4.6% system, 0.7% interrupt, 18.8% idle > > PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND > 42946 setiathome 139 15 16552K 15984K RUN 0 43.8H 98.00% 98.00% setiathome > 42945 setiathome 139 15 16944K 15732K CPU1 1 43.0H 97.56% 97.56% setiathome > 42947 setiathome 139 15 15524K 14956K CPU0 2 42.9H 94.14% 94.14% setiathome > > Note how the seti procs are getting 94-98% cpu time. > > When I do my scp thing, top looks like this: > > 98 processes: 8 running, 71 sleeping, 4 zombie, 12 waiting, 3 lock > CPU states: 1.7% user, 33.7% nice, 20.1% system, 0.6% interrupt, 43.9% idle > > PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU COMMAND > 42946 setiathome 139 15 16552K 15984K CPU3 2 44.0H 68.41% 68.41% setiathome > 50296 andyf 125 0 3084K 2176K RUN 2 7:55 64.21% 64.21% ssh > 12 root -16 0 0K 12K CPU2 2 153.6H 48.78% 48.78% idle: cpu2 > 11 root -16 0 0K 12K CPU3 3 153.6H 48.63% 48.63% idle: cpu3 > 13 root -16 0 0K 12K RUN 1 150.2H 48.44% 48.44% idle: cpu1 > 14 root -16 0 0K 12K RUN 0 144.8H 45.31% 45.31% idle: cpu0 > 42947 setiathome 130 15 15524K 14956K RUN 2 43.1H 28.56% 28.56% setiathome > 42945 setiathome 125 15 15916K 14700K RUN 0 43.2H 25.05% 25.05% setiathome > > Notice how 'nice' has gone to 33.7% and 'idle' to 43.9%, and the seti > procs have dropped well below 94%. > > > A truss of one of the seti processes may be useful too. setiathome > > really shouldn't be doing many syscalls at all. > > If setiathome is making lots of syscalls, then running the 3 instanses > should already show a problem, no? Not if it's ssh that's holding Giant for longer than it should. The setiathome processes may be calling some really fast syscall 500 times a second which doesn't cause a problem until ssh comes along and calls some other syscall that takes .1 ms to return but also locks Giant long enough to cause the other processes to all back up behind it. -- Dan Nelson dnelson@allantgroup.com From owner-freebsd-smp@FreeBSD.ORG Mon Jul 7 22:27:26 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EB5FC37B401; Mon, 7 Jul 2003 22:27:25 -0700 (PDT) Received: from franky.speednet.com.au (franky.speednet.com.au [203.57.65.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 07A2243F85; Mon, 7 Jul 2003 22:27:25 -0700 (PDT) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h685RHsw099527; Tue, 8 Jul 2003 15:27:17 +1000 (EST) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h685RG2b007910; Tue, 8 Jul 2003 15:27:16 +1000 (EST) (envelope-from andyf@speednet.com.au) Date: Tue, 8 Jul 2003 15:27:15 +1000 (EST) From: Andy Farkas X-X-Sender: andyf@hewey.af.speednet.com.au To: Dan Nelson In-Reply-To: <20030708044912.GF87950@dan.emsphone.com> Message-ID: <20030708152306.A6312-100000@hewey.af.speednet.com.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-current@freebsd.org cc: freebsd-smp@freebsd.org Subject: Re: whats going on with the scheduler? X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2003 05:27:26 -0000 On Mon, 7 Jul 2003, Dan Nelson wrote: > Not if it's ssh that's holding Giant for longer than it should. The > setiathome processes may be calling some really fast syscall 500 times > a second which doesn't cause a problem until ssh comes along and calls > some other syscall that takes .1 ms to return but also locks Giant long > enough to cause the other processes to all back up behind it. setiathome seems to only make syscalls once every few seconds: 50.625740361 open("outfile.sah",0x0,0666) = 5 (0x5) 50.632149151 fstat(5,0xbfbff7bc) = 0 (0x0) 50.638742382 fstat(5,0xbfbff874) = 0 (0x0) 50.646472300 lseek(5,0x0,1) = 0 (0x0) 50.654260852 lseek(5,0x0,0) = 0 (0x0) 50.660798154 read(0x5,0xe65000,0x4000) = 3756 (0xeac) 50.667928862 close(5) = 0 (0x0) 50.674330474 getrusage(0x0,0xbfbff994) = 0 (0x0) 50.685590259 open("outfile.sah",0x0,0666) = 5 (0x5) 50.694391188 fstat(5,0xbfbff7ac) = 0 (0x0) 50.703441591 fstat(5,0xbfbff864) = 0 (0x0) 50.711881896 lseek(5,0x0,1) = 0 (0x0) 50.719670900 lseek(5,0x0,0) = 0 (0x0) 50.725983906 read(0x5,0xd64000,0x4000) = 3756 (0xeac) 50.732710339 close(5) = 0 (0x0) 54.227980065 open("outfile.sah",0x0,0666) = 5 (0x5) 54.234289708 fstat(5,0xbfbff7bc) = 0 (0x0) 54.241518075 fstat(5,0xbfbff874) = 0 (0x0) 54.249555307 lseek(5,0x0,1) = 0 (0x0) 54.257416703 lseek(5,0x0,0) = 0 (0x0) 54.263766467 read(0x5,0xe65000,0x4000) = 3756 (0xeac) 54.270554826 close(5) = 0 (0x0) 54.276730780 getrusage(0x0,0xbfbff994) = 0 (0x0) 54.286228233 open("outfile.sah",0x0,0666) = 5 (0x5) 54.292575368 fstat(5,0xbfbff7ac) = 0 (0x0) 54.299568244 fstat(5,0xbfbff864) = 0 (0x0) 54.313140514 lseek(5,0x0,1) = 0 (0x0) 54.320997075 lseek(5,0x0,0) = 0 (0x0) 54.327394776 read(0x5,0xd64000,0x4000) = 3756 (0xeac) 54.334797742 close(5) = 0 (0x0) 57.830104692 open("outfile.sah",0x0,0666) = 5 (0x5) 57.836412958 fstat(5,0xbfbff7bc) = 0 (0x0) 57.842768341 fstat(5,0xbfbff874) = 0 (0x0) 57.850834611 lseek(5,0x0,1) = 0 (0x0) 57.862301512 lseek(5,0x0,0) = 0 (0x0) 57.868723869 read(0x5,0xe65000,0x4000) = 3756 (0xeac) 57.875396846 close(5) = 0 (0x0) 57.881638118 getrusage(0x0,0xbfbff994) = 0 (0x0) 57.890991955 open("outfile.sah",0x0,0666) = 5 (0x5) 57.897344227 fstat(5,0xbfbff7ac) = 0 (0x0) 57.903621248 fstat(5,0xbfbff864) = 0 (0x0) 57.913178984 lseek(5,0x0,1) = 0 (0x0) 57.922614702 lseek(5,0x0,0) = 0 (0x0) 57.931253628 read(0x5,0xd64000,0x4000) = 3756 (0xeac) 57.938755028 close(5) = 0 (0x0) -- :{ andyf@speednet.com.au Andy Farkas System Administrator Speednet Communications http://www.speednet.com.au/ From owner-freebsd-smp@FreeBSD.ORG Tue Jul 8 03:55:42 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C62D437B401; Tue, 8 Jul 2003 03:55:42 -0700 (PDT) Received: from stork.mail.pas.earthlink.net (stork.mail.pas.earthlink.net [207.217.120.188]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1314743FA3; Tue, 8 Jul 2003 03:55:42 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from user-38lc01p.dialup.mindspring.com ([209.86.0.57] helo=mindspring.com) by stork.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 19Zq8B-0005hg-00; Tue, 08 Jul 2003 03:55:32 -0700 Message-ID: <3F0AA2DE.13035C1@mindspring.com> Date: Tue, 08 Jul 2003 03:54:22 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Andy Farkas References: <20030708120950.S6312-100000@hewey.af.speednet.com.au> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a473297c82ea3c78ff06481c6bb063ab5693caf27dac41a8fd350badd9bab72f9c350badd9bab72f9c cc: freebsd-current@freebsd.org cc: freebsd-smp@freebsd.org cc: Lanny Baron Subject: Re: whats going on with the scheduler? X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2003 10:55:43 -0000 Andy Farkas wrote: > On 7 Jul 2003, Lanny Baron wrote: > > A load of 3 is pretty high. I think you have more going on. > > Not for my box. Its only running at 75% cpu power. Its got four > processors, so a load of 4 is when its running flat out. And I also said > that the box is idle other than the 3 setiathomes. > > What I'm trying to find out is why at load 3 when adding another semi-cpu > intensive process, the load starts fluctuating between 2 and 3, when it > should go to a steady 3.4 or something. So the load actually goes down > when I run another process! It's not clear what your link speed is, but it's possible that what's happening is that the setiathome processes are stalling waiting for work units because you are using up your available network bandwidth. This is usually the case when you have an asymmetric link speed (e.g. cablemodem or DSL) and are trying to push data through the small pipe: it's very easy to monopolize all the buffers in the router at the other end of the link, such that you don't end up getting ACK packets out to keep the data pipe down full. If this is what's ahppening, you might want to try bandwidth limiting the scp and/or running with Alt-Q and/or begging Julian for the code he wrote at Whistle, which beats the snot out of Alt-Q... 8-). -- Terry From owner-freebsd-smp@FreeBSD.ORG Tue Jul 8 04:01:41 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 52CA337B401; Tue, 8 Jul 2003 04:01:41 -0700 (PDT) Received: from stork.mail.pas.earthlink.net (stork.mail.pas.earthlink.net [207.217.120.188]) by mx1.FreeBSD.org (Postfix) with ESMTP id 52E6643FAF; Tue, 8 Jul 2003 04:01:40 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from user-38lc01p.dialup.mindspring.com ([209.86.0.57] helo=mindspring.com) by stork.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 19ZqDy-0006QN-00; Tue, 08 Jul 2003 04:01:31 -0700 Message-ID: <3F0AA444.28EC5A8E@mindspring.com> Date: Tue, 08 Jul 2003 04:00:20 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Dan Nelson References: <20030708035309.GE87950@dan.emsphone.com> <20030708044912.GF87950@dan.emsphone.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4393873c806a2959f9a181e51f5b02339548b785378294e88350badd9bab72f9c350badd9bab72f9c cc: freebsd-current@freebsd.org cc: freebsd-smp@freebsd.org cc: Andy Farkas Subject: Re: whats going on with the scheduler? X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2003 11:01:41 -0000 Dan Nelson wrote: > In the last episode (Jul 08), Andy Farkas said: > > If setiathome is making lots of syscalls, then running the 3 instanses > > should already show a problem, no? > > Not if it's ssh that's holding Giant for longer than it should. The > setiathome processes may be calling some really fast syscall 500 times > a second which doesn't cause a problem until ssh comes along and calls > some other syscall that takes .1 ms to return but also locks Giant long > enough to cause the other processes to all back up behind it. Specifically, if it's sleeping with Giant held because the Send-Q is full (use netstat to check) it could block things for a long time, waiting for the queue to drain. If this is the case, then you might want to ask Jeffrey Hsu if it's safe to drop the lock during the sosend() (it probably isn't). -- Terry From owner-freebsd-smp@FreeBSD.ORG Tue Jul 8 04:55:50 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 841DB37B401; Tue, 8 Jul 2003 04:55:50 -0700 (PDT) Received: from franky.speednet.com.au (franky.speednet.com.au [203.57.65.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 24DDB43FA3; Tue, 8 Jul 2003 04:55:49 -0700 (PDT) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h68Btksw017277; Tue, 8 Jul 2003 21:55:47 +1000 (EST) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h68Btj2b008966; Tue, 8 Jul 2003 21:55:46 +1000 (EST) (envelope-from andyf@speednet.com.au) Date: Tue, 8 Jul 2003 21:55:45 +1000 (EST) From: Andy Farkas X-X-Sender: andyf@hewey.af.speednet.com.au To: Terry Lambert In-Reply-To: <3F0AA2DE.13035C1@mindspring.com> Message-ID: <20030708214151.R8850-100000@hewey.af.speednet.com.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-current@freebsd.org cc: freebsd-smp@freebsd.org Subject: Re: whats going on with the scheduler? X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2003 11:55:50 -0000 On Tue, 8 Jul 2003, Terry Lambert wrote: > It's not clear what your link speed is, but it's possible that > what's happening is that the setiathome processes are stalling > waiting for work units because you are using up your available > network bandwidth. setiathome is a cpu intensive process. It touches the network maybe once in a 24 hour period when it downloads a ~400k work unit which takes less than 10 seconds over my 512k/128k dsl link and then proceeds to number crunch. Stalling on work units is not whats happening. > If this is what's ahppening, you might want to try bandwidth > limiting the scp and/or running with Alt-Q and/or begging Julian I *did* limit scp: > scp -c blowfish -p -l 100 remote.host:filename . -- :{ andyf@speednet.com.au Andy Farkas System Administrator Speednet Communications http://www.speednet.com.au/ From owner-freebsd-smp@FreeBSD.ORG Tue Jul 8 05:22:18 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4520837B404; Tue, 8 Jul 2003 05:22:18 -0700 (PDT) Received: from franky.speednet.com.au (franky.speednet.com.au [203.57.65.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 282FB43FE0; Tue, 8 Jul 2003 05:22:17 -0700 (PDT) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h68CM9sw018182; Tue, 8 Jul 2003 22:22:10 +1000 (EST) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [172.22.2.17])h68CM72b009049; Tue, 8 Jul 2003 22:22:08 +1000 (EST) (envelope-from andyf@speednet.com.au) Date: Tue, 8 Jul 2003 22:22:07 +1000 (EST) From: Andy Farkas X-X-Sender: andyf@hewey.af.speednet.com.au To: Terry Lambert In-Reply-To: <3F0AA444.28EC5A8E@mindspring.com> Message-ID: <20030708215553.F8850-100000@hewey.af.speednet.com.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-smp@freebsd.org cc: freebsd-current@freebsd.org cc: Dan Nelson Subject: Re: whats going on with the scheduler? X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2003 12:22:18 -0000 On Tue, 8 Jul 2003, Terry Lambert wrote: > Dan Nelson wrote: > > In the last episode (Jul 08), Andy Farkas said: > > > If setiathome is making lots of syscalls, then running the 3 instanses > > > should already show a problem, no? > > > > Not if it's ssh that's holding Giant for longer than it should. The > > setiathome processes may be calling some really fast syscall 500 times > > a second which doesn't cause a problem until ssh comes along and calls > > some other syscall that takes .1 ms to return but also locks Giant long > > enough to cause the other processes to all back up behind it. > > Specifically, if it's sleeping with Giant held because the > Send-Q is full (use netstat to check) it could block things > for a long time, waiting for the queue to drain. scp was retrieving a file, not sending, and it was bandwidth limited. Any other ideas? Why would 3 (niced) cpu intensive processes suddenly get reduced cpu time (on a 4 cpu system) when a 4th non-resource intensive process gets started? Also, from something that BDE said once, this command will produce unexpected results when run for more than a few hours: for i in `jot -n -s ' ' 20 0 19 1` do nice -$i sh -c "while :; do echo -n;done" & done -- :{ andyf@speednet.com.au Andy Farkas System Administrator Speednet Communications http://www.speednet.com.au/ From owner-freebsd-smp@FreeBSD.ORG Tue Jul 8 11:41:47 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8B07337B401; Tue, 8 Jul 2003 11:41:47 -0700 (PDT) Received: from carver.gumbysoft.com (carver.gumbysoft.com [66.220.23.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id B0FF243F3F; Tue, 8 Jul 2003 11:41:46 -0700 (PDT) (envelope-from dwhite@gumbysoft.com) Received: by carver.gumbysoft.com (Postfix, from userid 1000) id 9EF2672FE3; Tue, 8 Jul 2003 11:41:46 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by carver.gumbysoft.com (Postfix) with ESMTP id 918C072FDC; Tue, 8 Jul 2003 11:41:46 -0700 (PDT) Date: Tue, 8 Jul 2003 11:41:46 -0700 (PDT) From: Doug White To: Andy Farkas In-Reply-To: <20030708215553.F8850-100000@hewey.af.speednet.com.au> Message-ID: <20030708113618.P25140@carver.gumbysoft.com> References: <20030708215553.F8850-100000@hewey.af.speednet.com.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Dan Nelson cc: freebsd-smp@freebsd.org cc: freebsd-current@freebsd.org Subject: Re: whats going on with the scheduler? X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2003 18:41:47 -0000 On Tue, 8 Jul 2003, Andy Farkas wrote: > Any other ideas? Why would 3 (niced) cpu intensive processes suddenly get > reduced cpu time (on a 4 cpu system) when a 4th non-resource intensive > process gets started? Hm.. guess its time to explain how nice works again. Nice is a relative value. If you have 2 processes in a system, one with a lower nice value (== higher "priority") than the other, the lower-niced process will be scheduled in deference to the higher-niced process. The scheduler attempts to ensure that niced processes are not starved. (In practice, nice level 20 gets some special treatment.) If you don't want higher-niced processes to get their cpu time reduced when a lower-niced process starts doing work, then don't nice them. I'm sure Terry will pick this to death, buut you get the idea. I think the daemon book explains this better than I could (and with infinitely more detail). -- Doug White | FreeBSD: The Power to Serve dwhite@gumbysoft.com | www.FreeBSD.org From owner-freebsd-smp@FreeBSD.ORG Tue Jul 8 15:40:44 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 24D4137B401; Tue, 8 Jul 2003 15:40:44 -0700 (PDT) Received: from franky.speednet.com.au (franky.speednet.com.au [203.57.65.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id C7FF943F75; Tue, 8 Jul 2003 15:40:42 -0700 (PDT) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h68Matsw036900; Wed, 9 Jul 2003 08:40:40 +1000 (EST) (envelope-from andyf@speednet.com.au) Received: from hewey.af.speednet.com.au (hewey.af.speednet.com.au [203.38.96.242])h68Mas2b011363; Wed, 9 Jul 2003 08:36:54 +1000 (EST) (envelope-from andyf@speednet.com.au) Date: Wed, 9 Jul 2003 08:36:53 +1000 (EST) From: Andy Farkas X-X-Sender: andyf@hewey.af.speednet.com.au To: Doug White In-Reply-To: <20030708113618.P25140@carver.gumbysoft.com> Message-ID: <20030709080542.H11189-100000@hewey.af.speednet.com.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-current@freebsd.org cc: freebsd-smp@freebsd.org Subject: Re: whats going on with the scheduler? X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Jul 2003 22:40:44 -0000 On Tue, 8 Jul 2003, Doug White wrote: > On Tue, 8 Jul 2003, Andy Farkas wrote: > > > Any other ideas? Why would 3 (niced) cpu intensive processes suddenly get > > reduced cpu time (on a 4 cpu system) when a 4th non-resource intensive > > process gets started? > > Hm.. guess its time to explain how nice works again. > > Nice is a relative value. If you have 2 processes in a system, one with a > lower nice value (== higher "priority") than the other, the lower-niced > process will be scheduled in deference to the higher-niced process. The > scheduler attempts to ensure that niced processes are not starved. (In > practice, nice level 20 gets some special treatment.) That doesn't explain why the idle time goes up, in my case. If you have 4 processors in a box and start 3 cpu-intensive jobs, the system load will be 3.00 and idle time will be 25%. If you start another semi cpu-intensive process, one would expect the load to increase and the idle time to come down, regardless if the other 3 procs are niced or not. ps. setiathome procs run at idle level 15. -- :{ andyf@speednet.com.au Andy Farkas System Administrator Speednet Communications http://www.speednet.com.au/ From owner-freebsd-smp@FreeBSD.ORG Tue Jul 8 23:27:30 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7E12F37B401 for ; Tue, 8 Jul 2003 23:27:30 -0700 (PDT) Received: from dns11.mail.yahoo.co.jp (dns11.mail.yahoo.co.jp [210.81.151.144]) by mx1.FreeBSD.org (Postfix) with SMTP id 7C4EB43F85 for ; Tue, 8 Jul 2003 23:27:29 -0700 (PDT) (envelope-from redmonta@ybb.ne.jp) Received: from unknown (HELO ?192.168.1.11?) (219.11.70.5 with poptime) by dns11.mail.yahoo.co.jp with SMTP; 9 Jul 2003 06:27:27 -0000 Date: Wed, 09 Jul 2003 15:27:21 +0900 From: "redmonta@ybb.ne.jp" To: freebsd-smp@FreeBSD.org Message-Id: <20030709152716.526A.REDMONTA@ybb.ne.jp> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Mailer: Becky! ver. 2.06.02 Subject: X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2003 06:27:30 -0000 -- redmonta@ybb.ne.jp From owner-freebsd-smp@FreeBSD.ORG Wed Jul 9 10:03:19 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E5C9F37B401 for ; Wed, 9 Jul 2003 10:03:19 -0700 (PDT) Received: from andromeda.lunarpages.com (andromeda.lunarpages.com [64.235.234.215]) by mx1.FreeBSD.org (Postfix) with ESMTP id 697A943FAF for ; Wed, 9 Jul 2003 10:03:19 -0700 (PDT) (envelope-from bob@greenpeanut.com) Received: from c66.190.111.77.ts46v-01.rckprt.tx.charter.com ([66.190.111.77] helo=greenpeanut.com) by andromeda.lunarpages.com with asmtp (Exim 3.36 #1) id 19aILb-0001la-00 for freebsd-smp@freebsd.org; Wed, 09 Jul 2003 10:03:15 -0700 Message-ID: <3F0AF994.4020206@greenpeanut.com> Date: Tue, 08 Jul 2003 12:04:20 -0500 From: Bob Johnson User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-smp@freebsd.org Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - andromeda.lunarpages.com X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [0 0] / [0 0] X-AntiAbuse: Sender Address Domain - greenpeanut.com Subject: Interesting SMP timeouts X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Jul 2003 17:03:20 -0000 I am installing a SMP kernel on another one of my old dual Pentium II boxes, I have already done this on one before and it did work fine, (www.bluhayz.org) but I can't get it to work on the NFS Server. I cannot get you the exact messages, cause I can't get them myself. It gets to the point of saying SMP: AP CPU #1 Launched, or something like that. Then it will list out device timeouts, ad0 and ata0 Then the whole kernel will panic and reboot When I use a non-SMP kernel, everything works fine, please help me! I've searched the SMP archives and found similar problems, but none of their solutions have worked. thanks. From owner-freebsd-smp@FreeBSD.ORG Thu Jul 10 07:25:10 2003 Return-Path: Delivered-To: freebsd-smp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 954EB37B401 for ; Thu, 10 Jul 2003 07:25:10 -0700 (PDT) Received: from flamingo.mail.pas.earthlink.net (flamingo.mail.pas.earthlink.net [207.217.120.232]) by mx1.FreeBSD.org (Postfix) with ESMTP id 10A3943FAF for ; Thu, 10 Jul 2003 07:25:10 -0700 (PDT) (envelope-from nycvelo@earthlink.net) Received: from beaker.psp.pas.earthlink.net ([207.217.78.247]) by flamingo.mail.pas.earthlink.net with esmtp (Exim 3.33 #1) id 19acM9-0005Lg-00 for freebsd-smp@freebsd.org; Thu, 10 Jul 2003 07:25:09 -0700 Received: from [207.217.78.202] by EarthlinkWAM via HTTP; Thu Jul 10 07:25:09 PDT 2003 Message-ID: <7758413.1057847109643.JavaMail.nobody@beaker.psp.pas.earthlink.net> Date: Thu, 10 Jul 2003 07:25:08 -0700 (PDT) From: nycvelo@earthlink.net To: freebsd-smp@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Mailer: Earthlink Web Access Mail version 3.0 Subject: extended tabled hosed on Proliant X-BeenThere: freebsd-smp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: FreeBSD SMP implementation group List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2003 14:25:11 -0000 Apologies if this has been asked before. I've installed 4.8 on a Compaq Proliant 1850 6-550 and built a kernel with MP support. The system boots the MP kernel OK, but mptable reports "Extended Table HOSED!" I don't see anything unusual in dmesg, but I may be missing something. Output of dmesg and mptable below. Thanks very much for any ideas on how to fix this. Regards, David Newman dnewman@moser ~ 81$ dmesg Copyright (c) 1992-2003 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 4.8-STABLE #1: Fri Jul 4 11:51:56 PDT 2003 root@moser.int.networktest.com:/usr/obj/usr/src/sys/SMPKERN Timecounter "i8254" frequency 1193182 Hz CPU: Pentium III/Pentium III Xeon/Celeron (548.34-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x673 Stepping = 3 Features=0x383fbff real memory = 536854528 (524272K bytes) avail memory = 517677056 (505544K bytes) Changing APIC ID for IO APIC #0 from 0 to 8 on chip Programming 35 pins in IOAPIC #0 IOAPIC #0 intpin 2 -> irq 0 FreeBSD/SMP: Multiprocessor motherboard cpu0 (BSP): apic id: 1, version: 0x00040011, at 0xfee00000 cpu1 (AP): apic id: 0, version: 0x00040011, at 0xfee00000 io0 (APIC): apic id: 8, version: 0x00220011, at 0xfec00000 Preloaded elf kernel "kernel" at 0xc0490000. Pentium Pro MTRR support enabled md0: Malloc disk npx0: on motherboard npx0: INT 16 interface pcib0: on motherboard pci0: on pcib0 sym0: <875> port 0x2000-0x20ff mem 0xc6eff000-0xc6efffff,0xc6efcf00-0xc6efcfff irq 10 at device 6.0 on pci0 sym0: No NVRAM, ID 7, Fast-20, SE, parity checking sym1: <875> port 0x2400-0x24ff mem 0xc6efe000-0xc6efefff,0xc6efce00-0xc6efceff irq 11 at device 6.1 on pci0 sym1: No NVRAM, ID 7, Fast-20, SE, parity checking tl0: port 0x2c20-0x2c2f mem 0xc6efccf0-0xc6efccff irq 15 at device 7.0 on pci0 tl0: Ethernet address: 00:50:8b:8f:b9:f1 miibus0: on tl0 lxtphy0: on miibus0 lxtphy0: 100baseFX, 100baseFX-FDX, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto tlphy0: on miibus0 tlphy0: 10base2/BNC, 10base5/AUI pci0: at 8.0 pci0: (vendor=0x0e11, dev=0xa0f0) at 9.0 fxp0: port 0x2c00-0x2c1f mem 0xc6f00000-0xc6ffffff,0xc4fff000-0xc4ffffff irq 5 at device 15.0 on pci0 fxp0: Ethernet address 00:50:8b:6a:93:cf inphy0: on miibus1 inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto isab0: at device 20.0 on pci0 isa0: on isab0 atapci0: port 0xf100-0xf10f at device 20.1 on pci0 ata0: at 0x1f0 irq 14 on atapci0 ata1: at 0x170 irq 15 on atapci0 pci0: at 20.2 irq 0 piix0: at device 20.3 on pci0 eisa0: on motherboard mainboard0: on eisa0 slot 0 orm0: