From owner-freebsd-stable@FreeBSD.ORG Fri Apr 11 12:59:32 2003 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DC99337B401 for ; Fri, 11 Apr 2003 12:59:31 -0700 (PDT) Received: from hub.org (hub.org [64.117.224.146]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2178143FBD for ; Fri, 11 Apr 2003 12:59:31 -0700 (PDT) (envelope-from scrappy@hub.org) Received: from hub.org (hub.org [64.117.224.146]) by hub.org (Postfix) with ESMTP id 47C0610391C1 for ; Fri, 11 Apr 2003 16:59:27 -0300 (ADT) Date: Fri, 11 Apr 2003 16:59:27 -0300 (ADT) From: "Marc G. Fournier" To: freebsd-stable@freebsd.org Message-ID: <20030411165002.L9067@hub.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: no idle CPU ... system hogging it all ... X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Apr 2003 19:59:32 -0000 Its been looking like this pretty much all day ... top shows nothing major, and the drive looks reaonably quiet ... there is nothing in messages to indicate a problem that I can see (even those enclosure messages have been reasonably quiet) ... What consumes SYS CPU? Stuff like apache and jakarta-tomcat use up USER CPU, correct? neptune# iostat 5 tty aacd0 pass0 pass1 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 1 125 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 13 0 48 0 39 0 108 17.39 88 1.49 0.00 0 0.00 0.00 0 0.00 16 0 83 0 0 0 23 17.79 23 0.39 0.00 0 0.00 0.00 0 0.00 6 0 92 0 1 0 294 15.99 28 0.43 0.00 0 0.00 0.00 0 0.00 8 0 92 0 0 0 35 17.56 156 2.67 0.00 0 0.00 0.00 0 0.00 18 0 82 0 0 0 1048 15.86 68 1.05 0.00 0 0.00 0.00 0 0.00 22 0 77 0 0 3 981 14.63 79 1.13 0.00 0 0.00 0.00 0 0.00 10 0 89 1 0 0 353 14.85 77 1.12 0.00 0 0.00 0.00 0 0.00 18 0 81 0 0 0 728 14.75 31 0.45 0.00 0 0.00 0.00 0 0.00 8 0 91 0 0 1 669 13.93 11 0.15 0.00 0 0.00 0.00 0 0.00 11 0 89 0 0 0 15 10.42 4 0.04 0.00 0 0.00 0.00 0 0.00 6 0 93 0 0 0 15 13.42 31 0.40 0.00 0 0.00 0.00 0 0.00 3 0 97 0 0 0 15 8.52 25 0.21 0.00 0 0.00 0.00 0 0.00 3 0 97 0 0 5 686 15.71 64 0.98 0.00 0 0.00 0.00 0 0.00 9 0 91 0 0 5 463 8.94 10 0.09 0.00 0 0.00 0.00 0 0.00 6 0 93 0 0 5 24 10.67 3 0.03 0.00 0 0.00 0.00 0 0.00 14 0 85 1 0 6 41 16.95 128 2.12 0.00 0 0.00 0.00 0 0.00 9 0 90 0 0 8 274 16.14 114 1.80 0.00 0 0.00 0.00 0 0.00 7 0 93 0 0 8 33 15.13 8 0.11 0.00 0 0.00 0.00 0 0.00 11 0 89 0 0 3 20 15.65 33 0.51 0.00 0 0.00 0.00 0 0.00 4 0 96 0 0 doing a ps, the 'server processes' don't look to be consuming much CPU, other then the vmdaemon/syncer (that is 18 and 9 hrs respectively, right?) USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND root 0 0.0 0.0 0 0 ?? DLs Thu07AM 0:45.80 (swapper) root 1 0.0 0.0 552 196 ?? SLs Thu07AM 0:03.37 /sbin/init -- root 2 0.0 0.0 0 0 ?? DL Thu07AM 0:00.26 (aac0aif) root 3 0.0 0.0 0 0 ?? DL Thu07AM 0:19.50 (pagedaemon) root 4 0.4 0.0 0 0 ?? DL Thu07AM 18:27.81 (vmdaemon) root 5 0.0 0.0 0 0 ?? DL Thu07AM 0:01.62 (bufdaemon) root 6 0.0 0.0 0 0 ?? DL Thu07AM 0:01.03 (vnlru) root 7 0.0 0.0 0 0 ?? DL Thu07AM 9:03.81 (syncer) root 28 0.0 0.0 212 0 ?? IWs - 0:00.00 adjkerntz -i root 134 0.0 0.0 952 528 ?? Ss Thu10AM 0:06.17 /usr/sbin/sysl root 138 0.0 0.1 6304 5608 ?? Ss Thu10AM 0:51.81 /usr/sbin/name daemon 140 0.0 0.0 988 436 ?? Ss Thu10AM 0:02.75 /usr/sbin/port root 142 0.0 0.0 572 356 ?? Ss Thu10AM 0:02.76 mountd -r root 144 0.0 0.0 368 0 ?? IWs - 0:00.00 nfsd: master ( root 145 0.0 0.0 360 0 ?? IW - 0:00.00 nfsd: server ( root 146 0.0 0.0 360 0 ?? IW - 0:00.00 nfsd: server ( root 148 0.0 0.0 360 0 ?? IW - 0:00.00 nfsd: server ( root 149 0.0 0.0 360 0 ?? IW - 0:00.00 nfsd: server ( root 150 0.0 0.0 263088 0 ?? Ss Thu10AM 0:02.73 rpc.statd daemon 152 0.0 0.0 908 0 ?? IWs - 0:00.00 rwhod root 158 0.0 0.0 1024 0 ?? IWs - 0:00.00 /usr/sbin/cron root 160 0.0 0.0 2592 928 ?? Ss Thu10AM 0:03.87 /usr/sbin/sshd Right now, kvm and vnodes are looking like: vm.kvm_free: 809500672 - debug.numvnodes: 414708 - debug.freevnodes: 95815 which is pretty much standard for my servers ... and there are only ~1200 processes running on this one, which is a Dual PIII with 4Gb of RAM ... Starting commands seems to take a long time ... top takes forever, and pstat -s shows: neptune# time pstat -s Device 1K-blocks Used Avail Capacity Type /dev/aacd0s1b 8388480 119916 8268564 1% Interleaved 0.245u 3.670s 0:27.39 14.2% 16+218k 0+0io 0pf+0w And not much swap is being used for the # of processes .. What else should I be looking at? :(