From owner-freebsd-stable@FreeBSD.ORG Mon Apr 14 06:25:26 2003 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0FADC37B404 for ; Mon, 14 Apr 2003 06:25:26 -0700 (PDT) Received: from HAL9000.homeunix.com (12-233-57-131.client.attbi.com [12.233.57.131]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5A96343FAF for ; Mon, 14 Apr 2003 06:25:25 -0700 (PDT) (envelope-from das@FreeBSD.ORG) Received: from HAL9000.homeunix.com (localhost [127.0.0.1]) by HAL9000.homeunix.com (8.12.9/8.12.5) with ESMTP id h3EDPJN7019945; Mon, 14 Apr 2003 06:25:19 -0700 (PDT) (envelope-from das@FreeBSD.ORG) Received: (from das@localhost) by HAL9000.homeunix.com (8.12.9/8.12.5/Submit) id h3EDPJWn019944; Mon, 14 Apr 2003 06:25:19 -0700 (PDT) (envelope-from das@FreeBSD.ORG) Date: Mon, 14 Apr 2003 06:25:19 -0700 From: David Schultz To: "Marc G. Fournier" Message-ID: <20030414132519.GA19543@HAL9000.homeunix.com> Mail-Followup-To: "Marc G. Fournier" , freebsd-stable@freebsd.org References: <20030411165002.L9067@hub.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030411165002.L9067@hub.org> cc: freebsd-stable@FreeBSD.ORG Subject: Re: no idle CPU ... system hogging it all ... X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 Apr 2003 13:25:26 -0000 On Fri, Apr 11, 2003, Marc G. Fournier wrote: > > Its been looking like this pretty much all day ... top shows nothing > major, and the drive looks reaonably quiet ... there is nothing in > messages to indicate a problem that I can see (even those enclosure > messages have been reasonably quiet) ... You can profile the kernel to find out, although at this hour I probably can't recall all the necessary details. I believe you need to say 'make buildkernel CONFIGARGS=-p', then use kgmon(8) and gprof(1) to extract and analyze the data. Based on the ps output you gave, it looks like vmdaemon and syncer are taking up most of the time. This suggests that perhaps there is a hardware or driver problem with disk I/O. Some drivers perform very small delays to wait for the hardware by spinning instead of eating the overhead of a context switch. It could be that one of these drivers has a bug, or the hardware is taking longer than expected to respond. > What consumes SYS CPU? Stuff like apache and jakarta-tomcat use up USER > CPU, correct? Every process consumes system time when it is running in the kernel to process a trap or system call. They use user time when they are running their own (unprivileged) code. Kernel threads such as the pageout daemon and syncer use system time exclusively. time(1) will show you how much system and user time a process uses. Unfortunately, you can't use it on a process that is already running, and tools like 'ps' don't show both numbers (although they can be easily hacked to do so.) For example: > time du >/dev/null 0.074u 0.568s 0:03.36 18.7% 11+285k 1431+0io 0pf+0w This shows that the 'du' command took 74 ms of user time and 568 ms in system calls. If you're not using the C shell or you want more descriptive output, use '/usr/bin/time -l'.