From owner-freebsd-questions@FreeBSD.ORG Wed Feb 27 11:31:41 2008 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 116E51065676 for ; Wed, 27 Feb 2008 11:31:41 +0000 (UTC) (envelope-from zszalbot@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.187]) by mx1.freebsd.org (Postfix) with ESMTP id 63E918FC1A for ; Wed, 27 Feb 2008 11:31:40 +0000 (UTC) (envelope-from zszalbot@gmail.com) Received: by nf-out-0910.google.com with SMTP id b2so1640381nfb.33 for ; Wed, 27 Feb 2008 03:31:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=YzhkF/B/TaycyccNT1YzDXBaZsN95fFHCla/nYNa7BE=; b=fqgMKBuwbKwNalp5rWdjuIprmVCemZJW2NxQIYHmNL/j14Ae1VJeCLsbMBIgRpJLbcieOFbu66M8Yo79QCtTv13oUzG8DJW6SDxqbzwrwXtxu7nf70cHTk3Pc50WCGZ/uGbtFUZF9Q5J351FclbjKshXwN3ghocCefAL9aYX7X8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=kgh1hEVnx685EwPVKO6o/NJ+unyWxke4ASIA52g62tx6gWyF2mbVdlsgHj7aJEWLaU+vgay8sXT+MgfbXf5X0CgainjtR+E3Xkguu6126o858Gwox2He40YbceTBwRVnBsuP69X8VztKc20C5EEYD4BAlhIq8WG9uFbdhFzNmFQ= Received: by 10.78.168.6 with SMTP id q6mr5755279hue.65.1204111898783; Wed, 27 Feb 2008 03:31:38 -0800 (PST) Received: by 10.78.130.5 with HTTP; Wed, 27 Feb 2008 03:31:33 -0800 (PST) Message-ID: <94136a2c0802270331t59a3561dhf33f0662ef2ddec8@mail.gmail.com> Date: Wed, 27 Feb 2008 12:31:33 +0100 From: "Zbigniew Szalbot" To: "Kris Kennaway" In-Reply-To: <47C543AC.2090801@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <94136a2c0802270259i313331dcw45f8c26801f01636@mail.gmail.com> <47C543AC.2090801@FreeBSD.org> Cc: freebsd-questions Subject: Re: sudden peak in load average X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Feb 2008 11:31:41 -0000 Hello, 2008/2/27, Kris Kennaway : > Zbigniew Szalbot wrote: > > Hello, > > > >>From the MRTG data I can see that suddenly on a currently not very > > busy machine the load averege went over 15 or more. This happened > > around 10 in the morning. Not many entries in httpd access log, smtp > > server was not too much loaded (at that time it generally produces a > > load average of about 1). > > > > But this time: > > 2008-02-27 10:03:36 1JUICO-0002Bs-23 no immediate delivery: load average 15.88 > > > > Where can I look for more information about what happened. The server > > was rebooted early in the morning so its swap consumption was almost > > non-existent. Around 10 it suddenly consumed probably around 200MB of > > swap, now it stands at 135MB which is pretty normal with this machine > > (it only has 512 RAM). SWAP usage is normal with this machine but it > > usually is a matter of 2-3 days before it gradually takes over 10-13% > > of swap space. > > > > I tried by looking at: > > http access log > > auth.log > > maillog > > messages > > > > Nothing in there indicating an outburst of sudden activities. > > > > heh - I now think I may be wrong. The load average did not necessarily > > produce so much swap consumption so fast. At 10:13 I run a cron job > > optimising all mysql tables. So maybe in fact the swap was used by > > mysql operations, although I am still interested to now what casued > > load average to go above 15 at 10:03. > > > > 10:00AM up 3:14, 0 users, load averages: 0.40, 0.13, 0.05 > > 10:01AM up 3:15, 0 users, load averages: 0.23, 0.14, 0.05 > > 10:02AM up 3:16, 0 users, load averages: 0.08, 0.11, 0.05 > > 10:03AM up 3:17, 0 users, load averages: 11.54, 3.09, 1.15 > > 10:04AM up 3:18, 0 users, load averages: 13.26, 5.69, 2.28 > > 10:05AM up 3:19, 0 users, load averages: 4.98, 4.69, 2.14 > > 10:06AM up 3:20, 0 users, load averages: 1.79, 3.80, 1.99 > > > > > > Thanks for any pointers. > No way to know. You'll have to set up more detailed logging, e.g. a > script that runs ps if load is over a certain limit. Thank you Kris! Is anybody willing to share such a script (if there is one)? Thanks! -- Zbigniew Szalbot