From owner-freebsd-stable@FreeBSD.ORG Thu Apr 15 20:56:55 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4D6BC106564A for ; Thu, 15 Apr 2010 20:56:55 +0000 (UTC) (envelope-from pali.gabor@googlemail.com) Received: from mail-bw0-f214.google.com (mail-bw0-f214.google.com [209.85.218.214]) by mx1.freebsd.org (Postfix) with ESMTP id B5F558FC25 for ; Thu, 15 Apr 2010 20:56:54 +0000 (UTC) Received: by bwz6 with SMTP id 6so1692596bwz.13 for ; Thu, 15 Apr 2010 13:56:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :organization:user-agent:mime-version:to:cc:subject:references :in-reply-to:x-enigmail-version:content-type; bh=l21G3VMXsLw9sU13waa+fb0xW3Bp0BCu+MifiBFLB4Q=; b=P3zvMVQrMZJyrTpcRJNDFhRznZe240dCK7rYsGcsuN06Q/jkii/y/zxelaNtWv8LSi 0rLiS4O9/HFsFGj818KiJSXUydIKvFLGVU4kt0KGP5htPhDXirTrLG+SdiYMtQfpFTfU LaPq/gozax+H3FLWVhc9CDCVGZqIgDLl2yKhY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=sender:message-id:date:from:organization:user-agent:mime-version:to :cc:subject:references:in-reply-to:x-enigmail-version:content-type; b=CiCJFnLDLkCifr+5WPmThKK7bJnYNqD0cbWSXG415Lz2Hoat+NBdAxilZBHpS6pyCo YD/Ys2hW4Gy5v/NEzqp/WOToI/T404W5a7CRKfSnXsu0qKMvuMdab34hrO7WBNbPiMwu zMQjJK6N6p7ustW0gWvgcQHCP2GJ8Dt1nlZXI= Received: by 10.204.13.68 with SMTP id b4mr642917bka.133.1271365013484; Thu, 15 Apr 2010 13:56:53 -0700 (PDT) Received: from [129.16.197.55] (dhcp-197-55.nomad.chalmers.se [129.16.197.55]) by mx.google.com with ESMTPS id 16sm1470903bwz.9.2010.04.15.13.56.52 (version=SSLv3 cipher=RC4-MD5); Thu, 15 Apr 2010 13:56:52 -0700 (PDT) Sender: =?UTF-8?B?UMOBTEkgR8OhYm9yIErDoW5vcw==?= Message-ID: <4BC77D14.9010304@FreeBSD.org> Date: Thu, 15 Apr 2010 22:54:44 +0200 From: Gabor PALI Organization: The FreeBSD Project User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.9) Gecko/20100331 Thunderbird/3.0.4 MIME-Version: 1.0 To: Peter Jeremy References: <20100415184100.GA75755@server.vk2pj.dyndns.org> In-Reply-To: <20100415184100.GA75755@server.vk2pj.dyndns.org> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-stable@FreeBSD.org Subject: Re: Strange Swapping Issues(?) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Apr 2010 20:56:55 -0000 Hello, On 04/15/10 20:41, Peter Jeremy wrote: > The out-of-swap hander will kill the largest process so one of your > problems is probably "throwto003". I can't offer any suggestion as > to why the swap_pager_getswapspace() errors continued afterwards. Okay, it was my fault. After huge processes, like "throwto003" are killed, swap usage falls below 3-4% in a second. (I will need to consult the author of that processes.) For your amusement, here is a log excerpt of the situation (it can by easily reproduced on my system). Columns are as follows: - Time - Swap Used (KB) [swapinfo] - Swap Free (KB) [swapinfo] - Number of Processes [ps] - Active Virtual Pages (KB) [vmstat] - Size of Free List (KB) [vmstat] Messages are pounding the logs at 17:16:37: "kernel: swap_pager_getswapspace(12): failed", and at 17:16:39 "throwto003" is finally killed: "kernel: pid 117 (throwto003), uid 1001, was killed: out of swap space", no further fails. 17:15:30 81720 4112584 155 2033636 71464 17:15:31 81720 4112584 155 2777084 9732 17:15:41 439384 3754920 155 2953212 10104 17:15:42 493804 3700500 155 3121148 10152 17:15:44 730560 3463744 155 3280812 10208 17:15:47 817608 3376696 155 3459068 9980 17:15:49 988420 3205884 155 3625008 10268 17:15:51 1182748 3011556 155 3799036 10036 17:15:53 1326872 2867432 155 3963900 8280 17:15:54 1445332 2748972 155 3969020 9632 17:15:55 1457348 2736956 155 4093928 40304 17:15:57 1598944 2595360 155 4139004 10568 17:15:58 1646460 2547844 155 4276220 8360 17:15:59 1689776 2504528 155 4365308 8476 17:16:00 1882924 2311380 155 4453372 89048 17:16:04 2049676 2144628 155 4567036 9860 17:16:06 2219004 1975300 155 4733948 10264 17:16:08 2389028 1805276 155 4903932 10012 17:16:10 2558516 1635788 155 5047396 99916 17:16:12 2735864 1458440 157 5254428 10532 17:16:15 2917096 1277208 158 5475788 100944 17:16:17 3103632 1090672 159 5716856 98876 17:16:20 3231660 962644 159 5929908 40144 17:16:24 3498256 696048 159 6168504 68192 17:16:28 3675324 518980 159 6238136 10700 17:16:30 3861972 332332 159 6414172 8500 17:16:32 4032132 162172 157 6556588 10324 17:16:33 4054828 139476 155 6718460 7932 17:16:34 4176548 17756 155 6719536 9584 17:16:38 4193604 700 155 6799908 71096 17:16:39 100580 4093724 151 1132240 1640856 17:16:40 96336 4097968 151 1123620 1637372 17:16:41 96332 4097972 153 1189608 1613748 17:16:42 96328 4097976 156 1269164 1529760 17:16:43 96272 4098032 154 1147628 1602084 Cheers, :g