From owner-freebsd-questions@FreeBSD.ORG Thu Apr 6 13:53:39 2006 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DDD0816A426 for ; Thu, 6 Apr 2006 13:53:39 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from pi.codefab.com (pi.codefab.com [199.103.21.227]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1840343D67 for ; Thu, 6 Apr 2006 13:53:23 +0000 (GMT) (envelope-from cswiger@mac.com) Received: from localhost (localhost [127.0.0.1]) by pi.codefab.com (Postfix) with ESMTP id 7355B5CFF; Thu, 6 Apr 2006 09:53:22 -0400 (EDT) Received: from pi.codefab.com ([127.0.0.1]) by localhost (pi.codefab.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 00544-06; Thu, 6 Apr 2006 09:53:21 -0400 (EDT) Received: from [192.168.1.3] (pool-68-161-112-80.ny325.east.verizon.net [68.161.112.80]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by pi.codefab.com (Postfix) with ESMTP id 6D6015C6D; Thu, 6 Apr 2006 09:53:21 -0400 (EDT) Message-ID: <44351D4F.8020900@mac.com> Date: Thu, 06 Apr 2006 09:53:19 -0400 From: Chuck Swiger User-Agent: Thunderbird 1.5 (Windows/20051201) MIME-Version: 1.0 To: Karl Ma References: <42a5f58c0604060143g5e68e806r870761a1beeefbb5@mail.gmail.com> In-Reply-To: <42a5f58c0604060143g5e68e806r870761a1beeefbb5@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: amavisd-new at codefab.com Cc: freebsd-questions@freebsd.org Subject: Re: Giving more CPU time to a swapping process? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Apr 2006 13:53:39 -0000 Karl Ma wrote: > The process is so memory-hungry that it starts swap after the physical RAM > max out. (To be exact, I've lowered the per-process limitation to make this > possible). What did you lower, exactly? If you reduce the max resident datasize needlessly, you're going to make your program swap more and run much slower. > However, when I use top to monitor the status, the STATE of the process > started to stay as "swread" for most of the time (instead of RUN before > using swap) and its priority has dropped to -20; and the corresponding WCPU > drops to around 1% only. And the CPU consumption time in total (for the > whole job) would only increase a minute or two even the process has been > running for more than a few hours. Yes, because the task isn't using much CPU, it's entirely I/O bound. > In Windows XP, which has less per-task resource restriction (I guess?), I > did successfully complete the task on the same hardware machine; although it > takes more than 30 mins. > > How can I push up the priority of the whole paging task? How can I > allocate more CPU attention to this process? I've tried using "nice" > but it does not help. Won't help. Add more RAM, or adjust the program to be more clever about the use of memory, possibly by using Numeric/numarray. The size of your python process is surprising to me, python tends to run relatively lightweight process sizes even when handling large data sets (ie, > 1GB of data per day)... -- -Chuck