From owner-freebsd-current@FreeBSD.ORG Thu Jul 17 16:12:46 2008 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C32C4106567C for ; Thu, 17 Jul 2008 16:12:46 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: from web63902.mail.re1.yahoo.com (web63902.mail.re1.yahoo.com [69.147.97.117]) by mx1.freebsd.org (Postfix) with SMTP id 8E81B8FC19 for ; Thu, 17 Jul 2008 16:12:46 +0000 (UTC) (envelope-from barney_cordoba@yahoo.com) Received: (qmail 38983 invoked by uid 60001); 17 Jul 2008 16:12:45 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Received:X-Mailer:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Message-ID; b=n9QhO8RmygwJbOSMI4tIft4OzqdD4Nd1XQ7C0DS4KGYPy9WjgEuBw5Nrs1PvSGmfHjlIgz7GCGrXlkdDRAWkZ2wrv2YAuTl7S1Z7Rcbhoil0TZ/8NH2sfxOJHo18n0qmR3gHm3yLmHecZ4tyibIbzzo+JHZKjhpEVA2KwJdXa/o=; Received: from [98.203.28.38] by web63902.mail.re1.yahoo.com via HTTP; Thu, 17 Jul 2008 09:12:45 PDT X-Mailer: YahooMailWebService/0.7.218 Date: Thu, 17 Jul 2008 09:12:45 -0700 (PDT) From: Barney Cordoba To: Steve Kargl In-Reply-To: <20080716211317.GA92354@troutmask.apl.washington.edu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Message-ID: <452221.38826.qm@web63902.mail.re1.yahoo.com> Cc: current@freebsd.org Subject: Re: ULE scheduling oddity X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: barney_cordoba@yahoo.com List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Jul 2008 16:12:46 -0000 --- On Wed, 7/16/08, Steve Kargl wrote: > From: Steve Kargl > Subject: Re: ULE scheduling oddity > To: "Barney Cordoba" > Cc: current@freebsd.org > Date: Wednesday, July 16, 2008, 5:13 PM > On Wed, Jul 16, 2008 at 07:49:03AM -0700, Barney Cordoba > wrote: > > --- On Tue, 7/15/08, Steve Kargl > wrote: > > > last pid: 3874; load averages: 9.99, 9.76, > 9.43 up 0+19:54:44 10:51:18 > > > 41 processes: 11 running, 30 sleeping > > > CPU: 100% user, 0.0% nice, 0.0% system, 0.0% > interrupt, 0.0% idle > > > Mem: 5706M Active, 8816K Inact, 169M Wired, 84K > Cache, 108M > > > Buf, 25G Free > > > Swap: 4096M Total, 4096M Free > > > > > > PID USERNAME THR PRI NICE SIZE RES > STATE C TIME WCPU COMMAND > > > 3836 kargl 1 118 0 577M 572M CPU7 > 7 6:37 100.00% kzk90 > > > 3839 kargl 1 118 0 577M 572M CPU2 > 2 6:36 100.00% kzk90 > > > 3849 kargl 1 118 0 577M 572M CPU3 > 3 6:33 100.00% kzk90 > > > 3852 kargl 1 118 0 577M 572M CPU0 > 0 6:25 100.00% kzk90 > > > 3864 kargl 1 118 0 577M 572M RUN > 1 6:24 100.00% kzk90 > > > 3858 kargl 1 112 0 577M 572M RUN > 5 4:10 78.47% kzk90 > > > 3855 kargl 1 110 0 577M 572M CPU5 > 5 4:29 67.97% kzk90 > > > 3842 kargl 1 110 0 577M 572M CPU4 > 4 4:24 66.70% kzk90 > > > 3846 kargl 1 107 0 577M 572M RUN > 6 3:22 53.96% kzk90 > > > 3861 kargl 1 107 0 577M 572M CPU6 > 6 3:15 53.37% kzk90 > > > > > > I would have expected to see a more evenly > distributed WCPU > > > of around 80% for each process. > > > > I don't see why "equal" distribution is > or should be a goal, as that > > does not guarantee optimization. > > The above images may be parts of an MPI application. > Synchronization > problems simply kill performance. The PIDs with 100% WCPU > could be > spinning in a loop waiting for PID 3861 to send a message > after > completing a computation. The factor of 2 difference in > TIME for > PID 3836 and 3861 was still observed after more than an > hour of > accumulated time for 3836. It appears as if the algorithm > for > cpu affinity is punishing 3846 and 3861. > > > Given that the cache is shared between only 2 cpus, it > might very well > > be more efficient to run on 2 CPUs when the 3rd or 4th > isn't needed. > > > > It works pretty darn well, IMO. Its not like your > little app is the > > only thing going on in the system > > Actually, 10 copies of the little app are the only things > running except > top(1) and few sleeping system services (e.g., nfsd and > sshd). Apparently, > you missed the "41 processes: 11 running, 30 > sleeping" line above. > > -- > Steve Your apparent argument that somehow every cpu cycle can be sliced equally and automagically is as silly as the expectation that a first generation scheduler will exhibit 100% efficiency across 8 cpus. Its just as likely an inefficiency in the application as in the kernel.