From owner-freebsd-hackers@FreeBSD.ORG Fri Dec 2 07:01:16 2011 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 990E4106567E for ; Fri, 2 Dec 2011 07:01:16 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 1BA088FC18 for ; Fri, 2 Dec 2011 07:01:15 +0000 (UTC) Received: by wgbdr11 with SMTP id dr11so491140wgb.31 for ; Thu, 01 Dec 2011 23:01:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to; bh=bHVhO7koc00lU6qYalmHZo3FGUC7/uqU6fmcqPJies0=; b=ZgiZGIjXa1eWO7FBT8w0BgKpAPAoZB5Te2MVhGfLJqLs3bI+kS/G058H/dkwSg24us Ke5btG+Xq/sjpxtLQR5lXA677C4nDzVUtiOc8Qr26nLtgJEhPFjMCxgw82bl8Up5pFhT anHVAnMqjTMoqgzaQR/PncWGSpX91cK0HUx/Y= Received: by 10.227.207.67 with SMTP id fx3mr4620949wbb.0.1322809274759; Thu, 01 Dec 2011 23:01:14 -0800 (PST) Received: from DataIX.net (ppp-21.70.dialinfree.com. [209.172.21.70]) by mx.google.com with ESMTPS id m5sm2397062wie.2.2011.12.01.23.01.09 (version=TLSv1/SSLv3 cipher=OTHER); Thu, 01 Dec 2011 23:01:13 -0800 (PST) Sender: Jason Hellenthal Received: from DataIX.net (localhost [127.0.0.1]) by DataIX.net (8.14.5/8.14.5) with ESMTP id pB2715Ip000904 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 2 Dec 2011 02:01:05 -0500 (EST) (envelope-from jhell@DataIX.net) Received: (from jhell@localhost) by DataIX.net (8.14.5/8.14.5/Submit) id pB270t9Z000903; Fri, 2 Dec 2011 02:00:55 -0500 (EST) (envelope-from jhell@DataIX.net) Date: Fri, 2 Dec 2011 02:00:55 -0500 From: Jason Hellenthal To: Steven Hartland Message-ID: <20111202070055.GA98731@DataIX.net> References: <547298A3C38F407887E1AAAAC487DF6D@multiplay.co.uk> <20111201065722.GA97051@DataIX.net> <1A338C6C470940B386C307641E350A3E@multiplay.co.uk> <4ED77650.6050409@my.gd> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="5vNYLRcllDrimb99" Content-Disposition: inline In-Reply-To: Cc: freebsd-hackers@freebsd.org Subject: Re: Invalid memory stats from vmstat and sysctl vm.vmtotal? X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Dec 2011 07:01:16 -0000 --5vNYLRcllDrimb99 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Dec 01, 2011 at 01:23:35PM -0000, Steven Hartland wrote: > ----- Original Message -----=20 > From: "Damien Fleuriot" >=20 > >> I could understand a bit of overflow as stats are snapshots which may = not > >> be instuntanious, but 31GB instead of under 8GB is hardly a rounding > >> issue / > >> overflow. > >>=20 > >> With respect to top showing greater than 100% by how much are you talk= ing? > >> Do your realise that each core =3D 100%? So if you have a quad core yo= ur > >> system > >> total will be 400% not 100%? > >>=20 > >=20 > > That's his point, you cannot use 400% of a system as a whole, his point > > is that top should report 100% where each core accounts for 25% >=20 > Then I would have to disagree, keeping 100% to mean 100% of a single core > is much easier to manage than 100% of a machines total capacity. >=20 > If you went to 100% =3D the machine total capacity processes could be usi= ng > a lot of cpu without even registering 1% on today's machines where 24 cor= es > is common place. >=20 > It also makes detecting single process / thread bottlenecks easier as if > your seeing 100% you know its maxing a core, instead of having to calcula= te > it once you know how many cores the machine has. >=20 > If your looking for total machine usage then that's also their in the sum= mary > at the top of the screen e.g. > CPU states: 13.6% user, 0.0% nice, 1.3% system, 0.0% interrupt, 85.1% = idle >=20 > Anyway this is quite off topic, and I don't want to loose sight of the th= reads > goal which is to determine why we can see 31GB of usage on an 8GB machine > with very little shared memory usage an no swap usage. >=20 Just to put some visuals to this... =2E `-- DIE |-- Core1 [Idle] |-- Core2 [35% ] | `-- thread127 |-- Core3 [40% ] | `-- thread127 `-- Core4 [100%] `-- thread127 In this case you would say the DIE should be at a total of 175% ? $(((25*0)+(25*0.35)+(25*0.40)+(25*1.00))) Out of sanity and each core only being 25% of the total DIE it should be re= porting. It is using all together 43.75% of the total DIE. But thats not wh= at I see even on SP machines. 1 DIE 1 core and a report of 338% usage for 8= threads of firefox. If someone was attempting to write a scheduler to laun= ch processes & yield back to the scheduler to launch more based on processo= r usage either by core or by die totals that scheduler would be ineffective= at best without alot of kludges put in place to handle all the misinterpre= tation. Somewhere along the line our math has been distorted or the move fr= om SP -> MP, cores, hyperthreading etc.. has just not completed yet and sho= uld not be ignored. --5vNYLRcllDrimb99 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJO2HenAAoJEJBXh4mJ2FR+KtMH/jdx+xhAgNTyDOAUce/PtOHI zREdl3XEnhFfi868XLXYbx98/QSMokqzjXC6IqiP8XFtPG5UwB05KERe4U/MnxKN Ord/wg57gHZuBWROmSgnbDgCKmXMom605ZOT78DAZNfFKmnPg4tAqWISdb+ukzFg K6xyNjlQcnyhiIAN0q4wClSVkYVT+Aqk0L4X5+C3NfJBhqZBXqiZ0tr9Ppkw9+HZ BXIHka1FyJvUW9+Gr74/KWYGsAFfqh5fBLDdtAO0PvhJRjp4N16V4IE9nn6RK2H5 05sdzjal5vfx7pKy96A5z8gNsjy6Yjyflrg7eISpAy+rMpicllw5RLak02JP118= =wqDh -----END PGP SIGNATURE----- --5vNYLRcllDrimb99--