From owner-freebsd-virtualization@FreeBSD.ORG Thu Apr 24 16:09:00 2014 Return-Path: Delivered-To: freebsd-virtualization@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ED3556F9; Thu, 24 Apr 2014 16:09:00 +0000 (UTC) Received: from mail-lb0-x233.google.com (mail-lb0-x233.google.com [IPv6:2a00:1450:4010:c04::233]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 447451CD6; Thu, 24 Apr 2014 16:09:00 +0000 (UTC) Received: by mail-lb0-f179.google.com with SMTP id q8so2155761lbi.38 for ; Thu, 24 Apr 2014 09:08:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; bh=zj+moZUYwbHlp6VG+wvcR5fnPg1NRDoZF2AUKHJxpJY=; b=Nfkr+d3t/wr37RquNEXFoqfuzJw2YkpH4GcFUICPF1Cbjd2odB381DtapfeJK4jHQh ZsIwpJEptfDe2SKi7O58h5gzkE/PYmZJuXBjHG8jZ68B0+nru0ZLI1WqU+EsOQKCtBfL l0hCMS04S0pFL/+0LXNFkBUfAERv3v/pJoH8+/BfOdYtUA2cIrE0MqM/SxdQo9h/e5fD xlgJNWBz7RdLlfQrXEngjn0lFYP5yFtS3QAmO9Gr+Ih5ZE0+5y7K3yTNNJ5kDCJFu2xf PCO/13jYRkMbONqJxPNP5QypYk5clZ1PgcP5R1n0B2KAvKVODCv7ufFaqInWHRvjIehe flKA== X-Received: by 10.112.92.109 with SMTP id cl13mr1708220lbb.50.1398355736318; Thu, 24 Apr 2014 09:08:56 -0700 (PDT) Received: from kloomba ([217.65.211.142]) by mx.google.com with ESMTPSA id t3sm4334866laj.15.2014.04.24.09.08.53 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 24 Apr 2014 09:08:53 -0700 (PDT) Sender: Roman Bogorodskiy Date: Thu, 24 Apr 2014 20:08:17 +0400 From: Roman Bogorodskiy To: Peter Grehan Subject: Re: Understanding CPU and memory usage in Bhyve Message-ID: <20140424160816.GB3494@kloomba> References: <20140421102138.GA6157@kloomba> <535557BC.8030300@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="lEGEL1/lMxI0MVQ2" Content-Disposition: inline In-Reply-To: <535557BC.8030300@freebsd.org> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-virtualization@FreeBSD.org X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Apr 2014 16:09:01 -0000 --lEGEL1/lMxI0MVQ2 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Peter Grehan wrote: > Hi Roman, >=20 > > For the CPU, there's some info provided by bhyvectl, e.g.: > > > > /usr/sbin/bhyvectl --vm=3Dbhyve --get-stats --cpu=3D1 > > > > in the field 'vcpu total runtime', for example: > > > > vcpu total runtime 8178870653 > > > > Two question about that: > > > > 1. What are the units used here? Looks like it's ticks, so if I want = to > > convert this number, to, say, nanoseconds, would it be right to do > > something like: > > > > X * 1000**3 / kern.clockrate.stathz > > > > (X beeing the value returned by bhyvectl and kern.clockrate is a > > sysctl name)? >=20 > It's in TSC units so you can convert to nsecs using the=20 > machdep.tsc_freq sysctl node. >=20 > > 2. This value shows only vcpu time, without hypervisor time? >=20 > That's correct. To get the hypervisor and vCPU time together, you can= =20 > use existing CPU usage reporting (e.g. that used by top et al). Thanks for the clarification, it's very helpful. However, I need some more help with that. I'm trying to implement some domain stats reporting for the bhyve driver in libvirt and it looks like I choose a wrong approach initially.=20 It's described here: https://www.redhat.com/archives/libvir-list/2014-April/msg00912.html But the summary is: is there a way to figure out how much CPU time bhyve and the guest spends on host CPUs N (N =3D 0, 1, ...)? > > For the memory stats, it's less obvious. Does guest always use an amount > > of memory specified at startup with '-m' (i.e. bhyve -m 1G) or it could > > use less? >=20 > That's the maximum it can use. We currently don't expose the incore=20 > amount from the vmspace representing the guest's physical - that should= =20 > be added to bhyvectl soon. Looking forward to that. > > What is the format of bhyvectl --get-{low,high}mem output? >=20 > The number reported there isn't to do with guest usage, but how the=20 > memory is divided between < 4G and above 4G. For example, the default=20 > for bhyve is to use up to 3G below 4G, provide 1GB for PCI MMIO decode,= =20 > and then put the remainder of guest memory above 4G. Here's an example=20 > for an 8G VM, with 3G of guest mem below 4G (starting address 0), and 5G= =20 > above 4G (starting address 4G): >=20 > lowmem 0x0000000000000000/3221225472 > highmem 0x0000000100000000/5368709120 >=20 > > PS I found it a little confusing that bhyvectl displays vcpu0 stats by > > default if --cpu is not provided, expected it provide info of all vcpus > > in the guest by default. >=20 > Yes, that's a relic of the days when there was only a single vCPU=20 > supported :( >=20 > > Also, didn't find a way to get a number of vcpus in a running guest. >=20 > Until the vCPU state is exposed by bhyvectl, or we provide a sysctl,=20 > you can use heuristics: the number of vCPU threads for the bhyve=20 > process, or scan all vCPUs and only count those that have a non-zero RIP. >=20 > later, >=20 > Peter. >=20 Thanks, Roman Bogorodskiy --lEGEL1/lMxI0MVQ2 Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.13 (FreeBSD) iQEcBAEBAgAGBQJTWTbwAAoJEMltX/4IwiJqEygH/ippVMN8384c2RA8G8q2/jE1 LU2f4tyJaNQ/SMQtl3Ed0RTr8KThal2wXXGnujEw3TJaYTFDFaZ31rHAFT1/cPTG szuMgZ6mO47anNInoOBTQu0jOxgeUv/1z0V/zd+WDidFPRSI5YQFHeQe47/TjFqQ fJEUo6dExYeQl1tMkxi1oC1xvi04zbZgMuNPpeGUQBhpJ8MZkSILIEaiHWwFAOex g8hii/zHF9bEgNG/NnNPgrc56dIqn5kv736nLuWTHHog1pleTsW0qJL+X3IBIVZu CSKQ5twHMgWw57xXmpoMZwsIORkHb1AUHBHMZKbYA3WFKBNivrKB4x+6D5SR2S4= =ERSu -----END PGP SIGNATURE----- --lEGEL1/lMxI0MVQ2--