From owner-freebsd-virtualization@freebsd.org Tue Jul 24 22:13:16 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 693D51057A61 for ; Tue, 24 Jul 2018 22:13:16 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id F2BB5829DC for ; Tue, 24 Jul 2018 22:13:15 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id B75D91057A60; Tue, 24 Jul 2018 22:13:15 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 952DE1057A5F for ; Tue, 24 Jul 2018 22:13:15 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-ed1-x52b.google.com (mail-ed1-x52b.google.com [IPv6:2a00:1450:4864:20::52b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E90ED829D8 for ; Tue, 24 Jul 2018 22:13:14 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: by mail-ed1-x52b.google.com with SMTP id s16-v6so5394143edq.12 for ; Tue, 24 Jul 2018 15:13:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=LQjiMCU4zZr7SN8UZ3c30iqC8ER0LBZYn9UViCxFqMo=; b=aJjGbuOZJbzqD/RK8I7GOKZiiuXsAklfpGapWHeSi8rhzu0+45mDhcwKOrRmcAoVbl y8UbQc1yPGFmo72U0/FIA48JDZDQ5Sluk/GGQcTo9n/y1t8F8g0n2MELITplW1zdhbyc pXcqMdl4EigVoghDU30mBYEZGi6pUvi3s1a777CXnBPI6IeO157uGp3L0VOcfBbHXX3Y Jhs2XHCusZTAfXtww37mCdaPQoCcoeSbHnEn9HgX5mngZcyv+Gcsufvhi0PI6UplpI8C BNjAcwHxWE69gCa1D06H0gRbE6pu3Pq9B2UwBBldF2qQna+jLi2Odlin81PtoWiCZKio 4/hQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=LQjiMCU4zZr7SN8UZ3c30iqC8ER0LBZYn9UViCxFqMo=; b=f9kySQu4LEBw8GDnbDbN6weoZRe45SKNmokycMtFHx7SDdjtWQQu6SVUvxLhuHSghO gIsut5w/BPdGEftWKE1XAZub+rC6PRzo24wdJuKBNsJC/HyxzHNQeVCL8minDSJtOKj9 4r5Yl2rN8RLxB6qW5SXYzVpfdb1e9QNvmfEACcS0rKL5b0pDEhmG63GnzOxf2SQMEheI FVlnw9ZChrAdx6ZRfOqfYSDfGviO3MgN9gybuqFdDCXb6Gmikx1We9cfefR0h5hDRiFo TuPozIvAMHsIIuL/JSuyCGc272YeCuP+6q1CYPiT7i59kRSpK/0fKCO5VpdoGlEwhYPU eNnA== X-Gm-Message-State: AOUpUlEv1LZW2ePD1hrUfz1stHs9QEXN8bhoKxVm9ezxs1KXrMBNOGuh yfrUd5PGKKWO0iVF+QLwGHbH/dFJ090eGw== X-Google-Smtp-Source: AAOMgpd8DXcqLzoDeb1ttFNBsM+Rd8IrrFIqC1CBHLKQoIVecx7upTJifJmJfzvB3gyIXEGSRIlXdA== X-Received: by 2002:a50:b4b0:: with SMTP id w45-v6mr20614702edd.254.1532470392541; Tue, 24 Jul 2018 15:13:12 -0700 (PDT) Received: from mutt-hbsd ([178.19.111.150]) by smtp.gmail.com with ESMTPSA id d31-v6sm5843497edd.73.2018.07.24.15.13.09 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Jul 2018 15:13:10 -0700 (PDT) Date: Tue, 24 Jul 2018 18:12:30 -0400 From: Shawn Webb To: Alan Somers Cc: "freebsd-virtualization@freebsd.org" Subject: Re: Overcommitting CPUs with BHyve? Message-ID: <20180724221230.tcljojufbwdl5ywy@mutt-hbsd> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="fbaxk42dnpdw3af4" Content-Disposition: inline In-Reply-To: X-Operating-System: FreeBSD mutt-hbsd 12.0-CURRENT FreeBSD 12.0-CURRENT X-PGP-Key: http://pgp.mit.edu/pks/lookup?op=vindex&search=0x6A84658F52456EEE User-Agent: NeoMutt/20180622 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jul 2018 22:13:16 -0000 --fbaxk42dnpdw3af4 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Jul 24, 2018 at 03:30:32PM -0600, Alan Somers wrote: > What are people's experiences with overcommitting CPUs in BHyve? I have = an > 8-core machine that often runs VMs totalling up to 5 allocated CPUs witho= ut > problems. But today I got greedy. I assigned 8 cores to one VM for a big > build job. Obviously, some of those were shared with the host. I also > assigned it 8GB of RAM (out of 16 total). Build performance fell through > the floor, even though the host was idle. Eventually I killed the build > and restarted it with a more modest 2 make jobs (but the VM still had 8 > cores). Performance improved. But eventually the system seemed to be > mostly hung, while I had a build job running on the host as well as in the > VM. I killed both build jobs, which resolved the hung processes. Then I > restarted the host's build alone, and my system completely hung, with > top(1) indicating that many processes were in the pfault state. >=20 > So my questions are: > 1) Is it a known problem to overcommit CPUs with BHyve? > 2) Could this be related to the pfault hang, even though the guest was id= le > at the time? VMWare's ESXi uses a special scheduler to do what it does. I wonder if it would be worthwhile to investigate implementing a scheduler in FreeBSD that provides decent performance for virtualized workloads. Thanks, --=20 Shawn Webb Cofounder and Security Engineer HardenedBSD Tor-ified Signal: +1 443-546-8752 Tor+XMPP+OTR: lattera@is.a.hacker.sx GPG Key ID: 0x6A84658F52456EEE GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE --fbaxk42dnpdw3af4 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEKrq2ve9q9Ia+iT2eaoRlj1JFbu4FAltXpEoACgkQaoRlj1JF bu7rLw/8DnZEhK1tcH+AlRwd3WlXtAeoSnTM8OLW0fR6jOOIYf6cS/+cxL0ALqca Pod0YLDY+rVMYKCZVxS7F8T5eTbiOxYBuaSM1lWKFr0wXaIxJ/EHm3S/Rodoag9Y 1BMtmo65FAN9jBoBjCXT6db3f/KfLKvHKDWtZrS6GdpQRtZPmIIPsmSAbp0wcKlv 0Dw71lyY23krOKx7OHq2ICrxSr6ewrYdnIHx6hQHWxKcpBqYsMbsH5VrXpi4IfWr G9uWGb44jDGxbFcGsLUW1mAAIdlTzgpyItBzcAEX9vZpF/1OKI6RK1BSD6AI7cEp ZKuRggDWjGKwaVafH0wXhm85QEk/okhRnU+KXqfJ5snaw/RiKuIs4mdjE1Kvv+hA /xDctlA5xr06X80rRSySf26XPmg4rbDuAyoHmgJsPCKU1Fx3UunT+4c21lHI0drm 5V4Buf3R2u2lLG/7Dc6Guqh4fDDOg65cfI7ZgSM6/FDgd/KjTa6AM9YgA18E+XdQ VOIDSyyZO5rKwNR5MD6InLheCGi+wIHWaPpj+jb7qYJOJZ7NPMKK2oJgpKQIMms4 VyiJQ4ZXb8e255PiiWH0EctuPVcNpvgR2Hj+Pgu57a1G/OB4TnLCDbp3hWh/Gfu9 tNv33XtE2SV6FTQIjxJi/YfCLT5UiCCMCEE8145PijxYHSlqSdg= =vfCa -----END PGP SIGNATURE----- --fbaxk42dnpdw3af4--