From owner-freebsd-virtualization@freebsd.org Tue Jul 24 23:47:54 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ED13F105922A for ; Tue, 24 Jul 2018 23:47:53 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 64250853BF for ; Tue, 24 Jul 2018 23:47:53 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 137851D021 for ; Tue, 24 Jul 2018 23:47:52 +0000 (UTC) Subject: Re: Overcommitting CPUs with BHyve? To: freebsd-virtualization@freebsd.org References: From: Allan Jude Openpgp: preference=signencrypt Autocrypt: addr=allanjude@freebsd.org; prefer-encrypt=mutual; keydata= xsFNBFVwZcYBEADwrZDH0xe0ZVjc9ORCc6PcBLwS/RTXA6NkvpD6ea02pZ8lPOVgteuuugFc D34LdDbiWr+479vfrKBh+Y38GL0oZ0/13j10tIlDMHSa5BU0y6ACtnhupFvVlQ57+XaJAb/q 7qkfSiuxVwQ3FY3PL3cl1RrIP5eGHLA9hu4eVbu+FOX/q/XVKz49HaeIaxzo2Q54572VzIo6 C28McX9m65UL5fXMUGJDDLCItLmehZlHsQQ+uBxvODLFpVV2lUgDR/0rDa0B9zHZX8jY8qQ7 ZdCSy7CwClXI054CkXZCaBzgxYh/CotdI8ezmaw7NLs5vWNTxaDEFXaFMQtMVhvqQBpHkfOD 7rjjOmFw00nJL4FuPE5Yut0CPyx8vLjVmNJSt/Y8WxxmhutsqJYFgYfWl/vaWkrFLur/Zcmz IklwLw35HLsCZytCN5A3rGKdRbQjD6QPXOTJu0JPrJF6t2xFkWAT7oxnSV0ELhl2g+JfMMz2 Z1PDmS3NRnyEdqEm7NoRGXJJ7bgxDbN+9SXTyOletqGNXj/bSrBvhvZ0RQrzdHAPwQUfVSU2 qBhQEi2apSZstgVNMan0GUPqCdbE2zpysg+zT7Yhvf9EUQbzPL4LpdK1llT9fZbrdMzEXvEF oSvwJFdV3sqKmZc7b+E3PuxK6GTsKqaukd/3Cj8aLHG1T1im1QARAQABzSJBbGxhbiBKdWRl IDxhbGxhbmp1ZGVAZnJlZWJzZC5vcmc+wsF/BBMBAgApBQJVcGXGAhsjBQkSzAMABwsJCAcD AgEGFQgCCQoLBBYCAwECHgECF4AACgkQGZU1PhKYC34Muw/+JOKpSfhhysWFYiRXynGRDe07 Z6pVsn7DzrPUMRNZfHu8Uujmmy3p2nx9FelIY9yjd2UKHhug+whM54MiIFs90eCRVa4XEsPR 4FFAm0DAWrrb7qhZFcE/GhHdRWpZ341WAElWf6Puj2devtRjfYbikvj5+1V1QmDbju7cEw5D mEET44pTuD2VMRJpu2yZZzkM0i+wKFuPxlhqreufA1VNkZXI/rIfkYWK+nkXd9Efw3YdCyCQ zUgTUCb88ttSqcyhik/li1CDbXBpkzDCKI6I/8fAb7jjOC9LAtrZJrdgONywcVFoyK9ZN7EN AVA+xvYCmuYhR/3zHWH1g4hAm1v1+gIsufhajhfo8/wY1SetlzPaYkSkVQLqD8T6zZyhf+AN bC7ci44UsiKGAplB3phAXrtSPUEqM86kbnHg3fSx37kWKUiYNOnx4AC2VXvEiKsOBlpyt3dw WQbOtOYM+vkfbBwDtoGOOPYAKxc4LOIt9r+J8aD+gTooi9Eo5tvphATf9WkCpl9+aaGbSixB tUpvQMRnSMqTqq4Z7DeiG6VMRQIjsXDSLJEUqcfhnLFo0Ko/RiaHd5xyAQ4DhQ9QpkyQjjNf /3f/dYG7JAtoD30txaQ5V8uHrz210/77DRRX+HJjEj6xCxWUGvQgvEZf5XXyxeePvqZ+zQyT DX61bYw6w6bOwU0EVXBlxgEQAMy7YVnCCLN4oAOBVLZ5nUbVPvpUhsdA94/0/P+uqCIh28Cz ar56OCX0X19N/nAWecxL4H32zFbIRyDB2V/MEh4p9Qvyu/j4i1r3Ex5GhOT2hnit43Ng46z5 29Es4TijrHJP4/l/rB2VOqMKBS7Cq8zk1cWqaI9XZ59imxDNjtLLPPM+zQ1yE3OAMb475QwN UgWxTMw8rkA7CEaqeIn4sqpTSD5C7kT1Bh26+rbgJDZ77D6Uv1LaCZZOaW52okW3bFbdozV8 yM2u+xz2Qs8bHz67p+s+BlygryiOyYytpkiK6Iy4N7FTolyj5EIwCuqzfk0SaRHeOKX2ZRjC qatkgoD/t13PNT38V9tw3qZVOJDS0W6WM8VSg+F+bkM9LgJ8CmKV+Hj0k3pfGfYPOZJ/v18i +SmZmL/Uw2RghnwDWGAsPCKu4uZR777iw7n9Io6Vfxndw2dcS0e9klvFYoaGS6H2F13Asygr WBzFNGFQscN4mUW+ZYBzpTOcHkdT7w8WS55BmXYLna+dYer9/HaAuUrONjujukN4SPS1fMJ2 /CS/idAUKyyVVX5vozoNK2JVC1h1zUAVsdnmhEzNPsvBoqcVNfyqBFROEVLIPwq+lQMGNVjH ekLTKRWf59MEhUC2ztjSKkGmwdg73d6xSXMuq45EgIJV2wPvOgWQonoHH/kxABEBAAHCwWUE GAECAA8FAlVwZcYCGwwFCRLMAwAACgkQGZU1PhKYC34w5A//YViBtZyDV5O+SJT9FFO3lb9x Zdxf0trA3ooCt7gdBkdnBM6T5EmjgVZ3KYYyFfwXZVkteuCCycMF/zVw5eE9FL1+zz9gg663 nY9q2F77TZTKXVWOLlOV2bY+xaK94U4ytogOGhh9b4UnQ/Ct3+6aviCF78Go608BXbmF/GVT 7uhddemk7ItxM1gE5Hscx3saxGKlayaOsdPKeGTVJCDEtHDuOc7/+jGh5Zxpk/Hpi+DUt1ot 8e6hPYLIQa4uVx4f1xxxV858PQ7QysSLr9pTV7FAQ18JclCaMc7JWIa3homZQL/MNKOfST0S 2e+msuRwQo7AnnfFKBUtb02KwpA4GhWryhkjUh/kbVc1wmGxaU3DgXYQ5GV5+Zf4kk/wqr/7 KG0dkTz6NLCVLyDlmAzuFhf66DJ3zzz4yIo3pbDYi3HB/BwJXVSKB3Ko0oUo+6/qMrOIS02L s++QE/z7K12CCcs7WwOjfCYHK7VtE0Sr/PfybBdTbuDncOuAyAIeIKxdI2nmQHzl035hhvQX s4CSghsP319jAOQiIolCeSbTMD4QWMK8RL/Pe1FI1jC3Nw9s+jq8Dudtbcj2UwAP/STUEbJ9 5rznzuuhPjE0e++EU/RpWmcaIMK/z1zZDMN+ce2v1qzgV936ZhJ3iaVzyqbEE81gDxg3P+IM kiYh4ZtPB4Q= Message-ID: Date: Tue, 24 Jul 2018 19:47:47 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ufdwLLnHSBp9FFXIXB1kcmN0LElR0gzIA" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jul 2018 23:47:54 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --ufdwLLnHSBp9FFXIXB1kcmN0LElR0gzIA Content-Type: multipart/mixed; boundary="LnBBkp24fKFRDj36YuniA37gh3QUFhj40"; protected-headers="v1" From: Allan Jude To: freebsd-virtualization@freebsd.org Message-ID: Subject: Re: Overcommitting CPUs with BHyve? References: In-Reply-To: --LnBBkp24fKFRDj36YuniA37gh3QUFhj40 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 2018-07-24 17:30, Alan Somers wrote: > What are people's experiences with overcommitting CPUs in BHyve? I hav= e an > 8-core machine that often runs VMs totalling up to 5 allocated CPUs wit= hout > problems. But today I got greedy. I assigned 8 cores to one VM for a = big > build job. Obviously, some of those were shared with the host. I also= > assigned it 8GB of RAM (out of 16 total). Build performance fell throu= gh > the floor, even though the host was idle. Eventually I killed the buil= d > and restarted it with a more modest 2 make jobs (but the VM still had 8= > cores). Performance improved. But eventually the system seemed to be > mostly hung, while I had a build job running on the host as well as in = the > VM. I killed both build jobs, which resolved the hung processes. Then= I > restarted the host's build alone, and my system completely hung, with > top(1) indicating that many processes were in the pfault state. >=20 > So my questions are: > 1) Is it a known problem to overcommit CPUs with BHyve? > 2) Could this be related to the pfault hang, even though the guest was = idle > at the time? >=20 > -Alan > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@fr= eebsd.org" >=20 Bhyve has a command line flag, -p, to let you pin a vCPU to a physical CPU. This might avoid some of the issues with the threads hopping around all the time. If you were anyone else, I'd also ask if you ensured your vfs.zfs.arc_max was low enough to actually leave some ram for the VM to u= se. --=20 Allan Jude --LnBBkp24fKFRDj36YuniA37gh3QUFhj40-- --ufdwLLnHSBp9FFXIXB1kcmN0LElR0gzIA Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJbV7qnAAoJEBmVNT4SmAt+8fMQAJgOCZ4RdlDo0S6/La12v+Qm /TswUy8DFGtbNc0h6nDXUVPv4vjSFXUUS1mGV2dceRuqs567289EDYjK6DT0/jkd pPnhHDDWP5+kokLuLVbt34qp0vxia+5/CBcnxntFWSglH0bEzqN5wmqcpJYhUPuK 76k+soOdhvajUlBEE+UbE2cNZkFpYdgUo7WLGzh8116PJT2a+N0uIxHFbppwsSQb Vjh9M4rut/rI4gh0zxQyljbMFD/VW93AfwQ6KgE155GL+gVul2BNwX982AHlMxk5 vX1U5jUuLuatN9olEyhFhareVGM5mY8oAb8U6yUkLvn7+nN1FJcUCtsh/zsBIVAN boci1iH1IwsFCswlXk3NZbJqoON9M1z7woEXVqXQBtSsyPQ3hU1quymmTHM3Xb2d g1zBy3njmAZYUQ1DLxg0GV9j/X/1JVMOhZIR1NRyV0rFMdL3RGjH8QSXRlZrfdSR a8agfWLypqRK5dsxX2o7wx8GJa1iPRbXH6KSs5hQZpJ8izYizswRhv2inHWiWWvM B91md7GY1Olrk9B1LcyQ3ipOFvXTDKT6xrVYy2HNIPLtknSMXnab2mLnRIbqxiWo MVz0NMOIHPHVk58aaPnZikSlXViezSChJUX/HaYcuVE/0IORAoxDqIFuYdoo/+hm tWNPMUZOYzxkNnTLaIIz =9yAw -----END PGP SIGNATURE----- --ufdwLLnHSBp9FFXIXB1kcmN0LElR0gzIA--