From owner-freebsd-virtualization@freebsd.org Wed Jul 25 00:14:34 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 490081059D07 for ; Wed, 25 Jul 2018 00:14:34 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id DB629863BD for ; Wed, 25 Jul 2018 00:14:33 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id A00D21059D06; Wed, 25 Jul 2018 00:14:33 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7DFDE1059D05 for ; Wed, 25 Jul 2018 00:14:33 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: from mail-oi0-f65.google.com (mail-oi0-f65.google.com [209.85.218.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 21A70863BB; Wed, 25 Jul 2018 00:14:33 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: by mail-oi0-f65.google.com with SMTP id l10-v6so10804756oii.0; Tue, 24 Jul 2018 17:14:33 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=ZDed5HDT2HZ4AHtztTS7En1DcZrwiqU3zW/4o9MeGW8=; b=PBt2Q9j6hWxD0jqnUaXOeKanI2PpeouCytxY/6cZeuVdE9QmCitvHVf9SjwIpYrh2W Z/KN2BMfPKkrUcJD/Hb8/nDwXLtd/CQ+kshZUaYafJrVlVTw2tZAq+xSCg/H2sQgIHuI dRXoIybrb9Fb1fS2cJde5MzfblAKTMAf8+9YMZfP1wzQ0R76n9gulP9REIvW1doNTLcn wOLWvb1Eyam7pI0Wykmf7xDkrAyn3LKIKUG5N6eSbhM6ym4IvVTpUI00fNhHNB6eyiRY 8IV6xr3f7gmXkPhyv4tUX1XqoqLJxDc0oN6JA/Efn9Md3zxmi8DoUnua4i0++gZhRxw+ Yltg== X-Gm-Message-State: AOUpUlHpcmc2Ix8AVo21yu30VHqAbvihg1I8BcaNmJ4pBMZYYiwP0zt2 7pouj/PrKhynq1ivY755X3QoqwbVW6ttmh0LNxaO2A== X-Google-Smtp-Source: AAOMgpeqxd5ampMSAOeRqniy//lHF7ZgYSjm6jofwQlO/9fOLNhgMsAy/gd5duEb/M/zabSkFVy7oB5O497JO10Ivis= X-Received: by 2002:aca:6142:: with SMTP id v63-v6mr1006842oib.137.1532474363720; Tue, 24 Jul 2018 16:19:23 -0700 (PDT) MIME-Version: 1.0 References: <20180724221230.tcljojufbwdl5ywy@mutt-hbsd> In-Reply-To: <20180724221230.tcljojufbwdl5ywy@mutt-hbsd> From: Jason Tubnor Date: Wed, 25 Jul 2018 09:19:12 +1000 Message-ID: Subject: Re: Overcommitting CPUs with BHyve? To: Shawn Webb Cc: Alan Somers , "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.27 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jul 2018 00:14:34 -0000 On Wed, 25 Jul 2018 at 08:12, Shawn Webb wrote: > On Tue, Jul 24, 2018 at 03:30:32PM -0600, Alan Somers wrote: > > What are people's experiences with overcommitting CPUs in BHyve? I have > an > > 8-core machine that often runs VMs totalling up to 5 allocated CPUs > without > > problems. But today I got greedy. I assigned 8 cores to one VM for a > big > > build job. Obviously, some of those were shared with the host. I also > > assigned it 8GB of RAM (out of 16 total). Build performance fell through > > the floor, even though the host was idle. Eventually I killed the build > > and restarted it with a more modest 2 make jobs (but the VM still had 8 > > cores). Performance improved. But eventually the system seemed to be > > mostly hung, while I had a build job running on the host as well as in > the > > VM. I killed both build jobs, which resolved the hung processes. Then I > > restarted the host's build alone, and my system completely hung, with > > top(1) indicating that many processes were in the pfault state. > > > > So my questions are: > > 1) Is it a known problem to overcommit CPUs with BHyve? > > 2) Could this be related to the pfault hang, even though the guest was > idle > > at the time? > 1) Not that I have experienced. 2) More likely RAM pressure. Are you running ZFS? What is you ARC capped at? (Total guest + System + ARC < System Total Ram) > VMWare's ESXi uses a special scheduler to do what it does. I wonder if > it would be worthwhile to investigate implementing a scheduler in > FreeBSD that provides decent performance for virtualized workloads. > > >