From owner-freebsd-virtualization@freebsd.org Sat Dec 2 03:18:39 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 18F76DF02F7 for ; Sat, 2 Dec 2017 03:18:39 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: from mail-oi0-x22a.google.com (mail-oi0-x22a.google.com [IPv6:2607:f8b0:4003:c06::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D8C3C7E86F for ; Sat, 2 Dec 2017 03:18:38 +0000 (UTC) (envelope-from kmacybsd@gmail.com) Received: by mail-oi0-x22a.google.com with SMTP id r63so8469177oia.6 for ; Fri, 01 Dec 2017 19:18:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-transfer-encoding; bh=Cj0OC29vWntwxTcp2tw1Fio7AenCU1B+BsjxoBeUe2I=; b=Up89FjTFFT9tG+X7ZHULswAQxsdoL16fya9DPl7y6h87JcuOoUTfH8cYIHTvdxfgxM iqLJ+ittxxuNCVuGJXD88EiRmRDZh4OGG4JMpeyEubQscdibyfM0AUaQDSAvTqgQbyfN X+DB4CM8798d/0lfRUGaxR1KIk3/osuTmwM2aWeu6YS9/mOc7F2TmxU1X/2XuIMA/l41 y8SI0Z9/9HqThWZMVVeV6UOZ3ScXEMNhSxjZmVCps/9SgXm58sKGDF3yydhN5xkW9GO9 bSn70vsiFcyTU7lhpOuTdmIEdE56n1xuO9L9hq0QqFRuwZZVwoexGiUpIrW7bflVdv1j aHCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-transfer-encoding; bh=Cj0OC29vWntwxTcp2tw1Fio7AenCU1B+BsjxoBeUe2I=; b=ZStUJmZDlafypn8+YHPHQ95HDOjraCeorFiWxFFyUP1iobHurXiLfosflUm4VocBe7 5lIAw4Bm66KnjT3i8D4/mYw3ZOBG/g0xygXj/JD6CL4EPGUDcWSaJNNx69aP6Epbvbnt 9HstIkgUPdEYyujeAM6SbAFE0EkyNemNsspYwiHIdrtsgLSR02VrCeYRPt+uVcKch04K Hgmxo1yg0jTseRSsG2Djn1euGAnoZGhMwbvuH8a2x2OBF9ewHt5/PvTd3Fuyruoo7V0i 0f3u/cEmI68NYRcTzaXgtu6UEJgajzRPz7h7//o4k2TC20Fk+JtwJqDIbqDof9V6+V8k XBVw== X-Gm-Message-State: AJaThX6vNs0FBU8FoJUDH9VK0bK3GYiC2tRiXR5GA5IKnv0QTz8h3YVa IfPijzqySLxifMtjfiqR3jXqGAgck6eleCqqOCNFSw== X-Google-Smtp-Source: AGs4zMZsubcc0tcSnvYd3XWq0bnqTLsr7SiF2U7DffKWTK2P/lVW+AqLSgsn7wAXpknM5SNIhml2SPd2a+dOpnrDzXo= X-Received: by 10.202.231.197 with SMTP id e188mr9073377oih.131.1512184717581; Fri, 01 Dec 2017 19:18:37 -0800 (PST) MIME-Version: 1.0 Sender: kmacybsd@gmail.com Received: by 10.157.66.238 with HTTP; Fri, 1 Dec 2017 19:18:37 -0800 (PST) In-Reply-To: References: From: "K. Macy" Date: Fri, 1 Dec 2017 19:18:37 -0800 X-Google-Sender-Auth: 6UPeOrhOc8K8r7_zjfS9yjVqm8k Message-ID: Subject: Re: bhyve uses all available memory during IO-intensive operations To: Dustin Wenz Cc: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 02 Dec 2017 03:18:39 -0000 One thing to watch out for with chyves if your virtual disk is more than 20G is the fact that it uses 512 byte blocks for the zvols it creates. I ended up using up 1.4TB only half filling up a 250G zvol. Chyves is quick and easy, but it's not exactly production ready. -M On Thu, Nov 30, 2017 at 3:15 PM, Dustin Wenz wrote= : > I'm using chyves on FreeBSD 11.1 RELEASE to manage a few VMs (guest OS is= also FreeBSD 11.1). Their sole purpose is to house some medium-sized Postg= res databases (100-200GB). The host system has 64GB of real memory and 112G= B of swap. I have configured each guest to only use 16GB of memory, yet whi= le doing my initial database imports in the VMs, bhyve will quickly grow to= use all available system memory and then be killed by the kernel: > > kernel: swap_pager: I/O error - pageout failed; blkno 1735,size 4= 096, error 12 > kernel: swap_pager: I/O error - pageout failed; blkno 1610,size 4= 096, error 12 > kernel: swap_pager: I/O error - pageout failed; blkno 1763,size 4= 096, error 12 > kernel: pid 41123 (bhyve), uid 0, was killed: out of swap space > > The OOM condition seems related to doing moderate IO within the VM, thoug= h nothing within the VM itself shows high memory usage. This is the chyves = config for one of them: > > bargs -A -H -P -S > bhyve_disk_type virtio-blk > bhyve_net_type virtio-net > bhyveload_flags > chyves_guest_version 0300 > cpu 4 > creation Created on Mon Oct 23 16:17:04 CDT 201= 7 by chyves v0.2.0 2016/09/11 using __create() > loader bhyveload > net_ifaces tap51 > os default > ram 16G > rcboot 0 > revert_to_snapshot > revert_to_snapshot_method off > serial nmdm51 > template no > uuid 8495a130-b837-11e7-b092-0025909a8b56 > > > I've also tried using different bhyve_disk_types, with no improvement. Ho= w is it that bhyve can use far more memory that I'm specifying? > > - .Dustin