From owner-freebsd-virtualization@freebsd.org  Fri Jul 21 23:03:55 2017
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id 292B3C09E97
 for <freebsd-virtualization@mailman.ysv.freebsd.org>;
 Fri, 21 Jul 2017 23:03:55 +0000 (UTC)
 (envelope-from grehan@freebsd.org)
Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12])
 by mx1.freebsd.org (Postfix) with ESMTP id DD6E722B7
 for <freebsd-virtualization@freebsd.org>; Fri, 21 Jul 2017 23:03:54 +0000 (UTC)
 (envelope-from grehan@freebsd.org)
Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au
 [203.13.68.150])
 by alto.onthenet.com.au (Postfix) with ESMTPS id 31BE9204092E
 for <freebsd-virtualization@freebsd.org>;
 Sat, 22 Jul 2017 09:03:52 +1000 (AEST)
Received: from localhost (iredmail.onthenet.com.au [127.0.0.1])
 by iredmail.onthenet.com.au (Postfix) with ESMTP id 2AB86281C3B
 for <freebsd-virtualization@freebsd.org>;
 Sat, 22 Jul 2017 09:03:52 +1000 (AEST)
X-Amavis-Modified: Mail body modified (using disclaimer) -
 iredmail.onthenet.com.au
Received: from iredmail.onthenet.com.au ([127.0.0.1])
 by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026)
 with ESMTP id FAhvFj3Qjask for <freebsd-virtualization@freebsd.org>;
 Sat, 22 Jul 2017 09:03:52 +1000 (AEST)
Received: from Peters-MacBook-Pro-2.local
 (96-82-80-65-static.hfc.comcastbusiness.net [96.82.80.65])
 by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 7A394280F8B;
 Sat, 22 Jul 2017 09:03:51 +1000 (AEST)
Subject: Re: Bhyve Broken: whose fault (AMD, FreeBSD, ZFS ...?)
To: Zaphod Beeblebrox <zbeeble@gmail.com>
References: <CACpH0MewKeeif2egxD=hjwiXYa5tNV0ACTz6PrM=+Z3=7Nfryg@mail.gmail.com>
 <CACpH0MeozpUFakVg9z68hwyT9Ewx3sn1P6KorgzHPvoH7kcFdg@mail.gmail.com>
 <CACpH0Md4Dxq9jOSrzs9N7PbEG9F5mAsYaTjz3s+OfKCFkDQ2+Q@mail.gmail.com>
From: Peter Grehan <grehan@freebsd.org>
Cc: "freebsd-virtualization@freebsd.org" <freebsd-virtualization@freebsd.org>
Message-ID: <792ad5df-9e03-a332-c9a1-0b04ba69786a@freebsd.org>
Date: Fri, 21 Jul 2017 16:03:49 -0700
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:52.0)
 Gecko/20100101 Thunderbird/52.1.1
MIME-Version: 1.0
In-Reply-To: <CACpH0Md4Dxq9jOSrzs9N7PbEG9F5mAsYaTjz3s+OfKCFkDQ2+Q@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-CMAE-Score: 0
X-CMAE-Analysis: v=2.2 cv=XKlAcUpE c=1 sm=1 tr=0
 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=mwgbnDbW7alINpy3vhoKyg==:17
 a=IkcTkHD0fZMA:10 a=G3gG6ho9WtcA:10 a=fXAUDZ2eGBcBujxDsqUA:9
 a=QEXdDO2ut3YA:10 wl=host:3
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.23
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
 <freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>, 
 <mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>, 
 <mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 21 Jul 2017 23:03:55 -0000

> oh ... and ... the console spit out:
> 
> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 4522, size: 8192
> 
> (swap is on a separate zVol).

  Ok - you may have hit a separate issue. Is ZFS ARC limited on your 
setup ? If bhyve and ZFS (and other consumers) end up fighting for 
memory, everyone loses :( A general rule of thumb is to limit ARC to 
less than the bhyve VM usage + a few additional gig for the base system.

  The FreeBSD default of giving ZFS all memory minus 1GB doesn't work 
too well when running VMs.

later,

Peter.