From owner-freebsd-virtualization@freebsd.org  Fri Jul 21 22:58:25 2017
Return-Path: <owner-freebsd-virtualization@freebsd.org>
Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id B0D41C09B48
 for <freebsd-virtualization@mailman.ysv.freebsd.org>;
 Fri, 21 Jul 2017 22:58:25 +0000 (UTC)
 (envelope-from grehan@freebsd.org)
Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12])
 by mx1.freebsd.org (Postfix) with ESMTP id 720E41EB5
 for <freebsd-virtualization@freebsd.org>; Fri, 21 Jul 2017 22:58:25 +0000 (UTC)
 (envelope-from grehan@freebsd.org)
Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au
 [203.13.68.150])
 by alto.onthenet.com.au (Postfix) with ESMTPS id 0F04720AE974
 for <freebsd-virtualization@freebsd.org>;
 Sat, 22 Jul 2017 08:58:18 +1000 (AEST)
Received: from localhost (iredmail.onthenet.com.au [127.0.0.1])
 by iredmail.onthenet.com.au (Postfix) with ESMTP id 00178281258
 for <freebsd-virtualization@freebsd.org>;
 Sat, 22 Jul 2017 08:58:17 +1000 (AEST)
X-Amavis-Modified: Mail body modified (using disclaimer) -
 iredmail.onthenet.com.au
Received: from iredmail.onthenet.com.au ([127.0.0.1])
 by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026)
 with ESMTP id NEC8gytIPgoT for <freebsd-virtualization@freebsd.org>;
 Sat, 22 Jul 2017 08:58:17 +1000 (AEST)
Received: from Peters-MacBook-Pro-2.local
 (96-82-80-65-static.hfc.comcastbusiness.net [96.82.80.65])
 by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 5039F2809A8;
 Sat, 22 Jul 2017 08:58:16 +1000 (AEST)
Subject: Re: Bhyve Broken: whose fault (AMD, FreeBSD, ZFS ...?)
To: Zaphod Beeblebrox <zbeeble@gmail.com>
References: <CACpH0MewKeeif2egxD=hjwiXYa5tNV0ACTz6PrM=+Z3=7Nfryg@mail.gmail.com>
Cc: "freebsd-virtualization@freebsd.org" <freebsd-virtualization@freebsd.org>
From: Peter Grehan <grehan@freebsd.org>
Message-ID: <490d70e2-ff8b-2a60-2eb7-669dadbcc2a9@freebsd.org>
Date: Fri, 21 Jul 2017 15:58:12 -0700
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:52.0)
 Gecko/20100101 Thunderbird/52.1.1
MIME-Version: 1.0
In-Reply-To: <CACpH0MewKeeif2egxD=hjwiXYa5tNV0ACTz6PrM=+Z3=7Nfryg@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-CMAE-Score: 0
X-CMAE-Analysis: v=2.2 cv=XKlAcUpE c=1 sm=1 tr=0
 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=mwgbnDbW7alINpy3vhoKyg==:17
 a=IkcTkHD0fZMA:10 a=G3gG6ho9WtcA:10 a=fCqCzJ7R-124FQafWwsA:9
 a=QEXdDO2ut3YA:10 wl=host:3
X-BeenThere: freebsd-virtualization@freebsd.org
X-Mailman-Version: 2.1.23
Precedence: list
List-Id: "Discussion of various virtualization techniques FreeBSD supports."
 <freebsd-virtualization.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-virtualization>, 
 <mailto:freebsd-virtualization-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-virtualization/>
List-Post: <mailto:freebsd-virtualization@freebsd.org>
List-Help: <mailto:freebsd-virtualization-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization>, 
 <mailto:freebsd-virtualization-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 21 Jul 2017 22:58:25 -0000

Hi,

> What should be next steps here?  This is repeatable.  The host is stable
> (it can makeworld -j32 in about 25 minutes ... so it's hardware seems
> good).  Is this an AMD bug?  Is it bad to use ZFS ZVols?

  ZVols are fine. Is the guest panic a spinlock timeout ?

  I believe this is a bug in bhyve/SVM. It appears somewhat related to 
processor speed (I can't repro on a 2.3GHz 8 CPU Opteron 6320, but can 
hit it after 15 mins or so on a Ryzen 1700, with/without SMT).

  Anish and I are currently chasing this and have repros. An experiment 
you could try is to run with the vCPUs pinned i.e. for a 4 vCPU guest, 
add the options "-p 0:1 -p 1:2 -p 2:3 -p 3:4".

later,

Peter.