From owner-freebsd-virtualization@freebsd.org Tue Feb 27 23:31:00 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 91D49F33EA8 for ; Tue, 27 Feb 2018 23:31:00 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id 0B2EA76F0D for ; Tue, 27 Feb 2018 23:30:59 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id 9C09320768C0 for ; Wed, 28 Feb 2018 09:30:57 +1000 (AEST) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id 96B832820A8 for ; Wed, 28 Feb 2018 09:30:57 +1000 (AEST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id iCsWwKg3tWIu for ; Wed, 28 Feb 2018 09:30:57 +1000 (AEST) Received: from Peters-MacBook-Pro-2.local (c-67-180-92-13.hsd1.ca.comcast.net [67.180.92.13]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id E660D2809BD; Wed, 28 Feb 2018 09:30:55 +1000 (AEST) Subject: Re: Ryzen lockup on bhyve was (Re: new Ryzen lockup issue ?) To: Mike Tancsa Cc: "freebsd-virtualization@freebsd.org" References: <92a60e14-f532-2647-d45d-b500fc59ba88@sentex.net> <425be16f-9fdc-9ed6-72b1-02e28bfd130f@sentex.net> <2f2b6f96-d6ef-8e02-966d-dcbbfb0c92fb@freebsd.org> <48768738-e699-fd06-9154-9b5be1bbabcc@sentex.net> From: Peter Grehan Message-ID: Date: Tue, 27 Feb 2018 15:30:53 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <48768738-e699-fd06-9154-9b5be1bbabcc@sentex.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-CMAE-Score: 0 X-CMAE-Analysis: v=2.3 cv=KPZ08mNo c=1 sm=1 tr=0 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=5eVCmCvhg37cu/pjidAGzw==:17 a=IkcTkHD0fZMA:10 a=Op4juWPpsa0A:10 a=pwdUeM8-EA_1Fz4opM4A:9 a=QEXdDO2ut3YA:10 wl=host:3 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 27 Feb 2018 23:31:00 -0000 Hi Mike, >> =C2=A0I could reproduce this on a Ryzen 1700 with HT enabled - it too= k ~700 >> seconds, and the entire machine locked up. >> >> =C2=A0An interesting point is a single VM was able to run to completi= on. >> >> =C2=A0I'll keep testing with various combinations of numbers of VMs, >> different vCPUs per VM, and pinning guest vCPUs. 3 4*vCPU guests locked the machine up in ~170 secs, 4 guests ran to=20 completion :( > Not sure if its related to core count or not, but it takes several > instances on the Epyc to lock up and takes a lot longer. I enabled IOMM= U > on it as well, and it's dmesg shows below. But the lockups are still > there with or without it enabled. Would you be able to try on the Epyc with vCPUs pinned, and all guest=20 vCPUs on different physical CPUs ? i.e. for a configuration that will lock up, run the same configuration=20 but with vCPUs pinned. For example, a 4 vCPU guest that will have vCPUs=20 0-3 pinned to host physical CPUs 7-10 would look like: -c 4 -p 0:7 -p 1:8 -p 2:9 -p 3:10 later, Peter.