Date: Tue, 27 Feb 2018 12:55:20 -0500 From: Mike Tancsa <mike@sentex.net> To: Peter Grehan <grehan@freebsd.org> Cc: "freebsd-virtualization@freebsd.org" <freebsd-virtualization@freebsd.org> Subject: Re: Ryzen lockup on bhyve was (Re: new Ryzen lockup issue ?) Message-ID: <48768738-e699-fd06-9154-9b5be1bbabcc@sentex.net> In-Reply-To: <2f2b6f96-d6ef-8e02-966d-dcbbfb0c92fb@freebsd.org> References: <a687883a-b2a8-5b18-f63e-754a2ed445c0@sentex.net> <bbcc09cf-0072-8510-156f-5c20c301d43f@sentex.net> <92a60e14-f532-2647-d45d-b500fc59ba88@sentex.net> <CAMgUhpo1C_0L86Xkzmuz5%2Be3C3zk5RNkVS9aEBEwF-2XZ4d1sQ@mail.gmail.com> <425be16f-9fdc-9ed6-72b1-02e28bfd130f@sentex.net> <2f2b6f96-d6ef-8e02-966d-dcbbfb0c92fb@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2/25/2018 12:51 PM, Peter Grehan wrote: > Hi Mike, > > (moving this to -virtualization) > >> Actually I can confirm the same sort of hard lockup happens on my Epyc >> board with RELENG11. It also happens in current. I will file a PR and >> post on freebsd-current in case someone has any suggestions on how to >> try and figure out whats going on. > > I could reproduce this on a Ryzen 1700 with HT enabled - it took ~700 > seconds, and the entire machine locked up. > > An interesting point is a single VM was able to run to completion. > > I'll keep testing with various combinations of numbers of VMs, > different vCPUs per VM, and pinning guest vCPUs. Hi Peter, I upgraded both boxes (Ryzen and Epyc) to HEAD. For the record, the Epyc CPU is CPU1 Information AMD EPYC 7281 16-Core Family: 17h Model: 00h-0Fh Microcode Patch Level: 8001207 Bios Release Date: 01/08/2018 SuperMicro: H11SSL-i Not sure if its related to core count or not, but it takes several instances on the Epyc to lock up and takes a lot longer. I enabled IOMMU on it as well, and it's dmesg shows below. But the lockups are still there with or without it enabled. ivhd0: <AMD-Vi/IOMMU ivhd with EFR> on acpi0 ivhd0: Flag:b0<IotlbSup,Coherent> ivhd0: Features(type:0x11) MsiNumPPR = 0 PNBanks= 2 PNCounters= 0 ivhd0: Extended features[31:0]:22294ada<PPRSup,NXSup,GTSup,IASup> HATS = 0x2 GATS = 0x0 GLXSup = 0x1 SmiFSup = 0x1 SmiFRC = 0x2 GAMSup = 0x1 DualPortLogSup = 0x2 DualEventLogSup = 0x2 ivhd0: Extended features[62:32]:f77ef<USSup> Max PASID: 0x2f DevTblSegSup = 0x3 MarcSup = 0x1 ivhd0: supported paging level:7, will use only: 4 ivhd0: device range: 0x0 - 0xffff ivhd0: PCI cap 0x190b640f@0x40 feature:19<IOTLB,EFR,CapExt> ivhd1: <AMD-Vi/IOMMU ivhd with EFR> on acpi0 ivhd1: Flag:b0<IotlbSup,Coherent> ivhd1: Features(type:0x11) MsiNumPPR = 0 PNBanks= 2 PNCounters= 0 ivhd1: Extended features[31:0]:22294ada<PPRSup,NXSup,GTSup,IASup> HATS = 0x2 GATS = 0x0 GLXSup = 0x1 SmiFSup = 0x1 SmiFRC = 0x2 GAMSup = 0x1 DualPortLogSup = 0x2 DualEventLogSup = 0x2 ivhd1: Extended features[62:32]:f77ef<USSup> Max PASID: 0x2f DevTblSegSup = 0x3 MarcSup = 0x1 ivhd1: supported paging level:7, will use only: 4 ivhd1: device range: 0x0 - 0xffff ivhd1: PCI cap 0x190b640f@0x40 feature:19<IOTLB,EFR,CapExt> ivhd2: <AMD-Vi/IOMMU ivhd with EFR> on acpi0 ivhd2: Flag:b0<IotlbSup,Coherent> ivhd2: Features(type:0x11) MsiNumPPR = 0 PNBanks= 2 PNCounters= 0 ivhd2: Extended features[31:0]:22294ada<PPRSup,NXSup,GTSup,IASup> HATS = 0x2 GATS = 0x0 GLXSup = 0x1 SmiFSup = 0x1 SmiFRC = 0x2 GAMSup = 0x1 DualPortLogSup = 0x2 DualEventLogSup = 0x2 ivhd2: Extended features[62:32]:f77ef<USSup> Max PASID: 0x2f DevTblSegSup = 0x3 MarcSup = 0x1 ivhd2: supported paging level:7, will use only: 4 ivhd2: device range: 0x0 - 0x3ffe ivhd2: PCI cap 0x190b640f@0x40 feature:19<IOTLB,EFR,CapExt> ivhd3: <AMD-Vi/IOMMU ivhd with EFR> on acpi0 ivhd3: Flag:b0<IotlbSup,Coherent> ivhd3: Features(type:0x11) MsiNumPPR = 0 PNBanks= 2 PNCounters= 0 ivhd3: Extended features[31:0]:22294ada<PPRSup,NXSup,GTSup,IASup> HATS = 0x2 GATS = 0x0 GLXSup = 0x1 SmiFSup = 0x1 SmiFRC = 0x2 GAMSup = 0x1 DualPortLogSup = 0x2 DualEventLogSup = 0x2 ivhd3: Extended features[62:32]:f77ef<USSup> Max PASID: 0x2f DevTblSegSup = 0x3 MarcSup = 0x1 ivhd3: supported paging level:7, will use only: 4 ivhd3: device range: 0x0 - 0x3ffe ivhd3: PCI cap 0x190b640f@0x40 feature:19<IOTLB,EFR,CapExt> ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 x203 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?48768738-e699-fd06-9154-9b5be1bbabcc>