Date: Sun, 4 Dec 2022 22:18:06 +0100 From: Mario Marietto <marietto2008@gmail.com> To: FreeBSD virtualization <freebsd-virtualization@freebsd.org>, =?UTF-8?Q?Corvin_K=C3=B6hne?= <c.koehne@beckhoff.com> Subject: Failed to emulate instruction sequence [ 8b5810f7c340004078740a498b8548 ] at 0xfffff80516c4304a Message-ID: <CA%2B1FSigne%2Bgf8z_%2BtJs%2BQYtJ3U5vC=WYQ_XtjNWWBR8OvC8UEg@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
[-- Attachment #1 --]
Hello.
I would like to inform you about a bug that afflicts bhyve that doesn't let
me run more than one virtual machine at the same time. Let's take this
example to understand when it occurs. I launch the first VM using this
script :
#!/bin/sh
#ps ax | grep bhyve | awk '{ print $1; }' | xargs kill -9
setxkbmap it
vms="$(ls /dev/vmm/*)"
vncs="$(ps ax | awk '/vncviewer [0]/{print $6}')"
for vm in $vms; do
session="${vm##*/}"
echo "bhyve session = $session"
echo "vnc session = $vncs"
if ! printf '%s\n' "${vncs}" | grep "${session}"; then
printf 'VNC session not found,destroying
ghost vms\n'
bhyvectl --vm=$session --destroy
else
printf 'Found VNC session %s\n'
"${session},no ghost vms found,not destroying them"
fi
done
vmdisk=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (NM13N4CZ)/
&& d{print d}'`
echo $vmdisk
vmdisk1=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(2015020204055E)/ && d{print d}'`
echo $vmdisk1
vmdisk2=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(2022285F1175)/ && d{print d}'`
echo $vmdisk2
vmdisk3=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(BE0191500218)/ && d{print d}'`
echo $vmdisk3
vmdisk4=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(38434B4237354B45)/ && d{print d}'`
echo $vmdisk4
vmdisk5=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(WD-WCAV2X597309)/ && d{print d}'`
echo $vmdisk5
mount -t ufs /dev/$vmdisk1'p2' /mnt/$vmdisk1'p2'
bhyve -S -c sockets=2,cores=2,threads=2 -m 4G -w -H -A \
-s 0,hostbridge \
-s 1,ahci-hd,/mnt/$vmdisk1'p2'/bhyve/img/Windows/windows11.img,bootindex=1 \
-s 2,ahci-hd,/dev/$vmdisk3 \
-s 3,ahci-hd,/dev/$vmdisk4 \
-s 4,ahci-hd,/dev/$vmdisk5 \
-s 7,virtio-net,tap18 \
-s 21,hda,play=/dev/dsp,rec=/dev/dsp \
-s 29,fbuf,tcp=0.0.0.0:5918,w=1500,h=950 \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \
vm18 < /dev/null & sleep 2 && vncviewer 0:18
Windows 11 runs there and it works great. At this point,let's say that I
want to run the VM n. 2 without closing the first one and this time I
choose a Linux vm,that I launch using this script :
#!/bin/sh
setxkbmap it
vms="$(ls /dev/vmm/*)"
vncs="$(ps ax | awk '/vncviewer [0]/{print $6}')"
for vm in $vms; do
session="${vm##*/}"
echo "bhyve session = $session"
echo "vnc session = $vncs"
if ! printf '%s\n' "${vncs}" | grep "${session}"; then
printf 'VNC session not found,destroying
ghost vms\n'
bhyvectl --vm=$session --destroy
else
printf 'Found VNC session %s\n'
"${session},no ghost vms found,not destroying them"
fi
done
#bhyvectl --vm=vm7 --force-reset
#bhyvectl --vm=vm7 --destroy
vmdisk=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (NM13N4CZ)/
&& d{print d}'`
echo "Seagate M3 Portable 1.8 TB ; $vmdisk"
vmdisk0=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(2022285F1175)/ && d{print d}'`
echo "CT1000P1SSD8 ; $vmdisk0"
vmdisk1=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(2015020204055E)/ && d{print d}'`
echo "TOSHIBA External USB 3.0 1.8 TB ; $vmdisk1"
vmdisk2=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(20130506005976F)/ && d{print d}'`
echo "TOSHIBA External USB 3.0 932G ; $vmdisk2"
vmdisk3=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(BE0191500218)/ && d{print d}'`
echo "G-DRIVE USB ; $vmdisk3"
vmdisk4=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(38434B4237354B45)/ && d{print d}'`
echo "Elements 25A3 ; $vmdisk4"
vmdisk5=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(WD-WCAV2X597309)/ && d{print d}'`
echo "WDC WD3200AAJS-00L7A0-298 GB ; $vmdisk5"
vmdisk6=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(0774911DDC4200A6)/ && d{print d}'`
echo "SanDisk Cruzer-15GB ; $vmdisk6"
vmdisk7=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(60A44C4138D8F311190A0149)/ && d{print d}'`
echo "Kingston DataTraveler 2.0 ; $vmdisk7"
vmdisk8=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(20140100006C)/ && d{print d}'`
echo "Corsair Force 3 SSD ; $vmdisk8"
vmdisk9=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(1924E20B2AE5)/ && d{print d}'`
echo "CT500MX500SSD4 ; $vmdisk9"
vmdisk10=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident:
(S3Z2NB0KB99028V)/ && d{print d}'`
echo "Samsung SSD 860 EVO ; $vmdisk10"
mount -t ufs /dev/$vmdisk3'p2' /mnt/$vmdisk3'p2'
mount -t ufs /dev/$vmdisk1'p2' /mnt/$vmdisk1'p2'
bhyve -S -c sockets=1,cores=2,threads=2 -m 4G -w -H -A \
-s 0,hostbridge \
-s
2,virtio-blk,/mnt/$vmdisk1'p2'/bhyve/img/Linux/ubuntu2210.img,bootindex=1 \
-s 3,virtio-blk,/dev/$vmdisk4 \
-s 4,virtio-blk,/dev/$vmdisk2 \
-s 5,virtio-blk,/dev/$vmdisk10 \
-s 7,fbuf,tcp=0.0.0.0:5919,w=1600,h=950,wait \
-s 8:0,passthru,2/0/0 \
-s 8:1,passthru,2/0/1 \
-s 8:2,passthru,2/0/2 \
-s 8:3,passthru,2/0/3 \
-s 10,virtio-net,tap19 \
-s 11,virtio-9p,sharename=/ \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \
vm0:19 < /dev/null & sleep 2 && vncviewer 0:19
and boom. The first VM crash itself and close giving this error :
Failed to emulate instruction sequence [ 8b5810f7c340004078740a498b8548 ]
at 0xfffff80516c4304a
This is clearly a bug that should be fixed. Maybe @Corvin Köhne
<c.koehne@beckhoff.com> can help here.
Below you can see what are the hardware components of my system :
# lspci
00:00.0 Host bridge: Intel Corporation 8th/9th Gen Core 8-core Desktop
Processor Host Bridge/DRAM Re
gisters [Coffee Lake S] (rev 0d)
00:01.0 PCI bridge: Intel Corporation 6th-10th Gen Core Processor PCIe
Controller (x16) (rev 0d)
00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen
Core Processor PCIe Control
ler (x8) (rev 0d)
00:02.0 Display controller: Intel Corporation CoffeeLake-S GT2 [UHD
Graphics 630] (rev 02)
00:12.0 Signal processing controller: Intel Corporation Cannon Lake PCH
Thermal Controller (rev 10)
00:14.0 USB controller: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host
Controller (rev 10)
00:14.2 RAM memory: Intel Corporation Cannon Lake PCH Shared SRAM (rev 10)
00:16.0 Communication controller: Intel Corporation Cannon Lake PCH HECI
Controller (rev 10)
00:17.0 SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI
Controller (rev 10)
00:1b.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port
#17 (rev f0)
00:1c.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port
#1 (rev f0)
00:1d.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port
#9 (rev f0)
00:1f.0 ISA bridge: Intel Corporation Z390 Chipset LPC/eSPI Controller (rev
10)
00:1f.3 Audio device: Intel Corporation Cannon Lake PCH cAVS (rev 10)
00:1f.4 SMBus: Intel Corporation Cannon Lake PCH SMBus Controller (rev 10)
00:1f.5 Serial bus controller: Intel Corporation Cannon Lake PCH SPI
Controller (rev 10)
00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (7)
I219-V (rev 10)
01:00.0 VGA compatible controller: NVIDIA Corporation GP106 [GeForce GTX
1060 3GB] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GP106 High Definition Audio
Controller (rev a1)
02:00.0 VGA compatible controller: NVIDIA Corporation TU102 [GeForce RTX
2080 Ti] (rev a1)
02:00.1 Audio device: NVIDIA Corporation TU102 High Definition Audio
Controller (rev a1)
02:00.2 USB controller: NVIDIA Corporation TU102 USB 3.1 Host Controller
(rev a1)
02:00.3 Serial bus controller: NVIDIA Corporation TU102 USB Type-C UCSI
Controller (rev a1)
03:00.0 Non-Volatile memory controller: Micron/Crucial Technology Device
5403 (rev 03)
--
Mario.
[-- Attachment #2 --]
<div dir="ltr"><div>Hello.<br></div><div><br></div><div>I would like to inform you about a bug that afflicts bhyve that doesn't let me run more than one virtual machine at the same time. Let's take this example to understand when it occurs. I launch the first VM using this script :<br></div><div><br></div><div>#!/bin/sh<br><br>#ps ax | grep bhyve | awk '{ print $1; }' | xargs kill -9<br>setxkbmap it<br>vms="$(ls /dev/vmm/*)"<br>vncs="$(ps ax | awk '/vncviewer [0]/{print $6}')"<br>for vm in $vms; do<br> session="${vm##*/}" <br> echo "bhyve session = $session"<br> echo "vnc session = $vncs"<br> if ! printf '%s\n' "${vncs}" | grep "${session}"; then<br> printf 'VNC session not found,destroying ghost vms\n'<br> bhyvectl --vm=$session --destroy <br> else<br> printf 'Found VNC session %s\n' "${session},no ghost vms found,not destroying them"<br> fi<br>done<br><br>vmdisk=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (NM13N4CZ)/ && d{print d}'`<br>echo $vmdisk<br><br>vmdisk1=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (2015020204055E)/ && d{print d}'`<br>echo $vmdisk1<br><br>vmdisk2=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (2022285F1175)/ && d{print d}'`<br>echo $vmdisk2<br><br>vmdisk3=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (BE0191500218)/ && d{print d}'`<br>echo $vmdisk3<br><br>vmdisk4=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (38434B4237354B45)/ && d{print d}'`<br>echo $vmdisk4<br><br>vmdisk5=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (WD-WCAV2X597309)/ && d{print d}'`<br>echo $vmdisk5<br><br>mount -t ufs /dev/$vmdisk1'p2' /mnt/$vmdisk1'p2'<br><br>bhyve -S -c sockets=2,cores=2,threads=2 -m 4G -w -H -A \<br>-s 0,hostbridge \<br>-s 1,ahci-hd,/mnt/$vmdisk1'p2'/bhyve/img/Windows/windows11.img,bootindex=1 \<br>-s 2,ahci-hd,/dev/$vmdisk3 \<br>-s 3,ahci-hd,/dev/$vmdisk4 \<br>-s 4,ahci-hd,/dev/$vmdisk5 \<br>-s 7,virtio-net,tap18 \<br>-s 21,hda,play=/dev/dsp,rec=/dev/dsp \<br>-s 29,fbuf,tcp=<a href="http://0.0.0.0:5918">0.0.0.0:5918</a>,w=1500,h=950 \<br>-s 30,xhci,tablet \<br>-s 31,lpc \<br>-l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \<br>vm18 < /dev/null & sleep 2 && vncviewer 0:18 <br></div><div><br></div><div>Windows 11 runs there and it works great. At this point,let's say that I want to run the VM n. 2 without closing the first one and this time I choose a Linux vm,that I launch using this script :</div><div><br></div><div>#!/bin/sh<br><br>setxkbmap it<br>vms="$(ls /dev/vmm/*)"<br>vncs="$(ps ax | awk '/vncviewer [0]/{print $6}')"<br>for vm in $vms; do<br> session="${vm##*/}" <br> echo "bhyve session = $session"<br> echo "vnc session = $vncs"<br> if ! printf '%s\n' "${vncs}" | grep "${session}"; then<br> printf 'VNC session not found,destroying ghost vms\n'<br> bhyvectl --vm=$session --destroy <br> else<br> printf 'Found VNC session %s\n' "${session},no ghost vms found,not destroying them"<br> fi<br>done<br><br>#bhyvectl --vm=vm7 --force-reset<br>#bhyvectl --vm=vm7 --destroy<br><br>vmdisk=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (NM13N4CZ)/ && d{print d}'`<br>echo "Seagate M3 Portable 1.8 TB ; $vmdisk"<br><br>vmdisk0=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (2022285F1175)/ && d{print d}'`<br>echo "CT1000P1SSD8 ; $vmdisk0"<br><br>vmdisk1=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (2015020204055E)/ && d{print d}'`<br>echo "TOSHIBA External USB 3.0 1.8 TB ; $vmdisk1"<br><br>vmdisk2=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (20130506005976F)/ && d{print d}'`<br>echo "TOSHIBA External USB 3.0 932G ; $vmdisk2"<br><br>vmdisk3=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (BE0191500218)/ && d{print d}'`<br>echo "G-DRIVE USB ; $vmdisk3"<br><br>vmdisk4=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (38434B4237354B45)/ && d{print d}'`<br>echo "Elements 25A3 ; $vmdisk4"<br><br>vmdisk5=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (WD-WCAV2X597309)/ && d{print d}'`<br>echo "WDC WD3200AAJS-00L7A0-298 GB ; $vmdisk5"<br><br>vmdisk6=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (0774911DDC4200A6)/ && d{print d}'`<br>echo "SanDisk Cruzer-15GB ; $vmdisk6"<br><br>vmdisk7=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (60A44C4138D8F311190A0149)/ && d{print d}'`<br>echo "Kingston DataTraveler 2.0 ; $vmdisk7"<br><br>vmdisk8=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (20140100006C)/ && d{print d}'`<br>echo "Corsair Force 3 SSD ; $vmdisk8"<br><br>vmdisk9=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (1924E20B2AE5)/ && d{print d}'`<br>echo "CT500MX500SSD4 ; $vmdisk9"<br><br>vmdisk10=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (S3Z2NB0KB99028V)/ && d{print d}'`<br>echo "Samsung SSD 860 EVO ; $vmdisk10"<br><br>mount -t ufs /dev/$vmdisk3'p2' /mnt/$vmdisk3'p2'<br>mount -t ufs /dev/$vmdisk1'p2' /mnt/$vmdisk1'p2'<br><br>bhyve -S -c sockets=1,cores=2,threads=2 -m 4G -w -H -A \<br>-s 0,hostbridge \<br>-s 2,virtio-blk,/mnt/$vmdisk1'p2'/bhyve/img/Linux/ubuntu2210.img,bootindex=1 \<br>-s 3,virtio-blk,/dev/$vmdisk4 \<br>-s 4,virtio-blk,/dev/$vmdisk2 \<br>-s 5,virtio-blk,/dev/$vmdisk10 \<br>-s 7,fbuf,tcp=<a href="http://0.0.0.0:5919">0.0.0.0:5919</a>,w=1600,h=950,wait \<br>-s 8:0,passthru,2/0/0 \<br>-s 8:1,passthru,2/0/1 \<br>-s 8:2,passthru,2/0/2 \<br>-s 8:3,passthru,2/0/3 \<br>-s 10,virtio-net,tap19 \<br>-s 11,virtio-9p,sharename=/ \<br>-s 30,xhci,tablet \<br>-s 31,lpc \<br>-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \<br>vm0:19 < /dev/null & sleep 2 && vncviewer 0:19<br><br></div><div>and boom. The first VM crash itself and close giving this error :</div><div><br></div><div>Failed to emulate instruction sequence [ 8b5810f7c340004078740a498b8548 ] at 0xfffff80516c4304a</div><div><br></div><div>This is clearly a bug that should be fixed. Maybe <a class="gmail_plusreply" id="plusReplyChip-0" href="mailto:c.koehne@beckhoff.com" tabindex="-1">@Corvin Köhne</a> can help here.</div><div><br></div><div>Below you can see what are the hardware components of my system :</div><div><br></div><div><span style="font-family:arial black,sans-serif"><span style="color:rgb(0,0,0);background-color:rgb(255,255,255)"># lspci
</span><br></span></div><div><span style="font-family:arial black,sans-serif"><br></span></div><div><span style="font-family:monospace"><span style="font-family:arial black,sans-serif">00:00.0 Host bridge: Intel Corporation 8th/9th Gen Core 8-core Desktop Processor Host Bridge/DRAM Re<br>gisters [Coffee Lake S] (rev 0d)
<br>00:01.0 PCI bridge: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) (rev 0d)
<br>00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Control<br>ler (x8) (rev 0d)
<br>00:02.0 Display controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630] (rev 02)
<br>00:12.0 Signal processing controller: Intel Corporation Cannon Lake PCH Thermal Controller (rev 10)
<br>00:14.0 USB controller: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller (rev 10)
<br>00:14.2 RAM memory: Intel Corporation Cannon Lake PCH Shared SRAM (rev 10)
<br>00:16.0 Communication controller: Intel Corporation Cannon Lake PCH HECI Controller (rev 10)
<br>00:17.0 SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)
<br>00:1b.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #17 (rev f0)
<br>00:1c.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #1 (rev f0)
<br>00:1d.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #9 (rev f0)
<br>00:1f.0 ISA bridge: Intel Corporation Z390 Chipset LPC/eSPI Controller (rev 10)
<br>00:1f.3 Audio device: Intel Corporation Cannon Lake PCH cAVS (rev 10)
<br>00:1f.4 SMBus: Intel Corporation Cannon Lake PCH SMBus Controller (rev 10)
<br>00:1f.5 Serial bus controller: Intel Corporation Cannon Lake PCH SPI Controller (rev 10)
<br>00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (7) I219-V (rev 10)
<br>01:00.0 VGA compatible controller: NVIDIA Corporation GP106 [GeForce GTX 1060 3GB] (rev a1)
<br>01:00.1 Audio device: NVIDIA Corporation GP106 High Definition Audio Controller (rev a1)
<br>02:00.0 VGA compatible controller: NVIDIA Corporation TU102 [GeForce RTX 2080 Ti] (rev a1)
<br>02:00.1 Audio device: NVIDIA Corporation TU102 High Definition Audio Controller (rev a1)
<br>02:00.2 USB controller: NVIDIA Corporation TU102 USB 3.1 Host Controller (rev a1)
<br>02:00.3 Serial bus controller: NVIDIA Corporation TU102 USB Type-C UCSI Controller (rev a1)
<br>03:00.0 Non-Volatile memory controller: Micron/Crucial Technology Device 5403 (rev 03)<br></span></span></div><div><span style="font-family:monospace"><span style="font-family:arial black,sans-serif"><br></span></span></div>-- <br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature">Mario.<br></div></div>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2B1FSigne%2Bgf8z_%2BtJs%2BQYtJ3U5vC=WYQ_XtjNWWBR8OvC8UEg>