From nobody Sun Dec 4 21:18:06 2022 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4NQKKx6w6jz4jZxq for ; Sun, 4 Dec 2022 21:18:45 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: from mail-ej1-x62c.google.com (mail-ej1-x62c.google.com [IPv6:2a00:1450:4864:20::62c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4NQKKx1GkYz410n for ; Sun, 4 Dec 2022 21:18:45 +0000 (UTC) (envelope-from marietto2008@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=DSZ8Dl8W; spf=pass (mx1.freebsd.org: domain of marietto2008@gmail.com designates 2a00:1450:4864:20::62c as permitted sender) smtp.mailfrom=marietto2008@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-ej1-x62c.google.com with SMTP id ml11so23525849ejb.6 for ; Sun, 04 Dec 2022 13:18:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=ulGGBRbMRbBkHwbpW1RUCMUqva/KwAvC83t02R38WyU=; b=DSZ8Dl8WssJhLdv5FCKDGsM+8CUY0725ZezyV5ZLDeEoYg6FR05UWXycUcSS0gYHyW 98M1ZKyrLeZjiRuUzdMv1nzFjLwM5PXqJTom4YvKHr5DJL/oy/oKsd+m3VZ19q/y5oKR Didf6TK8yVO0fhJm+8dbgl2w3FZ9CpdDHN8i1pytW/XpKZyqPlTT/M8SQqICHFN6K/zz cQJLWjDtxVOOO4q41w8sfmyw1mJn/0xrNhG4/AXr1xhdAH619xu5t3RiGnl0DYMzdhg4 8Sffu7t/eqge46+FtBesjuTyR61wF6Q7X0QxhWV2o3p0q9pp5qwifp5jv0BDpCjI3WKl /oGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ulGGBRbMRbBkHwbpW1RUCMUqva/KwAvC83t02R38WyU=; b=uryWLI3JED1LQK0vJM9OsuVYIFmaHFeoBbMMRgTTgQEphJxbL6AjE5WdgLmJyz48ay eQEmjnzibIveggkDMZcP/22PytpO4UrkXT8lJBhTGZZC/CpfdaXpVVuVIE7g2Msz5vw2 7CIYCerPvc4vhGy8hEbCeLk5RvOzoJsjUmQKhepU7x353iqFBNdxwL0Wt3+Gah8nXHr/ +dpspv2wS2idstyfeYrPV6TaDkowt2Bcll4HwjEi2wycSpYVanAy5F9I26CCgesqqzq9 mWHcCBnQ+U9pk6B3pJxJqQBNEoiHdy6Jl313InERgwrpKLUxVkz85CT4M1kxdU3zjSKW /n5Q== X-Gm-Message-State: ANoB5plDvwEys80mZjnqxtyfVdH380RmZyw3VC3/V8B7Oy04FrS1DxNg AWBpAJSOqqvUEQV+aCCk2Me8g7WcU5/GgO8J+FiINNEEmPdznQ== X-Google-Smtp-Source: AA0mqf6NN1QkgrF2Kx3jwjLyTpPPj8LHiERGtrBzRfpLFs6X71BPGlmZEIMkB6djukLS4vexoyRoNxAerMhh/KRnsuk= X-Received: by 2002:a17:906:3293:b0:7bd:f540:9be7 with SMTP id 19-20020a170906329300b007bdf5409be7mr32361627ejw.434.1670188723187; Sun, 04 Dec 2022 13:18:43 -0800 (PST) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 From: Mario Marietto Date: Sun, 4 Dec 2022 22:18:06 +0100 Message-ID: Subject: Failed to emulate instruction sequence [ 8b5810f7c340004078740a498b8548 ] at 0xfffff80516c4304a To: FreeBSD virtualization , =?UTF-8?Q?Corvin_K=C3=B6hne?= Content-Type: multipart/alternative; boundary="0000000000008db33905ef071d88" X-Spamd-Result: default: False [-2.83 / 15.00]; HTTP_TO_IP(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-0.999]; NEURAL_HAM_SHORT(-0.83)[-0.831]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62c:from]; ARC_NA(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; DKIM_TRACE(0.00)[gmail.com:+]; TO_DN_ALL(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; RCPT_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4NQKKx1GkYz410n X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N --0000000000008db33905ef071d88 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hello. I would like to inform you about a bug that afflicts bhyve that doesn't let me run more than one virtual machine at the same time. Let's take this example to understand when it occurs. I launch the first VM using this script : #!/bin/sh #ps ax | grep bhyve | awk '{ print $1; }' | xargs kill -9 setxkbmap it vms=3D"$(ls /dev/vmm/*)" vncs=3D"$(ps ax | awk '/vncviewer [0]/{print $6}')" for vm in $vms; do session=3D"${vm##*/}" echo "bhyve session =3D $session" echo "vnc session =3D $vncs" if ! printf '%s\n' "${vncs}" | grep "${session}"; then printf 'VNC session not found,destroying ghost vms\n' bhyvectl --vm=3D$session --destroy else printf 'Found VNC session %s\n' "${session},no ghost vms found,not destroying them" fi done vmdisk=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (NM13N4C= Z)/ && d{print d}'` echo $vmdisk vmdisk1=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (2015020204055E)/ && d{print d}'` echo $vmdisk1 vmdisk2=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (2022285F1175)/ && d{print d}'` echo $vmdisk2 vmdisk3=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (BE0191500218)/ && d{print d}'` echo $vmdisk3 vmdisk4=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (38434B4237354B45)/ && d{print d}'` echo $vmdisk4 vmdisk5=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (WD-WCAV2X597309)/ && d{print d}'` echo $vmdisk5 mount -t ufs /dev/$vmdisk1'p2' /mnt/$vmdisk1'p2' bhyve -S -c sockets=3D2,cores=3D2,threads=3D2 -m 4G -w -H -A \ -s 0,hostbridge \ -s 1,ahci-hd,/mnt/$vmdisk1'p2'/bhyve/img/Windows/windows11.img,bootindex=3D= 1 \ -s 2,ahci-hd,/dev/$vmdisk3 \ -s 3,ahci-hd,/dev/$vmdisk4 \ -s 4,ahci-hd,/dev/$vmdisk5 \ -s 7,virtio-net,tap18 \ -s 21,hda,play=3D/dev/dsp,rec=3D/dev/dsp \ -s 29,fbuf,tcp=3D0.0.0.0:5918,w=3D1500,h=3D950 \ -s 30,xhci,tablet \ -s 31,lpc \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \ vm18 < /dev/null & sleep 2 && vncviewer 0:18 Windows 11 runs there and it works great. At this point,let's say that I want to run the VM n. 2 without closing the first one and this time I choose a Linux vm,that I launch using this script : #!/bin/sh setxkbmap it vms=3D"$(ls /dev/vmm/*)" vncs=3D"$(ps ax | awk '/vncviewer [0]/{print $6}')" for vm in $vms; do session=3D"${vm##*/}" echo "bhyve session =3D $session" echo "vnc session =3D $vncs" if ! printf '%s\n' "${vncs}" | grep "${session}"; then printf 'VNC session not found,destroying ghost vms\n' bhyvectl --vm=3D$session --destroy else printf 'Found VNC session %s\n' "${session},no ghost vms found,not destroying them" fi done #bhyvectl --vm=3Dvm7 --force-reset #bhyvectl --vm=3Dvm7 --destroy vmdisk=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (NM13N4C= Z)/ && d{print d}'` echo "Seagate M3 Portable 1.8 TB ; $vmdisk" vmdisk0=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (2022285F1175)/ && d{print d}'` echo "CT1000P1SSD8 ; $vmdisk0" vmdisk1=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (2015020204055E)/ && d{print d}'` echo "TOSHIBA External USB 3.0 1.8 TB ; $vmdisk1" vmdisk2=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (20130506005976F)/ && d{print d}'` echo "TOSHIBA External USB 3.0 932G ; $vmdisk2" vmdisk3=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (BE0191500218)/ && d{print d}'` echo "G-DRIVE USB ; $vmdisk3" vmdisk4=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (38434B4237354B45)/ && d{print d}'` echo "Elements 25A3 ; $vmdisk4" vmdisk5=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (WD-WCAV2X597309)/ && d{print d}'` echo "WDC WD3200AAJS-00L7A0-298 GB ; $vmdisk5" vmdisk6=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (0774911DDC4200A6)/ && d{print d}'` echo "SanDisk Cruzer-15GB ; $vmdisk6" vmdisk7=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (60A44C4138D8F311190A0149)/ && d{print d}'` echo "Kingston DataTraveler 2.0 ; $vmdisk7" vmdisk8=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (20140100006C)/ && d{print d}'` echo "Corsair Force 3 SSD ; $vmdisk8" vmdisk9=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (1924E20B2AE5)/ && d{print d}'` echo "CT500MX500SSD4 ; $vmdisk9" vmdisk10=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (S3Z2NB0KB99028V)/ && d{print d}'` echo "Samsung SSD 860 EVO ; $vmdisk10" mount -t ufs /dev/$vmdisk3'p2' /mnt/$vmdisk3'p2' mount -t ufs /dev/$vmdisk1'p2' /mnt/$vmdisk1'p2' bhyve -S -c sockets=3D1,cores=3D2,threads=3D2 -m 4G -w -H -A \ -s 0,hostbridge \ -s 2,virtio-blk,/mnt/$vmdisk1'p2'/bhyve/img/Linux/ubuntu2210.img,bootindex=3D1= \ -s 3,virtio-blk,/dev/$vmdisk4 \ -s 4,virtio-blk,/dev/$vmdisk2 \ -s 5,virtio-blk,/dev/$vmdisk10 \ -s 7,fbuf,tcp=3D0.0.0.0:5919,w=3D1600,h=3D950,wait \ -s 8:0,passthru,2/0/0 \ -s 8:1,passthru,2/0/1 \ -s 8:2,passthru,2/0/2 \ -s 8:3,passthru,2/0/3 \ -s 10,virtio-net,tap19 \ -s 11,virtio-9p,sharename=3D/ \ -s 30,xhci,tablet \ -s 31,lpc \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \ vm0:19 < /dev/null & sleep 2 && vncviewer 0:19 and boom. The first VM crash itself and close giving this error : Failed to emulate instruction sequence [ 8b5810f7c340004078740a498b8548 ] at 0xfffff80516c4304a This is clearly a bug that should be fixed. Maybe @Corvin K=C3=B6hne can help here. Below you can see what are the hardware components of my system : # lspci 00:00.0 Host bridge: Intel Corporation 8th/9th Gen Core 8-core Desktop Processor Host Bridge/DRAM Re gisters [Coffee Lake S] (rev 0d) 00:01.0 PCI bridge: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) (rev 0d) 00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Control ler (x8) (rev 0d) 00:02.0 Display controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630] (rev 02) 00:12.0 Signal processing controller: Intel Corporation Cannon Lake PCH Thermal Controller (rev 10) 00:14.0 USB controller: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller (rev 10) 00:14.2 RAM memory: Intel Corporation Cannon Lake PCH Shared SRAM (rev 10) 00:16.0 Communication controller: Intel Corporation Cannon Lake PCH HECI Controller (rev 10) 00:17.0 SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10) 00:1b.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #17 (rev f0) 00:1c.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #1 (rev f0) 00:1d.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #9 (rev f0) 00:1f.0 ISA bridge: Intel Corporation Z390 Chipset LPC/eSPI Controller (rev 10) 00:1f.3 Audio device: Intel Corporation Cannon Lake PCH cAVS (rev 10) 00:1f.4 SMBus: Intel Corporation Cannon Lake PCH SMBus Controller (rev 10) 00:1f.5 Serial bus controller: Intel Corporation Cannon Lake PCH SPI Controller (rev 10) 00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (7) I219-V (rev 10) 01:00.0 VGA compatible controller: NVIDIA Corporation GP106 [GeForce GTX 1060 3GB] (rev a1) 01:00.1 Audio device: NVIDIA Corporation GP106 High Definition Audio Controller (rev a1) 02:00.0 VGA compatible controller: NVIDIA Corporation TU102 [GeForce RTX 2080 Ti] (rev a1) 02:00.1 Audio device: NVIDIA Corporation TU102 High Definition Audio Controller (rev a1) 02:00.2 USB controller: NVIDIA Corporation TU102 USB 3.1 Host Controller (rev a1) 02:00.3 Serial bus controller: NVIDIA Corporation TU102 USB Type-C UCSI Controller (rev a1) 03:00.0 Non-Volatile memory controller: Micron/Crucial Technology Device 5403 (rev 03) --=20 Mario. --0000000000008db33905ef071d88 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hello.

I would like to i= nform you about a bug that afflicts bhyve that doesn't let me run more = than one virtual machine at the same time. Let's take this example to u= nderstand when it occurs. I launch the first VM using this script :

#!/bin/sh

#ps ax | grep bhyve | awk '{ pri= nt $1; }' | xargs kill -9
setxkbmap it
vms=3D"$(ls /dev/vmm/= *)"
vncs=3D"$(ps ax | awk '/vncviewer [0]/{print $6}')= "
for vm in $vms; do
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 session=3D"${vm##*/}" =C2=A0
=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 echo "bhyve session =3D $sessio= n"
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 echo &qu= ot;vnc session =3D $vncs"
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 if ! printf '%s\n' "${vncs}" | grep "= ${session}"; then
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 printf '= VNC session not found,destroying ghost vms\n'
=C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 bhyvectl --vm=3D$session --destroy =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 else
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 printf 'Found VNC sess= ion %s\n' "${session},no ghost vms found,not destroying them"=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 fi
done
<= br>vmdisk=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (= NM13N4CZ)/ && d{print d}'`
echo $vmdisk

vmdisk1=3D`ge= om disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (2015020204055E)= / && d{print d}'`
echo $vmdisk1

vmdisk2=3D`geom disk = list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (2022285F1175)/ &&am= p; d{print d}'`
echo $vmdisk2

vmdisk3=3D`geom disk list | awk= '/^Geom name: /{d=3D$NF} /^ *ident: (BE0191500218)/ && d{print= d}'`
echo $vmdisk3

vmdisk4=3D`geom disk list | awk '/^Ge= om name: /{d=3D$NF} /^ *ident: (38434B4237354B45)/ && d{print d}= 9;`
echo $vmdisk4

vmdisk5=3D`geom disk list | awk '/^Geom nam= e: /{d=3D$NF} /^ *ident: (WD-WCAV2X597309)/ && d{print d}'`
= echo $vmdisk5

mount -t ufs /dev/$vmdisk1'p2' /mnt/$vmdisk1&#= 39;p2'

bhyve -S -c sockets=3D2,cores=3D2,threads=3D2 -m 4G -w -H= -A \
-s 0,hostbridge \
-s 1,ahci-hd,/mnt/$vmdisk1'p2'/bhyve/= img/Windows/windows11.img,bootindex=3D1 \
-s 2,ahci-hd,/dev/$vmdisk3 \-s 3,ahci-hd,/dev/$vmdisk4 \
-s 4,ahci-hd,/dev/$vmdisk5 \
-s 7,virt= io-net,tap18 \
-s 21,hda,play=3D/dev/dsp,rec=3D/dev/dsp \
-s 29,fbuf,= tcp=3D0.0.0.0:5918,w=3D1500,h=3D950 \-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-fi= rmware/BHYVE_BHF_CODE.fd \
vm18 < /dev/null & sleep 2 && = vncviewer 0:18

Windows 11 runs there and it w= orks great. At this point,let's say that I want to run the VM n. 2 with= out closing the first one and this time I choose a Linux vm,that I launch u= sing this script :

#!/bin/sh

setxkbmap itvms=3D"$(ls /dev/vmm/*)"
vncs=3D"$(ps ax | awk '/vn= cviewer [0]/{print $6}')"
for vm in $vms; do
=C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 session=3D"${vm##*/}" = =C2=A0
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 echo &qu= ot;bhyve session =3D $session"
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 echo "vnc session =3D $vncs"
=C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if ! printf '%s\n' &q= uot;${vncs}" | grep "${session}"; then
=C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 printf 'VNC session not found,destroying ghost vms= \n'
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 bhyvectl --vm=3D$session -= -destroy =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0
=C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 else
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 printf 'Found VNC session %s\n' "${session},no ghost vms f= ound,not destroying them"
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A0 =C2=A0 fi
done

#bhyvectl --vm=3Dvm7 --force-reset
#bhy= vectl --vm=3Dvm7 --destroy

vmdisk=3D`geom disk list | awk '/^Geo= m name: /{d=3D$NF} /^ *ident: (NM13N4CZ)/ && d{print d}'`
ec= ho "Seagate M3 Portable 1.8 TB ; $vmdisk"

vmdisk0=3D`geom = disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (2022285F1175)/ &am= p;& d{print d}'`
echo "CT1000P1SSD8 ; $vmdisk0"
vmdisk1=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (2= 015020204055E)/ && d{print d}'`
echo "TOSHIBA External = USB 3.0 1.8 TB ; $vmdisk1"

vmdisk2=3D`geom disk list | awk '= ;/^Geom name: /{d=3D$NF} /^ *ident: (20130506005976F)/ && d{print d= }'`
echo "TOSHIBA External USB 3.0 932G ; $vmdisk2"
vmdisk3=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (B= E0191500218)/ && d{print d}'`
echo "G-DRIVE USB ; $vmdi= sk3"

vmdisk4=3D`geom disk list | awk '/^Geom name: /{d=3D$N= F} /^ *ident: (38434B4237354B45)/ && d{print d}'`
echo "= ;Elements 25A3 ; $vmdisk4"

vmdisk5=3D`geom disk list | awk '= ;/^Geom name: /{d=3D$NF} /^ *ident: (WD-WCAV2X597309)/ && d{print d= }'`
echo "WDC WD3200AAJS-00L7A0-298 GB ; $vmdisk5"

= vmdisk6=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (07= 74911DDC4200A6)/ && d{print d}'`
echo "SanDisk Cruzer-1= 5GB ; $vmdisk6"

vmdisk7=3D`geom disk list | awk '/^Geom nam= e: /{d=3D$NF} /^ *ident: (60A44C4138D8F311190A0149)/ && d{print d}&= #39;`
echo "Kingston DataTraveler 2.0 ; $vmdisk7"

vmdis= k8=3D`geom disk list | awk '/^Geom name: /{d=3D$NF} /^ *ident: (2014010= 0006C)/ && d{print d}'`
echo "Corsair Force 3 SSD ; $vm= disk8"

vmdisk9=3D`geom disk list | awk '/^Geom name: /{d=3D= $NF} /^ *ident: (1924E20B2AE5)/ && d{print d}'`
echo "C= T500MX500SSD4 ; $vmdisk9"

vmdisk10=3D`geom disk list | awk '= ;/^Geom name: /{d=3D$NF} /^ *ident: (S3Z2NB0KB99028V)/ && d{print d= }'`
echo "Samsung SSD 860 EVO ; $vmdisk10"

mount -t= ufs /dev/$vmdisk3'p2' /mnt/$vmdisk3'p2'
mount -t ufs /d= ev/$vmdisk1'p2' /mnt/$vmdisk1'p2'

bhyve -S -c socket= s=3D1,cores=3D2,threads=3D2 -m 4G -w -H -A \
-s 0,hostbridge \
-s 2,v= irtio-blk,/mnt/$vmdisk1'p2'/bhyve/img/Linux/ubuntu2210.img,bootinde= x=3D1 \
-s 3,virtio-blk,/dev/$vmdisk4 \
-s 4,virtio-blk,/dev/$vmdisk2= \
-s 5,virtio-blk,/dev/$vmdisk10 \
-s 7,fbuf,tcp=3D0.0.0.0:5919,w=3D1600,h=3D950,wait \
-s 8:0,passthru,= 2/0/0 \
-s 8:1,passthru,2/0/1 \
-s 8:2,passthru,2/0/2 \
-s 8:3,pas= sthru,2/0/3 \
-s 10,virtio-net,tap19 \
-s 11,virtio-9p,sharename=3D/ = \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi= -firmware/BHYVE_UEFI_CODE.fd \
vm0:19 < /dev/null & sleep 2 &= & vncviewer 0:19

and boom. The first VM crash itself = and close giving this error :

Failed to emulate in= struction sequence [ 8b5810f7c340004078740a498b8548 ] at 0xfffff80516c4304a=

This is clearly a bug that should be fixed. Maybe= @Corvin K=C3=B6hne can help here.
=

Below you can see what are the hardware components of m= y system :

# lspci

00:00.0 Host bridge: Intel Corpora= tion 8th/9th Gen Core 8-core Desktop Processor Host Bridge/DRAM Re
giste= rs [Coffee Lake S] (rev 0d)
00:01.0 PCI bridge: Intel Corporation 6th-10th Gen Core Processor PCIe = Controller (x16) (rev 0d)
00:01.1 PCI bridge: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Ge= n Core Processor PCIe Control
ler (x8) (rev 0d)
00:02.0 Display controller: Intel Corporation CoffeeLake-S GT2 [UHD Gra= phics 630] (rev 02)
00:12.0 Signal processing controller: Intel Corporation Cannon Lake PCH= Thermal Controller (rev 10)
00:14.0 USB controller: Intel Corporation Cannon Lake PCH USB 3.1 xHCI = Host Controller (rev 10)
00:14.2 RAM memory: Intel Corporation Cannon Lake PCH Shared SRAM (rev = 10)
00:16.0 Communication controller: Intel Corporation Cannon Lake PCH HEC= I Controller (rev 10)
00:17.0 SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Co= ntroller (rev 10)
00:1b.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root = Port #17 (rev f0)
00:1c.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root = Port #1 (rev f0)
00:1d.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root = Port #9 (rev f0)
00:1f.0 ISA bridge: Intel Corporation Z390 Chipset LPC/eSPI Controller = (rev 10)
00:1f.3 Audio device: Intel Corporation Cannon Lake PCH cAVS (rev 10)
00:1f.4 SMBus: Intel Corporation Cannon Lake PCH SMBus Controller (rev = 10)
00:1f.5 Serial bus controller: Intel Corporation Cannon Lake PCH SPI Co= ntroller (rev 10)
00:1f.6 Ethernet controller: Intel Corporation Ethernet Connection (7) = I219-V (rev 10)
01:00.0 VGA compatible controller: NVIDIA Corporation GP106 [GeForce GT= X 1060 3GB] (rev a1)
01:00.1 Audio device: NVIDIA Corporation GP106 High Definition Audio Co= ntroller (rev a1)
02:00.0 VGA compatible controller: NVIDIA Corporation TU102 [GeForce RT= X 2080 Ti] (rev a1)
02:00.1 Audio device: NVIDIA Corporation TU102 High Definition Audio Co= ntroller (rev a1)
02:00.2 USB controller: NVIDIA Corporation TU102 USB 3.1 Host Controlle= r (rev a1)
02:00.3 Serial bus controller: NVIDIA Corporation TU102 USB Type-C UCSI= Controller (rev a1)
03:00.0 Non-Volatile memory controller: Micron/Crucial Technology Devic= e 5403 (rev 03)

--
Mario.
--0000000000008db33905ef071d88--