Date: Sat, 18 Jan 2020 08:59:28 +0000 From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 236922] Virtio fails as QEMU-KVM guest with Q35 chipset on Ubuntu 18.04.2 LTS Message-ID: <bug-236922-27103-ybKSgAmBVa@https.bugs.freebsd.org/bugzilla/> In-Reply-To: <bug-236922-27103@https.bugs.freebsd.org/bugzilla/> References: <bug-236922-27103@https.bugs.freebsd.org/bugzilla/>
next in thread | previous in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D236922 --- Comment #49 from John Hartley <drum@graphica.com.au> --- (In reply to John Hartley from comment #47) Hi VirtIO'ers, I have not got: QEMU Q35 with OVMF, VirtIO SCSI (boot device), netmap + em & vmx. I had to sacrifice vtnet (VirtIO networking) as it inclusion seems to break= the rest of VirtIO (if you have netmap). Here is are key differences: 1. Remove VirtIO networking from kernel diff sys/amd64/conf/GENERIC sys/amd64/conf/GENERIC.bak.12-1-REL=20 349c349 < # device vtnet # VirtIO Ethernet device --- > device vtnet # VirtIO Ethernet device 2. Change netmap if_net.c NETMAP patch so it essentially stops VirtIO code generation, by moving it up to stop #include <dev/virtio/network/virtio_net= .h> # diff sys/dev/netmap/if_ptnet.c sys/dev/netmap/if_ptnet.c.bak.12-1-NETMAP= =20 88,89d87 <=20 < #ifdef WITH_PTNETMAP 91a90 > #ifdef WITH_PTNETMAP here is snippet of result: <<snip>> ... #include <dev/netmap/netmap_mem2.h> #ifdef WITH_PTNETMAP #include <dev/virtio/network/virtio_net.h> #ifndef INET #error "INET not defined, cannot support offloadings" #endif ... <<end snip>> NOTE: you could likely get same result by just remove compile from sys/conf/files 3. Remove VirtIO network compile from sys/conf/files , but keep other VirtIO patches # diff sys/conf/files sys/conf/files.bak.12-1-REL 3481,3483d3480 < dev/virtio/pci/virtio_pci_if.m optional virtio_pci < dev/virtio/pci/virtio_pci_legacy.c optional virtio_pci < dev/virtio/pci/virtio_pci_modern.c optional virtio_pci 3485a3483 > dev/virtio/network/if_vtnet.c optional vtnet <<snip>> ... dev/virtio/virtio.c optional virtio dev/virtio/virtqueue.c optional virtio dev/virtio/virtio_bus_if.m optional virtio dev/virtio/virtio_if.m optional virtio dev/virtio/pci/virtio_pci.c optional virtio_pci dev/virtio/pci/virtio_pci_if.m optional virtio_pci dev/virtio/pci/virtio_pci_legacy.c optional virtio_pci dev/virtio/pci/virtio_pci_modern.c optional virtio_pci dev/virtio/mmio/virtio_mmio.c optional virtio_mmio fdt dev/virtio/mmio/virtio_mmio_if.m optional virtio_mmio fdt dev/virtio/block/virtio_blk.c optional virtio_blk dev/virtio/balloon/virtio_balloon.c optional virtio_balloon dev/virtio/scsi/virtio_scsi.c optional virtio_scsi dev/virtio/random/virtio_random.c optional virtio_random dev/virtio/console/virtio_console.c optional virtio_console ... <<end snip>> 4. Replace VirtIO 12.1-RELEASE with updated VirtIO fixes from but stop Vir= tIO networking diff -q sys/dev/virtio sys/dev/virtio.bak.12-1-REL Common subdirectories: sys/dev/virtio/balloon and sys/dev/virtio.bak.12-1-REL/balloon Common subdirectories: sys/dev/virtio/block and sys/dev/virtio.bak.12-1-REL/block Common subdirectories: sys/dev/virtio/console and sys/dev/virtio.bak.12-1-REL/console Common subdirectories: sys/dev/virtio/mmio and sys/dev/virtio.bak.12-1-REL/= mmio Only in sys/dev/virtio.bak.12-1-REL: network Only in sys/dev/virtio: network.stop Common subdirectories: sys/dev/virtio/pci and sys/dev/virtio.bak.12-1-REL/p= ci Common subdirectories: sys/dev/virtio/random and sys/dev/virtio.bak.12-1-REL/random Common subdirectories: sys/dev/virtio/scsi and sys/dev/virtio.bak.12-1-REL/= scsi Files sys/dev/virtio/virtio.c and sys/dev/virtio.bak.12-1-REL/virtio.c diff= er Files sys/dev/virtio/virtio.h and sys/dev/virtio.bak.12-1-REL/virtio.h diff= er Files sys/dev/virtio/virtio_bus_if.m and sys/dev/virtio.bak.12-1-REL/virtio_bus_if.m differ Files sys/dev/virtio/virtio_config.h and sys/dev/virtio.bak.12-1-REL/virtio_config.h differ Only in sys/dev/virtio: virtio_endian.h Files sys/dev/virtio/virtio_ids.h and sys/dev/virtio.bak.12-1-REL/virtio_id= s.h differ Files sys/dev/virtio/virtio_if.m and sys/dev/virtio.bak.12-1-REL/virtio_if.m differ Files sys/dev/virtio/virtio_ring.h and sys/dev/virtio.bak.12-1-REL/virtio_ring.h differ Files sys/dev/virtio/virtqueue.c and sys/dev/virtio.bak.12-1-REL/virtqueue.c differ Only in sys/dev/virtio: virtqueue.c.ori Files sys/dev/virtio/virtqueue.h and sys/dev/virtio.bak.12-1-REL/virtqueue.h differ 5. Replace 12.1-RELEASE VirtIO modules config files with VirtIO batch versi= on, but also remove network from Makefile # diff -q sys/modules/virtio sys/modules/virtio.bak.12-1-REL/ Files sys/modules/virtio/Makefile and sys/modules/virtio.bak.12-1-REL/Makef= ile differ Common subdirectories: sys/modules/virtio/balloon and sys/modules/virtio.bak.12-1-REL/balloon Common subdirectories: sys/modules/virtio/block and sys/modules/virtio.bak.12-1-REL/block Common subdirectories: sys/modules/virtio/console and sys/modules/virtio.bak.12-1-REL/console Only in sys/modules/virtio.bak.12-1-REL: network Only in sys/modules/virtio: network.stop Common subdirectories: sys/modules/virtio/pci and sys/modules/virtio.bak.12-1-REL/pci Common subdirectories: sys/modules/virtio/random and sys/modules/virtio.bak.12-1-REL/random Common subdirectories: sys/modules/virtio/scsi and sys/modules/virtio.bak.12-1-REL/scsi Common subdirectories: sys/modules/virtio/virtio and sys/modules/virtio.bak.12-1-REL/virtio # diff sys/modules/virtio/Makefile sys/modules/virtio.bak.12-1-REL/Makefile 2c2 < # $FreeBSD$ --- > # $FreeBSD: releng/12.1/sys/modules/virtio/Makefile 273515 2014-10-23 04:= 47:32Z bryanv $ 26c26 < SUBDIR=3D virtio pci block balloon scsi random console --- > SUBDIR=3D virtio pci network block balloon scsi random console 6. Use the other 12.1 netmap patches as is: # diff sys/dev/netmap/netmap_kern.h sys/dev/netmap/netmap_kern.h.bak.12-1-R= EL=20 79c79 < /* #define WITH_PTNETMAP */ /* ptnetmap guest support */ --- > #define WITH_PTNETMAP /* ptnetmap guest support */ # diff sys/net/netmap_virt.h sys/net/netmap_virt.h.bak.12-1-REL 47,48c47,48 < #define PTNETMAP_PCI_DEVICE_ID 0xcccc /* memory device */ < #define PTNETMAP_PCI_NETIF_ID 0xcccd /* ptnet network interfac= e */ --- > #define PTNETMAP_PCI_DEVICE_ID 0x000c /* memory device */ > #define PTNETMAP_PCI_NETIF_ID 0x000d /* ptnet network interfac= e */ see (2) above for: sys/dev/netmap/if_ptnet.c Are test results: # ifconfig -a vmx0: flags=3D8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 =20=20=20=20=20=20=20 options=3De403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSU= M,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6> ether 52:54:00:a5:4c:f3 media: Ethernet autoselect status: active nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> em0: flags=3D8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 =20=20=20=20=20=20=20 options=3D81209b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,WOL_MAGI= C,VLAN_HWFILTER> ether 52:54:00:cb:db:07 inet 192.168.73.102 netmask 0xffffff00 broadcast 192.168.73.255 media: Ethernet autoselect (1000baseT <full-duplex>) status: active nd6 options=3D29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> lo0: flags=3D8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=3D680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3 inet 127.0.0.1 netmask 0xff000000 groups: lo nd6 options=3D21<PERFORMNUD,AUTO_LINKLOCAL> # pciconf -lcve hostb0@pci0:0:0:0: class=3D0x060000 card=3D0x11001af4 chip=3D0x29c0808= 6 rev=3D0x00 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82G33/G31/P35/P31 Express DRAM Controller' class =3D bridge subclass =3D HOST-PCI vgapci0@pci0:0:1:0: class=3D0x030000 card=3D0x11001af4 chip=3D0x01001b3= 6 rev=3D0x04 hdr=3D0x00 vendor =3D 'Red Hat, Inc.' device =3D 'QXL paravirtual graphic card' class =3D display subclass =3D VGA pcib1@pci0:0:2:0: class=3D0x060400 card=3D0x00001b36 chip=3D0x000c1b3= 6 rev=3D0x00 hdr=3D0x01 vendor =3D 'Red Hat, Inc.' device =3D 'QEMU PCIe Root port' class =3D bridge subclass =3D PCI-PCI cap 10[54] =3D PCI-Express 2 root port max data 128(128) ARI disabled link x1(x1) speed 2.5(2.5) ASPM disabled(L0s) slot 0 power limit 0 mW HotPlug(present) surprise Attn But= ton PC(on) EI(disengaged) cap 11[48] =3D MSI-X supports 1 message Table in map 0x10[0x0], PBA in map 0x10[0x800] cap 0d[40] =3D PCI Bridge card=3D0x00001b36 ecap 0001[100] =3D AER 2 0 fatal 0 non-fatal 0 corrected ecap 000d[148] =3D ACS 1 pcib3@pci0:0:2:1: class=3D0x060400 card=3D0x00001b36 chip=3D0x000c1b3= 6 rev=3D0x00 hdr=3D0x01 vendor =3D 'Red Hat, Inc.' device =3D 'QEMU PCIe Root port' class =3D bridge subclass =3D PCI-PCI cap 10[54] =3D PCI-Express 2 root port max data 128(128) ARI disabled link x1(x1) speed 2.5(2.5) ASPM disabled(L0s) slot 0 power limit 0 mW HotPlug(present) surprise Attn But= ton PC(on) EI(disengaged) cap 11[48] =3D MSI-X supports 1 message Table in map 0x10[0x0], PBA in map 0x10[0x800] cap 0d[40] =3D PCI Bridge card=3D0x00001b36 ecap 0001[100] =3D AER 2 0 fatal 0 non-fatal 0 corrected ecap 000d[148] =3D ACS 1 pcib4@pci0:0:2:2: class=3D0x060400 card=3D0x00001b36 chip=3D0x000c1b3= 6 rev=3D0x00 hdr=3D0x01 vendor =3D 'Red Hat, Inc.' device =3D 'QEMU PCIe Root port' class =3D bridge subclass =3D PCI-PCI cap 10[54] =3D PCI-Express 2 root port max data 128(128) ARI disabled link x1(x1) speed 2.5(2.5) ASPM disabled(L0s) slot 0 power limit 0 mW HotPlug(empty) surprise Attn Button PC(off) EI(disengaged) cap 11[48] =3D MSI-X supports 1 message Table in map 0x10[0x0], PBA in map 0x10[0x800] cap 0d[40] =3D PCI Bridge card=3D0x00001b36 ecap 0001[100] =3D AER 2 0 fatal 0 non-fatal 0 corrected ecap 000d[148] =3D ACS 1 pcib5@pci0:0:2:3: class=3D0x060400 card=3D0x00001b36 chip=3D0x000c1b3= 6 rev=3D0x00 hdr=3D0x01 vendor =3D 'Red Hat, Inc.' device =3D 'QEMU PCIe Root port' class =3D bridge subclass =3D PCI-PCI cap 10[54] =3D PCI-Express 2 root port max data 128(128) ARI disabled link x1(x1) speed 2.5(2.5) ASPM disabled(L0s) slot 0 power limit 0 mW HotPlug(present) surprise Attn But= ton PC(on) EI(disengaged) cap 11[48] =3D MSI-X supports 1 message Table in map 0x10[0x0], PBA in map 0x10[0x800] cap 0d[40] =3D PCI Bridge card=3D0x00001b36 ecap 0001[100] =3D AER 2 0 fatal 0 non-fatal 0 corrected ecap 000d[148] =3D ACS 1 pcib6@pci0:0:2:4: class=3D0x060400 card=3D0x00001b36 chip=3D0x000c1b3= 6 rev=3D0x00 hdr=3D0x01 vendor =3D 'Red Hat, Inc.' device =3D 'QEMU PCIe Root port' class =3D bridge subclass =3D PCI-PCI cap 10[54] =3D PCI-Express 2 root port max data 128(128) ARI disabled link x1(x1) speed 2.5(2.5) ASPM disabled(L0s) slot 0 power limit 0 mW HotPlug(empty) surprise Attn Button PC(off) EI(disengaged) cap 11[48] =3D MSI-X supports 1 message Table in map 0x10[0x0], PBA in map 0x10[0x800] cap 0d[40] =3D PCI Bridge card=3D0x00001b36 ecap 0001[100] =3D AER 2 0 fatal 0 non-fatal 0 corrected ecap 000d[148] =3D ACS 1 pcib7@pci0:0:2:5: class=3D0x060400 card=3D0x00001b36 chip=3D0x000c1b3= 6 rev=3D0x00 hdr=3D0x01 vendor =3D 'Red Hat, Inc.' device =3D 'QEMU PCIe Root port' class =3D bridge subclass =3D PCI-PCI cap 10[54] =3D PCI-Express 2 root port max data 128(128) ARI disabled link x1(x1) speed 2.5(2.5) ASPM disabled(L0s) slot 0 power limit 0 mW HotPlug(empty) surprise Attn Button PC(off) EI(disengaged) cap 11[48] =3D MSI-X supports 1 message Table in map 0x10[0x0], PBA in map 0x10[0x800] cap 0d[40] =3D PCI Bridge card=3D0x00001b36 ecap 0001[100] =3D AER 2 0 fatal 0 non-fatal 0 corrected ecap 000d[148] =3D ACS 1 uhci0@pci0:0:29:0: class=3D0x0c0300 card=3D0x11001af4 chip=3D0x2934808= 6 rev=3D0x03 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82801I (ICH9 Family) USB UHCI Controller' class =3D serial bus subclass =3D USB uhci1@pci0:0:29:1: class=3D0x0c0300 card=3D0x11001af4 chip=3D0x2935808= 6 rev=3D0x03 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82801I (ICH9 Family) USB UHCI Controller' class =3D serial bus subclass =3D USB uhci2@pci0:0:29:2: class=3D0x0c0300 card=3D0x11001af4 chip=3D0x2936808= 6 rev=3D0x03 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82801I (ICH9 Family) USB UHCI Controller' class =3D serial bus subclass =3D USB ehci0@pci0:0:29:7: class=3D0x0c0320 card=3D0x11001af4 chip=3D0x293a808= 6 rev=3D0x03 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82801I (ICH9 Family) USB2 EHCI Controller' class =3D serial bus subclass =3D USB isab0@pci0:0:31:0: class=3D0x060100 card=3D0x11001af4 chip=3D0x2918808= 6 rev=3D0x02 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82801IB (ICH9) LPC Interface Controller' class =3D bridge subclass =3D PCI-ISA ahci0@pci0:0:31:2: class=3D0x010601 card=3D0x11001af4 chip=3D0x2922808= 6 rev=3D0x02 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82801IR/IO/IH (ICH9R/DO/DH) 6 port SATA Controller [AHCI mode]' class =3D mass storage subclass =3D SATA cap 05[80] =3D MSI supports 1 message, 64 bit enabled with 1 message cap 12[a8] =3D SATA Index-Data Pair none0@pci0:0:31:3: class=3D0x0c0500 card=3D0x11001af4 chip=3D0x2930808= 6 rev=3D0x02 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82801I (ICH9 Family) SMBus Controller' class =3D serial bus subclass =3D SMBus pcib2@pci0:1:0:0: class=3D0x060400 card=3D0x00000000 chip=3D0x000e1b3= 6 rev=3D0x00 hdr=3D0x01 vendor =3D 'Red Hat, Inc.' class =3D bridge subclass =3D PCI-PCI cap 05[8c] =3D MSI supports 1 message, 64 bit, vector masks=20 cap 01[84] =3D powerspec 3 supports D0 D3 current D0 cap 10[48] =3D PCI-Express 2 PCI bridge max data 128(128) ARI disabled link x1(x1) speed 2.5(2.5) ASPM disabled(L0s) cap 0c[40] =3D unknown ecap 0001[100] =3D AER 2 0 fatal 0 non-fatal 0 corrected vmx0@pci0:2:1:0: class=3D0x020000 card=3D0x07b015ad chip=3D0x07b015a= d rev=3D0x01 hdr=3D0x00 vendor =3D 'VMware' device =3D 'VMXNET3 Ethernet Controller' class =3D network subclass =3D ethernet cap 11[9c] =3D MSI-X supports 25 messages, enabled Table in map 0x18[0x0], PBA in map 0x18[0x1000] cap 05[84] =3D MSI supports 1 message, 64 bit=20 em0@pci0:2:2:0: class=3D0x020000 card=3D0x11001af4 chip=3D0x100e8086 rev=3D= 0x03 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82540EM Gigabit Ethernet Controller' class =3D network subclass =3D ethernet vtpcim0@pci0:3:0:0: class=3D0x010000 card=3D0x11001af4 chip=3D0x10481af= 4 rev=3D0x01 hdr=3D0x00 vendor =3D 'Red Hat, Inc.' device =3D 'Virtio SCSI' class =3D mass storage subclass =3D SCSI cap 11[dc] =3D MSI-X supports 4 messages, enabled Table in map 0x14[0x0], PBA in map 0x14[0x800] cap 09[c8] =3D vendor (length 20) cap 09[b4] =3D vendor (length 20) cap 09[a4] =3D vendor (length 16) cap 09[94] =3D vendor (length 16) cap 09[84] =3D vendor (length 16) cap 01[7c] =3D powerspec 3 supports D0 D3 current D0 cap 10[40] =3D PCI-Express 2 endpoint max data 128(128) link x1(x1) speed 2.5(2.5) ASPM disabled(L0s) vtpcim1@pci0:5:0:0: class=3D0x00ff00 card=3D0x11001af4 chip=3D0x10451af= 4 rev=3D0x01 hdr=3D0x00 vendor =3D 'Red Hat, Inc.' device =3D 'Virtio memory balloon' class =3D old cap 09[c8] =3D vendor (length 20) cap 09[b4] =3D vendor (length 20) cap 09[a4] =3D vendor (length 16) cap 09[94] =3D vendor (length 16) cap 09[84] =3D vendor (length 16) cap 01[7c] =3D powerspec 3 supports D0 D3 current D0 cap 10[40] =3D PCI-Express 2 endpoint max data 128(128) link x1(x1) speed 2.5(2.5) ASPM disabled(L0s) ls /dev/da* /dev/da0 /dev/da0p1 /dev/da0p2 /dev/da0p3 As I am using Q35 VirtIO SCSI rather then VirtIO Block. I am still a bit puzzled why enabling VirtIO networking with netmap results= in losing VirtIO Disk support but again it looks like there is some subtle interaction across VirtIO and netmap that needs to be looked at closer. My testing is complete for the time being, as I have got degree of flexibil= ity required: Disk - SATA, SCSI (with VirtIO) and VirtIO Networking - em, vmx and re This is huge improvement from 11.3 & 12.x current situation. Thanks to all for helping. Cheers. John Hartley --=20 You are receiving this mail because: You are the assignee for the bug.=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-236922-27103-ybKSgAmBVa>