From owner-freebsd-virtualization@freebsd.org Thu Jan 9 08:44:25 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9A7FD22DB9B for ; Thu, 9 Jan 2020 08:44:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47tfmn3gqcz3Jfp for ; Thu, 9 Jan 2020 08:44:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 7E3B122DB9A; Thu, 9 Jan 2020 08:44:25 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7DF8622DB99 for ; Thu, 9 Jan 2020 08:44:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47tfmn2mslz3Jfn for ; Thu, 9 Jan 2020 08:44:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 59A2C26292 for ; Thu, 9 Jan 2020 08:44:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 0098iPmE026072 for ; Thu, 9 Jan 2020 08:44:25 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 0098iPfr026071 for virtualization@FreeBSD.org; Thu, 9 Jan 2020 08:44:25 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 241774] FreeBSD 11.3 & 12.0 has broken SCSI & Networking on KVM/QEMU Q35 with OVMF Date: Thu, 09 Jan 2020 08:44:25 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.3-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: drum@graphica.com.au X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Jan 2020 08:44:25 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D241774 --- Comment #18 from John Hartley --- (In reply to Tommy P from comment #16) Hi Tommy P, Hooray great to see you have got VirtIO sorted !! I have now got my mutant 11.3 with 11.2 network kernel going and ... It works !! uname -a FreeBSD newt.in.graphica.com.au 11.3-RELEASE FreeBSD 11.3-RELEASE #26: Thu = Jan=20 9 18:38:41 AEDT 2020=20=20=20=20 root@newt.in.graphica.com.au:/usr/obj/usr/src/sys/GENERIC2 amd64 Only tested e1000 at the moment: % ifconfig -a lo0: flags=3D8049 metric 0 mtu 16384 options=3D600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 inet 127.0.0.1 netmask 0xff000000 nd6 options=3D21 groups: lo em0: flags=3D8843 metric 0 mtu 1500 =20=20=20=20=20=20=20 options=3D209b ether 52:54:00:4e:50:91 hwaddr 52:54:00:4e:50:91 inet 192.168.73.131 netmask 0xffffff00 broadcast 192.168.73.255 nd6 options=3D29 media: Ethernet autoselect (1000baseT ) status: active My approach was a bit sledge hammer as I have ended up with the following modules all being regressed back to 11.2: % find . -name '*.bak*' -print ./sbin/ipfw.bak ./sys/modules/mlx5en.bak ./sys/modules/netmap/Makefile.bak ./sys/modules/cxgbe.bak ./sys/modules/cxgbe.bak/if_cxgbe/Makefile.bak ./sys/modules/ipfw_nat64.bak ./sys/modules/mlx5.bak ./sys/modules/urtwn.bak ./sys/amd64/vmm.bak ./sys/amd64/pci.bak ./sys/dev/virtio/network.bak ./sys/dev/vmware/vmxnet3.bak ./sys/dev/pci/pcireg.h.bak.11.2 ./sys/dev/cxgbe/firmware.bak.11.3 ./sys/dev/usb/wlan.bak ./sys/dev/pci.bak ./sys/dev/netmap.bak ./sys/dev/e1000.bak ./sys/dev/ixgbe.bak ./sys/dev/re.bak ./sys/dev/iwi.bak ./sys/dev/malo.bak ./sys/dev/mwl.bak ./sys/dev/ral.bak ./sys/dev/ixl.bak ./sys/dev/bwi.bak ./sys/dev/cxgbe.bak ./sys/dev/mlx5.bak ./sys/dev/oce.bak ./sys/dev/rtwn.bak ./sys/dev/urtwn.bak ./sys/conf/files.bak ./sys/net.bak ./sys/netinet.bak ./sys/net80211.bak ./sys/netinet6.bak ./sys/netpfil.bak The need to regress so much was due ripple up / down impacts of /dev/netmap= . As you can see this is pretty much then entire network subsystem... :-( I am only able to post this as now network is up I can ssh into box to get result out. First though I am going to test all the other network devices... Now done: % ifconfig -a lo0: flags=3D8049 metric 0 mtu 16384 options=3D600003 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 inet 127.0.0.1 netmask 0xff000000 nd6 options=3D21 groups: lo em0: flags=3D8843 metric 0 mtu 1500 =20=20=20=20=20=20=20 options=3D209b ether 52:54:00:4e:50:91 hwaddr 52:54:00:4e:50:91 inet 192.168.73.131 netmask 0xffffff00 broadcast 192.168.73.255 nd6 options=3D29 media: Ethernet autoselect (1000baseT ) status: active vmx0: flags=3D8802 metric 0 mtu 1500 =20=20=20=20=20=20=20 options=3D60039b ether 52:54:00:0a:cc:0d hwaddr 52:54:00:0a:cc:0d nd6 options=3D29 media: Ethernet autoselect status: active re0: flags=3D8802 metric 0 mtu 1500 =20=20=20=20=20=20=20 options=3D8009b ether 52:54:00:b2:20:53 hwaddr 52:54:00:b2:20:53 nd6 options=3D29 media: Ethernet autoselect (100baseTX ) status: active So I have: e1000 - intel - em0 vmxnet3 - VMWare vmxnet3 Virtual NIC driver (VirtIO alternate) - vmx0 rtl8139 - RealTec - re0 This test also include VirtIO, but as per your testing this does not work a= nd I need to include your fix. So in summary: Issue is not with QEMU / KVM , but with FreeBSD code from 11.2 -> 11.3 which 12.x has inherited There are two bugs: 1. confirmed - VirtIO - the one Tommy T has helped resolve 2. Speculatively - /dev/netmap bug or else where Bug (2) is that is breaking on the other Q35 (non VirtIO) interfaces (e1000, vmxnet3, rtl8139). Now that I have a working base I am going for first move forward the PCI co= de and validate that is ok and then look at the netmap code, which I believe is what is causing the issue with all the other network devices. Once I get to specific commit/s that introduced the bug I will provide upda= te. BTW - I am really surprise that such a large and impactful change on network sub-system was part of minor release cycle and nothing was mentioned in the release notes. Cheers and thanks again Tommy. John Hartley. --=20 You are receiving this mail because: You are the assignee for the bug.=