From owner-freebsd-virtualization@freebsd.org Sun Nov 12 10:45:12 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8B4C2E6EC5E for ; Sun, 12 Nov 2017 10:45:12 +0000 (UTC) (envelope-from me@cschwarz.com) Received: from orion.uberspace.de (orion.uberspace.de [95.143.172.79]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DFB1B63A6F for ; Sun, 12 Nov 2017 10:45:10 +0000 (UTC) (envelope-from me@cschwarz.com) Received: (qmail 32287 invoked from network); 12 Nov 2017 10:45:07 -0000 Received: from localhost (HELO csarch) (127.0.0.1) by orion.uberspace.de with SMTP; 12 Nov 2017 10:45:07 -0000 Message-ID: <1510483505.13439.16.camel@cschwarz.com> Subject: Re: grub-bhyve: support overriding just --root flag From: Christian Schwarz To: freebsd-virtualization@freebsd.org Date: Sun, 12 Nov 2017 11:45:05 +0100 In-Reply-To: <161c3f27-41fd-ada6-4edb-b8d2b3c1dc48@freebsd.org> References: <1510414698.13439.8.camel@cschwarz.com> <161c3f27-41fd-ada6-4edb-b8d2b3c1dc48@freebsd.org> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.26.2 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Nov 2017 10:45:12 -0000 Hi Alan, > Does libvirt support using the bhyve UEFI-CSM firmware instead? That > would let the VM boot using the native grub installed inside the VM, and > avoid this issue entirely. It also makes starting a bhyve a single > command instead of 2. Thanks for the tip, I just converted the disk to GPT and now use UEFI directly. The only problem I encountered with the UEFI firmware is that it will always prefer the virtualized cdrom over hdd or, more generally, one cannot define a boot order. This gist contains a working minimal UEFI-only libvirt domain: https://gist.github.com/problame/79a94ae05f5b17e11c3b5bc2fe5910c8 If you have any idea how to set the boot order via bhyve command line flags I would be "happy" to patch libvirt to support this feature. Otherwise, I hope this helps anyone reading this in the future, Christian From owner-freebsd-virtualization@freebsd.org Sun Nov 12 14:53:44 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C184CE73824 for ; Sun, 12 Nov 2017 14:53:44 +0000 (UTC) (envelope-from fabian.freyer@physik.tu-berlin.de) Received: from mail.tu-berlin.de (mail.tu-berlin.de [130.149.7.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 813596A981; Sun, 12 Nov 2017 14:53:44 +0000 (UTC) (envelope-from fabian.freyer@physik.tu-berlin.de) X-tubIT-Incoming-IP: 130.149.50.25 Received: from mail.physik-pool.tu-berlin.de ([130.149.50.25] helo=mail.physik.tu-berlin.de) by mail.tu-berlin.de (exim-4.89/mailfrontend-5) with esmtp id 1eDtdl-0008WD-7T; Sun, 12 Nov 2017 15:53:42 +0100 Received: from [192.168.0.18] (ip5f5bd7cc.dynamic.kabel-deutschland.de [95.91.215.204]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.physik.tu-berlin.de (Postfix) with ESMTPSA id 32A88C653; Sun, 12 Nov 2017 14:53:26 +0000 (UTC) From: "Fabian Freyer" To: "Allan Jude" , "Christian Schwarz" Cc: freebsd-virtualization@freebsd.org, novel@FreeBSD.org Subject: Re: grub-bhyve: support overriding just --root flag Date: Sun, 12 Nov 2017 15:53:11 +0100 Message-ID: In-Reply-To: <161c3f27-41fd-ada6-4edb-b8d2b3c1dc48@freebsd.org> References: <1510414698.13439.8.camel@cschwarz.com> <161c3f27-41fd-ada6-4edb-b8d2b3c1dc48@freebsd.org> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=_MailMate_8AFA511F-3A59-4771-86E3-F26C0DE4F8C0_="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Mailer: MailMate (1.9.7r5425) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Nov 2017 14:53:44 -0000 This is an OpenPGP/MIME signed message (RFC 3156 and 4880). --=_MailMate_8AFA511F-3A59-4771-86E3-F26C0DE4F8C0_= Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable On 12 Nov 2017, at 0:46, Allan Jude wrote: > Does libvirt support using the bhyve UEFI-CSM firmware instead? That > would let the VM boot using the native grub installed inside the VM, an= d > avoid this issue entirely. It also makes starting a bhyve a single > command instead of 2. Yes it does[1]. Also be aware that bootloader_args has some quoting issue= s. CC=E2=80=99ing novel@ as he does a lot of the libvirt+bhyve driver stuff.= Fabian. [1] https://libvirt.org/drvbhyve.html#uefi --=_MailMate_8AFA511F-3A59-4771-86E3-F26C0DE4F8C0_= Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc -----BEGIN PGP SIGNATURE----- iQJWBAEBCgBAFiEEX6JoxdmEemcFacQZmealkcs85+YFAloIYFciHGZhYmlhbi5m cmV5ZXJAcGh5c2lrLnR1LWJlcmxpbi5kZQAKCRCZ5qWRyzzn5jZ6D/0VGTjk10QS qsJujZrIFx3NE+WDY5GUhIrDABe1rAJtU3rlDkNdnfjweK7QNXig9djYL0Amrqx/ 7q1/TGrHLILqBKwsg/h3gYdw+9yiibB67A3Rz8Js5AK2vmfNa3U1EuZTliHZGmqo BMHNlQaf1Md8u3ZqbY1lBJTmn6BfUU80gL7aLCRjRxKtz2dpkJkPa/G0ithVYd+9 AgBpQxZ35aMHyiynCJIwT8WECoegwR47zAk1sONfeOY7flKeEcopHA/sM8AcF2DL YE7XI/vGlTRfAipOmak1pHBHRLCq58KIA42dVfIAGH7OLh8ShdM19TIkRcXXeZzh etAxASow+OmMwhkzqNPE2RbYvlNUhqz0a0rwEngTvZl02lLQtplBcy6Hm9kQHDZG 5tiDEuEuf5SdFxA0teYUO1PpfXFltpfuz62lNni+GI32KLaoDOUSPdqwdmDG+pHi HoguayuxLIVGUCiJLx+BYcfUZY99dsoMMnKmcgootVYDV+zUf2cCm/lQofwxIOHt kbuZ/V52Ih0YGcrvnsTEQhUBOpn9o1PGJrYrQ+i4or+P0Z65SPNLHh4AEPx+5xAR 0OnudI7AYZcgKvXoJPhwlk4bdxx8nehMCtyDD7lkF5pInvGObretB5gUCNxx5+GD mMKd5cU1wnxz+uwHlLmrUlyitbvzuEBXcw== =uhYj -----END PGP SIGNATURE----- --=_MailMate_8AFA511F-3A59-4771-86E3-F26C0DE4F8C0_=-- From owner-freebsd-virtualization@freebsd.org Sun Nov 12 15:10:26 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 16796E73B2C for ; Sun, 12 Nov 2017 15:10:26 +0000 (UTC) (envelope-from rajil.s@gmail.com) Received: from mail-qt0-x22a.google.com (mail-qt0-x22a.google.com [IPv6:2607:f8b0:400d:c0d::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C48F76ADF0 for ; Sun, 12 Nov 2017 15:10:25 +0000 (UTC) (envelope-from rajil.s@gmail.com) Received: by mail-qt0-x22a.google.com with SMTP id y5so607261qtk.4 for ; Sun, 12 Nov 2017 07:10:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=LumuRi/sN+oUMTjkC4nyPmrCwcAsToZxXIdwuBVcqXc=; b=nWhlnowFZV3RGIdX4IR8S0m0ogwuST94HgeVvm/JCE0UsH71C3qNbHzW5gy/E9UOfI Kp0mEOqndbJMjqlyykdr6poHBrHAqNMM75e8aVvK3TUQsbfZZMHCRJ6xGnHdUzu4lMjk UE+WDMKStT7QtHKYKrlZSzfPRwkixWIwkocW/vV65Bkd7Z/d1LhGkY8BZDSxsyHcjusi OGx8AWUkn3GJ0yHNYPGSRkFlmu61X4bJvuO5DkTDucx9sUvBJyqeiflx6bXM6zuAOw2q WnoZpZFgHfzyqD/Qw1D/nOqAYX9pi3G9/y2vUfwW5jFAf5PjmB8JPudPHL+caxjQlZYn FQdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=LumuRi/sN+oUMTjkC4nyPmrCwcAsToZxXIdwuBVcqXc=; b=fTfD4KyKOrV4xggMBBf2rY4gN0tdiI1M5DdXom9h5zoHL7E2VSxgrI+8zMTE8mdo6K y5OoMDTnefhpTb2JuKe+53gTdi+shHXvWGCeLLfuOKOQ7zbEXSAVs/u9dA1gfn/pOGq6 NrguRoHnrZ77Uw4nCRu5L91PAUdZFwBMAaQ8hIGC1F/G8CQEGXrJ/simbXlGcoXMawm7 LGfWEwycb84c6ANK1PyTk8UWv8pkHbemxHeM8JIt6Dc8hB4JAK6KSr+oFQbFb9r9jFJy dpOXlDk30TG3DyfgAF6QZf1ypPliSgz2ghUGacaYZWlz4mkRVvnxTftah9iszRGgDBIj qxQg== X-Gm-Message-State: AJaThX5jLEUBui2rTjBQzqeZNnYW/o1DylSytGZRdgu+2pCNH+o7Xhyf k6CVLJIGcNsm7OjB5gB32u1DOZsgcLA5Di3FFlT4DBLs X-Google-Smtp-Source: AGs4zMa0RNphcRst0ABysv9qwkLxo+ib3dTiWYkkK+FWBeXRJfUd4pVI3dmoEgCBdrAkjafhSc/55yG7SszANJQKSf8= X-Received: by 10.200.8.148 with SMTP id v20mr10426168qth.33.1510499424442; Sun, 12 Nov 2017 07:10:24 -0800 (PST) MIME-Version: 1.0 Received: by 10.200.1.9 with HTTP; Sun, 12 Nov 2017 07:10:23 -0800 (PST) In-Reply-To: <75ebab85-8259-10d7-687e-c77b9098387e@gmail.com> References: <201711051632.vA5GWF2u005895@pdx.rh.CN85.dnsmgr.net> <9207cb98-6c53-0173-4842-7f5c5f7324de@freebsd.org> <75ebab85-8259-10d7-687e-c77b9098387e@gmail.com> From: Rajil Saraswat Date: Sun, 12 Nov 2017 09:10:23 -0600 Message-ID: Subject: Re: Onboard USB controller passthrough To: freebsd-virtualization Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 Nov 2017 15:10:26 -0000 On 5 November 2017 at 13:26, Rajil Saraswat wrote: > The onboard xHCI controller (C610/X99 series chipset) does support MSI, > > # pciconf -l -BbcV xhci0@pci0:0:20:0 > xhci0@pci0:0:20:0: class=0x0c0330 card=0x72708086 chip=0x8d318086 > rev=0x05 hdr=0x00 > bar [10] = type Memory, range 64, base 0xc6100000, size 65536, enabled > cap 01[70] = powerspec 2 supports D0 D3 current D0 > cap 05[80] = MSI supports 8 messages, 64 bit enabled with 1 message > > I will give this a try. > > > Thanks, > > Rajil I passed the xHCI controller to a VM and it worked. The VM can see all the usb devices. However, I dont have any devices showing up as connected to the EHCI controller on the host, # usbconfig list ugen0.1: at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA) ugen1.1: at usbus1, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA) ugen0.2: at usbus0, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA) ugen1.2: at usbus1, cfg=0 md=HOST spd=HIGH (480Mbps) pwr=SAVE (0mA) I do have USB-2.0 devices plugged into these onboard controllers. Why are they being connected to the xHCI controller? From owner-freebsd-virtualization@freebsd.org Mon Nov 13 14:32:41 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 739A0DBBE34 for ; Mon, 13 Nov 2017 14:32:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 61626706D8 for ; Mon, 13 Nov 2017 14:32:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vADEWfRR016063 for ; Mon, 13 Nov 2017 14:32:41 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 222704] `sudo pkg update` segfaults in Vagrant Date: Mon, 13 Nov 2017 14:32:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: brd@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: DUPLICATE X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc resolution bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Nov 2017 14:32:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222704 Brad Davis changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |brd@FreeBSD.org Resolution|--- |DUPLICATE Status|New |Closed --- Comment #1 from Brad Davis --- This is a sudo bug, see the linked bug. *** This bug has been marked as a duplicate of bug 222510 *** --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Mon Nov 13 14:33:08 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A15A8DBBE84 for ; Mon, 13 Nov 2017 14:33:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8FA2770743 for ; Mon, 13 Nov 2017 14:33:08 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vADEX8kh016740 for ; Mon, 13 Nov 2017 14:33:08 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 222115] vagrant image sudo segfaults Date: Mon, 13 Nov 2017 14:33:08 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: brd@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: DUPLICATE X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution bug_status cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Nov 2017 14:33:08 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D222115 Brad Davis changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |DUPLICATE Status|New |Closed CC| |brd@FreeBSD.org --- Comment #1 from Brad Davis --- This is a bug in sudo, see the linked bug. *** This bug has been marked as a duplicate of bug 222510 *** --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Mon Nov 13 19:11:01 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 450B6DC1BF0 for ; Mon, 13 Nov 2017 19:11:01 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lf0-f45.google.com (mail-lf0-f45.google.com [209.85.215.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DEDFB792B3 for ; Mon, 13 Nov 2017 19:11:00 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lf0-f45.google.com with SMTP id 73so5244251lfu.10 for ; Mon, 13 Nov 2017 11:11:00 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:to:from:subject:message-id:date:user-agent :mime-version:content-language:content-transfer-encoding; bh=XcXy2XO1ek6srEf2ZX8xr0WRo4BPVDQ7KIAcKMTtpXI=; b=Lp6b1p4vUvzWjOwqBvsQ3F65I++OvWA5cbxa37EbwpB6i4XuCmVvV7Eg6Kn7NnkT+j Hcis8oFT+dL7ldxpp8HUgCzrMt8g8mCryRneIng4LLsPgNVpAq8e4aOOeplzHwId0CMD IprDHB3NUcMylaHBjOfx7412x1CFgrBQhKZkd8ar9AHwVYO3XVOmhYR/ZSvbSwYMtf/u uAogWsNBrWfjAso8I5Zqk/Ecau0/4MCn+9ulfIOAQhRiT0G/R8negesnQmaUU+LbWSpQ 0xQePUBUNoV58gaprB9UoIVwGc7+EwS5psoc7o5RQ9F+KGxzyYJsIBdbfImhteKhz56H dzWA== X-Gm-Message-State: AJaThX7tpSLaDQxMcFJXI4pGO2r1ypBCZj3k+ioLuzBBq0Zx5Fp4RoTA fI+LgmRzFusPzDEzTI6rw7DXnDFw X-Google-Smtp-Source: AGs4zMaZRiiHzuYBY4S3sF7kVYSLYMR2IHVGFiHUOU0bBRUrKgq+32w069TiKX4fwSP36iEy6zVTNA== X-Received: by 10.46.23.20 with SMTP id l20mr2973846lje.25.1510599771360; Mon, 13 Nov 2017 11:02:51 -0800 (PST) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id z81sm2973121lff.16.2017.11.13.11.02.49 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 Nov 2017 11:02:50 -0800 (PST) To: freebsd-virtualization@freebsd.org From: Andriy Gapon Subject: problem with pass-through on amd Message-ID: Date: Mon, 13 Nov 2017 21:02:48 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Nov 2017 19:11:01 -0000 I have a FreeBSD guest on a FreeBSD host, both amd64 head. I decided to experiment with PCI pass-through and used an AHCI controller for the experiment. It seems like that partially works. The guest detects the controller and attaches a driver to it, the messages (reported capabilities, etc) look identical to those on the host. But the guest can not detect any disks behind the controller. From what I can tell, the guest has no problems accessing PCI configuration registers and a memory mapped BAR. Not sure about the I/O ports. What's suspicious is that there are no interrupts either for the ppt device on the host or the ahci device in the guest. Maybe that is what causes the failure to see the disks. When the guest driver attaches to the controller I see these messages on the host: ppt0: attempting to allocate 1 MSI vectors (1 supported) ppt0: using IRQ 265 for MSI and in the guest: ahci0: attempting to allocate 1 MSI vectors (1 supported) ahci0: using IRQ 279 for MSI But vmstat -i does not report any interrupts. Just in case, the controller has some complex topology where the AHCI PCI device is behind two PCI-PCI bridges from the main bus. But I think that that should not matter and the guest needs to see only the AHCI device and the host handles the bridges. Here are some more messages from the guest: ahci0: port 0x21c0-0x21c7,0x21c8-0x21cb,0x21d0-0x21d7,0x21d8-0x21db,0x21e0-0x21ef mem 0xc000e000-0xc000efff irq 17 at device 9.0 on pci0 ahci0: attempting to allocate 1 MSI vectors (1 supported) ahci0: using IRQ 279 for MSI ahci0: AHCI v1.00 with 2 6Gbps ports, Port Multiplier supported with FBS ahci0: Caps: 64bit NCQ SNTF AL 6Gbps PM FBS PMD SSC PSC 32cmd eSATA 2ports ahcich0: at channel 0 on ahci0 ahcich0: Caps: CPD ESP FBSCP ahcich1: at channel 1 on ahci0 ahcich1: Caps: CPD ESP FBSCP ahcich0: AHCI reset... ahcich0: SATA connect time=100us status=00000123 ahcich0: AHCI reset: device found ahcich1: AHCI reset... ahcich1: SATA connect time=100us status=00000123 ahcich1: AHCI reset: device found ahcich1: AHCI reset: device ready after 100ms ahcich1: Poll timeout on slot 1 port 15 ahcich1: is 00000000 cs 00000002 ss 00000000 rs 00000002 tfd 150 serr 00000000 cmd 00318017 ahcich0: AHCI reset: device ready after 200ms ahcich0: Poll timeout on slot 1 port 15 ahcich0: is 00000000 cs 00000002 ss 00000000 rs 00000002 tfd 150 serr 00000000 cmd 00318017 And some more... Any ideas or suggestions? Or maybe some additional information from me? Thank you! -- Andriy Gapon From owner-freebsd-virtualization@freebsd.org Mon Nov 13 20:41:12 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 17B4FDD518F for ; Mon, 13 Nov 2017 20:41:12 +0000 (UTC) (envelope-from akgupt3@gmail.com) Received: from mail-qk0-x22e.google.com (mail-qk0-x22e.google.com [IPv6:2607:f8b0:400d:c09::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CA5A47D34F; Mon, 13 Nov 2017 20:41:11 +0000 (UTC) (envelope-from akgupt3@gmail.com) Received: by mail-qk0-x22e.google.com with SMTP id d125so6940723qkc.12; Mon, 13 Nov 2017 12:41:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=BUjjeOcYNmy85HowepM8FS3vhcr+14acNr4Eykd353U=; b=GiuCB0aepSCc7g8jgEpVHExNQUhfhU/QvDDPFd0YmT+rZ/4uQxzLMoR8nF0gJ3lm4n D9oY4gOCiUGXzzZxywPGdxTZTkuxqB1RrT9vDPXddrpWNpvK9ZgwvXodEP/DoCLnFRjZ QJT0lsv5jRhffGGO5Dq9uQOO4mUebo7fsAXyU8Wnk8NJDUUxYY/bTTvKRfEtOBy6Ap0Y 7UbZS2uGliRNNp+PYU7L8CO0AlhxMWIUl3+SaNmoPKJzFeLrgIHDXdRaX9hMlxVeVRbE lBibn/hdTxFexfCmMet8hmntzmZr5uL/B0nvDWGGIgpz4zZfgG2299aK56mWepIDcanA Vo5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=BUjjeOcYNmy85HowepM8FS3vhcr+14acNr4Eykd353U=; b=DDlZWTmHbWFOVTxbQJYIjgQ1wieyNtVE+ADMmJnTmmZLt8vlDe64JeQRrIinC4hD54 F0Rsdc4gteoW1EmsTEf8hoEyFRr2JwIY6VJzy0Ct+AJudv10252i32boc4wvZOGQjy7Y BBR5CSeLWOEOrXPXc2DLs/0RILHR6wgXLpcQyxSQEh+xiG/O5AWZRWv/Uqr/vVftstO4 mzrXsomTKO8fRCwaYGnUGdSz5139p0R2gRMXIQiV/cs81PUm1jsPsDZAglYgsE/6SF1L KVFIpF7tuSdYtBkf2ti8W4uH2AXhrdxfTcNoypqmeWvDANw2UHpKBL3AmjL3Nwp5sy6M 3w7Q== X-Gm-Message-State: AJaThX5yO+x70QF8xdKb/RN4llk+JADumPM8tIpEdz4VdGuAKnfktVEe IqLZ0JA5v0SBvivJ1V4JM5laCtGtoQOld8lBYr68QQ== X-Google-Smtp-Source: AGs4zMZJEjGHBLIO4gl9jw0cLxyA9i88LQqHWlywnQU2Si3lwznJAI9knlnWbUAZgPCScXQv8ZzM1UZjcs/CM54g5Mk= X-Received: by 10.55.183.134 with SMTP id h128mr15281501qkf.96.1510605670473; Mon, 13 Nov 2017 12:41:10 -0800 (PST) MIME-Version: 1.0 Received: by 10.12.172.247 with HTTP; Mon, 13 Nov 2017 12:41:10 -0800 (PST) In-Reply-To: References: From: Anish Date: Mon, 13 Nov 2017 12:41:10 -0800 Message-ID: Subject: Re: problem with pass-through on amd To: Andriy Gapon Cc: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Nov 2017 20:41:12 -0000 Hi Andriy, >What's suspicious is that there are no interrupts either for the ppt device on the host or the ahci device in the guest. I think most likely AMD IOMMU/amdvi is not enabled, by default it is disabled and can be confirmed by sysctl. [root@ryzen /usr/home/anish]# sysctl hw.vmm.amdvi hw.vmm.amdvi.domain_id: 3 hw.vmm.amdvi.disable_io_fault: 0 hw.vmm.amdvi.ptp_level: 4 hw.vmm.amdvi.host_ptp: 1 hw.vmm.amdvi.enable: 1 hw.vmm.amdvi.count: 2 If hw.vmm.amdvi.enable is 0, set it and reload vmm and things should work. Regards, Anish On Mon, Nov 13, 2017 at 11:02 AM, Andriy Gapon wrote: > > I have a FreeBSD guest on a FreeBSD host, both amd64 head. > I decided to experiment with PCI pass-through and used an AHCI controller > for > the experiment. It seems like that partially works. The guest detects the > controller and attaches a driver to it, the messages (reported > capabilities, > etc) look identical to those on the host. But the guest can not detect any > disks behind the controller. From what I can tell, the guest has no > problems > accessing PCI configuration registers and a memory mapped BAR. Not sure > about > the I/O ports. What's suspicious is that there are no interrupts either > for the > ppt device on the host or the ahci device in the guest. Maybe that is what > causes the failure to see the disks. > > When the guest driver attaches to the controller I see these messages on > the host: > ppt0: attempting to allocate 1 MSI vectors (1 supported) > ppt0: using IRQ 265 for MSI > and in the guest: > ahci0: attempting to allocate 1 MSI vectors (1 supported) > ahci0: using IRQ 279 for MSI > > But vmstat -i does not report any interrupts. > > Just in case, the controller has some complex topology where the AHCI PCI > device > is behind two PCI-PCI bridges from the main bus. But I think that that > should > not matter and the guest needs to see only the AHCI device and the host > handles > the bridges. > > Here are some more messages from the guest: > ahci0: port > 0x21c0-0x21c7,0x21c8-0x21cb,0x21d0-0x21d7,0x21d8-0x21db,0x21e0-0x21ef mem > 0xc000e000-0xc000efff irq 17 at device 9.0 on pci0 > ahci0: attempting to allocate 1 MSI vectors (1 supported) > ahci0: using IRQ 279 for MSI > ahci0: AHCI v1.00 with 2 6Gbps ports, Port Multiplier supported with FBS > ahci0: Caps: 64bit NCQ SNTF AL 6Gbps PM FBS PMD SSC PSC 32cmd eSATA 2ports > ahcich0: at channel 0 on ahci0 > ahcich0: Caps: CPD ESP FBSCP > ahcich1: at channel 1 on ahci0 > ahcich1: Caps: CPD ESP FBSCP > ahcich0: AHCI reset... > ahcich0: SATA connect time=100us status=00000123 > ahcich0: AHCI reset: device found > ahcich1: AHCI reset... > ahcich1: SATA connect time=100us status=00000123 > ahcich1: AHCI reset: device found > ahcich1: AHCI reset: device ready after 100ms > ahcich1: Poll timeout on slot 1 port 15 > ahcich1: is 00000000 cs 00000002 ss 00000000 rs 00000002 tfd 150 serr > 00000000 > cmd 00318017 > ahcich0: AHCI reset: device ready after 200ms > ahcich0: Poll timeout on slot 1 port 15 > ahcich0: is 00000000 cs 00000002 ss 00000000 rs 00000002 tfd 150 serr > 00000000 > cmd 00318017 > And some more... > > Any ideas or suggestions? > Or maybe some additional information from me? > > Thank you! > > -- > Andriy Gapon > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization- > unsubscribe@freebsd.org" > From owner-freebsd-virtualization@freebsd.org Mon Nov 13 21:02:13 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EDA06DD5B34 for ; Mon, 13 Nov 2017 21:02:13 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lf0-f50.google.com (mail-lf0-f50.google.com [209.85.215.50]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7DAE07E1EF for ; Mon, 13 Nov 2017 21:02:13 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lf0-f50.google.com with SMTP id a132so19912144lfa.7 for ; Mon, 13 Nov 2017 13:02:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=z15i16mZU0b8r+cRo0OVMEv5Nfj8fi718LG/3fGV3Qs=; b=eiiVRZ3CB9W1X+zMZBjiF5BninchLK+0bQ2Bg2X8u9rxIk8gw3UPajN9lUvFidy9T3 tdGv0U1LaIAsmQ87XadfzoUPhY3nozvQa7FptJPDIznuuIj6+WByEmgmhKERA0akVx/C 5KRZjEooAhcnYJycoKYqvxOXR2JNYMcqdeE1k2NfSMaTIEaMAWUiMrUWdYfEH4DvTPPw e3eCYnwWznS4H09n7HWHo48xEACEjjCWtIXRW2ETcbqbE1fvm3viIMZT9zyM16JDEjzc i6HDJ0SS+AnfEb7tcW6QEtc90DODFtL+u3b4Xon3e3jNp4bBD5eX68E2VHhOesBaaVpS LdgA== X-Gm-Message-State: AJaThX4EWuCF6vjuw+ca/TmDEYtV0s6OYq+GA7DuMzrcOkq3RDyhSq/o GVcmx4a8i28kXl2asHx386C/Q44D X-Google-Smtp-Source: AGs4zMYwC+ScCo2JvuJTfIG529P0awSc9IBeU8Vt/UitwkKtUt6y8gzTuVJlAiJxAkQvyKc727FzbA== X-Received: by 10.46.85.208 with SMTP id g77mr1447322lje.114.1510606548486; Mon, 13 Nov 2017 12:55:48 -0800 (PST) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id o21sm1135408lff.24.2017.11.13.12.55.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 13 Nov 2017 12:55:47 -0800 (PST) Subject: Re: problem with pass-through on amd To: Anish Cc: "freebsd-virtualization@freebsd.org" References: From: Andriy Gapon Message-ID: Date: Mon, 13 Nov 2017 22:55:46 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Nov 2017 21:02:14 -0000 On 13/11/2017 22:41, Anish wrote: > Hi Andriy, > >>What's suspicious is that there are no interrupts either for the > ppt device on the host or the ahci device in the guest. > > I think most likely AMD IOMMU/amdvi is not enabled, by default it is disabled > and can be confirmed by sysctl. > > [root@ryzen /usr/home/anish]# sysctl hw.vmm.amdvi > hw.vmm.amdvi.domain_id: 3 > hw.vmm.amdvi.disable_io_fault: 0 > hw.vmm.amdvi.ptp_level: 4 > hw.vmm.amdvi.host_ptp: 1 > hw.vmm.amdvi.enable: 1 > hw.vmm.amdvi.count: 2  > > If hw.vmm.amdvi.enable is 0, set it and reload vmm and things should work. I followed your how-to, so I didn't make this mistake :-) $ sysctl hw.vmm.amdvi hw.vmm.amdvi.domain_id: 3 hw.vmm.amdvi.disable_io_fault: 0 hw.vmm.amdvi.ptp_level: 4 hw.vmm.amdvi.host_ptp: 1 hw.vmm.amdvi.enable: 1 hw.vmm.amdvi.count: 1 Could it be that AMD-Vi support is broken in the hardware or BIOS? Are there any checks that I could do? Here is what I see in dmesg, but it is not a full dump of IVRS, of course: $ dmesg | egrep -i 'ivrs|ivh|amd-vi|mmu' Table 'IVRS' at 0xbdafaf80 ACPI: IVRS 0x00000000BDAFAF80 0000F8 (v01 AMD RD890S 00202031 AMD 00000000) AMD-Vi IVRS VAsize = 64 PAsize = 52 GVAsize = 0 flags:0 ivhd0: on acpi0 ivhd0: Unknown dev entry:0xff ivhd0: Flag:3e ivhd0: max supported paging level:7 restricting to: 4 ivhd0: device supported range [0x0 - 0xb01] ivhd0: device [0xa0 - 0xa0]config:d7 ivhd0: PCI cap 0x10b540f@0x40 feature:1 pci0: at device 0.2 (no driver attached) > On Mon, Nov 13, 2017 at 11:02 AM, Andriy Gapon > wrote: > > > I have a FreeBSD guest on a FreeBSD host, both amd64 head. > I decided to experiment with PCI pass-through and used an AHCI controller for > the experiment.  It seems like that partially works.  The guest detects the > controller and attaches a driver to it, the messages (reported capabilities, > etc) look identical to those on the host.  But the guest can not detect any > disks behind the controller.  From what I can tell, the guest has no problems > accessing PCI configuration registers and a memory mapped BAR.  Not sure about > the I/O ports.  What's suspicious is that there are no interrupts either for the > ppt device on the host or the ahci device in the guest.  Maybe that is what > causes the failure to see the disks. > > When the guest driver attaches to the controller I see these messages on the > host: > ppt0: attempting to allocate 1 MSI vectors (1 supported) > ppt0: using IRQ 265 for MSI > and in the guest: > ahci0: attempting to allocate 1 MSI vectors (1 supported) > ahci0: using IRQ 279 for MSI > > But vmstat -i does not report any interrupts. > > Just in case, the controller has some complex topology where the AHCI PCI device > is behind two PCI-PCI bridges from the main bus.  But I think that that should > not matter and the guest needs to see only the AHCI device and the host handles > the bridges. > > Here are some more messages from the guest: > ahci0: port > 0x21c0-0x21c7,0x21c8-0x21cb,0x21d0-0x21d7,0x21d8-0x21db,0x21e0-0x21ef mem > 0xc000e000-0xc000efff irq 17 at device 9.0 on pci0 > ahci0: attempting to allocate 1 MSI vectors (1 supported) > ahci0: using IRQ 279 for MSI > ahci0: AHCI v1.00 with 2 6Gbps ports, Port Multiplier supported with FBS > ahci0: Caps: 64bit NCQ SNTF AL 6Gbps PM FBS PMD SSC PSC 32cmd eSATA 2ports > ahcich0: at channel 0 on ahci0 > ahcich0: Caps: CPD ESP FBSCP > ahcich1: at channel 1 on ahci0 > ahcich1: Caps: CPD ESP FBSCP > ahcich0: AHCI reset... > ahcich0: SATA connect time=100us status=00000123 > ahcich0: AHCI reset: device found > ahcich1: AHCI reset... > ahcich1: SATA connect time=100us status=00000123 > ahcich1: AHCI reset: device found > ahcich1: AHCI reset: device ready after 100ms > ahcich1: Poll timeout on slot 1 port 15 > ahcich1: is 00000000 cs 00000002 ss 00000000 rs 00000002 tfd 150 serr 00000000 > cmd 00318017 > ahcich0: AHCI reset: device ready after 200ms > ahcich0: Poll timeout on slot 1 port 15 > ahcich0: is 00000000 cs 00000002 ss 00000000 rs 00000002 tfd 150 serr 00000000 > cmd 00318017 > And some more... > > Any ideas or suggestions? > Or maybe some additional information from me? > > Thank you! > > -- > Andriy Gapon > _______________________________________________ > freebsd-virtualization@freebsd.org > mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscribe@freebsd.org > " > > -- Andriy Gapon From owner-freebsd-virtualization@freebsd.org Tue Nov 14 04:22:08 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A732ACFF095 for ; Tue, 14 Nov 2017 04:22:08 +0000 (UTC) (envelope-from akgupt3@gmail.com) Received: from mail-qk0-x236.google.com (mail-qk0-x236.google.com [IPv6:2607:f8b0:400d:c09::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 662696D9C7; Tue, 14 Nov 2017 04:22:08 +0000 (UTC) (envelope-from akgupt3@gmail.com) Received: by mail-qk0-x236.google.com with SMTP id a194so20413948qkc.9; Mon, 13 Nov 2017 20:22:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=M9qqoydr9T0p+HwXsVxsTltBoqxbakTrxxCn6G0lD3M=; b=DDk7dYoZclwKX7cuNw6nRzDJq+lDfF6sgUky7BeNHWrN1XYyJoOAVMtflhcBW7ywjs ipTPKGV9gYHeqqkk0hqrB+05z//G2SxNb98kcms+5mAdgUrgYyfJ/SVoeL0EZlO1DOrB FOTUssJlenZCyHG00Hs8gDyXGGQ3JP/qiEGc4pmLsUoeOxv79Vl9RPJJCWZTwFoohctR JXYJ25xdtZJWSZ6VxukJGJu+kDqZskwGlHfsOhhCqwVY/V9pq091WljV8PPip8X3Gz65 3clQEtvkcXtysimKoaZvbKZoCbqpgu1T7tRdpAqm3FJXGmQBLdlx/ZN64Dujh4VbEDSP iDPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=M9qqoydr9T0p+HwXsVxsTltBoqxbakTrxxCn6G0lD3M=; b=coJinr8iw0RoCr0xkCbXw9g0CCHve/GRp+f/GvnSFsCIsB3uBmdMDEO0l5si9UTC0k m24FFZ4ax/eLDKmir2NzurWflOaPuh1AG4NqsaU5cb049gk7pjLP7kLasazYrrMM9VBM jLlfog2C2E5gpsJaNopan0i/9/QN6bzXLZYptH465FEwmlWbIh4C406lDXGTLu6eQxfA UpD3l+RvEZz+KvCCgTjfvw/un7bwnbG/5KWzm5uHU+RVtwXOptqVQ8URnCyAtJ8Jjfda Gyf9om1fqpD/qh7CZlZxyaC20fZGTOF0E/Uuk+el7Mn2rAsKsqBH7ieigQnl7mRVI2EI oMfQ== X-Gm-Message-State: AJaThX473CO2E6ANTNAPMnMY2ZL/mIxouY38Hc65If7z+/Zmr50fVL3q oXhHmAEdilRCdvr4PVMjRoINyFdQqHA4GO6WRq6GcQ== X-Google-Smtp-Source: AGs4zMbsrm2EGop89i7WSyNLNCW8UYKYQwNZFNEw2xDeTTPVZc9SSoE+3lYhM3RskD80OODoHqF0OQWCH8YvpmlxbIA= X-Received: by 10.233.232.13 with SMTP id a13mr17138146qkg.186.1510633327270; Mon, 13 Nov 2017 20:22:07 -0800 (PST) MIME-Version: 1.0 Received: by 10.12.172.247 with HTTP; Mon, 13 Nov 2017 20:22:06 -0800 (PST) In-Reply-To: References: From: Anish Date: Mon, 13 Nov 2017 20:22:06 -0800 Message-ID: Subject: Re: problem with pass-through on amd To: Andriy Gapon Cc: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Nov 2017 04:22:08 -0000 Hi Andriy, Newer AMD IOMMU has EFR capability which is missing here, something like: ivhd0: PCI cap 0x190b640f@0x40 feature:19 Is this an old chipset? Also ivhd has fault interrupt enabled which is very helpful in debugging: [root@ryzen /home/anish/FreeBSD/head]# vmstat -ia |grep ivh irq256: ivhd0:fault 0 0 irq257: ivhd1:fault 0 0 Regards, Anish On Mon, Nov 13, 2017 at 12:55 PM, Andriy Gapon wrote: > On 13/11/2017 22:41, Anish wrote: > > Hi Andriy, > > > >>What's suspicious is that there are no interrupts either for the > > ppt device on the host or the ahci device in the guest. > > > > I think most likely AMD IOMMU/amdvi is not enabled, by default it is > disabled > > and can be confirmed by sysctl. > > > > [root@ryzen /usr/home/anish]# sysctl hw.vmm.amdvi > > hw.vmm.amdvi.domain_id: 3 > > hw.vmm.amdvi.disable_io_fault: 0 > > hw.vmm.amdvi.ptp_level: 4 > > hw.vmm.amdvi.host_ptp: 1 > > hw.vmm.amdvi.enable: 1 > > hw.vmm.amdvi.count: 2 > > > > If hw.vmm.amdvi.enable is 0, set it and reload vmm and things should > work. > > > I followed your how-to, so I didn't make this mistake :-) > > $ sysctl hw.vmm.amdvi > hw.vmm.amdvi.domain_id: 3 > hw.vmm.amdvi.disable_io_fault: 0 > hw.vmm.amdvi.ptp_level: 4 > hw.vmm.amdvi.host_ptp: 1 > hw.vmm.amdvi.enable: 1 > hw.vmm.amdvi.count: 1 > > Could it be that AMD-Vi support is broken in the hardware or BIOS? > Are there any checks that I could do? > > Here is what I see in dmesg, but it is not a full dump of IVRS, of course: > $ dmesg | egrep -i 'ivrs|ivh|amd-vi|mmu' > Table 'IVRS' at 0xbdafaf80 > ACPI: IVRS 0x00000000BDAFAF80 0000F8 (v01 AMD RD890S 00202031 AMD > 00000000) > AMD-Vi IVRS VAsize = 64 PAsize = 52 GVAsize = 0 flags:0 > ivhd0: on acpi0 > ivhd0: Unknown dev entry:0xff > ivhd0: Flag:3e > ivhd0: max supported paging level:7 restricting to: 4 > ivhd0: device supported range [0x0 - 0xb01] > ivhd0: device [0xa0 - 0xa0]config:d7 > ivhd0: PCI cap 0x10b540f@0x40 feature:1 > pci0: at device 0.2 (no driver attached) > > > On Mon, Nov 13, 2017 at 11:02 AM, Andriy Gapon > > wrote: > > > > > > I have a FreeBSD guest on a FreeBSD host, both amd64 head. > > I decided to experiment with PCI pass-through and used an AHCI > controller for > > the experiment. It seems like that partially works. The guest > detects the > > controller and attaches a driver to it, the messages (reported > capabilities, > > etc) look identical to those on the host. But the guest can not > detect any > > disks behind the controller. From what I can tell, the guest has no > problems > > accessing PCI configuration registers and a memory mapped BAR. Not > sure about > > the I/O ports. What's suspicious is that there are no interrupts > either for the > > ppt device on the host or the ahci device in the guest. Maybe that > is what > > causes the failure to see the disks. > > > > When the guest driver attaches to the controller I see these > messages on the > > host: > > ppt0: attempting to allocate 1 MSI vectors (1 supported) > > ppt0: using IRQ 265 for MSI > > and in the guest: > > ahci0: attempting to allocate 1 MSI vectors (1 supported) > > ahci0: using IRQ 279 for MSI > > > > But vmstat -i does not report any interrupts. > > > > Just in case, the controller has some complex topology where the > AHCI PCI device > > is behind two PCI-PCI bridges from the main bus. But I think that > that should > > not matter and the guest needs to see only the AHCI device and the > host handles > > the bridges. > > > > Here are some more messages from the guest: > > ahci0: port > > 0x21c0-0x21c7,0x21c8-0x21cb,0x21d0-0x21d7,0x21d8-0x21db,0x21e0-0x21ef > mem > > 0xc000e000-0xc000efff irq 17 at device 9.0 on pci0 > > ahci0: attempting to allocate 1 MSI vectors (1 supported) > > ahci0: using IRQ 279 for MSI > > ahci0: AHCI v1.00 with 2 6Gbps ports, Port Multiplier supported with > FBS > > ahci0: Caps: 64bit NCQ SNTF AL 6Gbps PM FBS PMD SSC PSC 32cmd eSATA > 2ports > > ahcich0: at channel 0 on ahci0 > > ahcich0: Caps: CPD ESP FBSCP > > ahcich1: at channel 1 on ahci0 > > ahcich1: Caps: CPD ESP FBSCP > > ahcich0: AHCI reset... > > ahcich0: SATA connect time=100us status=00000123 > > ahcich0: AHCI reset: device found > > ahcich1: AHCI reset... > > ahcich1: SATA connect time=100us status=00000123 > > ahcich1: AHCI reset: device found > > ahcich1: AHCI reset: device ready after 100ms > > ahcich1: Poll timeout on slot 1 port 15 > > ahcich1: is 00000000 cs 00000002 ss 00000000 rs 00000002 tfd 150 > serr 00000000 > > cmd 00318017 > > ahcich0: AHCI reset: device ready after 200ms > > ahcich0: Poll timeout on slot 1 port 15 > > ahcich0: is 00000000 cs 00000002 ss 00000000 rs 00000002 tfd 150 > serr 00000000 > > cmd 00318017 > > And some more... > > > > Any ideas or suggestions? > > Or maybe some additional information from me? > > > > Thank you! > > > > -- > > Andriy Gapon > > _______________________________________________ > > freebsd-virtualization@freebsd.org > > mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > > > To unsubscribe, send any mail to > > "freebsd-virtualization-unsubscribe@freebsd.org > > " > > > > > > > -- > Andriy Gapon > From owner-freebsd-virtualization@freebsd.org Tue Nov 14 12:19:16 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3AABADBCCB0 for ; Tue, 14 Nov 2017 12:19:16 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lf0-f53.google.com (mail-lf0-f53.google.com [209.85.215.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E747C7C0AB for ; Tue, 14 Nov 2017 12:19:15 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lf0-f53.google.com with SMTP id a132so22058083lfa.7 for ; Tue, 14 Nov 2017 04:19:15 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=9W3x897ojDTH+hqhJbQsof60RT3xpzOkTmOCk4234FI=; b=YIAEQqJ2mfcs7NlCC9JvLdN/5p/E1ud18uU09E2J8A+zOuCyTOhp3I1c1use/HKJ+6 snVKrwEl6HjH8mSg0whYgfrjkfZeeAAgOuxvlZFG4YB3zTp0PGnOy3OUL5hHr6FNEmx6 /aSdqSqjZFkfKjhA+Yyrjkr6Gth2KOxuKZsoQ4UmVnLiWOq54iUx/+WAn+6zi2RFmld6 QfZOuEyJQ21yMKJ1ICQDyYV49ISqAjOB2Ot/z48gHoonqn+Y1UPiqBrjDjDlDuAGl87V zHq59H1qf5oKfwBQLOf+SzwAzZCUfSEdiofJAYN+JM9/b+w20Q8+YPmLlbIJw+TJUCUs UrEw== X-Gm-Message-State: AJaThX4kDVH6P/HTa/D0kcBBmA1abQQUEQOusaBH6Z+pVbrQCiW1VbaW ILw7sh8Ae8BxOqAM/OBQ5PH41CMi X-Google-Smtp-Source: AGs4zMZ4VnUGQpgEITgxKAM8JKGU8FofpHgyRltwvHGAzJ06HaSHj0PWn8spfzkOC6m47cHR6S4LhA== X-Received: by 10.46.80.16 with SMTP id e16mr4960769ljb.165.1510661444053; Tue, 14 Nov 2017 04:10:44 -0800 (PST) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id p66sm3271024lfp.45.2017.11.14.04.10.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 14 Nov 2017 04:10:43 -0800 (PST) Subject: Re: problem with pass-through on amd To: Anish Cc: "freebsd-virtualization@freebsd.org" References: From: Andriy Gapon Message-ID: Date: Tue, 14 Nov 2017 14:10:41 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Nov 2017 12:19:16 -0000 On 14/11/2017 06:22, Anish wrote: > Hi Andriy, > Newer AMD IOMMU has EFR capability which is missing here, something like: > ivhd0: PCI cap 0x190b640f@0x40 feature:19 > > Is this an old chipset? Yes, this is an old 900 series chipset (990X). > Also ivhd has fault interrupt enabled which is very > helpful in debugging: > > [root@ryzen /home/anish/FreeBSD/head]# vmstat -ia |grep ivh > irq256: ivhd0:fault                    0          0 > irq257: ivhd1:fault                    0          0 irq256: ivhd0:fault 0 0 > Another thing to worth checking is PCI config space of ivhd/IOMMU if you have not already done so. For what should I look there? -- Andriy Gapon From owner-freebsd-virtualization@freebsd.org Wed Nov 15 14:19:51 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2BE32DDE0F6 for ; Wed, 15 Nov 2017 14:19:51 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C27236CAAE; Wed, 15 Nov 2017 14:19:50 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (mh0.gentlemail.de [IPv6:2a00:e10:2800::a135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id vAFEJlUj074122; Wed, 15 Nov 2017 15:19:47 +0100 (CET) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 3F1BEC8B; Wed, 15 Nov 2017 15:19:47 +0100 (CET) Message-ID: <5A0C4CFF.7040902@omnilan.de> Date: Wed, 15 Nov 2017 15:19:43 +0100 From: Harry Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: Peter Grehan CC: freebsd-virtualization@freebsd.org Subject: Re: pci_virtio_block.c Assertion failed: line 216 References: <586BFC91.4090404@omnilan.de> <2a8308c3-1570-08a3-a993-0f699fcd267d@freebsd.org> In-Reply-To: <2a8308c3-1570-08a3-a993-0f699fcd267d@freebsd.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]); Wed, 15 Nov 2017 15:19:47 +0100 (CET) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Nov 2017 14:19:51 -0000 Bezüglich Peter Grehan's Nachricht vom 03.01.2017 20:36 (localtime): > Hi Harry, > >> trying to use bhyve(8) with virtio-blk and Windows guest results in core >> dump: >> Assertion failed: (n >= 2 && n <= BLOCKIF_IOV_MAX + 2), function >> pci_vtblk_proc, file usr.sbin/bhyve/pci_virtio_block.c, line >> 216. >> Abort trap (core dumped) >> >> Unfortunately this is on a production-test machine which lacks gdb etc. >> Will try to reproduce on antoher machine, but maybe someone already >> knows that problem? > > virtio-blk isn't currently supported with Windows guests. You'll need > to use ahci-hd for now. > > However, I do have a fix that can hopefully be committed shortly. … Mising in another reply: >> Wiadomość napisana przez Harry Schmalzbauer > > w dniu 03.01.2017, o godz. 20:33: … >> Will try to reproduce on antoher machine, but maybe someone already >> knows that problem? > > I've seen that problem and fixed it, will upstream the patch later today. > > JFYI, fixing > commit: https://github.com/freenas/os/commit/0e4d6e1826f8aa7041cbeeb4365c797eeec5c5f4 Thanks Jakub for the info. I can confirm that increasing BLOCKIF_IOV_MAX from 33 to 128, like the diff shows, solved the problem for me. I've successfully done some performace tests on Windows7 (virtio-blk vs. ahci,hd:) and also migrated one Server 2012R2 to bhyve using virtio-blk. Peter, is your mentioned fix different from just increasing BLOCKIF_IOV_MAX? If not, would you commit that please? Thanks, -harry From owner-freebsd-virtualization@freebsd.org Wed Nov 15 17:18:01 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BFB4CDE26C1 for ; Wed, 15 Nov 2017 17:18:01 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (smtp.digiware.nl [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8AA1772BBB for ; Wed, 15 Nov 2017 17:18:01 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from router.digiware.nl (localhost.digiware.nl [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 545AB2EEF9; Wed, 15 Nov 2017 18:17:58 +0100 (CET) X-Virus-Scanned: amavisd-new at digiware.com Received: from smtp.digiware.nl ([127.0.0.1]) by router.digiware.nl (router.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 3XPYyu8UUAbA; Wed, 15 Nov 2017 18:17:57 +0100 (CET) Received: from [192.168.10.67] (opteron [192.168.10.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 4B0482EEF8; Wed, 15 Nov 2017 18:17:57 +0100 (CET) To: Ceph Development , "freebsd-virtualization@freebsd.org" From: Willem Jan Withagen Subject: Running a FreeBSD bhyve instance on a Ceph cluster Message-ID: Date: Wed, 15 Nov 2017 18:17:55 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: nl Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Nov 2017 17:18:01 -0000 Hi, Just an info point. I'm preparing for a lecture tomorrow, and thought why not do an actual demo.... Like to be friends with Murphy :) So after I started the cluster: 5 jails with 7 OSDs This what I manually needed to do to boot a memory stick ================ # Start een Bhyve instance rbd --dest-pool rbd_data --no-progress import memstick.img memstick rbd-ggate map rbd_data/memstick # ggate-devvice is available on /dev/ggate1 kldload vmm kldload nmdm kldload if_tap kldload if_bridge kldload cpuctl sysctl net.link.tap.up_on_open=1 ifconfig bridge0 create ifconfig bridge0 addm em0 up ifconfig ifconfig tap11 create ifconfig bridge0 addm tap11 ifconfig tap11 up # load the GGate disk in bhyve bhyveload -c /dev/nmdm11A -m 2G -d /dev/ggate1 FB11 # and boot a single from it. bhyve -H -P -A -c 1 -m 2G -l com1,/dev/nmdm11A -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap11 -s 4,ahci-hd,/dev/ggate1 FB11 & bhyvectl --vm=FB11 --get-stats # Connect to the VM cu -l /dev/nmdm11B ========= And that'll give you a bhyve VM running on an RBD image over ggate. In the installer I tested reading from the bootdisk: ==== root@:/ # dd if=/dev/ada0 of=/dev/null bs=32M 21+1 records in 21+1 records out 734077952 bytes transferred in 5.306260 secs (138341865 bytes/sec) ==== which is a nice 138Mb/sec. Hope the demonstration does work out tomorrow. --WjW From owner-freebsd-virtualization@freebsd.org Wed Nov 15 18:39:06 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8F110DE3DB6 for ; Wed, 15 Nov 2017 18:39:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7D030751F4 for ; Wed, 15 Nov 2017 18:39:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vAFId571067578 for ; Wed, 15 Nov 2017 18:39:06 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 223531] Instant kernel panic on 'kldload virtio_console' on oVirt 3.5 VM Date: Wed, 15 Nov 2017 18:39:05 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: cem@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Nov 2017 18:39:06 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D223531 Conrad Meyer changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |jceel@FreeBSD.org, | |trasz@FreeBSD.org --- Comment #11 from Conrad Meyer --- FYI, I believe this was introduced in r305900. (prep_devname has rejected spaces in device names since r244584.) --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Nov 16 00:41:17 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A4C65DF0124 for ; Thu, 16 Nov 2017 00:41:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 92A89650C6 for ; Thu, 16 Nov 2017 00:41:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vAG0fHFS085499 for ; Thu, 16 Nov 2017 00:41:17 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 184046] bhyve(4) manpage references non-existant manpages bhyvectl(8), vmm(4) Date: Thu, 16 Nov 2017 00:41:17 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Documentation X-Bugzilla-Component: Documentation X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: sevan@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Nov 2017 00:41:17 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D184046 Sevan Janiyan changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |sevan@freebsd.org --- Comment #1 from Sevan Janiyan --- We have an incomplete bhyvectl manual, vmmm manual is still missing. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Nov 16 02:01:50 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5A8D3DB801E for ; Thu, 16 Nov 2017 02:01:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48518673D7 for ; Thu, 16 Nov 2017 02:01:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vAG21nkb075576 for ; Thu, 16 Nov 2017 02:01:50 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 184046] bhyve(4) manpage references non-existant manpages bhyvectl(8), vmm(4) Date: Thu, 16 Nov 2017 02:01:50 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Documentation X-Bugzilla-Component: Documentation X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: grehan@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Nov 2017 02:01:50 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D184046 Peter Grehan changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |grehan@FreeBSD.org --- Comment #2 from Peter Grehan --- There is a review for vmm(4) at https://reviews.freebsd.org/D13062 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Nov 16 09:32:42 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 03AF5DD72E6 for ; Thu, 16 Nov 2017 09:32:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E53C373C1E for ; Thu, 16 Nov 2017 09:32:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vAG9Wege085326 for ; Thu, 16 Nov 2017 09:32:41 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 223531] Instant kernel panic on 'kldload virtio_console' on oVirt 3.5 VM Date: Thu, 16 Nov 2017 09:32:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: wosch@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Nov 2017 09:32:42 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D223531 Wolfram Schneider changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |wosch@FreeBSD.org --- Comment #12 from Wolfram Schneider --- I have a virtual machine running with FreeBSD 11-stable at a cloud provider. It runs fine. I tried to upgrade to FreeBSD 12-current and the kernel panics at boot time with: make_dev_alias_v bad si_name (error=3D22) si_name=3Dvtcon/ org.qemu.guest_a= gent.0) and hangs. Looks like the same problem as described in this bug report. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Nov 16 09:36:31 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1D4ECDD7395 for ; Thu, 16 Nov 2017 09:36:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0ABE373CCB for ; Thu, 16 Nov 2017 09:36:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vAG9aUbC090260 for ; Thu, 16 Nov 2017 09:36:30 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 223531] Instant kernel panic on 'kldload virtio_console' on oVirt 3.5 VM Date: Thu, 16 Nov 2017 09:36:31 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: wosch@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Nov 2017 09:36:31 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D223531 Wolfram Schneider changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |bryanv@FreeBSD.org, | |grehan@FreeBSD.org --- Comment #13 from Wolfram Schneider --- See also: https://svnweb.freebsd.org/base?view=3Drevision&revision=3Dr305900 https://reviews.freebsd.org/D7182 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Nov 16 09:38:49 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BA2EEDD7443 for ; Thu, 16 Nov 2017 09:38:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A815573D6B for ; Thu, 16 Nov 2017 09:38:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vAG9cnD0093219 for ; Thu, 16 Nov 2017 09:38:49 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 223531] Instant kernel panic on 'kldload virtio_console' on oVirt 3.5 VM Date: Thu, 16 Nov 2017 09:38:49 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: wosch@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_file_loc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Nov 2017 09:38:49 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D223531 Wolfram Schneider changed: What |Removed |Added ---------------------------------------------------------------------------- URL| |https://reviews.freebsd.org | |/D7182 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Nov 16 09:40:36 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 81D73DD7551 for ; Thu, 16 Nov 2017 09:40:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6FB0B73E71 for ; Thu, 16 Nov 2017 09:40:36 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vAG9eZhv095712 for ; Thu, 16 Nov 2017 09:40:36 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 223531] Instant kernel panic on 'kldload virtio_console' on oVirt 3.5 VM Date: Thu, 16 Nov 2017 09:40:36 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: cem@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Nov 2017 09:40:36 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D223531 --- Comment #14 from Conrad Meyer --- (In reply to Wolfram Schneider from comment #12) Yes, r305900 was introduced in CURRENT and never MFCed to 11. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Nov 16 09:46:47 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0C758DD7898 for ; Thu, 16 Nov 2017 09:46:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EED09741C0 for ; Thu, 16 Nov 2017 09:46:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vAG9kksC012498 for ; Thu, 16 Nov 2017 09:46:46 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 223531] Instant kernel panic on 'kldload virtio_console' on oVirt 3.5 VM Date: Thu, 16 Nov 2017 09:46:46 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: wosch@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: dependson cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Nov 2017 09:46:47 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D223531 Wolfram Schneider changed: What |Removed |Added ---------------------------------------------------------------------------- Depends on| |144736, 161912 CC| |imp@FreeBSD.org Referenced Bugs: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D144736 [Bug 144736] devd(8) should consider spaces in event description when processing events https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D161912 [Bug 161912] [kernel] [patch] kernel sends incorrect notify to devctl about device changes --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Nov 16 11:18:35 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 44B6FDD992A for ; Thu, 16 Nov 2017 11:18:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 323C176965 for ; Thu, 16 Nov 2017 11:18:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vAGBIYcp027596 for ; Thu, 16 Nov 2017 11:18:35 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 223531] Instant kernel panic on 'kldload virtio_console' on oVirt 3.5 VM Date: Thu, 16 Nov 2017 11:18:34 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: wosch@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Nov 2017 11:18:35 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D223531 --- Comment #15 from Wolfram Schneider --- Created attachment 188043 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D188043&action= =3Dedit remove space check --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Nov 16 11:19:48 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8551CDD9988 for ; Thu, 16 Nov 2017 11:19:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6FBCC769DF for ; Thu, 16 Nov 2017 11:19:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vAGBJlfg029091 for ; Thu, 16 Nov 2017 11:19:48 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 223531] Instant kernel panic on 'kldload virtio_console' on oVirt 3.5 VM Date: Thu, 16 Nov 2017 11:19:48 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: wosch@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Nov 2017 11:19:48 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D223531 --- Comment #16 from Wolfram Schneider --- I agree that spaces in filenames are not a good idea.=20 However, a kernel which is not booting due spaces is very bad. I removed the space check in sys/kern/kern_conf.c and now I can boot my current kernel, a= nd upgrade the VM. Attached a patch with a one line change. It could be that this change has unknown side effect for devd (PR #144736, PR #161912). Lets see. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Nov 16 11:20:26 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 799ACDD99DE for ; Thu, 16 Nov 2017 11:20:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6789376A54 for ; Thu, 16 Nov 2017 11:20:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vAGBKPge029994 for ; Thu, 16 Nov 2017 11:20:26 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 223531] Instant kernel panic on 'kldload virtio_console' on oVirt 3.5 VM Date: Thu, 16 Nov 2017 11:20:25 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: wosch@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Nov 2017 11:20:26 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D223531 Wolfram Schneider changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |In Progress --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Nov 16 18:43:10 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 910A3DE5A0F for ; Thu, 16 Nov 2017 18:43:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7EAF265368 for ; Thu, 16 Nov 2017 18:43:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id vAGIh9Yd093015 for ; Thu, 16 Nov 2017 18:43:10 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 223531] Instant kernel panic on 'kldload virtio_console' on oVirt 3.5 VM Date: Thu, 16 Nov 2017 18:43:09 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: cem@freebsd.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Nov 2017 18:43:10 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D223531 --- Comment #17 from Conrad Meyer --- I think a better solution might be to strip or replace unwanted characters = from the name we receive from virtio. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Nov 16 20:45:41 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 43E5FDE8A8D for ; Thu, 16 Nov 2017 20:45:41 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lf0-f47.google.com (mail-lf0-f47.google.com [209.85.215.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C495769B16 for ; Thu, 16 Nov 2017 20:45:40 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lf0-f47.google.com with SMTP id k66so356307lfg.3 for ; Thu, 16 Nov 2017 12:45:40 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:subject:to:references:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=kR80g8RUT3VnvOwjHFhL0BOiEZQlaupGs2f76KoDs+s=; b=R9Ec+SzH/5m3I63cUvAYp1/Uzi9lWTkE5yDIG26Wx72zWc1ciMl3RKyZdVyA/bzxCq DIte65AgCpaVF39KAUfp2lQIR1rQX5ya1ImoZt6aIJvk5RlRzFDVD+QCDJyPhZP2CBF5 Q1SnuWsnfOblxcDw1RdLJZmTy4TiLBtbuEwg/+ncloAA0tkidJxW8U5brj7RfO8rF6Gx XlN+vKI95nl3octQFs77gq7l0a3oDKkjMGEz60S6KpcPkrtPgdA1QXjlb54UEdB6fZQU yB6KfnB/9n80qQsy97hR02K9KigwnP+zvXmN3ncbXZOQw6rb6rs+ksAuThUCXopD9s8T HGDg== X-Gm-Message-State: AJaThX6J33DEdtuGABEEHHa4Fi1/bCX7Y8prAR0iFmFPhdG5+FGMQauC F48D04i8sj1ywoRGQeBO7955t776 X-Google-Smtp-Source: AGs4zMYjiGhdSCDaLFHzRlMMl6UEWABTo5l5MWqYYnS0V840rfImgekd45Y1TGFR5ScK5ZdmLeq5Kw== X-Received: by 10.25.208.20 with SMTP id h20mr39952lfg.26.1510865138079; Thu, 16 Nov 2017 12:45:38 -0800 (PST) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id a78sm361474lfa.86.2017.11.16.12.45.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 16 Nov 2017 12:45:36 -0800 (PST) From: Andriy Gapon Subject: Re: problem with pass-through on amd To: Anish , "freebsd-virtualization@freebsd.org" References: Message-ID: Date: Thu, 16 Nov 2017 22:45:35 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Nov 2017 20:45:41 -0000 On 14/11/2017 06:22, Anish wrote: > Also ivhd has fault interrupt enabled which is very helpful in debugging: > > [root@ryzen /home/anish/FreeBSD/head]# vmstat -ia |grep ivh > irq256: ivhd0:fault                    0          0 > irq257: ivhd1:fault                    0          0 > Anish, I have made several interesting discoveries regarding my problem. One of them is that actually there were some IOMMU log events: dev.ivhd.0.event_tail: 240 dev.ivhd.0.event_head: 0 dev.ivhd.0.event_intr_count: 0 But there were no interrupts and the events are unconsumed and unreported. I examined MSI configuration of the IOMMU PCI device and the address and data registers were zeroed out. I looked at dmesg and at the code and I realized why that happened. So, first of all, I pre-load vmm via loader.conf. Probably as a result of that the ivhd device attaches before any bridges and buses on my system. And amdvi_alloc_intr_resources() does a rather untypical thing, it configures an MSI for a PCI device by directly writing to its configuration registers. The PCI bus code is completely unaware of those changes and it wipes them out in pci_add_child() -> pci_cfg_restore(). Also, I think that even if ivhd attached after the root PCI bus, then what it does would be still unsafe. I think that, for example, a suspend-resume cycle would wipe out the MSI configuration too. I think that in that case we should better use pci methods to configure MSI. Now, why does ivhd attach before the root Host-PCI bridge and what can we do to fix the order? ivrs_drv.c has this code: /* * Load this module at the end after PCI re-probing to configure interrupt. */ DRIVER_MODULE_ORDERED(ivhd, acpi, ivhd_driver, ivhd_devclass, 0, 0, SI_ORDER_ANY); But apparently this SI_ORDER_ANY does not help much. It affects only the driver registration order, but not the device probe and attachment order. This code is far more significant: ivhd_devs[i] = BUS_ADD_CHILD(parent, 1, "ivhd", i); ivhd passes 1 as the order. This is a very high order for the acpi bus. As a comment in acpi_probe_child() says: /* * Create a placeholder device for this node. Sort the * placeholder so that the probe/attach passes will run * breadth-first. Orders less than ACPI_DEV_BASE_ORDER * are reserved for special objects (i.e., system * resources). */ where ACPI_DEV_BASE_ORDER is 100. For example, order of the Host-PCI bridge on my system is 120. I must note that this is important only of vmm is preloaded (which is probably not an extremely rare case). If vmm is loaded after the system is booted then, of course, ivhd will be probed after the PCI buses / bridges. -- Andriy Gapon From owner-freebsd-virtualization@freebsd.org Thu Nov 16 20:52:06 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 00390DE8E3C for ; Thu, 16 Nov 2017 20:52:06 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lf0-f41.google.com (mail-lf0-f41.google.com [209.85.215.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A668969FBA for ; Thu, 16 Nov 2017 20:52:05 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lf0-f41.google.com with SMTP id 73so350448lfu.10 for ; Thu, 16 Nov 2017 12:52:05 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:subject:to:cc:references:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=vJWOc99V9FgL1kEfB6FcbSPLvgtfhigIOfc5ygzwj9U=; b=p0yjJl7uBgVOJzSLBjIywLfOYqhWzp6khPV8v/FEvOihOv29PQA2TF1sgbAg4vOeXc xOFCkJf7xOM8n4GYAu5hV8OmHPzK4E4VyKI7lRB4ydeWGIy0j9tulzJTHP88m/OcwDTD ryQiR8rzMWhgpaWRCRFyuDOQE3zXKleVVsQCJ+T0axSUn6bOLMAt5LqbZ/rzY1n7Kty/ 9MjU97dV52YuiqnQFR8IDoWD/rOqtXAJgqGkkM0XORd77fNMavYkzsNsHxmVwfVtS3p4 sSEt/Jf3VvrnqGgPlTqbBUaikAv2HpIVLohqBM6scsps2iOtxyWxanefOi+ex/p6X1G0 j4mw== X-Gm-Message-State: AJaThX66h/04+tclvaZHDHT9AVZeGO5yq+mx0Upwz+oYtIzDZ2OIrm9A Zkm0aSm/JhSTZmxhZol8MsrVk4uN X-Google-Smtp-Source: AGs4zMaKDmmMvKlPiiRrLkqnKdQZ8cn+i5ajbZBUlxw7Qy+y+jh/GSN1YddCaKQQXi9Cuc13eOFmNg== X-Received: by 10.25.167.213 with SMTP id q204mr32794lfe.219.1510865517526; Thu, 16 Nov 2017 12:51:57 -0800 (PST) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id a12sm349547ljd.19.2017.11.16.12.51.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 16 Nov 2017 12:51:56 -0800 (PST) From: Andriy Gapon Subject: Re: problem with pass-through on amd To: Anish Cc: "freebsd-virtualization@freebsd.org" References: Message-ID: Date: Thu, 16 Nov 2017 22:51:55 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Nov 2017 20:52:06 -0000 On 14/11/2017 06:22, Anish wrote: > [root@ryzen /home/anish/FreeBSD/head]# vmstat -ia |grep ivh > irq256: ivhd0:fault                    0          0 > irq257: ivhd1:fault                    0          0 After I fixed the MSI setup problem that I described in the previous email I was finally able to see the real problem. Now I have: irq263: ivhd0:fault 9 0 dev.ivhd.0.event_tail: 240 dev.ivhd.0.event_head: 240 dev.ivhd.0.event_intr_count: 9 I've got a bunch of messages like these in the log: [IO_PAGE_FAULT EVT: devId:0xa01 DomId:0x2 Addr:0x70f70400x30] [IO_PAGE_FAULT EVT: devId:0xa01 DomId:0x2 Addr:0x2fdf0400x30] (BTW, there seems to be a missing space before 0x30) Now it's obvious what the problem is. The controller has a RID of 0xa00 (its PCI locator is 10:0:0) but the requests are coming from 0xa01. The card actually has another PCI device at that location (10:0:1), it's a vestigial IDE controller (in a sense that it is not connected to any ports, so it cannot really provide any functionality). I've passed both of the PCI device to bhyve and everything started working. Thanks! I've googled a little bit and it seems that this is not an uncommon problem. Linux has a bunch of quirks for this and similar kinds of problems (they call it PCI or DMA aliasing): http://elixir.free-electrons.com/linux/v4.8.16/source/drivers/pci/quirks.c#L3701 The comments say that sometimes requests come from devices that do not exist at all (or hidden beyond non-transparent bridges). An interesting new world for me :-) By the way, my understanding is that the alias IVHD entries in IVRS are designed to report such aliasing issues when they are known to the platform. But it appears to me that FreeBSD currently ignores those aliases. Also, it looks like we are not making any use of IVHD entries beyond printing them. Based on the specification I think that we should use IVHD flags to set up appropriate bits in the corresponding device table entries. But we are not doing that. That's probably okay since we do not support the pass-though for devices with complex properties anyway. -- Andriy Gapon From owner-freebsd-virtualization@freebsd.org Fri Nov 17 03:33:46 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 791C8DB902E for ; Fri, 17 Nov 2017 03:33:46 +0000 (UTC) (envelope-from akgupt3@gmail.com) Received: from mail-qt0-x22e.google.com (mail-qt0-x22e.google.com [IPv6:2607:f8b0:400d:c0d::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2A84F781EB; Fri, 17 Nov 2017 03:33:46 +0000 (UTC) (envelope-from akgupt3@gmail.com) Received: by mail-qt0-x22e.google.com with SMTP id n61so2912625qte.10; Thu, 16 Nov 2017 19:33:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Dmc74Oo0JADAGm+CWfRfCL10Au0ENF7Xq9CgPZOoUsU=; b=AIzxspLvGum0Ynvq5N/KhYXBJuWz77DjRwp0Zp4DinDcueOMpA14Wj/Jt02N4bFRq+ 3dPUat7qy+GMdc9g8oGRaCRA0OQKr+VZ4Y0UOgW2kjlzKk3mmC73NRJ3mCqup8XF5CB9 S4XNYCsXLaA8fshgGp+mOVnaBtNeAEbTEzoGIafsZVLgVR2iindFPwc1H6wpk0GYs6Dy TQ8/VSIyiohUCm1lFrITFlBa2szi1Lnuyd3GxjUyAde6HJKn2ScbqJYxV5nQ043p+NMk 4b3PrHtvBz+WOXxIXZ5TrFhGi/c651l1p6lvg+Sw21+af/HTqzr7dGHCNAdbKiytihtM WIRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Dmc74Oo0JADAGm+CWfRfCL10Au0ENF7Xq9CgPZOoUsU=; b=XygzUhNGxYCldkIBv5NilOI3d2fxnSc3TLPYTTjVPfVZ1nqqM0XSi3libyDIhRuXC4 o9YRAnt1K4jrtC5I8dtqkaC45YgUh9UFK5FZTT1s6TTKrSUvm9I2MiRNVWUPTit9Ayoe 71JcwZwgXsbB5V0s0b61EBLWaCFlrE/cpY1mSQrsRw/mhxN6wn1bJDsuhbKMy35QStf/ aVF8HvVVvjRQnEolG7K5us7eEz4HfIpsCB6+Ym8OA6eBBfD4kQdjmUteHPkxaAHtoOI9 rq/ddk1NNCMm+Xyt4BK/BvI3S+lmPwEQBOWcPeicNqwsv7j+TgyAMDhqYy3yMFOPoPyA GNhw== X-Gm-Message-State: AJaThX68F5RmJANI4xT8WX1kxnmgaLo5LY+iSZ2Wl7s3SQ5CPbPsHcGH 2t2QNrm3Ps0oJOU16nbvQkN2hR/mw01Nz7g1EbY1qg== X-Google-Smtp-Source: AGs4zMYiN0MTC7CnTR1ymz/xl+sLnB+pjZzn3F+bVEo2jDw27oUf6KIGNkg++nAKeMqrCV75bVPX886yFHIUUmn4xrQ= X-Received: by 10.55.126.198 with SMTP id z189mr6422962qkc.100.1510889625011; Thu, 16 Nov 2017 19:33:45 -0800 (PST) MIME-Version: 1.0 Received: by 10.12.172.247 with HTTP; Thu, 16 Nov 2017 19:33:44 -0800 (PST) In-Reply-To: References: From: Anish Date: Thu, 16 Nov 2017 19:33:44 -0800 Message-ID: Subject: Re: problem with pass-through on amd To: Andriy Gapon Cc: "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Nov 2017 03:33:46 -0000 Hi Andriy, >After I fixed the MSI setup problem that I described in the previous email I was finally able to see the real problem. Awesome! fault interrupt handler has been very helpful in debugging AMD-Vi. >I've passed both of the PCI device to bhyve and everything started working. Thanks! Bhyve uses single domain for all the passthrough devices of a guest, that's why even with aliasing it worked. >But it appears to me that FreeBSD currently ignores those aliases. Also, it looks like we are not making any use of IVHD entries beyond printing them. We make use of of very few IVRS entries, see ivhd_dev_parse() but not much for aliasing entries. Found out AMD IOMMU spec has some details on it: "An Alias device type entry is used for each peripheral that does not use its own DeviceID information in bus transactions. For example, peripherals downstream of a bridge device that use the DeviceID of the bridge must have a corresponding Alias Select or Alias Start of Range entry to inform system software which IOMMU Device Table entry will be used for translation information." Thanks a lot for trying out AMD-Vi, let me know if you have any other feedback. Thanks and regards, Anish On Thu, Nov 16, 2017 at 12:51 PM, Andriy Gapon wrote: > On 14/11/2017 06:22, Anish wrote: > > [root@ryzen /home/anish/FreeBSD/head]# vmstat -ia |grep ivh > > irq256: ivhd0:fault 0 0 > > irq257: ivhd1:fault 0 0 > > After I fixed the MSI setup problem that I described in the previous email > I was > finally able to see the real problem. > Now I have: > > irq263: ivhd0:fault 9 0 > > dev.ivhd.0.event_tail: 240 > dev.ivhd.0.event_head: 240 > dev.ivhd.0.event_intr_count: 9 > > > I've got a bunch of messages like these in the log: > [IO_PAGE_FAULT EVT: devId:0xa01 DomId:0x2 Addr:0x70f70400x30] > [IO_PAGE_FAULT EVT: devId:0xa01 DomId:0x2 Addr:0x2fdf0400x30] > (BTW, there seems to be a missing space before 0x30) > > Now it's obvious what the problem is. > The controller has a RID of 0xa00 (its PCI locator is 10:0:0) but the > requests > are coming from 0xa01. The card actually has another PCI device at that > location (10:0:1), it's a vestigial IDE controller (in a sense that it is > not > connected to any ports, so it cannot really provide any functionality). > > I've passed both of the PCI device to bhyve and everything started working. > Thanks! > > I've googled a little bit and it seems that this is not an uncommon > problem. > Linux has a bunch of quirks for this and similar kinds of problems (they > call it > PCI or DMA aliasing): > http://elixir.free-electrons.com/linux/v4.8.16/source/ > drivers/pci/quirks.c#L3701 > The comments say that sometimes requests come from devices that do not > exist at > all (or hidden beyond non-transparent bridges). > > An interesting new world for me :-) > > By the way, my understanding is that the alias IVHD entries in IVRS are > designed > to report such aliasing issues when they are known to the platform. > But it appears to me that FreeBSD currently ignores those aliases. > Also, it looks like we are not making any use of IVHD entries beyond > printing > them. Based on the specification I think that we should use IVHD flags to > set > up appropriate bits in the corresponding device table entries. But we are > not > doing that. That's probably okay since we do not support the pass-though > for > devices with complex properties anyway. > > -- > Andriy Gapon >