From owner-freebsd-virtualization@freebsd.org Sun Mar 18 13:19:48 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 05BFFF53255 for ; Sun, 18 Mar 2018 13:19:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 951A270BA4 for ; Sun, 18 Mar 2018 13:19:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id DB2CB18431 for ; Sun, 18 Mar 2018 13:19:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w2IDJkZs003410 for ; Sun, 18 Mar 2018 13:19:46 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w2IDJklZ003409 for freebsd-virtualization@FreeBSD.org; Sun, 18 Mar 2018 13:19:46 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 176694] [kvm] KVM Guest Crash at Boot - kernel trap 12 with interrupts disabled [regression] Date: Sun, 18 Mar 2018 13:19:46 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: uefi X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: keywords assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Mar 2018 13:19:48 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D176694 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |uefi Assignee|freebsd-bugs@FreeBSD.org |freebsd-virtualization@Free | |BSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Sun Mar 18 21:00:40 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B14B4F534B1 for ; Sun, 18 Mar 2018 21:00:40 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 54F4C8687C for ; Sun, 18 Mar 2018 21:00:40 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 887921C3CE for ; Sun, 18 Mar 2018 21:00:39 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w2IL0d1J043821 for ; Sun, 18 Mar 2018 21:00:39 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w2IL0dZ6043818 for freebsd-virtualization@FreeBSD.org; Sun, 18 Mar 2018 21:00:39 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201803182100.w2IL0dZ6043818@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: freebsd-virtualization@FreeBSD.org Subject: Problem reports for freebsd-virtualization@FreeBSD.org that need special attention Date: Sun, 18 Mar 2018 21:00:39 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Mar 2018 21:00:40 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 212820 | FreeBSD 10-STABLE from latest HEAD and 11-RELEASE 1 problems total for which you should take action. From owner-freebsd-virtualization@freebsd.org Mon Mar 19 08:20:45 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 32E7BF5AB26 for ; Mon, 19 Mar 2018 08:20:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C5DDA822D7 for ; Mon, 19 Mar 2018 08:20:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 10F582229A for ; Mon, 19 Mar 2018 08:20:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w2J8KhXJ048537 for ; Mon, 19 Mar 2018 08:20:43 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w2J8Kh9Y048536 for freebsd-virtualization@FreeBSD.org; Mon, 19 Mar 2018 08:20:43 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 226583] FreeBSD VM on Hyper-V/Azure can't properly detected SCSI disk da2 Date: Mon, 19 Mar 2018 08:20:43 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: needs-qa, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? mfc-stable11? X-Bugzilla-Changed-Fields: keywords bug_status flagtypes.name Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Mar 2018 08:20:45 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D226583 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- Keywords| |needs-qa Status|New |Open Flags| |mfc-stable10?, | |mfc-stable11? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Mon Mar 19 08:22:23 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D40A5F5AE11 for ; Mon, 19 Mar 2018 08:22:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6F74F82586 for ; Mon, 19 Mar 2018 08:22:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 88A10223D6 for ; Mon, 19 Mar 2018 08:22:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w2J8MMlm060011 for ; Mon, 19 Mar 2018 08:22:22 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w2J8MM4h060010 for freebsd-virtualization@FreeBSD.org; Mon, 19 Mar 2018 08:22:22 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: freebsd-virtualization@FreeBSD.org Subject: [Bug 226583] FreeBSD VM on Hyper-V/Azure can't properly detected SCSI disk da2 Date: Mon, 19 Mar 2018 08:22:22 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: needs-qa, patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: koobs@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-virtualization@FreeBSD.org X-Bugzilla-Flags: mfc-stable10? mfc-stable11? X-Bugzilla-Changed-Fields: bug_file_loc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Mar 2018 08:22:24 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D226583 Kubilay Kocak changed: What |Removed |Added ---------------------------------------------------------------------------- URL| |https://reviews.freebsd.org | |/D14690 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Thu Mar 22 08:52:49 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9625BF51666 for ; Thu, 22 Mar 2018 08:52:49 +0000 (UTC) (envelope-from support@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1C0CF68A5B for ; Thu, 22 Mar 2018 08:52:49 +0000 (UTC) (envelope-from support@omnilan.de) Received: from mh0.gentlemail.de (mh0.gentlemail.de [IPv6:2a00:e10:2800::a135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id w2M8ql1B065751 for ; Thu, 22 Mar 2018 09:52:47 +0100 (CET) (envelope-from support@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id BB2AAD20 for ; Thu, 22 Mar 2018 09:52:47 +0100 (CET) Message-ID: <5AB36EDF.5060204@omnilan.de> Date: Thu, 22 Mar 2018 09:52:47 +0100 From: "H. Schmalzbauer, OmniLAN Support" Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: FreeBSD virtualization Subject: r327065+r330615 MFC candidate before 11.2? Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]); Thu, 22 Mar 2018 09:52:47 +0100 (CET) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: ; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Mar 2018 08:52:49 -0000 Hello, I have these two on my local stable/11 patchlist (named vmm_intel_vmx-RecognizeVirtIRQduringHALT-r327065+r330615.patch). https://svnweb.freebsd.org/base?view=revision&revision=327065 https://svnweb.freebsd.org/base?view=revision&revision=330615 I guess there are reasons if hackers don't flag for MFC. But maybe that one was a 11.2 candidate? Thanks, -harry From owner-freebsd-virtualization@freebsd.org Fri Mar 23 08:56:40 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 73CF4F5565A for ; Fri, 23 Mar 2018 08:56:40 +0000 (UTC) (envelope-from jmaloney@ixsystems.com) Received: from mail-qt0-x22f.google.com (mail-qt0-x22f.google.com [IPv6:2607:f8b0:400d:c0d::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0FC75680B2 for ; Fri, 23 Mar 2018 08:56:39 +0000 (UTC) (envelope-from jmaloney@ixsystems.com) Received: by mail-qt0-x22f.google.com with SMTP id g5so656090qth.7 for ; Fri, 23 Mar 2018 01:56:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ixsystems-com.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=dDBiRlqKHM/fFoHLWSYEonARB9KLP/nPx3xxpRaTn+o=; b=MGDyZY5TNzUHWOOVCdQgwLDeRKwSK9M013x+bqK7jItM6aM4DLCzorfydZfi/Jostu 0IkHORm71yIy5SGdkzwvmIaK9Eh6Re6/dZVnx0GWX2nT9LOeUU6P8TZAC2IrsRQBGi23 yTMYZ28yFNxfqHjNTbjMl1jFr1YTys58PYQFDYBVtoT254codvZXD6scyQxW+RAbHTLv Wugfb6ntgAAdCIqqP+qfuwWypM5AvAOyW9/p6RhvChLUGKSD+o+9gpMzAGsqL0shNGTQ KfFkGPlOtn3Z55RwWhR5l6TplCDpc8MU2OY6iFtJVyCsbtJiozYCTHclbCLayxI5+KYe 3Dyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=dDBiRlqKHM/fFoHLWSYEonARB9KLP/nPx3xxpRaTn+o=; b=qQqkQ2isRC4HavEe0YIbkBT+SqMJxVIHavbb9LbAzd7bSvV6Wg25Jp2MFKZ9OkDVVY LZKQsleKUikM3wUH/2h4zaarCT3bweIcprVZwnlnDpzt7UkuprwhhuMlrjK+flKjM2Eh 32197xsUXThaaEIMZmiPjzvkoeCbiqZG+VLWQN+cG9ZUtV2HeuhzUsdwAXiegioC3uIc J7tpJTlyRxDVMx5RX/WRGbiSFEp2xlvo5Vm7jHGGy1DJ7BFAfwVDmsaVeD6HRqkxc5z8 QQ7tbXYiHvSrl8ehFSZ8yEh+VQicn4z6fhYv0VkTZhm1Szu3HYCVcI53PYW7zOaFLxvl uW1A== X-Gm-Message-State: AElRT7EIXw1yXxZ2VBVLl4fnr3TXLOiXS/Ki1YVAcdhNwJrGQUgCzAbN ECwo+90shZuqFAH/3aj6Gnik9HRYui4/ob+SsoS+Nw== X-Google-Smtp-Source: AIpwx48rSUWcv8i0q8REOQHqLh15oOJntM/lRGw/hupbWI5dJ04vE2jVRNPICdFbEaLxbPe81QQKXdcacoWmeKUVXh8= X-Received: by 10.237.35.137 with SMTP id j9mr1878865qtc.181.1521795399152; Fri, 23 Mar 2018 01:56:39 -0700 (PDT) MIME-Version: 1.0 Received: by 10.237.63.119 with HTTP; Fri, 23 Mar 2018 01:56:38 -0700 (PDT) In-Reply-To: References: From: Joe Maloney Date: Fri, 23 Mar 2018 04:56:38 -0400 Message-ID: Subject: Re: Issue encountered booting FreeBSD STABLE and CURRENT snapshots with EFI To: "araujo@freebsd.org" Cc: Kyle Evans , "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 08:56:40 -0000 We narrowed the issue down to how vm-bhyve attaches a null.iso when starting the VM. For now a hack like so to vm-bhyve can get around the issue: https://github.com/araujobsd/vm-bhyve/commit/29db2d6c6ec4a29578dc111903107f25a78cdf82 This may simply be an issue with vm-bhyve attaching an invalid iso image to the VM, and I would conclude it is odd that it does so. Or there may still be an issue that affects even the latest 03-22-18 snapshots for example if other media is present when booting with bhyve, or natively. I would need to do some more testing at a later date to confirm but just wanted to pass along what was discovered to be the root cause thus far. On Friday, March 16, 2018, Marcelo Araujo wrote: > 2018-03-16 9:55 GMT+08:00 Kyle Evans : > > > On Thu, Mar 15, 2018 at 8:46 PM, Marcelo Araujo > > > wrote: > > > > > > > > > 2018-03-16 7:07 GMT+08:00 Kyle Evans : > > >> > > >> On Thu, Mar 15, 2018 at 5:01 PM, Kyle Evans > wrote: > > >> > On Thu, Mar 15, 2018 at 4:09 PM, Peter Grehan > > >> > wrote: > > >> >>> I believe the problem may have been introduced with this commit: > > > > > >> >>> https://svnweb.freebsd.org/base/stable/?view=log&pathrev=329114 > > >> >> > > >> >> Any chance of being able to work out where in that list of commits > > in > > >> >> CURRENT the loader stopped working ? > > >> >> > > >> > > > >> > Indeed- if you could work out the exact commit in that range from > head > > >> > that caused it, I wouldn't think it to be a tough fix. After tonight > > >> > I'm out until Sunday, but should have time Sunday or Monday to try > and > > >> > diagnose it further. > > >> > > >> Can one of you try this with boot1.efi+loader.efi built from today's > > >> head stand/? I'm not sure what I'm expecting here since these are > > >> among my first times trying bhyve, but this is what I'm seeing now > > >> (vs. from the mentioned head snapshot where I noted similar behavior > > >> as originally mentioned): > > >> > > >> 1.) Get to loader.efi, menu is good > > >> 2.) Break into loader prompt > > >> 3.) `lsdev`- pager is restricted to the line the prompt is on, so the > > >> output is useless > > >> 4.) `boot` > > >> 5.) "Unhandled ps2 mouse command 0xe1" > > >> > > >> At this point, the boot looks screwed until I VNC into it- it booted > > >> fine here, but the console stopped working after the kernel handoff. > > >> > > >> Thanks, > > >> > > >> Kyle Evans > > >> _______________________________________________ > > >> freebsd-virtualization@freebsd.org mailing list > > >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > >> To unsubscribe, send any mail to > > >> "freebsd-virtualization-unsubscribe@freebsd.org" > > > > > > > > > Hi Kyle, > > > > > > I will do that today and report back as soon as I have something. > > > > > > > Thanks! If it's still failing, I think capturing the output of "lsdev" > > and "show currdev" prior to a failed boot might be most helpful just > > to make sure there's not something obviously sketchy happening. > > > > Hi, > > I think we had two bad snapshots! > > I just finished the tests with HEAD and 11-STABLE latest snapshots: > 1) HEAD: FreeBSD-12.0-CURRENT-amd64-20180315-r331001-disc1.iso > 2) Stable: FreeBSD-11.1-STABLE-amd64-20180315-r330998-bootonly.iso > > I have installed it using ZFS and tested using AHCI and virtio-blk. > > So everything worked fine. > Seems those broken snapshots have missed some commits related with EFI. > > > Thank you all. > -- > > -- > Marcelo Araujo (__)araujo@FreeBSD.org > \\\'',)http://www.FreeBSD.org \/ \ ^ > Power To Server. .\. /_) > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to " > freebsd-virtualization-unsubscribe@freebsd.org" > -- Joe Maloney QA Manager / iXsystems Enterprise Storage & Servers Driven By Open Source From owner-freebsd-virtualization@freebsd.org Fri Mar 23 13:06:03 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E5B5FF6AC44 for ; Fri, 23 Mar 2018 13:06:02 +0000 (UTC) (envelope-from byond.lenox@gmail.com) Received: from mail-lf0-f44.google.com (mail-lf0-f44.google.com [209.85.215.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5C7BA73127; Fri, 23 Mar 2018 13:06:02 +0000 (UTC) (envelope-from byond.lenox@gmail.com) Received: by mail-lf0-f44.google.com with SMTP id g203-v6so18164682lfg.11; Fri, 23 Mar 2018 06:06:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=peKdfDljdbsYupwY+o9qGc9rec8oXLIjiviTPJDMQAs=; b=Y2CPe5/cS5qhGV+cG25OCxd0Sc+sfWRaw5I0OydVG0WIo83Bi0U9/ZI1eK4SGtWnos jkdc6VzCOxyWWywBO6ALgvVvcZIuO1XDKxcPOJwGfjSeAasp+wdHq8vxqXssgHWaSTcU QSedW1/Q98MjI6bdAO/Nusxifvt3Id8qNAAF8HnTyA4ZyOx/P/aHwCw6+RgzghCYIAZw 7AAjbRSEvUokXMh8TvYfr1TqhIYjcKRrMT1uyjS1+MOHQ8Uuq1kD4TEVaS03DeMK6+RL LXQ5d6XqyG2gU2lJhhbhv+Mv/xgo20OnPUXZs3JtKW84rqyG3tXJqtuVgFW7ZJ1ogCT8 dZ4Q== X-Gm-Message-State: AElRT7Eo6QcpK8stiNwdNxS7/Oxd+SgjTT5Sd3kXHIgYCV8DH8wb4Ijz hwkRKZqxS/DGeBnR/WnIgwjo8fUGRnA= X-Google-Smtp-Source: AG47ELs0x5ndZNANS6+prU1YIgXmxCQkiX/v6o/NF0E4RoQO/4eJ2WryZF5M3CIChxeKTbxV8Ow8ug== X-Received: by 10.46.134.205 with SMTP id n13mr19071335ljj.17.1521810354548; Fri, 23 Mar 2018 06:05:54 -0700 (PDT) Received: from mail-lf0-f47.google.com (mail-lf0-f47.google.com. [209.85.215.47]) by smtp.gmail.com with ESMTPSA id d2sm1859125lja.43.2018.03.23.06.05.54 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 23 Mar 2018 06:05:54 -0700 (PDT) Received: by mail-lf0-f47.google.com with SMTP id t132-v6so18202118lfe.2; Fri, 23 Mar 2018 06:05:54 -0700 (PDT) X-Received: by 2002:a19:114f:: with SMTP id g76-v6mr14928374lfi.108.1521810354088; Fri, 23 Mar 2018 06:05:54 -0700 (PDT) MIME-Version: 1.0 Received: by 10.46.129.90 with HTTP; Fri, 23 Mar 2018 06:05:33 -0700 (PDT) In-Reply-To: References: From: Kyle Evans Date: Fri, 23 Mar 2018 08:05:33 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Issue encountered booting FreeBSD STABLE and CURRENT snapshots with EFI To: Joe Maloney Cc: "araujo@freebsd.org" , "freebsd-virtualization@freebsd.org" , Warner Losh Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 13:06:03 -0000 On Fri, Mar 23, 2018 at 3:56 AM, Joe Maloney wrote: > We narrowed the issue down to how vm-bhyve attaches a null.iso when starting > the VM. > What exactly are the contents of this null.iso? It sounds like we're just ultimately choosing the wrong partition to boot with the null.iso attached. I'd be interested to see if a loader.efi built with D13784 [1] applied resolves this both replacing /boot/loader.efi initially then while installed on the ESP. Do note that I'm pretty sure we still had some small outstanding issues with D13784's loader.efi being installed as /boot/loader.efi (with console handling), so it may not be a good long-term solution, but this should hopefully land in the not-so-distant future. [1] https://reviews.freebsd.org/D13784 > For now a hack like so to vm-bhyve can get around the issue: > > https://github.com/araujobsd/vm-bhyve/commit/29db2d6c6ec4a29578dc111903107f25a78cdf82 > > This may simply be an issue with vm-bhyve attaching an invalid iso image to > the VM, and I would conclude it is odd that it does so. Or there may still > be an issue that affects even the latest 03-22-18 snapshots for example if > other media is present when booting with bhyve, or natively. I would need > to do some more testing at a later date to confirm but just wanted to pass > along what was discovered to be the root cause thus far. > > On Friday, March 16, 2018, Marcelo Araujo wrote: >> >> 2018-03-16 9:55 GMT+08:00 Kyle Evans : >> >> > On Thu, Mar 15, 2018 at 8:46 PM, Marcelo Araujo >> > >> > wrote: >> > > >> > > >> > > 2018-03-16 7:07 GMT+08:00 Kyle Evans : >> > >> >> > >> On Thu, Mar 15, 2018 at 5:01 PM, Kyle Evans >> > >> wrote: >> > >> > On Thu, Mar 15, 2018 at 4:09 PM, Peter Grehan >> > >> > wrote: >> > >> >>> I believe the problem may have been introduced with this commit: >> > >> >>> > > >> > >> >>> https://svnweb.freebsd.org/base/stable/?view=log&pathrev=329114 >> > >> >> >> > >> >> Any chance of being able to work out where in that list of >> > >> >> commits >> > in >> > >> >> CURRENT the loader stopped working ? >> > >> >> >> > >> > >> > >> > Indeed- if you could work out the exact commit in that range from >> > >> > head >> > >> > that caused it, I wouldn't think it to be a tough fix. After >> > >> > tonight >> > >> > I'm out until Sunday, but should have time Sunday or Monday to try >> > >> > and >> > >> > diagnose it further. >> > >> >> > >> Can one of you try this with boot1.efi+loader.efi built from today's >> > >> head stand/? I'm not sure what I'm expecting here since these are >> > >> among my first times trying bhyve, but this is what I'm seeing now >> > >> (vs. from the mentioned head snapshot where I noted similar behavior >> > >> as originally mentioned): >> > >> >> > >> 1.) Get to loader.efi, menu is good >> > >> 2.) Break into loader prompt >> > >> 3.) `lsdev`- pager is restricted to the line the prompt is on, so the >> > >> output is useless >> > >> 4.) `boot` >> > >> 5.) "Unhandled ps2 mouse command 0xe1" >> > >> >> > >> At this point, the boot looks screwed until I VNC into it- it booted >> > >> fine here, but the console stopped working after the kernel handoff. >> > >> >> > >> Thanks, >> > >> >> > >> Kyle Evans >> > >> _______________________________________________ >> > >> freebsd-virtualization@freebsd.org mailing list >> > >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> > >> To unsubscribe, send any mail to >> > >> "freebsd-virtualization-unsubscribe@freebsd.org" >> > > >> > > >> > > Hi Kyle, >> > > >> > > I will do that today and report back as soon as I have something. >> > > >> > >> > Thanks! If it's still failing, I think capturing the output of "lsdev" >> > and "show currdev" prior to a failed boot might be most helpful just >> > to make sure there's not something obviously sketchy happening. >> > >> >> Hi, >> >> I think we had two bad snapshots! >> >> I just finished the tests with HEAD and 11-STABLE latest snapshots: >> 1) HEAD: FreeBSD-12.0-CURRENT-amd64-20180315-r331001-disc1.iso >> 2) Stable: FreeBSD-11.1-STABLE-amd64-20180315-r330998-bootonly.iso >> >> I have installed it using ZFS and tested using AHCI and virtio-blk. >> >> So everything worked fine. >> Seems those broken snapshots have missed some commits related with EFI. >> >> >> Thank you all. >> -- >> >> -- >> Marcelo Araujo (__)araujo@FreeBSD.org >> \\\'',)http://www.FreeBSD.org \/ \ ^ >> Power To Server. .\. /_) >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to >> "freebsd-virtualization-unsubscribe@freebsd.org" > > > > -- > Joe Maloney > QA Manager / iXsystems > Enterprise Storage & Servers Driven By Open Source > From owner-freebsd-virtualization@freebsd.org Fri Mar 23 13:29:28 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 65DFBF6C61E for ; Fri, 23 Mar 2018 13:29:28 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from smtp-a.userve.net (smtp-outbound.userve.net [217.196.1.22]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.userve.net", Issuer "Go Daddy Secure Certificate Authority - G2" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0025A73EDA; Fri, 23 Mar 2018 13:29:27 +0000 (UTC) (envelope-from matt.churchyard@userve.net) Received: from owa.usd-group.com (owa.usd-group.com [217.196.1.2]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp-a.userve.net (Postfix) with ESMTPS id 8EFC1617; Fri, 23 Mar 2018 13:19:31 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=userve.net; s=201508; t=1521811171; bh=B09AF1i8AHFG3ADVhv0+yrnyhZuP9m5J+0U+LOOKPC4=; h=From:To:CC:Subject:Date:References:In-Reply-To; b=qnwP4NzMOoX/GXgobLW2fJudu4HPCYU7A+X9zEm/+SuSW8NmskDchmt4H5tiIGjG6 vcv8yu23sJAEMcNoJcZT3EC5V9aOYj/aaUtjHH3ut+mj9A8r4wRj+R+J0IXwaP7jlh bp0aLLB1g2yKwg8gWa47l1N1weTBTQ3kHt2Li+x0= Received: from SERVER.ad.usd-group.com (192.168.0.1) by SERVER.ad.usd-group.com (192.168.0.1) with Microsoft SMTP Server (TLS) id 15.0.847.32; Fri, 23 Mar 2018 13:19:31 +0000 Received: from SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9]) by SERVER.ad.usd-group.com ([fe80::b19d:892a:6fc7:1c9%12]) with mapi id 15.00.0847.030; Fri, 23 Mar 2018 13:19:30 +0000 From: Matt Churchyard To: Kyle Evans , Joe Maloney CC: Warner Losh , "freebsd-virtualization@freebsd.org" Subject: RE: Issue encountered booting FreeBSD STABLE and CURRENT snapshots with EFI Thread-Topic: Issue encountered booting FreeBSD STABLE and CURRENT snapshots with EFI Thread-Index: AQHTu99K027tu7e5CUWji7AupOElw6PRyzyAgAAAygCAAA5/gIAAEoAAgAAsWgCAAAKqAIAAKQMAgAtM6gCAAEWMgIAAARYg Date: Fri, 23 Mar 2018 13:19:30 +0000 Message-ID: <096c805a0aa84bf3bcff3256b9a22a14@SERVER.ad.usd-group.com> References: In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.0.10] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 13:29:28 -0000 -----Original Message----- From: owner-freebsd-virtualization@freebsd.org On Behalf Of Kyle Evans Sent: 23 March 2018 13:06 To: Joe Maloney Cc: Warner Losh ; freebsd-virtualization@freebsd.org Subject: Re: Issue encountered booting FreeBSD STABLE and CURRENT snapshots= with EFI On Fri, Mar 23, 2018 at 3:56 AM, Joe Maloney wrote= : > We narrowed the issue down to how vm-bhyve attaches a null.iso when=20 > starting the VM. > >What exactly are the contents of this null.iso? It sounds like we're just = ultimately choosing the wrong partition to boot with the null.iso attached.= I'd be interested to see if a loader.efi built with >D13784 [1] applied re= solves this both replacing /boot/loader.efi initially then while installed = on the ESP. >Do note that I'm pretty sure we still had some small outstanding issues wi= th D13784's loader.efi being installed as /boot/loader.efi (with console ha= ndling), so it may not be a good long-term solution, >but this should hopef= ully land in the not-so-distant future. >[1] https://reviews.freebsd.org/D13784 The original reason for null.iso was due to notes on the bhyve wiki https://wiki.freebsd.org/bhyve/Windows " The install.iso is only required for the first boot of Windows Server and= must be removed from the bhyve command after the first boot. Desktop editi= ons of Windows require that a null install.iso file remains and it can be c= reated with touch install.iso" At the time the UEFI support was primarily used for loading Windows guests,= and some desktop versions apparently needed an empty iso file to boot. I w= as not able to verify which guests needed this file and so it was supplied = to all, as it seemed to have no ill effects on the server versions of Windo= ws I was testing with. As mentioned above is this more an issue with the boot process? If bhyve at= some point allows an "empty" cd device to be attached, with the ability to= insert an iso on the fly, like many other hypervisors do, would that trigg= er the same issue? Matt > For now a hack like so to vm-bhyve can get around the issue: > > https://github.com/araujobsd/vm-bhyve/commit/29db2d6c6ec4a29578dc11190 > 3107f25a78cdf82 > > This may simply be an issue with vm-bhyve attaching an invalid iso=20 > image to the VM, and I would conclude it is odd that it does so. Or=20 > there may still be an issue that affects even the latest 03-22-18=20 > snapshots for example if other media is present when booting with=20 > bhyve, or natively. I would need to do some more testing at a later=20 > date to confirm but just wanted to pass along what was discovered to be t= he root cause thus far. > > On Friday, March 16, 2018, Marcelo Araujo wrote= : >> >> 2018-03-16 9:55 GMT+08:00 Kyle Evans : >> >> > On Thu, Mar 15, 2018 at 8:46 PM, Marcelo Araujo=20 >> > >> > wrote: >> > > >> > > >> > > 2018-03-16 7:07 GMT+08:00 Kyle Evans : >> > >> >> > >> On Thu, Mar 15, 2018 at 5:01 PM, Kyle Evans >> > >> wrote: >> > >> > On Thu, Mar 15, 2018 at 4:09 PM, Peter Grehan=20 >> > >> > >> > >> > wrote: >> > >> >>> I believe the problem may have been introduced with this commit= : >> > >> >>> > > >> > >> >>> https://svnweb.freebsd.org/base/stable/?view=3Dlog&pathrev=3D32= 9 >> > >> >>> 114 >> > >> >> >> > >> >> Any chance of being able to work out where in that list of=20 >> > >> >> commits >> > in >> > >> >> CURRENT the loader stopped working ? >> > >> >> >> > >> > >> > >> > Indeed- if you could work out the exact commit in that range=20 >> > >> > from head that caused it, I wouldn't think it to be a tough=20 >> > >> > fix. After tonight I'm out until Sunday, but should have time=20 >> > >> > Sunday or Monday to try and diagnose it further. >> > >> >> > >> Can one of you try this with boot1.efi+loader.efi built from=20 >> > >> today's head stand/? I'm not sure what I'm expecting here since=20 >> > >> these are among my first times trying bhyve, but this is what=20 >> > >> I'm seeing now (vs. from the mentioned head snapshot where I=20 >> > >> noted similar behavior as originally mentioned): >> > >> >> > >> 1.) Get to loader.efi, menu is good >> > >> 2.) Break into loader prompt >> > >> 3.) `lsdev`- pager is restricted to the line the prompt is on,=20 >> > >> so the output is useless >> > >> 4.) `boot` >> > >> 5.) "Unhandled ps2 mouse command 0xe1" >> > >> >> > >> At this point, the boot looks screwed until I VNC into it- it=20 >> > >> booted fine here, but the console stopped working after the kernel = handoff. >> > >> >> > >> Thanks, >> > >> >> > >> Kyle Evans >> > >> _______________________________________________ >> > >> freebsd-virtualization@freebsd.org mailing list=20 >> > >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualizatio >> > >> n To unsubscribe, send any mail to=20 >> > >> "freebsd-virtualization-unsubscribe@freebsd.org" >> > > >> > > >> > > Hi Kyle, >> > > >> > > I will do that today and report back as soon as I have something. >> > > >> > >> > Thanks! If it's still failing, I think capturing the output of "lsdev" >> > and "show currdev" prior to a failed boot might be most helpful=20 >> > just to make sure there's not something obviously sketchy happening. >> > >> >> Hi, >> >> I think we had two bad snapshots! >> >> I just finished the tests with HEAD and 11-STABLE latest snapshots: >> 1) HEAD: FreeBSD-12.0-CURRENT-amd64-20180315-r331001-disc1.iso >> 2) Stable: FreeBSD-11.1-STABLE-amd64-20180315-r330998-bootonly.iso >> >> I have installed it using ZFS and tested using AHCI and virtio-blk. >> >> So everything worked fine. >> Seems those broken snapshots have missed some commits related with EFI. >> >> >> Thank you all. >> -- >> >> -- >> Marcelo Araujo (__)araujo@FreeBSD.org >> \\\'',)http://www.FreeBSD.org \/ \ ^ >> Power To Server. .\. /_) >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list=20 >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to >> "freebsd-virtualization-unsubscribe@freebsd.org" > > > > -- > Joe Maloney > QA Manager / iXsystems > Enterprise Storage & Servers Driven By Open Source > _______________________________________________ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/m= ailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebs= d.org" From owner-freebsd-virtualization@freebsd.org Fri Mar 23 14:58:33 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B53BF4EA35 for ; Fri, 23 Mar 2018 14:58:33 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4ED3577FAB; Fri, 23 Mar 2018 14:58:31 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w2NEwMxS031549; Fri, 23 Mar 2018 07:58:22 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w2NEwMr7031548; Fri, 23 Mar 2018 07:58:22 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201803231458.w2NEwMr7031548@pdx.rh.CN85.dnsmgr.net> Subject: Re: Issue encountered booting FreeBSD STABLE and CURRENT snapshots with EFI In-Reply-To: To: Kyle Evans Date: Fri, 23 Mar 2018 07:58:22 -0700 (PDT) CC: Joe Maloney , Warner Losh , "freebsd-virtualization@freebsd.org" X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 14:58:33 -0000 > On Fri, Mar 23, 2018 at 3:56 AM, Joe Maloney wrote: > > We narrowed the issue down to how vm-bhyve attaches a null.iso when starting > > the VM. > > > > What exactly are the contents of this null.iso? It sounds like we're touch "null.iso" -rw-r--r-- 1 root wheel 0 Dec 3 22:55 /home/vmbhyve/.config/null.iso It is litterly a 0 byte file. This is just there to appease windows installer that you have a cd drive. You can test with bhyve yourself this problem by adding: -s 3:0,ahci-cd,${vm_dir}/.config/null.iso to your bhyve command. > just ultimately choosing the wrong partition to boot with the null.iso > attached. I'd be interested to see if a loader.efi built with D13784 > [1] applied resolves this both replacing /boot/loader.efi initially > then while installed on the ESP. I suspect it is new code unable to deal with a 0 byte device. There are some other test cases to run, like MBR's that have valid boot code signature but corrupt Partition tables, like especially they all 0 one, I have seen many bioses that attempt a divide by 0 when faced with this table, aggrivated if it is not really all 0, but if the active bit is set in a 0 valued slice. Robust code should also handle MBR entries that have END < START, and other nasties. Never make any assmption about the validity of the data in a MBR, even if it has a valid 0x55aa signature, as the partition table can easily be corrupted, some times intentionally. These can be hard cases to test for as many tools do not allow you to intentionally set bad values. > Do note that I'm pretty sure we still had some small outstanding > issues with D13784's loader.efi being installed as /boot/loader.efi > (with console handling), so it may not be a good long-term solution, > but this should hopefully land in the not-so-distant future. > > [1] https://reviews.freebsd.org/D13784 > > > For now a hack like so to vm-bhyve can get around the issue: > > > > https://github.com/araujobsd/vm-bhyve/commit/29db2d6c6ec4a29578dc111903107f25a78cdf82 > > > > This may simply be an issue with vm-bhyve attaching an invalid iso image to > > the VM, and I would conclude it is odd that it does so. Or there may still > > be an issue that affects even the latest 03-22-18 snapshots for example if > > other media is present when booting with bhyve, or natively. I would need > > to do some more testing at a later date to confirm but just wanted to pass > > along what was discovered to be the root cause thus far. > > > > On Friday, March 16, 2018, Marcelo Araujo wrote: > >> > >> 2018-03-16 9:55 GMT+08:00 Kyle Evans : > >> > >> > On Thu, Mar 15, 2018 at 8:46 PM, Marcelo Araujo > >> > > >> > wrote: > >> > > > >> > > > >> > > 2018-03-16 7:07 GMT+08:00 Kyle Evans : > >> > >> > >> > >> On Thu, Mar 15, 2018 at 5:01 PM, Kyle Evans > >> > >> wrote: > >> > >> > On Thu, Mar 15, 2018 at 4:09 PM, Peter Grehan > >> > >> > wrote: > >> > >> >>> I believe the problem may have been introduced with this commit: > >> > >> >>> > > > >> > >> >>> https://svnweb.freebsd.org/base/stable/?view=log&pathrev=329114 > >> > >> >> > >> > >> >> Any chance of being able to work out where in that list of > >> > >> >> commits > >> > in > >> > >> >> CURRENT the loader stopped working ? > >> > >> >> > >> > >> > > >> > >> > Indeed- if you could work out the exact commit in that range from > >> > >> > head > >> > >> > that caused it, I wouldn't think it to be a tough fix. After > >> > >> > tonight > >> > >> > I'm out until Sunday, but should have time Sunday or Monday to try > >> > >> > and > >> > >> > diagnose it further. > >> > >> > >> > >> Can one of you try this with boot1.efi+loader.efi built from today's > >> > >> head stand/? I'm not sure what I'm expecting here since these are > >> > >> among my first times trying bhyve, but this is what I'm seeing now > >> > >> (vs. from the mentioned head snapshot where I noted similar behavior > >> > >> as originally mentioned): > >> > >> > >> > >> 1.) Get to loader.efi, menu is good > >> > >> 2.) Break into loader prompt > >> > >> 3.) `lsdev`- pager is restricted to the line the prompt is on, so the > >> > >> output is useless > >> > >> 4.) `boot` > >> > >> 5.) "Unhandled ps2 mouse command 0xe1" > >> > >> > >> > >> At this point, the boot looks screwed until I VNC into it- it booted > >> > >> fine here, but the console stopped working after the kernel handoff. > >> > >> > >> > >> Thanks, > >> > >> > >> > >> Kyle Evans > >> > >> _______________________________________________ > >> > >> freebsd-virtualization@freebsd.org mailing list > >> > >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > >> > >> To unsubscribe, send any mail to > >> > >> "freebsd-virtualization-unsubscribe@freebsd.org" > >> > > > >> > > > >> > > Hi Kyle, > >> > > > >> > > I will do that today and report back as soon as I have something. > >> > > > >> > > >> > Thanks! If it's still failing, I think capturing the output of "lsdev" > >> > and "show currdev" prior to a failed boot might be most helpful just > >> > to make sure there's not something obviously sketchy happening. > >> > > >> > >> Hi, > >> > >> I think we had two bad snapshots! > >> > >> I just finished the tests with HEAD and 11-STABLE latest snapshots: > >> 1) HEAD: FreeBSD-12.0-CURRENT-amd64-20180315-r331001-disc1.iso > >> 2) Stable: FreeBSD-11.1-STABLE-amd64-20180315-r330998-bootonly.iso > >> > >> I have installed it using ZFS and tested using AHCI and virtio-blk. > >> > >> So everything worked fine. > >> Seems those broken snapshots have missed some commits related with EFI. > >> > >> > >> Thank you all. > >> -- > >> > >> -- > >> Marcelo Araujo (__)araujo@FreeBSD.org > >> \\\'',)http://www.FreeBSD.org \/ \ ^ > >> Power To Server. .\. /_) > >> _______________________________________________ > >> freebsd-virtualization@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > >> To unsubscribe, send any mail to > >> "freebsd-virtualization-unsubscribe@freebsd.org" > > > > > > > > -- > > Joe Maloney > > QA Manager / iXsystems > > Enterprise Storage & Servers Driven By Open Source > > > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" > -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Fri Mar 23 15:30:35 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A0B1F517CD for ; Fri, 23 Mar 2018 15:30:35 +0000 (UTC) (envelope-from nospam@ofloo.net) Received: from loki.ofloo.net (unknown [IPv6:2001:470:7acd:0:1:1:0:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "loki.ofloo.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 16C8E799B0 for ; Fri, 23 Mar 2018 15:30:34 +0000 (UTC) (envelope-from nospam@ofloo.net) Received: from [IPv6:fc03:1337:dead:cafe:247f:77bc:e37d:82ad] ([IPv6:fc03:1337:dead:cafe:247f:77bc:e37d:82ad]) by loki.ofloo.net (8.15.2/8.15.2) with ESMTPS id w2NFTb74038269 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Fri, 23 Mar 2018 15:29:43 GMT (envelope-from nospam@ofloo.net) To: freebsd-virtualization@freebsd.org From: Wouter Snels Subject: bhyve pci passthrough usb controller Message-ID: <686906fa-b531-455b-713a-796361d0741d@ofloo.net> Date: Fri, 23 Mar 2018 16:29:31 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="t920u2XkOZwgHAMsAVQlk9l9MBzLDBw8w" X-Spam-Status: No, score=-1.8 required=5.0 tests=ALL_TRUSTED,BAYES_00, HTML_MESSAGE,RDNS_NONE,SPF_PASS,TW_EV,TW_SV,TW_TD,TW_XH autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on loki.ofloo.net X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 15:30:35 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --t920u2XkOZwgHAMsAVQlk9l9MBzLDBw8w Content-Type: multipart/mixed; boundary="TlkQYThvklOLoToqeA1wi7vQnJSQadnSK"; protected-headers="v1" From: Wouter Snels To: freebsd-virtualization@freebsd.org Message-ID: <686906fa-b531-455b-713a-796361d0741d@ofloo.net> Subject: bhyve pci passthrough usb controller --TlkQYThvklOLoToqeA1wi7vQnJSQadnSK Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable |Hi, .. I'm trying to pci passhthrough a usb controller to a bhyve windows 7 virtualization. When I start the virtual machine it freezes the entire system and this shows up on the console. To be clear the system runs without any problems without the pci passthrough. | ||(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 e0 ff 40 40 29 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: s/n ZA4292G6 detached (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Poll timeout on slot 28 port 0 ahcich0: is 20000000 cs 10000000 ss 00000000 rs 10000000 tfd 1d0 serr 00000800 cmd 00005c17 (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted (ada0:ahcich0:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 40 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Poll timeout on slot 30 port 0 ahcich0: is 20000000 cs 40000000 ss 00000000 rs 40000000 tfd 1d0 serr 00000800 cmd 00005e17 (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted (ada0:ahcich0:0:0:0): SETFEATURES ENABLE WCACHE. ACB: ef 02 00 00 00 40 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error (ada0:ahcich0:0:0:0): Periph destroyed em0: Watchdog timeout Queue[0]-- resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 428, hw tdt =3D 471 em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 981 em0: Tx Descriptors avail failure =3D 0 em0: RX= Queue 0 ------ em0: hw rdh =3D 912, hw rdt =3D 908 em0: RX discarded pack= ets =3D 0 em0: RX Next to Check =3D 909 em0: RX Next to Refresh =3D 908 xl0: watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D 5= 8 em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 966 em= 0: Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh =3D= 0, hw rdt =3D 1023 em0: RX discarded packets =3D 0 em0: RX Next to Check =3D= 0 em0: RX Next to Refresh =3D 1023 em0: Watchdog timeout Queue[0]-- resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D 58 em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 966 em0: Tx Descriptors avail failure =3D 0 em0: RX= Queue 0 ------ em0: hw rdh =3D 0, hw rdt =3D 1023 em0: RX discarded packe= ts =3D 0 em0: RX Next to Check =3D 0 em0: RX Next to Refresh =3D 1023 xl0: watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D 5= 4 em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 970 em= 0: Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh =3D= 0, hw rdt =3D 1023 em0: RX discarded packets =3D 0include GENERIC ident OFL options IPFIREWALL options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options CONSPEED=3D115200 em0: RX Next to Check =3D 0 em0: RX Next to Refresh =3D= 1023 |||| |||I've recompiled the kernel without the xhci driver. I've tested both usb controllers. Both "|||||Sunrise Point-H USB 3.0 xHCI Controller|"||||| and "|||uPD720202 USB 3.0 Host Controller|", both showed the same error on the console screen.| # uname FreeBSD 11.1-STABLE #1 r331092M # custom kernel include GENERIC ident OFL options IPFIREWALL options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options CONSPEED=3D115200 * note commented "device xhci" in generic # svn info ||= |Path: . Working Copy Root Path: /usr/src URL: https://svn0.eu.freebsd.org/base/stable/11 Relative URL: ^/stable/11 Repository Root: https://svn0.eu.freebsd.org/base Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind: directory Schedule: normal Last Changed Author: eadler Last Changed Rev: 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018) |# vm passthru DEVICE BHYVE ID READY DESCRIPTION hostb0 0/0/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers pcib1 0/1/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) vgapci0 0/2/0 No HD Graphics 530 |||ppt0| 0/20/0 Yes Sun= rise Point-H USB 3.0 xHCI Controller none0 0/20/2 No Sunrise Point-H Thermal subsystem none1 0/22/0 No Sunrise Point-H CSME HECI ahci0 0/23/0 No Sunrise Point-H SATA controller [AHCI mode] pcib2 0/28/0 No Sunrise Point-H PCI Express Root Port pcib4 0/28/6 No Sunrise Point-H PCI Express Root Port isab0 0/31/0 No Sunrise Point-H LPC Controller none2 0/31/2 No Sunrise Point-H PMC hdac0 0/31/3 No Sunrise Point-H HD Audio none3 0/31/4 No Sunrise Point-H SMBus em0 0/31/6 No Ethernet Connection (2) I219-V pcib3 2/0/0 No XIO2001 PCI Express-to-PCI Bridge xl0 3/0/0 No 3c905B 100BaseTX [Cyclone] ppt1 4/0/0 Yes uPD720202 USB 3.0 Host Controller ||ppt1@pci0:4:0:0: class=3D0x0c0330= card=3D0x00151912 chip=3D0x00151912 rev=3D0x02 hdr=3D0x00 vendor =3D 'Renesas Technology Corp.' device =3D 'uPD720202 US= B 3.0 Host Controller' class =3D serial bus subclass =3D USB |||ppt|0@pci0:0:20= :0: class=3D0x0c0330 card=3D0x06ba1028 chip=3D0xa12f8086 rev=3D0x31 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Sunrise = Point-H USB 3.0 xHCI Controller' class =3D serial bus subclass =3D USB # grep -i pptdevfs loader.conf ||pptdevs=3D"0/20/0 4/0/0" # vm configure ||passthru= 0=3D"0/20/0"| |# vm configure ||passthru0=3D"4/0/0"| ||Any suggestions on what I'm doing wrong?|| =C2=A0 Regards, Ofloo |||||| --TlkQYThvklOLoToqeA1wi7vQnJSQadnSK-- --t920u2XkOZwgHAMsAVQlk9l9MBzLDBw8w Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJatR1bAAoJENWumses7+9mzDEQALA5HHUmC0/3IrJ+SzhUjlQk uLb0KGZdeK592J/2ERToMY8IN5WjndPDhuH4IOkGDSaNQhbUDYx0B3y7mKKHGygV vrT34hvcxaLYcuXu41vQBRxrRJB55CQJiGg0BKFkCoVknv2Opop6b0QawZrzHaH6 vGfOjXuMRIALrmvRK955kxWR9uIGoTanHJL8Hs4PGrIPL3LgJoefgvrkzXV/1DZF vMqZdpiaJqI5Aswj9ifrzpQonXxXtiSVi0XhyOAfnq0tO2mNWHjhDyhP1tTD1kQK ZAcBYk/TNqdjC0QY1Vvn3kpuL9qL0ElvhZimzRCEchg+UME7jUwTX/65y6gk5jbL 8c1ggHdOyDiy7lOn8BgMfJbBSVF5OrLh2rfsIS/so+tB1dDEmg+rdGVme9nUtKcE qtEG95S05jgrGG7UomhsfKgFAIDplJMdl8znXIJ5DutaZqGEvZvPpGidWfAci1kl esbw+SHSI9xmr39WJWqTj2jEGeqcjHpMNQPQOdUnGBx3h7Vp1pGiLvLotBFIin0J yM5kz62E9nNIewgbgy4L6DV0YJMc5mfvpmOUe/KC3oO53jsmUvXXcaOcgjt3PHwm Hd1Ry2LHvRoS7OLhsEGgNYWdm23x3BZ1djGxIsO9JwDC0gWqSj685bvfgF27NOuD aALqiWrkoNPChoNkOHgp =I8od -----END PGP SIGNATURE----- --t920u2XkOZwgHAMsAVQlk9l9MBzLDBw8w-- From owner-freebsd-virtualization@freebsd.org Fri Mar 23 15:41:16 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E4C28F527CD for ; Fri, 23 Mar 2018 15:41:15 +0000 (UTC) (envelope-from nospam@ofloo.net) Received: from loki.ofloo.net (unknown [IPv6:2001:470:7acd:0:1:1:0:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "loki.ofloo.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 35A557A5F2 for ; Fri, 23 Mar 2018 15:41:15 +0000 (UTC) (envelope-from nospam@ofloo.net) Received: from [IPv6:fc03:1337:dead:cafe:247f:77bc:e37d:82ad] ([IPv6:fc03:1337:dead:cafe:247f:77bc:e37d:82ad]) by loki.ofloo.net (8.15.2/8.15.2) with ESMTPS id w2NFeGjc038646 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Fri, 23 Mar 2018 15:40:22 GMT (envelope-from nospam@ofloo.net) To: freebsd-virtualization@freebsd.org From: Wouter Snels Subject: bhyve pci passthrough usb controller Message-ID: <5bd3c8b6-0579-0269-1ada-ab6f0a55ce8f@ofloo.net> Date: Fri, 23 Mar 2018 16:40:09 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="tmFcbYRTJUG6F4FbMt7xBElVpZTgU2QE1" X-Spam-Status: No, score=-1.8 required=5.0 tests=ALL_TRUSTED,BAYES_00, RDNS_NONE,SPF_PASS,TW_EV,TW_SV,TW_TD,TW_XH autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on loki.ofloo.net X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 15:41:16 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --tmFcbYRTJUG6F4FbMt7xBElVpZTgU2QE1 Content-Type: multipart/mixed; boundary="XfA8H9jUh5GZqRi88ModXSknxHWT1TF4r"; protected-headers="v1" From: Wouter Snels To: freebsd-virtualization@freebsd.org Message-ID: <5bd3c8b6-0579-0269-1ada-ab6f0a55ce8f@ofloo.net> Subject: bhyve pci passthrough usb controller --XfA8H9jUh5GZqRi88ModXSknxHWT1TF4r Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Content-Language: nl-BE |Hi, .. I'm trying to pci passhthrough a usb controller to a bhyve windows 7 virtualization. When I start the virtual machine it freezes the entire system and this shows up on the console. To be clear the system runs without any problems without the pci passthrough. | ||(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 e0 ff 40 40 29 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: s/n ZA4292G6 detached (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Poll timeout on slot 28 port 0 ahcich0: is 20000000 cs 10000000 ss 00000000 rs 10000000 tfd 1d0 serr 00000800 cmd 00005c17 (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted (ada0:ahcich0:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 40 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Poll timeout on slot 30 port 0 ahcich0: is 20000000 cs 40000000 ss 00000000 rs 40000000 tfd 1d0 serr 00000800 cmd 00005e17 (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted (ada0:ahcich0:0:0:0): SETFEATURES ENABLE WCACHE. ACB: ef 02 00 00 00 40 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error (ada0:ahcich0:0:0:0): Periph destroyed em0: Watchdog timeout Queue[0]-- resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 428, hw tdt =3D 471 em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 981 em0: Tx Descriptors avail failure =3D 0 em0: RX= Queue 0 ------ em0: hw rdh =3D 912, hw rdt =3D 908 em0: RX discarded pack= ets =3D 0 em0: RX Next to Check =3D 909 em0: RX Next to Refresh =3D 908 xl0: watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D 5= 8 em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 966 em= 0: Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh =3D= 0, hw rdt =3D 1023 em0: RX discarded packets =3D 0 em0: RX Next to Check =3D= 0 em0: RX Next to Refresh =3D 1023 em0: Watchdog timeout Queue[0]-- resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D 58 em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 966 em0: Tx Descriptors avail failure =3D 0 em0: RX= Queue 0 ------ em0: hw rdh =3D 0, hw rdt =3D 1023 em0: RX discarded packe= ts =3D 0 em0: RX Next to Check =3D 0 em0: RX Next to Refresh =3D 1023 xl0: watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D 5= 4 em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 970 em= 0: Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh =3D= 0, hw rdt =3D 1023 em0: RX discarded packets =3D 0include GENERIC ident OFL options IPFIREWALL|||||| |Hi, .. I'm trying to pci passhthrough a usb controller to a bhyve windows 7 virtualization. When I start the virtual machine it freezes the entire system and this shows up on the console. To be clear the system runs without any problems without the pci passthrough. | ||(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 e0 ff 40 40 29 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: s/n ZA4292G6 detached (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Poll timeout on slot 28 port 0 ahcich0: is 20000000 cs 10000000 ss 00000000 rs 10000000 tfd 1d0 serr 00000800 cmd 00005c17 (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted (ada0:ahcich0:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 40 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Poll timeout on slot 30 port 0 ahcich0: is 20000000 cs 40000000 ss 00000000 rs 40000000 tfd 1d0 serr 00000800 cmd 00005e17 (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted (ada0:ahcich0:0:0:0): SETFEATURES ENABLE WCACHE. ACB: ef 02 00 00 00 40 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error (ada0:ahcich0:0:0:0): Periph destroyed em0: Watchdog timeout Queue[0]-- resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 428, hw tdt =3D 471 em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 981 em0: Tx Descriptors avail failure =3D 0 em0: RX= Queue 0 ------ em0: hw rdh =3D 912, hw rdt =3D 908 em0: RX discarded pack= ets =3D 0 em0: RX Next to Check =3D 909 em0: RX Next to Refresh =3D 908 xl0: watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D 5= 8 em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 966 em= 0: Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh =3D= 0, hw rdt =3D 1023 em0: RX discarded packets =3D 0 em0: RX Next to Check =3D= 0 em0: RX Next to Refresh =3D 1023 em0: Watchdog timeout Queue[0]-- resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D 58 em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 966 em0: Tx Descriptors avail failure =3D 0 em0: RX= Queue 0 ------ em0: hw rdh =3D 0, hw rdt =3D 1023 em0: RX discarded packe= ts =3D 0 em0: RX Next to Check =3D 0 em0: RX Next to Refresh =3D 1023 xl0: watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D 5= 4 em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 970 em= 0: Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh =3D= 0, hw rdt =3D 1023 em0: RX discarded packets =3D 0include GENERIC ident OFL options IPFIREWALL options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options CONSPEED=3D115200 em0: RX Next to Check =3D 0 em0: RX Next to Refresh =3D= 1023 |||| |||I've recompiled the kernel without the xhci driver. I've tested both usb controllers. Both "|||||Sunrise Point-H USB 3.0 xHCI Controller|"||||| and "|||uPD720202 USB 3.0 Host Controller|", both showed the same error on the console screen.| # uname FreeBSD 11.1-STABLE #1 r331092M # custom kernel include GENERIC ident OFL options IPFIREWALL options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options CONSPEED=3D115200 * note commented "device xhci" in generic # svn info ||= |Path: . Working Copy Root Path: /usr/src URL: https://svn0.eu.freebsd.org/base/stable/11 Relative URL: ^/stable/11 Repository Root: https://svn0.eu.freebsd.org/base Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind: directory Schedule: normal Last Changed Author: eadler Last Changed Rev: 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018) |# vm passthru DEVICE BHYVE ID READY DESCRIPTION hostb0 0/0/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers pcib1 0/1/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) vgapci0 0/2/0 No HD Graphics 530 |||ppt0| 0/20/0 Yes Sun= rise Point-H USB 3.0 xHCI Controller none0 0/20/2 No Sunrise Point-H Thermal subsystem none1 0/22/0 No Sunrise Point-H CSME HECI ahci0 0/23/0 No Sunrise Point-H SATA controller [AHCI mode] pcib2 0/28/0 No Sunrise Point-H PCI Express Root Port pcib4 0/28/6 No Sunrise Point-H PCI Express Root Port isab0 0/31/0 No Sunrise Point-H LPC Controller none2 0/31/2 No Sunrise Point-H PMC hdac0 0/31/3 No Sunrise Point-H HD Audio none3 0/31/4 No Sunrise Point-H SMBus em0 0/31/6 No Ethernet Connection (2) I219-V pcib3 2/0/0 No XIO2001 PCI Express-to-PCI Bridge xl0 3/0/0 No 3c905B 100BaseTX [Cyclone] ppt1 4/0/0 Yes uPD720202 USB 3.0 Host Controller ||ppt1@pci0:4:0:0: class=3D0x0c0330= card=3D0x00151912 chip=3D0x00151912 rev=3D0x02 hdr=3D0x00 vendor =3D 'Renesas Technology Corp.' device =3D 'uPD720202 US= B 3.0 Host Controller' class =3D serial bus subclass =3D USB |||ppt|0@pci0:0:20= :0: class=3D0x0c0330 card=3D0x06ba1028 chip=3D0xa12f8086 rev=3D0x31 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Sunrise = Point-H USB 3.0 xHCI Controller' class =3D serial bus subclass =3D USB # grep -i pptdevfs loader.conf ||pptdevs=3D"0/20/0 4/0/0" # vm configure ||passthru= 0=3D"0/20/0"| |# vm configure ||passthru0=3D"4/0/0"| ||Any suggestions on what I'm doing wrong?|| =C2=A0 Regards, Ofloo |||||| ||options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT options= DUMMYNET options IPDIVERT options CONSPEED=3D115200 em0: RX Next to Check= =3D 0 em0: RX Next to Refresh =3D 1023 |||| |||I've recompiled the kernel without the xhci driver. I've tested both usb controllers. Both "|||||Sunrise Point-H USB 3.0 xHCI Controller|"||||| and "|||uPD720202 USB 3.0 Host Controller|", both showed the same error on the console screen.| # uname FreeBSD 11.1-STABLE #1 r331092M # custom kernel include GENERIC ident OFL options IPFIREWALL options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options CONSPEED=3D115200 * note commented "device xhci" in generic # svn info ||= |Path: . Working Copy Root Path: /usr/src URL: https://svn0.eu.freebsd.org/base/stable/11 Relative URL: ^/stable/11 Repository Root: https://svn0.eu.freebsd.org/base Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind: directory Schedule: normal Last Changed Author: eadler Last Changed Rev: 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018) |# vm passthru DEVICE BHYVE ID READY DESCRIPTION hostb0 0/0/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers pcib1 0/1/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) vgapci0 0/2/0 No HD Graphics 530 |||ppt0| 0/20/0 Yes Sun= rise Point-H USB 3.0 xHCI Controller none0 0/20/2 No Sunrise Point-H Thermal subsystem none1 0/22/0 No Sunrise Point-H CSME HECI ahci0 0/23/0 No Sunrise Point-H SATA controller [AHCI mode] pcib2 0/28/0 No Sunrise Point-H PCI Express Root Port pcib4 0/28/6 No Sunrise Point-H PCI Express Root Port isab0 0/31/0 No Sunrise Point-H LPC Controller none2 0/31/2 No Sunrise Point-H PMC hdac0 0/31/3 No Sunrise Point-H HD Audio none3 0/31/4 No Sunrise Point-H SMBus em0 0/31/6 No Ethernet Connection (2) I219-V pcib3 2/0/0 No XIO2001 PCI Express-to-PCI Bridge xl0 3/0/0 No 3c905B 100BaseTX [Cyclone] ppt1 4/0/0 Yes uPD720202 USB 3.0 Host Controller ||ppt1@pci0:4:0:0: class=3D0x0c0330= card=3D0x00151912 chip=3D0x00151912 rev=3D0x02 hdr=3D0x00 vendor =3D 'Renesas Technology Corp.' device =3D 'uPD720202 US= B 3.0 Host Controller' class =3D serial bus subclass =3D USB |||ppt|0@pci0:0:20= :0: class=3D0x0c0330 card=3D0x06ba1028 chip=3D0xa12f8086 rev=3D0x31 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Sunrise = Point-H USB 3.0 xHCI Controller' class =3D serial bus subclass =3D USB # grep -i pptdevfs loader.conf ||pptdevs=3D"0/20/0 4/0/0" # vm configure ||passthru= 0=3D"0/20/0"| |# vm configure ||passthru0=3D"4/0/0"| ||Any suggestions on what I'm doing wrong?|| edit: previous wrapping was all wrong. =C2=A0=C2=A0 Regards, Ofloo |||||| --XfA8H9jUh5GZqRi88ModXSknxHWT1TF4r-- --tmFcbYRTJUG6F4FbMt7xBElVpZTgU2QE1 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJatR/ZAAoJENWumses7+9mxacP/3RQB1+KEHu54SFeMlOFsDHe QeAFs7oB65F1gLNfYEMmPf4Mp0LyLf4IOKKXJXnw5KTix/0tEUGcg8iAYrVor3E9 UATkG6yNXN/bI+t+j2GNFCBkOM95Scqqtm5TWDf/y8dmSb2TDCuLnpXeBqYPtq4R f0TGKJrGqUOJ7fRWHr/AhixwHhVBHFgALka4/ph9pF9EUJsH5mzhgeUNbOhAKY2Z Bekc+bYhREV2niQUHVQYsgCkstT/xMAFFlNC2SDvLQCJgdmNVGPfRwXOpQoLpb4f lbCyRqUReu7d5a07ppqb9WWYQPIT1ezKy0JXhL+aDm+xdC6Tt+O9XW60gAQScnlY Cuy6MZ3TcHJWSMB2iEbR0DM1nx30yp3uAi9Xr+aqQruN3VgkfOoYKS9tgaXPdCZF hEWwbN1iVHI3No3qjDbg+rR+0mfmc8vDn7mSvoLGnHGPv80f74LOuGIXXkeHjX21 ydiPU7qefmlMxns3kuQF3F70K0zfUjpggxKpd9O0fYIQVgDhGb8/Lyuh8L4ugazD I7fF2nBt6Snwe329ty45BUyg3KPPpIRrGP3iOibnXg0CGVoS5JKQvbmtsHX1ygBW T7UIkkJUV66ihYLTPxObp7KAU+2bWOweE1zELa7yCAesCfvAd7OxVNviE/c3MaMY LNQqMsHYw8jA1PZeZ030 =+3Z4 -----END PGP SIGNATURE----- --tmFcbYRTJUG6F4FbMt7xBElVpZTgU2QE1-- From owner-freebsd-virtualization@freebsd.org Fri Mar 23 15:55:00 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2E4ABF53CF0 for ; Fri, 23 Mar 2018 15:55:00 +0000 (UTC) (envelope-from nospam@ofloo.net) Received: from loki.ofloo.net (unknown [IPv6:2001:470:7acd:0:1:1:0:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "loki.ofloo.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6E4737B4AE for ; Fri, 23 Mar 2018 15:54:59 +0000 (UTC) (envelope-from nospam@ofloo.net) Received: from [IPv6:fc03:1337:dead:cafe:247f:77bc:e37d:82ad] ([IPv6:fc03:1337:dead:cafe:247f:77bc:e37d:82ad]) by loki.ofloo.net (8.15.2/8.15.2) with ESMTPS id w2NFs09g039144 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Fri, 23 Mar 2018 15:54:08 GMT (envelope-from nospam@ofloo.net) To: freebsd-virtualization@freebsd.org From: Wouter Snels Subject: bhyve pci passthrough usb controller Message-ID: Date: Fri, 23 Mar 2018 16:53:54 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="61mbaeORzKUySXRBhLaLMGXAFrRm7FdKW" X-Spam-Status: No, score=-1.1 required=5.0 tests=ALL_TRUSTED,BAYES_00, HTML_MESSAGE,MIME_HTML_ONLY,RDNS_NONE,SPF_PASS,TW_EV,TW_SV,TW_TD,TW_XH autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on loki.ofloo.net X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 15:55:00 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --61mbaeORzKUySXRBhLaLMGXAFrRm7FdKW Content-Type: multipart/mixed; boundary="FT4sgTz9dPTjFiHF7o2NiUD3OKJFi9DfS"; protected-headers="v1" From: Wouter Snels To: freebsd-virtualization@freebsd.org Message-ID: Subject: bhyve pci passthrough usb controller --FT4sgTz9dPTjFiHF7o2NiUD3OKJFi9DfS Content-Language: nl-BE MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" --FT4sgTz9dPTjFiHF7o2NiUD3OKJFi9DfS-- --61mbaeORzKUySXRBhLaLMGXAFrRm7FdKW Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJatSMSAAoJENWumses7+9my5IQAN2idw0iRDZB37vZZ/j72wYo +o1VnTvfj/GS/TXuuYW/pstMIjBsI7AxIqGvbV+jDFJAKnfKqG8yhZUrVTuPRzhZ yGS2lcdKBaR2TfrdsMwIhtbhkaHsYgPS36uZxFeUaxgcRQe+688grU8D5RCXivk6 ldzK5w4/tR6LA2pMlIQwwKSmYtNVzF4UynADJdRkXVPdZvYN35dThp9YGrNNmWsb qD7Dpx1bQOy8vkbO3KbHAYpycHxfWsXPxnh9yWeDfraZfKn/rE2RMzRx56QAMlD5 qctix7/fh5SwrrPAzy4s05VVDyzXVltNSR71Im2MnD+rIstxWVB5rl+d7ZPC8mof NfBCiYtYXk1D3EsD3WOWHUvWMdXQ6f5EoBzkc0zw3cmpBEShPe7k0WrNvHgmi8yn lwyckwreDcworsX05OeD2bxa5KXGq3pWawwcUWBsH7hmOgFw2TMO4jQUmz4ttsAi xSmwIqTycK+wBfW3X/So3FrDTvQfTUdBrIOrePr/7N345i46urUlJl1PXwa4La8O We5jFBKHCwmJIH51T2Hb8KDAI7/K+0shegzaeV8Hu5L1J75U68gA60C01y2O3MwL Q/CcLu4yoD/5vUQLm5ri6wK4AGzUOQ+esmo+xbAr9sa7sQt+aiel7oYnkyIXxNZ2 CZxpFYvQ2CidNUAWaeH3 =JrUz -----END PGP SIGNATURE----- --61mbaeORzKUySXRBhLaLMGXAFrRm7FdKW-- From owner-freebsd-virtualization@freebsd.org Fri Mar 23 16:00:15 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 409D6F5453F for ; Fri, 23 Mar 2018 16:00:15 +0000 (UTC) (envelope-from nospam@ofloo.net) Received: from loki.ofloo.net (unknown [IPv6:2001:470:7acd:0:1:1:0:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "loki.ofloo.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AD9657B8AC for ; Fri, 23 Mar 2018 16:00:14 +0000 (UTC) (envelope-from nospam@ofloo.net) Received: from [IPv6:fc03:1337:dead:cafe:247f:77bc:e37d:82ad] ([IPv6:fc03:1337:dead:cafe:247f:77bc:e37d:82ad]) by loki.ofloo.net (8.15.2/8.15.2) with ESMTPS id w2NFxHPZ039360 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Fri, 23 Mar 2018 15:59:23 GMT (envelope-from nospam@ofloo.net) Subject: Re: bhyve pci passthrough usb controller To: freebsd-virtualization@freebsd.org References: From: Wouter Snels Message-ID: <4af9f7ee-d33e-86c4-d87c-fa8cf72bae33@ofloo.net> Date: Fri, 23 Mar 2018 16:59:11 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="DrLCvruDKXkqCJeI4idVkWRAolLVSftcr" X-Spam-Status: No, score=-1.8 required=5.0 tests=ALL_TRUSTED,BAYES_00, HTML_MESSAGE,RDNS_NONE,SPF_PASS,TW_EV,TW_SV,TW_TD,TW_XH autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on loki.ofloo.net X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 16:00:15 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --DrLCvruDKXkqCJeI4idVkWRAolLVSftcr Content-Type: multipart/mixed; boundary="omuoK4hhHKKCutSZohADwwphx9C4sEKty"; protected-headers="v1" From: Wouter Snels To: freebsd-virtualization@freebsd.org Message-ID: <4af9f7ee-d33e-86c4-d87c-fa8cf72bae33@ofloo.net> Subject: Re: bhyve pci passthrough usb controller References: In-Reply-To: --omuoK4hhHKKCutSZohADwwphx9C4sEKty Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable |Hi, .. I'm trying to pci passhthrough a usb controller to a bhyve windows 7 virtualization. When I start the virtual machine it freezes the entire system and this shows up on the console. To be clear the system runs without any problems without the pci passthrough. | ||(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 e0 ff 40 40 29 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ada0 at ahcich0 bus 0 scbus0 target 0 lun 0 ada0: s/n ZA4292G6 detached (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Poll timeout on slot 28 port 0 ahcich0: is 20000000 cs 10000000 ss 00000000 rs 10000000 tfd 1d0 serr 00000800 cmd 00005c17 (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted (ada0:ahcich0:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 40 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Poll timeout on slot 30 port 0 ahcich0: is 20000000 cs 40000000 ss 00000000 rs 40000000 tfd 1d0 serr 00000800 cmd 00005e17 (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted (ada0:ahcich0:0:0:0): SETFEATURES ENABLE WCACHE. ACB: ef 02 00 00 00 40 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error (ada0:ahcich0:0:0:0): Periph destroyed em0: Watchdog timeout Queue[0]-- resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 428, hw tdt =3D 471 em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 981 em0: Tx Descriptors avail failure =3D 0 em0: RX= Queue 0 ------ em0: hw rdh =3D 912, hw rdt =3D 908 em0: RX discarded pack= ets =3D 0 em0: RX Next to Check =3D 909 em0: RX Next to Refresh =3D 908 xl0: watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D 5= 8 em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 966 em= 0: Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh =3D= 0, hw rdt =3D 1023 em0: RX discarded packets =3D 0 em0: RX Next to Check =3D= 0 em0: RX Next to Refresh =3D 1023 em0: Watchdog timeout Queue[0]-- resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D 58 em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 966 em0: Tx Descriptors avail failure =3D 0 em0: RX= Queue 0 ------ em0: hw rdh =3D 0, hw rdt =3D 1023 em0: RX discarded packe= ts =3D 0 em0: RX Next to Check =3D 0 em0: RX Next to Refresh =3D 1023 xl0: watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D 5= 4 em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 970 em= 0: Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh =3D= 0, hw rdt =3D 1023 em0: RX discarded packets =3D 0include GENERIC ident OFL options IPFIREWALL options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options CONSPEED=3D115200 em0: RX Next to Check =3D 0 em0: RX Next to Refresh =3D= 1023 |||| |||I've recompiled the kernel without the xhci driver. I've tested both usb controllers. Both "|||||Sunrise Point-H USB 3.0 xHCI Controller|"||||| and "|||uPD720202 USB 3.0 Host Controller|", both showed the same error on the console screen.| # uname FreeBSD 11.1-STABLE #1 r331092M # custom kernel include GENERIC ident OFL options IPFIREWALL options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options CONSPEED=3D115200 * note commented "device xhci" in generic # svn info ||= |Path: . Working Copy Root Path: /usr/src URL: https://svn0.eu.freebsd.org/base/stable/11 Relative URL: ^/stable/11 Repository Root: https://svn0.eu.freebsd.org/base Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind: directory Schedule: normal Last Changed Author: eadler Last Changed Rev: 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018) |# vm passthru DEVICE BHYVE ID READY DESCRIPTION hostb0 0/0/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers pcib1 0/1/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) vgapci0 0/2/0 No HD Graphics 530 |||ppt0| 0/20/0 Yes Sun= rise Point-H USB 3.0 xHCI Controller none0 0/20/2 No Sunrise Point-H Thermal subsystem none1 0/22/0 No Sunrise Point-H CSME HECI ahci0 0/23/0 No Sunrise Point-H SATA controller [AHCI mode] pcib2 0/28/0 No Sunrise Point-H PCI Express Root Port pcib4 0/28/6 No Sunrise Point-H PCI Express Root Port isab0 0/31/0 No Sunrise Point-H LPC Controller none2 0/31/2 No Sunrise Point-H PMC hdac0 0/31/3 No Sunrise Point-H HD Audio none3 0/31/4 No Sunrise Point-H SMBus em0 0/31/6 No Ethernet Connection (2) I219-V pcib3 2/0/0 No XIO2001 PCI Express-to-PCI Bridge xl0 3/0/0 No 3c905B 100BaseTX [Cyclone] ppt1 4/0/0 Yes uPD720202 USB 3.0 Host Controller ||ppt1@pci0:4:0:0: class=3D0x0c0330= card=3D0x00151912 chip=3D0x00151912 rev=3D0x02 hdr=3D0x00 vendor =3D 'Renesas Technology Corp.'/mailman device =3D 'uPD7= 20202 USB 3.0 Host Controller' class =3D serial bus subclass =3D USB |||ppt|0@p= ci0:0:20:0: class=3D0x0c0330 card=3D0x06ba1028 chip=3D0xa12f8086 rev=3D0x31 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Sunrise = Point-H USB 3.0 xHCI Controller' class =3D serial bus subclass =3D USB # grep -i pptdevfs loader.conf ||pptdevs=3D"0/20/0 4/0/0" # vm configure ||passthru= 0=3D"0/20/0"| |# vm configure ||passthru0=3D"4/0/0"| ||Any suggestions on what I'm doing wrong?|| edit: sorry last attempt on getting the format of the email right. =C2=A0=C2=A0 Regards, Ofloo --omuoK4hhHKKCutSZohADwwphx9C4sEKty-- --DrLCvruDKXkqCJeI4idVkWRAolLVSftcr Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJatSRPAAoJENWumses7+9mgrIP/Rr0p1Hb4vN+bH2UuCZnVTYF rvCYK+M2zOgbpyOz2aDHaDII4GuPK6kNY7smSrUCMbowL22qI/9wBb/Zk0O3svkF j4EkAR8mENmr2BW2pWZrrd3wjfR97XbiFha03dwZ+0wOAAgImM2MiXng51szOdiw emhoIx7iCM5+qSeKIQUN+bSzI/Prr7/0ekE4L8WJXsKmymJ/3iBQyzm81G0o14vR EW0Y8cE5NTx1xTKz35ArGHbBad9jshtJPVkaY7YLXQb64ImCcxTyhb+owGC700EO UPUfdqslkdIFAdwSGnSHXtAojoJYH4lkIRYi9ihD2SDNNWb994zxU1Yo9N+26jJS vWDRo8hicb2VMmF7mBWy11Y2z1v5tad+U3aC5P9YSxLMQ0UoJirnRsV10R50EpSY Dq9KQ5xCvxdwSxtbY8itGUa8DqTZg7gqHWVR547myMCFe87ayqznAM67AVZy+lg5 Mxd3Up3WeGwKlNznQlSYMg8u/MAsKJeSxj3/45pTKwD/eXFBl7D7en6WmgxvFmce 3XE6lvyP5uEr9d0LqN365tgzTNCmfqC71x9cQx2KHdG3533Zn/6d6YPtMBHGYgx/ TR8DDOb8q+SXJBjI2Ade1RhDRKpa/SZosvNakxvMySitzE4EjSEB0ZPMi4Vlriap Fv0dGVKg55+5JQsefCj8 =0M9s -----END PGP SIGNATURE----- --DrLCvruDKXkqCJeI4idVkWRAolLVSftcr-- From owner-freebsd-virtualization@freebsd.org Fri Mar 23 17:25:13 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D4277F5C380 for ; Fri, 23 Mar 2018 17:25:12 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 56CD8810C3 for ; Fri, 23 Mar 2018 17:25:12 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (115-166-20-68.dyn.iinet.net.au [115.166.20.68]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id w2NHP6WF095954 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Fri, 23 Mar 2018 10:25:09 -0700 (PDT) (envelope-from julian@freebsd.org) Subject: Re: bhyve pci passthrough usb controller To: Wouter Snels , freebsd-virtualization@freebsd.org References: <5bd3c8b6-0579-0269-1ada-ab6f0a55ce8f@ofloo.net> From: Julian Elischer Message-ID: <2ca981b4-9651-253a-7549-eaece4a58ff0@freebsd.org> Date: Sat, 24 Mar 2018 01:25:01 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <5bd3c8b6-0579-0269-1ada-ab6f0a55ce8f@ofloo.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 17:25:13 -0000 On 23/3/18 11:40 pm, Wouter Snels wrote: new wrapping is no better It looks like the device you are passing though is not removed from freebsd first. but it's hard to tell. > |Hi, .. I'm trying to pci passhthrough a usb controller to a bhyve > windows 7 virtualization. When I start the virtual machine it freezes > the entire system and this shows up on the console. To be clear the > system runs without any problems without the pci passthrough. | > ||(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 e0 ff 40 40 29 00 > 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid > (ada0:ahcich0:0:0:0): Error 22, Unretryable error > (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 > 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid > (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ada0 at ahcich0 bus > 0 scbus0 target 0 lun 0 ada0: s/n ZA4292G6 > detached (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 > 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was > invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: > Poll timeout on slot 28 port 0 ahcich0: is 20000000 cs 10000000 ss > 00000000 rs 10000000 tfd 1d0 serr 00000800 cmd 00005c17 > (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 > 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout > (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted > (ada0:ahcich0:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 40 > 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was > invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Poll > timeout on slot 30 port 0 ahcich0: is 20000000 cs 40000000 ss 00000000 > rs 40000000 tfd 1d0 serr 00000800 cmd 00005e17 (aprobe0:ahcich0:0:0:0): > SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 > (aprobe0:ahcich0:0:0:0): CAM status: Command timeout > (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted > (ada0:ahcich0:0:0:0): SETFEATURES ENABLE WCACHE. ACB: ef 02 00 00 00 40 > 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was > invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error > (ada0:ahcich0:0:0:0): Periph destroyed em0: Watchdog timeout Queue[0]-- > resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw > tdh = 428, hw tdt = 471 em0: Tx Queue Status = -2147483648 em0: TX > descriptors avail = 981 em0: Tx Descriptors avail failure = 0 em0: RX > Queue 0 ------ em0: hw rdh = 912, hw rdt = 908 em0: RX discarded packets > = 0 em0: RX Next to Check = 909 em0: RX Next to Refresh = 908 xl0: > watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is > RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh = 0, hw tdt = 58 > em0: Tx Queue Status = -2147483648 em0: TX descriptors avail = 966 em0: > Tx Descriptors avail failure = 0 em0: RX Queue 0 ------ em0: hw rdh = 0, > hw rdt = 1023 em0: RX discarded packets = 0 em0: RX Next to Check = 0 > em0: RX Next to Refresh = 1023 em0: Watchdog timeout Queue[0]-- > resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw > tdh = 0, hw tdt = 58 em0: Tx Queue Status = -2147483648 em0: TX > descriptors avail = 966 em0: Tx Descriptors avail failure = 0 em0: RX > Queue 0 ------ em0: hw rdh = 0, hw rdt = 1023 em0: RX discarded packets > = 0 em0: RX Next to Check = 0 em0: RX Next to Refresh = 1023 xl0: > watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is > RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh = 0, hw tdt = 54 > em0: Tx Queue Status = -2147483648 em0: TX descriptors avail = 970 em0: > Tx Descriptors avail failure = 0 em0: RX Queue 0 ------ em0: hw rdh = 0, > hw rdt = 1023 em0: RX discarded packets = 0include GENERIC ident OFL > options IPFIREWALL|||||| > > |Hi, .. I'm trying to pci passhthrough a usb controller to a bhyve > windows 7 virtualization. When I start the virtual machine it freezes > the entire system and this shows up on the console. To be clear the > system runs without any problems without the pci passthrough. | > ||(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 e0 ff 40 40 29 00 > 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid > (ada0:ahcich0:0:0:0): Error 22, Unretryable error > (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 > 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid > (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ada0 at ahcich0 bus > 0 scbus0 target 0 lun 0 ada0: s/n ZA4292G6 > detached (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 > 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was > invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: > Poll timeout on slot 28 port 0 ahcich0: is 20000000 cs 10000000 ss > 00000000 rs 10000000 tfd 1d0 serr 00000800 cmd 00005c17 > (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 > 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout > (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted > (ada0:ahcich0:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 40 > 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was > invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Poll > timeout on slot 30 port 0 ahcich0: is 20000000 cs 40000000 ss 00000000 > rs 40000000 tfd 1d0 serr 00000800 cmd 00005e17 (aprobe0:ahcich0:0:0:0): > SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 > (aprobe0:ahcich0:0:0:0): CAM status: Command timeout > (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted > (ada0:ahcich0:0:0:0): SETFEATURES ENABLE WCACHE. ACB: ef 02 00 00 00 40 > 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was > invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error > (ada0:ahcich0:0:0:0): Periph destroyed em0: Watchdog timeout Queue[0]-- > resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw > tdh = 428, hw tdt = 471 em0: Tx Queue Status = -2147483648 em0: TX > descriptors avail = 981 em0: Tx Descriptors avail failure = 0 em0: RX > Queue 0 ------ em0: hw rdh = 912, hw rdt = 908 em0: RX discarded packets > = 0 em0: RX Next to Check = 909 em0: RX Next to Refresh = 908 xl0: > watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is > RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh = 0, hw tdt = 58 > em0: Tx Queue Status = -2147483648 em0: TX descriptors avail = 966 em0: > Tx Descriptors avail failure = 0 em0: RX Queue 0 ------ em0: hw rdh = 0, > hw rdt = 1023 em0: RX discarded packets = 0 em0: RX Next to Check = 0 > em0: RX Next to Refresh = 1023 em0: Watchdog timeout Queue[0]-- > resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw > tdh = 0, hw tdt = 58 em0: Tx Queue Status = -2147483648 em0: TX > descriptors avail = 966 em0: Tx Descriptors avail failure = 0 em0: RX > Queue 0 ------ em0: hw rdh = 0, hw rdt = 1023 em0: RX discarded packets > = 0 em0: RX Next to Check = 0 em0: RX Next to Refresh = 1023 xl0: > watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is > RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh = 0, hw tdt = 54 > em0: Tx Queue Status = -2147483648 em0: TX descriptors avail = 970 em0: > Tx Descriptors avail failure = 0 em0: RX Queue 0 ------ em0: hw rdh = 0, > hw rdt = 1023 em0: RX discarded packets = 0include GENERIC ident OFL > options IPFIREWALL options IPFIREWALL_VERBOSE options > IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options > CONSPEED=115200 em0: RX Next to Check = 0 em0: RX Next to Refresh = 1023 |||| > |||I've recompiled the kernel without the xhci driver. I've tested both > usb controllers. Both "|||||Sunrise Point-H USB 3.0 xHCI > Controller|"||||| and "|||uPD720202 USB 3.0 Host Controller|", both > showed the same error on the console screen.| # uname FreeBSD > 11.1-STABLE #1 r331092M # custom kernel include GENERIC ident OFL > options IPFIREWALL options IPFIREWALL_VERBOSE options > IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options > CONSPEED=115200 * note commented "device xhci" in generic # svn info |||Path: . Working Copy Root Path: /usr/src URL: > https://svn0.eu.freebsd.org/base/stable/11 Relative URL: ^/stable/11 > Repository Root: https://svn0.eu.freebsd.org/base Repository UUID: > ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind: > directory Schedule: normal Last Changed Author: eadler Last Changed Rev: > 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018) > |# vm passthru DEVICE BHYVE ID READY DESCRIPTION hostb0 0/0/0 No Xeon > E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers > pcib1 0/1/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe > Controller (x16) vgapci0 0/2/0 No HD Graphics 530 |||ppt0| 0/20/0 Yes Sunrise Point-H USB 3.0 xHCI Controller none0 0/20/2 > No Sunrise Point-H Thermal subsystem none1 0/22/0 No Sunrise Point-H > CSME HECI ahci0 0/23/0 No Sunrise Point-H SATA controller [AHCI mode] > pcib2 0/28/0 No Sunrise Point-H PCI Express Root Port pcib4 0/28/6 No > Sunrise Point-H PCI Express Root Port isab0 0/31/0 No Sunrise Point-H > LPC Controller none2 0/31/2 No Sunrise Point-H PMC hdac0 0/31/3 No > Sunrise Point-H HD Audio none3 0/31/4 No Sunrise Point-H SMBus em0 > 0/31/6 No Ethernet Connection (2) I219-V pcib3 2/0/0 No XIO2001 PCI > Express-to-PCI Bridge xl0 3/0/0 No 3c905B 100BaseTX [Cyclone] ppt1 4/0/0 > Yes uPD720202 USB 3.0 Host Controller ||ppt1@pci0:4:0:0: class=0x0c0330 card=0x00151912 chip=0x00151912 rev=0x02 > hdr=0x00 vendor = 'Renesas Technology Corp.' device = 'uPD720202 USB 3.0 > Host Controller' class = serial bus subclass = USB |||ppt|0@pci0:0:20:0: class=0x0c0330 card=0x06ba1028 chip=0xa12f8086 > rev=0x31 hdr=0x00 vendor = 'Intel Corporation' device = 'Sunrise Point-H > USB 3.0 xHCI Controller' class = serial bus subclass = USB # grep -i > pptdevfs loader.conf ||pptdevs="0/20/0 4/0/0" # vm configure ||passthru0="0/20/0"| > > |# vm configure ||passthru0="4/0/0"| > ||Any suggestions on what I'm doing wrong?|| > > Regards, Ofloo > |||||| > > ||options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT options > DUMMYNET options IPDIVERT options CONSPEED=115200 em0: RX Next to Check > = 0 em0: RX Next to Refresh = 1023 |||| > |||I've recompiled the kernel without the xhci driver. I've tested both > usb controllers. Both "|||||Sunrise Point-H USB 3.0 xHCI > Controller|"||||| and "|||uPD720202 USB 3.0 Host Controller|", both > showed the same error on the console screen.| # uname FreeBSD > 11.1-STABLE #1 r331092M # custom kernel include GENERIC ident OFL > options IPFIREWALL options IPFIREWALL_VERBOSE options > IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options > CONSPEED=115200 * note commented "device xhci" in generic # svn info |||Path: . Working Copy Root Path: /usr/src URL: > https://svn0.eu.freebsd.org/base/stable/11 Relative URL: ^/stable/11 > Repository Root: https://svn0.eu.freebsd.org/base Repository UUID: > ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind: > directory Schedule: normal Last Changed Author: eadler Last Changed Rev: > 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018) > |# vm passthru DEVICE BHYVE ID READY DESCRIPTION hostb0 0/0/0 No Xeon > E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers > pcib1 0/1/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe > Controller (x16) vgapci0 0/2/0 No HD Graphics 530 |||ppt0| 0/20/0 Yes Sunrise Point-H USB 3.0 xHCI Controller none0 0/20/2 > No Sunrise Point-H Thermal subsystem none1 0/22/0 No Sunrise Point-H > CSME HECI ahci0 0/23/0 No Sunrise Point-H SATA controller [AHCI mode] > pcib2 0/28/0 No Sunrise Point-H PCI Express Root Port pcib4 0/28/6 No > Sunrise Point-H PCI Express Root Port isab0 0/31/0 No Sunrise Point-H > LPC Controller none2 0/31/2 No Sunrise Point-H PMC hdac0 0/31/3 No > Sunrise Point-H HD Audio none3 0/31/4 No Sunrise Point-H SMBus em0 > 0/31/6 No Ethernet Connection (2) I219-V pcib3 2/0/0 No XIO2001 PCI > Express-to-PCI Bridge xl0 3/0/0 No 3c905B 100BaseTX [Cyclone] ppt1 4/0/0 > Yes uPD720202 USB 3.0 Host Controller ||ppt1@pci0:4:0:0: class=0x0c0330 card=0x00151912 chip=0x00151912 rev=0x02 > hdr=0x00 vendor = 'Renesas Technology Corp.' device = 'uPD720202 USB 3.0 > Host Controller' class = serial bus subclass = USB |||ppt|0@pci0:0:20:0: class=0x0c0330 card=0x06ba1028 chip=0xa12f8086 > rev=0x31 hdr=0x00 vendor = 'Intel Corporation' device = 'Sunrise Point-H > USB 3.0 xHCI Controller' class = serial bus subclass = USB # grep -i > pptdevfs loader.conf ||pptdevs="0/20/0 4/0/0" # vm configure ||passthru0="0/20/0"| > > |# vm configure ||passthru0="4/0/0"| > ||Any suggestions on what I'm doing wrong?|| > > edit: previous wrapping was all wrong. > > Regards, Ofloo > |||||| > > From owner-freebsd-virtualization@freebsd.org Fri Mar 23 17:32:11 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B866CF5CF88 for ; Fri, 23 Mar 2018 17:32:11 +0000 (UTC) (envelope-from byond.lenox@gmail.com) Received: from mail-lf0-f51.google.com (mail-lf0-f51.google.com [209.85.215.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2D65D818BD; Fri, 23 Mar 2018 17:32:10 +0000 (UTC) (envelope-from byond.lenox@gmail.com) Received: by mail-lf0-f51.google.com with SMTP id l4-v6so15090974lfg.12; Fri, 23 Mar 2018 10:32:10 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=0LAzJzfKTL3JkGerfOdR+5sAp2tQonI2yftIZa8MMf0=; b=HSN+9V0kGEEx+3GtvqUwyzMSwcBzrXrZ60G9Cq2M2uH+/5jRr0nKhiKWXQRpXVzvnP H1EA53ougU8oQ+vpI303MgwU7qFmyORzcP7tChdIcu2UBhFZvprOE3/v+eSxIV2U5rOB MF72Ei2w0Utpa0S6sWWZM8V0kUTnhY3uoB7dRo9+DjhFc4ILj3EoOlLzmIN8yHH20qYi XmMnbAlF/Tr7NXWFg6rWidl8XdS/hDV0UL0VUS9aSdRQS0hSIoDyZRUCEFKhHnlhVY3g hYQfvTDvz+xqZqMS2+WMMHC/nKAI8MbAGzOl9tQBvxqReayup54K3HFEk3e/J84dhU8c nvbQ== X-Gm-Message-State: AElRT7FrLlliUG9UvyNjj4oBkIO44fYvuQSmRt12yeSla027K8WQXrQq 7fXoifMhJJ+XP+cyqJeuMFDMzCZVd0M= X-Google-Smtp-Source: AG47ELtKSF2UyfD+xKUlBIdFftqfrEPpe8tiY3twK3kfwd3Nwl9MeKTrVTy+UEXBeEx3C0/DADkxWQ== X-Received: by 10.46.91.143 with SMTP id m15mr10523370lje.75.1521826328920; Fri, 23 Mar 2018 10:32:08 -0700 (PDT) Received: from mail-lf0-f50.google.com (mail-lf0-f50.google.com. [209.85.215.50]) by smtp.gmail.com with ESMTPSA id n5sm1975371ljc.71.2018.03.23.10.32.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 23 Mar 2018 10:32:08 -0700 (PDT) Received: by mail-lf0-f50.google.com with SMTP id g203-v6so19397453lfg.11; Fri, 23 Mar 2018 10:32:08 -0700 (PDT) X-Received: by 2002:a19:c4c8:: with SMTP id u191-v6mr21299249lff.109.1521826328646; Fri, 23 Mar 2018 10:32:08 -0700 (PDT) MIME-Version: 1.0 Received: by 10.46.129.90 with HTTP; Fri, 23 Mar 2018 10:31:48 -0700 (PDT) In-Reply-To: <201803231458.w2NEwMr7031548@pdx.rh.CN85.dnsmgr.net> References: <201803231458.w2NEwMr7031548@pdx.rh.CN85.dnsmgr.net> From: Kyle Evans Date: Fri, 23 Mar 2018 12:31:48 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Issue encountered booting FreeBSD STABLE and CURRENT snapshots with EFI To: "Rodney W. Grimes" Cc: Joe Maloney , Warner Losh , "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 17:32:12 -0000 On Fri, Mar 23, 2018 at 9:58 AM, Rodney W. Grimes wrote: >> On Fri, Mar 23, 2018 at 3:56 AM, Joe Maloney wrote: >> > We narrowed the issue down to how vm-bhyve attaches a null.iso when starting >> > the VM. >> > >> >> What exactly are the contents of this null.iso? It sounds like we're > > touch "null.iso" > -rw-r--r-- 1 root wheel 0 Dec 3 22:55 /home/vmbhyve/.config/null.iso > > It is litterly a 0 byte file. This is just there to appease windows > installer that you have a cd drive. > > You can test with bhyve yourself this problem by adding: > -s 3:0,ahci-cd,${vm_dir}/.config/null.iso > to your bhyve command. > Thanks for that! A couple hundred printf's later, I think this might be a firmware bug or something of the sort (?). In this loop [1], the address at 'dp' is getting immediately clobbered by the efi_lookup_devpath on every iteration, so we end up looking at basically all of the partitions and this null.iso unfortunately meets this criteria [2] for removable media with no media present. Therefore, we detect absolutely no disks. The only candidate in efi_lookup_devpath [3] is the firmware itself having overwritten the devpath at that address. We can confirm this with a printf just after the efi_lookup_devpath- the pointer it's returning is in fact exactly match 'dp' every single time. Is there a newer/different firmware you can try? This seems like really peculiar behavior for a firmware to be exhibiting. [1] https://svnweb.freebsd.org/base/head/stand/efi/libefi/efipart.c?view=markup#l232 [2] https://svnweb.freebsd.org/base/head/stand/efi/libefi/efipart.c?view=markup#l261 [3] https://svnweb.freebsd.org/base/head/stand/efi/libefi/devpath.c?view=markup#l53 From owner-freebsd-virtualization@freebsd.org Fri Mar 23 18:04:20 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8AB96F5F8F4 for ; Fri, 23 Mar 2018 18:04:20 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E7A6983189; Fri, 23 Mar 2018 18:04:19 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w2NI4Ft1032386; Fri, 23 Mar 2018 11:04:15 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w2NI4FQq032385; Fri, 23 Mar 2018 11:04:15 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201803231804.w2NI4FQq032385@pdx.rh.CN85.dnsmgr.net> Subject: Re: Issue encountered booting FreeBSD STABLE and CURRENT snapshots with EFI In-Reply-To: To: Kyle Evans Date: Fri, 23 Mar 2018 11:04:15 -0700 (PDT) CC: Joe Maloney , Warner Losh , "freebsd-virtualization@freebsd.org" X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 18:04:20 -0000 [ Charset UTF-8 unsupported, converting... ] > On Fri, Mar 23, 2018 at 9:58 AM, Rodney W. Grimes > wrote: > >> On Fri, Mar 23, 2018 at 3:56 AM, Joe Maloney wrote: > >> > We narrowed the issue down to how vm-bhyve attaches a null.iso when starting > >> > the VM. > >> > > >> > >> What exactly are the contents of this null.iso? It sounds like we're > > > > touch "null.iso" > > -rw-r--r-- 1 root wheel 0 Dec 3 22:55 /home/vmbhyve/.config/null.iso > > > > It is litterly a 0 byte file. This is just there to appease windows > > installer that you have a cd drive. > > > > You can test with bhyve yourself this problem by adding: > > -s 3:0,ahci-cd,${vm_dir}/.config/null.iso > > to your bhyve command. > > > > Thanks for that! A couple hundred printf's later, I think this might > be a firmware bug or something of the sort (?). > > In this loop [1], the address at 'dp' is getting immediately clobbered > by the efi_lookup_devpath on every iteration, so we end up looking at > basically all of the partitions and this null.iso unfortunately meets > this criteria [2] for removable media with no media present. > Therefore, we detect absolutely no disks. > > The only candidate in efi_lookup_devpath [3] is the firmware itself > having overwritten the devpath at that address. We can confirm this > with a printf just after the efi_lookup_devpath- the pointer it's > returning is in fact exactly match 'dp' every single time. > > Is there a newer/different firmware you can try? This seems like > really peculiar behavior for a firmware to be exhibiting. What happened to the boot device on the hd? There should be another -s x:0, ahci-hd drive that it boots from. -s 4:0,ahci-hd,${vm_dir}/${vmname}/disk0.img > [1] https://svnweb.freebsd.org/base/head/stand/efi/libefi/efipart.c?view=markup#l232 > [2] https://svnweb.freebsd.org/base/head/stand/efi/libefi/efipart.c?view=markup#l261 > [3] https://svnweb.freebsd.org/base/head/stand/efi/libefi/devpath.c?view=markup#l53 > -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Fri Mar 23 18:06:20 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7C79BF5FB8B for ; Fri, 23 Mar 2018 18:06:20 +0000 (UTC) (envelope-from byond.lenox@gmail.com) Received: from mail-lf0-f44.google.com (mail-lf0-f44.google.com [209.85.215.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DF0C883388; Fri, 23 Mar 2018 18:06:19 +0000 (UTC) (envelope-from byond.lenox@gmail.com) Received: by mail-lf0-f44.google.com with SMTP id c78-v6so14879482lfh.1; Fri, 23 Mar 2018 11:06:19 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=NRqooJ2sc7BD0bIG9WVRktLrtTFbdpv3VXapw9GwpMY=; b=lC81qEoYCRwfClKTEBgij1PnYZ9UKFRW599abaN/UFSi1IbPMBf2EQTlTcJmrsrbSB uLFS3cr/93Ljqv5D/5nnhnpqQCJNgHQ/NDQOwaMealG6ioSlpA0aFsef/EcG8GQLuMTh Cp6A+w06t6VVGbBGcuz5sUPDPKoHZqbcetjJ0ebvpaB+pP8Nz9J4lrXobg0wc4DUBX9i G894kWUoWpZt0HdSd0PZZNKDeqted4D/dtT0ZJJUD4EY8jYRM+iNy9CW2m6obBKUtnvj Auwe4IvSkhHYCwiugfQ69+zW6++SBf7a6TfxX5TWi6gqLxO6g85hFzYBtxUEE29W9wzO PCAw== X-Gm-Message-State: AElRT7H/fRDwxC0fKZHoUVhvp25lSamc+50WlzzK9BNiHgah8wW4B2Me Xq15AZO3p4W+L4DGzxe/38T/V19Nerc= X-Google-Smtp-Source: AG47ELscV+B2Ot7UhshP4ycROu32gaD6WleJRhD0uzFEV+c1xYPA4CdsyIdBG7GuSEVhb2k4KZ5kWg== X-Received: by 10.46.83.67 with SMTP id t3mr19585051ljd.63.1521828377922; Fri, 23 Mar 2018 11:06:17 -0700 (PDT) Received: from mail-lf0-f42.google.com (mail-lf0-f42.google.com. [209.85.215.42]) by smtp.gmail.com with ESMTPSA id e28-v6sm2332800lfb.52.2018.03.23.11.06.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 23 Mar 2018 11:06:17 -0700 (PDT) Received: by mail-lf0-f42.google.com with SMTP id p142-v6so19563701lfd.6; Fri, 23 Mar 2018 11:06:17 -0700 (PDT) X-Received: by 2002:a19:114f:: with SMTP id g76-v6mr15613102lfi.108.1521828377474; Fri, 23 Mar 2018 11:06:17 -0700 (PDT) MIME-Version: 1.0 Received: by 10.46.129.90 with HTTP; Fri, 23 Mar 2018 11:05:56 -0700 (PDT) In-Reply-To: <201803231804.w2NI4FQq032385@pdx.rh.CN85.dnsmgr.net> References: <201803231804.w2NI4FQq032385@pdx.rh.CN85.dnsmgr.net> From: Kyle Evans Date: Fri, 23 Mar 2018 13:05:56 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Issue encountered booting FreeBSD STABLE and CURRENT snapshots with EFI To: "Rodney W. Grimes" Cc: Joe Maloney , Warner Losh , "freebsd-virtualization@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 18:06:20 -0000 On Fri, Mar 23, 2018 at 1:04 PM, Rodney W. Grimes wrote: > [ Charset UTF-8 unsupported, converting... ] >> On Fri, Mar 23, 2018 at 9:58 AM, Rodney W. Grimes >> wrote: >> >> On Fri, Mar 23, 2018 at 3:56 AM, Joe Maloney wrote: >> >> > We narrowed the issue down to how vm-bhyve attaches a null.iso when starting >> >> > the VM. >> >> > >> >> >> >> What exactly are the contents of this null.iso? It sounds like we're >> > >> > touch "null.iso" >> > -rw-r--r-- 1 root wheel 0 Dec 3 22:55 /home/vmbhyve/.config/null.iso >> > >> > It is litterly a 0 byte file. This is just there to appease windows >> > installer that you have a cd drive. >> > >> > You can test with bhyve yourself this problem by adding: >> > -s 3:0,ahci-cd,${vm_dir}/.config/null.iso >> > to your bhyve command. >> > >> >> Thanks for that! A couple hundred printf's later, I think this might >> be a firmware bug or something of the sort (?). >> >> In this loop [1], the address at 'dp' is getting immediately clobbered >> by the efi_lookup_devpath on every iteration, so we end up looking at >> basically all of the partitions and this null.iso unfortunately meets >> this criteria [2] for removable media with no media present. >> Therefore, we detect absolutely no disks. >> >> The only candidate in efi_lookup_devpath [3] is the firmware itself >> having overwritten the devpath at that address. We can confirm this >> with a printf just after the efi_lookup_devpath- the pointer it's >> returning is in fact exactly match 'dp' every single time. >> >> Is there a newer/different firmware you can try? This seems like >> really peculiar behavior for a firmware to be exhibiting. > > What happened to the boot device on the hd? > There should be another -s x:0, ahci-hd drive that it boots from. > -s 4:0,ahci-hd,${vm_dir}/${vmname}/disk0.img > Right, it's there but we think it's removable media with no media present because of the above-mentioned clobbering by the firmware. We could probably work around it for this specific case, but I don't know if that's a great idea- there be snakes here... From owner-freebsd-virtualization@freebsd.org Fri Mar 23 18:10:46 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3719EF6026A for ; Fri, 23 Mar 2018 18:10:44 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A180A837D9; Fri, 23 Mar 2018 18:10:43 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w2NIAe7i032431; Fri, 23 Mar 2018 11:10:40 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w2NIAehD032430; Fri, 23 Mar 2018 11:10:40 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201803231810.w2NIAehD032430@pdx.rh.CN85.dnsmgr.net> Subject: Re: Issue encountered booting FreeBSD STABLE and CURRENT snapshots with EFI In-Reply-To: To: Kyle Evans Date: Fri, 23 Mar 2018 11:10:40 -0700 (PDT) CC: Warner Losh , "freebsd-virtualization@freebsd.org" X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 18:10:46 -0000 > On Fri, Mar 23, 2018 at 1:04 PM, Rodney W. Grimes > wrote: > > [ Charset UTF-8 unsupported, converting... ] > >> On Fri, Mar 23, 2018 at 9:58 AM, Rodney W. Grimes > >> wrote: > >> >> On Fri, Mar 23, 2018 at 3:56 AM, Joe Maloney wrote: > >> >> > We narrowed the issue down to how vm-bhyve attaches a null.iso when starting > >> >> > the VM. > >> >> > > >> >> > >> >> What exactly are the contents of this null.iso? It sounds like we're > >> > > >> > touch "null.iso" > >> > -rw-r--r-- 1 root wheel 0 Dec 3 22:55 /home/vmbhyve/.config/null.iso > >> > > >> > It is litterly a 0 byte file. This is just there to appease windows > >> > installer that you have a cd drive. > >> > > >> > You can test with bhyve yourself this problem by adding: > >> > -s 3:0,ahci-cd,${vm_dir}/.config/null.iso > >> > to your bhyve command. > >> > > >> > >> Thanks for that! A couple hundred printf's later, I think this might > >> be a firmware bug or something of the sort (?). > >> > >> In this loop [1], the address at 'dp' is getting immediately clobbered > >> by the efi_lookup_devpath on every iteration, so we end up looking at > >> basically all of the partitions and this null.iso unfortunately meets > >> this criteria [2] for removable media with no media present. > >> Therefore, we detect absolutely no disks. > >> > >> The only candidate in efi_lookup_devpath [3] is the firmware itself > >> having overwritten the devpath at that address. We can confirm this > >> with a printf just after the efi_lookup_devpath- the pointer it's > >> returning is in fact exactly match 'dp' every single time. > >> > >> Is there a newer/different firmware you can try? This seems like > >> really peculiar behavior for a firmware to be exhibiting. > > > > What happened to the boot device on the hd? > > There should be another -s x:0, ahci-hd drive that it boots from. > > -s 4:0,ahci-hd,${vm_dir}/${vmname}/disk0.img > > > > Right, it's there but we think it's removable media with no media > present because of the above-mentioned clobbering by the firmware. We > could probably work around it for this specific case, but I don't know > if that's a great idea- there be snakes here... Perhaps take a look at how and why this was working fine with the forth boot code? -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Fri Mar 23 18:12:24 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8F86BF607DB for ; Fri, 23 Mar 2018 18:12:24 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E680983C9E; Fri, 23 Mar 2018 18:12:23 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w2NICLvV032461; Fri, 23 Mar 2018 11:12:21 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w2NICLEW032460; Fri, 23 Mar 2018 11:12:21 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201803231812.w2NICLEW032460@pdx.rh.CN85.dnsmgr.net> Subject: Re: Issue encountered booting FreeBSD STABLE and CURRENT snapshots with EFI In-Reply-To: To: Kyle Evans Date: Fri, 23 Mar 2018 11:12:21 -0700 (PDT) CC: Warner Losh , "freebsd-virtualization@freebsd.org" X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 18:12:24 -0000 > On Fri, Mar 23, 2018 at 1:04 PM, Rodney W. Grimes > wrote: > > [ Charset UTF-8 unsupported, converting... ] > >> On Fri, Mar 23, 2018 at 9:58 AM, Rodney W. Grimes > >> wrote: > >> >> On Fri, Mar 23, 2018 at 3:56 AM, Joe Maloney wrote: > >> >> > We narrowed the issue down to how vm-bhyve attaches a null.iso when starting > >> >> > the VM. > >> >> > > >> >> > >> >> What exactly are the contents of this null.iso? It sounds like we're > >> > > >> > touch "null.iso" > >> > -rw-r--r-- 1 root wheel 0 Dec 3 22:55 /home/vmbhyve/.config/null.iso > >> > > >> > It is litterly a 0 byte file. This is just there to appease windows > >> > installer that you have a cd drive. > >> > > >> > You can test with bhyve yourself this problem by adding: > >> > -s 3:0,ahci-cd,${vm_dir}/.config/null.iso > >> > to your bhyve command. > >> > > >> > >> Thanks for that! A couple hundred printf's later, I think this might > >> be a firmware bug or something of the sort (?). > >> > >> In this loop [1], the address at 'dp' is getting immediately clobbered > >> by the efi_lookup_devpath on every iteration, so we end up looking at > >> basically all of the partitions and this null.iso unfortunately meets > >> this criteria [2] for removable media with no media present. > >> Therefore, we detect absolutely no disks. > >> > >> The only candidate in efi_lookup_devpath [3] is the firmware itself > >> having overwritten the devpath at that address. We can confirm this > >> with a printf just after the efi_lookup_devpath- the pointer it's > >> returning is in fact exactly match 'dp' every single time. > >> > >> Is there a newer/different firmware you can try? This seems like > >> really peculiar behavior for a firmware to be exhibiting. > > > > What happened to the boot device on the hd? > > There should be another -s x:0, ahci-hd drive that it boots from. > > -s 4:0,ahci-hd,${vm_dir}/${vmname}/disk0.img > > > > Right, it's there but we think it's removable media with no media > present because of the above-mentioned clobbering by the firmware. We > could probably work around it for this specific case, but I don't know > if that's a great idea- there be snakes here... And this also appers to works with the Windows boot code.. so I do suspect something has become broken in the new loader. -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Fri Mar 23 18:34:32 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 05C95F6290C for ; Fri, 23 Mar 2018 18:34:32 +0000 (UTC) (envelope-from byond.lenox@gmail.com) Received: from mail-lf0-f54.google.com (mail-lf0-f54.google.com [209.85.215.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6D48E85156; Fri, 23 Mar 2018 18:34:31 +0000 (UTC) (envelope-from byond.lenox@gmail.com) Received: by mail-lf0-f54.google.com with SMTP id x205-v6so19683841lfa.0; Fri, 23 Mar 2018 11:34:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Ls8CzWFMvNkACTEskLgjmQm8EbU1ahjznkUHSRVuVCM=; b=un5SGNTuYFhLgZBBijPCDSPb6sy6nz+f1FobRTBbTyB5pp/MfgKvCMSnHUVgE9WnPp qY1KLrpa0yz5QsYWlOK296blw7WSCfg2E4kpqj0V30AEn2ZOYPy5YsYtfSmjhPEDQSmz GSqJin94w6+BF39E/eiVPkOLSlkA3iMWiz4haFuakLtXbOTeIwBdk0gf4SK0U2HvikcQ OdaQ5fCnO5sPy2ZDGBisbrx825C4+JMPcamIs1pwLqYJgXEUDjJMxYpOmxQrmkt5fP/i Fjh+MdxPvMA5Mhjdl2w39j1OZIzoIwXU7+m8rukPcUjRx6BP/Tlhp/+I54XTVjHESel0 zf6Q== X-Gm-Message-State: AElRT7Ht6WQJoc8qWOu9RjyQl7/7qYSUprCwjjSdGCOFRh/hzTY6bSHM ZgWi2vFZa0l4jHV/Hz9VZWZKmKTUe4g= X-Google-Smtp-Source: AG47ELsyPOQ7C+dx0JPAipY0pE/fwfDU8fQ3jqDLc5rybhgBs37psLoh06AcfzSv4MhtumlOdl+7ZQ== X-Received: by 2002:a19:385e:: with SMTP id d30-v6mr19792927lfj.16.1521830063535; Fri, 23 Mar 2018 11:34:23 -0700 (PDT) Received: from mail-lf0-f45.google.com (mail-lf0-f45.google.com. [209.85.215.45]) by smtp.gmail.com with ESMTPSA id u84-v6sm2345925lfk.71.2018.03.23.11.34.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 23 Mar 2018 11:34:23 -0700 (PDT) Received: by mail-lf0-f45.google.com with SMTP id a22-v6so19657541lfg.9; Fri, 23 Mar 2018 11:34:23 -0700 (PDT) X-Received: by 2002:a19:e9cc:: with SMTP id j73-v6mr18144632lfk.99.1521830063066; Fri, 23 Mar 2018 11:34:23 -0700 (PDT) MIME-Version: 1.0 Received: by 10.46.129.90 with HTTP; Fri, 23 Mar 2018 11:34:02 -0700 (PDT) In-Reply-To: <201803231812.w2NICLEW032460@pdx.rh.CN85.dnsmgr.net> References: <201803231812.w2NICLEW032460@pdx.rh.CN85.dnsmgr.net> From: Kyle Evans Date: Fri, 23 Mar 2018 13:34:02 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Issue encountered booting FreeBSD STABLE and CURRENT snapshots with EFI To: "Rodney W. Grimes" Cc: Warner Losh , "freebsd-virtualization@freebsd.org" , Peter Grehan , Marcelo Araujo Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 18:34:32 -0000 On Fri, Mar 23, 2018 at 1:12 PM, Rodney W. Grimes wrote: >> On Fri, Mar 23, 2018 at 1:04 PM, Rodney W. Grimes >> wrote: >> > [ Charset UTF-8 unsupported, converting... ] >> >> On Fri, Mar 23, 2018 at 9:58 AM, Rodney W. Grimes >> >> wrote: >> >> >> On Fri, Mar 23, 2018 at 3:56 AM, Joe Maloney wrote: >> >> >> > We narrowed the issue down to how vm-bhyve attaches a null.iso when starting >> >> >> > the VM. >> >> >> > >> >> >> >> >> >> What exactly are the contents of this null.iso? It sounds like we're >> >> > >> >> > touch "null.iso" >> >> > -rw-r--r-- 1 root wheel 0 Dec 3 22:55 /home/vmbhyve/.config/null.iso >> >> > >> >> > It is litterly a 0 byte file. This is just there to appease windows >> >> > installer that you have a cd drive. >> >> > >> >> > You can test with bhyve yourself this problem by adding: >> >> > -s 3:0,ahci-cd,${vm_dir}/.config/null.iso >> >> > to your bhyve command. >> >> > >> >> >> >> Thanks for that! A couple hundred printf's later, I think this might >> >> be a firmware bug or something of the sort (?). >> >> >> >> In this loop [1], the address at 'dp' is getting immediately clobbered >> >> by the efi_lookup_devpath on every iteration, so we end up looking at >> >> basically all of the partitions and this null.iso unfortunately meets >> >> this criteria [2] for removable media with no media present. >> >> Therefore, we detect absolutely no disks. >> >> >> >> The only candidate in efi_lookup_devpath [3] is the firmware itself >> >> having overwritten the devpath at that address. We can confirm this >> >> with a printf just after the efi_lookup_devpath- the pointer it's >> >> returning is in fact exactly match 'dp' every single time. >> >> >> >> Is there a newer/different firmware you can try? This seems like >> >> really peculiar behavior for a firmware to be exhibiting. >> > >> > What happened to the boot device on the hd? >> > There should be another -s x:0, ahci-hd drive that it boots from. >> > -s 4:0,ahci-hd,${vm_dir}/${vmname}/disk0.img >> > >> >> Right, it's there but we think it's removable media with no media >> present because of the above-mentioned clobbering by the firmware. We >> could probably work around it for this specific case, but I don't know >> if that's a great idea- there be snakes here... > > And this also appers to works with the Windows boot code.. so I > do suspect something has become broken in the new loader. > Benno points out that this is due to a uefi-edk2 hack [1] that should be ripped out now. To quote him and his quote of grehan: 13:23 <@benno> https://github.com/freebsd/uefi-edk2/blob/bhyve/UDK2014.SP1/BhyvePkg/Library/PlatformBdsLib/BdsPlatform.c#L1030 13:23 <@benno> If you short-circuit that if so it always returns everything will be happier. 13:24 <@benno> Quoting grehan: 13:24 < kevans91> Oh no 13:25 < kevans91> ew 13:25 <@benno> "The change was put in as a workaround for a bug in the FreeBSD/UEFI loader, however, that issue was fixed with r285246. 13:25 <@benno> I'd think it unlikely that anyone would want to run <= 10.1 with bhyve/UEFI, so I the intercept could safely be removed." 13:25 <@benno> So the proper fix is to remove the override entirely. 13:25 <@benno> But in my testing I only got as far as just nerfing it via sticking a 1 || in front of the if condition. I don't think we want to workaround this in loader, and I can't explain why this is functional with Windows since it's Windows- if I could review some of the relevant source, I would. Also CC'ing grehan@, since he's the one that knows the stuff. [1] https://github.com/freebsd/uefi-edk2/blob/bhyve/UDK2014.SP1/BhyvePkg/Library/PlatformBdsLib/BdsPlatform.c#L1008 From owner-freebsd-virtualization@freebsd.org Fri Mar 23 18:54:41 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DF71FF6444F for ; Fri, 23 Mar 2018 18:54:40 +0000 (UTC) (envelope-from nospam@ofloo.net) Received: from loki.ofloo.net (unknown [IPv6:2001:470:7acd:0:1:1:0:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "loki.ofloo.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 57B5C865A6; Fri, 23 Mar 2018 18:54:40 +0000 (UTC) (envelope-from nospam@ofloo.net) Received: from [IPv6:fc03:1337:dead:cafe:448c:cc95:4a7e:75c1] ([IPv6:fc03:1337:dead:cafe:448c:cc95:4a7e:75c1]) by loki.ofloo.net (8.15.2/8.15.2) with ESMTPS id w2NIrhcc047854 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Fri, 23 Mar 2018 18:53:49 GMT (envelope-from nospam@ofloo.net) Subject: Re: bhyve pci passthrough usb controller To: Julian Elischer , freebsd-virtualization@freebsd.org References: <5bd3c8b6-0579-0269-1ada-ab6f0a55ce8f@ofloo.net> <2ca981b4-9651-253a-7549-eaece4a58ff0@freebsd.org> From: Wouter Snels Message-ID: <8945b900-ca21-26e1-fa15-5dbeac8716b4@ofloo.net> Date: Fri, 23 Mar 2018 19:53:37 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <2ca981b4-9651-253a-7549-eaece4a58ff0@freebsd.org> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="MQOKdVQJ8Vjc1Uz5qruwntr4Siq4VacOy" X-Spam-Status: No, score=-1.8 required=5.0 tests=ALL_TRUSTED,BAYES_00, RDNS_NONE,SPF_PASS,TW_EV,TW_SV,TW_TD,TW_XH autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on loki.ofloo.net X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 18:54:41 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --MQOKdVQJ8Vjc1Uz5qruwntr4Siq4VacOy Content-Type: multipart/mixed; boundary="eUUAWQ4DS1XcIw7pzMGtyK9ck2aFdgVIR"; protected-headers="v1" From: Wouter Snels To: Julian Elischer , freebsd-virtualization@freebsd.org Message-ID: <8945b900-ca21-26e1-fa15-5dbeac8716b4@ofloo.net> Subject: Re: bhyve pci passthrough usb controller References: <5bd3c8b6-0579-0269-1ada-ab6f0a55ce8f@ofloo.net> <2ca981b4-9651-253a-7549-eaece4a58ff0@freebsd.org> In-Reply-To: <2ca981b4-9651-253a-7549-eaece4a58ff0@freebsd.org> --eUUAWQ4DS1XcIw7pzMGtyK9ck2aFdgVIR Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Content-Language: nl-BE No the xhci driver is not attached only the the ppt driver is, but that's normal because you set pptdevs=3D"0/20/0 4/0/0" in /boot/loader.co= nf Op 23-03-18 om 18:25 schreef Julian Elischer: > On 23/3/18 11:40 pm, Wouter Snels wrote: > > > new wrapping is no better > It looks like the device you are passing though is not removed from > freebsd first. > but it's hard to tell. > >> |Hi, .. I'm trying to pci passhthrough a usb controller to a bhyve >> windows 7 virtualization. When I start the virtual machine it freezes >> the entire system and this shows up on the console. To be clear the >> system runs without any problems without the pci passthrough. | >> ||(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 e0 ff 40 40 29 = 00 >> 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid >> (ada0:ahcich0:0:0:0): Error 22, Unretryable error >> (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00= >> 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid >> (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ada0 at ahcich0 b= us >> 0 scbus0 target 0 lun 0 ada0: s/n ZA4292G6 >> detached (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40= >> 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was= >> invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: >> Poll timeout on slot 28 port 0 ahcich0: is 20000000 cs 10000000 ss >> 00000000 rs 10000000 tfd 1d0 serr 00000800 cmd 00005c17 >> (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 0= 0 >> 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted >> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 4= 0 >> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was >> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Pol= l >> timeout on slot 30 port 0 ahcich0: is 20000000 cs 40000000 ss 00000000= >> rs 40000000 tfd 1d0 serr 00000800 cmd 00005e17 (aprobe0:ahcich0:0:0:0)= : >> SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 >> (aprobe0:ahcich0:0:0:0): CAM status: Command timeout >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted >> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE WCACHE. ACB: ef 02 00 00 00 4= 0 >> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was >> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error >> (ada0:ahcich0:0:0:0): Periph destroyed em0: Watchdog timeout Queue[0]-= - >> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: = hw >> tdh =3D 428, hw tdt =3D 471 em0: Tx Queue Status =3D -2147483648 em0: = TX >> descriptors avail =3D 981 em0: Tx Descriptors avail failure =3D 0 em0:= RX >> Queue 0 ------ em0: hw rdh =3D 912, hw rdt =3D 908 em0: RX discarded p= ackets >> =3D 0 em0: RX Next to Check =3D 909 em0: RX Next to Refresh =3D 908 xl= 0: >> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface = is >> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D= 58 >> em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 966= em0: >> Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh = =3D 0, >> hw rdt =3D 1023 em0: RX discarded packets =3D 0 em0: RX Next to Check = =3D 0 >> em0: RX Next to Refresh =3D 1023 em0: Watchdog timeout Queue[0]-- >> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: = hw >> tdh =3D 0, hw tdt =3D 58 em0: Tx Queue Status =3D -2147483648 em0: TX >> descriptors avail =3D 966 em0: Tx Descriptors avail failure =3D 0 em0:= RX >> Queue 0 ------ em0: hw rdh =3D 0, hw rdt =3D 1023 em0: RX discarded pa= ckets >> =3D 0 em0: RX Next to Check =3D 0 em0: RX Next to Refresh =3D 1023 xl0= : >> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface = is >> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D= 54 >> em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 970= em0: >> Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh = =3D 0, >> hw rdt =3D 1023 em0: RX discarded packets =3D 0include GENERIC ident O= FL >> options IPFIREWALL|||||| >> >> |Hi, .. I'm trying to pci passhthrough a usb controller to a bhyve >> windows 7 virtualization. When I start the virtual machine it freezes >> the entire system and this shows up on the console. To be clear the >> system runs without any problems without the pci passthrough. | >> ||(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 e0 ff 40 40 29 = 00 >> 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid >> (ada0:ahcich0:0:0:0): Error 22, Unretryable error >> (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00= >> 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid >> (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ada0 at ahcich0 b= us >> 0 scbus0 target 0 lun 0 ada0: s/n ZA4292G6 >> detached (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40= >> 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was= >> invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: >> Poll timeout on slot 28 port 0 ahcich0: is 20000000 cs 10000000 ss >> 00000000 rs 10000000 tfd 1d0 serr 00000800 cmd 00005c17 >> (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 0= 0 >> 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted >> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 4= 0 >> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was >> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Pol= l >> timeout on slot 30 port 0 ahcich0: is 20000000 cs 40000000 ss 00000000= >> rs 40000000 tfd 1d0 serr 00000800 cmd 00005e17 (aprobe0:ahcich0:0:0:0)= : >> SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 >> (aprobe0:ahcich0:0:0:0): CAM status: Command timeout >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted >> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE WCACHE. ACB: ef 02 00 00 00 4= 0 >> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was >> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error >> (ada0:ahcich0:0:0:0): Periph destroyed em0: Watchdog timeout Queue[0]-= - >> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: = hw >> tdh =3D 428, hw tdt =3D 471 em0: Tx Queue Status =3D -2147483648 em0: = TX >> descriptors avail =3D 981 em0: Tx Descriptors avail failure =3D 0 em0:= RX >> Queue 0 ------ em0: hw rdh =3D 912, hw rdt =3D 908 em0: RX discarded p= ackets >> =3D 0 em0: RX Next to Check =3D 909 em0: RX Next to Refresh =3D 908 xl= 0: >> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface = is >> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D= 58 >> em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 966= em0: >> Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh = =3D 0, >> hw rdt =3D 1023 em0: RX discarded packets =3D 0 em0: RX Next to Check = =3D 0 >> em0: RX Next to Refresh =3D 1023 em0: Watchdog timeout Queue[0]-- >> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: = hw >> tdh =3D 0, hw tdt =3D 58 em0: Tx Queue Status =3D -2147483648 em0: TX >> descriptors avail =3D 966 em0: Tx Descriptors avail failure =3D 0 em0:= RX >> Queue 0 ------ em0: hw rdh =3D 0, hw rdt =3D 1023 em0: RX discarded pa= ckets >> =3D 0 em0: RX Next to Check =3D 0 em0: RX Next to Refresh =3D 1023 xl0= : >> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface = is >> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D= 54 >> em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 970= em0: >> Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh = =3D 0, >> hw rdt =3D 1023 em0: RX discarded packets =3D 0include GENERIC ident O= FL >> options IPFIREWALL options IPFIREWALL_VERBOSE options >> IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options= >> CONSPEED=3D115200 em0: RX Next to Check =3D 0 em0: RX Next to Refresh = =3D >> 1023 |||| >> |||I've recompiled the kernel without the xhci driver. I've tested bot= h >> usb controllers. Both "|||||Sunrise Point-H USB 3.0 xHCI >> Controller|"||||| and "|||uPD720202 USB 3.0 Host Controller|", both >> showed the same error on the console screen.| # uname FreeBSD >> 11.1-STABLE #1 r331092M # custom kernel include GENERIC ident OFL >> options IPFIREWALL options IPFIREWALL_VERBOSE options >> IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options= >> CONSPEED=3D115200 * note commented "device xhci" in generic # svn info= >> |||Path: . Working Copy Root Path: /usr/src URL: >> https://svn0.eu.freebsd.org/base/stable/11 Relative URL: ^/stable/11 >> Repository Root: https://svn0.eu.freebsd.org/base Repository UUID: >> ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind: >> directory Schedule: normal Last Changed Author: eadler Last Changed Re= v: >> 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018)= >> |# vm passthru DEVICE BHYVE ID READY DESCRIPTION hostb0 0/0/0 No Xeon >> E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Register= s >> pcib1 0/1/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe >> Controller (x16) vgapci0 0/2/0 No HD Graphics 530 |||ppt0| 0/20/0 Yes >> Sunrise Point-H USB 3.0 xHCI Controller none0 0/20/2 >> No Sunrise Point-H Thermal subsystem none1 0/22/0 No Sunrise Point-H >> CSME HECI ahci0 0/23/0 No Sunrise Point-H SATA controller [AHCI mode] >> pcib2 0/28/0 No Sunrise Point-H PCI Express Root Port pcib4 0/28/6 No >> Sunrise Point-H PCI Express Root Port isab0 0/31/0 No Sunrise Point-H >> LPC Controller none2 0/31/2 No Sunrise Point-H PMC hdac0 0/31/3 No >> Sunrise Point-H HD Audio none3 0/31/4 No Sunrise Point-H SMBus em0 >> 0/31/6 No Ethernet Connection (2) I219-V pcib3 2/0/0 No XIO2001 PCI >> Express-to-PCI Bridge xl0 3/0/0 No 3c905B 100BaseTX [Cyclone] ppt1 4/0= /0 >> Yes uPD720202 USB 3.0 Host Controller ||ppt1@pci0:4:0:0: >> class=3D0x0c0330 card=3D0x00151912 chip=3D0x00151912 rev=3D0x02 >> hdr=3D0x00 vendor =3D 'Renesas Technology Corp.' device =3D 'uPD720202= USB 3.0 >> Host Controller' class =3D serial bus subclass =3D USB >> |||ppt|0@pci0:0:20:0: class=3D0x0c0330 card=3D0x06ba1028 chip=3D0xa12f= 8086 >> rev=3D0x31 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Sunri= se Point-H >> USB 3.0 xHCI Controller' class =3D serial bus subclass =3D USB # grep = -i >> pptdevfs loader.conf ||pptdevs=3D"0/20/0 4/0/0" # vm configure >> ||passthru0=3D"0/20/0"| >> >> |# vm configure ||passthru0=3D"4/0/0"| >> ||Any suggestions on what I'm doing wrong?|| >> =C2=A0 Regards, Ofloo >> |||||| >> >> ||options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT >> options >> DUMMYNET options IPDIVERT options CONSPEED=3D115200 em0: RX Next to Ch= eck >> =3D 0 em0: RX Next to Refresh =3D 1023 |||| >> |||I've recompiled the kernel without the xhci driver. I've tested bot= h >> usb controllers. Both "|||||Sunrise Point-H USB 3.0 xHCI >> Controller|"||||| and "|||uPD720202 USB 3.0 Host Controller|", both >> showed the same error on the console screen.| # uname FreeBSD >> 11.1-STABLE #1 r331092M # custom kernel include GENERIC ident OFL >> options IPFIREWALL options IPFIREWALL_VERBOSE options >> IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options= >> CONSPEED=3D115200 * note commented "device xhci" in generic # svn info= >> |||Path: . Working Copy Root Path: /usr/src URL: >> https://svn0.eu.freebsd.org/base/stable/11 Relative URL: ^/stable/11 >> Repository Root: https://svn0.eu.freebsd.org/base Repository UUID: >> ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind: >> directory Schedule: normal Last Changed Author: eadler Last Changed Re= v: >> 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018)= >> |# vm passthru DEVICE BHYVE ID READY DESCRIPTION hostb0 0/0/0 No Xeon >> E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Register= s >> pcib1 0/1/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe >> Controller (x16) vgapci0 0/2/0 No HD Graphics 530 |||ppt0| 0/20/0 Yes >> Sunrise Point-H USB 3.0 xHCI Controller none0 0/20/2 >> No Sunrise Point-H Thermal subsystem none1 0/22/0 No Sunrise Point-H >> CSME HECI ahci0 0/23/0 No Sunrise Point-H SATA controller [AHCI mode] >> pcib2 0/28/0 No Sunrise Point-H PCI Express Root Port pcib4 0/28/6 No >> Sunrise Point-H PCI Express Root Port isab0 0/31/0 No Sunrise Point-H >> LPC Controller none2 0/31/2 No Sunrise Point-H PMC hdac0 0/31/3 No >> Sunrise Point-H HD Audio none3 0/31/4 No Sunrise Point-H SMBus em0 >> 0/31/6 No Ethernet Connection (2) I219-V pcib3 2/0/0 No XIO2001 PCI >> Express-to-PCI Bridge xl0 3/0/0 No 3c905B 100BaseTX [Cyclone] ppt1 4/0= /0 >> Yes uPD720202 USB 3.0 Host Controller ||ppt1@pci0:4:0:0: >> class=3D0x0c0330 card=3D0x00151912 chip=3D0x00151912 rev=3D0x02 >> hdr=3D0x00 vendor =3D 'Renesas Technology Corp.' device =3D 'uPD720202= USB 3.0 >> Host Controller' class =3D serial bus subclass =3D USB >> |||ppt|0@pci0:0:20:0: class=3D0x0c0330 card=3D0x06ba1028 chip=3D0xa12f= 8086 >> rev=3D0x31 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Sunri= se Point-H >> USB 3.0 xHCI Controller' class =3D serial bus subclass =3D USB # grep = -i >> pptdevfs loader.conf ||pptdevs=3D"0/20/0 4/0/0" # vm configure >> ||passthru0=3D"0/20/0"| >> >> |# vm configure ||passthru0=3D"4/0/0"| >> ||Any suggestions on what I'm doing wrong?|| >> >> edit: previous wrapping was all wrong. >> =C2=A0=C2=A0 Regards, Ofloo >> |||||| >> >> --eUUAWQ4DS1XcIw7pzMGtyK9ck2aFdgVIR-- --MQOKdVQJ8Vjc1Uz5qruwntr4Siq4VacOy Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJatU0xAAoJENWumses7+9mrIYQAIPKiaK4LkdeAZjrgtj48hv3 r18Af2jzNOTueIZde6v64T6AbWlh/P+U/N7ii4skV8hM0OIBX0zoHb0tmXHM1Amg hb8gGAFwQauHjfoV78VMjtqjVi7ps9Cu+ch4TlJHRwOvkRrYu+JyW1fQYgTsMRPg 2NkzwD44W2pizQpUHnL3SqFkxWZwRCQ7zYihzuTz0BtotXmxcb89iH9d0/iwGILv 4ub+qLuA6TA316Sa3sSir6tauBvQIRTqnPhlJPhxuZijisoGCt3gFy7UiuC4zj/v md4BgH35PqYQn4zPMaS8O5Q3eeVHGKHz99HDl+7GLUOqTXMHllAm27pJ4QMmmSdJ FaumUVthzA6+CE0kkHsxFZ5QU42rfz7mpZIytw8psobg80+xphEA2EPQcQB0UgcF TXBPJbVr/Qbh/3WSTK3IX4PesMW4rumieRRmapwuw6VOvz9VIPE3xmHjmkR7r2GH F9apMBkMeD9hjN9ppURxuPJd+UEBpOrAuRR/4t3g3exoTjZUhOnANaCHcSNIaszx H1zcWWtKl6jbQ/NYsmo5ZXRI08iqiUNF/JV2lLeAoQaRnL4Bnwn4l2p5i4ijVhm/ NhS3uI6U+28TUB5UgrHXKtNS0EtsQD2LTpXwtjI5SB9eVSvyLkmF0clKwdwXOLIf voVd7vAkQ+tAQTqBM92B =+2X9 -----END PGP SIGNATURE----- --MQOKdVQJ8Vjc1Uz5qruwntr4Siq4VacOy-- From owner-freebsd-virtualization@freebsd.org Fri Mar 23 18:56:29 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DED6EF64708 for ; Fri, 23 Mar 2018 18:56:28 +0000 (UTC) (envelope-from nospam@ofloo.net) Received: from loki.ofloo.net (unknown [IPv6:2001:470:7acd:0:1:1:0:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "loki.ofloo.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 295BA866CB; Fri, 23 Mar 2018 18:56:28 +0000 (UTC) (envelope-from nospam@ofloo.net) Received: from [IPv6:fc03:1337:dead:cafe:448c:cc95:4a7e:75c1] ([IPv6:fc03:1337:dead:cafe:448c:cc95:4a7e:75c1]) by loki.ofloo.net (8.15.2/8.15.2) with ESMTPS id w2NItQBk047897 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Fri, 23 Mar 2018 18:55:33 GMT (envelope-from nospam@ofloo.net) Subject: Re: bhyve pci passthrough usb controller To: Julian Elischer , freebsd-virtualization@freebsd.org References: <5bd3c8b6-0579-0269-1ada-ab6f0a55ce8f@ofloo.net> <2ca981b4-9651-253a-7549-eaece4a58ff0@freebsd.org> From: Wouter Snels Message-ID: <22e0ee2a-34f7-aa57-34e2-5c7404874b08@ofloo.net> Date: Fri, 23 Mar 2018 19:55:20 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <2ca981b4-9651-253a-7549-eaece4a58ff0@freebsd.org> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="qthS3UFkRgwFy6iMIyYDNWesq0emuE6um" X-Spam-Status: No, score=-1.8 required=5.0 tests=ALL_TRUSTED,BAYES_00, RDNS_NONE,SPF_PASS,TW_EV,TW_SV,TW_TD,TW_XH autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on loki.ofloo.net X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 18:56:29 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --qthS3UFkRgwFy6iMIyYDNWesq0emuE6um Content-Type: multipart/mixed; boundary="GhkdO3wjnld3TXnqeciJi2eWAt4iAzfto"; protected-headers="v1" From: Wouter Snels To: Julian Elischer , freebsd-virtualization@freebsd.org Message-ID: <22e0ee2a-34f7-aa57-34e2-5c7404874b08@ofloo.net> Subject: Re: bhyve pci passthrough usb controller References: <5bd3c8b6-0579-0269-1ada-ab6f0a55ce8f@ofloo.net> <2ca981b4-9651-253a-7549-eaece4a58ff0@freebsd.org> In-Reply-To: <2ca981b4-9651-253a-7549-eaece4a58ff0@freebsd.org> --GhkdO3wjnld3TXnqeciJi2eWAt4iAzfto Content-Type: multipart/mixed; boundary="------------E256576D65DAFBE2BE83698C" Content-Language: nl-BE This is a multi-part message in MIME format. --------------E256576D65DAFBE2BE83698C Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi, .. I'm trying to pci passhthrough a usb controller to a bhyve windows 7 virtualization. When I start the virtual machine it freezes the entire system. See console.log To be clear the system runs without any problems without the pci passthrough. kernel configuration include GENERIC ident=C2=A0=C2=A0 OFL options IPFIREWALL options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options CONSPEED=3D115200 * note: removed xhci driver from GENERIC I've recompiled the kernel without the xhci driver. I've tested both usb controllers. Both "Sunrise Point-H USB 3.0 xHCI Controller" and "uPD720202 USB 3.0 Host Controller", both showed the same error on the console screen. # uname FreeBSD 11.1-STABLE #1 r331092M # custom kernel include GENERIC ident=C2=A0=C2=A0 OFL options IPFIREWALL options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options CONSPEED=3D115200 * note commented "device xhci" in generic # svn info Path: . Working Copy Root Path: /usr/src URL: https://svn0.eu.freebsd.org/base/stable/11 Relative URL: ^/stable/11 Repository Root: https://svn0.eu.freebsd.org/base Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind: directory Schedule: normal Last Changed Author: eadler Last Changed Rev: 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018) # vm passthru DEVICE=C2=A0=C2=A0=C2=A0=C2=A0 BHYVE ID=C2=A0=C2=A0=C2=A0=C2=A0 READY=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 DESCRIPTION hostb0=C2=A0=C2=A0=C2=A0=C2=A0 0/0/0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 No=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Xeon E= 3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers pcib1=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0/1/0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 No=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) vgapci0=C2=A0=C2=A0=C2=A0 0/2/0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= No=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 HD Graphi= cs 530 ppt0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0/20/0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 Yes=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Sunri= se Point-H USB 3.0 xHCI Controller none0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0/20/2=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 No=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Sunris= e Point-H Thermal subsystem none1=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0/22/0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 No=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Sunris= e Point-H CSME HECI ahci0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0/23/0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 No=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Sunris= e Point-H SATA controller [AHCI mode] pcib2=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0/28/0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 No=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Sunris= e Point-H PCI Express Root Port pcib4=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0/28/6=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 No=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Sunris= e Point-H PCI Express Root Port isab0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0/31/0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 No=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Sunris= e Point-H LPC Controller none2=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0/31/2=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 No=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Sunris= e Point-H PMC hdac0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0/31/3=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 No=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Sunris= e Point-H HD Audio none3=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0/31/4=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 No=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Sunris= e Point-H SMBus em0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0/31/6=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 No=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 Ethernet Connection (2) I219-V pcib3=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 2/0/0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 No=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = XIO2001 PCI Express-to-PCI Bridge xl0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 3/0/0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 No=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 3c905B 100BaseTX [Cyclone] ppt1=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 4/0/0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 Yes=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= uPD720202 USB 3.0 Host Controller ppt1@pci0:4:0:0:=C2=A0=C2=A0=C2=A0 class=3D0x0c0330 card=3D0x00151912 chi= p=3D0x00151912 rev=3D0x02 hdr=3D0x00 =C2=A0=C2=A0=C2=A0 vendor=C2=A0=C2=A0=C2=A0=C2=A0 =3D 'Renesas Technology= Corp.'/mailman =C2=A0=C2=A0=C2=A0 device=C2=A0=C2=A0=C2=A0=C2=A0 =3D 'uPD720202 USB 3.0 = Host Controller' =C2=A0=C2=A0=C2=A0 class=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D serial bus =C2=A0=C2=A0=C2=A0 subclass=C2=A0=C2=A0 =3D USB ppt0@pci0:0:20:0:=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 class=3D0x0c0330 card=3D0= x06ba1028 chip=3D0xa12f8086 rev=3D0x31 hdr=3D0x00 =C2=A0=C2=A0=C2=A0 vendor=C2=A0=C2=A0=C2=A0=C2=A0 =3D 'Intel Corporation'= =C2=A0=C2=A0=C2=A0 device=C2=A0=C2=A0=C2=A0=C2=A0 =3D 'Sunrise Point-H US= B 3.0 xHCI Controller' =C2=A0=C2=A0=C2=A0 class=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =3D serial bus =C2=A0=C2=A0=C2=A0 subclass=C2=A0=C2=A0 =3D USB # grep -i pptdevfs loader.conf pptdevs=3D"0/20/0 4/0/0" # vm configure passthru0=3D"0/20/0" # vm configure passthru0=3D"4/0/0" Any suggestions on what I'm doing wrong? * Sorry kinda new to this mailing list thing Op 23-03-18 om 18:25 schreef Julian Elischer: > On 23/3/18 11:40 pm, Wouter Snels wrote: > > > new wrapping is no better > It looks like the device you are passing though is not removed from > freebsd first. > but it's hard to tell. > >> |Hi, .. I'm trying to pci passhthrough a usb controller to a bhyve >> windows 7 virtualization. When I start the virtual machine it freezes >> the entire system and this shows up on the console. To be clear the >> system runs without any problems without the pci passthrough. | >> ||(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 e0 ff 40 40 29 = 00 >> 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid >> (ada0:ahcich0:0:0:0): Error 22, Unretryable error >> (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00= >> 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid >> (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ada0 at ahcich0 b= us >> 0 scbus0 target 0 lun 0 ada0: s/n ZA4292G6 >> detached (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40= >> 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was= >> invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: >> Poll timeout on slot 28 port 0 ahcich0: is 20000000 cs 10000000 ss >> 00000000 rs 10000000 tfd 1d0 serr 00000800 cmd 00005c17 >> (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 0= 0 >> 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted >> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 4= 0 >> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was >> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Pol= l >> timeout on slot 30 port 0 ahcich0: is 20000000 cs 40000000 ss 00000000= >> rs 40000000 tfd 1d0 serr 00000800 cmd 00005e17 (aprobe0:ahcich0:0:0:0)= : >> SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 >> (aprobe0:ahcich0:0:0:0): CAM status: Command timeout >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted >> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE WCACHE. ACB: ef 02 00 00 00 4= 0 >> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was >> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error >> (ada0:ahcich0:0:0:0): Periph destroyed em0: Watchdog timeout Queue[0]-= - >> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: = hw >> tdh =3D 428, hw tdt =3D 471 em0: Tx Queue Status =3D -2147483648 em0: = TX >> descriptors avail =3D 981 em0: Tx Descriptors avail failure =3D 0 em0:= RX >> Queue 0 ------ em0: hw rdh =3D 912, hw rdt =3D 908 em0: RX discarded p= ackets >> =3D 0 em0: RX Next to Check =3D 909 em0: RX Next to Refresh =3D 908 xl= 0: >> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface = is >> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D= 58 >> em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 966= em0: >> Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh = =3D 0, >> hw rdt =3D 1023 em0: RX discarded packets =3D 0 em0: RX Next to Check = =3D 0 >> em0: RX Next to Refresh =3D 1023 em0: Watchdog timeout Queue[0]-- >> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: = hw >> tdh =3D 0, hw tdt =3D 58 em0: Tx Queue Status =3D -2147483648 em0: TX >> descriptors avail =3D 966 em0: Tx Descriptors avail failure =3D 0 em0:= RX >> Queue 0 ------ em0: hw rdh =3D 0, hw rdt =3D 1023 em0: RX discarded pa= ckets >> =3D 0 em0: RX Next to Check =3D 0 em0: RX Next to Refresh =3D 1023 xl0= : >> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface = is >> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D= 54 >> em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 970= em0: >> Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh = =3D 0, >> hw rdt =3D 1023 em0: RX discarded packets =3D 0include GENERIC ident O= FL >> options IPFIREWALL|||||| >> >> |Hi, .. I'm trying to pci passhthrough a usb controller to a bhyve >> windows 7 virtualization. When I start the virtual machine it freezes >> the entire system and this shows up on the console. To be clear the >> system runs without any problems without the pci passthrough. | >> ||(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 e0 ff 40 40 29 = 00 >> 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid >> (ada0:ahcich0:0:0:0): Error 22, Unretryable error >> (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00= >> 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid >> (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ada0 at ahcich0 b= us >> 0 scbus0 target 0 lun 0 ada0: s/n ZA4292G6 >> detached (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40= >> 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was= >> invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: >> Poll timeout on slot 28 port 0 ahcich0: is 20000000 cs 10000000 ss >> 00000000 rs 10000000 tfd 1d0 serr 00000800 cmd 00005c17 >> (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 0= 0 >> 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted >> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 4= 0 >> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was >> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Pol= l >> timeout on slot 30 port 0 ahcich0: is 20000000 cs 40000000 ss 00000000= >> rs 40000000 tfd 1d0 serr 00000800 cmd 00005e17 (aprobe0:ahcich0:0:0:0)= : >> SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 >> (aprobe0:ahcich0:0:0:0): CAM status: Command timeout >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted >> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE WCACHE. ACB: ef 02 00 00 00 4= 0 >> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was >> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error >> (ada0:ahcich0:0:0:0): Periph destroyed em0: Watchdog timeout Queue[0]-= - >> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: = hw >> tdh =3D 428, hw tdt =3D 471 em0: Tx Queue Status =3D -2147483648 em0: = TX >> descriptors avail =3D 981 em0: Tx Descriptors avail failure =3D 0 em0:= RX >> Queue 0 ------ em0: hw rdh =3D 912, hw rdt =3D 908 em0: RX discarded p= ackets >> =3D 0 em0: RX Next to Check =3D 909 em0: RX Next to Refresh =3D 908 xl= 0: >> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface = is >> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D= 58 >> em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 966= em0: >> Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh = =3D 0, >> hw rdt =3D 1023 em0: RX discarded packets =3D 0 em0: RX Next to Check = =3D 0 >> em0: RX Next to Refresh =3D 1023 em0: Watchdog timeout Queue[0]-- >> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: = hw >> tdh =3D 0, hw tdt =3D 58 em0: Tx Queue Status =3D -2147483648 em0: TX >> descriptors avail =3D 966 em0: Tx Descriptors avail failure =3D 0 em0:= RX >> Queue 0 ------ em0: hw rdh =3D 0, hw rdt =3D 1023 em0: RX discarded pa= ckets >> =3D 0 em0: RX Next to Check =3D 0 em0: RX Next to Refresh =3D 1023 xl0= : >> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface = is >> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt =3D= 54 >> em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 970= em0: >> Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh = =3D 0, >> hw rdt =3D 1023 em0: RX discarded packets =3D 0include GENERIC ident O= FL >> options IPFIREWALL options IPFIREWALL_VERBOSE options >> IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options= >> CONSPEED=3D115200 em0: RX Next to Check =3D 0 em0: RX Next to Refresh = =3D >> 1023 |||| >> |||I've recompiled the kernel without the xhci driver. I've tested bot= h >> usb controllers. Both "|||||Sunrise Point-H USB 3.0 xHCI >> Controller|"||||| and "|||uPD720202 USB 3.0 Host Controller|", both >> showed the same error on the console screen.| # uname FreeBSD >> 11.1-STABLE #1 r331092M # custom kernel include GENERIC ident OFL >> options IPFIREWALL options IPFIREWALL_VERBOSE options >> IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options= >> CONSPEED=3D115200 * note commented "device xhci" in generic # svn info= >> |||Path: . Working Copy Root Path: /usr/src URL: >> https://svn0.eu.freebsd.org/base/stable/11 Relative URL: ^/stable/11 >> Repository Root: https://svn0.eu.freebsd.org/base Repository UUID: >> ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind: >> directory Schedule: normal Last Changed Author: eadler Last Changed Re= v: >> 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018)= >> |# vm passthru DEVICE BHYVE ID READY DESCRIPTION hostb0 0/0/0 No Xeon >> E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Register= s >> pcib1 0/1/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe >> Controller (x16) vgapci0 0/2/0 No HD Graphics 530 |||ppt0| 0/20/0 Yes >> Sunrise Point-H USB 3.0 xHCI Controller none0 0/20/2 >> No Sunrise Point-H Thermal subsystem none1 0/22/0 No Sunrise Point-H >> CSME HECI ahci0 0/23/0 No Sunrise Point-H SATA controller [AHCI mode] >> pcib2 0/28/0 No Sunrise Point-H PCI Express Root Port pcib4 0/28/6 No >> Sunrise Point-H PCI Express Root Port isab0 0/31/0 No Sunrise Point-H >> LPC Controller none2 0/31/2 No Sunrise Point-H PMC hdac0 0/31/3 No >> Sunrise Point-H HD Audio none3 0/31/4 No Sunrise Point-H SMBus em0 >> 0/31/6 No Ethernet Connection (2) I219-V pcib3 2/0/0 No XIO2001 PCI >> Express-to-PCI Bridge xl0 3/0/0 No 3c905B 100BaseTX [Cyclone] ppt1 4/0= /0 >> Yes uPD720202 USB 3.0 Host Controller ||ppt1@pci0:4:0:0: >> class=3D0x0c0330 card=3D0x00151912 chip=3D0x00151912 rev=3D0x02 >> hdr=3D0x00 vendor =3D 'Renesas Technology Corp.' device =3D 'uPD720202= USB 3.0 >> Host Controller' class =3D serial bus subclass =3D USB >> |||ppt|0@pci0:0:20:0: class=3D0x0c0330 card=3D0x06ba1028 chip=3D0xa12f= 8086 >> rev=3D0x31 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Sunri= se Point-H >> USB 3.0 xHCI Controller' class =3D serial bus subclass =3D USB # grep = -i >> pptdevfs loader.conf ||pptdevs=3D"0/20/0 4/0/0" # vm configure >> ||passthru0=3D"0/20/0"| >> >> |# vm configure ||passthru0=3D"4/0/0"| >> ||Any suggestions on what I'm doing wrong?|| >> =C2=A0 Regards, Ofloo >> |||||| >> >> ||options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT >> options >> DUMMYNET options IPDIVERT options CONSPEED=3D115200 em0: RX Next to Ch= eck >> =3D 0 em0: RX Next to Refresh =3D 1023 |||| >> |||I've recompiled the kernel without the xhci driver. I've tested bot= h >> usb controllers. Both "|||||Sunrise Point-H USB 3.0 xHCI >> Controller|"||||| and "|||uPD720202 USB 3.0 Host Controller|", both >> showed the same error on the console screen.| # uname FreeBSD >> 11.1-STABLE #1 r331092M # custom kernel include GENERIC ident OFL >> options IPFIREWALL options IPFIREWALL_VERBOSE options >> IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options= >> CONSPEED=3D115200 * note commented "device xhci" in generic # svn info= >> |||Path: . Working Copy Root Path: /usr/src URL: >> https://svn0.eu.freebsd.org/base/stable/11 Relative URL: ^/stable/11 >> Repository Root: https://svn0.eu.freebsd.org/base Repository UUID: >> ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind: >> directory Schedule: normal Last Changed Author: eadler Last Changed Re= v: >> 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018)= >> |# vm passthru DEVICE BHYVE ID READY DESCRIPTION hostb0 0/0/0 No Xeon >> E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Register= s >> pcib1 0/1/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe >> Controller (x16) vgapci0 0/2/0 No HD Graphics 530 |||ppt0| 0/20/0 Yes >> Sunrise Point-H USB 3.0 xHCI Controller none0 0/20/2 >> No Sunrise Point-H Thermal subsystem none1 0/22/0 No Sunrise Point-H >> CSME HECI ahci0 0/23/0 No Sunrise Point-H SATA controller [AHCI mode] >> pcib2 0/28/0 No Sunrise Point-H PCI Express Root Port pcib4 0/28/6 No >> Sunrise Point-H PCI Express Root Port isab0 0/31/0 No Sunrise Point-H >> LPC Controller none2 0/31/2 No Sunrise Point-H PMC hdac0 0/31/3 No >> Sunrise Point-H HD Audio none3 0/31/4 No Sunrise Point-H SMBus em0 >> 0/31/6 No Ethernet Connection (2) I219-V pcib3 2/0/0 No XIO2001 PCI >> Express-to-PCI Bridge xl0 3/0/0 No 3c905B 100BaseTX [Cyclone] ppt1 4/0= /0 >> Yes uPD720202 USB 3.0 Host Controller ||ppt1@pci0:4:0:0: >> class=3D0x0c0330 card=3D0x00151912 chip=3D0x00151912 rev=3D0x02 >> hdr=3D0x00 vendor =3D 'Renesas Technology Corp.' device =3D 'uPD720202= USB 3.0 >> Host Controller' class =3D serial bus subclass =3D USB >> |||ppt|0@pci0:0:20:0: class=3D0x0c0330 card=3D0x06ba1028 chip=3D0xa12f= 8086 >> rev=3D0x31 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Sunri= se Point-H >> USB 3.0 xHCI Controller' class =3D serial bus subclass =3D USB # grep = -i >> pptdevfs loader.conf ||pptdevs=3D"0/20/0 4/0/0" # vm configure >> ||passthru0=3D"0/20/0"| >> >> |# vm configure ||passthru0=3D"4/0/0"| >> ||Any suggestions on what I'm doing wrong?|| >> >> edit: previous wrapping was all wrong. >> =C2=A0=C2=A0 Regards, Ofloo >> |||||| >> >> --------------E256576D65DAFBE2BE83698C-- --GhkdO3wjnld3TXnqeciJi2eWAt4iAzfto-- --qthS3UFkRgwFy6iMIyYDNWesq0emuE6um Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJatU2YAAoJENWumses7+9mq+cP/i7VMhN9IA+Ea3JjBiEOlTbc L3wLoatZnUqlPOVV5llOjnM0/u0zLBr8LPfaC2Yf7jmlcJbP6iXqG6f8kuqtCwVg ISj0NzbwG/UnQMCypHOWgWO1XBSrpHO3rKQd+vfqIXoSDNOofeWhP1G1RvkAv1H/ JeOWkTdnRRZT7cPpJgirKUg+6uvQePVSBeTb0gDwxwGLYpWKvD+jklyhpYVgT3Ah gpg27UByvaDYGVVOWbB1869oV8n4tl0iqRTVc+bw6KhKx4BEcx6rz1yieOkoHjYY HreCuoFdPa7krVtYlSutNCC14PPcJPBaGJsxEXO168sx12Iri9UyQpG7h7Ftbwyi 8yLR2wkUJh+edRrL3qG+VGKFqBE4iYHJPKi0LDDD92I1s6h3keCSnVTz0qw/dyOo 8j6ZOXyYDNRHJwxdDdV5Du0u5MRnN5s83QHDanD+wijq2GG/yQL7865XwaTQzorC By3GsStKlsKDcDoca27617x0WVZ5+/LXdGZj6ntBdjKs3iu6hymEfaaq8lbQI3RV lKCHcvKhHng42AXq8ywUH25IqHWXrpw4j0bnDUVUCy2go7ECSQr3pjFUaNbBChFq BdZFb3EbTyEqD3kmHk+tCE1kKZF7mPamDlHEe4Xnu6P8UwpF0r28iICQ07Tp2Bdh MTaVmlBME6p21nuG4cDc =sMEh -----END PGP SIGNATURE----- --qthS3UFkRgwFy6iMIyYDNWesq0emuE6um-- From owner-freebsd-virtualization@freebsd.org Fri Mar 23 19:36:30 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6EDA4F67EEB for ; Fri, 23 Mar 2018 19:36:30 +0000 (UTC) (envelope-from byond.lenox@gmail.com) Received: from mail-lf0-f53.google.com (mail-lf0-f53.google.com [209.85.215.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BD8D988717; Fri, 23 Mar 2018 19:36:29 +0000 (UTC) (envelope-from byond.lenox@gmail.com) Received: by mail-lf0-f53.google.com with SMTP id v207-v6so19908597lfa.10; Fri, 23 Mar 2018 12:36:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=ubmZq/pzW1oMeMpRWCNuckOERrJYEJZGEy6ALHVtJsc=; b=EiPeyPFIyQgWEC6rq0baDKxJH1xpWhXTkThAoICmxJTNGsK8dLZ4NAvJM9JIMGxdai Ovi/EILxXHwt5WV+bDfdSdc9wIxq6RESrCJTRu0rK9LdxtuFLFUOB1XOnmCM38Iq0wDW S/82TXvt19PJwXJk8KlswoFrJCnaUwY7K1B9rVjMDh60jU9/uHlWX4zCIh2BSD5lt7Ur OivpHypteYSFkpIWdjErzXA+pZ30QktO6O9nMKGNXlx73/i6fZru8DmDim7HUpoB+x30 2VgPUOZ7r4MIj1/g5kWvZcJZERBQcJ59H6X0msey0zoduiGw4QHJRkw9d3UVGcSV+PkK ewNg== X-Gm-Message-State: AElRT7GAHi9WTBzzaKjx1sAoWJ2PCteSmQMlbnEZfCxCcMfvPI0OeBFb rneFHm6Fn6y26SasHGsvZejq4rMd X-Google-Smtp-Source: AG47ELtXzNHUr5ua+OzKgl+9KBBsF0ZQzxwVTrQoRz/e6Lwpvqw/K6IyAVfRlaZWJ247sILoe3TfqA== X-Received: by 10.46.80.71 with SMTP id v7mr8035738ljd.45.1521833761698; Fri, 23 Mar 2018 12:36:01 -0700 (PDT) Received: from mail-lf0-f41.google.com (mail-lf0-f41.google.com. [209.85.215.41]) by smtp.gmail.com with ESMTPSA id 31-v6sm2376375lfs.57.2018.03.23.12.36.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 23 Mar 2018 12:36:01 -0700 (PDT) Received: by mail-lf0-f41.google.com with SMTP id p142-v6so19902622lfd.6; Fri, 23 Mar 2018 12:36:01 -0700 (PDT) X-Received: by 10.46.155.204 with SMTP id w12mr2212906ljj.76.1521833761318; Fri, 23 Mar 2018 12:36:01 -0700 (PDT) MIME-Version: 1.0 Received: by 10.46.129.90 with HTTP; Fri, 23 Mar 2018 12:35:40 -0700 (PDT) In-Reply-To: References: <201803231812.w2NICLEW032460@pdx.rh.CN85.dnsmgr.net> From: Kyle Evans Date: Fri, 23 Mar 2018 14:35:40 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Issue encountered booting FreeBSD STABLE and CURRENT snapshots with EFI To: Kyle Evans Cc: "Rodney W. Grimes" , "freebsd-virtualization@freebsd.org" , Peter Grehan , Marcelo Araujo Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Mar 2018 19:36:30 -0000 On Fri, Mar 23, 2018 at 1:34 PM, Kyle Evans wrote: > On Fri, Mar 23, 2018 at 1:12 PM, Rodney W. Grimes > wrote: >>> On Fri, Mar 23, 2018 at 1:04 PM, Rodney W. Grimes >>> wrote: >>> > [ Charset UTF-8 unsupported, converting... ] >>> >> On Fri, Mar 23, 2018 at 9:58 AM, Rodney W. Grimes >>> >> wrote: >>> >> >> On Fri, Mar 23, 2018 at 3:56 AM, Joe Maloney wrote: >>> >> >> > We narrowed the issue down to how vm-bhyve attaches a null.iso when starting >>> >> >> > the VM. >>> >> >> > >>> >> >> >>> >> >> What exactly are the contents of this null.iso? It sounds like we're >>> >> > >>> >> > touch "null.iso" >>> >> > -rw-r--r-- 1 root wheel 0 Dec 3 22:55 /home/vmbhyve/.config/null.iso >>> >> > >>> >> > It is litterly a 0 byte file. This is just there to appease windows >>> >> > installer that you have a cd drive. >>> >> > >>> >> > You can test with bhyve yourself this problem by adding: >>> >> > -s 3:0,ahci-cd,${vm_dir}/.config/null.iso >>> >> > to your bhyve command. >>> >> > >>> >> >>> >> Thanks for that! A couple hundred printf's later, I think this might >>> >> be a firmware bug or something of the sort (?). >>> >> >>> >> In this loop [1], the address at 'dp' is getting immediately clobbered >>> >> by the efi_lookup_devpath on every iteration, so we end up looking at >>> >> basically all of the partitions and this null.iso unfortunately meets >>> >> this criteria [2] for removable media with no media present. >>> >> Therefore, we detect absolutely no disks. >>> >> >>> >> The only candidate in efi_lookup_devpath [3] is the firmware itself >>> >> having overwritten the devpath at that address. We can confirm this >>> >> with a printf just after the efi_lookup_devpath- the pointer it's >>> >> returning is in fact exactly match 'dp' every single time. >>> >> >>> >> Is there a newer/different firmware you can try? This seems like >>> >> really peculiar behavior for a firmware to be exhibiting. >>> > >>> > What happened to the boot device on the hd? >>> > There should be another -s x:0, ahci-hd drive that it boots from. >>> > -s 4:0,ahci-hd,${vm_dir}/${vmname}/disk0.img >>> > >>> >>> Right, it's there but we think it's removable media with no media >>> present because of the above-mentioned clobbering by the firmware. We >>> could probably work around it for this specific case, but I don't know >>> if that's a great idea- there be snakes here... >> >> And this also appers to works with the Windows boot code.. so I >> do suspect something has become broken in the new loader. >> > > Benno points out that this is due to a uefi-edk2 hack [1] that should > be ripped out now. To quote him and his quote of grehan: > > 13:23 <@benno> https://github.com/freebsd/uefi-edk2/blob/bhyve/UDK2014.SP1/BhyvePkg/Library/PlatformBdsLib/BdsPlatform.c#L1030 > 13:23 <@benno> If you short-circuit that if so it always returns > everything will be happier. > 13:24 <@benno> Quoting grehan: > 13:24 < kevans91> Oh no > 13:25 < kevans91> ew > 13:25 <@benno> "The change was put in as a workaround for a bug in the > FreeBSD/UEFI loader, however, that issue was fixed with > r285246. > 13:25 <@benno> I'd think it unlikely that anyone would want to run <= > 10.1 with bhyve/UEFI, so I the intercept could safely be > removed." > 13:25 <@benno> So the proper fix is to remove the override entirely. > 13:25 <@benno> But in my testing I only got as far as just nerfing it > via sticking a 1 || in front of the if condition. > > I don't think we want to workaround this in loader, and I can't > explain why this is functional with Windows since it's Windows- if I > could review some of the relevant source, I would. > > Also CC'ing grehan@, since he's the one that knows the stuff. > > [1] https://github.com/freebsd/uefi-edk2/blob/bhyve/UDK2014.SP1/BhyvePkg/Library/PlatformBdsLib/BdsPlatform.c#L1008 FYI- I've created PR #2 [1] against the freebsd/uefi-edk2 repository and have confirmed that this fixes the broken-looking firmware along with the booting problem experienced in this thread. [1] https://github.com/freebsd/uefi-edk2/pull/2 From owner-freebsd-virtualization@freebsd.org Sat Mar 24 02:30:09 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9F21DF62444 for ; Sat, 24 Mar 2018 02:30:09 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B05D072374; Sat, 24 Mar 2018 02:30:08 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w2O2U43R033918; Fri, 23 Mar 2018 19:30:04 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w2O2U21F033917; Fri, 23 Mar 2018 19:30:02 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201803240230.w2O2U21F033917@pdx.rh.CN85.dnsmgr.net> Subject: Re: bhyve pci passthrough usb controller In-Reply-To: <22e0ee2a-34f7-aa57-34e2-5c7404874b08@ofloo.net> To: Wouter Snels Date: Fri, 23 Mar 2018 19:30:02 -0700 (PDT) CC: Julian Elischer , freebsd-virtualization@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Mar 2018 02:30:10 -0000 -- Start of PGP signed section. > Hi, .. > > I'm trying to pci passhthrough a usb controller to a bhyve windows 7 > virtualization. > When I start the virtual machine it freezes the entire system. See > console.log > To be clear the system runs without any problems without the pci > passthrough. > > kernel configuration > > include GENERIC > ident?? OFL > options IPFIREWALL > options IPFIREWALL_VERBOSE > options IPFIREWALL_DEFAULT_TO_ACCEPT > options DUMMYNET > options IPDIVERT > options CONSPEED=115200 > > * note: removed xhci driver from GENERIC > > > I've recompiled the kernel without the xhci driver. I've tested both usb > controllers. > Both "Sunrise Point-H USB 3.0 xHCI Controller" and "uPD720202 USB 3.0 > Host Controller", both showed > the same error on the console screen. > > # uname > FreeBSD 11.1-STABLE #1 r331092M > > # custom kernel > include GENERIC > ident?? OFL > options IPFIREWALL > options IPFIREWALL_VERBOSE > options IPFIREWALL_DEFAULT_TO_ACCEPT > options DUMMYNET > options IPDIVERT > options CONSPEED=115200 > > * note commented "device xhci" in generic > > # svn info > Path: . > Working Copy Root Path: /usr/src > URL: https://svn0.eu.freebsd.org/base/stable/11 > Relative URL: ^/stable/11 > Repository Root: https://svn0.eu.freebsd.org/base > Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f > Revision: 331092 > Node Kind: directory > Schedule: normal > Last Changed Author: eadler > Last Changed Rev: 331092 > Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018) > > # vm passthru > DEVICE???? BHYVE ID???? READY??????? DESCRIPTION > hostb0???? 0/0/0??????? No?????????? Xeon E3-1200 v5/E3-1500 v5/6th Gen > Core Processor Host Bridge/DRAM Registers > pcib1????? 0/1/0??????? No?????????? Xeon E3-1200 v5/E3-1500 v5/6th Gen > Core Processor PCIe Controller (x16) > vgapci0??? 0/2/0??????? No?????????? HD Graphics 530 > ppt0?????? 0/20/0?????? Yes????????? Sunrise Point-H USB 3.0 xHCI Controller 0/20/0 > none0????? 0/20/2?????? No?????????? Sunrise Point-H Thermal subsystem has a subdevice 0/20/2 that is not being passed, this may cause odd things to happen. > none1????? 0/22/0?????? No?????????? Sunrise Point-H CSME HECI > ahci0????? 0/23/0?????? No?????????? Sunrise Point-H SATA controller > [AHCI mode] > pcib2????? 0/28/0?????? No?????????? Sunrise Point-H PCI Express Root Port > pcib4????? 0/28/6?????? No?????????? Sunrise Point-H PCI Express Root Port > isab0????? 0/31/0?????? No?????????? Sunrise Point-H LPC Controller > none2????? 0/31/2?????? No?????????? Sunrise Point-H PMC > hdac0????? 0/31/3?????? No?????????? Sunrise Point-H HD Audio > none3????? 0/31/4?????? No?????????? Sunrise Point-H SMBus > em0??????? 0/31/6?????? No?????????? Ethernet Connection (2) I219-V > pcib3????? 2/0/0??????? No?????????? XIO2001 PCI Express-to-PCI Bridge > xl0??????? 3/0/0??????? No?????????? 3c905B 100BaseTX [Cyclone] > ppt1?????? 4/0/0??????? Yes????????? uPD720202 USB 3.0 Host Controller > > ppt1@pci0:4:0:0:??? class=0x0c0330 card=0x00151912 chip=0x00151912 > rev=0x02 hdr=0x00 > ??? vendor???? = 'Renesas Technology Corp.'/mailman > ??? device???? = 'uPD720202 USB 3.0 Host Controller' > ??? class????? = serial bus > ??? subclass?? = USB > > ppt0@pci0:0:20:0:????? class=0x0c0330 card=0x06ba1028 chip=0xa12f8086 > rev=0x31 hdr=0x00 > ??? vendor???? = 'Intel Corporation' > ??? device???? = 'Sunrise Point-H USB 3.0 xHCI Controller' > ??? class????? = serial bus > ??? subclass?? = USB > > # grep -i pptdevfs loader.conf > pptdevs="0/20/0 4/0/0" add 0/20/2 to this > > # vm configure > passthru0="0/20/0" > > # vm configure > passthru0="4/0/0" > > Any suggestions on what I'm doing wrong? > > > * Sorry kinda new to this mailing list thing > > > Op 23-03-18 om 18:25 schreef Julian Elischer: > > On 23/3/18 11:40 pm, Wouter Snels wrote: > > > > > > new wrapping is no better > > It looks like the device you are passing though is not removed from > > freebsd first. > > but it's hard to tell. > > > >> |Hi, .. I'm trying to pci passhthrough a usb controller to a bhyve > >> windows 7 virtualization. When I start the virtual machine it freezes > >> the entire system and this shows up on the console. To be clear the > >> system runs without any problems without the pci passthrough. | > >> ||(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 e0 ff 40 40 29 00 > >> 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid > >> (ada0:ahcich0:0:0:0): Error 22, Unretryable error > >> (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 > >> 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid > >> (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ada0 at ahcich0 bus > >> 0 scbus0 target 0 lun 0 ada0: s/n ZA4292G6 > >> detached (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 > >> 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was > >> invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: > >> Poll timeout on slot 28 port 0 ahcich0: is 20000000 cs 10000000 ss > >> 00000000 rs 10000000 tfd 1d0 serr 00000800 cmd 00005c17 > >> (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 > >> 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout > >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted > >> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 40 > >> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was > >> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Poll > >> timeout on slot 30 port 0 ahcich0: is 20000000 cs 40000000 ss 00000000 > >> rs 40000000 tfd 1d0 serr 00000800 cmd 00005e17 (aprobe0:ahcich0:0:0:0): > >> SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 > >> (aprobe0:ahcich0:0:0:0): CAM status: Command timeout > >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted > >> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE WCACHE. ACB: ef 02 00 00 00 40 > >> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was > >> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error > >> (ada0:ahcich0:0:0:0): Periph destroyed em0: Watchdog timeout Queue[0]-- > >> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw > >> tdh = 428, hw tdt = 471 em0: Tx Queue Status = -2147483648 em0: TX > >> descriptors avail = 981 em0: Tx Descriptors avail failure = 0 em0: RX > >> Queue 0 ------ em0: hw rdh = 912, hw rdt = 908 em0: RX discarded packets > >> = 0 em0: RX Next to Check = 909 em0: RX Next to Refresh = 908 xl0: > >> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is > >> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh = 0, hw tdt = 58 > >> em0: Tx Queue Status = -2147483648 em0: TX descriptors avail = 966 em0: > >> Tx Descriptors avail failure = 0 em0: RX Queue 0 ------ em0: hw rdh = 0, > >> hw rdt = 1023 em0: RX discarded packets = 0 em0: RX Next to Check = 0 > >> em0: RX Next to Refresh = 1023 em0: Watchdog timeout Queue[0]-- > >> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw > >> tdh = 0, hw tdt = 58 em0: Tx Queue Status = -2147483648 em0: TX > >> descriptors avail = 966 em0: Tx Descriptors avail failure = 0 em0: RX > >> Queue 0 ------ em0: hw rdh = 0, hw rdt = 1023 em0: RX discarded packets > >> = 0 em0: RX Next to Check = 0 em0: RX Next to Refresh = 1023 xl0: > >> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is > >> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh = 0, hw tdt = 54 > >> em0: Tx Queue Status = -2147483648 em0: TX descriptors avail = 970 em0: > >> Tx Descriptors avail failure = 0 em0: RX Queue 0 ------ em0: hw rdh = 0, > >> hw rdt = 1023 em0: RX discarded packets = 0include GENERIC ident OFL > >> options IPFIREWALL|||||| > >> > >> |Hi, .. I'm trying to pci passhthrough a usb controller to a bhyve > >> windows 7 virtualization. When I start the virtual machine it freezes > >> the entire system and this shows up on the console. To be clear the > >> system runs without any problems without the pci passthrough. | > >> ||(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 e0 ff 40 40 29 00 > >> 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid > >> (ada0:ahcich0:0:0:0): Error 22, Unretryable error > >> (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 > >> 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid > >> (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ada0 at ahcich0 bus > >> 0 scbus0 target 0 lun 0 ada0: s/n ZA4292G6 > >> detached (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 > >> 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was > >> invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: > >> Poll timeout on slot 28 port 0 ahcich0: is 20000000 cs 10000000 ss > >> 00000000 rs 10000000 tfd 1d0 serr 00000800 cmd 00005c17 > >> (aprobe0:ahcich0:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 > >> 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout > >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted > >> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 40 > >> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was > >> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Poll > >> timeout on slot 30 port 0 ahcich0: is 20000000 cs 40000000 ss 00000000 > >> rs 40000000 tfd 1d0 serr 00000800 cmd 00005e17 (aprobe0:ahcich0:0:0:0): > >> SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00 > >> (aprobe0:ahcich0:0:0:0): CAM status: Command timeout > >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted > >> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE WCACHE. ACB: ef 02 00 00 00 40 > >> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was > >> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error > >> (ada0:ahcich0:0:0:0): Periph destroyed em0: Watchdog timeout Queue[0]-- > >> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw > >> tdh = 428, hw tdt = 471 em0: Tx Queue Status = -2147483648 em0: TX > >> descriptors avail = 981 em0: Tx Descriptors avail failure = 0 em0: RX > >> Queue 0 ------ em0: hw rdh = 912, hw rdt = 908 em0: RX discarded packets > >> = 0 em0: RX Next to Check = 909 em0: RX Next to Refresh = 908 xl0: > >> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is > >> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh = 0, hw tdt = 58 > >> em0: Tx Queue Status = -2147483648 em0: TX descriptors avail = 966 em0: > >> Tx Descriptors avail failure = 0 em0: RX Queue 0 ------ em0: hw rdh = 0, > >> hw rdt = 1023 em0: RX discarded packets = 0 em0: RX Next to Check = 0 > >> em0: RX Next to Refresh = 1023 em0: Watchdog timeout Queue[0]-- > >> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw > >> tdh = 0, hw tdt = 58 em0: Tx Queue Status = -2147483648 em0: TX > >> descriptors avail = 966 em0: Tx Descriptors avail failure = 0 em0: RX > >> Queue 0 ------ em0: hw rdh = 0, hw rdt = 1023 em0: RX discarded packets > >> = 0 em0: RX Next to Check = 0 em0: RX Next to Refresh = 1023 xl0: > >> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface is > >> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh = 0, hw tdt = 54 > >> em0: Tx Queue Status = -2147483648 em0: TX descriptors avail = 970 em0: > >> Tx Descriptors avail failure = 0 em0: RX Queue 0 ------ em0: hw rdh = 0, > >> hw rdt = 1023 em0: RX discarded packets = 0include GENERIC ident OFL > >> options IPFIREWALL options IPFIREWALL_VERBOSE options > >> IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options > >> CONSPEED=115200 em0: RX Next to Check = 0 em0: RX Next to Refresh = > >> 1023 |||| > >> |||I've recompiled the kernel without the xhci driver. I've tested both > >> usb controllers. Both "|||||Sunrise Point-H USB 3.0 xHCI > >> Controller|"||||| and "|||uPD720202 USB 3.0 Host Controller|", both > >> showed the same error on the console screen.| # uname FreeBSD > >> 11.1-STABLE #1 r331092M # custom kernel include GENERIC ident OFL > >> options IPFIREWALL options IPFIREWALL_VERBOSE options > >> IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options > >> CONSPEED=115200 * note commented "device xhci" in generic # svn info > >> |||Path: . Working Copy Root Path: /usr/src URL: > >> https://svn0.eu.freebsd.org/base/stable/11 Relative URL: ^/stable/11 > >> Repository Root: https://svn0.eu.freebsd.org/base Repository UUID: > >> ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind: > >> directory Schedule: normal Last Changed Author: eadler Last Changed Rev: > >> 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018) > >> |# vm passthru DEVICE BHYVE ID READY DESCRIPTION hostb0 0/0/0 No Xeon > >> E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers > >> pcib1 0/1/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe > >> Controller (x16) vgapci0 0/2/0 No HD Graphics 530 |||ppt0| 0/20/0 Yes > >> Sunrise Point-H USB 3.0 xHCI Controller none0 0/20/2 > >> No Sunrise Point-H Thermal subsystem none1 0/22/0 No Sunrise Point-H > >> CSME HECI ahci0 0/23/0 No Sunrise Point-H SATA controller [AHCI mode] > >> pcib2 0/28/0 No Sunrise Point-H PCI Express Root Port pcib4 0/28/6 No > >> Sunrise Point-H PCI Express Root Port isab0 0/31/0 No Sunrise Point-H > >> LPC Controller none2 0/31/2 No Sunrise Point-H PMC hdac0 0/31/3 No > >> Sunrise Point-H HD Audio none3 0/31/4 No Sunrise Point-H SMBus em0 > >> 0/31/6 No Ethernet Connection (2) I219-V pcib3 2/0/0 No XIO2001 PCI > >> Express-to-PCI Bridge xl0 3/0/0 No 3c905B 100BaseTX [Cyclone] ppt1 4/0/0 > >> Yes uPD720202 USB 3.0 Host Controller ||ppt1@pci0:4:0:0: > >> class=0x0c0330 card=0x00151912 chip=0x00151912 rev=0x02 > >> hdr=0x00 vendor = 'Renesas Technology Corp.' device = 'uPD720202 USB 3.0 > >> Host Controller' class = serial bus subclass = USB > >> |||ppt|0@pci0:0:20:0: class=0x0c0330 card=0x06ba1028 chip=0xa12f8086 > >> rev=0x31 hdr=0x00 vendor = 'Intel Corporation' device = 'Sunrise Point-H > >> USB 3.0 xHCI Controller' class = serial bus subclass = USB # grep -i > >> pptdevfs loader.conf ||pptdevs="0/20/0 4/0/0" # vm configure > >> ||passthru0="0/20/0"| > >> > >> |# vm configure ||passthru0="4/0/0"| > >> ||Any suggestions on what I'm doing wrong?|| > >> ? Regards, Ofloo > >> |||||| > >> > >> ||options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT > >> options > >> DUMMYNET options IPDIVERT options CONSPEED=115200 em0: RX Next to Check > >> = 0 em0: RX Next to Refresh = 1023 |||| > >> |||I've recompiled the kernel without the xhci driver. I've tested both > >> usb controllers. Both "|||||Sunrise Point-H USB 3.0 xHCI > >> Controller|"||||| and "|||uPD720202 USB 3.0 Host Controller|", both > >> showed the same error on the console screen.| # uname FreeBSD > >> 11.1-STABLE #1 r331092M # custom kernel include GENERIC ident OFL > >> options IPFIREWALL options IPFIREWALL_VERBOSE options > >> IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT options > >> CONSPEED=115200 * note commented "device xhci" in generic # svn info > >> |||Path: . Working Copy Root Path: /usr/src URL: > >> https://svn0.eu.freebsd.org/base/stable/11 Relative URL: ^/stable/11 > >> Repository Root: https://svn0.eu.freebsd.org/base Repository UUID: > >> ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind: > >> directory Schedule: normal Last Changed Author: eadler Last Changed Rev: > >> 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018) > >> |# vm passthru DEVICE BHYVE ID READY DESCRIPTION hostb0 0/0/0 No Xeon > >> E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers > >> pcib1 0/1/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe > >> Controller (x16) vgapci0 0/2/0 No HD Graphics 530 |||ppt0| 0/20/0 Yes > >> Sunrise Point-H USB 3.0 xHCI Controller none0 0/20/2 > >> No Sunrise Point-H Thermal subsystem none1 0/22/0 No Sunrise Point-H > >> CSME HECI ahci0 0/23/0 No Sunrise Point-H SATA controller [AHCI mode] > >> pcib2 0/28/0 No Sunrise Point-H PCI Express Root Port pcib4 0/28/6 No > >> Sunrise Point-H PCI Express Root Port isab0 0/31/0 No Sunrise Point-H > >> LPC Controller none2 0/31/2 No Sunrise Point-H PMC hdac0 0/31/3 No > >> Sunrise Point-H HD Audio none3 0/31/4 No Sunrise Point-H SMBus em0 > >> 0/31/6 No Ethernet Connection (2) I219-V pcib3 2/0/0 No XIO2001 PCI > >> Express-to-PCI Bridge xl0 3/0/0 No 3c905B 100BaseTX [Cyclone] ppt1 4/0/0 > >> Yes uPD720202 USB 3.0 Host Controller ||ppt1@pci0:4:0:0: > >> class=0x0c0330 card=0x00151912 chip=0x00151912 rev=0x02 > >> hdr=0x00 vendor = 'Renesas Technology Corp.' device = 'uPD720202 USB 3.0 > >> Host Controller' class = serial bus subclass = USB > >> |||ppt|0@pci0:0:20:0: class=0x0c0330 card=0x06ba1028 chip=0xa12f8086 > >> rev=0x31 hdr=0x00 vendor = 'Intel Corporation' device = 'Sunrise Point-H > >> USB 3.0 xHCI Controller' class = serial bus subclass = USB # grep -i > >> pptdevfs loader.conf ||pptdevs="0/20/0 4/0/0" # vm configure > >> ||passthru0="0/20/0"| > >> > >> |# vm configure ||passthru0="4/0/0"| > >> ||Any suggestions on what I'm doing wrong?|| > >> > >> edit: previous wrapping was all wrong. > >> ?? Regards, Ofloo > >> |||||| > >> > >> > -- End of PGP section, PGP failed! -- Rod Grimes rgrimes@freebsd.org From owner-freebsd-virtualization@freebsd.org Sat Mar 24 05:59:26 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 59F4FF6F8FF for ; Sat, 24 Mar 2018 05:59:26 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from alto.onthenet.com.au (alto.OntheNet.com.au [203.13.68.12]) by mx1.freebsd.org (Postfix) with ESMTP id C15C87AB61 for ; Sat, 24 Mar 2018 05:59:25 +0000 (UTC) (envelope-from grehan@freebsd.org) Received: from iredmail.onthenet.com.au (iredmail.onthenet.com.au [203.13.68.150]) by alto.onthenet.com.au (Postfix) with ESMTPS id E555B20B4BB4 for ; Sat, 24 Mar 2018 15:59:22 +1000 (AEST) Received: from localhost (iredmail.onthenet.com.au [127.0.0.1]) by iredmail.onthenet.com.au (Postfix) with ESMTP id DE9252820AD for ; Sat, 24 Mar 2018 15:59:22 +1000 (AEST) X-Amavis-Modified: Mail body modified (using disclaimer) - iredmail.onthenet.com.au Received: from iredmail.onthenet.com.au ([127.0.0.1]) by localhost (iredmail.onthenet.com.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id E8NhfLKpEiui for ; Sat, 24 Mar 2018 15:59:22 +1000 (AEST) Received: from Peters-MacBook-Pro-2.local (c-67-180-92-13.hsd1.ca.comcast.net [67.180.92.13]) by iredmail.onthenet.com.au (Postfix) with ESMTPSA id 3F3362804D6; Sat, 24 Mar 2018 15:59:20 +1000 (AEST) Subject: Re: Issue encountered booting FreeBSD STABLE and CURRENT snapshots with EFI To: Kyle Evans Cc: "Rodney W. Grimes" , "freebsd-virtualization@freebsd.org" , Marcelo Araujo References: <201803231812.w2NICLEW032460@pdx.rh.CN85.dnsmgr.net> From: Peter Grehan Message-ID: <624f4ac2-ada1-64af-1c4f-250d3de551de@freebsd.org> Date: Fri, 23 Mar 2018 22:59:19 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-CMAE-Score: 0 X-CMAE-Analysis: v=2.3 cv=dNCIZtRb c=1 sm=1 tr=0 a=A6CF0fG5TOl4vs6YHvqXgw==:117 a=5eVCmCvhg37cu/pjidAGzw==:17 a=IkcTkHD0fZMA:10 a=v2DPQv5-lfwA:10 a=NEAV23lmAAAA:8 a=v-5Oyjvk_WBqChVrRF0A:9 a=QEXdDO2ut3YA:10 a=bAsVuuXFBG0A:10 a=VlZU0XKO32wA:10 wl=host:3 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Mar 2018 05:59:26 -0000 Hi Kyle, > FYI- I've created PR #2 [1] against the freebsd/uefi-edk2 repository > and have confirmed that this fixes the broken-looking firmware along > with the booting problem experienced in this thread. > > [1] https://github.com/freebsd/uefi-edk2/pull/2 Thanks - merged. I've asked Fabian to update the UEFI ports. later, Peter. From owner-freebsd-virtualization@freebsd.org Sat Mar 24 06:48:52 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DDA7CF4D031 for ; Sat, 24 Mar 2018 06:48:51 +0000 (UTC) (envelope-from nospam@ofloo.net) Received: from loki.ofloo.net (unknown [IPv6:2001:470:7acd:0:1:1:0:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "loki.ofloo.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3925E7C9D0; Sat, 24 Mar 2018 06:48:51 +0000 (UTC) (envelope-from nospam@ofloo.net) Received: from [10.13.21.20] ([10.13.21.20]) by loki.ofloo.net (8.15.2/8.15.2) with ESMTPS id w2O6lbOj087334 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 24 Mar 2018 06:47:46 GMT (envelope-from nospam@ofloo.net) In-Reply-To: <201803240230.w2O2U21F033917@pdx.rh.CN85.dnsmgr.net> References: <201803240230.w2O2U21F033917@pdx.rh.CN85.dnsmgr.net> X-Referenced-Uid: 34773 Thread-Topic: Re: bhyve pci passthrough usb controller User-Agent: Android X-Is-Generated-Message-Id: true MIME-Version: 1.0 Subject: Re: bhyve pci passthrough usb controller From: Wouter Snels Date: Sat, 24 Mar 2018 07:47:31 +0100 To: "Rodney W. Grimes" CC: Julian Elischer , freebsd-virtualization@freebsd.org Message-ID: X-Spam-Status: No, score=-1.8 required=5.0 tests=ALL_TRUSTED,BAYES_00, HTML_MESSAGE,RDNS_NONE,SPF_PASS,TW_EV,TW_SV,TW_TD,TW_XH autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on loki.ofloo.net Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Mar 2018 06:48:52 -0000 I surely will try this out, but what about device 4/0/0 showing the same is= sue=2E Both tests where performed separately showing the exact same result,= of course I'd be happy if either one of them worked=2E Regards,=2E=2E = -------- Oorspronkelijk bericht -------- Van: "Rodney W=2E Grimes" Verzonden: Sat Mar 24 03:30:02 GMT+01:= 00 2018 Aan: Wouter Snels Cc: Julian Elischer , freebsd-virtualization@freebsd=2Eorg Onderwerp: Re: bhyve p= ci passthrough usb controller -- Start of PGP signed section=2E > Hi, =2E=2E >=20 > I'm trying to pci passhthrough a usb controller to a bhyve windows 7 > virtualization=2E > When I start the virtual machine it freezes the entire system=2E See > console=2Elog > To be clear the system runs without any problems without the pci > passthrough=2E >=20 > kernel configuration >=20 > include GENERIC > ident?? OFL > options IPFIREWALL > options IPFIREWALL_VERBOSE > options IPFIREWALL_DEFAULT_TO_ACCEPT > options DUMMYNET > options IPDIVERT > options CONSPEED=3D115200 >=20 > * note: removed xhci driver from GENERIC >=20 >=20 > I've recompiled the kernel without the xhci driver=2E I've tested both u= sb > controllers=2E > Both "Sunrise Point-H USB 3=2E0 xHCI Controller" and "uPD720202 USB 3=2E= 0 > Host Controller", both showed > the same error on the console screen=2E >=20 > # uname > FreeBSD 11=2E1-STABLE #1 r331092M >=20 > # custom kernel > include GENERIC > ident?? OFL > options IPFIREWALL > options IPFIREWALL_VERBOSE > options IPFIREWALL_DEFAULT_TO_ACCEPT > options DUMMYNET > options IPDIVERT > options CONSPEED=3D115200 >=20 > * note commented "device xhci" in generic >=20 > # svn info > Path: =2E > Working Copy Root Path: /usr/src > URL: https://svn0=2Eeu=2Efreebsd=2Eorg/base/stable/11 > Relative URL: ^/stable/11 > Repository Root: https://svn0=2Eeu=2Efreebsd=2Eorg/base > Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f > Revision: 331092 > Node Kind: directory > Schedule: normal > Last Changed Author: eadler > Last Changed Rev: 331092 > Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018) >=20 > # vm passthru > DEVICE???? BHYVE ID???? READY??????? DESCRIPTION > hostb0???? 0/0/0??????? No?????????? Xeon E3-1200 v5/E3-1500 v5/6th Gen > Core Processor Host Bridge/DRAM Registers > pcib1????? 0/1/0??????? No?????????? Xeon E3-1200 v5/E3-1500 v5/6th Gen > Core Processor PCIe Controller (x16) > vgapci0??? 0/2/0??????? No?????????? HD Graphics 530 > ppt0?????? 0/20/0?????? Yes????????? Sunrise Point-H USB 3=2E0 xHCI Cont= roller 0/20/0 > none0????? 0/20/2?????? No?????????? Sunrise Point-H Thermal subsystem has a subdevice 0/20/2 that is not being passed, this may cause odd things to happen=2E > none1????? 0/22/0?????? No?????????? Sunrise Point-H CSME HECI > ahci0????? 0/23/0?????? No?????????? Sunrise Point-H SATA controller > [AHCI mode] > pcib2????? 0/28/0?????? No?????????? Sunrise Point-H PCI Express Root Po= rt > pcib4????? 0/28/6?????? No?????????? Sunrise Point-H PCI Express Root Po= rt > isab0????? 0/31/0?????? No?????????? Sunrise Point-H LPC Controller > none2????? 0/31/2?????? No?????????? Sunrise Point-H PMC > hdac0????? 0/31/3?????? No?????????? Sunrise Point-H HD Audio > none3????? 0/31/4?????? No?????????? Sunrise Point-H SMBus > em0??????? 0/31/6?????? No?????????? Ethernet Connection (2) I219-V > pcib3????? 2/0/0??????? No?????????? XIO2001 PCI Express-to-PCI Bridge > xl0??????? 3/0/0??????? No?????????? 3c905B 100BaseTX [Cyclone] > ppt1?????? 4/0/0??????? Yes????????? uPD720202 USB 3=2E0 Host Controller >=20 > ppt1@pci0:4:0:0:??? class=3D0x0c0330 card=3D0x00151912 chip=3D0x00151912 > rev=3D0x02 hdr=3D0x00 > ??? vendor???? =3D 'Renesas Technology Corp=2E'/mailman > ??? device???? =3D 'uPD720202 USB 3=2E0 Host Controller' > ??? class????? =3D serial bus > ??? subclass?? =3D USB >=20 > ppt0@pci0:0:20:0:????? class=3D0x0c0330 card=3D0x06ba1028 chip=3D0xa12f8= 086 > rev=3D0x31 hdr=3D0x00 > ??? vendor???? =3D 'Intel Corporation' > ??? device???? =3D 'Sunrise Point-H USB 3=2E0 xHCI Controller' > ??? class????? =3D serial bus > ??? subclass?? =3D USB >=20 > # grep -i pptdevfs loader=2Econf > pptdevs=3D"0/20/0 4/0/0" add 0/20/2 to this >=20 > # vm configure > passthru0=3D"0/20/0" >=20 > # vm configure > passthru0=3D"4/0/0" >=20 > Any suggestions on what I'm doing wrong? >=20 >=20 > * Sorry kinda new to this mailing list thing >=20 >=20 > Op 23-03-18 om 18:25 schreef Julian Elischer: > > On 23/3/18 11:40 pm, Wouter Snels wrote: > > > > > > new wrapping is no better > > It looks like the device you are passing though is not removed from > > freebsd first=2E > > but it's hard to tell=2E > > > >> |Hi, =2E=2E I'm trying to pci passhthrough a usb controller to a bhyv= e > >> windows 7 virtualization=2E When I start the virtual machine it freez= es > >> the entire system and this shows up on the console=2E To be clear the > >> system runs without any problems without the pci passthrough=2E | > >> ||(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED=2E ACB: 61 08 e0 ff 40 40 = 29 00 > >> 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid > >> (ada0:ahcich0:0:0:0): Error 22, Unretryable error > >> (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY=2E ACB: ec 00 00 00 00 40 00 00= 00 > >> 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid > >> (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ada0 at ahcich0 = bus > >> 0 scbus0 target 0 lun 0 ada0: s/n ZA4292G6 > >> detached (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY=2E ACB: ec 00 00 00 00= 40 > >> 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request wa= s > >> invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: > >> Poll timeout on slot 28 port 0 ahcich0: is 20000000 cs 10000000 ss > >> 00000000 rs 10000000 tfd 1d0 serr 00000800 cmd 00005c17 > >> (aprobe0:ahcich0:0:0:0): SOFT_RESET=2E ACB: 00 00 00 00 00 00 00 00 0= 0 00 > >> 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout > >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted > >> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE RCACHE=2E ACB: ef aa 00 00 0= 0 40 > >> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was > >> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Po= ll > >> timeout on slot 30 port 0 ahcich0: is 20000000 cs 40000000 ss 0000000= 0 > >> rs 40000000 tfd 1d0 serr 00000800 cmd 00005e17 (aprobe0:ahcich0:0:0:0= ): > >> SOFT_RESET=2E ACB: 00 00 00 00 00 00 00 00 00 00 00 00 > >> (aprobe0:ahcich0:0:0:0): CAM status: Command timeout > >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted > >> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE WCACHE=2E ACB: ef 02 00 00 0= 0 40 > >> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was > >> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error > >> (ada0:ahcich0:0:0:0): Periph destroyed em0: Watchdog timeout Queue[0]= -- > >> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0:= hw > >> tdh =3D 428, hw tdt =3D 471 em0: Tx Queue Status =3D -2147483648 em0:= TX > >> descriptors avail =3D 981 em0: Tx Descriptors avail failure =3D 0 em0= : RX > >> Queue 0 ------ em0: hw rdh =3D 912, hw rdt =3D 908 em0: RX discarded = packets > >> =3D 0 em0: RX Next to Check =3D 909 em0: RX Next to Refresh =3D 908 x= l0: > >> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface= is > >> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt = =3D 58 > >> em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 96= 6 em0: > >> Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh= =3D 0, > >> hw rdt =3D 1023 em0: RX discarded packets =3D 0 em0: RX Next to Check= =3D 0 > >> em0: RX Next to Refresh =3D 1023 em0: Watchdog timeout Queue[0]-- > >> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0:= hw > >> tdh =3D 0, hw tdt =3D 58 em0: Tx Queue Status =3D -2147483648 em0: TX > >> descriptors avail =3D 966 em0: Tx Descriptors avail failure =3D 0 em0= : RX > >> Queue 0 ------ em0: hw rdh =3D 0, hw rdt =3D 1023 em0: RX discarded p= ackets > >> =3D 0 em0: RX Next to Check =3D 0 em0: RX Next to Refresh =3D 1023 xl= 0: > >> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface= is > >> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt = =3D 54 > >> em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 97= 0 em0: > >> Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh= =3D 0, > >> hw rdt =3D 1023 em0: RX discarded packets =3D 0include GENERIC ident = OFL > >> options IPFIREWALL|||||| > >> > >> |Hi, =2E=2E I'm trying to pci passhthrough a usb controller to a bhyv= e > >> windows 7 virtualization=2E When I start the virtual machine it freez= es > >> the entire system and this shows up on the console=2E To be clear the > >> system runs without any problems without the pci passthrough=2E | > >> ||(ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED=2E ACB: 61 08 e0 ff 40 40 = 29 00 > >> 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was invalid > >> (ada0:ahcich0:0:0:0): Error 22, Unretryable error > >> (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY=2E ACB: ec 00 00 00 00 40 00 00= 00 > >> 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request was invalid > >> (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ada0 at ahcich0 = bus > >> 0 scbus0 target 0 lun 0 ada0: s/n ZA4292G6 > >> detached (aprobe0:ahcich0:0:0:0): ATA_IDENTIFY=2E ACB: ec 00 00 00 00= 40 > >> 00 00 00 00 00 00 (aprobe0:ahcich0:0:0:0): CAM status: CCB request wa= s > >> invalid (aprobe0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: > >> Poll timeout on slot 28 port 0 ahcich0: is 20000000 cs 10000000 ss > >> 00000000 rs 10000000 tfd 1d0 serr 00000800 cmd 00005c17 > >> (aprobe0:ahcich0:0:0:0): SOFT_RESET=2E ACB: 00 00 00 00 00 00 00 00 0= 0 00 > >> 00 00 (aprobe0:ahcich0:0:0:0): CAM status: Command timeout > >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted > >> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE RCACHE=2E ACB: ef aa 00 00 0= 0 40 > >> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was > >> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error ahcich0: Po= ll > >> timeout on slot 30 port 0 ahcich0: is 20000000 cs 40000000 ss 0000000= 0 > >> rs 40000000 tfd 1d0 serr 00000800 cmd 00005e17 (aprobe0:ahcich0:0:0:0= ): > >> SOFT_RESET=2E ACB: 00 00 00 00 00 00 00 00 00 00 00 00 > >> (aprobe0:ahcich0:0:0:0): CAM status: Command timeout > >> (aprobe0:ahcich0:0:0:0): Error 5, Retries exhausted > >> (ada0:ahcich0:0:0:0): SETFEATURES ENABLE WCACHE=2E ACB: ef 02 00 00 0= 0 40 > >> 00 00 00 00 00 00 (ada0:ahcich0:0:0:0): CAM status: CCB request was > >> invalid (ada0:ahcich0:0:0:0): Error 22, Unretryable error > >> (ada0:ahcich0:0:0:0): Periph destroyed em0: Watchdog timeout Queue[0]= -- > >> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0:= hw > >> tdh =3D 428, hw tdt =3D 471 em0: Tx Queue Status =3D -2147483648 em0:= TX > >> descriptors avail =3D 981 em0: Tx Descriptors avail failure =3D 0 em0= : RX > >> Queue 0 ------ em0: hw rdh =3D 912, hw rdt =3D 908 em0: RX discarded = packets > >> =3D 0 em0: RX Next to Check =3D 909 em0: RX Next to Refresh =3D 908 x= l0: > >> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface= is > >> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt = =3D 58 > >> em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 96= 6 em0: > >> Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh= =3D 0, > >> hw rdt =3D 1023 em0: RX discarded packets =3D 0 em0: RX Next to Check= =3D 0 > >> em0: RX Next to Refresh =3D 1023 em0: Watchdog timeout Queue[0]-- > >> resetting Interface is RUNNING and ACTIVE em0: TX Queue 0 ------ em0:= hw > >> tdh =3D 0, hw tdt =3D 58 em0: Tx Queue Status =3D -2147483648 em0: TX > >> descriptors avail =3D 966 em0: Tx Descriptors avail failure =3D 0 em0= : RX > >> Queue 0 ------ em0: hw rdh =3D 0, hw rdt =3D 1023 em0: RX discarded p= ackets > >> =3D 0 em0: RX Next to Check =3D 0 em0: RX Next to Refresh =3D 1023 xl= 0: > >> watchdog timeout em0: Watchdog timeout Queue[0]-- resetting Interface= is > >> RUNNING and ACTIVE em0: TX Queue 0 ------ em0: hw tdh =3D 0, hw tdt = =3D 54 > >> em0: Tx Queue Status =3D -2147483648 em0: TX descriptors avail =3D 97= 0 em0: > >> Tx Descriptors avail failure =3D 0 em0: RX Queue 0 ------ em0: hw rdh= =3D 0, > >> hw rdt =3D 1023 em0: RX discarded packets =3D 0include GENERIC ident = OFL > >> options IPFIREWALL options IPFIREWALL_VERBOSE options > >> IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT option= s > >> CONSPEED=3D115200 em0: RX Next to Check =3D 0 em0: RX Next to Refresh= =3D > >> 1023 |||| > >> |||I've recompiled the kernel without the xhci driver=2E I've tested = both > >> usb controllers=2E Both "|||||Sunrise Point-H USB 3=2E0 xHCI > >> Controller|"||||| and "|||uPD720202 USB 3=2E0 Host Controller|", both > >> showed the same error on the console screen=2E| # uname FreeBSD > >> 11=2E1-STABLE #1 r331092M # custom kernel include GENERIC ident OFL > >> options IPFIREWALL options IPFIREWALL_VERBOSE options > >> IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT option= s > >> CONSPEED=3D115200 * note commented "device xhci" in generic # svn inf= o > >> |||Path: =2E Working Copy Root Path: /usr/src URL: > >> https://svn0=2Eeu=2Efreebsd=2Eorg/base/stable/11 Relative URL: ^/stab= le/11 > >> Repository Root: https://svn0=2Eeu=2Efreebsd=2Eorg/base Repository UU= ID: > >> ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind: > >> directory Schedule: normal Last Changed Author: eadler Last Changed R= ev: > >> 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018= ) > >> |# vm passthru DEVICE BHYVE ID READY DESCRIPTION hostb0 0/0/0 No Xeon > >> E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registe= rs > >> pcib1 0/1/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe > >> Controller (x16) vgapci0 0/2/0 No HD Graphics 530 |||ppt0| 0/20/0 Yes > >> Sunrise Point-H USB 3=2E0 xHCI Controller none0 0/20/2 > >> No Sunrise Point-H Thermal subsystem none1 0/22/0 No Sunrise Point-H > >> CSME HECI ahci0 0/23/0 No Sunrise Point-H SATA controller [AHCI mode] > >> pcib2 0/28/0 No Sunrise Point-H PCI Express Root Port pcib4 0/28/6 No > >> Sunrise Point-H PCI Express Root Port isab0 0/31/0 No Sunrise Point-H > >> LPC Controller none2 0/31/2 No Sunrise Point-H PMC hdac0 0/31/3 No > >> Sunrise Point-H HD Audio none3 0/31/4 No Sunrise Point-H SMBus em0 > >> 0/31/6 No Ethernet Connection (2) I219-V pcib3 2/0/0 No XIO2001 PCI > >> Express-to-PCI Bridge xl0 3/0/0 No 3c905B 100BaseTX [Cyclone] ppt1 4/= 0/0 > >> Yes uPD720202 USB 3=2E0 Host Controller ||ppt1@pci0:4:0:0: > >> class=3D0x0c0330 card=3D0x00151912 chip=3D0x00151912 rev=3D0x02 > >> hdr=3D0x00 vendor =3D 'Renesas Technology Corp=2E' device =3D 'uPD720= 202 USB 3=2E0 > >> Host Controller' class =3D serial bus subclass =3D USB > >> |||ppt|0@pci0:0:20:0: class=3D0x0c0330 card=3D0x06ba1028 chip=3D0xa12= f8086 > >> rev=3D0x31 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Sunr= ise Point-H > >> USB 3=2E0 xHCI Controller' class =3D serial bus subclass =3D USB # gr= ep -i > >> pptdevfs loader=2Econf ||pptdevs=3D"0/20/0 4/0/0" # vm configure > >> ||passthru0=3D"0/20/0"| > >> > >> |# vm configure ||passthru0=3D"4/0/0"| > >> ||Any suggestions on what I'm doing wrong?|| > >> ? Regards, Ofloo > >> |||||| > >> > >> ||options IPFIREWALL_VERBOSE options IPFIREWALL_DEFAULT_TO_ACCEPT > >> options > >> DUMMYNET options IPDIVERT options CONSPEED=3D115200 em0: RX Next to C= heck > >> =3D 0 em0: RX Next to Refresh =3D 1023 |||| > >> |||I've recompiled the kernel without the xhci driver=2E I've tested = both > >> usb controllers=2E Both "|||||Sunrise Point-H USB 3=2E0 xHCI > >> Controller|"||||| and "|||uPD720202 USB 3=2E0 Host Controller|", both > >> showed the same error on the console screen=2E| # uname FreeBSD > >> 11=2E1-STABLE #1 r331092M # custom kernel include GENERIC ident OFL > >> options IPFIREWALL options IPFIREWALL_VERBOSE options > >> IPFIREWALL_DEFAULT_TO_ACCEPT options DUMMYNET options IPDIVERT option= s > >> CONSPEED=3D115200 * note commented "device xhci" in generic # svn inf= o > >> |||Path: =2E Working Copy Root Path: /usr/src URL: > >> https://svn0=2Eeu=2Efreebsd=2Eorg/base/stable/11 Relative URL: ^/stab= le/11 > >> Repository Root: https://svn0=2Eeu=2Efreebsd=2Eorg/base Repository UU= ID: > >> ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f Revision: 331092 Node Kind: > >> directory Schedule: normal Last Changed Author: eadler Last Changed R= ev: > >> 331092 Last Changed Date: 2018-03-17 07:44:39 +0100 (Sat, 17 Mar 2018= ) > >> |# vm passthru DEVICE BHYVE ID READY DESCRIPTION hostb0 0/0/0 No Xeon > >> E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registe= rs > >> pcib1 0/1/0 No Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe > >> Controller (x16) vgapci0 0/2/0 No HD Graphics 530 |||ppt0| 0/20/0 Yes > >> Sunrise Point-H USB 3=2E0 xHCI Controller none0 0/20/2 > >> No Sunrise Point-H Thermal subsystem none1 0/22/0 No Sunrise Point-H > >> CSME HECI ahci0 0/23/0 No Sunrise Point-H SATA controller [AHCI mode] > >> pcib2 0/28/0 No Sunrise Point-H PCI Express Root Port pcib4 0/28/6 No > >> Sunrise Point-H PCI Express Root Port isab0 0/31/0 No Sunrise Point-H > >> LPC Controller none2 0/31/2 No Sunrise Point-H PMC hdac0 0/31/3 No > >> Sunrise Point-H HD Audio none3 0/31/4 No Sunrise Point-H SMBus em0 > >> 0/31/6 No Ethernet Connection (2) I219-V pcib3 2/0/0 No XIO2001 PCI > >> Express-to-PCI Bridge xl0 3/0/0 No 3c905B 100BaseTX [Cyclone] ppt1 4/= 0/0 > >> Yes uPD720202 USB 3=2E0 Host Controller ||ppt1@pci0:4:0:0: > >> class=3D0x0c0330 card=3D0x00151912 chip=3D0x00151912 rev=3D0x02 > >> hdr=3D0x00 vendor =3D 'Renesas Technology Corp=2E' device =3D 'uPD720= 202 USB 3=2E0 > >> Host Controller' class =3D serial bus subclass =3D USB > >> |||ppt|0@pci0:0:20:0: class=3D0x0c0330 card=3D0x06ba1028 chip=3D0xa12= f8086 > >> rev=3D0x31 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Sunr= ise Point-H > >> USB 3=2E0 xHCI Controller' class =3D serial bus subclass =3D USB # gr= ep -i > >> pptdevfs loader=2Econf ||pptdevs=3D"0/20/0 4/0/0" # vm configure > >> ||passthru0=3D"0/20/0"| > >> > >> |# vm configure ||passthru0=3D"4/0/0"| > >> ||Any suggestions on what I'm doing wrong?|| > >> > >> edit: previous wrapping was all wrong=2E > >> ?? Regards, Ofloo > >> |||||| > >> > >> >=20 -- End of PGP section, PGP failed! --=20 Rod Grimes rgrimes@freebsd= =2Eorg From owner-freebsd-virtualization@freebsd.org Sat Mar 24 14:27:08 2018 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E36CEF4E57A for ; Sat, 24 Mar 2018 14:27:07 +0000 (UTC) (envelope-from byond.lenox@gmail.com) Received: from mail-lf0-f45.google.com (mail-lf0-f45.google.com [209.85.215.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 58FB06EDEE; Sat, 24 Mar 2018 14:27:07 +0000 (UTC) (envelope-from byond.lenox@gmail.com) Received: by mail-lf0-f45.google.com with SMTP id l4-v6so17876171lfg.12; Sat, 24 Mar 2018 07:27:07 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=P8MZa4BFODkwDcg88/5RKC84bGDAnxmAWVXkxEIYv/8=; b=akMJS6nYFbclwJGEevM/d2mv+WoEanxZpvG/eQqhu0JEXsqa7XwF8+2kZvThXbdn0a +SOOWmasE1RhNni0dI7+9DToOKH/0HdllmpR0voFy6RYvef6WtQUHmO1J3Jxil9LSnOW P8A1fyPw8N+h/a9nKWUwoYBHZkkbL5oIeeQ3jfvcuWWEK6N/j7dhq44u8wAwkjBWtsXW UsvOggB5/fe3CLRKt+GfBWsKNTmTC7onxUt0Nw02a37ZrRr58x16W4N3QCg7HA9xquRa yPfrsJtM/NkKehlaXY60bEz7WNLX2AqgXYdiAg89c09Se33UUYn7Lemv3S60492Kyp36 cbbA== X-Gm-Message-State: AElRT7GHAoKOyOAHZlYE/AiopdhLhPnJw5WEoB1MUZ3z+fsnIXJeUuto /bFfDDqMzP03DOKb3WANt3Qak8D+ X-Google-Smtp-Source: AG47ELt1Tm8e4oRdYl+jN0Tla+O9VVL8c8aqn6eLBCUfQ1BM4u9ImzGCrsHtw01b3NaNxGEuOGLaKg== X-Received: by 2002:a19:e48e:: with SMTP id x14-v6mr22959735lfi.115.1521901619881; Sat, 24 Mar 2018 07:26:59 -0700 (PDT) Received: from mail-lf0-f49.google.com (mail-lf0-f49.google.com. [209.85.215.49]) by smtp.gmail.com with ESMTPSA id v6sm2436231ljc.85.2018.03.24.07.26.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sat, 24 Mar 2018 07:26:59 -0700 (PDT) Received: by mail-lf0-f49.google.com with SMTP id a22-v6so22201315lfg.9; Sat, 24 Mar 2018 07:26:59 -0700 (PDT) X-Received: by 10.46.128.132 with SMTP id i4mr15955433ljg.51.1521901619486; Sat, 24 Mar 2018 07:26:59 -0700 (PDT) MIME-Version: 1.0 Received: by 10.46.129.90 with HTTP; Sat, 24 Mar 2018 07:26:38 -0700 (PDT) In-Reply-To: <624f4ac2-ada1-64af-1c4f-250d3de551de@freebsd.org> References: <201803231812.w2NICLEW032460@pdx.rh.CN85.dnsmgr.net> <624f4ac2-ada1-64af-1c4f-250d3de551de@freebsd.org> From: Kyle Evans Date: Sat, 24 Mar 2018 09:26:38 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Issue encountered booting FreeBSD STABLE and CURRENT snapshots with EFI To: Peter Grehan Cc: "Rodney W. Grimes" , "freebsd-virtualization@freebsd.org" , Marcelo Araujo Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Mar 2018 14:27:08 -0000 On Sat, Mar 24, 2018 at 12:59 AM, Peter Grehan wrote: > Hi Kyle, > >> FYI- I've created PR #2 [1] against the freebsd/uefi-edk2 repository >> and have confirmed that this fixes the broken-looking firmware along >> with the booting problem experienced in this thread. >> >> [1] https://github.com/freebsd/uefi-edk2/pull/2 > > > Thanks - merged. I've asked Fabian to update the UEFI ports. > Awesome, thanks! =) It didn't seem right to hack around the hack around that was implemented for our formerly bad behavior, but we can look at it again if we run into further problems with the way we do this. I don't think our devpath usage is unreasonable, though- we don't expect these pointers to live forever, just long enough to locate all of the partitions on it. Thanks, Kyle Evans