From owner-freebsd-virtualization@freebsd.org Thu Feb 28 04:38:24 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 18E0A1516B87 for ; Thu, 28 Feb 2019 04:38:24 +0000 (UTC) (envelope-from toasty@dragondata.com) Received: from mail-it1-x12f.google.com (mail-it1-x12f.google.com [IPv6:2607:f8b0:4864:20::12f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A5F12754FF for ; Thu, 28 Feb 2019 04:38:21 +0000 (UTC) (envelope-from toasty@dragondata.com) Received: by mail-it1-x12f.google.com with SMTP id l15so13744781iti.4 for ; Wed, 27 Feb 2019 20:38:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dragondata.com; s=google; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=nSSI4lTssWPS1UHaYV6UahcsaGnBtuKPyiEhUo5ZKNU=; b=f+imOoX/zmKyg6sGY3ZW1pusFHhIYXtPhaaaRZsUAvhGhaHTJBdAQeMfjPbYsSZPvI WtnwrYlWXlpq7qbAF786V9wAqmlA9d4t9seqr6T3LyI05aiU8iYtSTxggX8IW3XAU6T1 xqRMpICujSgzgQLlcT/6LLkFqYB3qbVp0Hh9I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=nSSI4lTssWPS1UHaYV6UahcsaGnBtuKPyiEhUo5ZKNU=; b=gHKKW7wdi3caSX7s5d7DyHNQoiGkoVT3XFYoA3PbNKmq91f+pmZvkGAjj0TIBSw7vD c2mf+wi/4fZ8vy5AxhTDS34le6un6UpKiklfsBZyIcjcIyuu9YeSakTmSxdb1vTWcYyg wLnzP8eL+HQFD4JtB14OPZ61JD+kK0+e/BpkVlfY3fLuDVo+5mIEbLf4FTJPo9sgZEpH kPt4u12NL9jIgHLsPnHx3w3nqU96iQkrSMzf42V9vDaH386nslUocHMPQnlpgOu7475V NPMLE5hR2EPMTkUb8m9ABYwrjiw83N0KCgAeW8w+L+GNLjvtISBiOxkrr/73TfX33KeB Stmg== X-Gm-Message-State: APjAAAXoFE4EuU0aFxjCWXi1cnvWAxUMUyiRPlot+GXh7Gs5dQXzuTVr EkfxfkXt1Qro/zbPDxSf6vNnHwfz+jI= X-Google-Smtp-Source: APXvYqxbSUygCqQEuRGsK1U8EHvDQKTIg+u3Ca0h1lNaq+5Ncf7l/k6HcnwLBsZaXGn2iib1Kkoe+g== X-Received: by 2002:a24:6948:: with SMTP id e69mr1733110itc.140.1551328700268; Wed, 27 Feb 2019 20:38:20 -0800 (PST) Received: from i82.cfv.net (i82.cfv.net. [204.9.51.82]) by smtp.gmail.com with ESMTPSA id f24sm7084631ioo.24.2019.02.27.20.38.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 27 Feb 2019 20:38:19 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Re: bhyve PCIe passthrough on AMD Epyc From: Kevin Day In-Reply-To: <201902270632.x1R6W8ec049419@pdx.rh.CN85.dnsmgr.net> Date: Wed, 27 Feb 2019 22:38:18 -0600 Cc: freebsd-virtualization@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <8A44404F-C6E5-47BF-82E8-3CF94D5487E9@dragondata.com> References: <201902270632.x1R6W8ec049419@pdx.rh.CN85.dnsmgr.net> To: "Rodney W. Grimes" X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: A5F12754FF X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=dragondata.com header.s=google header.b=f+imOoX/ X-Spamd-Result: default: False [-5.54 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[dragondata.com:s=google]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; DMARC_NA(0.00)[dragondata.com]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[dragondata.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[f.2.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[ASPMX3.GOOGLEMAIL.com,ALT2.ASPMX.L.GOOGLE.com,ASPMX.L.GOOGLE.com,ALT1.ASPMX.L.GOOGLE.com,ASPMX5.GOOGLEMAIL.com,ASPMX4.GOOGLEMAIL.com,ASPMX2.GOOGLEMAIL.com]; R_SPF_NA(0.00)[]; NEURAL_HAM_SHORT(-0.96)[-0.957,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(-2.78)[ip: (-9.10), ipnet: 2607:f8b0::/32(-2.69), asn: 15169(-2.02), country: US(-0.07)] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Feb 2019 04:38:24 -0000 > On Feb 27, 2019, at 12:32 AM, Rodney W. Grimes = wrote: >=20 >> I'm trying to get PCIe device (video card) passthrough working to a = Windows bhyve VM, should this work now? >=20 > No, this should not work now. Ok, easy enough. Should *any* PCIe passthrough work on Epyc right now? = Just trying the same thing with a simple PCIe USB card gives me: CPU0: local APIC error 0x40 CPU0: local APIC error 0x40 ivhd0: Error: completion failed tail:0xda0, head:0x0. ivhd0: Dump all the commands: [CMD0, off:0xff0] opcode=3D 0x0 0x0 0x0 0x0 [CMD1, off:0x0] opcode=3D 0x3 0x0 0x0 0x7ffffffffffff003 [CMD2, off:0x10] opcode=3D 0x0 0x0 0x0 0x0 [CMD3, off:0x20] opcode=3D 0x2 0x0 0x0 0x0 [CMD4, off:0x30] opcode=3D 0x2 0x2 0x0 0x0 [...snip... lots of similar stuff] Then I get a bunch more APIC errors, and the system freezes for about 60 = seconds before recovering. -- Kevin