From owner-freebsd-virtualization@freebsd.org Sun May 19 13:20:13 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 396E915AE035 for ; Sun, 19 May 2019 13:20:13 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5D1D98C4E4 for ; Sun, 19 May 2019 13:20:12 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id x4JDKAJR069441; Sun, 19 May 2019 06:20:10 -0700 (PDT) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id x4JDK9rx069440; Sun, 19 May 2019 06:20:09 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201905191320.x4JDK9rx069440@gndrsh.dnsmgr.net> Subject: Re: Windows 10 guests fail to boot when attempting to passthrough network card In-Reply-To: To: Nick Principe Date: Sun, 19 May 2019 06:20:09 -0700 (PDT) CC: "Rodney W. Grimes" , freebsd-virtualization@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 5D1D98C4E4 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [0.43 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.84)[-0.844,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-0.63)[-0.631,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[dnsmgr.net]; AUTH_NA(1.00)[]; NEURAL_SPAM_SHORT(0.97)[0.970,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: gndrsh.dnsmgr.net]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.05)[ip: (0.16), ipnet: 69.59.192.0/19(0.08), asn: 13868(0.05), country: US(-0.06)] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 May 2019 13:20:13 -0000 > Yeah, that triggers a barf with error 2: > mmio_rb_lookup: barf ENOENTerror is 2 > mr.name is passthru-pci-5 > mr.flags is 0 > mr.arg2 is 0 > mr.base is 18446744073709027328 > mr.size is 524288 > Assertion failed: (error == 0), function modify_bar_registration, file > /usr/src/usr.sbin/bhyve/pci_emul.c, line 510. > > Interestingly enough, after we talked I simply commented out the assert at > 504 in pci_emul.c and Windows booted fine and can see the Chelsio card with > both interfaces. I'm not sure if it would fall over once I started actually > using it or not. I would go forward with testing, worst that happens is a host panic. > `nap > > > On Sat, May 18, 2019 at 4:06 PM Rodney W. Grimes < > freebsd-rwg@gndrsh.dnsmgr.net> wrote: > > > > I have noticed that Windows 10 guests fail to boot when attempting to > > pass > > > through a network card. I believe I have observed this with both cxgbe > > > (t580) and mlx5en cards, but only have a cxgbe to test with now. Without > > > passthrough, the Windows 10 guest boots and operates normally. > > > > > > FreeBSD guests (12.0-RELEASE) have no issue when booting with the cxgbe > > > card passed through - I can kldload cxgbe and I get both cxl ports > > showing > > > up in the FreeBSD guest. > > > > > > I have tested this with both 12.0-RELEASE and head (13-CURRENT r347883) > > as > > > the host OS with no change in behavior. The bhyve output is: > > > Unhandled ps2 keyboard command 0x02 > > > Unhandled ps2 keyboard command 0x02 > > > Assertion failed: (error == 0), function modify_bar_registration, file > > > /usr/src/usr.sbin/bhyve/pci_emul.c, line 504. > > > fbuf frame buffer base: 0x943600000 [sz 16777216] > > > > > > Two main suggestions from discussions at BSDCan this week were: > > > - Capture pciconf -lvb from the FreeBSD guest > > > - Add some printf to pci_emul.c to capture some values when there is an > > > error > > > > > > I've captured the above, and a lot of other relevant info, in a Google > > Doc > > > here (too big to post directly): > > > > > https://docs.google.com/document/d/1t-UVIO9Aq0TPUFHyo1nVscqaW1LoPuNhfLPitL8oeTs/edit?usp=sharing > > > > > > `nap > > > > To confirm what I see from looking at your data could you tell me if this > > patch triggers a barf? > > > > --- mem.c.orig 2019-05-18 20:04:26.707995000 +0000 > > +++ mem.c 2019-05-18 20:04:02.205119000 +0000 > > @@ -97,6 +97,7 @@ > > return (0); > > } > > > > +printf("mmio_rb_lookup: barf ENOENT"); > > return (ENOENT); > > } > > > > -- > > Rod Grimes > > rgrimes@freebsd.org > > > > > -- > Nick Principe > Performance Engineering Supervisor > iXsystems, Inc. > Ph: (408) 943-4100 x341 > Fx: (408) 943-4101 -- Rod Grimes rgrimes@freebsd.org