Date: Sun, 19 May 2019 06:24:24 -0700 (PDT) From: "Rodney W. Grimes" <freebsd-rwg@gndrsh.dnsmgr.net> To: Robert Crowston <crowston@protonmail.com> Cc: Nick Principe <nap@ixsystems.com>, "Rodney W. Grimes" <freebsd-rwg@gndrsh.dnsmgr.net>, freebsd-virtualization@freebsd.org Subject: Re: Windows 10 guests fail to boot when attempting to passthrough network card Message-ID: <201905191324.x4JDOOoh069468@gndrsh.dnsmgr.net> In-Reply-To: <tKPiToUyaEcMweodfqp15OdJqaJnGY1gQchZMTzr5dYD0I4XfNzaTPM3Ebij6Rh954JzmJ3JpQJB2TH84DICyIFy4JZIFRSakZbekcuplTk=@protonmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> Does the card have multiple BARs? Can you send us the output of pciconf -lvb for the card? > > There is a bug in bhyve where if the OS assigns the 0xffffffff sentinel to more than one BAR simultaneously it is not compatible with bhyve?s memory model for the guest. The "we talked" below is myself and Nick talking about exactly that which came from jhb@ pointing out to me in a hall wall conversation here at BSDCan that this is the cause of this problem and is a known issue. Does anyone have fixes for this issue? > > On Sat, May 18, 2019 at 21:14, Nick Principe <nap@ixsystems.com> wrote: > > > Yeah, that triggers a barf with error 2: > > mmio_rb_lookup: barf ENOENTerror is 2 > > mr.name is passthru-pci-5 > > mr.flags is 0 > > mr.arg2 is 0 > > mr.base is 18446744073709027328 > > mr.size is 524288 > > Assertion failed: (error == 0), function modify_bar_registration, file > > /usr/src/usr.sbin/bhyve/pci_emul.c, line 510. > > > > Interestingly enough, after we talked I simply commented out the assert at > > 504 in pci_emul.c and Windows booted fine and can see the Chelsio card with > > both interfaces. I'm not sure if it would fall over once I started actually > > using it or not. > > > > `nap > > > > On Sat, May 18, 2019 at 4:06 PM Rodney W. Grimes < > > freebsd-rwg@gndrsh.dnsmgr.net> wrote: > > > >> > I have noticed that Windows 10 guests fail to boot when attempting to > >> pass > >> > through a network card. I believe I have observed this with both cxgbe > >> > (t580) and mlx5en cards, but only have a cxgbe to test with now. Without > >> > passthrough, the Windows 10 guest boots and operates normally. > >> > > >> > FreeBSD guests (12.0-RELEASE) have no issue when booting with the cxgbe > >> > card passed through - I can kldload cxgbe and I get both cxl ports > >> showing > >> > up in the FreeBSD guest. > >> > > >> > I have tested this with both 12.0-RELEASE and head (13-CURRENT r347883) > >> as > >> > the host OS with no change in behavior. The bhyve output is: > >> > Unhandled ps2 keyboard command 0x02 > >> > Unhandled ps2 keyboard command 0x02 > >> > Assertion failed: (error == 0), function modify_bar_registration, file > >> > /usr/src/usr.sbin/bhyve/pci_emul.c, line 504. > >> > fbuf frame buffer base: 0x943600000 [sz 16777216] > >> > > >> > Two main suggestions from discussions at BSDCan this week were: > >> > - Capture pciconf -lvb from the FreeBSD guest > >> > - Add some printf to pci_emul.c to capture some values when there is an > >> > error > >> > > >> > I've captured the above, and a lot of other relevant info, in a Google > >> Doc > >> > here (too big to post directly): > >> > > >> https://docs.google.com/document/d/1t-UVIO9Aq0TPUFHyo1nVscqaW1LoPuNhfLPitL8oeTs/edit?usp=sharing > >> > > >> > `nap > >> > >> To confirm what I see from looking at your data could you tell me if this > >> patch triggers a barf? > >> > >> --- mem.c.orig 2019-05-18 20:04:26.707995000 +0000 > >> +++ mem.c 2019-05-18 20:04:02.205119000 +0000 > >> @@ -97,6 +97,7 @@ > >> return (0); > >> } > >> > >> +printf("mmio_rb_lookup: barf ENOENT"); > >> return (ENOENT); > >> } > >> > >> -- > >> Rod Grimes > >> rgrimes@freebsd.org > >> > > > > -- > > Nick Principe > > Performance Engineering Supervisor > > iXsystems, Inc. > > Ph: (408) 943-4100 x341 > > Fx: (408) 943-4101 > > _______________________________________________ > > freebsd-virtualization@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > > To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" -- Rod Grimes rgrimes@freebsd.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201905191324.x4JDOOoh069468>