Date: Sat, 2 Jan 2021 11:06:44 -0800 From: Doug Ambrisko <ambrisko@ambrisko.com> To: Neel Chauhan <neel@neelc.org> Cc: Mark Johnston <markj@freebsd.org>, freebsd-hackers@freebsd.org, ambrisko@freebsd.org Subject: Re: Debugging a WIP PCI/ACPI patch: Bad tailq NEXT(0xffffffff81cde660->tqh_last) != NULL Message-ID: <20210102190644.GB87535@ambrisko.com> In-Reply-To: <7cda3be6594d5ad5bdc69019f72b03d3@neelc.org> References: <44528336fa9168966d121bf771e1e229@neelc.org> <X%2ByzpNIclmFYgbr7@raichu> <3c9ff844e527daacd04c51f48836b57d@neelc.org> <dbcc0e54eeb0080620ee4fb6d14845fc@neelc.org> <e73228a75b8f05c83214c62ed7e1ba68@neelc.org> <X%2B3tfbxHGdiW1Kvt@raichu> <20201231200744.GA95383@ambrisko.com> <4f3f6a02a452f766063ae2acb060dc64@neelc.org> <7cda3be6594d5ad5bdc69019f72b03d3@neelc.org>
next in thread | previous in thread | raw e-mail | index | archive | help
With VMD, the PCI "root" is hidden behind it. To access devices behind
the VMD device, a new domain is created and when PCI config. space
is accessed, it is indexed via the VMD device via the offset. Intel
seems to have reduced the available bus space on some HW. So for a bus
access less then what they implemented in HW we have to return error
that nothing is there. Then when we get to the starting bus device,
we need to offset that to 0 based in the HW. The PCI probe will run
look for busses from 0 to 255. From the Linux driver your HW only works
from 224 to 255. So we need to fail anything under 224 and for bus
requests 224 and higher then subtract 224. Thus the b - sc->vmd_bus_start
part. I'm not sure if we could do it the other way in which we allow
0-12 bus requests to pass and fail if it is over. I'm not sure if
there is any specific reason why that wouldn't work. Linux didn't
do that but that doesn't mean it wouldn't work. It would be good to
start with the Linux method and then test 0-n, where n is the max.
busses that HW allows. Anything n or more would have to return a fail.
Doug A.
On Sat, Jan 02, 2021 at 09:20:20AM -0800, Neel Chauhan wrote:
| Just to ping you in case you may have missed my reply (I understand, New
| Years Day).
|
| Is there a reason why "b = pci_get_bus(dev);" return 0 even when the bus
| number is shifted (as it is on Linux)?
|
| -Neel
|
| On 2020-12-31 21:49, Neel Chauhan wrote:
| > Hi Doug,
| >
| > Thank you so much for this information.
| >
| > On 2020-12-31 12:07, Doug Ambrisko wrote:
| >> FYI, looks like this needs to be ported over from Linux:
| >> static char __iomem *vmd_cfg_addr(struct vmd_dev *vmd, struct pci_bus
| >> *bus,
| >> unsigned int devfn, int reg, int
| >> len)
| >> {
| >> char __iomem *addr = vmd->cfgbar +
| >> ((bus->number - vmd->busn_start) << 20) +
| >> (devfn << 12) + reg;
| >>
| >> to
| >> vmd_read_config
| >> offset = (b << 20) + (s << 15) + (f << 12) + reg;
| >>
| >> vmd_write_config(device_t dev, u_int b, u_int s, u_int f, u_int reg,
| >> offset = (b << 20) + (s << 15) + (f << 12) + reg;
| >>
| >> ie.
| >> offset = ((b - sc->vmd_bus_start) << 20) + (s << 15) + (f << 12) +
| >> reg;
| >>
| >> vmd_bus_start should be added to the softc as a uint8_t type and needs
| >> to
| >> be set via attach. We need range checks to make sure
| >> vmd_write_config/vmd_read_config doesn't read something out of range
| >> since it has been reduced.
| >
| > One thing I noticed is that the "b" variable (which corresponds to the
| > Linux bus->number) is 0 (thanks to printf). This should be the bus
| > number if we want to attach.
| >
| > If I use: "b = pci_get_bus(dev);" in the attach, b is still 0.
| >
| > And that leads to a kernel panic.
| >
| >> Not sure what the shadow registers do. These both seem to be new
| >> Intel
| >> features and Intel doc's have been minimal. Looks like Intel is doing
| >> a sparse map now on newer devices.
| >
| > I guess Linux is our best hope. Unless the new Intel docs is the Linux
| > kernel source.
| >
| >> I'm concerned about the Linux comment of:
| >> * Certain VMD devices may have a root port configuration
| >> option which
| >> * limits the bus range to between 0-127, 128-255, or 224-255
| >>
| >> since I don't see anything to limit it between 0-127 only starting
| >> at 0, 128 or 224, Maybe there is max of 128 busses overall?
| >
| > I could be wrong, but I guess that's a typo.
| >
| >> I don't have this type of HW to test things.
| >
| > I can use my hardware for testing. In the worse case scenario, I can
| > donate an entry-level 11th Gen/TigerLake system if I have the funds
| > and/or can get a tax credit.
| >
| >> Doug A.
| >
| > -Neel
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20210102190644.GB87535>