Date: Tue, 1 Dec 1998 21:01:22 -0700 (MST) From: vanmaren@cs.utah.edu To: FreeBSD-gnats-submit@FreeBSD.ORG Subject: kern/8928: 450NX-based computers only probe the first PCI bus Message-ID: <199812020401.VAA01553@vanmaren.aros.net>
next in thread | raw e-mail | index | archive | help
>Number: 8928
>Category: kern
>Synopsis: 450NX-based computers only probe the first PCI bus
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: freebsd-bugs
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Tue Dec 1 20:10:01 PST 1998
>Last-Modified:
>Originator: Kevin Van Maren
>Organization:
University of Utah
>Release: FreeBSD 3.0-CURRENT i386
>Environment:
This occurs in both 3.0-CURRENT and 2.2-STABLE with the 450NX chipset.
>Description:
MIOC: Memory and I/O controller. It connects the CPU bus to the
memory subsystem and the PCI controller. PXB: PCI Expander bridge,
connects the MIOC to either two 32-bit PCI busses or a 64-bit bus.
The MIOC can control one or two PXBs.
Any devices not plugged into the first PCI bus simply won't be found.
Most current 450NX-based computers are quad-capable with either
two 32-bit PCI busses (using one PXB), or with two 32-bit busses
and a 64-bit PCI bus (using two PXB chips). In either case, only
devices on the first 32-bit PCI bus will be probed, as FreeBSD
doesn't know about the other PCI busses.
The problem stems from how the PCI busses are provided by the
chipset. The processor bus interface attaches to the MIOC, which
in turn can connect to either one or two PXB PCI bus controller
chips (with either one or two PCI bus interfaces each). Instead
of there being one PCI bus with the other busses "bridged" from
that one, they are connected in parallel to the MIOC. Fancy
circuitry makes it all "work". However, since the PXB chips are
probed as regular devices, and not as PCI-PCI bridges, the
PCI bus count isn't incremented. Additionally, it is actually
possible for the BIOS to skip PCI busses when programming the
bus IDs for the PXBs in the MIOC (to allow for hot-plug devices
with a PCI-PCI bridge). Also, since the configuration for the
PXB is actually controlled by configuration registers in the
MIOC, the MIOC must be asked how many PCI busses there are. It
would be much cleaner to set up the subordinatebus for each PXB
as it is probed, but the info is in the MIOC, not the PXB.
>How-To-Repeat:
Buy a quad Xeon.
Install FreeBSD.
Try to use a device not on the first bus.
[PCI busses bridged from the first bus will, of course, work,
but not the other ones.]
>Fix:
The hardware knows how to route requests to higher bus numbers properly;
the kernel just doesn't know to look for something there.
The fix is to read the BIOS-programmed configuration from the
MIOC configuration registers. The highest bus number must belong
to the last bus on the last PXB, and there are 4 choices. The
code just searches backwards until it finds one that has been configured.
Here are the patches. Note that I haven't actually tested the ones
for 2.2-STABLE, but I think they will work...they do compile.
[Unfortunalty, I don't have good access to a 450NX box now.]
Patches for 2.2-STABLE: (in src/sys/pci)
Index: pcisupport.c
===================================================================
RCS file: /usr/lsrc/FreeBSD/CVS/src/sys/pci/pcisupport.c,v
retrieving revision 1.40.2.8
diff -u -r1.40.2.8 pcisupport.c
--- pcisupport.c 1998-06-17 09:29:09-06 1.40.2.8
+++ pcisupport.c 1998-12-01 20:58:41-07
@@ -172,6 +172,10 @@
return ("Intel 82450KX (Orion) PCI memory controller");
case 0x84c58086:
return ("Intel 82454GX (Orion) host to PCI bridge");
+ case 0x84ca8086:
+ return ("Intel 82451NX Memory and I/O Controller");
+ case 0x84cb8086:
+ return ("Intel 82454NX PCI Expander Bridge");
case 0x00051166:
return ("Ross (?) host to PCI bridge");
case 0x00221014:
@@ -726,6 +730,56 @@
}
static void
+config_450nx(pcici_t tag)
+{
+ unsigned long devmap;
+ int i;
+
+ /* Read the MIOC devmap to determine which PCI expanders are present */
+ devmap = (pci_conf_read(tag, 0xd4) >> 16) & 0x3c;
+
+ if (!devmap) {
+ printf("Error: 450NX MIOC: No PCI Expander Bridge present.\n");
+ return;
+ }
+
+#if 0
+ /*
+ * This hack is in the spirit of the config_orion() routine.
+ * It *would* work, except that some 450NX-based servers are
+ * set up to SKIP PCI busses, presumably to support hot-plug
+ * PCI cards that contain a single PCI-PCI bridge chip.
+ */
+
+ /* Now pciroots just needs to get set to the number of bits set... */
+
+ pciroots = 0;
+ for (i = 2; i < 6; i++)
+ if (devmap & (1 << i))
+ pciroots++;
+#else
+ /*
+ * Since the buses are configured in order, we just have to
+ * find the highest bus, and use those numbers.
+ * This is `wrong', but there is nothing else I can really do...
+ */
+ if (devmap & 0x20) { /* B1, 0xd5 */
+ pciroots = (pci_conf_read(tag, 0xd4) >> 8) & 0xff;
+ } else if (devmap & 0x10) { /* A1, 0xd4 */
+ pciroots = pci_conf_read(tag, 0xd4) & 0xff;
+ } else if (devmap & 0x8) { /* B0, 0xd2 */
+ pciroots = (pci_conf_read(tag, 0xd0) >> 16) & 0xff;
+ } else /* if (devmap & 0x4) */ { /* A0, 0xd1 */
+ pciroots = (pci_conf_read(tag, 0xd0) >> 8) & 0xff;
+ }
+#endif
+
+ if (bootverbose)
+ printf("config_450nx: %d PCI busses found\n", pciroots);
+}
+
+
+static void
chipset_attach (pcici_t config_id, int unit)
{
switch (pci_conf_read (config_id, PCI_ID_REG)) {
@@ -738,6 +792,9 @@
break;
case 0x00051166: /* Ross ??? */
config_Ross (config_id);
+ break;
+ case 0x84ca8086: /* Intel 450NX */
+ config_450nx (config_id);
break;
}
#ifndef PCI_QUIET
Patches for 3.0-CURRENT: in src/sys/pci
Index: pcisupport.c
===================================================================
RCS file: /usr/lsrc/FreeBSD/CVS/src/sys/pci/pcisupport.c,v
retrieving revision 1.74
diff -u -r1.74 pcisupport.c
--- pcisupport.c 1998-11-26 14:57:52-07 1.74
+++ pcisupport.c 1998-12-01 20:28:31-07
@@ -118,6 +122,7 @@
* XXX Both fixbushigh_orion() and fixbushigh_i1225() are bogus in that way,
* that they store the highest bus number to scan in this device's config
* data, though it is about PCI buses attached to the CPU independently!
+ * The same goes for fixbushigh_450nx.
*/
static void
@@ -137,6 +142,57 @@
tag->secondarybus = tag->subordinatebus = sublementarybus +1;
}
+
+/*
+ * This reads the PCI config space for the 82451NX MIOC in the 450NX
+ * chipset to determine the PCI bus configuration.
+ *
+ * Assuming the BIOS has set up the MIOC properly, this will correctly
+ * report the number of PCI busses in the system.
+ *
+ * A small problem is that the Host to PCI bridge control is in the MIOC,
+ * while the host-pci bridges are separate PCI devices. So it really
+ * isn't easily possible to set up the subordinatebus mappings as the
+ * 82454NX PCI expander bridges are probed, although that makes the
+ * most sense.
+ */
+static void
+fixbushigh_450nx(pcici_t tag)
+{
+ int subordinatebus;
+ unsigned long devmap;
+
+ /*
+ * Read the DEVMAP field, so we know which fields to check.
+ * If the Host-PCI bridge isn't marked as present by the BIOS,
+ * we have to assume it doesn't exist.
+ * If this doesn't find all the PCI busses, complain to the
+ * BIOS vendor. There is nothing more we can do.
+ */
+ devmap = pci_cfgread(tag, 0xd6, 2) & 0x3c;
+ if (!devmap)
+ panic("450NX MIOC: No host to PCI bridges marked present.\n");
+ /*
+ * Since the buses are configured in order, we just have to
+ * find the highest bus, and use those numbers.
+ */
+ if (devmap & 0x20) { /* B1 */
+ subordinatebus = pci_cfgread(tag, 0xd5, 1);
+ } else if (devmap & 0x10) { /* A1 */
+ subordinatebus = pci_cfgread(tag, 0xd4, 1);
+ } else if (devmap & 0x8) { /* B0 */
+ subordinatebus = pci_cfgread(tag, 0xd2, 1);
+ } else /* if (devmap & 0x4) */ { /* A0 */
+ subordinatebus = pci_cfgread(tag, 0xd1, 1);
+ }
+ if (bootverbose)
+ printf("fixbushigh_450nx: subordinatebus is %d\n",
+ subordinatebus);
+
+ tag->secondarybus = tag->subordinatebus = subordinatebus;
+}
+
+
static void
fixwsc_natoma(pcici_t tag)
{
@@ -231,6 +287,7 @@
case 0x84c58086:
return ("Intel 82453KX/GX (Orion) PCI memory controller");
case 0x84ca8086:
+ fixbushigh_450nx(tag);
return ("Intel 82451NX Memory and I/O Controller");
case 0x84cb8086:
return ("Intel 82454NX PCI Expander Bridge");
>Audit-Trail:
>Unformatted:
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-bugs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199812020401.VAA01553>