From owner-freebsd-acpi@FreeBSD.ORG Wed Jul 8 17:08:38 2009 Return-Path: Delivered-To: freebsd-acpi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B68E7106566C for ; Wed, 8 Jul 2009 17:08:38 +0000 (UTC) (envelope-from jkempf@davisvision.com) Received: from zixvpm02.zixvpm.davisvision.com (zixvpm02.zixvpm.davisvision.com [65.213.99.45]) by mx1.freebsd.org (Postfix) with ESMTP id 765F98FC16 for ; Wed, 8 Jul 2009 17:08:38 +0000 (UTC) (envelope-from jkempf@davisvision.com) Received: from zixvpm02.zixvpm.davisvision.com (ZixVPM [127.0.0.1]) by Outbound.davisvision.com (Proprietary) with ESMTP id DF1385DC014 for ; Wed, 8 Jul 2009 12:51:52 -0400 (EDT) Received: from riley.davisvision.com (unknown [10.51.10.12]) by zixvpm02.zixvpm.davisvision.com (Proprietary) with ESMTP id 83DAC1F4020 for ; Wed, 8 Jul 2009 12:51:50 -0400 (EDT) Received: from riley.davisvision.com (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id 7452024663 for ; Wed, 8 Jul 2009 12:51:50 -0400 (EDT) Received: from waffle.davisvision.com (jkempf-lt.davisvision.com [192.168.149.60]) by riley.davisvision.com (Postfix) with ESMTP id 603142464D for ; Wed, 8 Jul 2009 12:51:50 -0400 (EDT) Message-ID: <17574_1247071910_4A54CEA6_17574_42_1_4A54CE95.4090503@davisvision.com> Date: Wed, 08 Jul 2009 12:51:33 -0400 From: Jesse Kempf User-Agent: Thunderbird 2.0.0.19 (X11/20090128) MIME-Version: 1.0 To: freebsd-acpi@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Banner: Set Subject: SunFire x4275 X-BeenThere: freebsd-acpi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: ACPI and power management development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jul 2009 17:08:39 -0000 Hi, We recently got in a SunFire x4275, one of Sun's new Nehalem boxes. The full server architecture whitepaper is here: https://www.sun.com/offers/details/X4x70_server_architecture.html (requires registration). The upshot of the system is that there are a pair of "intelligent" risers in the machine, which take a x8 PCIe link and mux it to a pair of x8 PCIe links. Sun uses the IDT PES24T6G2 PCIe switch to do the muxing. FreeBSD, when it boots, can't see past the risers. This holds for both 7.2p1 and 8-CURRENT. It can see the IDT PCIe switches, and if a PCIe card is moved off the intelligent riser, to the one dumb riser in the machine, the card can be seen. Interestingly, when I boot FreeBSD with ACPI turned off, all the devices past the risers can be seen, but device attachment fails spectacularly. I've tried setting debug.acpi.disabled for individual subsystems that make sense, and have not been able to come up with a combination that allows the system to boot as well as see past the risers. From what I understand, this is not too surprising. When I say "can't see past the risers", I mean: pcib9: at device 0.0 on pci25 pcib9: domain 0 pcib9: secondary bus 26 pcib9: subordinate bus 38 pcib9: I/O decode 0x8000-0x9fff pcib9: memory decode 0xfaa00000-0xfabfffff pcib9: no prefetched decode device_attach: pcib9 attach returned 6 is what the kernel spits out when I do a verbose boot. None of the PCI buses past the riser are enumerated. I dumped the ACPI AML block and decompiled it. The only errors that iasl spit out when I recompiled it was use of the reserved variables '_T_0' and '_T_1'. Switching them to 'LOL0' and 'LOL1', recompiling, and telling the loader where to find the new blob did not help. I am, at this point, at a loss for what to do next. Here's the dmesg from the verbose boot: http://www.cs.rpi.edu/~kempfj2/x4275.dmesg (GENERIC-DMESG simply ups the kernel message buffer to 100 pages from 10 so that I could capture everything into /var/run/boot.dmesg). The ASL: http://www.cs.rpi.edu/~kempfj2/x4275.asl And pciconf -vl: http://www.cs.rpi.edu/~kempfj2/x4275pci.txt sysctl hw.acpi: http://www.cs.rpi.edu/~kempfj2/x4275hw.acpi Boot -v with ACPI disabled explodes on discovering igb0 with a page fault in kernel mode: http://www.cs.rpi.edu/~kempfj2/x4275.verbose-noacpi And this is a non-verbose boot with acpi disabled: http://www.cs.rpi.edu/~kempfj2/x4275.noacpi What's the next step in debugging this, and how can I be of assistance? Thanks, -Jesse Kempf ------------------------------------------------------------------------ The information contained in this communication is intended only for the use of the recipient(s) named above. It may contain information that is privileged or confidential, and may be protected by State and/or Federal Regulations. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please return it to the sender immediately and delete the original message and any copy of it from your computer system. If you have any questions concerning this message, please contact the sender. ------------------------------------------------------------------------