From owner-freebsd-current@FreeBSD.ORG Tue May 3 02:30:00 2011 Return-Path: Delivered-To: current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 09F5C1065673 for ; Tue, 3 May 2011 02:30:00 +0000 (UTC) (envelope-from imb@protected-networks.net) Received: from sarah.protected-networks.net (sarah.protected-networks.net [IPv6:2001:470:1f07:4e1::1]) by mx1.freebsd.org (Postfix) with ESMTP id 60EB68FC0A for ; Tue, 3 May 2011 02:29:59 +0000 (UTC) Received: from toshi.auburn.protected-networks.net (toshi.auburn.protected-networks.net [202.12.127.84]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "Iain Butler", Issuer "RSA Class 2 Personal CA" (verified OK)) (Authenticated sender: imb@protected-networks.net) by sarah.protected-networks.net (Postfix) with ESMTPSA id 3084160E5 for ; Mon, 2 May 2011 22:29:58 -0400 (EDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=protected-networks.net; s=200705; t=1304389798; bh=yX9pn8jDAPt+6bPNdrrSuKDEm6Q/6vaqPYy4VnsLI8M=; h=Message-ID:Date:From:MIME-Version:To:Subject:Content-Type: Content-Transfer-Encoding; b=WW6pWQQi1+QASp0/dc7Hbj/FfR0qMDBZ3nHJUYfxrrJ5+xbRicwd6WOLhpW6dO7HS FDdAbq14mBrkSNVsM98ilLCq4xn+ia5mnRnC3eUSNHUxPhrhME4Lya9JfksAeOw DomainKey-Signature: a=rsa-sha1; s=200509; d=protected-networks.net; c=nofws; q=dns; h=message-id:date:from:user-agent:mime-version:to:subject: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=HmgolPw7z39i5N8cMMBCrvJC4I9It/T++EnTlxTlPsy6wvT4C3pIqiFS7Q0ssPEhO vS0bxiPuaUDC1UMO3BtrEWKVeD2umBp1hXYJI/dMmv1YuRx4MrfNEzey+sdgcEC Message-ID: <4DBF68A4.6050405@protected-networks.net> Date: Mon, 02 May 2011 22:29:56 -0400 From: Michael Butler User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.17) Gecko/20110429 Thunderbird/3.1.10 MIME-Version: 1.0 To: current@FreeBSD.org X-Enigmail-Version: 1.1.2 OpenPGP: id=0442D492 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: cardbus memory allocation problem X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 May 2011 02:30:00 -0000 I've stared at this for a (long) while but haven't come to any reasonable conclusion as to why it does what it does or how to fix it :-( Specifically, the BIOS in this machine doesn't set up a memory window for the cardbus controller nor does it properly configure the PCI bridge to route to the correct buses. BSD tries but allocates memory from the wrong space. My question is - how to get PCI-cardbus bridge to allocate memory inside the window of the parent PCI-PCI bridge? .. the bus tree looks like .. imb@toshi:/home/imb> sudo lspci -t -[0000:00]-+-00.0 +-02.0 +-02.1 +-1b.0 +-1c.0-[02]-- +-1c.1-[03-04]-- +-1c.2-[05-06]----00.0 +-1d.0 +-1d.1 +-1d.2 +-1d.3 +-1d.7 +-1e.0-[07]--+-06.0 | +-06.1 | +-06.2 | +-06.3 | \-08.0 +-1f.0 +-1f.2 \-1f.3 I've annotated the verbose dmesg below to highlight the issues .. pcib4: at device 30.0 on pci0 pcib4: domain 0 pcib4: secondary bus 7 pcib4: subordinate bus 7 *** subordinate bus needs to be '9' so as to include both '8' & '9' *** for the PCI-cardbus bridge pcib4: I/O decode 0x4000-0x4fff pcib4: memory decode 0xf0900000-0xf09fffff *** this memory widow is what I expected all children to allocate from pcib4: no prefetched decode pcib4: Subtractively decoded bridge. pci7: on pcib4 pci7: domain=0, physical bus=7 found-> vendor=0x104c, dev=0x8039, revid=0x00 domain=0, bus=7, slot=6, func=0 class=06-07-00, hdrtype=0x02, mfdev=1 cmdreg=0x0000, statreg=0x0210, cachelnsz=0 (dwords) lattimer=0x40 (1920 ns), mingnt=0x40 (16000 ns), maxlat=0x03 (750 ns) intpin=a, irq=255 powerspec 2 supports D0 D1 D2 D3 current D0 map[10]: type Memory, range 32, base 0, size 12, memory disabled *** silly uninitialized value here outside of the expected window found-> vendor=0x104c, dev=0x803a, revid=0x00 domain=0, bus=7, slot=6, func=1 class=0c-00-10, hdrtype=0x00, mfdev=1 cmdreg=0x0006, statreg=0x0210, cachelnsz=4 (dwords) lattimer=0x80 (3840 ns), mingnt=0x02 (500 ns), maxlat=0x04 (1000 ns) intpin=b, irq=11 powerspec 2 supports D0 D1 D2 D3 current D0 map[10]: type Memory, range 32, base 0xf0906000, size 11, enabled *** everything else under the pcib4 bridge is in the right window pcib4: requested memory range 0xf0906000-0xf09067ff: good map[14]: type Memory, range 32, base 0xf0900000, size 14, enabled pcib4: requested memory range 0xf0900000-0xf0903fff: good pcib4: matched entry for 7.6.INTB pcib4: slot 6 INTB hardwired to IRQ 17 found-> vendor=0x104c, dev=0x803b, revid=0x00 domain=0, bus=7, slot=6, func=2 class=01-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0006, statreg=0x0210, cachelnsz=4 (dwords) lattimer=0x80 (3840 ns), mingnt=0x07 (1750 ns), maxlat=0x04 (1000 ns) intpin=a, irq=11 powerspec 2 supports D0 D1 D2 D3 current D0 map[10]: type Memory, range 32, base 0xf0904000, size 12, enabled pcib4: requested memory range 0xf0904000-0xf0904fff: good pcib4: matched entry for 7.6.INTA pcib4: slot 6 INTA hardwired to IRQ 18 found-> vendor=0x104c, dev=0x803c, revid=0x00 domain=0, bus=7, slot=6, func=3 class=08-05-01, hdrtype=0x00, mfdev=1 cmdreg=0x0006, statreg=0x0210, cachelnsz=4 (dwords) lattimer=0x80 (3840 ns), mingnt=0x07 (1750 ns), maxlat=0x04 (1000 ns) intpin=a, irq=11 powerspec 2 supports D0 D1 D2 D3 current D0 map[10]: type Memory, range 32, base 0xf0906800, size 8, enabled pcib4: requested memory range 0xf0906800-0xf09068ff: good pcib4: matched entry for 7.6.INTA pcib4: slot 6 INTA hardwired to IRQ 18 found-> vendor=0x8086, dev=0x1092, revid=0x02 domain=0, bus=7, slot=8, func=0 class=02-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0017, statreg=0x0290, cachelnsz=16 (dwords) lattimer=0x42 (1980 ns), mingnt=0x08 (2000 ns), maxlat=0x38 (14000 ns) intpin=a, irq=10 powerspec 2 supports D0 D1 D2 D3 current D0 map[10]: type Memory, range 32, base 0xf0905000, size 12, enabled pcib4: requested memory range 0xf0905000-0xf0905fff: good map[14]: type I/O Port, range 32, base 0x4000, size 6, enabled pcib4: requested I/O range 0x4000-0x403f: in range pcib4: matched entry for 7.8.INTA pcib4: slot 8 INTA hardwired to IRQ 20 cbb0: at device 6.0 on pci7 pcib4: cbb0 requested memory range 0x0-0xffffffff: good *** what appears to be a "wildcard" alloc request cbb0: Lazy allocation of 0x1000 bytes rid 0x10 type 3 at 0xbf670000 *** but which isn't constrained to be within the parent bridge's space cardbus0: on cbb0 pccard0: <16-bit PCCard bus> on cbb0 pcib4: matched entry for 7.6.INTA pcib4: slot 6 INTA hardwired to IRQ 18 cbb0: PCI Configuration space: 0x00: 0x8039104c 0x02100007 0x06070000 0x00824000 0x10: 0xbf670000 0x020000a0 0x20090807 0xfffff000 ^^^^^^^^^^ ^ ^ ^ | | + primary bus | + secondary bus + subordinate bus 0x20: 0x00000000 0xfffff000 0x00000000 0xfffffffc 0x30: 0x00000000 0xfffffffc 0x00000000 0x07400112 0x40: 0xff101179 0x00000001 0x00000000 0x00000000 0x50: 0x00000000 0x00000000 0x00000000 0x00000000 0x60: 0x00000000 0x00000000 0x00000000 0x00000000 0x70: 0x00000000 0x00000000 0x00000000 0x00000000 0x80: 0x48409060 0x02830019 0x000f0000 0x01a01b22 0x90: 0x606600c0 0x00000000 0x00000000 0x00000000 0xa0: 0xfe120001 0x00c00000 0x00000000 0x00000000 0xb0: 0x00000000 0x00000000 0x00000000 0x00000000 0xc0: 0x00000000 0x00000000 0x00000000 0x00000000 0xd0: 0x00000000 0x00000000 0x00000000 0x00000000 0xe0: 0x00000000 0x00000000 0x00000000 0x00000000 0xf0: 0x44072b31 0x8d019449 0x00000000 0x00000000 'lspci' reports the following nonsense configuration .. 07:06.0 CardBus bridge: Texas Instruments PCIxx12 Cardbus Controller Subsystem: Toshiba America Info Systems Device ff10 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- SERR- Reset+ 16bInt- PostWrite+ 16-bit legacy interface ports at 0001 Any hints as to where to look to work around this would be appreciated, imb