From owner-freebsd-sparc64@FreeBSD.ORG Mon Jun 6 20:38:14 2005 Return-Path: X-Original-To: sparc64@FreeBSD.org Delivered-To: freebsd-sparc64@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D649116A45C; Mon, 6 Jun 2005 20:38:14 +0000 (GMT) (envelope-from dwhite@gumbysoft.com) Received: from carver.gumbysoft.com (carver.gumbysoft.com [66.220.23.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 90CF643D5E; Mon, 6 Jun 2005 20:38:14 +0000 (GMT) (envelope-from dwhite@gumbysoft.com) Received: by carver.gumbysoft.com (Postfix, from userid 1000) id 6D81B72DD4; Mon, 6 Jun 2005 13:38:14 -0700 (PDT) Received: from localhost (localhost [127.0.0.1]) by carver.gumbysoft.com (Postfix) with ESMTP id 68A0772DCB; Mon, 6 Jun 2005 13:38:14 -0700 (PDT) Date: Mon, 6 Jun 2005 13:38:14 -0700 (PDT) From: Doug White To: Hiroki Sato In-Reply-To: <20050607.015510.21897573.hrs@allbsd.org> Message-ID: <20050606132756.X16994@carver.gumbysoft.com> References: <20050530.025302.64839649.hrs@allbsd.org> <429A7B4D.3080102@samsco.org> <20050607.015510.21897573.hrs@allbsd.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: sparc64@FreeBSD.org Subject: Re: E4500 with 24GB RAM X-BeenThere: freebsd-sparc64@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting FreeBSD to the Sparc List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Jun 2005 20:38:15 -0000 On Tue, 7 Jun 2005, Hiroki Sato wrote: > Scott Long wrote > in <429A7B4D.3080102@samsco.org>: > > sc> I'd guess that you're the first to have access to so much memory > sc> and a machine to hold it. The error means that vm_map_find() > sc> returned KERN_NO_SPACE. It could be that there is a 64-bit bug > sc> in the code, or it could be that the page tables to index so much > sc> memory consume all available space in the kernel map. > > Thanks. Jake gave me an advice about kern.maxbcache tunable, and > I finally make it boot with kern.maxbcache=524288000. > However, the hme driver seems to have a problem: > > |panic: iommu_enter: XXX: physical address too large (0x5e3fcc000) This panic occurs since the IOMMU can only handle physical addresses up to 34 bits (16GB), it appears. From src/sys/sparc64/include/iommureg.h: 67 #define IOMMU_BITS 34 68 #define IOMMU_MAXADDR (1UL << IOMMU_BITS) Since the panic is in a KASSERT thats why it only pops up under INVARIANTS. However the bus_dma tags specify that hme can only handle 32 bit addresses so it should be using a bounce buffer. I'm guessing that bus_dma needs to be taught about this limitation and use bounce buffers for the IOMMU on >16GB systems. Someone should also research this value a bit more ... I wonder if its from some older system (E450?), and newer machines have larger limits. I would hope they'd build E4500s with IOMMUs that can address all of physical memory. > |cpuid = 0 > |KDB: enter: panic > |[thread pid 442 tid 100215 ] > |Stopped at kdb_enter+0x3c: ta %xcc, 1 > |db> tr > |Tracing pid 442 tid 100215 td 0xfffff800af657b80 > |panic() at panic+0x16c > |iommu_enter() at iommu_enter+0x3c > |iommu_dvmamap_load_buffer() at iommu_dvmamap_load_buffer+0x118 > |iommu_dvmamap_load_mbuf() at iommu_dvmamap_load_mbuf+0x170 > |hme_load_txmbuf() at hme_load_txmbuf+0x68 > |hme_start_locked() at hme_start_locked+0x1bc > |hme_start() at hme_start+0x2c > |if_start() at if_start+0xa0 > |ether_output_frame() at ether_output_frame+0x244 > |ether_output() at ether_output+0x474 > |ip_output() at ip_output+0xa98 > |udp_output() at udp_output+0x5b4 > |udp_send() at udp_send+0x14 > |sosend() at sosend+0x654 > > When INVARIANTS enabled this panic occurs instantly. > > -- Doug White | FreeBSD: The Power to Serve dwhite@gumbysoft.com | www.FreeBSD.org