From owner-freebsd-ppc@FreeBSD.ORG Wed Apr 16 05:54:56 2008 Return-Path: Delivered-To: freebsd-ppc@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4940B106566C; Wed, 16 Apr 2008 05:54:56 +0000 (UTC) (envelope-from xcllnt@mac.com) Received: from smtpoutm.mac.com (smtpoutm.mac.com [17.148.16.78]) by mx1.freebsd.org (Postfix) with ESMTP id 294A68FC1A; Wed, 16 Apr 2008 05:54:55 +0000 (UTC) (envelope-from xcllnt@mac.com) Received: from mac.com (asmtp002-s [10.150.69.65]) by smtpoutm.mac.com (Xserve/smtpout015/MantshX 4.0) with ESMTP id m3G5st0H007611; Tue, 15 Apr 2008 22:54:55 -0700 (PDT) Received: from [192.168.1.100] (209-128-86-226.bayarea.net [209.128.86.226]) (authenticated bits=0) by mac.com (Xserve/asmtp002/MantshX 4.0) with ESMTP id m3G5sosn028894 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 15 Apr 2008 22:54:52 -0700 (PDT) Message-Id: <9F6F2C83-79F1-4463-B9FF-4BBEB55B95B2@mac.com> From: Marcel Moolenaar To: grehan@freebsd.org In-Reply-To: <48054DE6.10508@freebsd.org> Content-Type: multipart/mixed; boundary=Apple-Mail-1-237339805 Mime-Version: 1.0 (Apple Message framework v919.2) Date: Tue, 15 Apr 2008 22:54:50 -0700 References: <4804AE13.2060600@uchicago.edu> <4804C9E9.6010303@freebsd.org> <5CC81F06-7B59-4163-9AB8-2ACE4235A5AA@mac.com> <4804DD02.10304@freebsd.org> <058EEFE3-09D7-447A-93AB-3E90EC59ECDC@mac.com> <48053E46.4090700@freebsd.org> <48054DE6.10508@freebsd.org> X-Mailer: Apple Mail (2.919.2) Cc: freebsd-ppc@freebsd.org Subject: Re: kernel stacks [eas: Re: G5 Bridge-mode MMU] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Apr 2008 05:54:56 -0000 --Apple-Mail-1-237339805 Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit On Apr 15, 2008, at 5:52 PM, Peter Grehan wrote: > Hi Marcel, > >>> Are you sure it isn't a genuine stack overflow ? >> Positive. The panic happens after 4KB of stack has been used. >>> You may be able to tell by bumping the size of tmpstk on a non- >>> kstack0 boot and see how far up it's been used. >> The backtrace also shows that. From inner-most to out-most function >> in >> the backtrace the stack pointers are roughly 4KB apart. > > Can you send the code snippet that you're using to set up the > stack ? I can desk-check that, and then use it for my testing so we > have the exact same setup. Diff attached. This is the problem I'm running into: Kernel entry at 0x100100 ... GDB: debug ports: uartGDB: current port: uart KDB: debugger backends: ddb gdb KDB: current backend: ddb Copyright (c) 1992-2008 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.0-CURRENT #8: Tue Apr 15 22:44:23 PDT 2008 marcel@xserve.xcllnt.net:/nfs/freebsd/8.x/src/sys/powerpc/compile/ XSERVE WARNING: WITNESS option enabled, expect reduced performance. cpu0: Motorola PowerPC 7455 revision 2.1, 1000.00 MHz cpu0: HID0 8450c0bc real memory = 527314944 (502 MB) avail memory = 510078976 (486 MB) nexus0: unin0: on nexus0 unin0: Version 36 pcib0: on nexus0 pci0: on pcib0 bge0: mem 0xa0000000-0xa000ffff irq 48 at device 16.0 on pci0 miibus0: on bge0 brgphy0: PHY 1 on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto bge0: Ethernet address: 00:03:93:c0:54:18 bge0: [ITHREAD] pcib1: on nexus0 pci1: on pcib1 pcib2: at device 13.0 on pci1 pci2: on pcib2 macio0: mem 0x80000000-0x8007ffff at device 7.0 on pci2 openpic0: mem 0x40000-0x7ffff on macio0 scc0: mem 0x13000-0x13fff,0x8400-0x84ff, 0x8500-0x85ff,0x8600-0x86ff,0x8700-0x87ff irq 22,23 on macio0 scc0: [FILTER] scc0: [FILTER] uart0: on scc0 uart0: [FILTER] uart0: console (57600,n,8,1) uart1: on scc0 uart1: [FILTER] ata0 mem 0x1f000-0x1ffff,0x8a00-0x8aff irq 19 on macio0 ata0: [ITHREAD] ohci0: mem 0x80081000-0x80081fff irq 27 at device 8.0 on pci2 ohci0: [GIANT-LOCKED] ohci0: [ITHREAD] usb0: OHCI version 1.0 usb0: on ohci0 usb0: USB revision 1.0 [thread pid 0 tid 100000 ] Stopped at 0x3e9cc0: stwux r0, r1, r9, db> bt Tracing pid 0 tid 100000 td 0x4cb340 0xd00040f0: at usbd_transfer+0xb0 0xd0004110: at usbd_sync_transfer+0x20 0xd0004120: at usbd_do_request_flags_pipe+0xa4 0xd0004170: at usbd_do_request_flags+0x40 0xd0004190: at usbd_get_string_desc+0x78 0xd00041c0: at usbd_get_string+0x94 0xd00042f0: at usbd_devinfo_vp+0x64 0xd0004310: at usbd_devinfo+0x48 0xd0004440: at usbd_new_device+0x5ac 0xd00048b0: at usb_attach+0x130 0xd0004a60: at device_attach+0x338 0xd0004a90: at device_probe_and_attach+0x134 0xd0004ab0: at ohci_pci_attach+0x6a8 0xd0004af0: at device_attach+0x338 0xd0004b20: at device_probe_and_attach+0x134 0xd0004b40: at bus_generic_attach+0x28 0xd0004b50: at pci_attach+0x118 0xd0004b80: at device_attach+0x338 0xd0004bb0: at device_probe_and_attach+0x134 0xd0004bd0: at bus_generic_attach+0x28 0xd0004be0: at ofw_pcib_pci_attach+0x78 0xd0004c10: at device_attach+0x338 0xd0004c40: at device_probe_and_attach+0x134 0xd0004c60: at bus_generic_attach+0x28 0xd0004c70: at pci_attach+0x118 0xd0004ca0: at device_attach+0x338 0xd0004cd0: at device_probe_and_attach+0x134 0xd0004cf0: at bus_generic_attach+0x28 0xd0004d00: at uninorth_attach+0x3e8 0xd0004d70: at device_attach+0x338 0xd0004da0: at device_probe_and_attach+0x134 0xd0004dc0: at bus_generic_attach+0x28 0xd0004dd0: at device_attach+0x338 0xd0004e00: at device_probe_and_attach+0x134 0xd0004e20: at root_bus_configure+0x30 0xd0004e30: at configure+0x14 0xd0004e40: at mi_startup+0x11c 0xd0004e70: at __start+0x98 db> show reg r0 0xd00040f0 r1 0xd00040b0 r2 0 r3 0xca76c0 r4 0 r5 0xd00041c8 r6 0x2 r7 0x1b998c usbd_start_transfer r8 0 r9 0xfffffee0 r10 0x200 dsisize+0x15c r11 0xd00040f0 r12 0x8c0 dsisize+0x81c r13 0 r14 0 r15 0 r16 0xcadd80 r17 0x100 dsisize+0x5c r18 0 r19 0xcae100 r20 0 r21 0xca7594 r22 0xcae080 r23 0x5 vectrapsize+0x1 r24 0xcade00 r25 0xd00041a0 r26 0x4 vectrapsize r27 0x1b998c usbd_start_transfer r28 0xc25600 r29 0xd00040b0 r30 0xc25600 r31 0xd00040b0 srr0 0x3e9cc0 bus_dmamap_load+0x4c srr1 0x3032 dsisize+0x2f8e lr 0x1ba190 usbd_transfer+0xb4 ctr 0 cr 0x24000082 xer 0 dar 0xd0003f90 dsisr 0 0x3e9cc0: stwux r0, r1, r9, db> As the backtrace shows, about 4K has been used, which means we're running into the second page. The reason we're hitting the debugger without a panic is because we're tripping over the stack overflow logic. In other words: we have a DSI trap. -- Marcel Moolenaar xcllnt@mac.com --Apple-Mail-1-237339805 Content-Disposition: attachment; filename=ppc.diff Content-Type: application/octet-stream; x-unix-mode=0644; name="ppc.diff" Content-Transfer-Encoding: 7bit Index: locore.S =================================================================== RCS file: /home/ncvs/src/sys/powerpc/aim/locore.S,v retrieving revision 1.25 diff -u -r1.25 locore.S --- locore.S 7 Mar 2008 22:27:05 -0000 1.25 +++ locore.S 16 Apr 2008 01:08:25 -0000 @@ -182,6 +182,7 @@ mr 7,21 bl powerpc_init + mr %r1, %r3 bl mi_startup b OF_exit Index: machdep.c =================================================================== RCS file: /home/ncvs/src/sys/powerpc/aim/machdep.c,v retrieving revision 1.111 diff -u -r1.111 machdep.c --- machdep.c 16 Mar 2008 10:58:08 -0000 1.111 +++ machdep.c 16 Apr 2008 05:40:29 -0000 @@ -132,9 +132,6 @@ static struct pcpu pcpu0; static struct trapframe frame0; -vm_offset_t kstack0; -vm_offset_t kstack0_phys; - char machine[] = "powerpc"; SYSCTL_STRING(_hw, HW_MACHINE, machine, CTLFLAG_RD, machine, 0, ""); @@ -145,7 +142,7 @@ static void cpu_startup(void *); SYSINIT(cpu, SI_SUB_CPU, SI_ORDER_FIRST, cpu_startup, NULL); -void powerpc_init(u_int, u_int, u_int, void *); +u_int powerpc_init(u_int, u_int, u_int, void *); int save_ofw_mapping(void); int restore_ofw_mapping(void); @@ -248,11 +245,11 @@ extern void *dblow, *dbsize; extern void *vectrap, *vectrapsize; -void +u_int powerpc_init(u_int startkernel, u_int endkernel, u_int basekernel, void *mdp) { struct pcpu *pc; - vm_offset_t end, off; + vm_offset_t end; void *kmdp; char *env; @@ -295,7 +292,6 @@ pc = &pcpu0; pcpu_init(pc, 0, sizeof(struct pcpu)); pc->pc_curthread = &thread0; - pc->pc_curpcb = thread0.td_pcb; pc->pc_cpuid = 0; __asm __volatile("mtsprg 0, %0" :: "r"(pc)); @@ -379,15 +375,12 @@ /* * Finish setting up thread0. */ - thread0.td_kstack = kstack0; thread0.td_pcb = (struct pcb *) - (thread0.td_kstack + KSTACK_PAGES * PAGE_SIZE) - 1; + ((thread0.td_kstack + thread0.td_kstack_pages * PAGE_SIZE - + sizeof(struct pcb)) & ~0xfU); + pc->pc_curpcb = thread0.td_pcb; - /* - * Map and initialise the message buffer. - */ - for (off = 0; off < round_page(MSGBUF_SIZE); off += PAGE_SIZE) - pmap_kenter((vm_offset_t)msgbufp + off, msgbuf_phys + off); + /* Initialise the message buffer. */ msgbufinit(msgbufp, MSGBUF_SIZE); #ifdef KDB @@ -395,6 +388,8 @@ kdb_enter(KDB_WHY_BOOTFLAGS, "Boot flags requested debugger"); #endif + + return (((uintptr_t)thread0.td_pcb - 16) & ~15); } void Index: mmu_oea.c =================================================================== RCS file: /home/ncvs/src/sys/powerpc/aim/mmu_oea.c,v retrieving revision 1.117 diff -u -r1.117 mmu_oea.c --- mmu_oea.c 14 Dec 2007 22:39:34 -0000 1.117 +++ mmu_oea.c 16 Apr 2008 05:37:46 -0000 @@ -785,11 +785,6 @@ MTX_RECURSE); /* - * Allocate the message buffer. - */ - msgbuf_phys = moea_bootstrap_alloc(MSGBUF_SIZE, 0); - - /* * Initialise the unmanaged pvo pool. */ moea_bpvo_pool = (struct pvo_entry *)moea_bootstrap_alloc( @@ -872,48 +867,56 @@ kernel_pmap->pm_active = ~0; /* - * Allocate a kernel stack with a guard page for thread0 and map it - * into the kernel page map. + * Initialize hardware. */ - pa = moea_bootstrap_alloc(KSTACK_PAGES * PAGE_SIZE, 0); - kstack0_phys = pa; - kstack0 = virtual_avail + (KSTACK_GUARD_PAGES * PAGE_SIZE); - CTR2(KTR_PMAP, "moea_bootstrap: kstack0 at %#x (%#x)", kstack0_phys, - kstack0); - virtual_avail += (KSTACK_PAGES + KSTACK_GUARD_PAGES) * PAGE_SIZE; - for (i = 0; i < KSTACK_PAGES; i++) { - pa = kstack0_phys + i * PAGE_SIZE; - va = kstack0 + i * PAGE_SIZE; - moea_kenter(mmup, va, pa); - TLBIE(va); + for (i = 0; i < 16; i++) { + mtsrin(i << ADDR_SR_SHFT, EMPTY_SEGMENT); } + __asm __volatile ("mtsr %0,%1" + :: "n"(KERNEL_SR), "r"(KERNEL_SEGMENT)); + __asm __volatile ("mtsr %0,%1" + :: "n"(KERNEL2_SR), "r"(KERNEL2_SEGMENT)); + __asm __volatile ("sync; mtsdr1 %0; isync" + :: "r"((u_int)moea_pteg_table | (moea_pteg_mask >> 10))); + tlbia(); /* - * Calculate the last available physical address. + * Allocate a kernel stack with a guard page for thread0 and map it + * into the kernel page map. */ - for (i = 0; phys_avail[i + 2] != 0; i += 2) - ; - Maxmem = powerpc_btop(phys_avail[i + 1]); + pa = moea_bootstrap_alloc(KSTACK_PAGES * PAGE_SIZE, PAGE_SIZE); + va = virtual_avail + KSTACK_GUARD_PAGES * PAGE_SIZE; + virtual_avail = va + KSTACK_PAGES * PAGE_SIZE; + CTR2(KTR_PMAP, "moea_bootstrap: kstack0 at %#x (%#x)", pa, va); + thread0.td_kstack = va; + thread0.td_kstack_pages = KSTACK_PAGES; + + for (i = 0; i < KSTACK_PAGES; i++) { + moea_kenter(mmup, va, pa);; + pa += PAGE_SIZE; + va += PAGE_SIZE; + } /* * Allocate virtual address space for the message buffer. */ + pa = msgbuf_phys = moea_bootstrap_alloc(MSGBUF_SIZE, PAGE_SIZE); msgbufp = (struct msgbuf *)virtual_avail; + va = virtual_avail; virtual_avail += round_page(MSGBUF_SIZE); + while (va < virtual_avail) { + moea_kenter(mmup, va, pa);; + pa += PAGE_SIZE; + va += PAGE_SIZE; + } + /* - * Initialize hardware. + * Calculate the last available physical address. */ - for (i = 0; i < 16; i++) { - mtsrin(i << ADDR_SR_SHFT, EMPTY_SEGMENT); - } - __asm __volatile ("mtsr %0,%1" - :: "n"(KERNEL_SR), "r"(KERNEL_SEGMENT)); - __asm __volatile ("mtsr %0,%1" - :: "n"(KERNEL2_SR), "r"(KERNEL2_SEGMENT)); - __asm __volatile ("sync; mtsdr1 %0; isync" - :: "r"((u_int)moea_pteg_table | (moea_pteg_mask >> 10))); - tlbia(); + for (i = 0; phys_avail[i + 2] != 0; i += 2) + ; + Maxmem = powerpc_btop(phys_avail[i + 1]); pmap_bootstrapped++; } --Apple-Mail-1-237339805 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit --Apple-Mail-1-237339805--