From owner-freebsd-current@FreeBSD.ORG Wed Mar 3 06:41:51 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E5C5C16A4D6 for ; Wed, 3 Mar 2004 06:41:50 -0800 (PST) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 79A6943D1F for ; Wed, 3 Mar 2004 06:41:50 -0800 (PST) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.12.10/8.12.10) with ESMTP id i23EfnF6026093 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Wed, 3 Mar 2004 09:41:49 -0500 (EST) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.12.9p2/8.12.9/Submit) id i23EfhdR080411; Wed, 3 Mar 2004 09:41:44 -0500 (EST) (envelope-from gallatin) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <16453.61095.940884.597029@grasshopper.cs.duke.edu> Date: Wed, 3 Mar 2004 09:41:43 -0500 (EST) To: freebsd-current@freebsd.org X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Subject: reproducable "panic: pmap_enter: attempted pmap_enter on 4MB page" X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Mar 2004 14:41:51 -0000 I can reproducably panic -current with the above message by allocating a lot of memory in 4MB chunks in an out-of-tree os-bypass network driver. I've reproduced the failure in the appended stand-alone kernel module, which allocates 100MB of kernel memory in 4MB chunks. I don't think I'm running out of address space, as I've bumped that up to over 400MB: % sysctl -a | grep mem <....> vm.kmem_size: 429391872 hw.physmem: 532041728 hw.usermem: 513851392 <....> This is a single P4 (dual-core HTT), with 512MB of ram running an SMP, ULE no-INVARIANTS, no-WITNESS kernel. Adding WITNESS and INVARIANTS seems to prevent the panic somehow. The panic looks like this: #0 doadump () at ../../../kern/kern_shutdown.c:240 #1 0xc04dd69f in boot (howto=260) at ../../../kern/kern_shutdown.c:374 #2 0xc04dd98f in __panic () at ../../../kern/kern_shutdown.c:552 #3 0xc05e4990 in pmap_enter (pmap=0xc06a1fc0, va=3362578432, m=0xc10cca48, prot=7 '\a', wired=1) at ../../../i386/i386/pmap.c:1949 #4 0xc05abe41 in kmem_malloc (map=0xc0c3b0a0, size=4194304, flags=258) at ../../../vm/vm_kern.c:444 #5 0xc05b853a in page_alloc (zone=0x0, bytes=4194304, pflag=0x0, wait=258) at ../../../vm/uma_core.c:855 #6 0xc05b986b in uma_large_malloc (size=4194304, wait=258) at ../../../vm/uma_core.c:2037 #7 0xc04d5661 in malloc (size=4194304, type=0xc064b560, flags=258) at ../../../kern/kern_malloc.c:255 #8 0xc448f4b8 in load () from ./pmap_panic.ko #9 0xc04d618e in module_register_init (arg=0xc4490548) at ../../../kern/kern_module.c:108 #10 0xc04d1a61 in linker_file_sysinit (lf=0xc448d200) at ../../../kern/kern_linker.c:192 #11 0xc04d1d15 in linker_load_file (filename=0xc41e76d0 "./pmap_panic.ko", result=0xe981bcb0) at ../../../kern/kern_linker.c:357 #12 0xc04d3d6b in linker_load_module (kldname=0xc41e76d0 "./pmap_panic.ko", modname=0x0, parent=0x0, verinfo=0x0, lfpp=0xe981bcdc) at ../../../kern/kern_linker.c:1670 #13 0xc04d270b in kldload (td=0xc4333930, uap=0x0) at ../../../kern/kern_linker.c:773 #14 0xc05e7d7f in syscall (frame= {tf_fs = 47, tf_es = 47, tf_ds = 47, tf_edi = 0, tf_esi = -1077942136, tf_ebp = -1077942212, tf_isp = -377373324, tf_ebx = 0, tf_edx = -1077942140, tf_ecx = 2, tf_eax = 304, tf_trapno = 12, tf_err = 2, tf_eip = 671833943, tf_cs = 31, tf_eflags = 662, tf_esp = -1077942260, tf_ss = 47}) at ../../../i386/i386/trap.c:1008 #15 0x280b5f57 in ?? () (kgdb) frame 3 #3 0xc05e4990 in pmap_enter (pmap=0xc06a1fc0, va=3362578432, m=0xc10cca48, prot=7 '\a', wired=1) at ../../../i386/i386/pmap.c:1949 1949 panic("pmap_enter: attempted pmap_enter on 4MB page"); (kgdb) info locals pa = 269754368 pte = (pt_entry_t *) 0xbff21b38 opa = 0 origpte = 480 newpte = 0 mpte = 0x0 (kgdb) frame 4 #4 0xc05abe41 in kmem_malloc (map=0xc0c3b0a0, size=4194304, flags=258) at ../../../vm/vm_kern.c:444 444 pmap_enter(kernel_pmap, addr + i, m, VM_PROT_ALL, 1); (kgdb) info locals offset = 140058624 i = 2347008 entry = 0xc0c39870 addr = 3360231424 m = 0xc10cca48 pflags = 98 (kgdb) frame 5 #5 0xc05b853a in page_alloc (zone=0x0, bytes=4194304, pflag=0x0, wait=258) at ../../../vm/uma_core.c:855 855 p = (void *) kmem_malloc(kmem_map, bytes, wait); (kgdb) info locals p = (void *) 0x0 (kgdb) frame 6 #6 0xc05b986b in uma_large_malloc (size=4194304, wait=258) at ../../../vm/uma_core.c:2037 2037 mem = page_alloc(NULL, size, &flags, wait); (kgdb) info local mem = (void *) 0x102 slab = 0xc43cfb88 flags = 2 '\002' And a look at the page we're trying to pmap_enter: (kgdb) frame 4 #4 0xc05abe41 in kmem_malloc (map=0xc0c3b0a0, size=4194304, flags=258) at ../../../vm/vm_kern.c:444 444 pmap_enter(kernel_pmap, addr + i, m, VM_PROT_ALL, 1); (kgdb) p/x *m $3 = { pageq = { tqe_next = 0x0, tqe_prev = 0xc10cc148 }, listq = { tqe_next = 0xc10c2890, tqe_prev = 0xc10bbc08 }, left = 0xc10bbc00, right = 0xc10c2890, object = 0xc0688a00, pindex = 0x87cf, phys_addr = 0x10142000, md = { pv_list_count = 0x0, pv_list = { tqh_first = 0x0, tqh_last = 0xc10cca74 } }, queue = 0x0, flags = 0x841, pc = 0x2, wire_count = 0x1, hold_count = 0x0, act_count = 0x0, busy = 0x0, valid = 0xff, dirty = 0x0, cow = 0x0 } Here is the module: #include #include #include #include #include #include #include #define ALLOC_SIZE (1024 * 4096) #define ALLOC_COUNT 25 static void *pmp_mem[ALLOC_COUNT]; static int load (struct module *module, int cmd, void *arg) { int i; int error = 0; switch (cmd) { case MOD_LOAD : printf ("allocating %d bytes %d times\n", ALLOC_SIZE, ALLOC_COUNT); for (i = 0; i < ALLOC_COUNT; i++) pmp_mem[i] = malloc(ALLOC_SIZE, M_DEVBUF, M_WAITOK|M_ZERO); break; case MOD_UNLOAD : printf ("freeing memory\n"); for (i = 0; i < ALLOC_COUNT; i++) free(pmp_mem[i], M_DEVBUF); break; default : error = EINVAL; break; } return error; } static moduledata_t mod_data= { "pmap_panic", load, 0 }; DECLARE_MODULE(pmap_panic, mod_data, SI_SUB_EXEC, SI_ORDER_ANY); Thanks in advance for any help... Drew