From owner-freebsd-stable@FreeBSD.ORG Tue Aug 12 13:29:09 2003 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D13C937B404; Tue, 12 Aug 2003 13:29:09 -0700 (PDT) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4A2E443FBF; Tue, 12 Aug 2003 13:29:08 -0700 (PDT) (envelope-from mike@sentex.net) Received: from simian.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.12.9/8.12.8) with ESMTP id h7CKT321081604; Tue, 12 Aug 2003 16:29:03 -0400 (EDT) (envelope-from mike@sentex.net) Message-Id: <5.2.0.9.0.20030812161901.09b02558@209.112.4.2> X-Sender: mdtpop@209.112.4.2 (Unverified) X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Tue, 12 Aug 2003 16:31:02 -0400 To: stable@freebsd.org From: Mike Tancsa Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: By Sentex Communications (lava/20020517) cc: tegge@freebsd.org Subject: panic with today's stable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Aug 2003 20:29:10 -0000 Did cvsup on a machine that does just mail processing (well, a lot of spam scanning) and it crashed not too much later. This kernel does not include MFC src/sys/kern/sys_process.c revisions 1.111 and 1.112: Use kmem_alloc_nofault() rather than kmem_alloc_pageable() in procfs_rwmem(). Use vm_page_hold() in place of vm_page_wire() since the page can be freed. Don't hold extra reference to the containing vm object while page is mapped. which went in after I cvsup'd. Is there a change the crash is due to the above ? Hardware is pretty generic, we have the same configs on a number of other boxes. I know there were quite a few VM changes as well as twe changes since the late May kernel it was running prior to today. ns4# gdb -k /kernel.debug /var/crash/vmcore.1 GNU gdb 4.18 (FreeBSD) Copyright 1998 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-unknown-freebsd"...Deprecated bfd_read called at /usr/src/gnu/usr.bin/binutils/gdb/../../../../contrib/gdb/gdb/dbxread.c line 2627 in elfstab_build_psymtabs Deprecated bfd_read called at /usr/src/gnu/usr.bin/binutils/gdb/../../../../contrib/gdb/gdb/dbxread.c line 933 in fill_symbuf IdlePTD at phsyical address 0x003ad000 initial pcb at physical address 0x0030cb40 panicstr: page fault panic messages: --- Fatal trap 12: page fault while in kernel mode fault virtual address = 0xc408b0c8 fault code = supervisor read, page not present instruction pointer = 0x8:0xc025fa19 stack pointer = 0x10:0xe8ba6e24 frame pointer = 0x10:0xe8ba6e28 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 31044 (perl) interrupt mask = net tty bio cam trap number = 12 panic: page fault syncing disks... 11 2 2 2 2 2 2 2 8 done Uptime: 4h39m31s twe0: failed to delete unit 0 dumping to dev #twed/1, offset 2176 dump 1022 1021 1020 1019 1018 1017 1016 1015 1014 1013 1012 1011 1010 1009 1008 1007 1006 1005 1004 6 5 4 3 2 1 0 --- #0 dumpsys () at /usr/src/sys/kern/kern_shutdown.c:487 487 if (dumping++) { (kgdb) bt #0 dumpsys () at /usr/src/sys/kern/kern_shutdown.c:487 #1 0xc01618fc in boot (howto=256) at /usr/src/sys/kern/kern_shutdown.c:316 #2 0xc0161d49 in panic (fmt=0xc02db8cc "%s") at /usr/src/sys/kern/kern_shutdown.c:595 #3 0xc028cb94 in trap_fatal (frame=0xe8ba6de4, eva=3288903880) at /usr/src/sys/i386/i386/trap.c:974 #4 0xc028c829 in trap_pfault (frame=0xe8ba6de4, usermode=0, eva=3288903880) at /usr/src/sys/i386/i386/trap.c:867 #5 0xc028c38f in trap (frame={tf_fs = 16, tf_es = 16, tf_ds = 16, tf_edi = 7314456, tf_esi = -386904664, tf_ebp = -390435288, tf_isp = -390435312, tf_ebx = -1053617864, tf_edx = -1006063416, tf_ecx = -386904664, tf_eax = -1006063424, tf_trapno = 12, tf_err = 0, tf_eip = -1071252967, tf_cs = 8, tf_eflags = 66182, tf_esp = -1053617864, tf_ss = -390435260}) at /usr/src/sys/i386/i386/trap.c:466 #6 0xc025fa19 in vm_page_remove (m=0xc1331138) at /usr/src/sys/vm/vm_page.c:462 #7 0xc02600ac in vm_page_free_toq (m=0xc1331138) at /usr/src/sys/vm/vm_page.c:1104 #8 0xc025de22 in vm_object_terminate (object=0xe8f04da8) at /usr/src/sys/vm/vm_page.h:514 #9 0xc025dce0 in vm_object_deallocate (object=0xe8f04da8) at /usr/src/sys/vm/vm_object.c:399 #10 0xc025af21 in vm_map_entry_delete (map=0xe8b3e040, entry=0xe8906c60) at /usr/src/sys/vm/vm_map.c:2054 #11 0xc025b0d2 in vm_map_delete (map=0xe8b3e040, start=0, end=3217031168) at /usr/src/sys/vm/vm_map.c:2174 #12 0xc025b161 in vm_map_remove (map=0xe8b3e040, start=0, end=3217031168) at /usr/src/sys/vm/vm_map.c:2199 #13 0xc0159c3d in exit1 (p=0xe8a89ea0, rv=1054720) at /usr/src/sys/kern/kern_exit.c:226 #14 0xc0159a09 in sys_exit (p=0xe8a89ea0, uap=0xe8ba6f80) at /usr/src/sys/kern/kern_exit.c:104 #15 0xc028ce05 in syscall2 (frame={tf_fs = -1071185873, tf_es = 47, tf_ds = 47, tf_edi = 0, tf_esi = -1, tf_ebp = -1077936848, tf_isp = -390434860, tf_ebx = 672950156, tf_edx = 673025784, tf_ecx = 5, tf_eax = 1, tf_trapno = 12, tf_err = 2, tf_eip = 672622500, tf_cs = 31, tf_eflags = 647, tf_esp = -1077936892, tf_ss = 47}) at /usr/src/sys/i386/i386/trap.c:1175 #16 0xc027fb05 in Xint0x80_syscall () Cannot access memory at address 0xbfbffd30. (kgdb) (kgdb) list 482 dumpsys(void) 483 { 484 int error; 485 486 savectx(&dumppcb); 487 if (dumping++) { 488 printf("Dump already in progress, bailing...\n"); 489 return; 490 } 491 if (!dodump) (kgdb) up #1 0xc01618fc in boot (howto=256) at /usr/src/sys/kern/kern_shutdown.c:316 316 dumpsys(); (kgdb) list 311 * been completed. 312 */ 313 EVENTHANDLER_INVOKE(shutdown_post_sync, howto); 314 splhigh(); 315 if ((howto & (RB_HALT|RB_DUMP)) == RB_DUMP && !cold) 316 dumpsys(); 317 318 /* Now that we're going to really halt the system... */ 319 EVENTHANDLER_INVOKE(shutdown_final, howto); 320 (kgdb) up #2 0xc0161d49 in panic (fmt=0xc02db8cc "%s") at /usr/src/sys/kern/kern_shutdown.c:595 595 boot(bootopt); (kgdb) list 590 591 #if defined(DDB) 592 if (debugger_on_panic) 593 Debugger ("panic"); 594 #endif 595 boot(bootopt); 596 } 597 598 /* 599 * Support for poweroff delay. (kgdb) up #3 0xc028cb94 in trap_fatal (frame=0xe8ba6de4, eva=3288903880) at /usr/src/sys/i386/i386/trap.c:974 974 panic("%s", trap_msg[type]); (kgdb) list 969 if ((debugger_on_panic || db_active) && kdb_trap(type, 0, frame)) 970 return; 971 #endif 972 printf("trap number = %d\n", type); 973 if (type <= MAX_TRAP_MSG) 974 panic("%s", trap_msg[type]); 975 else 976 panic("unknown/reserved trap"); 977 } 978 (kgdb) up #4 0xc028c829 in trap_pfault (frame=0xe8ba6de4, usermode=0, eva=3288903880) at /usr/src/sys/i386/i386/trap.c:867 867 trap_fatal(frame, eva); (kgdb) list 862 if (!usermode) { 863 if (intr_nesting_level == 0 && curpcb && curpcb->pcb_onfault) { 864 frame->tf_eip = (int)curpcb->pcb_onfault; 865 return (0); 866 } 867 trap_fatal(frame, eva); 868 return (-1); 869 } 870 871 /* kludge to pass faulting virtual address to sendsig */ (kgdb) up #5 0xc028c38f in trap (frame={tf_fs = 16, tf_es = 16, tf_ds = 16, tf_edi = 7314456, tf_esi = -386904664, tf_ebp = -390435288, tf_isp = -390435312, tf_ebx = -1053617864, tf_edx = -1006063416, tf_ecx = -386904664, tf_eax = -1006063424, tf_trapno = 12, tf_err = 0, tf_eip = -1071252967, tf_cs = 8, tf_eflags = 66182, tf_esp = -1053617864, tf_ss = -390435260}) at /usr/src/sys/i386/i386/trap.c:466 466 (void) trap_pfault(&frame, FALSE, eva); (kgdb) list 461 kernel_trap: 462 /* kernel trap */ 463 464 switch (type) { 465 case T_PAGEFLT: /* page fault */ 466 (void) trap_pfault(&frame, FALSE, eva); 467 return; 468 469 case T_DNA: 470 #if NNPX > 0 (kgdb) up #6 0xc025fa19 in vm_page_remove (m=0xc1331138) at /usr/src/sys/vm/vm_page.c:462 462 bucket = &(*bucket)->hnext; (kgdb) list 457 458 bucket = &vm_page_buckets[vm_page_hash(m->object, m->pindex)]; 459 while (*bucket != m) { 460 if (*bucket == NULL) 461 panic("vm_page_remove(): page not found in hash"); 462 bucket = &(*bucket)->hnext; 463 } 464 *bucket = m->hnext; 465 m->hnext = NULL; 466 vm_page_bucket_generation++; (kgdb) up #7 0xc02600ac in vm_page_free_toq (m=0xc1331138) at /usr/src/sys/vm/vm_page.c:1104 1104 vm_page_remove(m); (kgdb) list 1099 * callback routine until after we've put the page on the 1100 * appropriate free queue. 1101 */ 1102 1103 vm_page_unqueue_nowakeup(m); 1104 vm_page_remove(m); 1105 1106 /* 1107 * If fictitious remove object association and 1108 * return, otherwise delay object association removal. (kgdb) #8 0xc025de22 in vm_object_terminate (object=0xe8f04da8) at /usr/src/sys/vm/vm_page.h:514 514 vm_page_free_toq(m); (kgdb) list 509 static __inline void 510 vm_page_free(m) 511 vm_page_t m; 512 { 513 vm_page_flag_clear(m, PG_ZERO); 514 vm_page_free_toq(m); 515 } 516 517 /* 518 * vm_page_free_zero: (kgdb) up #9 0xc025dce0 in vm_object_deallocate (object=0xe8f04da8) at /usr/src/sys/vm/vm_object.c:399 399 vm_object_terminate(object); (kgdb) list 394 * Don't double-terminate, we could be in a termination 395 * recursion due to the terminate having to sync data 396 * to disk. 397 */ 398 if ((object->flags & OBJ_DEAD) == 0) 399 vm_object_terminate(object); 400 object = temp; 401 } 402 } 403 (kgdb) up #10 0xc025af21 in vm_map_entry_delete (map=0xe8b3e040, entry=0xe8906c60) at /usr/src/sys/vm/vm_map.c:2054 2054 vm_object_deallocate(entry->object.vm_object); (kgdb) list 2049 { 2050 vm_map_entry_unlink(map, entry); 2051 map->size -= entry->end - entry->start; 2052 2053 if ((entry->eflags & MAP_ENTRY_IS_SUB_MAP) == 0) { 2054 vm_object_deallocate(entry->object.vm_object); 2055 } 2056 2057 vm_map_entry_dispose(map, entry); 2058 } (kgdb) up #11 0xc025b0d2 in vm_map_delete (map=0xe8b3e040, start=0, end=3217031168) at /usr/src/sys/vm/vm_map.c:2174 2174 vm_map_entry_delete(map, entry); (kgdb) list 2169 * Delete the entry (which may delete the object) only after 2170 * removing all pmap entries pointing to its pages. 2171 * (Otherwise, its page frames may be reallocated, and any 2172 * modify bits will be set in the wrong object!) 2173 */ 2174 vm_map_entry_delete(map, entry); 2175 entry = next; 2176 } 2177 return (KERN_SUCCESS); 2178 } (kgdb) up #12 0xc025b161 in vm_map_remove (map=0xe8b3e040, start=0, end=3217031168) at /usr/src/sys/vm/vm_map.c:2199 2199 result = vm_map_delete(map, start, end); (kgdb) list 2194 if (map == kmem_map || map == mb_map) 2195 s = splvm(); 2196 2197 vm_map_lock(map); 2198 VM_MAP_RANGE_CHECK(map, start, end); 2199 result = vm_map_delete(map, start, end); 2200 vm_map_unlock(map); 2201 2202 if (map == kmem_map || map == mb_map) 2203 splx(s); (kgdb) up #13 0xc0159c3d in exit1 (p=0xe8a89ea0, rv=1054720) at /usr/src/sys/kern/kern_exit.c:226 226 (void) vm_map_remove(&vm->vm_map, VM_MIN_ADDRESS, (kgdb) list 221 if (--vm->vm_refcnt == 0) { 222 if (vm->vm_shm) 223 shmexit(p); 224 pmap_remove_pages(vmspace_pmap(vm), VM_MIN_ADDRESS, 225 VM_MAXUSER_ADDRESS); 226 (void) vm_map_remove(&vm->vm_map, VM_MIN_ADDRESS, 227 VM_MAXUSER_ADDRESS); 228 } 229 230 if (SESS_LEADER(p)) { (kgdb) up #14 0xc0159a09 in sys_exit (p=0xe8a89ea0, uap=0xe8ba6f80) at /usr/src/sys/kern/kern_exit.c:104 104 exit1(p, W_EXITCODE(uap->rval, 0)); (kgdb) list 99 struct sys_exit_args /* { 100 int rval; 101 } */ *uap; 102 { 103 104 exit1(p, W_EXITCODE(uap->rval, 0)); 105 /* NOTREACHED */ 106 } 107 108 /* (kgdb) ns4# cat /var/run/dmesg.boot Copyright (c) 1992-2003 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 4.8-STABLE #0: Tue Aug 12 11:15:45 EDT 2003 mdtancsa@ns4.recycle.net:/usr/obj/usr/src/sys/ns4 Timecounter "i8254" frequency 1193182 Hz Timecounter "TSC" frequency 2398856892 Hz CPU: Intel(R) Pentium(R) 4 CPU 2.40GHz (2398.86-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0xf27 Stepping = 7 Features=0xbfebfbff real memory = 1072627712 (1047488K bytes) config> q avail memory = 1040453632 (1016068K bytes) Preloaded elf kernel "kernel" at 0xc038e000. Preloaded userconfig_script "/boot/kernel.conf" at 0xc038e09c. Pentium Pro MTRR support enabled md0: Malloc disk Using $PIR table, 7 entries at 0xc00fded0 npx0: on motherboard npx0: INT 16 interface pcib0: on motherboard pci0: on pcib0 agp0: mem 0xd8d00000-0xd8d7ffff,0xd0000000-0xd7ffffff irq 12 at device 2.0 on pci0 agp0: detected 892k stolen memory agp0: aperture size is 128M pcib1: at device 30.0 on pci0 pci1: on pcib1 fxp0: port 0xc000-0xc01f mem 0xd8800000-0xd88fffff,0xd8c00000-0xd8c00fff irq 12 at device 0.0 on pci1 fxp0: Ethernet address 00:a0:c9:e7:a6:e6 inphy0: on miibus0 inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto fxp1: port 0xc400-0xc43f mem 0xd8a00000-0xd8afffff,0xd8b00000-0xd8b00fff irq 10 at device 1.0 on pci1 fxp1: Ethernet address 00:02:b3:07:fd:8d inphy1: on miibus1 inphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto twe0: <3ware Storage Controller> port 0xc800-0xc80f irq 15 at device 5.0 on pci1 twe0: 2 ports, Firmware FE6X 1.02.00.029, BIOS BEXX 1.07.00.009 fxp2: port 0xcc00-0xcc3f mem 0xd8b01000-0xd8b01fff irq 11 at device 8.0 on pci1 fxp2: Ethernet address 00:01:80:38:46:36 inphy2: on miibus2 inphy2: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto isab0: at device 31.0 on pci0 isa0: on isab0 pci0: (vendor=0x8086, dev=0x24c3) at 31.3 irq 10 orm0: