Date: Tue, 23 Jan 2007 08:40:25 -0800 From: Micah <micahjon@ywave.com> To: FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Page fault in vfs_hash_get on 6.2 Message-ID: <45B63A79.3030805@ywave.com>
next in thread | raw e-mail | index | archive | help
Back in October 2006, I was having the same problem in 6.1 (original thread at: http://lists.freebsd.org/pipermail/freebsd-hackers/2006-October/018393.html) The update to 6.2 decreased the frequency of the panics, but I still get the panic periodically - since October I've had 6-8 panics judging by my vmcore count. trisha# uname -a FreeBSD trisha.lan 6.2-RELEASE FreeBSD 6.2-RELEASE #0: Sat Jan 20 16:18:57 PST 2007 root@trisha.lan:/usr/obj/usr/src/sys/TRISHA i386 Here are the last two dumps with some prints of the interesting vars (I have more dumps, but these are the only two for 6.2-release). Let me know if I can provide anything else. trisha# kgdb /usr/obj/usr/src/sys/TRISHA/kernel.debug vmcore.13 kgdb: kvm_nlist(_stopped_cpus): kgdb: kvm_nlist(_stoppcbs): [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode fault virtual address = 0xd92c1358 fault code = supervisor read, page not present instruction pointer = 0x20:0xc0595304 stack pointer = 0x28:0xf039c89c frame pointer = 0x28:0xf039c8bc code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 15173 (kphotoalbum) trap number = 12 panic: page fault Uptime: 2d4h53m29s Dumping 1534 MB (2 chunks) chunk 0: 1MB (159 pages) ... ok chunk 1: 1534MB (392672 pages) 1518 1502 1486 1470 1454 1438 1422 1406 1390 1374 1358 1342 1326 1310 1294 1278 1262 1246 1230 1214 1198 1182 1166 1150 1134 1118 1102 1086 1070 1054 1038 1022 1006 990 974 958 942 926 910 894 878 862 846 830 814 798 782 766 750 734 718 702 686 670 654 638 622 (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) 606 590 574 558 542 526 510 494 478 462 446 430 414 398 382 366 350 334 318 302 286 270 254 238 222 206 190 174 (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) 158 (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) 142 (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) 126 110 (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) 94 (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) 78 (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) 62 (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) 46 (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) 30 (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) 14 (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) #0 doadump () at pcpu.h:165 165 pcpu.h: No such file or directory. in pcpu.h (kgdb) bt #0 doadump () at pcpu.h:165 #1 0xc0535f54 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 #2 0xc0536286 in panic (fmt=0xc071898d "%s") at /usr/src/sys/kern/kern_shutdown.c:565 #3 0xc06f34bc in trap_fatal (frame=0xf039c85c, eva=0) at /usr/src/sys/i386/i386/trap.c:837 #4 0xc06f31c2 in trap_pfault (frame=0xf039c85c, usermode=0, eva=3643544408) at /usr/src/sys/i386/i386/trap.c:745 #5 0xc06f2d8d in trap (frame= {tf_fs = 8, tf_es = 40, tf_ds = 40, tf_edi = 4, tf_esi = 4, tf_ebp = -264648516, tf_isp = -264648568, tf_ebx = -651422928, tf_edx = -980348928, tf_ecx = -978780160, tf_eax = 8055459, tf_trapno = 12, tf_err = 0, tf_eip = -1067887868, tf_cs = 32, tf_eflags = 2163334, tf_esp = -978780160, tf_ss = 8055459}) at /usr/src/sys/i386/i386/trap.c:435 #6 0xc06df32a in calltrap () at /usr/src/sys/i386/i386/exception.s:139 #7 0xc0595304 in vfs_hash_get (mp=0xc5a90000, hash=8055459, flags=2, td=0xc6552600, vpp=0xf039c99c, fn=0, arg=0x0) at /usr/src/sys/kern/vfs_hash.c:73 #8 0xc067f379 in ffs_vget (mp=0xc5a90000, ino=8055459, flags=2, vpp=0xf039c99c) at pcpu.h:162 #9 0xc06879f3 in ufs_lookup (ap=0xf039ca40) at /usr/src/sys/ufs/ufs/ufs_lookup.c:572 #10 0xc0707993 in VOP_CACHEDLOOKUP_APV (vop=0x7aeaa3, a=0xc5911000) at vnode_if.c:150 #11 0xc05913ea in vfs_cache_lookup (ap=0x7aeaa3) at vnode_if.h:82 #12 0xc0707908 in VOP_LOOKUP_APV (vop=0xc076c500, a=0xf039caec) at vnode_if.c:99 #13 0xc0596a3b in lookup (ndp=0xf039cb94) at vnode_if.h:56 #14 0xc05961d8 in namei (ndp=0xf039cb94) at /usr/src/sys/kern/vfs_lookup.c:211 #15 0xc05a86cf in kern_lstat (td=0xc6552600, path=0xc5911000 "", pathseg=3314618368, sbp=0x7aeaa3) at /usr/src/sys/kern/vfs_syscalls.c:2143 #16 0xc05a864f in lstat (td=0x7aeaa3, uap=0xf039cd04) at /usr/src/sys/kern/vfs_syscalls.c:2126 #17 0xc06f3892 in syscall (frame= {tf_fs = 59, tf_es = 59, tf_ds = 59, tf_edi = 139441408, tf_esi = -1077945056, tf_ebp = -1077945816, tf_isp = -264647324, tf_ebx = 1230067672, tf_edx = 69, tf_ecx = 142276000, tf_eax = 190, tf_trapno = 0, tf_err = 2, tf_eip = 1233487479, tf_cs = 51, tf_eflags = 2097794, tf_esp = -1077945892, tf_ss = 59}) at /usr/src/sys/i386/i386/trap.c:983 #18 0xc06df37f in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:200 #19 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) frame 7 #7 0xc0595304 in vfs_hash_get (mp=0xc5a90000, hash=8055459, flags=2, td=0xc6552600, vpp=0xf039c99c, fn=0, arg=0x0) at /usr/src/sys/kern/vfs_hash.c:73 73 if (vp->v_hash != hash) (kgdb) p mp $1 = (struct mount *) 0xc5a90000 (kgdb) p *mp $2 = {mnt_list = {tqe_next = 0x0, tqe_prev = 0xc5a90298}, mnt_op = 0xc076bc60, mnt_vfc = 0xc076bca0, mnt_vnodecovered = 0xc5ad0aa0, mnt_syncer = 0xc5ad4cc0, mnt_nvnodelist = {tqh_first = 0xc5ad4dd0, tqh_last = 0xcaabc014}, mnt_lock = {lk_interlock = 0xc077f11c, lk_flags = 0, lk_sharecount = 0, lk_waitcount = 0, lk_exclusivecount = 0, lk_prio = 80, lk_wmesg = 0xc072d526 "vfslock", lk_timo = 0, lk_lockholder = 0xffffffff, lk_newlock = 0x0}, mnt_mtx = {mtx_object = { lo_class = 0xc075a8a4, lo_name = 0xc072d515 "struct mount mtx", lo_type = 0xc072d515 "struct mount mtx", lo_flags = 196608, lo_list = { tqe_next = 0x0, tqe_prev = 0x0}, lo_witness = 0x0}, mtx_lock = 4, mtx_recurse = 0}, mnt_writeopcount = 0, mnt_flag = 2101248, mnt_opt = 0xc5a498a0, mnt_optnew = 0x0, mnt_kern_flag = 536870912, mnt_maxsymlinklen = 120, mnt_stat = {f_version = 537068824, f_type = 5, f_flags = 2101248, f_bsize = 2048, f_iosize = 16384, f_blocks = 47731967, f_bfree = 11478076, f_bavail = 7659519, f_files = 12341246, f_ffree = 11381805, f_syncwrites = 0, f_asyncwrites = 0, f_syncreads = 0, f_asyncreads = 0, f_spare = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, f_namemax = 255, f_owner = 0, f_fsid = {val = {1161696182, 824004859}}, f_charspare = '\0' <repeats 79 times>, f_fstypename = "ufs", '\0' <repeats 12 times>, f_mntfromname = "/dev/ad4s1h", '\0' <repeats 76 times>, f_mntonname = "/home", '\0' <repeats 82 times>}, mnt_cred = 0xc5a24c00, mnt_data = 0xc5a6b900, mnt_time = 0, mnt_iosize_max = 131072, mnt_export = 0x0, mnt_mntlabel = 0x0, mnt_fslabel = 0x0, mnt_nvnodelistsize = 81992, mnt_hashseed = 1412285663, mnt_markercnt = 0, mnt_holdcnt = 0, mnt_holdcntwaiters = 0, mnt_secondary_writes = 0, mnt_secondary_accwrites = 2436404, mnt_ref = 81992, mnt_gen = 1} (kgdb) p vp $3 = (struct vnode *) 0xd92c1330 (kgdb) p *vp Cannot access memory at address 0xd92c1330 (kgdb) trisha# kgdb /usr/obj/usr/src/sys/TRISHA/kernel.debug vmcore.14 kgdb: kvm_nlist(_stopped_cpus): kgdb: kvm_nlist(_stoppcbs): [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode fault virtual address = 0xce7b0df8 fault code = supervisor read, page not present instruction pointer = 0x20:0xc0595304 stack pointer = 0x28:0xf03ab89c frame pointer = 0x28:0xf03ab8bc code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 2897 (find) trap number = 12 panic: page fault Uptime: 5h17m47s Dumping 1534 MB (2 chunks) chunk 0: 1MB (159 pages) ... ok chunk 1: 1534MB (392672 pages) 1518 1502 1486 1470 1454 1438 1422 1406 1390 1374 1358 1342 1326 1310 1294 1278 1262 1246 1230 1214 1198 1182 1166 1150 1134 1118 1102 1086 1070 1054 1038 1022 1006 990 974 958 942 926 910 894 878 862 846 830 814 798 782 766 750 734 718 702 686 670 654 638 622 606 590 574 558 542 526 510 494 478 462 446 430 414 398 382 366 350 334 318 302 286 270 254 238 222 206 190 174 158 142 126 110 94 78 62 46 30 14 #0 doadump () at pcpu.h:165 165 pcpu.h: No such file or directory. in pcpu.h (kgdb) bt #0 doadump () at pcpu.h:165 #1 0xc0535f54 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409 #2 0xc0536286 in panic (fmt=0xc071898d "%s") at /usr/src/sys/kern/kern_shutdown.c:565 #3 0xc06f34bc in trap_fatal (frame=0xf03ab85c, eva=0) at /usr/src/sys/i386/i386/trap.c:837 #4 0xc06f31c2 in trap_pfault (frame=0xf03ab85c, usermode=0, eva=3464171000) at /usr/src/sys/i386/i386/trap.c:745 #5 0xc06f2d8d in trap (frame= {tf_fs = 8, tf_es = 40, tf_ds = 40, tf_edi = 4, tf_esi = 4, tf_ebp = -264587076, tf_isp = -264587128, tf_ebx = -830796336, tf_edx = -980348928, tf_ecx = -978780160, tf_eax = 4027855, tf_trapno = 12, tf_err = 0, tf_eip = -1067887868, tf_cs = 32, tf_eflags = 66178, tf_esp = -978780160, tf_ss = 4027855}) at /usr/src/sys/i386/i386/trap.c:435 #6 0xc06df32a in calltrap () at /usr/src/sys/i386/i386/exception.s:139 #7 0xc0595304 in vfs_hash_get (mp=0xc5a90000, hash=4027855, flags=2, td=0xc64a3d80, vpp=0xf03ab99c, fn=0, arg=0x0) at /usr/src/sys/kern/vfs_hash.c:73 #8 0xc067f379 in ffs_vget (mp=0xc5a90000, ino=4027855, flags=2, vpp=0xf03ab99c) at pcpu.h:162 #9 0xc06879f3 in ufs_lookup (ap=0xf03aba40) at /usr/src/sys/ufs/ufs/ufs_lookup.c:572 #10 0xc0707993 in VOP_CACHEDLOOKUP_APV (vop=0x3d75cf, a=0xc5911000) at vnode_if.c:150 #11 0xc05913ea in vfs_cache_lookup (ap=0x3d75cf) at vnode_if.h:82 #12 0xc0707908 in VOP_LOOKUP_APV (vop=0xc076c500, a=0xf03abaec) at vnode_if.c:99 #13 0xc0596a3b in lookup (ndp=0xf03abb94) at vnode_if.h:56 #14 0xc05961d8 in namei (ndp=0xf03abb94) at /usr/src/sys/kern/vfs_lookup.c:211 #15 0xc05a86cf in kern_lstat (td=0xc64a3d80, path=0xc5911000 "@ÔLÆ0ÓLÆ ÒLÆ\020ÑLÆ°\233LÆ \232LÆ\220\231LÆÀ\034MÆ°\033MÆ \032MÆÀ¬MÆ", pathseg=3314618368, sbp=0x3d75cf) at /usr/src/sys/kern/vfs_syscalls.c:2143 #16 0xc05a864f in lstat (td=0x3d75cf, uap=0xf03abd04) at /usr/src/sys/kern/vfs_syscalls.c:2126 #17 0xc06f3892 in syscall (frame= {tf_fs = 59, tf_es = 59, tf_ds = 59, tf_edi = 134687816, tf_esi = 134687744, tf_ebp = -1077941128, tf_isp = -264585884, tf_ebx = 1209422816, tf_edx = 134687744, tf_ecx = 134565888, tf_eax = 190, tf_trapno = 12, tf_err = 2, tf_eip = 1209300599, tf_cs = 51, tf_eflags = 582, tf_esp = -1077941284, tf_ss = 59}) at /usr/src/sys/i386/i386/trap.c:983 ---Type <return> to continue, or q <return> to quit--- #18 0xc06df37f in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:200 #19 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) frame 7 #7 0xc0595304 in vfs_hash_get (mp=0xc5a90000, hash=4027855, flags=2, td=0xc64a3d80, vpp=0xf03ab99c, fn=0, arg=0x0) at /usr/src/sys/kern/vfs_hash.c:73 73 if (vp->v_hash != hash) (kgdb) p mp $1 = (struct mount *) 0xc5a90000 (kgdb) p *mp $2 = {mnt_list = {tqe_next = 0x0, tqe_prev = 0xc5a90298}, mnt_op = 0xc076bc60, mnt_vfc = 0xc076bca0, mnt_vnodecovered = 0xc5ad0aa0, mnt_syncer = 0xc5ad4cc0, mnt_nvnodelist = {tqh_first = 0xc5ad4dd0, tqh_last = 0xc7145344}, mnt_lock = {lk_interlock = 0xc077f11c, lk_flags = 0, lk_sharecount = 0, lk_waitcount = 0, lk_exclusivecount = 0, lk_prio = 80, lk_wmesg = 0xc072d526 "vfslock", lk_timo = 0, lk_lockholder = 0xffffffff, lk_newlock = 0x0}, mnt_mtx = {mtx_object = { lo_class = 0xc075a8a4, lo_name = 0xc072d515 "struct mount mtx", lo_type = 0xc072d515 "struct mount mtx", lo_flags = 196608, lo_list = { tqe_next = 0x0, tqe_prev = 0x0}, lo_witness = 0x0}, mtx_lock = 4, mtx_recurse = 0}, mnt_writeopcount = 0, mnt_flag = 2101248, mnt_opt = 0xc5a496e0, mnt_optnew = 0x0, mnt_kern_flag = 536870912, mnt_maxsymlinklen = 120, mnt_stat = {f_version = 537068824, f_type = 5, f_flags = 2101248, f_bsize = 2048, f_iosize = 16384, f_blocks = 47731967, f_bfree = 9820243, f_bavail = 6001686, f_files = 12341246, f_ffree = 11382010, f_syncwrites = 0, f_asyncwrites = 0, f_syncreads = 0, f_asyncreads = 0, f_spare = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, f_namemax = 255, f_owner = 0, f_fsid = {val = {1161696182, 824004859}}, f_charspare = '\0' <repeats 79 times>, f_fstypename = "ufs", '\0' <repeats 12 times>, f_mntfromname = "/dev/ad4s1h", '\0' <repeats 76 times>, f_mntonname = "/home", '\0' <repeats 82 times>}, mnt_cred = 0xc5aaed00, mnt_data = 0xc5a99e00, mnt_time = 0, mnt_iosize_max = 131072, mnt_export = 0x0, mnt_mntlabel = 0x0, mnt_fslabel = 0x0, mnt_nvnodelistsize = 10746, mnt_hashseed = 1412285663, mnt_markercnt = 0, mnt_holdcnt = 0, mnt_holdcntwaiters = 0, mnt_secondary_writes = 0, mnt_secondary_accwrites = 334543, mnt_ref = 10746, mnt_gen = 1} (kgdb) p vp $3 = (struct vnode *) 0xce7b0dd0 (kgdb) p *vp Cannot access memory at address 0xce7b0dd0 (kgdb) Thanks, Micah
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?45B63A79.3030805>