From owner-freebsd-stable@FreeBSD.ORG Fri Jan 5 11:31:27 2007 Return-Path: X-Original-To: stable@FreeBSD.org Delivered-To: freebsd-stable@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 11C5B16A403 for ; Fri, 5 Jan 2007 11:31:27 +0000 (UTC) (envelope-from ceri@submonkey.net) Received: from shrike.submonkey.net (cpc2-cdif2-0-0-cust107.cdif.cable.ntl.com [81.104.168.108]) by mx1.freebsd.org (Postfix) with ESMTP id 729D413C44B for ; Fri, 5 Jan 2007 11:31:26 +0000 (UTC) (envelope-from ceri@submonkey.net) Received: from ceri by shrike.submonkey.net with local (Exim 4.64 (FreeBSD)) (envelope-from ) id 1H2n75-0004wz-4e; Fri, 05 Jan 2007 11:19:55 +0000 Date: Fri, 5 Jan 2007 11:19:55 +0000 From: Ceri Davies To: stable@FreeBSD.org Message-ID: <20070105111954.GA51511@submonkey.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="xHFwDpU9dbj6ez1V" Content-Disposition: inline X-PGP: finger ceri@FreeBSD.org User-Agent: Mutt/1.5.13 (2006-08-11) Sender: Ceri Davies Cc: rwatson@FreeBSD.org Subject: (audit?) Panic in 6.2-PRERELEASE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Jan 2007 11:31:27 -0000 --xHFwDpU9dbj6ez1V Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable For the last two mornings, my system decided to panic() in the exact same place. I have dumps from both but they almost exactly the same. Any pointers on where to go next are welcomed. Here's the first, and I don't see much in there: {root@shrike}-{~} # uname -a FreeBSD shrike.private.submonkey.net 6.2-PRERELEASE FreeBSD 6.2-PRERELEASE = #69: Fri Dec 29 00:25:52 GMT 2006 root@shrike.private.submonkey.net:/us= r/obj/usr/src/sys/SHRIKE i386 {root@shrike}-{~} # kgdb /usr/obj/usr/src/sys/SHRIKE/kernel.debug /var/cras= h/vmcore.29 kgdb: kvm_nlist(_stopped_cpus):=20 kgdb: kvm_nlist(_stoppcbs):=20 [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so:= Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain condition= s. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode fault virtual address =3D 0x53892047 fault code =3D supervisor write, page not present instruction pointer =3D 0x20:0xc05cda7c stack pointer =3D 0x28:0xd610dc48 frame pointer =3D 0x28:0xd610dc60 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, def32 1, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 65381 (imapd) trap number =3D 12 panic: page fault Uptime: 5d19h44m40s Dumping 503 MB (2 chunks) chunk 0: 1MB (160 pages) ... ok chunk 1: 503MB (128752 pages) 487 471 455 439 423 407 391 375 359 343 327= 311 295 279 263 247 231 215 199 183 167 151 135 119 103 87 71 55 39 23 7 #0 doadump () at pcpu.h:165 165 pcpu.h: No such file or directory. in pcpu.h (kgdb) where #0 doadump () at pcpu.h:165 #1 0xc04e85aa in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:4= 09 #2 0xc04e8840 in panic (fmt=3D0xc066f61a "%s") at /usr/src/sys/kern/kern_s= hutdown.c:565 #3 0xc0653ed4 in trap_fatal (frame=3D0xd610dc08, eva=3D1401495623) at /usr/src/sys/i386/i386/trap.c:837 #4 0xc0653c3b in trap_pfault (frame=3D0xd610dc08, usermode=3D0, eva=3D1401= 495623) at /usr/src/sys/i386/i386/trap.c:745 #5 0xc0653899 in trap (frame=3D {tf_fs =3D 8, tf_es =3D 40, tf_ds =3D 40, tf_edi =3D -1024544384, tf_= esi =3D -1024544384, tf_ebp =3D -703538080, tf_isp =3D -703538124, tf_ebx = =3D 0, tf_edx =3D -703538092, tf_ecx =3D 4, tf_eax =3D 0, tf_trapno =3D 12,= tf_err =3D 2, tf_eip =3D -1067656580, tf_cs =3D 32, tf_eflags =3D 66050, t= f_esp =3D -1068742797, tf_ss =3D -1022955520}) at /usr/src/sys/i386/i386/tr= ap.c:435 #6 0xc064287a in calltrap () at /usr/src/sys/i386/i386/exception.s:139 #7 0xc05cda7c in audit_arg_auditon () at /usr/src/sys/security/audit/audit= _arg.c:586 #8 0xc04c470d in fstat (td=3D0xc2eeb180, uap=3D0xd610dc74) at /usr/src/sys= /kern/kern_descrip.c:1075 #9 0xc0654203 in syscall (frame=3D {tf_fs =3D 59, tf_es =3D 59, tf_ds =3D 59, tf_edi =3D -1077949408, tf= _esi =3D 135666752, tf_ebp =3D -1077949448, tf_isp =3D -703537820, tf_ebx = =3D 135432156, tf_edx =3D -1077949112, tf_ecx =3D 135826416, tf_eax =3D 189= , tf_trapno =3D 0, tf_err =3D 2, tf_eip =3D 675755895, tf_cs =3D 51, tf_efl= ags =3D 662, tf_esp =3D -1077949732, tf_ss =3D 59}) at /usr/src/sys/i386/i3= 86/trap.c:983 #10 0xc06428cf in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s= :200 #11 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) up 8 #8 0xc04c470d in fstat (td=3D0xc2eeb180, uap=3D0xd610dc74) at /usr/src/sys= /kern/kern_descrip.c:1075 1075 error =3D kern_fstat(td, uap->fd, &ub); (kgdb) p ub $1 =3D {st_dev =3D 89, st_ino =3D 1907905, st_mode =3D 33152, st_nlink =3D = 1, st_uid =3D 60, st_gid =3D 60,=20 st_rdev =3D 7624272, st_atimespec =3D {tv_sec =3D 1167893059, tv_nsec =3D= -703537996}, st_mtimespec =3D { tv_sec =3D -703537916, tv_nsec =3D -1024544384}, st_ctimespec =3D {tv_s= ec =3D 43018, tv_nsec =3D 43018},=20 st_size =3D -3021672509244264064, st_blocks =3D -1067658896, st_blksize = =3D 43018, st_flags =3D 4,=20 st_gen =3D 3, st_lspare =3D 0, st_birthtimespec =3D {tv_sec =3D -1, tv_ns= ec =3D 4}} (kgdb) p td $2 =3D (struct thread *) 0xc2eeb180 (kgdb) p uap->fd $3 =3D 89 (kgdb) The second one seems more promising, in that the fd seems to be rubbish. {root@shrike}-{~} # kgdb /usr/obj/usr/src/sys/SHRIKE/kernel.debug /var/cras= h/vmcore.30 kgdb: kvm_nlist(_stopped_cpus):=20 kgdb: kvm_nlist(_stoppcbs):=20 [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so:= Undefined symbol "ps_pglobal_lookup"] GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain condition= s. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd". Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode fault virtual address =3D 0x53892047 fault code =3D supervisor write, page not present instruction pointer =3D 0x20:0xc05cda7c stack pointer =3D 0x28:0xd617ec48 frame pointer =3D 0x28:0xd617ec60 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, def32 1, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 9943 (imapd) trap number =3D 12 panic: page fault Uptime: 22h39m3s Dumping 503 MB (2 chunks) chunk 0: 1MB (160 pages) ... ok chunk 1: 503MB (128752 pages) 487 471 455 439 423 407 391 375 359 343 327= 311 295 279 263 247 231 215 199 183 167 151 135 119 103 87 71 55 39 23 7 #0 doadump () at pcpu.h:165 165 pcpu.h: No such file or directory. in pcpu.h (kgdb) where #0 doadump () at pcpu.h:165 #1 0xc04e85aa in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:4= 09 #2 0xc04e8840 in panic (fmt=3D0xc066f61a "%s") at /usr/src/sys/kern/kern_s= hutdown.c:565 #3 0xc0653ed4 in trap_fatal (frame=3D0xd617ec08, eva=3D1401495623) at /usr/src/sys/i386/i386/trap.c:837 #4 0xc0653c3b in trap_pfault (frame=3D0xd617ec08, usermode=3D0, eva=3D1401= 495623) at /usr/src/sys/i386/i386/trap.c:745 #5 0xc0653899 in trap (frame=3D {tf_fs =3D 8, tf_es =3D 40, tf_ds =3D 40, tf_edi =3D -1022323968, tf_= esi =3D -1022323968, tf_ebp =3D -703075232, tf_isp =3D -703075276, tf_ebx = =3D 0, tf_edx =3D -703075244, tf_ecx =3D 4, tf_eax =3D 0, tf_trapno =3D 12,= tf_err =3D 2, tf_eip =3D -1067656580, tf_cs =3D 32, tf_eflags =3D 66050, t= f_esp =3D -1068742797, tf_ss =3D -1022327760}) at /usr/src/sys/i386/i386/tr= ap.c:435 #6 0xc064287a in calltrap () at /usr/src/sys/i386/i386/exception.s:139 #7 0xc05cda7c in audit_arg_auditon () at /usr/src/sys/security/audit/audit= _arg.c:586 #8 0xc04c470d in fstat (td=3D0xc3109300, uap=3D0xd617ec74) at /usr/src/sys= /kern/kern_descrip.c:1075 #9 0xc0654203 in syscall (frame=3D {tf_fs =3D 59, tf_es =3D 59, tf_ds =3D 59, tf_edi =3D 135488384, tf_e= si =3D -1077948560, tf_ebp =3D -1077948888, tf_isp =3D -703074972, tf_ebx = =3D 135432156, tf_edx =3D -1077948712, tf_ecx =3D 25, tf_eax =3D 189, tf_tr= apno =3D 0, tf_err =3D 2, tf_eip =3D 675755895, tf_cs =3D 51, tf_eflags =3D= 662, tf_esp =3D -1077949124, tf_ss =3D 59}) at /usr/src/sys/i386/i386/trap= =2Ec:983 #10 0xc06428cf in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s= :200 #11 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (kgdb) up 8 #8 0xc04c470d in fstat (td=3D0xc3109300, uap=3D0xd617ec74) at /usr/src/sys= /kern/kern_descrip.c:1075 1075 error =3D kern_fstat(td, uap->fd, &ub); (kgdb) p uap->fd $1 =3D -1023449232 (kgdb)=20 Ceri --=20 That must be wonderful! I don't understand it at all. -- Moliere --xHFwDpU9dbj6ez1V Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFFnjRaocfcwTS3JF8RAks0AKCtTVVI95FO06d7M5OuK1pNMn2XLQCgjNMO bHB45pHbhSA0CRUBFYXH3vg= =TaBm -----END PGP SIGNATURE----- --xHFwDpU9dbj6ez1V--