From owner-freebsd-stable Sun Sep 1 10:11:48 2002 Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AADA537B400 for ; Sun, 1 Sep 2002 10:11:40 -0700 (PDT) Received: from earth.hub.org (earth.hub.org [64.49.215.11]) by mx1.FreeBSD.org (Postfix) with ESMTP id 41A3743E3B for ; Sun, 1 Sep 2002 10:11:40 -0700 (PDT) (envelope-from scrappy@hub.org) Received: from earth.hub.org (earth.hub.org [64.49.215.11]) by earth.hub.org (Postfix) with ESMTP id 65B332CC803 for ; Sun, 1 Sep 2002 14:11:37 -0300 (ADT) Date: Sun, 1 Sep 2002 14:11:37 -0300 (ADT) From: "Marc G. Fournier" To: freebsd-stable@freebsd.org Subject: Crash with KVM monitoring in place ... Message-ID: <20020901135655.L27764-100000@hub.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-stable@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Morning all ... Okay, according to my logs, here is what I have: Aug 31 22:13:47 venus scrappy[35952]: Starting KVM Monitoring Aug 31 22:13:47 venus scrappy[35962]: kernel_vm_end: 0xf6400000 Aug 31 23:00:03 venus scrappy[41669]: kernel_vm_end: 0xf6800000 Sep 1 00:00:25 venus scrappy[50135]: kernel_vm_end: 0xf7000000 Note that I modified my script so that it only reports to syslog when the value changes, instead of every minute ... On the other server, she's still running with: Sep 1 02:05:30 jupiter root[93918]: kernel_vm_end: 0xfa000000 Sep 1 02:07:32 jupiter root[95929]: kernel_vm_end: 0xfa400000 Sep 1 02:09:34 jupiter root[98699]: kernel_vm_end: 0xfa800000 Sep 1 02:10:34 jupiter root[1076]: kernel_vm_end: 0xfac00000 Sep 1 04:30:42 jupiter root[33295]: kernel_vm_end: 0xfb000000 Both server(s) are running kernels updated to sources of last night. Next? /var/log/messages shows several reboots: Sep 1 00:29:21 venus sm-mta[54119]: g811TILb000223: SYSERR(root): makeconnection_ds: unsafe domain socket: No such file or directory Sep 1 01:05:43 venus /kernel: Sep 1 01:05:43 venus /kernel: Sep 1 01:05:43 venus /kernel: Fatal trap 12: page fault while in kernel mode Sep 1 01:05:43 venus /kernel: mp_lock = 00000002; cpuid = 0; lapic.id = 00000000 Sep 1 01:05:43 venus /kernel: fault virtual address = 0xef27be58 Sep 1 01:05:43 venus /kernel: fault code = supervisor read, page not present Sep 1 01:05:43 venus /kernel: instruction pointer = 0x8:0xc01749ff Sep 1 01:05:43 venus /kernel: stack pointer = 0x10:0xf5850d74 Sep 1 01:05:43 venus /kernel: frame pointer = 0x10:0xf5850d8c Sep 1 01:05:43 venus /kernel: code segment = base 0x0, limit 0xfffff, type 0x1b Sep 1 01:05:43 venus /kernel: = DPL 0, pres 1, def32 1, gran 1 Sep 1 01:05:43 venus /kernel: processor eflags = interrupt enabled, resume, IOPL = 0 Sep 1 01:05:43 venus /kernel: current process = 47365 (pickup) Sep 1 01:05:43 venus /kernel: interrupt mask = none <- SMP: XXX Sep 1 01:05:43 venus /kernel: trap number = 12 Sep 1 01:05:43 venus /kernel: panic: page fault Sep 1 01:05:43 venus /kernel: mp_lock = 00000002; cpuid = 0; lapic.id = 00000000 Sep 1 01:05:43 venus /kernel: boot() called on cpu#0 Sep 1 01:05:43 venus /kernel: Sep 1 01:05:43 venus /kernel: syncing disks... venus# nm -n /kernel | grep c01749ff venus# nm -n /kernel | grep c01749f venus# nm -n /kernel | grep c01749 c01749d0 t vn_poll Sep 1 01:05:43 venus /kernel: Sep 1 01:05:43 venus /kernel: Fatal trap 12: page fault while in kernel mode Sep 1 01:05:43 venus /kernel: mp_lock = 00000003; cpuid = 0; lapic.id = 00000000 Sep 1 01:05:43 venus /kernel: fault virtual address = 0xef27f514 Sep 1 01:05:43 venus /kernel: fault code = supervisor read, page not present Sep 1 01:05:43 venus /kernel: instruction pointer = 0x8:0xc016e6fb Sep 1 01:05:43 venus /kernel: stack pointer = 0x10:0xf5850c0c Sep 1 01:05:43 venus /kernel: frame pointer = 0x10:0xf5850c28 Sep 1 01:05:43 venus /kernel: code segment = base 0x0, limit 0xfffff, type 0x1b Sep 1 01:05:43 venus /kernel: = DPL 0, pres 1, def32 1, gran 1 Sep 1 01:05:43 venus /kernel: processor eflags = interrupt enabled, resume, IOPL = 0 Sep 1 01:05:43 venus /kernel: current process = 47365 (pickup) Sep 1 01:05:43 venus /kernel: interrupt mask = none <- SMP: XXX Sep 1 01:05:43 venus /kernel: trap number = 12 Sep 1 01:05:43 venus /kernel: panic: page fault Sep 1 01:05:43 venus /kernel: mp_lock = 00000003; cpuid = 0; lapic.id = 00000000 Sep 1 01:05:43 venus /kernel: boot() called on cpu#0 Sep 1 01:05:43 venus /kernel: Uptime: 4h26m4s Sep 1 01:05:43 venus /kernel: amr0: flushing cache...done Sep 1 01:05:43 venus /kernel: Automatic reboot in 15 seconds - press a key on the console to abort venus# nm -n /kernel | grep c016e6fb venus# nm -n /kernel | grep c016e6f venus# nm -n /kernel | grep c016e6 venus# nm -n /kernel | grep c016e c016e028 t sysctl_ovfs_conf c016e08c T vfs_mountedon c016e0a8 T vfs_unmountall c016e14c t vfs_hang_addrlist c016e334 t vfs_free_netcred c016e35c t vfs_free_addrlist c016e3b8 T vfs_export c016e42c T vfs_setpublicfs c016e598 T vfs_export_lookup c016e5f0 T vfs_msync c016e720 T vfs_object_create c016e758 T vfree c016e7f4 T vbusy c016e858 T vn_pollrecord c016e8cc T vn_pollevent c016e924 T vn_pollgone c016e968 T vfs_allocate_syncvnode c016ea40 t sync_fsync c016ead0 t sync_inactive c016eae4 t sync_reclaim c016eb30 t sync_print c016eb6c T vn_todev c016eb8c T vn_isdisk c016ebf8 T NDFREE c016ed14 T mount Sep 1 01:05:43 venus /kernel: Rebooting... Sep 1 01:05:43 venus /kernel: cpu_reset called on cpu#0 Sep 1 01:05:43 venus /kernel: cpu_reset: Stopping other CPUs Sep 1 01:05:46 venus /kernel: fxp0: promiscuous mode enabled Sep 1 01:05:46 venus trafd[239]: (fxp0) traffic collector started Sep 1 01:24:08 venus /kernel: Sep 1 01:24:08 venus /kernel: Sep 1 01:24:08 venus /kernel: Fatal trap 12: page fault while in kernel mode Sep 1 01:24:08 venus /kernel: mp_lock = 01000002; cpuid = 1; lapic.id = 01000000 Sep 1 01:24:08 venus /kernel: fault virtual address = 0x5 Sep 1 01:24:08 venus /kernel: fault code = supervisor read, page not present Sep 1 01:24:08 venus /kernel: instruction pointer = 0x8:0xc01e5d4b Sep 1 01:24:08 venus /kernel: stack pointer = 0x10:0xf3892d00 Sep 1 01:24:08 venus /kernel: frame pointer = 0x10:0xf3892d10 Sep 1 01:24:08 venus /kernel: code segment = base 0x0, limit 0xfffff, type 0x1b Sep 1 01:24:08 venus /kernel: = DPL 0, pres 1, def32 1, gran 1 Sep 1 01:24:08 venus /kernel: processor eflags = interrupt enabled, resume, IOPL = 0 Sep 1 01:24:08 venus /kernel: current process = 12787 (master) Sep 1 01:24:08 venus /kernel: interrupt mask = net bio cam <- SMP: XXX Sep 1 01:24:08 venus /kernel: trap number = 12 Sep 1 01:24:08 venus /kernel: panic: page fault Sep 1 01:24:08 venus /kernel: mp_lock = 01000002; cpuid = 1; lapic.id = 01000000 Sep 1 01:24:08 venus /kernel: boot() called on cpu#1 Sep 1 01:24:08 venus /kernel: Sep 1 01:24:08 venus /kernel: syncing disks... 219 103 99 99 99 99 99 99 99 78 78 78 78 78 78 78 52 52 52 52 52 52 52 52 52 52 52 52 52 52 52 52 52 52 52 52 Sep 1 01:24:08 venus /kernel: gi18>a.out ldconfig path: /usr/lib/aout /usr/lib/compat/aout /usr/X11R6/lib/aout Sep 1 01:24:08 venus /kernel: cpu_reset_proxy: Grabbed mp lock for BSP Sep 1 01:24:08 venus /kernel: cpu_reset_proxy: Stopped CPU 1 venus# nm -n /kernel | grep c01e5d4b venus# nm -n /kernel | grep c01e5d4 venus# nm -n /kernel | grep c01e5d c01e5d04 T pmap_remove_pages To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-stable" in the body of the message