From owner-freebsd-current@FreeBSD.ORG Mon Nov 26 11:25:40 2007 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F1B2A16A419 for ; Mon, 26 Nov 2007 11:25:39 +0000 (UTC) (envelope-from biancalana@gmail.com) Received: from py-out-1112.google.com (py-out-1112.google.com [64.233.166.178]) by mx1.freebsd.org (Postfix) with ESMTP id 9D27413C474 for ; Mon, 26 Nov 2007 11:25:39 +0000 (UTC) (envelope-from biancalana@gmail.com) Received: by py-out-1112.google.com with SMTP id u77so1489274pyb for ; Mon, 26 Nov 2007 03:25:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=GJtyK4s6t5Dd84VTPZ7kt0NCVrVg8ae/wAueItRmfTc=; b=rrp8sa0hWHxSVOyQjB3lokknJaeczQq1FMF2kkK0HsV8N1Y76gz3Wp9mOhUg0GDDfj3LzNEbjuf1dnWOob+9x3aZa39V2KhtT3PDUYPcd1+/Fd5zSVI5uVmNq6XhFYbv6bvzaEX8cBlRUH0omE3z4YmWDpRrEkt7ZOfQEbVtPOU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=jD4UKARablJVqHMN8Bx0CbVh6zLRsdSk+COS+d9V/lCbjLBsh/Oo6n2mcQkdwjdpgXpNZX6emggFQlhzOIDKrIP0EUDEr5Yes61sqsc8J38TGfpPo9XQL4sSd8hPxbK2qocAegsYVy7TZrQWfXo6uiGlrpz0NGBhRRM4DBg8JnY= Received: by 10.65.93.19 with SMTP id v19mr5656587qbl.1196076337906; Mon, 26 Nov 2007 03:25:37 -0800 (PST) Received: by 10.64.184.9 with HTTP; Mon, 26 Nov 2007 03:25:37 -0800 (PST) Message-ID: <8e10486b0711260325o5cf899cbu35a126e156320e26@mail.gmail.com> Date: Mon, 26 Nov 2007 09:25:37 -0200 From: "Alexandre Biancalana" To: freebsd-current@freebsd.org In-Reply-To: <8e10486b0711240428v86b36b3y69ce5f004bbd4156@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <8e10486b0711240409m699c78afy65bfefbb486ab838@mail.gmail.com> <474815F6.5030400@FreeBSD.org> <8e10486b0711240428v86b36b3y69ce5f004bbd4156@mail.gmail.com> Subject: Re: BETA3 crash (zfs related ?) X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Nov 2007 11:25:40 -0000 On Nov 24, 2007 10:28 AM, Alexandre Biancalana wrote: > On Nov 24, 2007 9:15 AM, Remko Lodder wrote: > > > > Can you look at the developers handbook and follow the ways to get > > a proper backtrace which we can investigate? I see from the logfile > > that you send along, that a dump had been made, so the information > > might be there :-) > > > > http://www.freebsd.org/doc/en/books/developers-handbook/kerneldebug.html > > My kernel is already compiled with all debug stuff. I forget to > mention that I tried to debug but I can't look at this: > > Manny:/var/crash # kgdb /boot/kernel/kernel vmcore.6 > [GDB will not be able to debug user-mode threads: > /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd". > (no debugging symbols found)...Attempt to extract a component of a > value that is not a structure pointer. > (kgdb) bt > #0 0xffffffff8032668a in doadump () > #1 0xffffffffd8272450 in ?? () > #2 0xffffffff80326ba5 in boot () > Previous frame inner to this frame (corrupt stack?) > (kgdb) > > > Can you help with that ? > > Thank you ! > Hi People ! Last night I get a similar crash, follow the messages from /var/log/messages Nov 26 02:53:41 Manny kernel: uma_zalloc_arg: zone "256" with the following non-sleepable locks held: Nov 26 02:53:41 Manny kernel: exclusive sleep mutex struct mount mtx r = 0 (0xffffff0003aba6a0) locked @ /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_vfs.c:206 Nov 26 02:53:41 Manny kernel: KDB: stack backtrace: Nov 26 02:53:41 Manny kernel: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a Nov 26 02:53:41 Manny kernel: witness_warn() at witness_warn+0x248 Nov 26 02:53:41 Manny kernel: uma_zalloc_arg() at uma_zalloc_arg+0x33a Nov 26 02:53:41 Manny kernel: malloc() at malloc+0x8a Nov 26 02:53:41 Manny kernel: crget() at crget+0x1f Nov 26 02:53:41 Manny kernel: crdup() at crdup+0x19 Nov 26 02:53:41 Manny kernel: domount() at domount+0x1fe Nov 26 02:53:41 Manny kernel: zfsctl_snapdir_lookup() at zfsctl_snapdir_lookup+0x307 Nov 26 02:53:41 Manny kernel: VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x95 Nov 26 02:53:41 Manny kernel: lookup() at lookup+0x4cd Nov 26 02:53:41 Manny kernel: namei() at namei+0x2cb Nov 26 02:53:41 Manny kernel: kern_stat() at kern_stat+0x61 Nov 26 02:53:41 Manny kernel: stat() at stat+0x2a Nov 26 02:53:41 Manny kernel: syscall() at syscall+0x1ce Nov 26 02:53:41 Manny kernel: Xfast_syscall() at Xfast_syscall+0xab Nov 26 02:53:41 Manny kernel: --- syscall (188, FreeBSD ELF64, stat), rip = 0x800c6547c, rsp = 0x7fffffffe798, rbp = 0x2 --- Nov 26 02:53:47 Manny kernel: uma_zalloc_arg: zone "256" with the following non-sleepable locks held: Nov 26 02:53:47 Manny kernel: exclusive sleep mutex struct mount mtx r = 0 (0xffffff00032a96a0) locked @ /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_vfs.c:206 Nov 26 02:53:47 Manny kernel: KDB: stack backtrace: Nov 26 02:53:47 Manny kernel: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a Nov 26 02:53:47 Manny kernel: witness_warn() at witness_warn+0x248 Nov 26 02:53:47 Manny kernel: uma_zalloc_arg() at uma_zalloc_arg+0x33a Nov 26 02:53:47 Manny kernel: malloc() at malloc+0x8a Nov 26 02:53:47 Manny kernel: crget() at crget+0x1f Nov 26 02:53:47 Manny kernel: crdup() at crdup+0x19 Nov 26 02:53:47 Manny kernel: domount() at domount+0x1fe Nov 26 02:53:47 Manny kernel: zfsctl_snapdir_lookup() at zfsctl_snapdir_lookup+0x307 Nov 26 02:53:47 Manny kernel: VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x95 Nov 26 02:53:47 Manny kernel: lookup() at lookup+0x4cd Nov 26 02:53:47 Manny kernel: namei() at namei+0x2cb Nov 26 02:53:47 Manny kernel: kern_stat() at kern_stat+0x61 Nov 26 02:53:47 Manny kernel: stat() at stat+0x2a Nov 26 02:53:47 Manny kernel: syscall() at syscall+0x1ce Nov 26 02:53:47 Manny kernel: Xfast_syscall() at Xfast_syscall+0xab Nov 26 02:53:47 Manny kernel: --- syscall (188, FreeBSD ELF64, stat), rip = 0x800c6547c, rsp = 0x7fffffffe798, rbp = 0x2 --- Nov 26 02:53:48 Manny kernel: uma_zalloc_arg: zone "256" with the following non-sleepable locks held: Nov 26 02:53:48 Manny kernel: exclusive sleep mutex struct mount mtx r = 0 (0xffffff00032a9378) locked @ /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_vfs.c:206 Nov 26 02:53:48 Manny kernel: KDB: stack backtrace: Nov 26 02:53:48 Manny kernel: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a Nov 26 02:53:48 Manny kernel: witness_warn() at witness_warn+0x248 Nov 26 02:53:48 Manny kernel: uma_zalloc_arg() at uma_zalloc_arg+0x33a Nov 26 02:53:48 Manny kernel: malloc() at malloc+0x8a Nov 26 02:53:48 Manny kernel: crget() at crget+0x1f Nov 26 02:53:48 Manny kernel: Nov 26 02:53:48 Manny kernel: crdup() at Nov 26 02:53:48 Manny kernel: crdup+0x19 Nov 26 02:53:48 Manny kernel: domount() at domount+0x1fe Nov 26 02:53:48 Manny kernel: zfsctl_snapdir_lookup() at zfsctl_snapdir_lookup Nov 26 02:53:48 Manny kernel: +0x307 Nov 26 02:53:48 Manny kernel: Nov 26 02:53:48 Manny kernel: VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x95 Nov 26 02:53:48 Manny kernel: lookup() at Nov 26 02:53:48 Manny kernel: lookup+0x4cd Nov 26 02:53:48 Manny kernel: namei() at namei+0x2cb Nov 26 02:53:48 Manny kernel: kern_stat() at kern_stat+0x61 Nov 26 02:53:48 Manny kernel: Nov 26 02:53:48 Manny kernel: stat() at stat+0x2a Nov 26 02:53:48 Manny kernel: syscall() at Nov 26 02:53:48 Manny kernel: syscall+0x1ce Nov 26 02:53:48 Manny kernel: Xfast_sys Nov 26 02:53:48 Manny kernel: call() at Nov 26 02:53:48 Manny kernel: Xfast_syscall+0xab Nov 26 02:53:48 Manny kernel: --- syscall (188, FreeBSD ELF64, stat), rip = 0x800c6547c, rsp = 0x7fffffffe79 Nov 26 02:53:48 Manny kernel: 8 Nov 26 02:53:48 Manny kernel: , rbp = 0x2 --- Nov 26 02:56:34 Manny kernel: uma_zalloc_arg: zone "256" with the following non-sleepable locks held: Nov 26 02:56:34 Manny kernel: exclusive sleep mutex struct mount mtx r = 0 (0xffffff0003ab96a0) locked @ /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_vfs.c:206 Nov 26 02:56:34 Manny kernel: KDB: stack backtrace: Nov 26 02:56:34 Manny kernel: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a Nov 26 02:56:34 Manny kernel: witness_warn() at witness_warn+0x248 Nov 26 02:56:34 Manny kernel: uma_zalloc_arg() at uma_zalloc_arg+0x33a Nov 26 02:56:34 Manny kernel: malloc() at malloc+0x8a Nov 26 02:56:34 Manny kernel: crget() at crget+0x1f Nov 26 02:56:34 Manny kernel: crdup() at crdup+0x19 Nov 26 02:56:34 Manny kernel: domount() at domount+0x1fe Nov 26 02:56:34 Manny kernel: zfsctl_snapdir_lookup() at zfsctl_snapdir_lookup+0x307 Nov 26 02:56:34 Manny kernel: VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x95 Nov 26 02:56:34 Manny kernel: lookup() at lookup+0x4cd Nov 26 02:56:34 Manny kernel: namei() at namei+0x2cb Nov 26 02:56:34 Manny kernel: kern_stat() at kern_stat+0x61 Nov 26 02:56:34 Manny kernel: stat() at stat+0x2a Nov 26 02:56:34 Manny kernel: syscall() at syscall+0x1ce Nov 26 02:56:34 Manny kernel: Xfast_syscall() at Xfast_syscall+0xab Nov 26 02:56:34 Manny kernel: --- syscall (188, FreeBSD ELF64, stat), rip = 0x800c6547c, rsp = 0x7fffffffe798, rbp = 0x2 --- Nov 26 02:56:37 Manny kernel: uma_zalloc_arg: zone "256" with the following non-sleepable locks held: Nov 26 02:56:37 Manny kernel: exclusive sleep mutex struct mount mtx r = 0 (0xffffff0003ab9378) locked @ /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_vfs.c:206 Nov 26 02:56:37 Manny kernel: KDB: stack backtrace: Nov 26 02:56:37 Manny kernel: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a Nov 26 02:56:37 Manny kernel: witness_warn() at witness_warn+0x248 Nov 26 02:56:37 Manny kernel: uma_zalloc_arg() at uma_zalloc_arg+0x33a Nov 26 02:56:37 Manny kernel: malloc() at malloc+0x8a Nov 26 02:56:37 Manny kernel: crget() at crget+0x1f Nov 26 02:56:37 Manny kernel: crdup() at crdup+0x19 Nov 26 02:56:37 Manny kernel: domount() at domount+0x1fe Nov 26 02:56:37 Manny kernel: zfsctl_snapdir_lookup() at Nov 26 02:56:37 Manny kernel: zfsctl_snapdir_lookup+0x307 Nov 26 02:56:37 Manny kernel: VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x95 Nov 26 02:56:37 Manny kernel: Nov 26 02:56:37 Manny kernel: lookup() at lookup+0x4cd Nov 26 02:56:37 Manny kernel: namei() at Nov 26 02:56:37 Manny kernel: namei+0x2cb Nov 26 02:56:37 Manny kernel: k Nov 26 02:56:37 Manny kernel: ern_stat Nov 26 02:56:37 Manny kernel: () at Nov 26 02:56:37 Manny kernel: kern_stat+0x61 Nov 26 02:56:37 Manny kernel: stat() at stat+0x2a Nov 26 02:56:37 Manny kernel: Nov 26 02:56:37 Manny kernel: syscall() at Nov 26 02:56:37 Manny kernel: syscall+0x1ce Nov 26 02:56:37 Manny kernel: Xfast_syscall() at Nov 26 02:56:37 Manny kernel: Xfast_syscall+0xab Nov 26 02:56:37 Manny kernel: --- syscall (188, FreeBSD ELF64, stat), rip = 0x800c6547c, rsp = 0x7fffffffe798, rbp = 0x2 --- Nov 26 02:56:37 Manny kernel: uma_zalloc_arg: zone "256" with the following non-sleepable locks held: Nov 26 02:56:37 Manny kernel: exclusive sleep mutex struct mount mtx r = 0 (0xffffff0003ab9050) locked @ /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_vfs.c:206 Nov 26 02:56:37 Manny kernel: KDB: stack backtrace: Nov 26 02:56:37 Manny kernel: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a Nov 26 02:56:37 Manny kernel: witness_warn() at witness_warn+0x248 Nov 26 02:56:37 Manny kernel: uma_zalloc_arg() at uma_zalloc_arg+0x33a Nov 26 02:56:37 Manny kernel: malloc() at malloc+0x8a Nov 26 02:56:37 Manny kernel: crget() at crget+0x1f Nov 26 02:56:37 Manny kernel: crdup() at crdup+0x19 Nov 26 02:56:37 Manny kernel: domount() at domount+0x1fe Nov 26 02:56:37 Manny kernel: zfsctl_snapdir_lookup() at zfsctl_snapdir_lookup+0x307 Nov 26 02:56:37 Manny kernel: VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x95 Nov 26 02:56:37 Manny kernel: lookup() at lookup+0x4cd Nov 26 02:56:37 Manny kernel: namei() at namei+0x2cb Nov 26 02:56:37 Manny kernel: kern_stat() at kern_stat+0x61 Nov 26 02:56:37 Manny kernel: stat() at stat+0x2a Nov 26 02:56:37 Manny kernel: syscall() at syscall+0x1ce Nov 26 02:56:37 Manny kernel: Xfast_syscall() at Xfast_syscall+0xab Nov 26 02:56:37 Manny kernel: --- syscall (188, FreeBSD ELF64, stat), rip = 0x800c6547c, rsp = 0x7fffffffe798, rbp = 0x2 --- Nov 26 02:56:37 Manny kernel: uma_zalloc_arg: zone "256" with the following non-sleepable locks held: Nov 26 02:56:37 Manny kernel: exclusive sleep mutex struct mount mtx r = 0 (0xffffff0003944050) locked @ /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_vfs.c:206 Nov 26 02:56:37 Manny kernel: KDB: stack backtrace: Nov 26 02:56:37 Manny kernel: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a Nov 26 02:56:37 Manny kernel: witness_warn() at witness_warn+0x248 Nov 26 02:56:37 Manny kernel: uma_zalloc_arg() at uma_zalloc_arg+0x33a Nov 26 02:56:37 Manny kernel: malloc() at malloc+0x8a Nov 26 02:56:37 Manny kernel: crget() at crget+0x1f Nov 26 02:56:37 Manny kernel: crdup() at crdup+0x19 Nov 26 02:56:37 Manny kernel: domount() at domount+0x1fe Nov 26 02:56:37 Manny kernel: zfsctl_snapdir_lookup() at zfsctl_snapdir_lookup+0x307 Nov 26 02:56:37 Manny kernel: VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x95 Nov 26 02:56:37 Manny kernel: lookup() at Nov 26 02:56:37 Manny kernel: lookup+0x4cd Nov 26 02:56:37 Manny kernel: namei() at namei+0x2cb Nov 26 02:56:37 Manny kernel: Nov 26 02:56:37 Manny kernel: kern_stat() at kern_stat+0x61 Nov 26 02:56:37 Manny kernel: stat() at stat+0x2a Nov 26 02:56:37 Manny kernel: Nov 26 02:56:37 Manny kernel: syscall() at Nov 26 02:56:37 Manny kernel: syscall+0x1ce Nov 26 02:56:37 Manny kernel: Xfast_syscall() at Xfast_syscall+0xab Nov 26 02:56:37 Manny kernel: Nov 26 02:56:37 Manny kernel: --- syscall (1 Nov 26 02:56:37 Manny kernel: 88 Nov 26 02:56:37 Manny kernel: , FreeBSD ELF64, stat), rip = 0x800c6547c, rsp = 0x7fffffffe798, rbp = 0x2 --- Nov 26 02:56:44 Manny kernel: uma_zalloc_arg: zone "256" with the following non-sleepable locks held: Nov 26 02:56:44 Manny kernel: exclusive sleep mutex struct mount mtx r = 0 (0xffffff00039b06a0) locked @ /usr/src/sys/modules/zfs/../../compat/opensolaris/kern/opensolaris_vfs.c:206 Nov 26 02:56:44 Manny kernel: KDB: stack backtrace: Nov 26 02:56:44 Manny kernel: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a Nov 26 02:56:44 Manny kernel: witness_warn() at witness_warn+0x248 Nov 26 02:56:44 Manny kernel: uma_zalloc_arg() at uma_zalloc_arg+0x33a Nov 26 02:56:44 Manny kernel: malloc() at malloc+0x8a Nov 26 02:56:44 Manny kernel: crget() at crget+0x1f Nov 26 02:56:44 Manny kernel: crdup() at crdup+0x19 Nov 26 02:56:44 Manny kernel: domount() at domount+0x1fe Nov 26 02:56:44 Manny kernel: zfsctl_snapdir_lookup() at zfsctl_snapdir_lookup+0x307 Nov 26 02:56:44 Manny kernel: VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0x95 Nov 26 02:56:44 Manny kernel: lookup() at lookup+0x4cd Nov 26 02:56:44 Manny kernel: namei() at namei+0x2cb Nov 26 02:56:44 Manny kernel: kern_stat() at kern_stat+0x61 Nov 26 02:56:44 Manny kernel: stat() at stat+0x2a Nov 26 02:56:44 Manny kernel: syscall() at syscall+0x1ce Nov 26 02:56:44 Manny kernel: Xfast_syscall() at Xfast_syscall+0xab Nov 26 02:56:44 Manny kernel: --- syscall (188, FreeBSD ELF64, stat), rip = 0x800c6547c, rsp = 0x7fffffffe798 Nov 26 02:56:44 Manny kernel: , rbp = 0x2 --- Nov 26 03:10:55 Manny syslogd: kernel boot file is /boot/kernel/kernel Nov 26 03:10:55 Manny kernel: panic: vm_fault: fault on nofault entry, addr: ffffffffd8547000 Nov 26 03:10:55 Manny kernel: cpuid = 1 Nov 26 03:10:55 Manny kernel: KDB: stack backtrace: Nov 26 03:10:55 Manny kernel: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a Nov 26 03:10:55 Manny kernel: panic() at panic+0x17a Nov 26 03:10:55 Manny kernel: vm_fault() at vm_fault+0x14c1 Nov 26 03:10:55 Manny kernel: trap_pfault() at trap_pfault+0x218 Nov 26 03:10:55 Manny kernel: trap() at trap+0x30c Nov 26 03:10:55 Manny kernel: calltrap() at calltrap+0x8 Nov 26 03:10:55 Manny kernel: --- trap 0xc, rip = 0xffffffffd8198e54, rsp = 0xffffffffd8168960, rbp = 0xffffffffd81689a0 --- Nov 26 03:10:55 Manny kernel: dsl_dir_set_reservation_check() at dsl_dir_set_reservation_check+0x24 Nov 26 03:10:55 Manny kernel: dsl_sync_task_group_sync() at dsl_sync_task_group_sync+0x96 Nov 26 03:10:55 Manny kernel: dsl_pool_sync() at dsl_pool_sync+0xc3 Nov 26 03:10:55 Manny kernel: spa_sync() at spa_sync+0x390 Nov 26 03:10:55 Manny kernel: txg_sync_thread() at txg_sync_thread+0x12f Nov 26 03:10:55 Manny kernel: fork_exit() at fork_exit+0x12a Nov 26 03:10:55 Manny kernel: fork_trampoline() at fork_trampoline+0xe Nov 26 03:10:55 Manny kernel: --- trap 0, rip = 0, rsp = 0xffffffffd8168d30, rbp = 0 --- Nov 26 03:10:55 Manny kernel: Uptime: 12h55m27s Nov 26 03:10:55 Manny kernel: Physical memory: 2036 MB Nov 26 03:10:55 Manny kernel: Dumping 1178 MB: 1163 1147 1131 1115 1099 1083 1067 1051 1035 1019 1003 987 971 955 939 923 907 891 875 859 843 827 811 795 779 763 747 731 715 699 683 667 651 635 619 603 587 571 555 539 523 507 491 475 459 443 427 411 395 379 363 347 331 315 299 283 267 251 235 219 203 187 171 155 139 123 107 91 75 59 43 27 11 Nov 26 03:10:55 Manny kernel: Dump complete Nov 26 03:10:55 Manny kernel: Automatic reboot in 15 seconds - press a key on the console to abort Nov 26 03:10:55 Manny kernel: Rebooting... Nov 26 03:10:55 Manny kernel: cpu_reset: Stopping other CPUs