Date: Sat, 30 Jul 2011 12:05:33 -0700 From: maestro something <maestro82@gmail.com> To: Andriy Gapon <avg@freebsd.org> Cc: freebsd-stable@freebsd.org Subject: Re: dtrace ustack kernel panic Message-ID: <CAJ_JOqu2SR2n1Cm__3x3rfcJpYY=ihnoE9oyuKeNmKGuUGgnGA@mail.gmail.com> In-Reply-To: <CAJ_JOqvCDE1Zcv4jz14rtZu5_NmqfY04tz91uy76WnaW2trO7Q@mail.gmail.com> References: <CAJ_JOqvEmXBTBABhUcJ66=bh9%2B8S%2BC9v30hXxVZiCXuEpGPJ1A@mail.gmail.com> <4E2E9F60.1060808@FreeBSD.org> <CAJ_JOqszViwLi6TeQxAxeX2Mte5eBPsGJpjQPVOQs2BOwAq9JQ@mail.gmail.com> <4E33B7CF.90200@FreeBSD.org> <CAJ_JOqt4VdgJm3NnB1KUf1RFuk75nu6-Rh=Bqb53h5TAEzB0%2BA@mail.gmail.com> <4E344D15.1040508@FreeBSD.org> <CAJ_JOqvCDE1Zcv4jz14rtZu5_NmqfY04tz91uy76WnaW2trO7Q@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi,
>> Have you started kgdb with the correct kernel and core file?
>> If yes, then I am out of ideas.
>>
>
> I hope so, I only recompiled the kernel once according to the DTRACE wiki
> instructions and I certainly only have one /var/crash/vmcore.* file.
>
> I'll try recompiling the kernel with -O1 and try again. In the meantime,
> I'm wondering whether I'm really the only/first one that ran into this
> problem or if there are people that actually successfully used the ustack()
> target on freebsd-8.2?
>
I could not get the information even after recompiling the kernel here is
the relevant (I think information).
fb82i386# cat /etc/make.conf
CFLAGS= -O
(accodring to man make.conf only -O and -O2 is supported for CFLAGS anyways)
kernel.debug is the newly compiled kernel (according to the timestamp)
fb82i386# kgdb kernel.debug /var/crash/vmcore.0
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...
Unread portion of the kernel message buffer:
kernel trap 12 with interrupts disabled
Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address = 0x108
fault code = supervisor write, page not present
instruction pointer = 0x20:0xc1100847
stack pointer = 0x28:0xcd39a9e4
frame pointer = 0x28:0xcd39a9fc
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = resume, IOPL = 0
current process = 1060 (nc)
trap number = 12
panic: page fault
cpuid = 0
KDB: stack backtrace:
#0 0xc09036a7 at kdb_backtrace+0x47
#1 0xc08d1a07 at panic+0x117
#2 0xc0c158c3 at trap_fatal+0x323
#3 0xc0c15bc0 at trap_pfault+0x2f0
#4 0xc0c1612a at trap+0x48a
#5 0xc0bfc97c at calltrap+0x6
#6 0xc10e99db at dtrace_panic+0x1b
#7 0xc10e9a0d at dtrace_assfail+0x2d
#8 0xc10fa6a6 at dtrace_probe+0xfd6
#9 0xc1237ce4 at systrace_probe+0x84
#10 0xc090f63f at syscallenter+0x47f
#11 0xc0c15c14 at syscall+0x34
#12 0xc0bfca11 at Xint0x80_syscall+0x21
Uptime: 2m39s
Physical memory: 239 MB
Dumping 78 MB: 63 47 31 15
(kgdb) where
#0 doadump () at pcpu.h:231
#1 0xc08d17a3 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:419
#2 0xc08d1a40 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:592
#3 0xc0c158c3 in trap_fatal (frame=0xcd39a9a4, eva=264) at
/usr/src/sys/i386/i386/trap.c:946
#4 0xc0c15bc0 in trap_pfault (frame=0xcd39a9a4, usermode=0, eva=264) at
/usr/src/sys/i386/i386/trap.c:859
#5 0xc0c1612a in trap (frame=0xcd39a9a4) at
/usr/src/sys/i386/i386/trap.c:532
#6 0xc0bfc97c in calltrap () at /usr/src/sys/i386/i386/exception.s:166
#7 0xc1100847 in dtrace_panic_trigger () from /boot/kernel/dtrace.ko
Previous frame inner to this frame (corrupt stack?)
(kgdb) list *dtrace_probe+0xfd6
No source file for address 0xc10fa6a6.
So I'm stuck at the same point.
any other ideas?
cheers
--m
>
> cheers
> --m
>
>
>>
>> >
>> > > On Tue, Jul 26, 2011 at 4:05 AM, Andriy Gapon <avg@freebsd.org
>> > <mailto:avg@freebsd.org>
>> > > <mailto:avg@freebsd.org <mailto:avg@freebsd.org>>> wrote:
>> > >
>> > > on 26/07/2011 04:20 maestro something said the following:
>> > > > Hi,
>> > > >
>> > > > when using the ustack action on the latest FreeBSD8.2 i386
>> the kernel
>> > > > panics.
>> > > >
>> > > > Here is the information I could gather:
>> > > >
>> > > > let me know if I can provide more information. (i.e., i have
>> the
>> > full crash
>> > > > information 80+mbs handy)
>> > >
>> > > Use kgdb on the crash dump, go to the dtrace_probe frame and
>> see which
>> > exactly
>> > > assert fails and why.
>> > >
>> > > > Here is the backtrace:
>> > > >
>> > > > Fatal trap 12: page fault while in kernel mode
>> > > > cpuid = 0; apic id = 00
>> > > > fault virtual address = 0x108
>> > > > fault code = supervisor write, page not present
>> > > > instruction pointer = 0x20:0xc11012d7
>> > > > stack pointer = 0x28:0xcd3ed9f4
>> > > > frame pointer = 0x28:0xcd3eda0c
>> > > > code segment = base 0x0, limit 0xfffff, type 0x1b
>> > > > = DPL 0, pres 1, def32 1, gran 1
>> > > > processor eflags = resume, IOPL = 0
>> > > > current process = 1132 (nc)
>> > > > trap number = 12
>> > > > panic: page fault
>> > > > cpuid = 0
>> > > > KDB: stack backtrace:
>> > > > #0 0xc09036a7 at kdb_backtrace+0x47
>> > > > #1 0xc08d1a07 at panic+0x117
>> > > > #2 0xc0c158c3 at trap_fatal+0x323
>> > > > #3 0xc0c15bc0 at trap_pfault+0x2f0
>> > > > #4 0xc0c1612a at trap+0x48a
>> > > > #5 0xc0bfc97c at calltrap+0x6
>> > > > #6 0xc10e992b at dtrace_panic+0x1b
>> > > > #7 0xc10e995d at dtrace_assfail+0x2d
>> > > > #8 0xc10fb28c at dtrace_probe+0x135c
>> > > > #9 0xc1237f72 at systrace_probe+0x62
>> > > > #10 0xc090f63f at syscallenter+0x47f
>> > > > #11 0xc0c15c14 at syscall+0x34
>> > > > #12 0xc0bfca11 at Xint0x80_syscall+0x21
>> > > > Uptime: 3m0s
>> > > > Physical memory: 239 MB
>> > > > Dumping 79 MB: 64 48 32 16
>> > > >
>> > > >
>> > > > Steps To reproduce:
>> > > >
>> > > > the dtrace program (i.e., test.d) was:
>> > > >
>> > > > #!/usr/sbin/dtrace -s
>> > > >
>> > > > syscall::accept:return
>> > > > / execname == "nc" /
>> > > > {
>> > > > printf("%s accept:return\n", execname);
>> > > > ustack();
>> > > > }
>> > > >
>> > > > % dtrace -s test.d
>> > > >
>> > > > then running
>> > > > % nc -vl 8080
>> > > > on one shell and
>> > > > % nc localhost 8080
>> > > > on another would make the kernel panic
>> >
>> >
>> > --
>> > Andriy Gapon
>> >
>> >
>>
>>
>> --
>> Andriy Gapon
>>
>
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAJ_JOqu2SR2n1Cm__3x3rfcJpYY=ihnoE9oyuKeNmKGuUGgnGA>