Date: Mon, 09 Nov 2009 15:33:11 -0500 From: Mike Tancsa <mike@sentex.net> To: pyunyh@gmail.com, Mykola Dzham <freebsd@levsha.org.ua> Cc: jfv@freebsd.org, current@freebsd.org Subject: Re: page fault in igb driver on 8.0-RC2 Message-ID: <200911092033.nA9KX6dD013378@lava.sentex.ca> In-Reply-To: <20091017222314.GB19204@michelle.cdnetworks.com> References: <20091017170351.GZ29771@expo.ukrweb.net> <20091017222314.GB19204@michelle.cdnetworks.com>
next in thread | previous in thread | raw e-mail | index | archive | help
At 05:23 PM 10/17/2009, Pyun YongHyeon wrote:
>On Sat, Oct 17, 2009 at 08:03:51PM +0300, Mykola Dzham wrote:
> > Hi!
> > On hight network load system panics:
> >
> > GET BUF: dmamap load failure - 12
> > GET BUF: dmamap load failure - 12
> > GET BUF: dmamap load failure - 12
> > GET BUF: dmamap load failure - 12
> > GET BUF: dmamap load failure - 12
> > GET BUF: dmamap load failure - 12
> > GET BUF: dmamap load failure - 12
> > GET BUF: dmamap load failure - 12
> > GET BUF: dmamap load failure - 12
> > GET BUF: dmamap load failure - 12
> > GET BUF: dmamap load failure - 12
> > GET BUF: dmamap load failure - 12
> > GET BUF: dmamap load failure - 12
> > GET BUF: dmamap load failure - 12
> > GET BUF: dmamap load failure - 12
> >
>
>I believe this type of message should not be in fast path and it
>should be rate-limited.
>
> >
> > Fatal trap 12: page fault while in kernel mode
> > cpuid = 2; apic id = 02
> > fault virtual address = 0x0
> > fault code = supervisor read data, page not present
> > instruction pointer = 0x20:0xffffffff8025e4a5
> > stack pointer = 0x28:0xffffff87312f3a60
> > frame pointer = 0x28:0xffffff87312f3a80
> > code segment = base 0x0, limit 0xfffff, type 0x1b
> > = DPL 0, pres 1, long 1, def32 0, gran 1
> > processor eflags = interrupt enabled, resume, IOPL = 0
> > current process = 0 (igb0 taskq)
> > trap number = 12
> > panic: page fault
> > cpuid = 2
> > KDB: stack backtrace:
> > db_trace_self_wrapper() at 0xffffffff80185baa =
> > db_trace_self_wrapper+0x2a
> > panic() at 0xffffffff8020e992 = panic+0x182
> > trap_fatal() at 0xffffffff8040eefd = trap_fatal+0x2ad
> > trap_pfault() at 0xffffffff8040f27d = trap_pfault+0x22d
> > trap() at 0xffffffff8040fbff = trap+0x3cf
> > calltrap() at 0xffffffff803f6e13 = calltrap+0x8
> > --- trap 0xc, rip = 0xffffffff8025e4a5, rsp = 0xffffff87312f3a60, rbp =
> > 0xffffff87312f3a80 ---
> > mb_free_ext() at 0xffffffff8025e4a5 = mb_free_ext+0x15
> > igb_get_buf() at 0xffffffff80a3a6e5 = igb_get_buf+0x2e5
> > igb_rxeof() at 0xffffffff80a3abd5 = igb_rxeof+0x425
> > igb_handle_rx() at 0xffffffff80a3b14b = igb_handle_rx+0x3b
> > taskqueue_run() at 0xffffffff80243ec1 = taskqueue_run+0x91
> > taskqueue_thread_loop() at 0xffffffff8024404f =
> > taskqueue_thread_loop+0x3f
> > fork_exit() at 0xffffffff801ea9b2 = fork_exit+0x112
> > fork_trampoline() at 0xffffffff803f72ee = fork_trampoline+0xe
> > --- trap 0, rip = 0, rsp = 0xffffff87312f3d30, rbp = 0 ---
> > Uptime: 1h46m18s
I was just about to start testing such hardware and found the bug
fairly easy to reproduce. From /var/crash/core.txt
panic: page fault
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...
Unread portion of the kernel message buffer:
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 12 (irq257: igb0)
trap number = 12
panic: page fault
cpuid = 5
Uptime: 30m34s
Physical memory: 3560 MB
Dumping 252 MB: 237 221 205 189 173 157 141 125 109 93 77 61 45 29 13
#0 doadump () at pcpu.h:246
246 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) #0 doadump () at pcpu.h:246
#1 0xc08853b7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:416
#2 0xc08856a9 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:579
#3 0xc0bb6c7c in trap_fatal (frame=0xe75f8bd8, eva=16)
at /usr/src/sys/i386/i386/trap.c:938
#4 0xc0bb6f00 in trap_pfault (frame=0xe75f8bd8, usermode=0, eva=16)
at /usr/src/sys/i386/i386/trap.c:851
#5 0xc0bb7905 in trap (frame=0xe75f8bd8) at /usr/src/sys/i386/i386/trap.c:533
#6 0xc0b9a0cb in calltrap () at /usr/src/sys/i386/i386/exception.s:165
#7 0xc061eecc in igb_rxeof (rxr=0xc7400e00, count=100)
at /usr/src/sys/dev/e1000/if_igb.c:4074
#8 0xc061f409 in igb_msix_rx (arg=0xc7400e00)
at /usr/src/sys/dev/e1000/if_igb.c:1455
#9 0xc085d3db in intr_event_execute_handlers (p=0xc71527f8, ie=0xc748d680)
at /usr/src/sys/kern/kern_intr.c:1165
#10 0xc085e97b in ithread_loop (arg=0xc74b8970)
at /usr/src/sys/kern/kern_intr.c:1178
#11 0xc085b121 in fork_exit (callout=0xc085e910 <ithread_loop>,
arg=0xc74b8970, frame=0xe75f8d38) at /usr/src/sys/kern/kern_fork.c:843
#12 0xc0b9a140 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:270
(kgdb)
using the patch provided by Pyun YongHyeon provided, allows the
second port on the dual port nic to partially work, but the box still
panics after a bit of traffic. If I just use igb1, it seems to be ok.
But if I try and bring up igb0 and use it, 'bad things
happen'. Panic with patch below
panic: page fault
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...
Unread portion of the kernel message buffer:
Fatal trap 12: page fault while in kernel mode
cpuid = 5; apic id = 05
fault virtual address = 0xc
fault code = supervisor write, page not present
instruction pointer = 0x20:0xc061f147
stack pointer = 0x28:0xe75f7c24
frame pointer = 0x28:0xe75f7c78
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 12 (irq257: igb0)
trap number = 12
panic: page fault
cpuid = 5
Uptime: 8m27s
Physical memory: 3560 MB
Dumping 127 MB: 112 96 80 64 48 32 16
#0 doadump () at pcpu.h:246
246 pcpu.h: No such file or directory.
in pcpu.h
(kgdb) #0 doadump () at pcpu.h:246
#1 0xc0885597 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:416
#2 0xc0885889 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:579
#3 0xc0bb6e1c in trap_fatal (frame=0xe75f7be4, eva=12)
at /usr/src/sys/i386/i386/trap.c:938
#4 0xc0bb70a0 in trap_pfault (frame=0xe75f7be4, usermode=0, eva=12)
at /usr/src/sys/i386/i386/trap.c:851
#5 0xc0bb7aa5 in trap (frame=0xe75f7be4) at /usr/src/sys/i386/i386/trap.c:533
#6 0xc0b9a26b in calltrap () at /usr/src/sys/i386/i386/exception.s:165
#7 0xc061f147 in igb_rxeof (rxr=0xc7400e00, count=100)
at /usr/src/sys/dev/e1000/if_igb.c:4130
#8 0xc061f679 in igb_msix_rx (arg=0xc7400e00)
at /usr/src/sys/dev/e1000/if_igb.c:1456
#9 0xc085d5bb in intr_event_execute_handlers (p=0xc71527f8, ie=0xc748d680)
at /usr/src/sys/kern/kern_intr.c:1165
#10 0xc085eb5b in ithread_loop (arg=0xc74b8960)
at /usr/src/sys/kern/kern_intr.c:1178
#11 0xc085b301 in fork_exit (callout=0xc085eaf0 <ithread_loop>,
arg=0xc74b8960, frame=0xe75f7d38) at /usr/src/sys/kern/kern_fork.c:843
#12 0xc0b9a2e0 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:270
(kgdb)
>_______________________________________________
>freebsd-current@freebsd.org mailing list
>http://lists.freebsd.org/mailman/listinfo/freebsd-current
>To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
--------------------------------------------------------------------
Mike Tancsa, tel +1 519 651 3400
Sentex Communications, mike@sentex.net
Providing Internet since 1994 www.sentex.net
Cambridge, Ontario Canada www.sentex.net/mike
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200911092033.nA9KX6dD013378>