From owner-freebsd-current@FreeBSD.ORG Mon Nov 9 20:33:08 2009 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D75E2106566B for ; Mon, 9 Nov 2009 20:33:08 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by mx1.freebsd.org (Postfix) with ESMTP id B0EEB8FC19 for ; Mon, 9 Nov 2009 20:33:08 +0000 (UTC) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.14.3/8.14.3) with ESMTP id nA9KX6dD013378; Mon, 9 Nov 2009 15:33:06 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200911092033.nA9KX6dD013378@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Mon, 09 Nov 2009 15:33:11 -0500 To: pyunyh@gmail.com, Mykola Dzham From: Mike Tancsa In-Reply-To: <20091017222314.GB19204@michelle.cdnetworks.com> References: <20091017170351.GZ29771@expo.ukrweb.net> <20091017222314.GB19204@michelle.cdnetworks.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Cc: jfv@freebsd.org, current@freebsd.org Subject: Re: page fault in igb driver on 8.0-RC2 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Nov 2009 20:33:09 -0000 At 05:23 PM 10/17/2009, Pyun YongHyeon wrote: >On Sat, Oct 17, 2009 at 08:03:51PM +0300, Mykola Dzham wrote: > > Hi! > > On hight network load system panics: > > > > GET BUF: dmamap load failure - 12 > > GET BUF: dmamap load failure - 12 > > GET BUF: dmamap load failure - 12 > > GET BUF: dmamap load failure - 12 > > GET BUF: dmamap load failure - 12 > > GET BUF: dmamap load failure - 12 > > GET BUF: dmamap load failure - 12 > > GET BUF: dmamap load failure - 12 > > GET BUF: dmamap load failure - 12 > > GET BUF: dmamap load failure - 12 > > GET BUF: dmamap load failure - 12 > > GET BUF: dmamap load failure - 12 > > GET BUF: dmamap load failure - 12 > > GET BUF: dmamap load failure - 12 > > GET BUF: dmamap load failure - 12 > > > >I believe this type of message should not be in fast path and it >should be rate-limited. > > > > > Fatal trap 12: page fault while in kernel mode > > cpuid = 2; apic id = 02 > > fault virtual address = 0x0 > > fault code = supervisor read data, page not present > > instruction pointer = 0x20:0xffffffff8025e4a5 > > stack pointer = 0x28:0xffffff87312f3a60 > > frame pointer = 0x28:0xffffff87312f3a80 > > code segment = base 0x0, limit 0xfffff, type 0x1b > > = DPL 0, pres 1, long 1, def32 0, gran 1 > > processor eflags = interrupt enabled, resume, IOPL = 0 > > current process = 0 (igb0 taskq) > > trap number = 12 > > panic: page fault > > cpuid = 2 > > KDB: stack backtrace: > > db_trace_self_wrapper() at 0xffffffff80185baa = > > db_trace_self_wrapper+0x2a > > panic() at 0xffffffff8020e992 = panic+0x182 > > trap_fatal() at 0xffffffff8040eefd = trap_fatal+0x2ad > > trap_pfault() at 0xffffffff8040f27d = trap_pfault+0x22d > > trap() at 0xffffffff8040fbff = trap+0x3cf > > calltrap() at 0xffffffff803f6e13 = calltrap+0x8 > > --- trap 0xc, rip = 0xffffffff8025e4a5, rsp = 0xffffff87312f3a60, rbp = > > 0xffffff87312f3a80 --- > > mb_free_ext() at 0xffffffff8025e4a5 = mb_free_ext+0x15 > > igb_get_buf() at 0xffffffff80a3a6e5 = igb_get_buf+0x2e5 > > igb_rxeof() at 0xffffffff80a3abd5 = igb_rxeof+0x425 > > igb_handle_rx() at 0xffffffff80a3b14b = igb_handle_rx+0x3b > > taskqueue_run() at 0xffffffff80243ec1 = taskqueue_run+0x91 > > taskqueue_thread_loop() at 0xffffffff8024404f = > > taskqueue_thread_loop+0x3f > > fork_exit() at 0xffffffff801ea9b2 = fork_exit+0x112 > > fork_trampoline() at 0xffffffff803f72ee = fork_trampoline+0xe > > --- trap 0, rip = 0, rsp = 0xffffff87312f3d30, rbp = 0 --- > > Uptime: 1h46m18s I was just about to start testing such hardware and found the bug fairly easy to reproduce. From /var/crash/core.txt panic: page fault GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd"... Unread portion of the kernel message buffer: processor eflags = interrupt enabled, resume, IOPL = 0 current process = 12 (irq257: igb0) trap number = 12 panic: page fault cpuid = 5 Uptime: 30m34s Physical memory: 3560 MB Dumping 252 MB: 237 221 205 189 173 157 141 125 109 93 77 61 45 29 13 #0 doadump () at pcpu.h:246 246 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump () at pcpu.h:246 #1 0xc08853b7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:416 #2 0xc08856a9 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:579 #3 0xc0bb6c7c in trap_fatal (frame=0xe75f8bd8, eva=16) at /usr/src/sys/i386/i386/trap.c:938 #4 0xc0bb6f00 in trap_pfault (frame=0xe75f8bd8, usermode=0, eva=16) at /usr/src/sys/i386/i386/trap.c:851 #5 0xc0bb7905 in trap (frame=0xe75f8bd8) at /usr/src/sys/i386/i386/trap.c:533 #6 0xc0b9a0cb in calltrap () at /usr/src/sys/i386/i386/exception.s:165 #7 0xc061eecc in igb_rxeof (rxr=0xc7400e00, count=100) at /usr/src/sys/dev/e1000/if_igb.c:4074 #8 0xc061f409 in igb_msix_rx (arg=0xc7400e00) at /usr/src/sys/dev/e1000/if_igb.c:1455 #9 0xc085d3db in intr_event_execute_handlers (p=0xc71527f8, ie=0xc748d680) at /usr/src/sys/kern/kern_intr.c:1165 #10 0xc085e97b in ithread_loop (arg=0xc74b8970) at /usr/src/sys/kern/kern_intr.c:1178 #11 0xc085b121 in fork_exit (callout=0xc085e910 , arg=0xc74b8970, frame=0xe75f8d38) at /usr/src/sys/kern/kern_fork.c:843 #12 0xc0b9a140 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:270 (kgdb) using the patch provided by Pyun YongHyeon provided, allows the second port on the dual port nic to partially work, but the box still panics after a bit of traffic. If I just use igb1, it seems to be ok. But if I try and bring up igb0 and use it, 'bad things happen'. Panic with patch below panic: page fault GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd"... Unread portion of the kernel message buffer: Fatal trap 12: page fault while in kernel mode cpuid = 5; apic id = 05 fault virtual address = 0xc fault code = supervisor write, page not present instruction pointer = 0x20:0xc061f147 stack pointer = 0x28:0xe75f7c24 frame pointer = 0x28:0xe75f7c78 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, def32 1, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 12 (irq257: igb0) trap number = 12 panic: page fault cpuid = 5 Uptime: 8m27s Physical memory: 3560 MB Dumping 127 MB: 112 96 80 64 48 32 16 #0 doadump () at pcpu.h:246 246 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump () at pcpu.h:246 #1 0xc0885597 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:416 #2 0xc0885889 in panic (fmt=Variable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:579 #3 0xc0bb6e1c in trap_fatal (frame=0xe75f7be4, eva=12) at /usr/src/sys/i386/i386/trap.c:938 #4 0xc0bb70a0 in trap_pfault (frame=0xe75f7be4, usermode=0, eva=12) at /usr/src/sys/i386/i386/trap.c:851 #5 0xc0bb7aa5 in trap (frame=0xe75f7be4) at /usr/src/sys/i386/i386/trap.c:533 #6 0xc0b9a26b in calltrap () at /usr/src/sys/i386/i386/exception.s:165 #7 0xc061f147 in igb_rxeof (rxr=0xc7400e00, count=100) at /usr/src/sys/dev/e1000/if_igb.c:4130 #8 0xc061f679 in igb_msix_rx (arg=0xc7400e00) at /usr/src/sys/dev/e1000/if_igb.c:1456 #9 0xc085d5bb in intr_event_execute_handlers (p=0xc71527f8, ie=0xc748d680) at /usr/src/sys/kern/kern_intr.c:1165 #10 0xc085eb5b in ithread_loop (arg=0xc74b8960) at /usr/src/sys/kern/kern_intr.c:1178 #11 0xc085b301 in fork_exit (callout=0xc085eaf0 , arg=0xc74b8960, frame=0xe75f7d38) at /usr/src/sys/kern/kern_fork.c:843 #12 0xc0b9a2e0 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:270 (kgdb) >_______________________________________________ >freebsd-current@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-current >To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" -------------------------------------------------------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet since 1994 www.sentex.net Cambridge, Ontario Canada www.sentex.net/mike