From owner-freebsd-bugs@FreeBSD.ORG Tue May 1 20:50:11 2012 Return-Path: Delivered-To: freebsd-bugs@hub.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 4DCCF106566B for ; Tue, 1 May 2012 20:50:11 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 274968FC0A for ; Tue, 1 May 2012 20:50:11 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q41KoBEL081309 for ; Tue, 1 May 2012 20:50:11 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q41KoALM081308; Tue, 1 May 2012 20:50:10 GMT (envelope-from gnats) Resent-Date: Tue, 1 May 2012 20:50:10 GMT Resent-Message-Id: <201205012050.q41KoALM081308@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, J David Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 023E71065804 for ; Tue, 1 May 2012 20:42:18 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from red.freebsd.org (red.freebsd.org [IPv6:2001:4f8:fff6::22]) by mx1.freebsd.org (Postfix) with ESMTP id D6A248FC0A for ; Tue, 1 May 2012 20:42:17 +0000 (UTC) Received: from red.freebsd.org (localhost [127.0.0.1]) by red.freebsd.org (8.14.4/8.14.4) with ESMTP id q41KgHwG034741 for ; Tue, 1 May 2012 20:42:17 GMT (envelope-from nobody@red.freebsd.org) Received: (from nobody@localhost) by red.freebsd.org (8.14.4/8.14.4/Submit) id q41KgHN2034740; Tue, 1 May 2012 20:42:17 GMT (envelope-from nobody) Message-Id: <201205012042.q41KgHN2034740@red.freebsd.org> Date: Tue, 1 May 2012 20:42:17 GMT From: J David To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.1 Cc: Subject: kern/167500: Kernel panics in em driver X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 May 2012 20:50:11 -0000 >Number: 167500 >Category: kern >Synopsis: Kernel panics in em driver >Confidential: no >Severity: serious >Priority: high >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Tue May 01 20:50:10 UTC 2012 >Closed-Date: >Last-Modified: >Originator: J David >Release: 8.3-STABLE r234781 >Organization: >Environment: FreeBSD r1 8.3-STABLE FreeBSD 8.3-STABLE #5 r234781: Sun Apr 29 21:28:10 UTC 2012 root@r1:/data/r1/freebsd/obj/data/r1/freebsd/8-STABLE/sys/ROUTER64 amd64 >Description: We experience frequent panics under heavy network load with FreeBSD 8.3 that appear to be related to the em driver: Fatal trap 9: general protection fault while in kernel mode cpuid = 1; apic id = 01 instruction pointer = 0x20:0xffffffff806cd1d5 stack pointer = 0x28:0xffffff80000f1450 frame pointer = 0x28:0xffffff80000f1470 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 0 (em1 que) trap number = 9 panic: general protection fault cpuid = 1 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 panic() at panic+0x183 trap_fatal() at trap_fatal+0x290 trap() at trap+0x105 calltrap() at calltrap+0x8 --- trap 0x9, rip = 0xffffffff806cd1d5, rsp = 0xffffff80000f1450, rbp = 0xffffff80000f1470 --- m_freem() at m_freem+0x25 em_txeof() at em_txeof+0x164 em_start_locked() at em_start_locked+0x718 em_start() at em_start+0x5c if_transmit() at if_transmit+0xea lagg_start() at lagg_start+0x17b if_transmit() at if_transmit+0xea vlan_transmit() at vlan_transmit+0x11d ether_output_frame() at ether_output_frame+0x33 ether_output() at ether_output+0x518 ip_fastforward() at ip_fastforward+0x4a5 ether_demux() at ether_demux+0x198 ether_input() at ether_input+0x197 ether_demux() at ether_demux+0x6f ether_input() at ether_input+0x197 em_rxeof() at em_rxeof+0x1c7 em_handle_que() at em_handle_que+0x52 taskqueue_run_locked() at taskqueue_run_locked+0x85 taskqueue_thread_loop() at taskqueue_thread_loop+0x4e fork_exit() at fork_exit+0x11f fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000f1d00, rbp = 0 --- Uptime: 9h55m2s Cannot dump. Device not defined or unavailable. Automatic reboot in 15 seconds - press a key on the console to abort --> Press a key on the console to reboot, em2: discard frame w/o packet header Fatal trap 9: general protection fault while in kernel mode cpuid = 1; apic id = 01 instruction pointer = 0x20:0xffffffff806cd1d5 stack pointer = 0x28:0xffffff800011ba50 frame pointer = 0x28:0xffffff800011ba70 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 0 (em2 que) trap number = 9 panic: general protection fault cpuid = 1 This machine is a pf firewall under a pretty heavy traffic load. The line "em2: discard frame w/o packet header" seems to be the most informative diagnostic, because all that interface does is pfsync to the another machine; it's under very light load compared to em0, em1, and em3. >How-To-Repeat: 1) Configure machine as FreeBSD OpenBGPd router/ pf firewall. 2) Punish it. 3) Wait. >Fix: >Release-Note: >Audit-Trail: >Unformatted: