From owner-freebsd-net@FreeBSD.ORG Tue May 1 01:48:20 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id AF4FC106566C for ; Tue, 1 May 2012 01:48:20 +0000 (UTC) (envelope-from jdavidlists@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 6A9068FC1A for ; Tue, 1 May 2012 01:48:20 +0000 (UTC) Received: by yhgm50 with SMTP id m50so2054867yhg.13 for ; Mon, 30 Apr 2012 18:48:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=0eFe1eJDfP4IVEruOMnNCfuTI8NyCGW3i0y1lOzQzpY=; b=pnMdZE3XKu4w9ahUd9S2nQwmdOABtmB/sHN0aO9vEthIgk/nd3PIiOAlRnd6DYZYui 8F0QwNeYAVQi0ezcB0D7xA0d7W5UqwOSg3fZLe/a0sovoAlSmNVuUznJSXi3F3BmbuDb Ek+iA3a8oz8awZqcLU67I7jARNVd4rVE8wEcGNYa7xzg8Kt7D6eW4PqvESl/kxO/gGaw xi3Jd9TwUMLgSNEiyQRqw2P/ZYk3GRC1z4IFqkz74qeBrAJgUl0rpIS0X8TwBWYmdejG ZmaF5QIa8Po02obbOXGQaPQDX3pSst/pqHbjF31OTvpvEXaB2oHshlDSnik8TTz0lwG0 0Vtg== MIME-Version: 1.0 Received: by 10.236.75.74 with SMTP id y50mr24727978yhd.42.1335836899723; Mon, 30 Apr 2012 18:48:19 -0700 (PDT) Sender: jdavidlists@gmail.com Received: by 10.236.30.164 with HTTP; Mon, 30 Apr 2012 18:48:19 -0700 (PDT) Date: Mon, 30 Apr 2012 21:48:19 -0400 X-Google-Sender-Auth: Gg-KxWCRt2CkADytS-KcqyBvldc Message-ID: From: J David To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: Panics in 8.3 with em & pf X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 01 May 2012 01:48:20 -0000 Hello, I have started getting frequent panics related to the em driver on 8.3-STABLE (about every 8 hours). Fatal trap 9: general protection fault while in kernel mode cpuid = 1; apic id = 01 instruction pointer = 0x20:0xffffffff806cd1d5 stack pointer = 0x28:0xffffff80000f1450 frame pointer = 0x28:0xffffff80000f1470 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 0 (em1 que) trap number = 9 panic: general protection fault cpuid = 1 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a kdb_backtrace() at kdb_backtrace+0x37 panic() at panic+0x183 trap_fatal() at trap_fatal+0x290 trap() at trap+0x105 calltrap() at calltrap+0x8 --- trap 0x9, rip = 0xffffffff806cd1d5, rsp = 0xffffff80000f1450, rbp = 0xffffff80000f1470 --- m_freem() at m_freem+0x25 em_txeof() at em_txeof+0x164 em_start_locked() at em_start_locked+0x718 em_start() at em_start+0x5c if_transmit() at if_transmit+0xea lagg_start() at lagg_start+0x17b if_transmit() at if_transmit+0xea vlan_transmit() at vlan_transmit+0x11d ether_output_frame() at ether_output_frame+0x33 ether_output() at ether_output+0x518 ip_fastforward() at ip_fastforward+0x4a5 ether_demux() at ether_demux+0x198 ether_input() at ether_input+0x197 ether_demux() at ether_demux+0x6f ether_input() at ether_input+0x197 em_rxeof() at em_rxeof+0x1c7 em_handle_que() at em_handle_que+0x52 taskqueue_run_locked() at taskqueue_run_locked+0x85 taskqueue_thread_loop() at taskqueue_thread_loop+0x4e fork_exit() at fork_exit+0x11f fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000f1d00, rbp = 0 --- Uptime: 9h55m2s Cannot dump. Device not defined or unavailable. Automatic reboot in 15 seconds - press a key on the console to abort --> Press a key on the console to reboot, em2: discard frame w/o packet header Fatal trap 9: general protection fault while in kernel mode cpuid = 1; apic id = 01 instruction pointer = 0x20:0xffffffff806cd1d5 stack pointer = 0x28:0xffffff800011ba50 frame pointer = 0x28:0xffffff800011ba70 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 0 (em2 que) trap number = 9 panic: general protection fault cpuid = 1 This machine is a pf firewall under a pretty heavy traffic load. The line "em2: discard frame w/o packet header" seems to be the most informative diagnostic, because all that interface does is pfsync to the another machine; it's under very light load compared to em0, em1, and em3. The uname is: FreeBSD r1 8.3-STABLE FreeBSD 8.3-STABLE #5 r234781: Sun Apr 29 21:28:10 UTC 2012 root@r1:/data/r1/freebsd/obj/data/r1/freebsd/8-STABLE/sys/ROUTER64 amd64 I don't know if it's relevant, but I have also noticed behavior on this machine that's been reported elsewhere -- heavy load on the "busy" interfaces makes it prone to neglect the less-busy ones, even though it is not bound by any resource I can see.. it's a dual core that runs ~25% CPU utilized at its busiest. I've tried to tune it to prevent this to no real effect so far. Could anyone point me in the right direction on this? Thanks!