From owner-freebsd-stable@FreeBSD.ORG Sat Oct 23 09:35:08 2010 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ACD0C1065670 for ; Sat, 23 Oct 2010 09:35:08 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.cksoft.de (mail.cksoft.de [IPv6:2001:4068:10::3]) by mx1.freebsd.org (Postfix) with ESMTP id 270648FC18 for ; Sat, 23 Oct 2010 09:35:08 +0000 (UTC) Received: from localhost (amavis.fra.cksoft.de [192.168.74.71]) by mail.cksoft.de (Postfix) with ESMTP id 21DF941C7AD; Sat, 23 Oct 2010 11:35:07 +0200 (CEST) X-Virus-Scanned: amavisd-new at cksoft.de Received: from mail.cksoft.de ([192.168.74.103]) by localhost (amavis.fra.cksoft.de [192.168.74.71]) (amavisd-new, port 10024) with ESMTP id to-pb4ZRyzCb; Sat, 23 Oct 2010 11:35:05 +0200 (CEST) Received: by mail.cksoft.de (Postfix, from userid 66) id D80C141C7B6; Sat, 23 Oct 2010 11:35:05 +0200 (CEST) Received: from maildrop.int.zabbadoz.net (maildrop.int.zabbadoz.net [10.111.66.10]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.int.zabbadoz.net (Postfix) with ESMTP id C1F5E4448F3; Sat, 23 Oct 2010 09:33:20 +0000 (UTC) Date: Sat, 23 Oct 2010 09:33:20 +0000 (UTC) From: "Bjoern A. Zeeb" X-X-Sender: bz@maildrop.int.zabbadoz.net To: Mike Tancsa In-Reply-To: <201010230821.o9N8LVuR001382@lava.sentex.ca> Message-ID: <20101023091555.W66242@maildrop.int.zabbadoz.net> References: <201010221416.o9MEGSa0094817@lava.sentex.ca> <201010221425.o9MEPcWC094867@lava.sentex.ca> <201010221848.o9MIm7WF096197@lava.sentex.ca> <4CC1F3B8.3010302@bogus.com> <4CC225D3.1030502@ops-netman.net> <7.1.0.9.0.20101022210145.06fe25e8@sentex.net> <201010230159.o9N1xGGF098363@lava.sentex.ca> <201010230821.o9N8LVuR001382@lava.sentex.ca> X-OpenPGP-Key: 0x14003F198FEFA3E77207EE8D2B58B8F83CCF1842 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Randy Bush , stable@freebsd.org, Jack Vogel Subject: Re: repeating crashes with 8.1 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 23 Oct 2010 09:35:08 -0000 On Sat, 23 Oct 2010, Mike Tancsa wrote: Hi all, > At 12:41 AM 10/23/2010, Jack Vogel wrote: >> Odd, can you make any connection between this and the em complaints?? > > I dont think so. neither do I think so. The initial issue > This is on an igb nic and a different panic/behaviour. I > have the box sitting at the debugger prompt in the FreeBSD netperf cluster, > so hopefully someone can take a look and see what is the issue. Ok, and I seem to have another email telling me the machine;-) >> >>>>> Oct 22 02:06:02 i4 kernel: em1: discard frame w/o packet header >> >>>>> Oct 22 02:06:10 i4 kernel: em2: discard frame w/o packet header is if_em(4) receiving a packet and then passing it up in em_rxeof() by calling ifp->if_input ending up in sys/net/if_ethersubr.c:ether_input() and there's an early check on 1) that the interface is up, and 2) 580 /* 581 * Do consistency checks to verify assumptions 582 * made by code past this point. 583 */ 584 if ((m->m_flags & M_PKTHDR) == 0) { 585 if_printf(ifp, "discard frame w/o packet header\n"); 586 ifp->if_ierrors++; 587 m_freem(m); 588 return; 589 } So whatever mbuf if_em(4) is passing up here isn't setup correctly and it sounds like the mbuf chaining logic or receive ring handling could have a problem, though I doubt that em_refresh_mbufs() would be at fault ... but I'll leave that to Jack. For the following one, I hope to know the problem for it and am working on fixing this already. This is most likely more new-arp fallout and the last we currrently know of (despite some nd6 locking problems which will need to be addressed separately and are under discussion) >> Fatal trap 9: general protection fault while in kernel mode >> cpuid = 0; apic id = 00 >> instruction pointer = 0x20:0xffffffff80740a50 >> stack pointer = 0x28:0xffffff800005a890 >> frame pointer = 0x28:0xffffff800005a930 >> >> code segment = base 0x0, limit 0xfffff, type 0x1b >> = DPL 0, pres 1, long 1, def32 0, gran 1 >> processor eflags = interrupt enabled, resume, IOPL = 0 >> current process = 12 (swi4: clock) >> [thread pid 12 tid 100007 ] >> Stopped at in6_cksum+0x410: movzwl (%rsi),%r10d >> db> bt >> Tracing pid 12 tid 100007 td 0xffffff00025083e0 >> in6_cksum() at in6_cksum+0x410 >> icmp6_reflect() at icmp6_reflect+0x312 >> icmp6_error() at icmp6_error+0x1ec >> nd6_llinfo_timer() at nd6_llinfo_timer+0x208 >> softclock() at softclock+0x2a6 >> intr_event_execute_handlers() at intr_event_execute_handlers+0x66 >> ithread_loop() at ithread_loop+0xb2 >> fork_exit() at fork_exit+0x12a >> fork_trampoline() at fork_trampoline+0xe >> --- trap 0, rip = 0, rsp = 0xffffff800005ad30, rbp = 0 --- >> db> Mike, as you found that in a lab setup and I was never able to reproduce any of the reports before, I'll send you a patch over the weekend and we'll see if you can still reproduce it after patching. /bz -- Bjoern A. Zeeb Welcome a new stage of life. Going to jail sucks -- All my daemons like it! http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/jails.html