From owner-freebsd-net@FreeBSD.ORG Tue Oct 9 20:36:41 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 61B4060F; Tue, 9 Oct 2012 20:36:41 +0000 (UTC) (envelope-from guy.helmer@gmail.com) Received: from mail-ia0-f182.google.com (mail-ia0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id 153458FC18; Tue, 9 Oct 2012 20:36:40 +0000 (UTC) Received: by mail-ia0-f182.google.com with SMTP id k10so1506377iag.13 for ; Tue, 09 Oct 2012 13:36:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to:x-mailer; bh=lJf2GEMPcpXKxuHRjeQjL2egP9L84RMY4PW8owFsxV4=; b=NYcI5qY4L6j62OQPPf9FQ0cSEi6kQxMRgKtaVdBB3h3sWK8tATWnW2jLY6yy2pZvb/ 2J7OCpSD05bB8eVbBDd3KHzsN73Z1tXLpOfFzDN86Ln0YfsTVShhYb6ZvhpvxiK1+0I1 RLP4UXBfQucjLBKdM3I6dSi5RhArv2lmIiqvv7f7uQ8XVzLHA5y/xggPJ3HVxYUqFnqu rDogcBRS9Xk9+tMefssd4d3fJTU8g6E02NYih9NCUpmJAK/enjr99zRs1OwtfgXBX9Kz 0FLMMk03O4Mo6ZwwnCNJZ34qUWfzjAT5NnBDiy8ClDFc+cv5e97pcTjAoB07bDfO2Qou +BXw== Received: by 10.50.150.167 with SMTP id uj7mr3100194igb.33.1349815000154; Tue, 09 Oct 2012 13:36:40 -0700 (PDT) Received: from guysmbp.dyn.palisadesys.com ([216.81.189.9]) by mx.google.com with ESMTPS id pp3sm10805308igb.8.2012.10.09.13.36.38 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 09 Oct 2012 13:36:39 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.1 \(1498\)) Subject: Re: 8.3: kernel panic in bpf.c catchpacket() From: Guy Helmer In-Reply-To: <4B5399BF-4EE0-4182-8297-3BB97C4AA884@gmail.com> Date: Tue, 9 Oct 2012 15:36:33 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: <59F9A36E-3DB2-4F6F-BB2A-A4C9DA76A70C@gmail.com> References: <4B5399BF-4EE0-4182-8297-3BB97C4AA884@gmail.com> To: FreeBSD Stable , freebsd-net@freebsd.org X-Mailer: Apple Mail (2.1498) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Oct 2012 20:36:41 -0000 On Oct 8, 2012, at 8:09 AM, Guy Helmer wrote: > I'm seeing a consistent new kernel panic in FreeBSD 8.3: >=20 > #0 doadump () at pcpu.h:224 > 224 __asm("movq %%gs:0,%0" : "=3Dr" (td)); > (kgdb) #0 doadump () at pcpu.h:224 > #1 0xffffffff804c82e0 in boot (howto=3D260) at = ../../../kern/kern_shutdown.c:441 > #2 0xffffffff804c8763 in panic (fmt=3D0x0) at = ../../../kern/kern_shutdown.c:614 > #3 0xffffffff8069f3cd in trap_fatal (frame=3D0xffffffff809ecfc0, = eva=3DVariable "eva" is not available.) > at ../../../amd64/amd64/trap.c:825 > #4 0xffffffff8069f701 in trap_pfault (frame=3D0xffffff800014a8b0, = usermode=3D0) > at ../../../amd64/amd64/trap.c:741 > #5 0xffffffff8069fadf in trap (frame=3D0xffffff800014a8b0) > at ../../../amd64/amd64/trap.c:478 > #6 0xffffffff806870f4 in calltrap () at = ../../../amd64/amd64/exception.S:228 > #7 0xffffffff8069d026 in bcopy () at = ../../../amd64/amd64/support.S:124 > #8 0xffffffff8056ea8e in catchpacket (d=3D0xffffff00aee2c600,=20 > pkt=3D0xffffff00253ac600 "", pktlen=3D1434, snaplen=3DVariable = "snaplen" is not available.) > at ../../../net/bpf.c:2005 > #9 0xffffffff8056f0e5 in bpf_mtap (bp=3D0xffffff0001be1780,=20 > m=3D0xffffff00253ac600) at ../../../net/bpf.c:1832 > #10 0xffffffff80579035 in ether_input (ifp=3D0xffffff0001b73800,=20 > m=3D0xffffff00253ac600) at ../../../net/if_ethersubr.c:635 > #11 0xffffffff802b694a in em_rxeof (rxr=3D0xffffff0001bca200, = count=3D99, done=3D0x0) > at ../../../dev/e1000/if_em.c:4404 > #12 0xffffffff802b6db8 in em_handle_que (context=3DVariable "context" = is not available.) > at ../../../dev/e1000/if_em.c:1494 > #13 0xffffffff80506de5 in taskqueue_run_locked = (queue=3D0xffffff0001bdd600) > at ../../../kern/subr_taskqueue.c:250 > #14 0xffffffff80506f7e in taskqueue_thread_loop (arg=3DVariable "arg" = is not available.) > at ../../../kern/subr_taskqueue.c:387 > #15 0xffffffff8049da2f in fork_exit ( > callout=3D0xffffffff80506f30 ,=20 > arg=3D0xffffff8000945768, frame=3D0xffffff800014ac50) > at ../../../kern/kern_fork.c:876 > #16 0xffffffff8068763e in fork_trampoline () > at ../../../amd64/amd64/exception.S:602 > #17 0x0000000000000000 in ?? () >=20 > (kgdb) frame 8 > #8 0xffffffff8056ea8e in catchpacket (d=3D0xffffff00aee2c600,=20 > pkt=3D0xffffff00253ac600 "", pktlen=3D1434, snaplen=3DVariable = "snaplen" is not available. > ) > at ../../../net/bpf.c:2005 > 2005 bpf_append_bytes(d, d->bd_sbuf, curlen, &hdr, = sizeof(hdr)); > (kgdb) list > 2000 bzero(&hdr, sizeof(hdr)); > 2001 hdr.bh_tstamp =3D *tv; > 2002 hdr.bh_datalen =3D pktlen; > 2003 hdr.bh_hdrlen =3D hdrlen; > 2004 hdr.bh_caplen =3D totlen - hdrlen; > 2005 bpf_append_bytes(d, d->bd_sbuf, curlen, &hdr, = sizeof(hdr)); > 2006=09 > 2007 /* > 2008 * Copy the packet data into the store buffer and update = its length. > 2009 */ > (kgdb) print *d > $2 =3D {bd_next =3D {le_next =3D 0x0, le_prev =3D 0xffffff0001be1790}, = bd_sbuf =3D 0x0,=20 > bd_hbuf =3D 0x0, bd_fbuf =3D 0xffffff8000eae000 "?OoP", bd_slen =3D = 0,=20 > bd_hlen =3D 0, bd_bufsize =3D 8388608, bd_bif =3D 0xffffff0001be1780,=20= > bd_rtout =3D 1, bd_rfilter =3D 0xffffff0001e89180, bd_wfilter =3D = 0x0,=20 > bd_bfilter =3D 0x0, bd_rcount =3D 4, bd_dcount =3D 0, bd_promisc =3D = 1 '\001',=20 > bd_state =3D 2 '\002', bd_immediate =3D 1 '\001', bd_hdrcmplt =3D 1,=20= > bd_direction =3D 0, bd_feedback =3D 0, bd_async =3D 0, bd_sig =3D 23,=20= > bd_sigio =3D 0x0, bd_sel =3D {si_tdlist =3D {tqh_first =3D 0x0,=20 > tqh_last =3D 0xffffff00aee2c690}, si_note =3D {kl_list =3D = {slh_first =3D 0x0},=20 > kl_lock =3D 0xffffffff80497980 ,=20 > kl_unlock =3D 0xffffffff80497950 ,=20 > kl_assert_locked =3D 0xffffffff80494630 = ,=20 > kl_assert_unlocked =3D 0xffffffff80494640 = ,=20 > kl_lockarg =3D 0xffffff00aee2c6d8}, si_mtx =3D = 0xffffff8000de5270},=20 > bd_mtx =3D {lock_object =3D {lo_name =3D 0xffffff0001a5fce0 "bpf",=20 > lo_flags =3D 16973824, lo_data =3D 0, lo_witness =3D 0x0},=20 > mtx_lock =3D 18446742974226712770}, bd_callout =3D {c_links =3D = {sle =3D { > sle_next =3D 0x0}, tqe =3D {tqe_next =3D 0x0,=20 > tqe_prev =3D 0xffffff80f69c0c00}}, c_time =3D 20424328,=20 > c_arg =3D 0xffffff00aee2c600, c_func =3D 0xffffffff8056b690 = ,=20 > c_lock =3D 0xffffff00aee2c6d8, c_flags =3D 2, c_cpu =3D 0}, = bd_label =3D 0x0,=20 > bd_fcount =3D 4, bd_pid =3D 95393, bd_locked =3D 0, bd_bufmode =3D 1, = bd_wcount =3D 0,=20 > bd_wfcount =3D 0, bd_wdcount =3D 0, bd_zcopy =3D 0, bd_compat32 =3D 0 = '\0'} > (kgdb)=20 >=20 > I'm not seeing how bd_sbuf would be NULL here. Any ideas? Since I've not had any replies, I hope nobody minds if I reply with more = information. This panic seems to be occasionally triggered now that my user land code = is changing the packet filter a while after the bpd device has been = opened and an initial packet filter was set (previously, my code did not = change the filter after it was initially set). I'm focusing on bpf_setf() since that seems to be the place that could = be tickling a problem, and I see that bpf_setf() calls reset_d(d) to = clear the hold buffer. I have manually verified that the BPFD lock is = held during the call to reset_d(), and the lock is held every other = place that the buffers are manipulated, so I haven't been able to find = any place that seems vulnerable to losing one of the bpf buffers. Still = searching, but any help would be appreciated. Guy