From owner-freebsd-net@freebsd.org Tue Jun 25 12:59:07 2019 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BA56015CBED5; Tue, 25 Jun 2019 12:59:07 +0000 (UTC) (envelope-from ler@lerctr.org) Received: from thebighonker.lerctr.org (ns-b.lerctr.org [IPv6:2001:470:1f0f:3ad::53:2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.lerctr.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CD11E89E7E; Tue, 25 Jun 2019 12:59:06 +0000 (UTC) (envelope-from ler@lerctr.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=ler2019; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date: Content-Transfer-Encoding:Content-Type:MIME-Version:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=3GY08dUA9q3ZW4U3qX5ndPow0LXp7bn+OyC8b9gsMwg=; b=r6B9dron/b648lNtiptrQOcTxd a0gPmuVrdmVEOtVamTiy5+vRQgbYUe9ynUi5e3AKXv81q3z7Qqu4krq5wUhWK+KD3gU3tLl69Oi4J VRA3RCtJXrGqloFNysvTSn7MZ4yNkZ/UUH3UYA2BNGUAXCeFxFe0j65JOyG7AgXly2fcExIZa3qhI ZVvHjswnOVMHylRWYQRR8eLnbc3zG9Jh4P3gnRpw8Q7M/m6alI+92F7AUhzdeFnxrrGMBt1Knx97s kiyTO05ZCAG+D2ASSaMW1LaHxcbuTaE6I3Bb+ICVBxriA5kKXEtZmxyOeiTPiL80G5fv7O1fni/la PkjNhEBg==; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:bb:dcff:fe50:d900]:26572 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92 (FreeBSD)) (envelope-from ) id 1hfl2N-000F81-0h; Tue, 25 Jun 2019 07:59:03 -0500 Received: from 2600:1700:210:b180:618b:e19b:4ff6:4b3e by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Tue, 25 Jun 2019 07:59:02 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Tue, 25 Jun 2019 07:59:02 -0500 From: Larry Rosenman To: "Andrey V. Elsukov" Cc: freebsd-current@freebsd.org, freebsd-net@freebsd.org Subject: Re: ng_snd_item: Panic? In-Reply-To: References: <20190624183200.hu4vzocjsopjsnnz@ler-imac.local> Message-ID: X-Sender: ler@lerctr.org User-Agent: Roundcube Webmail/1.3.9 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jun 2019 12:59:08 -0000 On 06/25/2019 4:18 am, Andrey V. Elsukov wrote: > On 24.06.2019 23:10, Larry Rosenman wrote: >>>> #5  0xffffffff828ee5b7 in ng_snd_item (item=0xfffff8021e3b4d80, >>>> flags=0) >>>>     at /usr/src/sys/netgraph/ng_base.c:2252 >>> >>> It looks like you use some netgraph based ethernet interface. >>> The system got received ARP request and is going to send the reply, >>> but somehow mbuf with this ARP request has initialized m_next >>> pointer, >>> thus it is considered as a chain of mbufs. >>> >>> in_arpinput() reuses received mbuf to construct the reply, but it >>> doesn't check that an mbut is a chain. It just sets m_len and sends >>> it. >>> Then since you have INVARIANTS in your kernel, the netgraph code >>> check >>> the actual length of the chain, and it doesn't match to m_len. It >>> panics. >> >> >> so, is this a bug?  Timing race? Other? > > I think we should determine that my assumption is correct :) > Can you show the output of the following commands from the kgdb for > this > core? > > (kgdb) f 7 > (kgdb) p *m > (kgdb) p *m->m_next (kgdb) fr 7 #7 0xffffffff805b1e43 in ether_output (ifp=, m=0xfffff81f59eefb00, dst=0xfffffe012628d740, ro=) at /usr/src/sys/net/if_ethersubr.c:430 430 if ((error = (*ng_ether_output_p)(ifp, &m)) != 0) { (kgdb) p *m $1 = {{m_next = 0xfffff81086c9dd00, m_slist = {sle_next = 0xfffff81086c9dd00}, m_stailq = {stqe_next = 0xfffff81086c9dd00}}, {m_nextpkt = 0x0, m_slistpkt = {sle_next = 0x0}, m_stailqpkt = {stqe_next = 0x0}}, m_data = 0xfffff81f59eefb72 "\004\311\331c\"\207\244\272\333)f\225\b\006", m_len = 42, m_type = 1, m_flags = 2, {{m_pkthdr = {{snd_tag = 0x0, rcvif = 0x0}, tags = {slh_first = 0xfffff8104c530d60}, len = 42, flowid = 0, csum_flags = 0, fibnum = 0, numa_domain = 255 '\377', rsstype = 0 '\000', {rcv_tstmp = 0, {l2hlen = 0 '\000', l3hlen = 0 '\000', l4hlen = 0 '\000', l5hlen = 0 '\000', spare = 0}}, PH_per = {eight = "\000\000\000\000\000\000\000", sixteen = {0, 0, 0, 0}, thirtytwo = {0, 0}, sixtyfour = { 0}, unintptr = {0}, ptr = 0x0}, PH_loc = {eight = "\000\000\000\000\000\000\000", sixteen = {0, 0, 0, 0}, thirtytwo = {0, 0}, sixtyfour = {0}, unintptr = {0}, ptr = 0x0}}, {m_ext = {{ext_count = 3735929054, ext_cnt = 0xdeadc0dedeadc0de}, ext_buf = 0x200000207 , ext_size = 99483648, ext_type = 0, ext_flags = 0, ext_free = 0x872263d9c9040000, ext_arg1 = 0x608956629dbbaa4, ext_arg2 = 0x200040600080100}, m_pktdat = 0xfffff81f59eefb58 "\336\300\255\336\336\300\255\336\a\002"}}, m_dat = 0xfffff81f59eefb20 ""}} (kgdb) p *m->m_next $2 = {{m_next = 0x0, m_slist = {sle_next = 0x0}, m_stailq = {stqe_next = 0x0}}, {m_nextpkt = 0x0, m_slistpkt = {sle_next = 0x0}, m_stailqpkt = {stqe_next = 0x0}}, m_data = 0xfffff8002144b800 "\004\002\b\n\001tWg\001tWg\001\003\003\006\255\336zڭ\336zڭ\336zڭ\336zڭ\336zڭ\336zڭ\336zڭ\336zڭ\336zڭ\336zڭ\336zڭ\336", , m_len = 1372, m_type = 1, m_flags = 1, {{m_pkthdr = {{snd_tag = 0x0, rcvif = 0x0}, tags = {slh_first = 0x0}, len = 0, flowid = 0, csum_flags = 0, fibnum = 0, numa_domain = 0 '\000', rsstype = 0 '\000', {rcv_tstmp = 0, {l2hlen = 0 '\000', l3hlen = 0 '\000', l4hlen = 0 '\000', l5hlen = 0 '\000', spare = 0}}, PH_per = { eight = "\000\000\000\000\000\000\000", sixteen = {0, 0, 0, 0}, thirtytwo = {0, 0}, sixtyfour = {0}, unintptr = {0}, ptr = 0x0}, PH_loc = {eight = "\000\000\000\000\000\000\000", sixteen = {0, 0, 0, 0}, thirtytwo = {0, 0}, sixtyfour = {0}, unintptr = {0}, ptr = 0x0}}, {m_ext = {{ext_count = 1, ext_cnt = 0xdeadc0de00000001}, ext_buf = 0xfffff8002144b800 "\004\002\b\n\001tWg\001tWg\001\003\003\006\255\336zڭ\336zڭ\336zڭ\336zڭ\336zڭ\336zڭ\336zڭ\336zڭ\336zڭ\336zڭ\336zڭ\336", , ext_size = 2048, ext_type = 6, ext_flags = 1, ext_free = 0x0, ext_arg1 = 0x0, ext_arg2 = 0x0}, m_pktdat = 0xfffff81086c9dd58 "\001"}}, m_dat = 0xfffff81086c9dd20 ""}} (kgdb) -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106