From owner-freebsd-net@freebsd.org Mon Jun 24 20:01:27 2019 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9DC2A15D7C0A; Mon, 24 Jun 2019 20:01:27 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward105p.mail.yandex.net (forward105p.mail.yandex.net [77.88.28.108]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8280385E43; Mon, 24 Jun 2019 20:01:23 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from mxback13g.mail.yandex.net (mxback13g.mail.yandex.net [IPv6:2a02:6b8:0:1472:2741:0:8b7:92]) by forward105p.mail.yandex.net (Yandex) with ESMTP id 1E5CB4D411C8; Mon, 24 Jun 2019 23:01:15 +0300 (MSK) Received: from smtp1j.mail.yandex.net (smtp1j.mail.yandex.net [2a02:6b8:0:801::ab]) by mxback13g.mail.yandex.net (nwsmtp/Yandex) with ESMTP id DdDjq1FlEH-1E7Kf5xs; Mon, 24 Jun 2019 23:01:15 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1561406475; bh=9TLcODuoPjBkyrRUY+Es1r9o8VAwBPPlPDcBYrgl3/Y=; h=In-Reply-To:From:Date:References:To:Subject:Message-ID; b=a84Hp/GAq9M6hNRyAsuAnJhcVJGKN9j+zA/o4+u2sTrvY9JWbi0LhKpz1gZhPvZnb WsEqtT/vbdfsnXDbiPX82DnQVFbGjY6K0pXrEeavB6J1AF5EjRHmeDY7p72h0Me6Vm 0pLrcd8FKT1fmVOaVgHXv6fwT2kao3OjyVSNNHPw= Received: by smtp1j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id h5li9zDiEN-1Ei0WYeh; Mon, 24 Jun 2019 23:01:14 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client certificate not present) Subject: Re: ng_snd_item: Panic? To: "freebsd-current@freebsd.org" , freebsd-net@FreeBSD.org References: <20190624183200.hu4vzocjsopjsnnz@ler-imac.local> From: "Andrey V. Elsukov" Openpgp: preference=signencrypt Autocrypt: addr=bu7cher@yandex.ru; prefer-encrypt=mutual; keydata= mQENBEwBF1kBCADB9sXFhBEUy8qQ4X63Y8eBatYMHGEFWN9ypS5lI3RE6qQW2EYbxNk7qUC5 21YIIS1mMFVBEfvR7J9uc7yaYgFCEb6Sce1RSO4ULN2mRKGHP3/Sl0ijZEjWHV91hY1YTHEF ZW/0GYinDf56sYpDDehaBF5wkWIo1+QK5nmj3vl0DIDCMNd7QEiWpyLVwECgLX2eOAXByT8B bCqVhJGcG6iFP7/B9Ll6uX5gb8thM9LM+ibwErDBVDGiOgvfxqidab7fdkh893IBCXa82H9N CNwnEtcgzh+BSKK5BgvPohFMgRwjti37TSxwLu63QejRGbZWSz3OK3jMOoF63tCgn7FvABEB AAG0JUFuZHJleSBWLiBFbHN1a292IDxidTdjaGVyQHlhbmRleC5ydT6JATgEEwECACIFAkwB F1kCGwMGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEAHF6gQQyKF6qmYIAI6ekfm1VA4T vqankI1ISE6ku4jV7UlpIQlEbE7/8n3Zd6teJ+pGOQhN5qk8QE7utdPdbktAzi+x7LIJVzUw 4TywZLXGrkP7VKYkfg6oyCGyzITghefQeJtr2TN4hYCkzPWpylkue8MtmqfZv/6royqwTbN+ +E09FQNvTgRUYJYTeQ1qOsxNRycwvw3dr2rOfuxShbzaHBB1pBIjGrMg8fC5pd65ACH5zuFV A0CoTNGMDrEZSfBkTW604UUHFFXeCoC3dwDZRKOWJ3GmMXns65Ai5YkA63BSHEE1Qle3VBhd cG1w0CB5FBV3pB27UVnf0jEbysrDqW4qN7XMRFSWNAy5AQ0ETAEXWQEIAJ2p6l9LBoqdH/0J PEFDY2t2gTvAuzz+8zs3R03dFuHcNbOwjvWCG0aOmVpAzkRa8egn5JB4sZaFUtKPYJEQ1Iu+ LUBwgvtXf4vWpzC67zs2dDuiW4LamH5p6xkTD61aHR7mCB3bg2TUjrDWn2Jt44cvoYxj3dz4 S49U1rc9ZPgD5axCNv45j72tggWlZvpefThP7xT1OlNTUqye2gAwQravXpZkl5JG4eOqJVIU X316iE3qso0iXRUtO7OseBf0PiVmk+wCahdreHOeOxK5jMhYkPKVn7z1sZiB7W2H2TojbmcK HZC22sz7Z/H36Lhg1+/RCnGzdEcjGc8oFHXHCxUAEQEAAYkBHwQYAQIACQUCTAEXWQIbDAAK CRABxeoEEMihegkYCAC3ivGYNe2taNm/4Nx5GPdzuaAJGKWksV+w9mo7dQvU+NmI2az5w8vw 98OmX7G0OV9snxMW+6cyNqBrVFTu33VVNzz9pnqNCHxGvj5dL5ltP160JV2zw2bUwJBYsgYQ WfyJJIM7l3gv5ZS3DGqaGIm9gOK1ANxfrR5PgPzvI9VxDhlr2juEVMZYAqPLEJe+SSxbwLoz BcFCNdDAyXcaAzXsx/E02YWm1hIWNRxanAe7Vlg7OL+gvLpdtrYCMg28PNqKNyrQ87LQ49O9 50IIZDOtNFeR0FGucjcLPdS9PiEqCoH7/waJxWp6ydJ+g4OYRBYNM0EmMgy1N85JJrV1mi5i Message-ID: Date: Mon, 24 Jun 2019 23:01:07 +0300 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:60.0) Gecko/20100101 Thunderbird/60.7.2 MIME-Version: 1.0 In-Reply-To: <20190624183200.hu4vzocjsopjsnnz@ler-imac.local> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="UTKnhMvwuBEUCggdqxEeeAsYT7ZHhXOTY" X-Rspamd-Queue-Id: 8280385E43 X-Spamd-Bar: --------- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yandex.ru header.s=mail header.b=a84Hp/GA; dmarc=pass (policy=none) header.from=yandex.ru; spf=pass (mx1.freebsd.org: domain of bu7cher@yandex.ru designates 77.88.28.108 as permitted sender) smtp.mailfrom=bu7cher@yandex.ru X-Spamd-Result: default: False [-9.75 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEMAIL_FROM(0.00)[yandex.ru]; R_SPF_ALLOW(-0.20)[+ip4:77.88.0.0/18]; HAS_ATTACHMENT(0.00)[]; RCVD_COUNT_THREE(0.00)[4]; DKIM_TRACE(0.00)[yandex.ru:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yandex.ru,none]; MX_GOOD(-0.01)[cached: mx.yandex.ru]; SIGNED_PGP(-2.00)[]; NEURAL_HAM_SHORT(-1.00)[-0.997,0]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+]; FREEMAIL_ENVFROM(0.00)[yandex.ru]; ASN(0.00)[asn:13238, ipnet:77.88.0.0/18, country:RU]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[108.28.88.77.list.dnswl.org : 127.0.5.1]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[yandex.ru:s=mail]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; SUBJECT_ENDS_QUESTION(1.00)[]; DWL_DNSWL_LOW(-1.00)[yandex.ru.dwl.dnswl.org : 127.0.5.1]; IP_SCORE(-3.55)[ip: (-9.44), ipnet: 77.88.0.0/18(-4.54), asn: 13238(-3.76), country: RU(0.01)]; RWL_MAILSPIKE_POSSIBLE(0.00)[108.28.88.77.rep.mailspike.net : 127.0.0.17] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jun 2019 20:01:27 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --UTKnhMvwuBEUCggdqxEeeAsYT7ZHhXOTY Content-Type: multipart/mixed; boundary="QYmkI9qn7SMzT7EC9IbN65mFxA008IUXa"; protected-headers="v1" From: "Andrey V. Elsukov" To: "freebsd-current@freebsd.org" , freebsd-net@FreeBSD.org Message-ID: Subject: Re: ng_snd_item: Panic? References: <20190624183200.hu4vzocjsopjsnnz@ler-imac.local> In-Reply-To: <20190624183200.hu4vzocjsopjsnnz@ler-imac.local> --QYmkI9qn7SMzT7EC9IbN65mFxA008IUXa Content-Type: text/plain; charset=UTF-8 Content-Language: ru Content-Transfer-Encoding: quoted-printable 24.06.2019 21:32, Larry Rosenman =D0=BF=D0=B8=D1=88=D0=B5=D1=82: > Got 2 of these today, and I have cores.... > Ideas? > r349200. >=20 > Unread portion of the kernel message buffer: > panic: ng_snd_item: 42 !=3D 1414 > cpuid =3D 10 > time =3D 1561382494 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe012= 628d400 > vpanic() at vpanic+0x19d/frame 0xfffffe012628d450 > panic() at panic+0x43/frame 0xfffffe012628d4b0 > ng_snd_item() at ng_snd_item+0x477/frame 0xfffffe012628d4f0 > ng_ether_output() at ng_ether_output+0x5e/frame 0xfffffe012628d520 > ether_output() at ether_output+0x473/frame 0xfffffe012628d5c0 > arpintr() at arpintr+0xfe3/frame 0xfffffe012628d780 > netisr_dispatch_src() at netisr_dispatch_src+0x89/frame 0xfffffe012628d= 7f0 > ether_demux() at ether_demux+0x137/frame 0xfffffe012628d820 > ng_ether_rcv_upper() at ng_ether_rcv_upper+0x95/frame 0xfffffe012628d84= 0 > ng_apply_item() at ng_apply_item+0xf1/frame 0xfffffe012628d8c0 > ng_snd_item() at ng_snd_item+0x2ab/frame 0xfffffe012628d900 > ng_apply_item() at ng_apply_item+0xf1/frame 0xfffffe012628d980 > ng_snd_item() at ng_snd_item+0x2ab/frame 0xfffffe012628d9c0 > ng_ether_input() at ng_ether_input+0x4c/frame 0xfffffe012628d9f0 > ether_nh_input() at ether_nh_input+0x2cd/frame 0xfffffe012628da40 > netisr_dispatch_src() at netisr_dispatch_src+0x89/frame 0xfffffe012628d= ab0 > ether_input() at ether_input+0x48/frame 0xfffffe012628dad0 > bce_intr() at bce_intr+0x697/frame 0xfffffe012628db50 > ithread_loop() at ithread_loop+0x187/frame 0xfffffe012628dbb0 > fork_exit() at fork_exit+0x84/frame 0xfffffe012628dbf0 > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe012628dbf0 > --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- > Uptime: 4d18h45m34s > Dumping 24921 out of 131026 MB:..1%..11%..21%..31%..41%..51%..61%..71%.= =2E81%..91% >=20 > #5 0xffffffff828ee5b7 in ng_snd_item (item=3D0xfffff8021e3b4d80, flags= =3D0) > at /usr/src/sys/netgraph/ng_base.c:2252 It looks like you use some netgraph based ethernet interface. The system got received ARP request and is going to send the reply, but somehow mbuf with this ARP request has initialized m_next pointer, thus it is considered as a chain of mbufs. in_arpinput() reuses received mbuf to construct the reply, but it doesn't check that an mbut is a chain. It just sets m_len and sends it. Then since you have INVARIANTS in your kernel, the netgraph code check the actual length of the chain, and it doesn't match to m_len. It panics.= > #6 0xffffffff82900c2e in ng_ether_output (ifp=3D,=20 > mp=3D0xfffffe012628d578) at /usr/src/sys/netgraph/ng_ether.c:294 > #7 0xffffffff805b1e43 in ether_output (ifp=3D,=20 > m=3D0xfffff81f59eefb00, dst=3D0xfffffe012628d740, ro=3D) > at /usr/src/sys/net/if_ethersubr.c:430 > #8 0xffffffff805cb3e3 in in_arpinput (m=3D) > at /usr/src/sys/netinet/if_ether.c:1152 > #9 arpintr (m=3D0xfffff81f59eefb00) at /usr/src/sys/netinet/if_ether.c= :749 > #10 0xffffffff805bcf89 in netisr_dispatch_src (proto=3D4,=20 > source=3D, m=3D) at /usr/src/sys/net/ne= tisr.c:1123 > #11 0xffffffff805b22d7 in ether_demux (ifp=3D0xfffff8012c902000,=20 > m=3D) at /usr/src/sys/net/if_ethersubr.c:913 > #12 0xffffffff82901045 in ng_ether_rcv_upper (hook=3D,=20 > item=3D) at /usr/src/sys/netgraph/ng_ether.c:741 > #13 0xffffffff828ee6e1 in ng_apply_item (node=3D0xfffff81054f43400,=20 > item=3D0xfffff8021e3b4d80, rw=3D0) at /usr/src/sys/netgraph/ng_base= =2Ec:2403 > #14 0xffffffff828ee3eb in ng_snd_item (item=3D0xfffff8021e3b4d80, flags= =3D0) > at /usr/src/sys/netgraph/ng_base.c:2320 > #15 0xffffffff828ee6e1 in ng_apply_item (node=3D0xfffff8012c2d3e00,=20 > item=3D0xfffff8021e3b4d80, rw=3D0) at /usr/src/sys/netgraph/ng_base= =2Ec:2403 > #16 0xffffffff828ee3eb in ng_snd_item (item=3D0xfffff8021e3b4d80, flags= =3D0) > at /usr/src/sys/netgraph/ng_base.c:2320 > #17 0xffffffff82900cbc in ng_ether_input (ifp=3D,=20 > mp=3D0xfffffe012628da18) at /usr/src/sys/netgraph/ng_ether.c:255 > #18 0xffffffff805b34fd in ether_input_internal (ifp=3D0xfffff8012c90200= 0,=20 > m=3D0xfffff81f59eefb00) at /usr/src/sys/net/if_ethersubr.c:654 > #19 ether_nh_input (m=3D) at /usr/src/sys/net/if_ethersu= br.c:735 > #20 0xffffffff805bcf89 in netisr_dispatch_src (proto=3D5,=20 > source=3D, m=3D) at /usr/src/sys/net/ne= tisr.c:1123 > #21 0xffffffff805b26f8 in ether_input (ifp=3D0xfffff8012c902000, m=3D0x= 0) > at /usr/src/sys/net/if_ethersubr.c:823 > #22 0xffffffff8273c7f7 in bce_rx_intr (sc=3D) > at /usr/src/sys/dev/bce/if_bce.c:6848 > #23 bce_intr (xsc=3D0xfffffe01665c2000) at /usr/src/sys/dev/bce/if_bce.= c:8017 > #24 0xffffffff8047e0e7 in intr_event_execute_handlers (p=3D,=20 > ie=3D) at /usr/src/sys/kern/kern_intr.c:1148 > #25 ithread_execute_handlers (p=3D, ie=3D= ) > at /usr/src/sys/kern/kern_intr.c:1161 > #26 ithread_loop (arg=3D) at /usr/src/sys/kern/kern_intr= =2Ec:1241 > #27 0xffffffff8047ac74 in fork_exit ( > callout=3D0xffffffff8047df60 , arg=3D0xfffff8012c8831= 00,=20 > frame=3D0xfffffe012628dc00) at /usr/src/sys/kern/kern_fork.c:1056 > #28 > (kgdb)=20 > --=20 WBR, Andrey V. Elsukov --QYmkI9qn7SMzT7EC9IbN65mFxA008IUXa-- --UTKnhMvwuBEUCggdqxEeeAsYT7ZHhXOTY Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEE5lkeG0HaFRbwybwAAcXqBBDIoXoFAl0RLAkACgkQAcXqBBDI oXoG3Af/W4d4t8a7plih7X5QIOyXUzm6MqfbWXQ6/fdnS2pKmldjpjMUrbiGxoRv Nll67121BUHlTnKxUt+mfh97oYyFX8M9fJhlD1FaDIDEAGoyIoxOZuU0WSMuLLMA t+Xt0Lq4PVQbNVu/9CIDqNbkAANGeLmwoAPUx8C+nSnPMVx/C6aFARKnYAuB6U/f ad6AZL5ysdmxdVwx7bbUvrlWnb8U7EGOEEi9ZBY3Qr2AIpCB1B4tFqZCbBZ8oifz gHwougvvDl6k2ns/Fa6r8GlHRhT3KDhgn6ZroR089zY5IqjCUXlpapFpCL9UD1j3 qwLRZwcU/G7Bxa5cv3AADzM1LElYiQ== =MbY3 -----END PGP SIGNATURE----- --UTKnhMvwuBEUCggdqxEeeAsYT7ZHhXOTY--