From owner-freebsd-current@freebsd.org Tue Aug 20 14:16:09 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E1196D3278 for ; Tue, 20 Aug 2019 14:16:09 +0000 (UTC) (envelope-from rrs@netflix.com) Received: from mail-qk1-x736.google.com (mail-qk1-x736.google.com [IPv6:2607:f8b0:4864:20::736]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46CXs46lCfz4PPs for ; Tue, 20 Aug 2019 14:16:08 +0000 (UTC) (envelope-from rrs@netflix.com) Received: by mail-qk1-x736.google.com with SMTP id 201so4592496qkm.9 for ; Tue, 20 Aug 2019 07:16:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=QmXM1xxWtmXFyYYoa2SJ+R7Z/3oU0M8gH47hYOiQvhU=; b=AWHDXbrc8k1lOhT1141rdezvEEFEjG2oAJK4CorwZVwGVTiD5jcwmnJRXeTHek2OWo 6wUhMWjCLFCWifHE76wynxcKlgluxidyOF8sIed+XKJ+mIF1TPmbheEt8WMtPz9BNuiy 7+TciU+N5kqIzlLrHmd5C4N+9pJ50P2v0AmmqIk+68oRwf6u+PJzbTpq1AjHxuRmK4lN +mFb7AODfosx1G2jH9BvGFGV4KJtlNyFE+QQZp9Wzmku4yy1Zl78MYPCMRf3WXKQlIPK TreXWUZxvsqO3SeIK4iPlFTJPIsflcgzxgk6159QjZjBsWa/VagCzPMDO/Fc2Hij1K/9 i3yQ== X-Gm-Message-State: APjAAAW3VgRlAhv0PV+KzU8+JVLTtPKQwalUWFz2dKwTyGyAkVZ4H3rY 0xaxbtSosAI0sPpRfnDabGb3qw== X-Google-Smtp-Source: APXvYqxuxvAM2QvZLt7I1R+YtsDrtn018cAejdg1ikumZaroo99EkJMj+R/rxcCW+Ix27BQqcbFhqg== X-Received: by 2002:a37:79c4:: with SMTP id u187mr6999514qkc.91.1566310567322; Tue, 20 Aug 2019 07:16:07 -0700 (PDT) Received: from ?IPv6:2607:fb10:7061:7fd::3ebf? ([2607:fb10:7061:7fd::3ebf]) by smtp.gmail.com with ESMTPSA id r4sm10196971qta.93.2019.08.20.07.16.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 20 Aug 2019 07:16:06 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Re: panic... r350849M panic: ng_snd_item: 42 != 1414 From: Randall Stewart In-Reply-To: Cc: Freebsd current Content-Transfer-Encoding: quoted-printable Message-Id: <617CF250-5E1B-4667-886C-95F33387F19D@netflix.com> References: To: Larry Rosenman X-Mailer: Apple Mail (2.3445.9.1) X-Rspamd-Queue-Id: 46CXs46lCfz4PPs X-Spamd-Bar: ---------- X-Spamd-Result: default: False [-10.94 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(0.00)[+ip6:2607:f8b0:4000::/36]; MV_CASE(0.50)[]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; SUBJECT_HAS_EXCLAIM(0.00)[]; DKIM_TRACE(0.00)[netflix.com:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.99)[-0.993,0]; DMARC_POLICY_ALLOW(0.00)[netflix.com,reject]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(-2.94)[ip: (-9.37), ipnet: 2607:f8b0::/32(-2.93), asn: 15169(-2.37), country: US(-0.05)]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; RSPAMD_URIBL(4.50)[lerctr.org]; R_DKIM_ALLOW(0.00)[netflix.com:s=google]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; BAD_REP_POLICIES(0.10)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; WHITELIST_DMARC(-7.00)[netflix.com:D:+]; RCVD_IN_DNSWL_NONE(0.00)[6.3.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; WHITELIST_SPF_DKIM(-3.00)[netflix.com:d:+,netflix.com:s:+]; RCVD_TLS_ALL(0.00)[] X-Mailman-Approved-At: Sun, 13 Oct 2019 15:23:47 +0000 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Date: Tue, 20 Aug 2019 14:16:09 -0000 X-Original-Date: Tue, 20 Aug 2019 10:16:05 -0400 X-List-Received-Date: Tue, 20 Aug 2019 14:16:09 -0000 Larry: I am clueless when it comes to net graph :o.. I will be committing the fix/patch for rack shortly :) R > On Aug 20, 2019, at 9:26 AM, Larry Rosenman wrote: >=20 > the M is a patch from rrs@ for the previous crash on m_pullup. >=20 > Ideas? I have a core. >=20 >=20 > Unread portion of the kernel message buffer: > panic: ng_snd_item: 42 !=3D 1414 > cpuid =3D 0 > time =3D 1566303841 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame = 0xfffffe01265c0400 > vpanic() at vpanic+0x19d/frame 0xfffffe01265c0450 > panic() at panic+0x43/frame 0xfffffe01265c04b0 > ng_snd_item() at ng_snd_item+0x455/frame 0xfffffe01265c04f0 > ng_ether_output() at ng_ether_output+0x5e/frame 0xfffffe01265c0520 > ether_output() at ether_output+0x665/frame 0xfffffe01265c05c0 > arpintr() at arpintr+0xfe3/frame 0xfffffe01265c0780 > netisr_dispatch_src() at netisr_dispatch_src+0x89/frame = 0xfffffe01265c07f0 > ether_demux() at ether_demux+0x13b/frame 0xfffffe01265c0820 > ng_ether_rcv_upper() at ng_ether_rcv_upper+0x95/frame = 0xfffffe01265c0840 > ng_apply_item() at ng_apply_item+0xf9/frame 0xfffffe01265c08c0 > ng_snd_item() at ng_snd_item+0x2af/frame 0xfffffe01265c0900 > ng_apply_item() at ng_apply_item+0xf9/frame 0xfffffe01265c0980 > ng_snd_item() at ng_snd_item+0x2af/frame 0xfffffe01265c09c0 > ng_ether_input() at ng_ether_input+0x4c/frame 0xfffffe01265c09f0 > ether_nh_input() at ether_nh_input+0x2cd/frame 0xfffffe01265c0a40 > netisr_dispatch_src() at netisr_dispatch_src+0x89/frame = 0xfffffe01265c0ab0 > ether_input() at ether_input+0x48/frame 0xfffffe01265c0ad0 > bce_intr() at bce_intr+0x697/frame 0xfffffe01265c0b50 > ithread_loop() at ithread_loop+0x187/frame 0xfffffe01265c0bb0 > fork_exit() at fork_exit+0x84/frame 0xfffffe01265c0bf0 > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe01265c0bf0 > --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- > Uptime: 19h52m2s > Dumping 27502 out of 131026 = MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% >=20 > __curthread () at /usr/src/sys/amd64/include/pcpu.h:246 > 246 __asm("movq %%gs:%P1,%0" : "=3Dr" (td) : "n" = (OFFSETOF_CURTHREAD)); > (kgdb) #0 __curthread () at /usr/src/sys/amd64/include/pcpu.h:246 > #1 doadump (textdump=3D1) at /usr/src/sys/kern/kern_shutdown.c:392 > #2 0xffffffff804bb950 in kern_reboot (howto=3D260) > at /usr/src/sys/kern/kern_shutdown.c:479 > #3 0xffffffff804bbdc9 in vpanic (fmt=3D, ap=3D) > at /usr/src/sys/kern/kern_shutdown.c:905 > #4 0xffffffff804bbb03 in panic (fmt=3D) > at /usr/src/sys/kern/kern_shutdown.c:832 > #5 0xffffffff828e1515 in ng_snd_item (item=3D0xfffff81769193580, = flags=3D0) > at /usr/src/sys/netgraph/ng_base.c:2252 > #6 0xffffffff828f2c2e in ng_ether_output (ifp=3D, > mp=3D0xfffffe01265c0578) at /usr/src/sys/netgraph/ng_ether.c:294 > #7 0xffffffff805c9975 in ether_output (ifp=3D0xfffff80122488000, > m=3D0xfffff818181a8b00, dst=3D0xfffffe01265c0740, ro=3D) > at /usr/src/sys/net/if_ethersubr.c:430 > #8 0xffffffff805e2e43 in in_arpinput (m=3D) > at /usr/src/sys/netinet/if_ether.c:1152 > #9 arpintr (m=3D0xfffff818181a8b00) at = /usr/src/sys/netinet/if_ether.c:749 > #10 0xffffffff805d4959 in netisr_dispatch_src (proto=3D4, > source=3D, m=3D) at = /usr/src/sys/net/netisr.c:1123 > #11 0xffffffff805c9c3b in ether_demux (ifp=3D0xfffff80122488000, = m=3D) > at /usr/src/sys/net/if_ethersubr.c:913 > #12 0xffffffff828f3045 in ng_ether_rcv_upper (hook=3D, > item=3D) at /usr/src/sys/netgraph/ng_ether.c:741 > #13 0xffffffff828e1639 in ng_apply_item (node=3D0xfffff801c6ae3c00, > item=3D0xfffff81769193580, rw=3D0) at = /usr/src/sys/netgraph/ng_base.c:2403 > #14 0xffffffff828e136f in ng_snd_item (item=3D0xfffff81769193580, = flags=3D0) > at /usr/src/sys/netgraph/ng_base.c:2320 > #15 0xffffffff828e1639 in ng_apply_item (node=3D0xfffff810410a0400, > item=3D0xfffff81769193580, rw=3D0) at = /usr/src/sys/netgraph/ng_base.c:2403 > #16 0xffffffff828e136f in ng_snd_item (item=3D0xfffff81769193580, = flags=3D0) > at /usr/src/sys/netgraph/ng_base.c:2320 > #17 0xffffffff828f2cbc in ng_ether_input (ifp=3D, > mp=3D0xfffffe01265c0a18) at /usr/src/sys/netgraph/ng_ether.c:255 > #18 0xffffffff805cae8d in ether_input_internal = (ifp=3D0xfffff80122488000, > m=3D0xfffff818181a8b00) at /usr/src/sys/net/if_ethersubr.c:654 > #19 ether_nh_input (m=3D) at = /usr/src/sys/net/if_ethersubr.c:735 > #20 0xffffffff805d4959 in netisr_dispatch_src (proto=3D5, > source=3D, m=3D) at = /usr/src/sys/net/netisr.c:1123 > #21 0xffffffff805ca078 in ether_input (ifp=3D0xfffff80122488000, = m=3D0x0) > at /usr/src/sys/net/if_ethersubr.c:823 > #22 0xffffffff8272f877 in bce_rx_intr (sc=3D) > at /usr/src/sys/dev/bce/if_bce.c:6848 > #23 bce_intr (xsc=3D0xfffffe013abc2000) at = /usr/src/sys/dev/bce/if_bce.c:8017 > #24 0xffffffff80484997 in intr_event_execute_handlers (p=3D, > ie=3D) at /usr/src/sys/kern/kern_intr.c:1148 > #25 ithread_execute_handlers (p=3D, ie=3D) > at /usr/src/sys/kern/kern_intr.c:1161 > #26 ithread_loop (arg=3D) at = /usr/src/sys/kern/kern_intr.c:1241 > #27 0xffffffff80481544 in fork_exit ( > callout=3D0xffffffff80484810 , = arg=3D0xfffff81058226460, > frame=3D0xfffffe01265c0c00) at /usr/src/sys/kern/kern_fork.c:1057 > #28 > (kgdb) >=20 > --=20 > Larry Rosenman http://www.lerctr.org/~ler > Phone: +1 214-642-9640 E-Mail: ler@lerctr.org > US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 ------ Randall Stewart rrs@netflix.com