From owner-freebsd-current@freebsd.org Sat Jul 25 19:21:39 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id EAE7536D1B3; Sat, 25 Jul 2020 19:21:39 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BDbXf7492z4YTH; Sat, 25 Jul 2020 19:21:38 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id 06PJLUYR082228 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sat, 25 Jul 2020 12:21:30 -0700 (PDT) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id 06PJLUsu082227; Sat, 25 Jul 2020 12:21:30 -0700 (PDT) (envelope-from jmg) Date: Sat, 25 Jul 2020 12:21:30 -0700 From: John-Mark Gurney To: Kristof Provost , "Bjoern A. Zeeb" , freebsd-net@freebsd.org, freebsd-current@freebsd.org Subject: Re: somewhat reproducable vimage panic Message-ID: <20200725192129.GN4213@funkthat.com> Mail-Followup-To: Kristof Provost , "Bjoern A. Zeeb" , freebsd-net@freebsd.org, freebsd-current@freebsd.org References: <38F5A3A6-B578-4BA4-8F69-C248163CB6E0@libassi.se> <20200722060514.GF4213@funkthat.com> <20200722193443.GG4213@funkthat.com> <6C149617-55BB-4A87-B993-195E5E133790@lists.zabbadoz.net> <20200722221509.GI4213@funkthat.com> <2FFC49F9-83DE-4FA1-A47F-1D8A7AF4B241@FreeBSD.org> <6847FB6B-0B1A-43C7-B567-15BF21AB5D56@FreeBSD.org> <8B72C0B9-9CF0-4557-81D7-77190775805C@lists.zabbadoz.net> <20200723234925.GL4213@funkthat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200723234925.GL4213@funkthat.com> X-Operating-System: FreeBSD 11.3-STABLE amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Sat, 25 Jul 2020 12:21:30 -0700 (PDT) X-Rspamd-Queue-Id: 4BDbXf7492z4YTH X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of jmg@gold.funkthat.com has no SPF policy when checking 208.87.223.18) smtp.mailfrom=jmg@gold.funkthat.com X-Spamd-Result: default: False [1.68 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-0.19)[-0.193]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[funkthat.com]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.65)[0.646]; NEURAL_SPAM_SHORT(0.03)[0.026]; TO_MATCH_ENVRCPT_SOME(0.00)[]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[jmg@funkthat.com,jmg@gold.funkthat.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; MID_RHS_MATCH_FROM(0.00)[]; FROM_NEQ_ENVFROM(0.00)[jmg@funkthat.com,jmg@gold.funkthat.com]; ASN(0.00)[asn:32354, ipnet:208.87.216.0/21, country:US] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Jul 2020 19:21:40 -0000 John-Mark Gurney wrote this message on Thu, Jul 23, 2020 at 16:49 -0700: > Kristof Provost wrote this message on Thu, Jul 23, 2020 at 11:02 +0200: > > On 23 Jul 2020, at 11:00, Bjoern A. Zeeb wrote: > > > On 23 Jul 2020, at 8:09, Kristof Provost wrote: > > > > > >> On 23 Jul 2020, at 9:19, Kristof Provost wrote: > > >>> On 23 Jul 2020, at 0:15, John-Mark Gurney wrote: > > >>>> So, it's pretty easy to trigger, just attach a couple USB ethernet > > >>>> adapters, in my case, they were ure, but likely any two spare > > >>>> ethernet > > >>>> interfaces will work, and wire them back to back.. > > >>>> > > >>> I???ve been able to trigger it using epair as well: > > >>> > > >>> `sudo sh testinterfaces.txt epair0a epair0b` > > >>> > > >>> I did have to comment out the waitcarrier() check. > > >>> > > >> I???ve done a little bit of digging, and I think I???m starting to > > >> see how this breaks. > > >> > > >> This always affects the jailed vlan interfaces. They???re getting > > >> deleted, but the ifp doesn???t go away just yet because it???s still > > >> in use by the multicast code. > > >> The multicast code does its cleanup in task queues, > > > > > > Wow, did I miss that back then? Did I review a change and not notice? > > > Sorry if that was the case. > > > > > > Vnet teardown is blocking and forceful. > > > Doing deferred cleanup work isn???t a good idea at all. > > > I think that is the real problem here. > > > > > > I???d rather have us fix this than putting more bandaids into the > > > code. > > > > > Yeah, agreed. I think hselasky has a better fix: > > https://reviews.freebsd.org/D24914 > > > > I just saw his e-mail in a different thread. > > I'm testing out this patch now, and let people know how it goes.. It'll > be nice to not have to worry about these panics.. So far so good... I am getting these on occasion: in6_purgeaddr: err=65, destination address delete failed But that's more that the patch prevented a panic. The other issue that I'm now seeing is that because we don't forcefully clear out the multicast task, it can take a good 20+ seconds from the time a jail is destroyed to the interface appearing again in vnet0. Pretty sure this is related to the dmesg from above... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."