From nobody Fri Apr 1 21:15:39 2022 X-Original-To: freebsd-net@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 002A71A63FB7 for ; Fri, 1 Apr 2022 21:15:47 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mx1.sbone.de (cross.sbone.de [195.201.62.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.sbone.de", Issuer "SBone.DE" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KVXyT07VJz3vRk; Fri, 1 Apr 2022 21:15:44 +0000 (UTC) (envelope-from bzeeb-lists@lists.zabbadoz.net) Received: from mail.sbone.de (mail.sbone.de [IPv6:fde9:577b:c1a9:31::2013:587]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mx1.sbone.de (Postfix) with ESMTPS id A557B8D4A170; Fri, 1 Apr 2022 21:15:42 +0000 (UTC) Received: from content-filter.sbone.de (content-filter.sbone.de [IPv6:fde9:577b:c1a9:31::2013:2742]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPS id 2BD2AE707BF; Fri, 1 Apr 2022 21:15:42 +0000 (UTC) X-Virus-Scanned: amavisd-new at sbone.de Received: from mail.sbone.de ([IPv6:fde9:577b:c1a9:31::2013:587]) by content-filter.sbone.de (content-filter.sbone.de [fde9:577b:c1a9:31::2013:2742]) (amavisd-new, port 10024) with ESMTP id CkLjLlFaWuye; Fri, 1 Apr 2022 21:15:40 +0000 (UTC) Received: from [192.168.2.110] (unknown [IPv6:fde9:577b:c1a9:31:9172:8746:6a70:afcd]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.sbone.de (Postfix) with ESMTPSA id B25D5E707B8; Fri, 1 Apr 2022 21:15:40 +0000 (UTC) From: "Bjoern A. Zeeb" To: "Peter Holm" Cc: freebsd-net Subject: Re: epoch callback panic Date: Fri, 01 Apr 2022 21:15:39 +0000 X-Mailer: MailMate (2.0BETAr6151) Message-ID: <2E5707B9-63C6-4F8C-AAD5-692B14FF5DE1@lists.zabbadoz.net> In-Reply-To: References: <23673db5-5798-5c06-c04c-7c680ebe41d4@selasky.org> List-Id: Networking and TCP/IP with FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-net List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-net@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8"; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4KVXyT07VJz3vRk X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of bzeeb-lists@lists.zabbadoz.net designates 195.201.62.131 as permitted sender) smtp.mailfrom=bzeeb-lists@lists.zabbadoz.net X-Spamd-Result: default: False [-3.29 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; NEURAL_HAM_MEDIUM(-0.99)[-0.993]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:195.201.62.131]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[zabbadoz.net]; NEURAL_HAM_LONG(-1.00)[-1.000]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RCPT_COUNT_TWO(0.00)[2]; MLMMJ_DEST(0.00)[freebsd-net]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:195.201.0.0/16, country:DE]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On 1 Apr 2022, at 20:51, Peter Holm wrote: > On Fri, Apr 01, 2022 at 10:33:15PM +0200, Hans Petter Selasky wrote: >> On 4/1/22 19:07, Peter Holm wrote: >>> markj@ asked me to post this one: >>> >>> panic: rw lock 0xfffff801bccb1410 not unlocked >>> cpuid = 4 >>> time = 1648770125 >>> KDB: stack backtrace: >>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame >>> 0xfffffe00e48a3d10 >>> vpanic() at vpanic+0x17f/frame 0xfffffe00e48a3d60 >>> panic() at panic+0x43/frame 0xfffffe00e48a3dc0 >>> _rw_destroy() at _rw_destroy+0x35/frame 0xfffffe00e48a3dd0 >>> in_lltable_destroy_lle_unlocked() at >>> in_lltable_destroy_lle_unlocked+0x1a/frame 0xfffffe00e48a3df0 >>> epoch_call_task() at epoch_call_task+0x13a/frame 0xfffffe00e48a3e40 >>> gtaskqueue_run_locked() at gtaskqueue_run_locked+0xa7/frame >>> 0xfffffe00e48a3ec0 >>> gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0xc2/frame >>> 0xfffffe00e48a3ef0 >>> fork_exit() at fork_exit+0x80/frame 0xfffffe00e48a3f30 >>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe00e48a3f30 >>> --- trap 0, rip = 0, rsp = 0, rbp = 0 --- >>> >>> Details @ https://people.freebsd.org/~pho/stress/log/log0275.txt >>> >> >> Hi, >> >> Maybe you need to grab the lock before destroying it? >> > > This was on a pristine main-n254137-31190aa02eef0. If there was no other emory corruption and my memory serves me right, la_flags = 0x1 was LLE_DELETED which gives a good hint on the call path. If I had to bet it’s coming out of the 2nd condition in in_scrubprefixlle ... I’d check lltable_delete_addr .. and so on .. /bz