From owner-freebsd-net@freebsd.org Tue Jun 26 07:52:13 2018 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 43A911017859; Tue, 26 Jun 2018 07:52:13 +0000 (UTC) (envelope-from dch@skunkwerks.at) Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D9C8984B84; Tue, 26 Jun 2018 07:52:12 +0000 (UTC) (envelope-from dch@skunkwerks.at) Received: from compute7.internal (compute7.nyi.internal [10.202.2.47]) by mailout.nyi.internal (Postfix) with ESMTP id EF94421E00; Tue, 26 Jun 2018 03:52:11 -0400 (EDT) Received: from web6 ([10.202.2.216]) by compute7.internal (MEProxy); Tue, 26 Jun 2018 03:52:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=skunkwerks.at; h=content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; bh=IZ+apDUwMy1Ds9XYcf//sEDWsUDSS Zkcou9aWugrCcc=; b=Eyp8IL2FXR5Y76SP6BEI2BYjbMBgNXV3C6vgsVQffWtP5 x6feYm+O3QbrZk36eN2DbL4Q88ROTvy2YwflseaYfGXgESh2bcKTLIhJLGGumTUD VBf9DI2MS/065Sbbt26MM65F+9fNbt7MqInewgcb2rxQ/3pnsS5zA5Y6eUUBW7bK Z5PVFXgthsYTUk/kdfZshL8IWm0aLVFc4zO+khdocU3R/TCGU8JiTLspvOTVjHLC lXa+3qGiEI8u+asN2zJxjAEdZTtxra0cf5viQcEtAognJAevr27sjuw7gZUCkwyT U3oXrh8G+4mlxg80UEaHxijf9RHmPdC/uLZx0ghzw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=IZ+apD UwMy1Ds9XYcf//sEDWsUDSSZkcou9aWugrCcc=; b=DnZ1h/JucL1VD9w1NGIb6H NQZR0cyU6dDYTEBh7wi0hDV6ffyJi90DoCn6/+A1Qe2GuxOi3DSacui5Ic5DWsAI hGk6K5tihO/f0ywZ93CHZVERgkevhHoYy8IkQ5LrhB6lBrCscrzdffKZ5aS4EJPp i03GW3BZ41kXGy7IKt9qMh2kdTkJCZCz8NnKFFBg/kbZTq4+6F2UrWEJuOwsmNrZ O3DHnJI9r5imI0xyaf5021C3QMqSTDbehs8CL9dMbBe6yvBYPmz/xly6xxG8ykFM pPxmq9GsYPKHfcJWs128yvA2GMHUCoc3YhqnQaXrZ6IYW7lbvc94SKFRaeD6dNjA == X-ME-Proxy: X-ME-Sender: Received: by mailuser.nyi.internal (Postfix, from userid 99) id 9C6CF411E; Tue, 26 Jun 2018 03:52:11 -0400 (EDT) Message-Id: <1529999531.2330685.1420550472.409ED830@webmail.messagingengine.com> From: Dave Cottlehuber To: freebsd-net@freebsd.org, freebsd-current@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="utf-8" X-Mailer: MessagingEngine.com Webmail Interface - ajax-0d8ea36c Subject: Re: unloading pf causes desktop system to freeze since ~ r335651 [coredump] Date: Tue, 26 Jun 2018 09:52:11 +0200 In-Reply-To: <1529960902.613046.1420067144.3B445531@webmail.messagingengine.com> References: <1529960902.613046.1420067144.3B445531@webmail.messagingengine.com> X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jun 2018 07:52:13 -0000 On Mon, 25 Jun 2018, at 23:08, Dave Cottlehuber wrote: > [cross-posting for advice on general debugging + network-specific thoughts] The HPET NMI watchdog patch was very timely - works a treat: https://reviews.freebsd.org/D15630 > However each time there's no crashdump, & the usual ctrl-alt-esc does't > work either. I bumped my /usr/src to latest HEAD, applied HPET NMI watchdog hack & after freezing via `service pf stop`, I was rewarded with a coredump on next reboot; full log: https://git.io/f4Q4P [1202] panic: Assertion !in_epoch() && !mtx_owned(&(&(*(__typeof(vnet_entry_tcbinfo)*) (((((__curthread())->td_vnet))->vnet_data_base) + (uintptr_t)&vnet_entry_tcbinfo)))->ipi_lock) failed at /usr/src/sys/netinet/tcp_input.c:802 [1202] cpuid = 4 [1202] time = 1529997533 [1202] KDB: stack backtrace: [1202] db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0077ddc4a0 [1202] vpanic() at vpanic+0x1a3/frame 0xfffffe0077ddc500 [1202] doadump() at doadump/frame 0xfffffe0077ddc580 [1202] tcp_input() at tcp_input+0x940/frame 0xfffffe0077ddc6c0 [1202] ip_input() at ip_input+0x3f7/frame 0xfffffe0077ddc720 [1202] netisr_dispatch_src() at netisr_dispatch_src+0xa2/frame 0xfffffe0077ddc780 [1202] ether_demux() at ether_demux+0x16e/frame 0xfffffe0077ddc7b0 [1202] ether_nh_input() at ether_nh_input+0x402/frame 0xfffffe0077ddc820 [1202] netisr_dispatch_src() at netisr_dispatch_src+0xa2/frame 0xfffffe0077ddc880 [1202] ether_input() at ether_input+0x8f/frame 0xfffffe0077ddc8c0 [1202] iflib_rxeof() at iflib_rxeof+0xcce/frame 0xfffffe0077ddc9b0 [1202] _task_fn_rx() at _task_fn_rx+0x7f/frame 0xfffffe0077ddc9f0 [1202] gtaskqueue_run_locked() at gtaskqueue_run_locked+0x139/frame 0xfffffe0077ddca40 [1202] gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x88/frame 0xfffffe0077ddca70 [1202] fork_exit() at fork_exit+0x84/frame 0xfffffe0077ddcab0 [1202] fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0077ddcab0 [1202] --- trap 0, rip = 0, rsp = 0, rbp = 0 --- [1202] KDB: enter: panic db:0:kdb.enter.panic> run lockinfo db:1:lockinfo> show locks db:1:lockinfo> show alllocks Process 12789 (h2o) thread 0xfffff8020dd4f580 (101673) Process 17635 (pflogd) thread 0xfffff8017c91c580 (101027) db:1:lockinfo> show lockedvnods Locked vnodes db:0:kdb.enter.panic> show pcpu cpuid = 4 dynamic pcpu = 0xfffffe00848dd8c0 curthread = 0xfffff801067f2000: pid 0 tid 100029 "if_io_tqg_4" curpcb = 0xfffffe0077ddcb80 fpcurthread = none idlethread = 0xfffff80106796000: tid 100007 "idle: cpu4" curpmap = 0xffffffff81ffbe08 tssp = 0xffffffff82066ac0 commontssp = 0xffffffff82066ac0 rsp0 = 0xfffffe0077ddcb80 gs32p = 0xffffffff8206d6f8 ldt = 0xffffffff8206d738 tss = 0xffffffff8206d728 curvnet = 0xfffff801000ca0c0 spin locks held: db:0:kdb.enter.panic> bt Tracing pid 0 tid 100029 td 0xfffff801067f2000 kdb_enter() at kdb_enter+0x3b/frame 0xfffffe0077ddc4a0 vpanic() at vpanic+0x1c0/frame 0xfffffe0077ddc500 doadump() at doadump/frame 0xfffffe0077ddc580 tcp_input() at tcp_input+0x940/frame 0xfffffe0077ddc6c0 ip_input() at ip_input+0x3f7/frame 0xfffffe0077ddc720 netisr_dispatch_src() at netisr_dispatch_src+0xa2/frame 0xfffffe0077ddc780 ether_demux() at ether_demux+0x16e/frame 0xfffffe0077ddc7b0 ether_nh_input() at ether_nh_input+0x402/frame 0xfffffe0077ddc820 netisr_dispatch_src() at netisr_dispatch_src+0xa2/frame 0xfffffe0077ddc880 ether_input() at ether_input+0x8f/frame 0xfffffe0077ddc8c0 iflib_rxeof() at iflib_rxeof+0xcce/frame 0xfffffe0077ddc9b0 _task_fn_rx() at _task_fn_rx+0x7f/frame 0xfffffe0077ddc9f0 gtaskqueue_run_locked() at gtaskqueue_run_locked+0x139/frame 0xfffffe0077ddca40 gtaskqueue_thread_loop() at gtaskqueue_thread_loop+0x88/frame 0xfffffe0077ddca70 fork_exit() at fork_exit+0x84/frame 0xfffffe0077ddcab0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0077ddcab0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- Please keep replies just to freebsd-net@ from here on. A+ Dave