From nobody Mon Jun 10 16:08:45 2024 X-Original-To: bugs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VycDZ1B4Wz5M9nl for ; Mon, 10 Jun 2024 16:08:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VycDY6Kbhz4rDw for ; Mon, 10 Jun 2024 16:08:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1718035725; a=rsa-sha256; cv=none; b=Gm0aMnywKnFwyhsN1nF+lTCJPulMv17ZzPsIZWv+Of+DyjxDEOZxNVLDG91ff3k8N6pmEq nfNEtAAn2qxd2ca/VlLOHatbzy79KcSd6r5E13ZEtw5qJu35Nvo7AmmP5vOBv81BFJ5qpy 4J9urG93B1AUjccAqPHdGXnMv0Ol/TLuogF6sokVlLv/mtitxVE+xvfj7JKqCK1xcqvtnH o/tb8/h06El0a6aNdCVbQ6CvVh1l331cizrTaA03iOBqCvUH2rbfkxA37bdxg3xuRKVdP9 HuJWgmIoXbUnCBPQVE6bMhD9Z24vn4SjUjUnb/gpX8/vVaDBiudRziLd92H/XA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1718035725; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+dMpuwKhSACDh0ND0brpHo0lvPGzk8IdkhRrj5TYAOs=; b=E4j9fz0RxMfPCj4s9eMmI95T/Pw3TDp1IrrGgqmyyEI4cye2eLx7yDvlR97AjZfMelerDL tFFUfgPSfVZUOzQozfISwVGSqrTUPy8wzxkYZSh34QIMPt+FtBTz0oiEge0ITty8R9ZTH4 xxfBTi5BDnWM8ac0uhQY9D0wUrnUiGSQH6t+8GE0nyRUWoPgzc0++DtrSwAFGUV2cwUS5k KMvK466bf1cckFf9LEYjWmFirKDIjj1OLMVwO/DBuvV3SrABb+TY6fn+0yBIT9ma7YChRc T3ptYgrpSVPRBi4/x+LJ1E/i9vrPTWzSSsMMZPI4oWeb5kg89Pn7DSGQWOf1uw== Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4VycDY5qMtzXtk for ; Mon, 10 Jun 2024 16:08:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 45AG8jRB003598 for ; Mon, 10 Jun 2024 16:08:45 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 45AG8jql003597 for bugs@FreeBSD.org; Mon, 10 Jun 2024 16:08:45 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 277671] 14-RELEASE/14-STABLE crash with heavy disk IO on AMD Asus x670e motherboard and Intel i225 (igc) breakage NIC non-functioning Date: Mon, 10 Jun 2024 16:08:45 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 14.0-STABLE X-Bugzilla-Keywords: crash X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: cam@vasteel.io X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: bugs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Bug reports List-Archive: https://lists.freebsd.org/archives/freebsd-bugs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-bugs@FreeBSD.org MIME-Version: 1.0 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D277671 --- Comment #9 from Cameron --- Tried running monerod for the first time in a while... And my system no lon= ger crashes! This could be resolved by one or more of the following changes: 1. Upgraded to 14.1-RELEASE. I tried 14-STABLE maybe within a few months of 14.1-RELEASE and still had the problem. 2. Started using "zpool trim"... But I have another FreeBSD that had 14.0-RELEASE where I didn't run trim and had no problems. 3. I'm on a beta BIOS for this motherboard that's more recent than current latest official release. I notice after monerod has run for a while, I start getting tons of these messages in dmesg: Jun 5 02:19:11 hostname kernel: sonewconn: pcb 0xfffff802963b9540 (0.0.0.0:18080 (proto 6)): Listen queue overflow: 193 already in queue awai= ting acceptance (1 occurrences), euid 781, rgid 781, jail 0 Jun 5 02:25:11 hostname kernel: sonewconn: pcb 0xfffff802963b9540=20 Increasing kern.ipc.soacceptqueue doesn't seem to help at all. I wonder if = IO is so slow that monerod can't keep up with the connections? The first few times I ran "zpool trim", it only took a few minutes... But o= ver time, it has progressively gotten worse, now taking 21+ minutes. Suggesting there's still some IO issue. Perhaps the same issue I've had in the past wh= en running monerod, but now it no longer causes my box to completely lockup. I can now run monerod constantly without locking up my box though, which is= a nice improvement! In /var/log/monerod.log, I see a lot of traces: 2024-06-10 15:46:31.253 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:134 Exception: boost::wrapexcept 2024-06-10 15:46:31.253 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:135 Unwound call stack: 2024-06-10 15:46:31.385 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:163 1 0x9ab808 __cxa_thro= w + 0xc8 2024-06-10 15:46:31.510 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 2 0x50b05f 2024-06-10 15:46:31.633 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 3 0x7e1f4a 2024-06-10 15:46:31.757 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 4 0x7dc205 2024-06-10 15:46:31.879 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 5 0x788439 2024-06-10 15:46:32.001 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 6 0x78886c 2024-06-10 15:46:32.122 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 7 0x7c05e2 2024-06-10 15:46:32.244 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 8 0x7b2e5b 2024-06-10 15:46:32.365 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 9 0x7bc49d 2024-06-10 15:46:32.486 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 a 0x4d9b88 2024-06-10 15:46:32.607 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 b 0x491100 2024-06-10 15:46:32.728 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 c 0x48eddd 2024-06-10 15:46:32.849 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 d 0x48c562 2024-06-10 15:46:32.970 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 e 0x7e39a5 2024-06-10 15:46:33.091 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 f 0x7fd24f 2024-06-10 15:46:33.212 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 10 0x7fd118 2024-06-10 15:46:33.333 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 11 0x4fb1b2 2024-06-10 15:46:33.453 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 12 0x4f03c4 2024-06-10 15:46:33.575 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 13 0x4efe94 2024-06-10 15:46:33.695 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 14 0x4efbcc 2024-06-10 15:46:33.816 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 15 0x7deaa2 2024-06-10 15:46:33.937 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 16 0x82bec79bd 2024-06-10 15:46:34.058 [P2P6] INFO stacktrace=20=20=20=20=20 src/common/stack_trace.cpp:159 17 0x8324bcb05 I see similar traces on my other box where monerod has never given me probl= ems, but the traces become more far more common on the box that does give me problems once the sonewconn errors start appearing. The sonewconn errors ha= ve never appeared on the other working box. It seems monerod is mostly or entirely unable to continue syncing the block chain with constant stacktraces once it gets to this point unless I complet= ely reboot the system. Completely stopping and starting monerod doesn't help. Looking at sockstat -c the last time I was in this state, I only had a bit = over 200 connections. --=20 You are receiving this mail because: You are the assignee for the bug.=