From nobody Mon Nov 8 07:06:43 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id E428418563BB for ; Mon, 8 Nov 2021 07:06:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HnhxG5q3Zz3Kth for ; Mon, 8 Nov 2021 07:06:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A95234D64 for ; Mon, 8 Nov 2021 07:06:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 1A876gR5015008 for ; Mon, 8 Nov 2021 07:06:42 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 1A876g29015007 for fs@FreeBSD.org; Mon, 8 Nov 2021 07:06:42 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 259607] prometheus_sysctl_exporter: Need better encoding support for sysctl OIDs Date: Mon, 08 Nov 2021 07:06:43 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: delphij@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: delphij@FreeBSD.org X-Bugzilla-Flags: maintainer-feedback+ X-Bugzilla-Changed-Fields: component product assigned_to short_desc version bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D259607 Xin LI changed: What |Removed |Added ---------------------------------------------------------------------------- Component|Individual Port(s) |bin Product|Ports & Packages |Base System Assignee|dor.bsd@xm0.uk |delphij@FreeBSD.org Summary|sysutils/node_exporter / |prometheus_sysctl_exporter: |OpenZFS: need better |Need better encoding |handling for ZFS statistic |support for sysctl OIDs |numbers | Version|Latest |CURRENT Status|New |In Progress --- Comment #2 from Xin LI --- Thanks, I've created a patch for this at https://reviews.freebsd.org/D32886= . --=20 You are receiving this mail because: You are on the CC list for the bug.= From nobody Mon Nov 8 07:07:02 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id E23DA1856D44 for ; Mon, 8 Nov 2021 07:07:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hnhxf5fDXz3L1X for ; Mon, 8 Nov 2021 07:07:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9D52354B8 for ; Mon, 8 Nov 2021 07:07:02 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 1A8772MS015118 for ; Mon, 8 Nov 2021 07:07:02 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 1A8772kd015117 for fs@FreeBSD.org; Mon, 8 Nov 2021 07:07:02 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 259607] prometheus_sysctl_exporter: Need better encoding support for sysctl OIDs Date: Mon, 08 Nov 2021 07:07:02 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: delphij@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: delphij@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: flagtypes.name Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D259607 Xin LI changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|maintainer-feedback+ | --=20 You are receiving this mail because: You are on the CC list for the bug.= From nobody Tue Nov 9 22:03:34 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 9843C183E31E for ; Tue, 9 Nov 2021 22:03:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hphnf3mTYz4vQK for ; Tue, 9 Nov 2021 22:03:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 628D24EAF for ; Tue, 9 Nov 2021 22:03:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 1A9M3Ymo075952 for ; Tue, 9 Nov 2021 22:03:34 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 1A9M3YTR075951 for fs@FreeBSD.org; Tue, 9 Nov 2021 22:03:34 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 245916] Superblock check-hash failed Date: Tue, 09 Nov 2021 22:03:34 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: cragapito@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D245916 cragapito@gmail.com changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |cragapito@gmail.com --- Comment #1 from cragapito@gmail.com --- I had the same problem, but fsck made my old system unusable trying recover that check-hashes, and worst, it damage my backup disk! I can't mount it because after the messages it gave me 'mount: Integrity check failed'. fsck says that's clean, but I still cant mount (despite messages that it was ignored). I share that backup disk between machines with different versions, but I ne= ver even dream about that level of regression next to my 20 year with FreeBSD. I appreciate the attempt to evolve, but I found it rash and irresponsible. --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Wed Nov 10 03:40:16 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 8483A185CB4F for ; Wed, 10 Nov 2021 03:40:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HprGB2yc7z4kV4 for ; Wed, 10 Nov 2021 03:40:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 473C511451 for ; Wed, 10 Nov 2021 03:40:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 1AA3eIDS062328 for ; Wed, 10 Nov 2021 03:40:18 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 1AA3eIHo062327 for fs@FreeBSD.org; Wed, 10 Nov 2021 03:40:18 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 255979] fsck bad inode number 2 (256) to nextinode Date: Wed, 10 Nov 2021 03:40:16 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 13.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: emaste@freebsd.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D255979 Ed Maste changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |emaste@freebsd.org --- Comment #30 from Ed Maste --- This will become FreeBSD-EN-21:30.fsck_ffs --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Wed Nov 10 17:46:34 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A3D97183E4E0 for ; Wed, 10 Nov 2021 17:46:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HqC2f49rdz4ZFn for ; Wed, 10 Nov 2021 17:46:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6FE211CE24 for ; Wed, 10 Nov 2021 17:46:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 1AAHkY8D039761 for ; Wed, 10 Nov 2021 17:46:34 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 1AAHkYMc039760 for fs@FreeBSD.org; Wed, 10 Nov 2021 17:46:34 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 258208] [zfs] locks up when using rollback or destroy on both 13.0-RELEASE & sysutils/openzfs port Date: Wed, 10 Nov 2021 17:46:34 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 13.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: markj@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: markj@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D258208 Mark Johnston changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|fs@FreeBSD.org |markj@FreeBSD.org --- Comment #21 from Mark Johnston --- See https://reviews.freebsd.org/D32931 for an attempt at a fix. --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Wed Nov 10 18:48:44 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 232961838055 for ; Wed, 10 Nov 2021 18:48:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HqDQP0Ry3z4yGh for ; Wed, 10 Nov 2021 18:48:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id E4CBA1DB44 for ; Wed, 10 Nov 2021 18:48:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 1AAImiY2079407 for ; Wed, 10 Nov 2021 18:48:44 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 1AAImiYi079406 for fs@FreeBSD.org; Wed, 10 Nov 2021 18:48:44 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 245916] Superblock check-hash failed Date: Wed, 10 Nov 2021 18:48:44 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: cragapito@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D245916 --- Comment #2 from cragapito@gmail.com --- Googling I could recover the backup of SB, got by 'newfs -N /dev/da0' at 192 and using 'fsck_ufs -b 192 /dev/da0'. Now it's working. Sorry by my previous message... --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Thu Nov 11 04:43:46 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 7A839184AAFC; Thu, 11 Nov 2021 04:43:55 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from relay.wiredblade.com (relay.wiredblade.com [168.235.95.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4HqTd709LXz4tvj; Thu, 11 Nov 2021 04:43:54 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from mail.distal.com (pool-108-48-165-176.washdc.fios.verizon.net [108.48.165.176]) by relay.wiredblade.com with ESMTPSA (version=TLSv1.2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256) ; Thu, 11 Nov 2021 04:43:53 +0000 Received: from smtpclient.apple ( [2001:420:c0c8:1002::200]) by tristain.distal.com (OpenSMTPD) with ESMTPSA id dbeee53b (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Wed, 10 Nov 2021 23:43:52 -0500 (EST) Content-Type: text/plain; charset=utf-8 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 15.0 \(3693.20.0.1.32\)) Subject: Re: swap_pager: cannot allocate bio From: Chris Ross In-Reply-To: <9FE99EEF-37C5-43D1-AC9D-17F3EDA19606@distal.com> Date: Wed, 10 Nov 2021 23:43:46 -0500 Cc: freebsd-stable@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <09989390-FED9-45A6-A866-4605D3766DFE@distal.com> References: <9FE99EEF-37C5-43D1-AC9D-17F3EDA19606@distal.com> To: freebsd-fs X-Mailer: Apple Mail (2.3693.20.0.1.32) X-Rspamd-Queue-Id: 4HqTd709LXz4tvj X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[freebsd] X-ThisMailContainsUnwantedMimeParts: N > On Nov 10, 2021, at 23:35, Chris Ross = wrote: >=20 > Hey all. I have a system that I=E2=80=99m trying to do some intensive = CPU and I/O on. FreeBSD 13.0-RELEASE, amd64, 128GB RAM, hardware RAID1 = OS volume, and a large (40TB) zpool where most of the I/O is happening. >=20 > Initially, it was failing for me because it was running out of swap = space. It had only the normal small (4-8G) swap partition, so I resized = the filesystems on the root disk and now have 400+GB swap. The system = had frozen up and I wasn=E2=80=99t able to log in. When I go to the = console, I find a long list of: >=20 > swap_pager: cannot allocate bio >=20 > lines. I was able to log into the console as root and pstat -s shows = the swap minimally used (7.5GB used). Attempting a =E2=80=9Czpool = status=E2=80=9D at that point locked up. I don=E2=80=99t know if the = problem is the memory subsystem, or zfs. >=20 > But, based on the error, is there perhaps some kernel parameter I can = tune that might prevent the swap pager from encountering that error? Moving to freebsd-fs. More information makes it looks more like a ZFS = problem than anything else. I am able to log into another root virtual console, and I can run ps = (shows many things, including dozens of "cron: running job (cron)=E2=80=9D= jobs, in D state), and I=E2=80=99m able to wander around the root disk = (3T ufs filesystem) without trouble. But, as mentioned above the = =E2=80=9Czpool status=E2=80=9D is hung, and I suspect if I tried to = access anything in that filesystem it would hang to. Those cron jobs, = which aren=E2=80=99t anything I added, I assume are just system =E2=80=9Cc= heck around the system=E2=80=9D cron jobs that are getting stuck there. So, if anyone has any suggestions. I can leave this system stuck like = this for a little while, but I=E2=80=99ll probably want to bring it back = before the end of the day tomorrow. (I=E2=80=99m US EST, so it=E2=80=99s = almost midnight here. I=E2=80=99ll check in on email for suggestions or = ideas in the morning.) Thanks all. - Chris= From nobody Thu Nov 11 13:27:42 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id C2343184E679 for ; Thu, 11 Nov 2021 13:27:50 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from relay.wiredblade.com (relay.wiredblade.com [168.235.95.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4HqjFd5XKZz4ZNB; Thu, 11 Nov 2021 13:27:49 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from mail.distal.com (pool-108-48-165-176.washdc.fios.verizon.net [108.48.165.176]) by relay.wiredblade.com with ESMTPSA (version=TLSv1.2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256) ; Thu, 11 Nov 2021 13:27:48 +0000 Received: from smtpclient.apple ( [2001:420:c0c8:1002::200]) by tristain.distal.com (OpenSMTPD) with ESMTPSA id 906e6cc0 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Thu, 11 Nov 2021 08:27:46 -0500 (EST) From: Chris Ross Message-Id: <665A4DDB-6973-4A2D-A427-492F30D27851@distal.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_418595D6-236C-433E-A3B7-CBE045A968A1" List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 15.0 \(3693.20.0.1.32\)) Subject: Re: ZFS operations hanging, but no visible errors? Date: Thu, 11 Nov 2021 08:27:42 -0500 In-Reply-To: Cc: freebsd-fs To: Andriy Gapon References: <20211105173935.7aa53269@fabiankeil.de> <86999084-7007-4F08-A4C4-4A835A7E1C78@distal.com> X-Mailer: Apple Mail (2.3693.20.0.1.32) X-Rspamd-Queue-Id: 4HqjFd5XKZz4ZNB X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of cross@distal.com designates 168.235.95.80 as permitted sender) smtp.mailfrom=cross@distal.com X-Spamd-Result: default: False [2.37 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; R_SPF_ALLOW(-0.20)[+a:relay.dynu.com]; ARC_NA(0.00)[]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; DMARC_NA(0.00)[distal.com]; NEURAL_HAM_LONG(-0.83)[-0.827]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_SPAM_SHORT(1.00)[0.998]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:3842, ipnet:168.235.92.0/22, country:US]; TAGGED_FROM(0.00)[freebsd]; RCVD_TLS_ALL(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[108.48.165.176:received] X-Spam: Yes X-ThisMailContainsUnwantedMimeParts: Y --Apple-Mail=_418595D6-236C-433E-A3B7-CBE045A968A1 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Following up on a new hang this same system had (yesterday freebsd-fs = mail subject "swap_pager: cannot allocate bio=E2=80=9D), I think the = same problem might have occurred again. Certainly the system got stuck = again Based on the below, my executing that dtrace command caused the system = to report "ACPI Error: AE_NO_MEMORY=E2=80=9D. In what way is the system = out of memory here? And, does that failure running dtrace suggest that = that =E2=80=9Cout of memory=E2=80=9D problem is the core problem causing = the ZFS hang in the first place? My system has 128GB, which is nothing = to sneeze at. Are there parameters that I should change because the = normal parameters just don=E2=80=99t work well with a pool or fs this = large? And, from earlier in this thread from last week: Now that I have the = system running again, I can provide the "zpool status=E2=80=9D for = information. Let me know if I=E2=80=99ve just tried something crazy = here, this is the largest ZFS filesystem I=E2=80=99ve attempted. I have = a 30T pool on another system without issue, and with less RAM. (The = largest fs on that pool is about 18T) % zfs status pool: tank state: ONLINE scan: scrub repaired 0B in 05:05:55 with 0 errors on Sat Oct 23 = 04:38:36 2021 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 da3 ONLINE 0 0 0 da2 ONLINE 0 0 0 da1 ONLINE 0 0 0 raidz1-1 ONLINE 0 0 0 da4 ONLINE 0 0 0 da5 ONLINE 0 0 0 da6 ONLINE 0 0 0 raidz1-2 ONLINE 0 0 0 da7 ONLINE 0 0 0 da8 ONLINE 0 0 0 da9 ONLINE 0 0 0 errors: No known data errors % zfs list tank NAME USED AVAIL REFER MOUNTPOINT tank 14.2T 35.0T 14.2T /tank - Chris > On Nov 7, 2021, at 03:35, Andriy Gapon wrote: >=20 > On 05/11/2021 18:59, Chris Ross wrote: >> Running prostate -kk on the rsync that was hung, then killed, then = SIGKILL=E2=80=99d shows: >> procstat -kk 35220 >> PID TID COMM TDNAME KSTACK >> 35220 102499 rsync - mi_switch+0xc1 = _sleep+0x1cb vm_wait_doms+0xe2 vm_wait_domain+0x51 = vm_domain_alloc_fail+0x86 vm_page_alloc_domain_after+0x7e = uma_small_alloc+0x58 keg_alloc_slab+0xba zone_import+0xee = zone_alloc_item+0x6f abd_alloc_chunks+0x61 abd_alloc+0x102 = arc_hdr_alloc_abd+0xb0 arc_hdr_alloc+0x11e arc_read+0x4f4 = dbuf_issue_final_prefetch+0x108 dbuf_prefetch_impl+0x3d0 = dmu_zfetch+0x558 >=20 > Looks like the system is out of memory. > It seems that you already established that. >=20 > --=20 > Andriy Gapon >=20 --Apple-Mail=_418595D6-236C-433E-A3B7-CBE045A968A1-- From nobody Thu Nov 11 18:50:39 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 83F35183D6E5 for ; Thu, 11 Nov 2021 18:50:50 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smarthost1.greenhost.nl (smarthost1.greenhost.nl [195.190.28.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4HqrQK40FBz3sFg for ; Thu, 11 Nov 2021 18:50:49 +0000 (UTC) (envelope-from ronald-lists@klop.ws) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=klop.ws; s=mail; h=In-Reply-To:Message-ID:From:Content-Transfer-Encoding:MIME-Version: Date:References:Subject:To:Content-Type:Sender:Reply-To:Cc:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=ALPUixyQ+n6pEZvdvaOluSYxiDP8NL2EqAlewY1iP5M=; b=mNsFid2wyTxBZjc9+c2MB4zI7D 0N6jeiizYaZH2tQnI66xemvdsAIBnu+PqnjwLiP5w5RvOOKAOp+/fa37FQe7tZTyhOsDGhiYZH3l/ ticV7Wx5AMq3K/L1hjsdGjqHvgrK01LFpZay+HQ5t5FPXeVvEXCoCsiWTYQwWt20bd54=; Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: freebsd-fs@freebsd.org Subject: Re: swap_pager: cannot allocate bio References: <9FE99EEF-37C5-43D1-AC9D-17F3EDA19606@distal.com> <09989390-FED9-45A6-A866-4605D3766DFE@distal.com> Date: Thu, 11 Nov 2021 19:50:39 +0100 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: Quoted-Printable Message-ID: In-Reply-To: <09989390-FED9-45A6-A866-4605D3766DFE@distal.com> User-Agent: Opera Mail/1.0 (Win32) X-Authenticated-As-Hash: bdb49c4ff80bd276e321aade33e76e02752072e2 X-Virus-Scanned: by clamav at smarthost1.greenhost.nl X-Spam-Level: / X-Spam-Score: -0.4 X-Spam-Status: No, score=-0.4 required=5.0 tests=ALL_TRUSTED,BAYES_50,DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF autolearn=disabled version=3.4.2 X-Scan-Signature: d9b0ae15ee993d77aea4f0208a5c5b8c X-Rspamd-Queue-Id: 4HqrQK40FBz3sFg X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=klop.ws header.s=mail header.b=mNsFid2w; dmarc=pass (policy=quarantine) header.from=klop.ws; spf=pass (mx1.freebsd.org: domain of ronald-lists@klop.ws designates 195.190.28.88 as permitted sender) smtp.mailfrom=ronald-lists@klop.ws X-Spamd-Result: default: False [-1.38 / 15.00]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[klop.ws:s=mail]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:195.190.28.64/27]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_SPAM_MEDIUM(1.00)[1.000]; RCPT_COUNT_ONE(0.00)[1]; DKIM_TRACE(0.00)[klop.ws:+]; DMARC_POLICY_ALLOW(-0.50)[klop.ws,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[195.190.28.88:from]; NEURAL_HAM_SHORT(-0.88)[-0.882]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_NOT_FQDN(0.50)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:47172, ipnet:195.190.28.0/24, country:NL]; RWL_MAILSPIKE_POSSIBLE(0.00)[195.190.28.88:from] Reply-To: ronald-lists@klop.ws From: Ronald Klop via freebsd-fs X-Original-From: Ronald Klop X-ThisMailContainsUnwantedMimeParts: N On Thu, 11 Nov 2021 05:43:46 +0100, Chris Ross = wrote: > > >> On Nov 10, 2021, at 23:35, Chris Ross wrot= e: >> >> Hey all. I have a system that I=E2=80=99m trying to do some intensiv= e CPU and = >> I/O on. FreeBSD 13.0-RELEASE, amd64, 128GB RAM, hardware RAID1 OS = >> volume, and a large (40TB) zpool where most of the I/O is happening. >> >> Initially, it was failing for me because it was running out of swap = >> space. It had only the normal small (4-8G) swap partition, so I = >> resized the filesystems on the root disk and now have 400+GB swap. T= he = >> system had frozen up and I wasn=E2=80=99t able to log in. When I go = to the = >> console, I find a long list of: >> >> swap_pager: cannot allocate bio >> >> lines. I was able to log into the console as root and pstat -s shows= = >> the swap minimally used (7.5GB used). Attempting a =E2=80=9Czpool st= atus=E2=80=9D at = >> that point locked up. I don=E2=80=99t know if the problem is the mem= ory = >> subsystem, or zfs. >> >> But, based on the error, is there perhaps some kernel parameter I can= = >> tune that might prevent the swap pager from encountering that error? > > Moving to freebsd-fs. More information makes it looks more like a ZFS= = > problem than anything else. > > I am able to log into another root virtual console, and I can run ps = > (shows many things, including dozens of "cron: running job (cron)=E2=80= =9D jobs, = > in D state), and I=E2=80=99m able to wander around the root disk (3T u= fs = > filesystem) without trouble. But, as mentioned above the =E2=80=9Czpo= ol status=E2=80=9D = > is hung, and I suspect if I tried to access anything in that filesyste= m = > it would hang to. Those cron jobs, which aren=E2=80=99t anything I ad= ded, I = > assume are just system =E2=80=9Ccheck around the system=E2=80=9D cron = jobs that are = > getting stuck there. > > So, if anyone has any suggestions. I can leave this system stuck like= = > this for a little while, but I=E2=80=99ll probably want to bring it ba= ck before = > the end of the day tomorrow. (I=E2=80=99m US EST, so it=E2=80=99s alm= ost midnight = > here. I=E2=80=99ll check in on email for suggestions or ideas in the = morning.) > > Thanks all. > > - Chris > Can you press ctrl-t on the hanging process? That should print the = stacktrace indicating where it is waiting on. Ronald. From nobody Thu Nov 11 19:05:33 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 643EE1845068 for ; Thu, 11 Nov 2021 19:05:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HqrlK2L2Wz4R5Z for ; Thu, 11 Nov 2021 19:05:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2DC42118A9 for ; Thu, 11 Nov 2021 19:05:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 1ABJ5Xv1017954 for ; Thu, 11 Nov 2021 19:05:33 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 1ABJ5XXQ017953 for fs@FreeBSD.org; Thu, 11 Nov 2021 19:05:33 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 259276] sysutils/openzfs-kmod/: Read errors with openzfs-kmod-2021101400 Date: Thu, 11 Nov 2021 19:05:33 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: needs-qa X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: freqlabs@FreeBSD.org X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D259276 --- Comment #5 from Ryan Moeller --- I've just updated the port to openzfs/zfs@637771a06 which brings in commits that are intended to fix this issue. Please let me know if the errors persi= st after this. --=20 You are receiving this mail because: You are on the CC list for the bug.= From nobody Thu Nov 11 21:49:21 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A0A6718482DB for ; Thu, 11 Nov 2021 21:49:27 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from relay.wiredblade.com (relay.wiredblade.com [168.235.95.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4HqwNR2Gc9z3Kv3 for ; Thu, 11 Nov 2021 21:49:27 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from mail.distal.com (pool-108-48-165-176.washdc.fios.verizon.net [108.48.165.176]) by relay.wiredblade.com with ESMTPSA (version=TLSv1.2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256) ; Thu, 11 Nov 2021 21:49:24 +0000 Received: from smtpclient.apple ( [2001:420:c0c4:1004::ab]) by tristain.distal.com (OpenSMTPD) with ESMTPSA id fe44d9e4 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Thu, 11 Nov 2021 16:49:23 -0500 (EST) Content-Type: multipart/alternative; boundary="Apple-Mail=_A7A226AD-D288-4FA6-8B73-D4E088DDCBCF" List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 15.0 \(3693.20.0.1.32\)) Subject: Re: swap_pager: cannot allocate bio From: Chris Ross In-Reply-To: Date: Thu, 11 Nov 2021 16:49:21 -0500 Cc: freebsd-fs Message-Id: <4E5511DF-B163-4928-9CC3-22755683999E@distal.com> References: <9FE99EEF-37C5-43D1-AC9D-17F3EDA19606@distal.com> <09989390-FED9-45A6-A866-4605D3766DFE@distal.com> To: ronald-lists@klop.ws X-Mailer: Apple Mail (2.3693.20.0.1.32) X-Rspamd-Queue-Id: 4HqwNR2Gc9z3Kv3 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[freebsd] X-ThisMailContainsUnwantedMimeParts: Y --Apple-Mail=_A7A226AD-D288-4FA6-8B73-D4E088DDCBCF Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Nov 11, 2021, at 13:50, Ronald Klop via freebsd-fs = wrote: >=20 > Can you press ctrl-t on the hanging process? That should print the = stacktrace indicating where it is waiting on. So, I rebooted the machine this morning, but now have [tried to] log = into it to check on it and find that an ssh connection doesn=E2=80=99t = result in a shell. I logged into the console, tried to start a = =E2=80=9Cscreen=E2=80=9D to get more prompts, and it hung. Ctrl-T on = that shows (after running a console screen-capture through OCR, and hand = correction, so may not be 100%): root@host:~ # screen load: 0.07 cmd: csh 56116 [vmwait] 35.00r 0.00u 0.01s 0% 3984k mi_switch+0xc1 _sleep+0x1cb vm_wait_doms+0xe2 vm_wait_domain+0x51 = vm_domain_alloc_fail+0x86 vm_page_alloc_domain_after+0x7e = uma_small_alloc+0x58 keg_alloc_slab+0xba zone_import+0xee = zone_alloc_item+0x6f malloc+0x5d sigacts_alloc+0x1c fork1+0x9fb = sys_fork+0x54 amd64_syscall+0x10c fast_syscall_common+0xf8=20 As before, ps and even mount and df work here on console. But, a = =E2=80=9Czpool status tank=E2=80=9D will hang as before. A Ctrl+D on it root@host:~ # screen load: 0.00 cmd: zpool 62829 [aw.aew_cv] 37.89r 0.00u 0.00s 0% 6976k mi_switch+0xc1 _cv_wait+0xf2 arc_wait_for_eviction+0x14a = arc_get_data_impl+0xdb arc_hdr_alloc_abd+0xa6 arc_hdr_alloc+0x11e = arc_read+0x4f4 dbuf_read+0xc08 dmu_buf_hold+0x46 zap_lookup_norm+0x35 = zap_contains+0x26 vdev_rebuild_get_stats+0xac vdev_config_generate+0x3e9 = vdev_config_generate+0x74f spa_config_generate+0x2a2 = spa_open_common+0x25c spa_get_stats+0x4e zfs_ioc_pool_stats+0x22 > On Nov 11, 2021, at 14:10, Dave Cottlehuber <> wrote: >=20 > Grab output of =E2=80=98procstat-kk=E2=80=99 and see if this is = similar to https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D258208 a = few more prods might get this one addressed! procstat -kk 62829 yields the same as above. Which I presume is = expected, I=E2=80=99d just never used procstate -kk before. Unfortunately, I can=E2=80=99t tell if this is sufficiently similar to = bug 258208. A different ZFS operation is happening here, so the calls = behind my zpool status are different. The other non-zfs stat above = (screen in my case) doesn=E2=80=99t seem to be hitting zfs at all, but I = may be missing something. Andriy, Mark J, let me know if you think this = is relevant, I can build a 13-STABLE with D32931 if you think it will be = of use. Thanks. Let me know any thoughts you have. - Chris --Apple-Mail=_A7A226AD-D288-4FA6-8B73-D4E088DDCBCF-- From nobody Fri Nov 12 07:36:07 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 6131E18494A3 for ; Fri, 12 Nov 2021 07:36:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hr9PM2GVdz3Lly for ; Fri, 12 Nov 2021 07:36:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2E4241B650 for ; Fri, 12 Nov 2021 07:36:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 1AC7a78D024698 for ; Fri, 12 Nov 2021 07:36:07 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 1AC7a7Xj024697 for fs@FreeBSD.org; Fri, 12 Nov 2021 07:36:07 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 259276] sysutils/openzfs-kmod/: Read errors with openzfs-kmod-2021101400 Date: Fri, 12 Nov 2021 07:36:07 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: needs-qa X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: ottkostner@gmail.com X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D259276 --- Comment #6 from Ott K=C3=B6stner --- Installed the latest port (openzfs-kmod-2021111100) on a test machine and t= he first impression is that the issue is fixed. Will test some time and also on another machine before closing this ticket. --=20 You are receiving this mail because: You are on the CC list for the bug.= From nobody Fri Nov 12 09:35:57 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id D624F183D32E for ; Fri, 12 Nov 2021 09:36:05 +0000 (UTC) (envelope-from SRS0=2NMC=P7=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4HrD3n4zSPz4k7C for ; Fri, 12 Nov 2021 09:36:05 +0000 (UTC) (envelope-from SRS0=2NMC=P7=klop.ws=ronald-lists@realworks.nl) Date: Fri, 12 Nov 2021 10:35:57 +0100 (CET) DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=klop.ws; s=rw1; t=1636709758; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=tl4LVuX2AI8oHxnWCccTh5fL3ag5tVDI1cwc8HFyDlI=; b=S2JaLFmqGsNA365oJ0A9yfQzKneajUExFkMnajgc9udZdp3V1WASDQR50QIg7PMqCx92wg ayYM4fd57lS2QBAg== To: freebsd-fs Message-ID: <42006135.15.1636709757975@mailrelay> In-Reply-To: <4E5511DF-B163-4928-9CC3-22755683999E@distal.com> References: <9FE99EEF-37C5-43D1-AC9D-17F3EDA19606@distal.com> <09989390-FED9-45A6-A866-4605D3766DFE@distal.com> <4E5511DF-B163-4928-9CC3-22755683999E@distal.com> Subject: Re: swap_pager: cannot allocate bio List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_14_1187132451.1636709757927" X-Mailer: Realworks (585.1291.90acf81) Importance: Normal X-Priority: 3 (Normal) X-Rspamd-Queue-Id: 4HrD3n4zSPz4k7C X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Reply-To: ronald-lists@klop.ws From: Ronald Klop via freebsd-fs X-Original-From: Ronald Klop X-ThisMailContainsUnwantedMimeParts: Y ------=_Part_14_1187132451.1636709757927 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable =20 Van: Chris Ross Datum: donderdag, 11 november 2021 22:49 Aan: ronald-lists@klop.ws CC: freebsd-fs Onderwerp: Re: swap_pager: cannot allocate bio >=20 > =20 > >>=20 >> On Nov 11, 2021, at 13:50, Ronald Klop via freebsd-fs wrote: >>=20 >> Can you press ctrl-t on the hanging process? That should print the stack= trace indicating where it is waiting on. >=20 > =20 > So, I rebooted the machine this morning, but now have [tried to] log into= it to check on it and find that an ssh connection doesn=E2=80=99t result i= n a shell. I logged into the console, tried to start a =E2=80=9Cscreen=E2= =80=9D to get more prompts, and it hung. Ctrl-T on that shows (after runni= ng a console screen-capture through OCR, and hand correction, so may not be= 100%): > =20 > root@host:~ # screen > load: 0.07 cmd: csh 56116 [vmwait] 35.00r 0.00u 0.01s 0% 3984k > mi_switch+0xc1 _sleep+0x1cb vm_wait_doms+0xe2 vm_wait_domain+0x51 vm_doma= in_alloc_fail+0x86 vm_page_alloc_domain_after+0x7e uma_small_alloc+0x58 keg= _alloc_slab+0xba zone_import+0xee zone_alloc_item+0x6f malloc+0x5d sigacts_= alloc+0x1c fork1+0x9fb sys_fork+0x54 amd64_syscall+0x10c fast_syscall_commo= n+0xf8=20 > =20 > As before, ps and even mount and df work here on console. But, a =E2=80= =9Czpool status tank=E2=80=9D will hang as before. A Ctrl+D on it > =20 > root@host:~ # screen > load: 0.00 cmd: zpool 62829 [aw.aew_cv] 37.89r 0.00u 0.00s 0% 6976k > mi_switch+0xc1 _cv_wait+0xf2 arc_wait_for_eviction+0x14a arc_get_data_imp= l+0xdb arc_hdr_alloc_abd+0xa6 arc_hdr_alloc+0x11e arc_read+0x4f4 dbuf_read+= 0xc08 dmu_buf_hold+0x46 zap_lookup_norm+0x35 zap_contains+0x26 vdev_rebuild= _get_stats+0xac vdev_config_generate+0x3e9 vdev_config_generate+0x74f spa_c= onfig_generate+0x2a2 spa_open_common+0x25c spa_get_stats+0x4e zfs_ioc_pool_= stats+0x22 > =20 > =20 > =20 >>=20 >> On Nov 11, 2021, at 14:10, Dave Cottlehuber <> wrote: >> =20 >> Grab output of =E2=80=98procstat-kk=E2=80=99 and see if this is similar = to https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D258208 a few more p= rods might get this one addressed! >=20 > procstat -kk 62829 yields the same as above. Which I presume is expected= , I=E2=80=99d just never used procstate -kk before. > =20 > Unfortunately, I can=E2=80=99t tell if this is sufficiently similar to bu= g 258208. A different ZFS operation is happening here, so the calls behind= my zpool status are different. The other non-zfs stat above (screen in my= case) doesn=E2=80=99t seem to be hitting zfs at all, but I may be missing = something. Andriy, Mark J, let me know if you think this is relevant, I ca= n build a 13-STABLE with D32931 if you think it will be of use. > =20 > Thanks. Let me know any thoughts you have. > =20 > - Chris > =20 Hi, Interesting. The details of these stacktraces are unknown to me. But it loo= ks like it is waiting for available memory in both cases. What is the memor= y usage of the system while all this is happening. Is it swapping a lot? And what is the real setup of the disks? Are things like GELI used (not tha= t the stack shows that) or swap-on-zfs? And is there something else interesting in the logs than "swap_pager: canno= t allocate bio"? Maybe a reason why it can't allocate the bio. I would not know a pointer on how to debug this except for checking tools l= ike iostat, vmstat, etc.. Of course running 13-STABLE can give an interesti= ng data point. Regards, Ronald. =20 ------=_Part_14_1187132451.1636709757927-- From nobody Fri Nov 12 13:49:17 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 6A47F1850E3C for ; Fri, 12 Nov 2021 13:49:28 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from relay.wiredblade.com (relay.wiredblade.com [168.235.95.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4HrKh818Qtz3NHW for ; Fri, 12 Nov 2021 13:49:28 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from mail.distal.com (pool-108-48-165-176.washdc.fios.verizon.net [108.48.165.176]) by relay.wiredblade.com with ESMTPSA (version=TLSv1.2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256) ; Fri, 12 Nov 2021 13:49:21 +0000 Received: from smtpclient.apple ( [2001:420:c0c4:1004::ab]) by tristain.distal.com (OpenSMTPD) with ESMTPSA id 0e9ebacb (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Fri, 12 Nov 2021 08:49:19 -0500 (EST) Content-Type: text/plain; charset=utf-8 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 15.0 \(3693.20.0.1.32\)) Subject: Re: swap_pager: cannot allocate bio From: Chris Ross X-Priority: 3 (Normal) In-Reply-To: <42006135.15.1636709757975@mailrelay> Date: Fri, 12 Nov 2021 08:49:17 -0500 Cc: freebsd-fs Content-Transfer-Encoding: quoted-printable Message-Id: <7B41B7D7-0C74-4F87-A49C-A666DB970CC3@distal.com> References: <9FE99EEF-37C5-43D1-AC9D-17F3EDA19606@distal.com> <09989390-FED9-45A6-A866-4605D3766DFE@distal.com> <4E5511DF-B163-4928-9CC3-22755683999E@distal.com> <42006135.15.1636709757975@mailrelay> To: ronald-lists@klop.ws X-Mailer: Apple Mail (2.3693.20.0.1.32) X-Rspamd-Queue-Id: 4HrKh818Qtz3NHW X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; TAGGED_FROM(0.00)[freebsd]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N >> root@host:~ # screen >> load: 0.07 cmd: csh 56116 [vmwait] 35.00r 0.00u 0.01s 0% 3984k >> mi_switch+0xc1 _sleep+0x1cb vm_wait_doms+0xe2 vm_wait_domain+0x51 = vm_domain_alloc_fail+0x86 vm_page_alloc_domain_after+0x7e = uma_small_alloc+0x58 keg_alloc_slab+0xba zone_import+0xee = zone_alloc_item+0x6f malloc+0x5d sigacts_alloc+0x1c fork1+0x9fb = sys_fork+0x54 amd64_syscall+0x10c fast_syscall_common+0xf8 As before, = ps and even mount and df work here on console. But, a =E2=80=9Czpool = status tank=E2=80=9D will hang as before. A Ctrl+D on it >> load: 0.00 cmd: zpool 62829 [aw.aew_cv] 37.89r 0.00u 0.00s 0% 6976k >> mi_switch+0xc1 _cv_wait+0xf2 arc_wait_for_eviction+0x14a = arc_get_data_impl+0xdb arc_hdr_alloc_abd+0xa6 arc_hdr_alloc+0x11e = arc_read+0x4f4 dbuf_read+0xc08 dmu_buf_hold+0x46 zap_lookup_norm+0x35 = zap_contains+0x26 vdev_rebuild_get_stats+0xac vdev_config_generate+0x3e9 = vdev_config_generate+0x74f spa_config_generate+0x2a2 = spa_open_common+0x25c spa_get_stats+0x4e zfs_ioc_pool_stats+0x22 > Hi, >=20 > Interesting. The details of these stacktraces are unknown to me. But = it looks like it is waiting for available memory in both cases. What is = the memory usage of the system while all this is happening. Is it = swapping a lot? > And what is the real setup of the disks? Are things like GELI used = (not that the stack shows that) or swap-on-zfs? It=E2=80=99s pretty simple. No GELI, just three 3-disk raidz=E2=80=99s. = And swap is a partition on a physical (ish: hardware RAID1) disk, which = is also where the OS and everything other than the one large ZFS = filesystem are. > And is there something else interesting in the logs than "swap_pager: = cannot allocate bio"? Maybe a reason why it can't allocate the bio. Not that I saw. A new execution of procstat -kk (started yesterday), as = well as a dmesg, both hang now. They seem to be stuck with the same = stack-trace as screen is. And the zpool status shows the same stack = with Ctrl-T as it has. Looking at the logs now, Since I rebooted the = system 24 hours ago, there are no kernel logs after the failure that = began yesterday afternoon. Apparently, this is a reproducible problem, = it takes a day or less to get stuck. So, that=E2=80=99s valuable in a = way. ;-) =20 > I would not know a pointer on how to debug this except for checking = tools like iostat, vmstat, etc.. Of course running 13-STABLE can give an = interesting data point. So, tl;dr; no data from the most recent hang other than what the = stack-traces show. Not even the =E2=80=9Ccannot allocate bio=E2=80=9D I = saw two days ago after increasing swap size. I can take a look at = 13-STABLE, when I give up on this and reboot (likely today) I=E2=80=99ll = try building that. - Chris From nobody Fri Nov 12 16:15:31 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 2C46A18590B7 for ; Fri, 12 Nov 2021 16:15:44 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ua1-x931.google.com (mail-ua1-x931.google.com [IPv6:2607:f8b0:4864:20::931]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HrNww0Qxwz3J3k for ; Fri, 12 Nov 2021 16:15:43 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-ua1-x931.google.com with SMTP id i6so19849353uae.6 for ; Fri, 12 Nov 2021 08:15:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=WBFQh2MX1xLIuAxAi8aaOr7HN/NsuFiJzLx97B8O1GU=; b=6BykbJZSQqZ1p/W2UnzVRDk9QcLFumuINeukBM3AlF/+59lJPgodK9IVt49Aw7mJWa aVwaTScevSfntn2yg9TJHR/TJg11uI96YLJVEeNVqUOeUeVOpqfjomiIOvW7zenZJkmG Q0o2Xdm2dfFMToyxNyTpZ1NwYT4DgSrdZbzdwTt7w35GgV669Fsls5KRtjCQydSQQ0LI V+SHxslDGeBG04Ioqm2n5KNFirpGFUqooH7kBY3/b7QZOKl2GYzIsEmOCNbCFx2emg8b 6dbCcDJtOksTr9hx/2l2/OFK3k+8Ud/opBlYwasa4dlR59R5mEeZWeAkrSwThigoIc+D ZEHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=WBFQh2MX1xLIuAxAi8aaOr7HN/NsuFiJzLx97B8O1GU=; b=hjj6UphkqY0UdJqmZJ24VLF6XEa4zXvFHLrejIGHLn8L2kZQ6/eb1pl3Is6FASgbiJ ByE6HF+C6Y6kGM98noeyO8o8GP7pXBuniOKS/1ceCCtVoEkq+EoeyE8RW8eGHDL/dF/q uEHjer7Ur9kaSZkaH+9Ja7dvY1KvmyEcHV7lRjuOxicVi4RvDQTR5P7WtprtubxHEhXa iwtWEVUuU6RcnD9b7/ukAnJGXbdLk3tAtxwCo8Dh1JlTnEQk/DaxH526sCFrb1wjJk1d jXRR/0N58sw2T+LdfXxlboM/jxxAYGxPHu9XAkt5t8CLefkTsaNdyBkmqMh46lhqdkCK w9yQ== X-Gm-Message-State: AOAM5319QEE8pDWbrHCCh8nIST8G/Z/cMkPdQT1HOwV7tRXAqpk5kxCm AsMRbdSa6UQMFdSFwFjM6i3b5tpsLHzAVwYiP6EUnu+F2nY= X-Google-Smtp-Source: ABdhPJyelRmk+NvnhnizdJE/llgsp719FvHGmQEtrCJr4KIAmuaKgnTL+mpOJIcsKk1UjGd1VFDWOoR+8g6Sme5F01w= X-Received: by 2002:a05:6102:5f2:: with SMTP id w18mr12045951vsf.6.1636733743353; Fri, 12 Nov 2021 08:15:43 -0800 (PST) List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 References: <9FE99EEF-37C5-43D1-AC9D-17F3EDA19606@distal.com> <09989390-FED9-45A6-A866-4605D3766DFE@distal.com> <4E5511DF-B163-4928-9CC3-22755683999E@distal.com> <42006135.15.1636709757975@mailrelay> <7B41B7D7-0C74-4F87-A49C-A666DB970CC3@distal.com> In-Reply-To: <7B41B7D7-0C74-4F87-A49C-A666DB970CC3@distal.com> From: Warner Losh Date: Fri, 12 Nov 2021 09:15:31 -0700 Message-ID: Subject: Re: swap_pager: cannot allocate bio To: Chris Ross Cc: Ronald Klop , freebsd-fs Content-Type: multipart/alternative; boundary="0000000000005d6b1905d099c5e3" X-Rspamd-Queue-Id: 4HrNww0Qxwz3J3k X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; TAGGED_RCPT(0.00)[freebsd]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: Y --0000000000005d6b1905d099c5e3 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Nov 12, 2021, 6:49 AM Chris Ross wrote: > > >> root@host:~ # screen > >> load: 0.07 cmd: csh 56116 [vmwait] 35.00r 0.00u 0.01s 0% 3984k > >> mi_switch+0xc1 _sleep+0x1cb vm_wait_doms+0xe2 vm_wait_domain+0x51 > vm_domain_alloc_fail+0x86 vm_page_alloc_domain_after+0x7e > uma_small_alloc+0x58 keg_alloc_slab+0xba zone_import+0xee > zone_alloc_item+0x6f malloc+0x5d sigacts_alloc+0x1c fork1+0x9fb > sys_fork+0x54 amd64_syscall+0x10c fast_syscall_common+0xf8 As before, ps > and even mount and df work here on console. But, a =E2=80=9Czpool status= tank=E2=80=9D > will hang as before. A Ctrl+D on it > > >> load: 0.00 cmd: zpool 62829 [aw.aew_cv] 37.89r 0.00u 0.00s 0% 6976k > >> mi_switch+0xc1 _cv_wait+0xf2 arc_wait_for_eviction+0x14a > arc_get_data_impl+0xdb arc_hdr_alloc_abd+0xa6 arc_hdr_alloc+0x11e > arc_read+0x4f4 dbuf_read+0xc08 dmu_buf_hold+0x46 zap_lookup_norm+0x35 > zap_contains+0x26 vdev_rebuild_get_stats+0xac vdev_config_generate+0x3e9 > vdev_config_generate+0x74f spa_config_generate+0x2a2 spa_open_common+0x25= c > spa_get_stats+0x4e zfs_ioc_pool_stats+0x22 > > > Hi, > > > > Interesting. The details of these stacktraces are unknown to me. But it > looks like it is waiting for available memory in both cases. What is the > memory usage of the system while all this is happening. Is it swapping a > lot? > > And what is the real setup of the disks? Are things like GELI used (not > that the stack shows that) or swap-on-zfs? > > It=E2=80=99s pretty simple. No GELI, just three 3-disk raidz=E2=80=99s. = And swap is a > partition on a physical (ish: hardware RAID1) disk, which is also where t= he > OS and everything other than the one large ZFS filesystem are. > > > And is there something else interesting in the logs than "swap_pager: > cannot allocate bio"? Maybe a reason why it can't allocate the bio. > > Not that I saw. A new execution of procstat -kk (started yesterday), as > well as a dmesg, both hang now. They seem to be stuck with the same > stack-trace as screen is. And the zpool status shows the same stack with > Ctrl-T as it has. Looking at the logs now, Since I rebooted the system 2= 4 > hours ago, there are no kernel logs after the failure that began yesterda= y > afternoon. Apparently, this is a reproducible problem, it takes a day or > less to get stuck. So, that=E2=80=99s valuable in a way. ;-) > > > I would not know a pointer on how to debug this except for checking > tools like iostat, vmstat, etc.. Of course running 13-STABLE can give an > interesting data point. > > So, tl;dr; no data from the most recent hang other than what the > stack-traces show. Not even the =E2=80=9Ccannot allocate bio=E2=80=9D I = saw two days ago > after increasing swap size. I can take a look at 13-STABLE, when I give > up on this and reboot (likely today) I=E2=80=99ll try building that. > So the root cause of this problem is well known. You have a memory shortage, so you want to page out dirty pages to reclaim memory. However, there's not enough memory to allocate the structures you need to do I/O and so the swapout I/O fails half way down the stack not being able to allocate a bio. Some paths through the swapper cope with this well, other parts that execute less often cope less well. There's some hacks in the tree today to help with the GELI case: we prioritize swapping I/O. But there's no g_alloc_bio_swapping() interface for swapping I/O to get priority on allocating a bio to start with. Places that use g_clone_bio() could have the clone's copy allocated from a special swap pool, but that starts to get messy and isn't done today. And the upper layers like geom_cfs and ZFS are inconsistent in allocations, so there's work needed to make it robust in ZFS, but I have only a vague notion of what's needed. At the very least, the swapping I/O that comes into the top of ZFS won't have swapping I/O marked coming out the bottom because the BIO_SWAP flag is quite new. So until then, swapping on zvols is fraught with deadlocks like this and in the past there's been a strong admonishment against it. Warner - Chris > > > > --0000000000005d6b1905d099c5e3-- From nobody Fri Nov 12 19:50:00 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id F03201859C5E for ; Fri, 12 Nov 2021 19:50:11 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from relay.wiredblade.com (relay.wiredblade.com [168.235.95.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4HrThM4r7mz3pkJ for ; Fri, 12 Nov 2021 19:50:11 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from mail.distal.com (pool-108-48-165-176.washdc.fios.verizon.net [108.48.165.176]) by relay.wiredblade.com with ESMTPSA (version=TLSv1.2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256) ; Fri, 12 Nov 2021 19:50:04 +0000 Received: from smtpclient.apple ( [2001:420:c0c4:1004::ab]) by tristain.distal.com (OpenSMTPD) with ESMTPSA id 65eabce2 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Fri, 12 Nov 2021 14:50:02 -0500 (EST) Content-Type: text/plain; charset=utf-8 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 15.0 \(3693.20.0.1.32\)) Subject: Re: swap_pager: cannot allocate bio From: Chris Ross In-Reply-To: Date: Fri, 12 Nov 2021 14:50:00 -0500 Cc: Ronald Klop , freebsd-fs Content-Transfer-Encoding: quoted-printable Message-Id: <4008C512-31F1-4BE3-B674-A270CF674757@distal.com> References: <9FE99EEF-37C5-43D1-AC9D-17F3EDA19606@distal.com> <09989390-FED9-45A6-A866-4605D3766DFE@distal.com> <4E5511DF-B163-4928-9CC3-22755683999E@distal.com> <42006135.15.1636709757975@mailrelay> <7B41B7D7-0C74-4F87-A49C-A666DB970CC3@distal.com> To: Warner Losh X-Mailer: Apple Mail (2.3693.20.0.1.32) X-Rspamd-Queue-Id: 4HrThM4r7mz3pkJ X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[freebsd] X-ThisMailContainsUnwantedMimeParts: N > On Nov 12, 2021, at 11:15, Warner Losh wrote: > So the root cause of this problem is well known. You have a memory = shortage, so you want to page out dirty pages to reclaim memory. > However, there's not enough memory to allocate the structures you need = to do I/O and so the swapout I/O fails half way down > the stack not being able to allocate a bio. Some paths through the = swapper cope with this well, other parts that execute less > often cope less well. >=20 > There's some hacks in the tree today to help with the GELI case: we = prioritize swapping I/O. But there's no g_alloc_bio_swapping() interface > for swapping I/O to get priority on allocating a bio to start with. = Places that use g_clone_bio() could have the clone's copy allocated > from a special swap pool, but that starts to get messy and isn't done = today. And the upper layers like geom_cfs and ZFS are > inconsistent in allocations, so there's work needed to make it robust = in ZFS, but I have only a vague notion of what's needed. At the very > least, the swapping I/O that comes into the top of ZFS won't have = swapping I/O marked coming out the bottom because the > BIO_SWAP flag is quite new. >=20 > So until then, swapping on zvols is fraught with deadlocks like this = and in the past there's been a strong admonishment > against it. Apologies, Warner, but I=E2=80=99m not sure I=E2=80=99m understanding = this last statement. If you mean swapping _onto_ zvols, I=E2=80=99m not = doing that. If you mean swapping in any way, while having zvols, then = yes I am doing that. =20 My swap is on a partition on the non-ZFS disk. A physical disk as far = as the kernel knows, hardware RAID1. # pstat -s Device 1K-blocks Used Avail Capacity /dev/da0p3 445682648 1018524 444664124 0% Let me know if what you=E2=80=99re saying above is true to my case, and = any advice as to how I can avoid it. I had a =E2=80=9Cnot enough swap = space=E2=80=9D a while back, and accordingly increased the size of my = swap partition. I have 128GB of memory, though between the ARC and the = big process I was running, that fills it easily. - Chris From nobody Fri Nov 12 19:52:52 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id BA10A185A147 for ; Fri, 12 Nov 2021 19:53:04 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-vk1-xa32.google.com (mail-vk1-xa32.google.com [IPv6:2607:f8b0:4864:20::a32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HrTlh4bg0z3qMZ for ; Fri, 12 Nov 2021 19:53:04 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-vk1-xa32.google.com with SMTP id bc10so5719930vkb.1 for ; Fri, 12 Nov 2021 11:53:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20210112.gappssmtp.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=8A4UEJDhUSBySq3W4OssQpyAn/HvTVRLW0A+ROqVazQ=; b=Ad2LsUyzu3yx3PAbiathpAt3Qurn5lVBPdqBdb+3E0MmRGgN6XzanU2Eautg0/k1tx xL5KEXCwsQ4djC2vcT38z8DV9UUrcqpwV+hkm9AbfAf3+eytaqaGlofHTAQvyl+xqJwR LErNNSoUj3I4shzWr8tllOJJV3HQlVSpexQjhmbI1sXYZZkfQz6K5T8NTZMfs00CoUUj nAfUVAIT/aNfftDQSJOzECfoEuYieHPGtrNpzBCefF8w35Y7oaTv+OXCC9Sv3ac14BrL 9gD9ycX6oNI153m0ijNcrqRZBUS+lZZY2wDwaTPg7c6u5MEKKV6PKbfW41b5onhB65Mz 9fhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=8A4UEJDhUSBySq3W4OssQpyAn/HvTVRLW0A+ROqVazQ=; b=XbBWJH4+59J+1+clfX8g+usOIjCR7OTava2sYjo9mfTRVlZjO7LApuj3GbD4A73v8v EAfLb5EqKL3WsS62BLOGkb74RIS6ZCviFug8wooO/gUsFZmsbNS6MxUxcaGoMVZyWmZX pFLVkGgpMuKx6wNZZCyoJCaZT2TLkDCOaT49v9AQ6p6X2teqCfv9iYttd+RIo7vLsiW0 aiuQwBbGnA0EVnzPeHwMludtmla2ZlYgeuv9Pk/1zl/3nH3IK+kLwuDIFxiPONj+t/md rRgwuQ/f/xN9PdEgKVfEu1mkXXcf7E4ny0kOPxPTjIwSQMKWOEhM1pw33PI2Peo+IZui L1DA== X-Gm-Message-State: AOAM5312neKt6SXYEMRbVoRDtwCgeTrngnmKOprV/of8VDdCZEqzhatT RkgwZXnxHR5/z40AlIUFUADvoE5cxv38pX8bYUDAKQgwrS8= X-Google-Smtp-Source: ABdhPJxM9X6XdbKAm+rnsbumtNDBPynSUHymR/kJNEo+1nlZYkAG3/SwkXgV8fR+m377Ytkoo53OmxLpGKDs2dyhmpo= X-Received: by 2002:a05:6122:78c:: with SMTP id k12mr27347502vkr.22.1636746784041; Fri, 12 Nov 2021 11:53:04 -0800 (PST) List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 References: <9FE99EEF-37C5-43D1-AC9D-17F3EDA19606@distal.com> <09989390-FED9-45A6-A866-4605D3766DFE@distal.com> <4E5511DF-B163-4928-9CC3-22755683999E@distal.com> <42006135.15.1636709757975@mailrelay> <7B41B7D7-0C74-4F87-A49C-A666DB970CC3@distal.com> <4008C512-31F1-4BE3-B674-A270CF674757@distal.com> In-Reply-To: <4008C512-31F1-4BE3-B674-A270CF674757@distal.com> From: Warner Losh Date: Fri, 12 Nov 2021 12:52:52 -0700 Message-ID: Subject: Re: swap_pager: cannot allocate bio To: Chris Ross Cc: Ronald Klop , freebsd-fs Content-Type: multipart/alternative; boundary="000000000000a67ddf05d09ccebc" X-Rspamd-Queue-Id: 4HrTlh4bg0z3qMZ X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; TAGGED_RCPT(0.00)[freebsd]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: Y --000000000000a67ddf05d09ccebc Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Nov 12, 2021 at 12:50 PM Chris Ross wrote: > > > > On Nov 12, 2021, at 11:15, Warner Losh wrote: > > So the root cause of this problem is well known. You have a memory > shortage, so you want to page out dirty pages to reclaim memory. > > However, there's not enough memory to allocate the structures you need > to do I/O and so the swapout I/O fails half way down > > the stack not being able to allocate a bio. Some paths through the > swapper cope with this well, other parts that execute less > > often cope less well. > > > > There's some hacks in the tree today to help with the GELI case: we > prioritize swapping I/O. But there's no g_alloc_bio_swapping() interface > > for swapping I/O to get priority on allocating a bio to start with. > Places that use g_clone_bio() could have the clone's copy allocated > > from a special swap pool, but that starts to get messy and isn't done > today. And the upper layers like geom_cfs and ZFS are > > inconsistent in allocations, so there's work needed to make it robust i= n > ZFS, but I have only a vague notion of what's needed. At the very > > least, the swapping I/O that comes into the top of ZFS won't have > swapping I/O marked coming out the bottom because the > > BIO_SWAP flag is quite new. > > > > So until then, swapping on zvols is fraught with deadlocks like this an= d > in the past there's been a strong admonishment > > against it. > > Apologies, Warner, but I=E2=80=99m not sure I=E2=80=99m understanding thi= s last > statement. If you mean swapping _onto_ zvols, I=E2=80=99m not doing that= . If you > mean swapping in any way, while having zvols, then yes I am doing that. > > My swap is on a partition on the non-ZFS disk. A physical disk as far as > the kernel knows, hardware RAID1. > > # pstat -s > Device 1K-blocks Used Avail Capacity > /dev/da0p3 445682648 1018524 444664124 0% > OK. That's well supported and should work w/o some of the issues that I raised. I'd misunderstood and thought you were swapping to zvols... > Let me know if what you=E2=80=99re saying above is true to my case, and a= ny advice > as to how I can avoid it. I had a =E2=80=9Cnot enough swap space=E2=80= =9D a while back, > and accordingly increased the size of my swap partition. I have 128GB of > memory, though between the ARC and the big process I was running, that > fills it easily. > Yea, this is a 'memory is exhausted' problem, and more swap won't help that. It's unclear why we run out so fast, and why the separate zones for the bio isn't providing a good level of insulation from out of memory scenarios. Warner --000000000000a67ddf05d09ccebc-- From nobody Fri Nov 12 19:59:30 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id BE7C7185D8FA for ; Fri, 12 Nov 2021 19:59:35 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from relay.wiredblade.com (relay.wiredblade.com [168.235.95.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4HrTvC3wfKz3sRM for ; Fri, 12 Nov 2021 19:59:35 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from mail.distal.com (pool-108-48-165-176.washdc.fios.verizon.net [108.48.165.176]) by relay.wiredblade.com with ESMTPSA (version=TLSv1.2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256) ; Fri, 12 Nov 2021 19:59:34 +0000 Received: from smtpclient.apple ( [2001:420:c0c4:1004::33c]) by tristain.distal.com (OpenSMTPD) with ESMTPSA id 6c544083 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Fri, 12 Nov 2021 14:59:32 -0500 (EST) From: Chris Ross Message-Id: <953DD67A-1A37-4D03-B878-E65396641B7D@distal.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_8E68A48E-A757-49F9-978D-9D0329E5539A" List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 15.0 \(3693.20.0.1.32\)) Subject: Re: swap_pager: cannot allocate bio Date: Fri, 12 Nov 2021 14:59:30 -0500 In-Reply-To: Cc: Ronald Klop , freebsd-fs To: Warner Losh References: <9FE99EEF-37C5-43D1-AC9D-17F3EDA19606@distal.com> <09989390-FED9-45A6-A866-4605D3766DFE@distal.com> <4E5511DF-B163-4928-9CC3-22755683999E@distal.com> <42006135.15.1636709757975@mailrelay> <7B41B7D7-0C74-4F87-A49C-A666DB970CC3@distal.com> <4008C512-31F1-4BE3-B674-A270CF674757@distal.com> X-Mailer: Apple Mail (2.3693.20.0.1.32) X-Rspamd-Queue-Id: 4HrTvC3wfKz3sRM X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[freebsd] X-ThisMailContainsUnwantedMimeParts: Y --Apple-Mail=_8E68A48E-A757-49F9-978D-9D0329E5539A Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Nov 12, 2021, at 14:52, Warner Losh wrote: > My swap is on a partition on the non-ZFS disk. A physical disk as far = as the kernel knows, hardware RAID1. >=20 > # pstat -s > Device 1K-blocks Used Avail Capacity > /dev/da0p3 445682648 1018524 444664124 0% >=20 > OK. That's well supported and should work w/o some of the issues that = I raised. I'd misunderstood and thought you were swapping to zvols... > =20 > Let me know if what you=E2=80=99re saying above is true to my case, = and any advice as to how I can avoid it. I had a =E2=80=9Cnot enough = swap space=E2=80=9D a while back, and accordingly increased the size of = my swap partition. I have 128GB of memory, though between the ARC and = the big process I was running, that fills it easily. >=20 > Yea, this is a 'memory is exhausted' problem, and more swap won't help = that. It's unclear why we run out so fast, and why the separate zones = for the bio isn't providing a good level of insulation from out of = memory scenarios. Okay. Well, I can=E2=80=99t easily add more memory to this machine, = though I am investigating it. I certainly can=E2=80=99t do it in short = order. I presume the problem is that I recently increased the size of = this pool by adding a large raidz vdev to it. I=E2=80=99ve only been = seeing this since. Is there any way I can =E2=80=9Climit=E2=80=9D the = perceived size of the ZFS filesystem to ease the problem? Is there = anything I can tune to help? Can I turn off or drastically reduce the = ARC? A decrease in performance would be better than getting stuck after = a day or so. :-) - Chris --Apple-Mail=_8E68A48E-A757-49F9-978D-9D0329E5539A-- From nobody Fri Nov 12 20:10:50 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 1FA8D183D398 for ; Fri, 12 Nov 2021 20:10:55 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-il1-x133.google.com (mail-il1-x133.google.com [IPv6:2607:f8b0:4864:20::133]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HrV8H09ccz4SXJ for ; Fri, 12 Nov 2021 20:10:55 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-il1-x133.google.com with SMTP id l8so10115638ilv.3 for ; Fri, 12 Nov 2021 12:10:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:content-transfer-encoding:in-reply-to; bh=lBha5EHXdzvQqpyz/VQUzBHzFu8ZuihEbgL8oIYBv6I=; b=a6jD7VfXT/mAPdtO6nvOcsVL0KvjmD1heq//h0eRs0nW/1mYPQZRxXVUQky21f62kx gfqF9+7q75Z8LsJ+5laFk/q4EXxHGH6JpTIV5h0BIA3axaNyHzR5Txh4eIwIS+r5BKr9 FMefuoYptBsHW1IgwPh9Pm7vpTnew2jTBBRAYG+p1pmz2uDDi4UN+VqnwfWkPGaWKLmF 9bYU6x5pfgwwkk/fwW0qGgqKRizNkV+wYcfoaLhqRxgNtU2DLFkS1GCRtcmfGqvLQUND SIHu/+7z8Sr/n8S81Dq9sAXants+YY4wB+r+OCWQ8xMTmUUUlBoXUvp0/E9MuPsr0dII tN4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition :content-transfer-encoding:in-reply-to; bh=lBha5EHXdzvQqpyz/VQUzBHzFu8ZuihEbgL8oIYBv6I=; b=MtzSSMw+fpclm0OBPKWgJfe2V9Wx/fQrBaaJ53v3BWYmlC/t6/2NxWcSRCRZhOhNtW 0qFSABtFszXk3k0NJdAvID7QWfOyGvCZIwtCgtVfwjlnmlXiZ42UbnhrURaWrq5gokJY RkQ+O6Cwe2SyT+DcodVMTbMZzlm0YDzs9/xHp3SO9AMOgDaZYw+1ifpLHwjBfOISNMLh wmuKhTQ65GQpwIc/gxVf13s2ncqBpZrXf4T8jTAh96bx5Yb4MCjDSp4xBkRe+ycyQTBD UgZnzgaRVVBxzm6rgDrTCrN6ks/j+A+dTTJ7CfJDJ6fok8T+lPMj6ZiXCCsL2inaoxuN 9riA== X-Gm-Message-State: AOAM532qNGHt9pKf6ad3el6MC4KRuKCAxLC8Lvr4gO+pSyzH8orktS3O 9/bSqbwVCXYIDgKX7r7EAZcAsDKoSzg= X-Google-Smtp-Source: ABdhPJyt5B+ZfBhiSbcIJdvYI0U9d3r0OfYHgPVnbc5wpY2gyWBFW0scMaIWfgaWFkt0sBbV71vgdQ== X-Received: by 2002:a05:6e02:1c0e:: with SMTP id l14mr10325843ilh.8.1636747852916; Fri, 12 Nov 2021 12:10:52 -0800 (PST) Received: from nuc ([142.126.186.191]) by smtp.gmail.com with ESMTPSA id f2sm3431248ilu.54.2021.11.12.12.10.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Nov 2021 12:10:52 -0800 (PST) Date: Fri, 12 Nov 2021 15:10:50 -0500 From: Mark Johnston To: Chris Ross Cc: ronald-lists@klop.ws, freebsd-fs Subject: Re: swap_pager: cannot allocate bio Message-ID: References: <9FE99EEF-37C5-43D1-AC9D-17F3EDA19606@distal.com> <09989390-FED9-45A6-A866-4605D3766DFE@distal.com> <4E5511DF-B163-4928-9CC3-22755683999E@distal.com> List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <4E5511DF-B163-4928-9CC3-22755683999E@distal.com> X-Rspamd-Queue-Id: 4HrV8H09ccz4SXJ X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; TAGGED_RCPT(0.00)[freebsd]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Thu, Nov 11, 2021 at 04:49:21PM -0500, Chris Ross wrote: > > > > On Nov 11, 2021, at 13:50, Ronald Klop via freebsd-fs wrote: > > > > Can you press ctrl-t on the hanging process? That should print the stacktrace indicating where it is waiting on. > > So, I rebooted the machine this morning, but now have [tried to] log into it to check on it and find that an ssh connection doesn’t result in a shell. I logged into the console, tried to start a “screen” to get more prompts, and it hung. Ctrl-T on that shows (after running a console screen-capture through OCR, and hand correction, so may not be 100%): > > root@host:~ # screen > load: 0.07 cmd: csh 56116 [vmwait] 35.00r 0.00u 0.01s 0% 3984k > mi_switch+0xc1 _sleep+0x1cb vm_wait_doms+0xe2 vm_wait_domain+0x51 vm_domain_alloc_fail+0x86 vm_page_alloc_domain_after+0x7e uma_small_alloc+0x58 keg_alloc_slab+0xba zone_import+0xee zone_alloc_item+0x6f malloc+0x5d sigacts_alloc+0x1c fork1+0x9fb sys_fork+0x54 amd64_syscall+0x10c fast_syscall_common+0xf8 > > As before, ps and even mount and df work here on console. But, a “zpool status tank” will hang as before. A Ctrl+D on it > > root@host:~ # screen > load: 0.00 cmd: zpool 62829 [aw.aew_cv] 37.89r 0.00u 0.00s 0% 6976k > mi_switch+0xc1 _cv_wait+0xf2 arc_wait_for_eviction+0x14a arc_get_data_impl+0xdb arc_hdr_alloc_abd+0xa6 arc_hdr_alloc+0x11e arc_read+0x4f4 dbuf_read+0xc08 dmu_buf_hold+0x46 zap_lookup_norm+0x35 zap_contains+0x26 vdev_rebuild_get_stats+0xac vdev_config_generate+0x3e9 vdev_config_generate+0x74f spa_config_generate+0x2a2 spa_open_common+0x25c spa_get_stats+0x4e zfs_ioc_pool_stats+0x22 > > > > > On Nov 11, 2021, at 14:10, Dave Cottlehuber <> wrote: > > > > Grab output of ‘procstat-kk’ and see if this is similar to https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=258208 a few more prods might get this one addressed! > > procstat -kk 62829 yields the same as above. Which I presume is expected, I’d just never used procstate -kk before. > > Unfortunately, I can’t tell if this is sufficiently similar to bug 258208. A different ZFS operation is happening here, so the calls behind my zpool status are different. The other non-zfs stat above (screen in my case) doesn’t seem to be hitting zfs at all, but I may be missing something. Andriy, Mark J, let me know if you think this is relevant, I can build a 13-STABLE with D32931 if you think it will be of use. No, this looks like a different problem. If it's possible to reproduce this and procstat -kka is usable, it would be helpful to see the full output. In particular, I am wondering if the page daemon is getting blocked waiting for the arc evict handler to successfully allocate memory. https://cgit.freebsd.org/src/commit/?id=97ed4babb51636d8a4b11bc7b207c3219ffcd0e3 is an example of a fix for such a problem, and is not present in 13.0. I would also suggest trying to apply that patch, though I'm fairly sure there are other such problems still lurking. > Thanks. Let me know any thoughts you have. > > - Chris > From nobody Fri Nov 12 21:47:49 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 93A5C18508A6 for ; Fri, 12 Nov 2021 21:47:59 +0000 (UTC) (envelope-from pete@nomadlogic.org) Received: from mail.nomadlogic.org (mail.nomadlogic.org [66.165.241.226]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail.nomadlogic.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HrXJH2QJyz3GjD for ; Fri, 12 Nov 2021 21:47:59 +0000 (UTC) (envelope-from pete@nomadlogic.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nomadlogic.org; s=04242021; t=1636753671; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WfMLWDIm+D62d3HSAY8401ZaJl/6bbKNYaJpiQniSaE=; b=06yiLh5/bCjratPeqQIPL+5z8uOuBj/iZNrWln0uJC2viLX186bJAxsBh6mx4JrETqT/Dq wKSmOJcT1KiNf1VwPJlYsgCDMzN/cMv04Hm4BafBfOHgp6QCEfTPHv8GVY+9WRqT3nBZ8Z L88oW9GkjIP4cTF4VFEbtEVoPFlmkz0= Received: from [192.168.1.160] (cpe-24-24-163-126.socal.res.rr.com [24.24.163.126]) by mail.nomadlogic.org (OpenSMTPD) with ESMTPSA id 2280925a (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Fri, 12 Nov 2021 21:47:51 +0000 (UTC) Message-ID: <3b2b6c10-4a76-e7d4-c816-82fd8965316a@nomadlogic.org> Date: Fri, 12 Nov 2021 13:47:49 -0800 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:91.0) Gecko/20100101 Thunderbird/91.3.0 Subject: Re: swap_pager: cannot allocate bio Content-Language: en-US To: Chris Ross , Warner Losh Cc: Ronald Klop , freebsd-fs References: <9FE99EEF-37C5-43D1-AC9D-17F3EDA19606@distal.com> <09989390-FED9-45A6-A866-4605D3766DFE@distal.com> <4E5511DF-B163-4928-9CC3-22755683999E@distal.com> <42006135.15.1636709757975@mailrelay> <7B41B7D7-0C74-4F87-A49C-A666DB970CC3@distal.com> <4008C512-31F1-4BE3-B674-A270CF674757@distal.com> <953DD67A-1A37-4D03-B878-E65396641B7D@distal.com> In-Reply-To: <953DD67A-1A37-4D03-B878-E65396641B7D@distal.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4HrXJH2QJyz3GjD X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; TAGGED_RCPT(0.00)[freebsd]; REPLY(-4.00)[] Reply-To: pete@nomadlogic.org From: Pete Wright via freebsd-fs X-Original-From: Pete Wright X-ThisMailContainsUnwantedMimeParts: N On 11/12/21 11:59, Chris Ross wrote: > >> On Nov 12, 2021, at 14:52, Warner Losh wrote: >> My swap is on a partition on the non-ZFS disk. A physical disk as far as the kernel knows, hardware RAID1. >> >> # pstat -s >> Device 1K-blocks Used Avail Capacity >> /dev/da0p3 445682648 1018524 444664124 0% >> >> OK. That's well supported and should work w/o some of the issues that I raised. I'd misunderstood and thought you were swapping to zvols... >> >> Let me know if what you’re saying above is true to my case, and any advice as to how I can avoid it. I had a “not enough swap space” a while back, and accordingly increased the size of my swap partition. I have 128GB of memory, though between the ARC and the big process I was running, that fills it easily. >> >> Yea, this is a 'memory is exhausted' problem, and more swap won't help that. It's unclear why we run out so fast, and why the separate zones for the bio isn't providing a good level of insulation from out of memory scenarios. > Okay. Well, I can’t easily add more memory to this machine, though I am investigating it. I certainly can’t do it in short order. I presume the problem is that I recently increased the size of this pool by adding a large raidz vdev to it. I’ve only been seeing this since. Is there any way I can “limit” the perceived size of the ZFS filesystem to ease the problem? Is there anything I can tune to help? Can I turn off or drastically reduce the ARC? A decrease in performance would be better than getting stuck after a day or so. :-) I don't think this is "the right way to do things" *but* I have begun using this sysctl to limit the size of my arc*.  the reason i say it's not the right way is because it may just paper over a real bug and preventing us from getting it fixed.  might be worth testing though to see if it helps: # 25GB arc vfs.zfs.arc.max=25000000000 cheers, -pete *my use-case is for a system running a bunch of VM's and this has allowed me to avoid swapping.  perf has been acceptable. -- Pete Wright pete@nomadlogic.org @nomadlogicLA From nobody Sat Nov 13 04:15:49 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id C95A5185C382 for ; Sat, 13 Nov 2021 04:16:01 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from relay.wiredblade.com (relay.wiredblade.com [168.235.95.80]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hrhw11Gdfz4tnw; Sat, 13 Nov 2021 04:16:00 +0000 (UTC) (envelope-from cross+freebsd@distal.com) Received: from mail.distal.com (pool-108-48-165-176.washdc.fios.verizon.net [108.48.165.176]) by relay.wiredblade.com with ESMTPSA (version=TLSv1.2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256) ; Sat, 13 Nov 2021 04:15:52 +0000 Received: from smtpclient.apple ( [2001:470:e24c:200:4cec:7b2b:1c51:2e79]) by tristain.distal.com (OpenSMTPD) with ESMTPSA id 72777776 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Fri, 12 Nov 2021 23:15:50 -0500 (EST) From: Chris Ross Message-Id: <19A3AAF6-149B-4A3C-8C27-4CFF22382014@distal.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_B97C0999-1199-44D4-89D8-7DB849CC1194" List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: swap_pager: cannot allocate bio Date: Fri, 12 Nov 2021 23:15:49 -0500 In-Reply-To: Cc: ronald-lists@klop.ws, freebsd-fs To: Mark Johnston References: <9FE99EEF-37C5-43D1-AC9D-17F3EDA19606@distal.com> <09989390-FED9-45A6-A866-4605D3766DFE@distal.com> <4E5511DF-B163-4928-9CC3-22755683999E@distal.com> X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Rspamd-Queue-Id: 4Hrhw11Gdfz4tnw X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_FROM(0.00)[freebsd] X-Spam: Yes X-ThisMailContainsUnwantedMimeParts: Y --Apple-Mail=_B97C0999-1199-44D4-89D8-7DB849CC1194 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Nov 12, 2021, at 15:10, Mark Johnston wrote: >=20 > If it's possible to reproduce this and procstat -kka is usable, it = would > be helpful to see the full output. In particular, I am wondering if = the > page daemon is getting blocked waiting for the arc evict handler to > successfully allocate memory. >=20 > = https://cgit.freebsd.org/src/commit/?id=3D97ed4babb51636d8a4b11bc7b207c321= 9ffcd0e3 = > is an example of a fix for such a problem, and is not present in 13.0. > I would also suggest trying to apply that patch, though I'm fairly = sure > there are other such problems still lurking. I=E2=80=99ve built a stable/13 as of today, and updated the system. = I=E2=80=99ll see If the problem recurs, it usually takes about 24 hours to show. If It does, I=E2=80=99ll see if I can run a procstat -kka and get it off of = the system. - Chris --Apple-Mail=_B97C0999-1199-44D4-89D8-7DB849CC1194-- From nobody Sat Nov 13 12:46:12 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 03FEB18559FA for ; Sat, 13 Nov 2021 12:46:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HrwDg6cGwz4g4y for ; Sat, 13 Nov 2021 12:46:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C467C137F5 for ; Sat, 13 Nov 2021 12:46:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 1ADCkBAs074352 for ; Sat, 13 Nov 2021 12:46:11 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 1ADCkBfh074351 for fs@FreeBSD.org; Sat, 13 Nov 2021 12:46:11 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 259276] sysutils/openzfs-kmod/: Read errors with openzfs-kmod-2021101400 Date: Sat, 13 Nov 2021 12:46:12 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: needs-qa X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: ottkostner@gmail.com X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D259276 --- Comment #7 from Ott K=C3=B6stner --- Installed on two different machines and it looks good now. --=20 You are receiving this mail because: You are on the CC list for the bug.= From nobody Sat Nov 13 12:46:29 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 234A71855ECB for ; Sat, 13 Nov 2021 12:46:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HrwF159GJz4gNM for ; Sat, 13 Nov 2021 12:46:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 924B313A90 for ; Sat, 13 Nov 2021 12:46:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 1ADCkTIM074403 for ; Sat, 13 Nov 2021 12:46:29 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 1ADCkT0R074402 for fs@FreeBSD.org; Sat, 13 Nov 2021 12:46:29 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 259276] sysutils/openzfs-kmod/: Read errors with openzfs-kmod-2021101400 Date: Sat, 13 Nov 2021 12:46:29 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: needs-qa X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: ottkostner@gmail.com X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: ports-bugs@FreeBSD.org X-Bugzilla-Flags: maintainer-feedback? X-Bugzilla-Changed-Fields: bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D259276 Ott K=C3=B6stner changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Open |Closed Resolution|--- |FIXED --=20 You are receiving this mail because: You are on the CC list for the bug.= From nobody Sat Nov 13 13:21:38 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 7AD061848830 for ; Sat, 13 Nov 2021 13:21:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hrx1Z30Ytz4r9n for ; Sat, 13 Nov 2021 13:21:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 47BC013FC8 for ; Sat, 13 Nov 2021 13:21:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 1ADDLca9090572 for ; Sat, 13 Nov 2021 13:21:38 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 1ADDLctA090571 for fs@FreeBSD.org; Sat, 13 Nov 2021 13:21:38 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 256483] zfs doesn't automount encrypted datasets Date: Sat, 13 Nov 2021 13:21:38 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: herbert@mailbox.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D256483 Herbert J. Skuhra changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |herbert@mailbox.org --- Comment #9 from Herbert J. Skuhra --- zfskeys is not listed in libexec/rc/rc.d/Makefile and tools/build/mk/OptionalObsoleteFiles.inc. Test build running... diff --git a/libexec/rc/rc.d/Makefile b/libexec/rc/rc.d/Makefile index 4c7267552526..36b8a91727dc 100644 --- a/libexec/rc/rc.d/Makefile +++ b/libexec/rc/rc.d/Makefile @@ -323,6 +323,7 @@ CONFGROUPS+=3D ZFS ZFS+=3D zfs ZFS+=3D zfsbe ZFS+=3D zfsd +ZFS+=3D zfskeys ZFS+=3D zpool ZFS+=3D zvol ZFSPACKAGE=3D rc diff --git a/tools/build/mk/OptionalObsoleteFiles.inc b/tools/build/mk/OptionalObsoleteFiles.inc index 96156a80f2e3..0cfa3428ee21 100644 --- a/tools/build/mk/OptionalObsoleteFiles.inc +++ b/tools/build/mk/OptionalObsoleteFiles.inc @@ -1118,8 +1118,9 @@ OLD_FILES+=3Dboot/gptzfsboot OLD_FILES+=3Dboot/zfsboot OLD_FILES+=3Dboot/zfsloader OLD_FILES+=3Detc/rc.d/zfs -OLD_FILES+=3Detc/rc.d/zfsd OLD_FILES+=3Detc/rc.d/zfsbe +OLD_FILES+=3Detc/rc.d/zfsd +OLD_FILES+=3Detc/rc.d/zfskeys OLD_FILES+=3Detc/rc.d/zvol OLD_FILES+=3Detc/devd/zfs.conf OLD_FILES+=3Detc/periodic/daily/404.status-zfs --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Sun Nov 14 21:00:26 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id C81BA183A523 for ; Sun, 14 Nov 2021 21:00:26 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Hsl8V4Cwhz4RZK for ; Sun, 14 Nov 2021 21:00:26 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 31E9E5516 for ; Sun, 14 Nov 2021 21:00:26 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 1AEL0Qvm018217 for ; Sun, 14 Nov 2021 21:00:26 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 1AEL0Qg2018216 for fs@FreeBSD.org; Sun, 14 Nov 2021 21:00:26 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202111142100.1AEL0Qg2018216@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: fs@FreeBSD.org Subject: Problem reports for fs@FreeBSD.org that need special attention Date: Sun, 14 Nov 2021 21:00:26 +0000 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="16369236262.6aAEBcCe.16889" Content-Transfer-Encoding: 7bit X-ThisMailContainsUnwantedMimeParts: Y --16369236262.6aAEBcCe.16889 Date: Sun, 14 Nov 2021 21:00:26 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- Open | 211491 | System hangs after "Uptime" on reboot with ZFS Open | 237067 | ZFS: Crash in vdev_dtl_reassess when using GELI w Open | 240831 | zfs: Panic during snapshot on 12.1-STABLE r352648 Open | 243973 | [zfs] rollback segmentation fault Open | 244656 | zfs: resilver doesn't provide enough information Open | 244692 | gjournal: Does not support TRIM Open | 244899 | [PATCH] zfs: xattr on a symlink target > 136 caus Open | 251035 | ZFS: Allow 64 bit ZFS to support 32 bit ioctls (W 8 problems total for which you should take action. --16369236262.6aAEBcCe.16889--