From nobody Mon Aug 16 20:19:47 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 5E7F41760E4C for ; Mon, 16 Aug 2021 20:19:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GpQW722WTz3M2v for ; Mon, 16 Aug 2021 20:19:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2BAA57FF1 for ; Mon, 16 Aug 2021 20:19:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 17GKJlq3054436 for ; Mon, 16 Aug 2021 20:19:47 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 17GKJlFr054435 for fs@FreeBSD.org; Mon, 16 Aug 2021 20:19:47 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 257890] Storage controller lockup on zfs scrub [smartpqi][zfs] Date: Mon, 16 Aug 2021 20:19:47 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D257890 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|bugs@FreeBSD.org |fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Tue Aug 17 00:55:39 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 5F525174B178 for ; Tue, 17 Aug 2021 00:55:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GpXdR239pz4VYd for ; Tue, 17 Aug 2021 00:55:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2F7E613E86 for ; Tue, 17 Aug 2021 00:55:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 17H0tdDO099760 for ; Tue, 17 Aug 2021 00:55:39 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 17H0tdWh099759 for fs@FreeBSD.org; Tue, 17 Aug 2021 00:55:39 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 251035] ZFS: Allow 64 bit ZFS to support 32 bit ioctls (Wine) Date: Tue, 17 Aug 2021 00:55:39 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.2-RELEASE X-Bugzilla-Keywords: needs-qa X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: damjan.jov@gmail.com X-Bugzilla-Status: Open X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: maintainer-feedback? mfc-stable13? mfc-stable12? mfc-stable11? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D251035 --- Comment #4 from Damjan Jovanovic --- (In reply to Kubilay Kocak from comment #3) That code has changed a lot in FreeBSD 13 and upstream. 32 bit "zpool list" still crashes, as per truss: 21250: openat(AT_FDCWD,"/dev/zfs",O_RDWR,00) =3D 5 (0x5) 21250: ioctl(3,0xc0145a04 { IORW 0x5a('Z'), 4, 20 },0xffffb338) ERR#22 'Inv= alid argument' where /var/log/messages has: Aug 17 02:37:31 pc kernel: len 20 vecnum: 4 sizeof (zfs_cmd_t) 4528 probably from: if (len !=3D sizeof (zfs_iocparm_t)) { printf("len %d vecnum: %d sizeof (zfs_cmd_t) %ju\n", len, vecnum, (uintmax_t)sizeof (zfs_cmd_t)); return (EINVAL); } in sys/contrib/openzfs/module/os/freebsd/zfs/kmod_core.c That has another bug in logging, as zfs_iocparm_t and zfs_cmd_t are differe= nt structs. We should be printing sizeof(zfs_iocparm_t), not sizeof(zfs_cmd_t). This bug probably doesn't affect Linux, as it uses zfs_cmd_t directly, which has the same field sizes/alignments on 32 and 64 bit already. We wrap zfs_c= md_t in our zfs_iocparm_t, and add this bug in the process. It will take me a while to test a new patch. --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Tue Aug 17 17:47:28 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 867C21748648 for ; Tue, 17 Aug 2021 17:47:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gpz4w3GkWz3Qtb for ; Tue, 17 Aug 2021 17:47:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 593C221D76 for ; Tue, 17 Aug 2021 17:47:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 17HHlSPI002372 for ; Tue, 17 Aug 2021 17:47:28 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 17HHlSfQ002371 for fs@FreeBSD.org; Tue, 17 Aug 2021 17:47:28 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 257890] Storage controller lockup on zfs scrub [smartpqi][zfs] Date: Tue, 17 Aug 2021 17:47:28 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: peter@guenschel.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D257890 --- Comment #1 from Peter --- Created attachment 227287 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D227287&action= =3Dedit log/messages See last lines --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Tue Aug 17 20:32:16 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A9013175AC69 for ; Tue, 17 Aug 2021 20:32:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gq2l44MhVz3tBS for ; Tue, 17 Aug 2021 20:32:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7FC7A24140 for ; Tue, 17 Aug 2021 20:32:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 17HKWGv5094124 for ; Tue, 17 Aug 2021 20:32:16 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 17HKWGsU094123 for fs@FreeBSD.org; Tue, 17 Aug 2021 20:32:16 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 257890] Storage controller lockup on zfs scrub [smartpqi][zfs] Date: Tue, 17 Aug 2021 20:32:16 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: rainer@ultra-secure.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D257890 rainer@ultra-secure.de changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |rainer@ultra-secure.de --- Comment #2 from rainer@ultra-secure.de --- This is split off from https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D240145 Please include papani.srikanth, emaste and imp in CCs so that Papani can he= lp gather debug information or a never version of the driver. --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Tue Aug 17 22:56:51 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 6874617632A5 for ; Tue, 17 Aug 2021 22:56:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gq5xv2NWBz4YYY for ; Tue, 17 Aug 2021 22:56:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 3AB72262B0 for ; Tue, 17 Aug 2021 22:56:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 17HMup7r071386 for ; Tue, 17 Aug 2021 22:56:51 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 17HMup7m071385 for fs@FreeBSD.org; Tue, 17 Aug 2021 22:56:51 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 257890] Storage controller lockup on zfs scrub [smartpqi][zfs] Date: Tue, 17 Aug 2021 22:56:51 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: imp@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D257890 Warner Losh changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |imp@FreeBSD.org --- Comment #3 from Warner Losh --- Is there a newer driver ready to integrate? --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Tue Aug 17 22:57:29 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A71831763370 for ; Tue, 17 Aug 2021 22:57:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gq5yd489Bz4Z7n for ; Tue, 17 Aug 2021 22:57:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 76A07260C0 for ; Tue, 17 Aug 2021 22:57:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 17HMvTKD071538 for ; Tue, 17 Aug 2021 22:57:29 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 17HMvTvf071537 for fs@FreeBSD.org; Tue, 17 Aug 2021 22:57:29 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 257890] Storage controller lockup on zfs scrub [smartpqi][zfs] Date: Tue, 17 Aug 2021 22:57:29 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: imp@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D257890 --- Comment #4 from Warner Losh --- Is there a new version of the smartpqi driver ready to integrate into FreeB= SD? --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Tue Aug 17 23:47:52 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 71A1B1766D29 for ; Tue, 17 Aug 2021 23:47:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gq74m2fTSz4c2S for ; Tue, 17 Aug 2021 23:47:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 43D5726F1B for ; Tue, 17 Aug 2021 23:47:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 17HNlqdf096761 for ; Tue, 17 Aug 2021 23:47:52 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 17HNlqE7096760 for fs@FreeBSD.org; Tue, 17 Aug 2021 23:47:52 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 257156] ZFS: zfs_alloc()/zfs_free() mismatch on boot Date: Tue, 17 Aug 2021 23:47:52 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: mirror176@hotmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D257156 Edward.Sanford.Sutton, III changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mirror176@hotmail.com --- Comment #1 from Edward.Sanford.Sutton, III --- https://www.reddit.com/r/freebsd/comments/p50oun/zfs_zfs_alloczfs_free_mism= atch_message_upon_boot/ mentions a different user who resolved the issue by rewriting the bootcode.= Is your bootcode up to date? Is that true for any/all disks that have bootcode installed to them? --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Wed Aug 18 00:00:02 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 3959C1767744 for ; Wed, 18 Aug 2021 00:00:12 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-to1can01on060d.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe5d::60d]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "DigiCert Cloud Services CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gq7Ly5NTCz4cZl; Wed, 18 Aug 2021 00:00:10 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MMlWzNqdtC/6KTJcLwF/oHCkuuZ54/DS1JouZTYwaj4+DCJjXlE8nsUXqfe3PQ54HoK8IfJWIMBLXfyFYOf4AxzgK/K30AkQPolE9u3BSvLZ+eEdVD0wgeCLNHWH4e8hw7Z6GPRy2LgeD/w6NEuosujEo//Z5Hkcww4wVddBb5BggzcONqxRg7lwILq+5LBNkFV1nBdVP73Pp2aFSVzO8UenXqXhGudRAWeS7PF6em0yntqW52rHcjqT7IK/9J3oE+HUStWjy9JzY26T24IPAqXe4E/dSWXhEwcreQ2/J5Be5SkDh91EWFHMjYjFe09lYNasujfqK65Kxh62URZN6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=us35njFqh9UpFqZjxjtzAolKxylwYyWnE7SIA0OBk/I=; b=WkZD9TMu4pDzIbF0CGxoG20ankekg1ooUm739XtMVIIas2vwU01G9Uf1viG8X4MorDUeIc+r57kS4ncfT5OTCZiwf7V3Zoam8pZuyNdiUYNTGKrpMxPuLzrwtek7x6S8j0tOIk0OCJAj/3yD9rzqDcEpkhXEU2AyedXhI7ICBGDH09/JOZye8II5xHCNs36QKaWLysw4lmTuceS5CD7CjhStUSU7EUvKfJvKoLqsPTm5QdGlv267JTUtX3FDjsh1sWL6lvdlwy8RhvYFJ0iN+zHzFWJ6Ey9PkwwvJNskBfls+D9VmzFF7INi0RR9g7BwIVjKOoTGfquAbXoiublXbw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=uoguelph.ca; dmarc=pass action=none header.from=uoguelph.ca; dkim=pass header.d=uoguelph.ca; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uoguelph.ca; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=us35njFqh9UpFqZjxjtzAolKxylwYyWnE7SIA0OBk/I=; b=R5kgrk3SKiSoNjv94fhuqKT3xKz1pIhgTtxUP5PDf6qG9CKIUmIwFmeeNTP99dMYSnXB8AKKAEaZk3WW9Zgquznq+8pQW4rcRFwclcDgn5DDP6J1CwM4qN7NXisFxIjs1YXrzNJGK3TriXi6E6V5BbML4XifHhhekPt81MuoQVHHHT52qV5pDeKvkBGMximDeabquuvDADQgplnYtPQCRKLgBpMCaaYKdn57zspwvlVhy+MAxX8L8hTG9t6/6oh8sQHyEDb8qhjmpH6GkwFfN43KmFprCBfJbhh8MZ+SKOAQxi06otR/hbNC92ubI6IEXU1QRq76fZtplWGRBJOBsw== Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c00:19::29) by YQBPR0101MB4780.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:c01:18::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4415.17; Wed, 18 Aug 2021 00:00:03 +0000 Received: from YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::1174:bfb8:7a34:f67c]) by YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM ([fe80::1174:bfb8:7a34:f67c%7]) with mapi id 15.20.4415.024; Wed, 18 Aug 2021 00:00:03 +0000 From: Rick Macklem To: Ka Ho Ng CC: FreeBSD Filesystems Subject: fspacectl(2) questions Thread-Topic: fspacectl(2) questions Thread-Index: AQHXk8H96862pfDLFk6WeR3YvpWGCQ== Date: Wed, 18 Aug 2021 00:00:02 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: bc333452-991a-4261-a557-08d961db2114 x-ms-traffictypediagnostic: YQBPR0101MB4780: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:6790; x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: sM2WvVXBhuA+k74UBLJ2Yhcq5HSool7R4fqNQzY3wMO0qkSjs4XVtOJLdDCRFdNwUx1CSL8wdICCvdKu8NKo3uIwlgxBXDbxYslgsintngawxkJTr2b6RcgGPYnBEKg+TLoTUAk0B7HyOZ4CRm5BLQXSvLWJzcWbxp/sAVwJswRE07PDMBr7lIbwseLFAWPMpPKllWRphErxyOLnX273mIiyQNxRneKzoqbfyeB/DCkgFE+9ZCWWOXuBjPfsB6sCgsSc+Vpqh04hqy+tTaEfjviMlIkwXeIhTebUH1JcJnRMjlSazqG4WCs9ZnuR9Bghg3WJHin2QC7GHuEcROm7PcQQpUw+FZufrQHWGnFGA308cjE82SE266i0OGuC+TWbBJ9p7/kHfDRVxFeY0aMTwrE5v3+rfu9BEJuQ/kaaLxeQ+5/dTRnUqUQJRI08P0YgvSGSOsMEaweYIkN2QAgjYHQfP34ftpp1IgOKlELeSjaWTU0niwmCXxN94YeY4Do7iVtAxeB/z7xs1Ynmyh14iIdEXuDGdCkcZyg++3wU2A1OWm+UHB+LE3dpvQDHhvEP+bbJ9s3Llow8cquVSOjUUkYW2iX4hIro4/svoafY3flYgFCml7r+aVbW+JauJmSQe0mAguMT7aJifqL7TCbQwUfH5ZGIcV8tdmpQ3s+/EyeAYdAs2gr6eUfKZIGoBAYsPrORPT34zlgZh3jC2eBhtw== x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM;PTR:;CAT:NONE;SFS:(366004)(38070700005)(2906002)(8676002)(86362001)(508600001)(8936002)(6916009)(4326008)(7696005)(450100002)(316002)(786003)(5660300002)(66446008)(66556008)(66476007)(66946007)(76116006)(52536014)(55016002)(91956017)(64756008)(9686003)(71200400001)(38100700002)(122000001)(186003)(33656002)(6506007);DIR:OUT;SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?aRauy+Ia9VFQT1JIa1YdM+/nymRqE7NFf+EHdxzZPsMSZdv6E7Vtr+PxH4?= =?iso-8859-1?Q?c53AgGBH65ozoaWi9RY6HJPE2DmXFQRRTm+lZRlUV6uxxypb2fDFk5JFwy?= =?iso-8859-1?Q?D2B3szSDJqoWGK+VgcC1A4JxmfNWuPlt3RIRsDkJ7A9XOHwdGz/nnrnY2a?= =?iso-8859-1?Q?2l7Kag76DMeWxvC7Ytm0qR2jEBFF0GbgDogSk3xrciyES5V2Qxz0UqNBwc?= =?iso-8859-1?Q?bxGkK/ZiEIQ2qV7+MoWHnQ6NTTqH/5XOO6/3kHpUEWNulzSY15pcSCO4Lr?= =?iso-8859-1?Q?d7xX9B+sYCms0izFQAN9QNkR/vq4qV65Aq91bN5fAAlOdSuNX9J442cJIt?= =?iso-8859-1?Q?oxWL8KvT5s7GksbkfuMooFsz1mZGO/bxBYwC8Cwz0T/LSjWXjMGA+eUyvY?= =?iso-8859-1?Q?DT2Ul7T8FceVDnjGdUaUoy+0adOWKFeBR59H+qKut4Q+fdfLlCyYxlv6nU?= =?iso-8859-1?Q?dBsMxxiKoXS4RINNEo85uxJcMzi51VPtdYy9Tm6mi2weILoj4VbsbE75W3?= =?iso-8859-1?Q?1TCAa81QSMmkaSK+nGup7UiC0X1VRX0rE9wpgzMvDLZyIRnqxFyGYqPAjC?= =?iso-8859-1?Q?Ua6EIoEwV5QoPLent3jh7zqbf+1ZJy1sDDAfQI74WtfgoZpD+8JsIDnx+X?= =?iso-8859-1?Q?sxe1OEXxHedRukbgWUHoKaCISGmgtIHgOH2eNr+XC2VwpE9zzzIF12nbxz?= =?iso-8859-1?Q?XdkSBDYIUmMTf/nLXxePoaCCUDzDGowCO8PmKNQ233S7hajx4E7DoJgxjB?= =?iso-8859-1?Q?Sxk7SeFldSpyCzNIyCYdrH3tCEbmirt3WTqBPQjxsSHHVmS21e7UiVspTe?= =?iso-8859-1?Q?v5WienUAKCkebrp2X541IjcOgJN6wfJ50eyoGuv7jMUUR37pIh8J5hPNu7?= =?iso-8859-1?Q?yD81dpIiY5HsoH58bD/pFnvSizpHxcxFfbsr8NT1hsihIOhF33xCidqjnc?= =?iso-8859-1?Q?iRiRIgynIo1n5jPsMVjB0IsHP1fk175lPgU9QSwl6455NTdMBBJK9sNSqS?= =?iso-8859-1?Q?4+EEYOfBIITOEzD4/tcGWT112VqshlAQA70jZkwG7hb/8/sArHpSohKemu?= =?iso-8859-1?Q?Cyt2HSixJf6gL+Dbpd3llFJ2D2ZlYlXTLUoMTYyM/mZ9a9qL5pRvOVcMbg?= =?iso-8859-1?Q?mrW8YtCnXkZEYbRoCBH+rTsi5XzsJb/lbGK/UgfDzED44RpxdPAtEG2mo9?= =?iso-8859-1?Q?QQTQT6mBulSIkLs+JO3j1TiMww+EQTTgNObWc1KI/Nk/Dd5zZjs+Yfjd+a?= =?iso-8859-1?Q?WaheEDVyYp6lDx4Creh6Vk5VrMFkVEydlKE8Vobxyem1TIAXbNwo0VH0J1?= =?iso-8859-1?Q?UMiFVqLWpwp4PWXXljMybfpARNZCtY3wdn34F6m+gctb3coMDuDTooW4Q5?= =?iso-8859-1?Q?BmRHiDbKJ+/+FHuy55SHLyci4gfP7Yh441qkJlsOJrBhk0f+x4CBnlhk6l?= =?iso-8859-1?Q?2tn21LCfQwG+a4xh?= x-ms-exchange-transport-forked: True Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-Network-Message-Id: bc333452-991a-4261-a557-08d961db2114 X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Aug 2021 00:00:02.9199 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: Sv66q+8DwnDMXJw+rXhuIR5g/AjdThGg01j7JFGPMW8W4UGcLYQI5UmwovZ3Co52R+4nQOn8rLd4DUcLitNTdw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQBPR0101MB4780 X-Rspamd-Queue-Id: 4Gq7Ly5NTCz4cZl X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=uoguelph.ca header.s=selector2 header.b=R5kgrk3S; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=none) header.from=uoguelph.ca; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 2a01:111:f400:fe5d::60d as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-6.00 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[uoguelph.ca:s=selector2]; FREEFALL_USER(0.00)[rmacklem]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a01:111:f400::/48]; MIME_GOOD(-0.10)[text/plain]; NEURAL_HAM_LONG(-1.00)[-1.000]; DWL_DNSWL_LOW(-1.00)[uoguelph.ca:dkim]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[uoguelph.ca:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[uoguelph.ca,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:8075, ipnet:2a01:111:f000::/36, country:US]; ARC_ALLOW(-1.00)[microsoft.com:s=arcselector9901:i=1] X-ThisMailContainsUnwantedMimeParts: N First off, thanks for doing this. I am just implementing the NFSv4.2=0A= Deallocate operation using it. (I'll put the patches up on phabricator=0A= soon.)=0A= =0A= There are a couple of things I've spotted during testing.=0A= 1 - If you specify (for example):=0A= rqst.r_offset =3D 65536=0A= rqst.r_len =3D 1048576=0A= and then use these to do a fspacectl(fd, SPACECTL_DEALLOC, &rqst, 0= , &rqst)=0A= on a file < 1048576 bytes in length, it works and zeros out 65536->EOF,= =0A= which is what I would have expected.=0A= However, rqst.r_offset is set to 1114112 after the syscall.=0A= Is that intended? I just thought it was a little weird to return r_offse= t > EOF instead=0A= of the size of the file.=0A= If that is the intended behaviour, I'm fine with that, but maybe "man fs= pacectl"=0A= should indicate that? Maybe something like "bytes beyond EOF in the spec= ified=0A= range will not be set to zero (ie. file size not increased), but that r_= offset will be=0A= advanced to the end of the range, even if beyond EOF".=0A= =0A= 2 - Setting=0A= rqst.r_offset =3D OFF_MAX=0A= rqst.r_len =3D 65536=0A= results in an EINVAL errno, which seems fine, but isn't mentioned in t= he=0A= man page.=0A= =0A= But overall, works fine for me. (Of course, it will be nice when UFS and ZF= S=0A= get VOP_DEALLOCATE() calls that really deallocate.)=0A= =0A= Again, thanks for doing this, rick=0A= From nobody Wed Aug 18 01:35:53 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 46313176D4B1 for ; Wed, 18 Aug 2021 01:35:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gq9TR1F42z4jkg for ; Wed, 18 Aug 2021 01:35:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0C0C53B1 for ; Wed, 18 Aug 2021 01:35:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 17I1ZsX7053449 for ; Wed, 18 Aug 2021 01:35:54 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 17I1ZsTG053448 for fs@FreeBSD.org; Wed, 18 Aug 2021 01:35:54 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 224292] processes are hanging in state ufs / possible deadlock in file system Date: Wed, 18 Aug 2021 01:35:53 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: cseh.donat@gmail.com X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Overcome By Events X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224292 ATAG changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |cseh.donat@gmail.com --- Comment #21 from ATAG --- This happened to me too after upgrade to 13-RELEASE. Mostly mv and rm proce= sses stuck at biowr state. After 'sync' everything returns normal. FreeBSD 13.0-RELEASE-p3 #0: Tue Jun 29 19:46:20 UTC 2021=20=20=20=20 root@amd64-builder.daemonology.net:/usr/obj/usr/src/amd64.amd64/sys/GENERIC= =20 amd64 --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Wed Aug 18 06:35:12 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 0EAB1174AA29 for ; Wed, 18 Aug 2021 06:35:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GqJ6l6nBnz3Dsf for ; Wed, 18 Aug 2021 06:35:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CC3C74728 for ; Wed, 18 Aug 2021 06:35:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 17I6ZB0K007870 for ; Wed, 18 Aug 2021 06:35:11 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 17I6ZBCT007869 for fs@FreeBSD.org; Wed, 18 Aug 2021 06:35:11 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 224292] processes are hanging in state ufs / possible deadlock in file system Date: Wed, 18 Aug 2021 06:35:12 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D224292 Kirk McKusick changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|Overcome By Events |FIXED --- Comment #22 from Kirk McKusick --- (In reply to Tomasz Sowa from comment #20) (In reply to ATAG from comment #21) The fix for this problem was MFC'ed to stable/13 on Wed Jul 7 13:50:44 2021= in commit d36505493d2a876b37da0c7850ef906b1f32df08. You need to run a kernel compiled from stable/13 sources on or after this d= ate to get the needed fix. --=20 You are receiving this mail because: You are the assignee for the bug.= From nobody Thu Aug 19 09:37:39 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 7DCA0176E575 for ; Thu, 19 Aug 2021 09:38:03 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vtr.rulingia.com (vtr.rulingia.com [IPv6:2001:19f0:5801:ebe:5400:1ff:fe53:30fd]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "vtr.rulingia.com", Issuer "R3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gr07G0qJmz4WZC for ; Thu, 19 Aug 2021 09:38:01 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from server.rulingia.com (2001-44b8-31fc-0d00-41ad-e251-cf00-b0b1.static.ipv6.internode.on.net [IPv6:2001:44b8:31fc:d00:41ad:e251:cf00:b0b1]) by vtr.rulingia.com (8.16.1/8.15.2) with ESMTPS id 17J9biZ4026313 (version=TLSv1.3 cipher=AEAD-AES256-GCM-SHA384 bits=256 verify=OK) for ; Thu, 19 Aug 2021 19:37:50 +1000 (AEST) (envelope-from peter@rulingia.com) DKIM-Filter: OpenDKIM Filter v2.10.3 vtr.rulingia.com 17J9biZ4026313 X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.16.1/8.16.1) with ESMTPS id 17J9bdnD039514 (version=TLSv1.3 cipher=AEAD-AES256-GCM-SHA384 bits=256 verify=NO) for ; Thu, 19 Aug 2021 19:37:39 +1000 (AEST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.16.1/8.16.1/Submit) id 17J9bdta039513 for freebsd-fs@freebsd.org; Thu, 19 Aug 2021 19:37:39 +1000 (AEST) (envelope-from peter) Date: Thu, 19 Aug 2021 19:37:39 +1000 From: Peter Jeremy To: freebsd-fs@freebsd.org Subject: ZFS on high-latency devices Message-ID: List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="/blLqS9gkC3/xMqZ" Content-Disposition: inline X-PGP-Key: http://www.rulingia.com/keys/peter.pgp X-Rspamd-Queue-Id: 4Gr07G0qJmz4WZC X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=quarantine) header.from=rulingia.com; spf=pass (mx1.freebsd.org: domain of peter@rulingia.com designates 2001:19f0:5801:ebe:5400:1ff:fe53:30fd as permitted sender) smtp.mailfrom=peter@rulingia.com X-Spamd-Result: default: False [-5.87 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FREEFALL_USER(0.00)[peter]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCVD_COUNT_THREE(0.00)[3]; MID_RHS_MATCH_FROMTLD(0.00)[]; NEURAL_HAM_SHORT(-0.97)[-0.965]; DMARC_POLICY_ALLOW(-0.50)[rulingia.com,quarantine]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:20473, ipnet:2001:19f0:5800::/38, country:US]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N --/blLqS9gkC3/xMqZ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable I'm looking at backing up my local ZFS pools to a remote system without the remote system having access to the pool content. The options for the backup pools seem to be: a) Use OpenZFS native encryption with ZFS running on the remote system backed up using "zfs send --raw". b) Run ZFS over geli locally over geom_gate[1] to the remote system. The first approach removes RTT as an issue but requires that the local pools first be converted to native encryption - a process that seems to be generously defined as "very difficult". The second approach removes the need to encrypt the local pool but is far more sensitive to RTT issues and I've been unable to get a reasonable throughput. The main problems I've found are: * Even with a quite high write aggregation limit, I still get lots of relatively small writes. * Snapshot boundaries appear to wait for all queued writes to be flushed. I've found https://openzfs.org/wiki/ZFS_on_high_latency_devices but I can't get the procedure to work. "zfs send" of a zvol seems to bear very little resemblance to a "zfs send" of a "normal" filesystem. Sending a zvol, I can't get ndirty _down_ to the suggested 70-80%, whereas with (eg) my mail spool, I can't get ndirty _up_ to the suggested 70-80%. And most of the suggested adjustments are system- wide so the suggested changes are likely to adversely impact local ZFS performance. Does anyone have any suggestions as to a way forward? Either a reliable process to encrypt an existing pool or a way to improve throughput doing "zfs recv" to a pool with a high RTT. --=20 Peter Jeremy --/blLqS9gkC3/xMqZ Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEE7rKYbDBnHnTmXCJ+FqWXoOSiCzQFAmEeJlxfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEVF QjI5ODZDMzA2NzFFNzRFNjVDMjI3RTE2QTU5N0EwRTRBMjBCMzQACgkQFqWXoOSi CzS6LRAAiBR1BHdBtlf7jnNmPuvMn+oaYwW1HeEmLzdcD/YrGwJN6vP0d7LV2AP9 ceY3E2SiRTN/VDcYAYuWBk4UzNbV16tYbs6f6qQpAYH0diF0lFzjYe+hsnvDZFMo NXskXqAyhGPqciDTn+lRbNZETDJbnonNVmGUw7HfzG7wTPZOyr/5odq+YXyep6s0 zmpFTheLStYKMXnbaxBazoV6ydzQ9y4Elc/aCqswMV29vHxDSp5j/fdDwG6ZQRFs B2+4AqzF9Ea8eASzDeX0qzwhiH8jXNDCP0Xp0uPusSnzYJBu2i/x42wbT7Dp9DVE oTIXYMYlX2WZTLrUyB7evgEdce/DLH01LP/ZPLTPSK+vHPtoLIV/FiVre0AqUO3j ilJli+O+OKeITyWFjI60bXOoWWJpl+zyirwCtC2viKwtwyj4nj1jTrSHK9d8dCbq KI///r89DivnU2tKgSIbvCgwtE8YUi4T5dc0VJzr2c+b0Vf39R3LbC+C4FdvperW AqU6kuN9sVgqsJK5kLUBh4buAWDGhpfNxllbEh9sGioEodnnEi1nM5MmWqPhLnAM Lz44UMeSJEu4+XoQt9MO5QObWfi86382oN9qA7khMIad1aEN4eqwLKH9f3QutfR5 kkCbG+BRq7StBkeV8YzqPBJe2BKgBDA3BUHZe5p+5/kIXe+OC+k= =CRZo -----END PGP SIGNATURE----- --/blLqS9gkC3/xMqZ-- From nobody Thu Aug 19 23:16:28 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 54120177568D for ; Thu, 19 Aug 2021 23:16:52 +0000 (UTC) (envelope-from freebsd-fs@m.gmane-mx.org) Received: from ciao.gmane.io (ciao.gmane.io [116.202.254.214]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GrLJ32wfZz4SS8 for ; Thu, 19 Aug 2021 23:16:51 +0000 (UTC) (envelope-from freebsd-fs@m.gmane-mx.org) Received: from list by ciao.gmane.io with local (Exim 4.92) (envelope-from ) id 1mGrH9-0008fG-Qf for freebsd-fs@freebsd.org; Fri, 20 Aug 2021 01:16:43 +0200 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Johannes Totz Subject: Re: ZFS on high-latency devices Date: Fri, 20 Aug 2021 00:16:28 +0100 Message-ID: References: List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.0.1 Content-Language: en-GB In-Reply-To: X-Rspamd-Queue-Id: 4GrLJ32wfZz4SS8 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="SPF not aligned (relaxed), No valid DKIM" header.from=jo-t.de (policy=none); spf=pass (mx1.freebsd.org: domain of freebsd-fs@m.gmane-mx.org designates 116.202.254.214 as permitted sender) smtp.mailfrom=freebsd-fs@m.gmane-mx.org X-Spamd-Result: default: False [0.30 / 15.00]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; DMARC_POLICY_SOFTFAIL(0.10)[jo-t.de : SPF not aligned (relaxed), No valid DKIM,none]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-0.89)[-0.891]; RCPT_COUNT_ONE(0.00)[1]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_SHORT(-0.91)[-0.913]; FORGED_SENDER(0.30)[johannes@jo-t.de,freebsd-fs@m.gmane-mx.org]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:116.202.0.0/16, country:DE]; FROM_NEQ_ENVFROM(0.00)[johannes@jo-t.de,freebsd-fs@m.gmane-mx.org]; FORGED_MUA_THUNDERBIRD_MSGID_UNKNOWN(2.50)[]; RCVD_COUNT_TWO(0.00)[2] X-Spam: Yes X-ThisMailContainsUnwantedMimeParts: N On 19/08/2021 10:37, Peter Jeremy wrote: > I'm looking at backing up my local ZFS pools to a remote system > without the remote system having access to the pool content. The > options for the backup pools seem to be: > a) Use OpenZFS native encryption with ZFS running on the remote system > backed up using "zfs send --raw". > b) Run ZFS over geli locally over geom_gate[1] to the remote system. > > The first approach removes RTT as an issue but requires that the local > pools first be converted to native encryption - a process that seems > to be generously defined as "very difficult". > > The second approach removes the need to encrypt the local pool but > is far more sensitive to RTT issues and I've been unable to get > a reasonable throughput. The main problems I've found are: > * Even with a quite high write aggregation limit, I still get lots of > relatively small writes. > * Snapshot boundaries appear to wait for all queued writes to be flushed. > > I've found https://openzfs.org/wiki/ZFS_on_high_latency_devices but I > can't get the procedure to work. "zfs send" of a zvol seems to bear > very little resemblance to a "zfs send" of a "normal" filesystem. > Sending a zvol, I can't get ndirty _down_ to the suggested 70-80%, > whereas with (eg) my mail spool, I can't get ndirty _up_ to the > suggested 70-80%. And most of the suggested adjustments are system- > wide so the suggested changes are likely to adversely impact local > ZFS performance. > > Does anyone have any suggestions as to a way forward? Either a > reliable process to encrypt an existing pool or a way to improve > throughput doing "zfs recv" to a pool with a high RTT. > Do you have geli included in those perf tests? Any difference if you leave it out? What's making the throughout slow? zfs issuing a bunch of small writes and then trying to read something (unrelated)? Is there just not enough data to be written to saturate the link? Totally random thought: there used to be a vdev cache (not sure if that's still around) that would inflate read requests to hopefully drag in more data that might be useful soon. Have you tried hastd? From nobody Fri Aug 20 08:18:49 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 89C641772641 for ; Fri, 20 Aug 2021 08:18:58 +0000 (UTC) (envelope-from ben.rubson@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.15.19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass Class 2 CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GrZKY393Rz4yws for ; Fri, 20 Aug 2021 08:18:57 +0000 (UTC) (envelope-from ben.rubson@gmx.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1629447529; bh=SoAXD0jHrw17xK5hv+80zERoNdWWmxc0boWX0KoAbjw=; h=X-UI-Sender-Class:From:Subject:Date:References:To:In-Reply-To; b=fzJXaBBYUruRw5P043Hsx5u5UfWR20LpZ8hbqBT64ztAvIOqBsdB0gxBi6c/yMPuB N09soyyeEIAaFAknZc0piGQ69goZY5HeUn5S2aSae7Jvhg7u+PwFrx4rKN53oltQbG g5DSN+n9j6e3D55qyGx19NKuH144CymmR3IDxlY4= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from smtpclient.apple ([185.101.28.209]) by mail.gmx.net (mrgmx005 [212.227.17.184]) with ESMTPSA (Nemesis) id 1MysVs-1n2vjH2ojq-00w0fv for ; Fri, 20 Aug 2021 10:18:49 +0200 From: Ben RUBSON Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: ZFS on high-latency devices Date: Fri, 20 Aug 2021 10:18:49 +0200 References: To: freebsd-fs@freebsd.org In-Reply-To: Message-Id: <023225AD-2A97-47C5-9FE4-3ABF1BFD66F1@gmx.com> X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Provags-ID: V03:K1:X616uPTyBQLFkxESmGa0lRTaK8EC5kX8EoBnH3vjLW3kmppFnDx mjc5OoOmPiMK9+/cS6EqUzuqhg1L4+DZQkjsiscyCxzXrvgGSJGQz5XPuQnIZSRtGzBPBgM MtG8iayjjVzvt3g7T8HMVj63Ssiz8+aASIah2Mdldxezoo/g5X2c3gLLwgd3Wa8/f4Ap3zK P8zbBY+3mBvHpol00paxQ== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:l8hEDiO9HFg=:SWsqvsFiF8yZ4LICIFeTSA 7JbdbBgrCfuOy6Uxf12YYZx9XHXLl4JlXeei5HWcNICSycxMW7ZLyZggT9r6WM7H0Vq8E7Wtd zpSlFweYlSqzp9qGH37NepNrtnWAL3EVE5feSlDntXGoSPmOVPLOd2gOdO7C3+ZCY9iTbCADv KhoiSnYNbK7I783IR6nud9NCE5M77YKSQjG6AcZ/hlRH/CSdZJKUtdcdVj29vwXV5cHzLloIA wsYbKvTsygra9KdsX+D4D04w3E8Afm25M1/Oevrp4lL3WDvEHpB+wJqq2gG/Pkc3CsD0bC6Q1 LJMVlBFgM3oKdskIh/sgtUZMlFs1Moj1M9vlUOv5ghIhDiEMZJ6mT6VxOczFP15j0Ik4V0vMO Psef3wiXdKV1n9eja3gntzGrEUQWo2H1J0NLrLnNoU8vl5arlA2ZaqDVuwU/NNuvmDYsNiXRK 7wtNlE8fehhRGGVb2f9YGYqV+73y7IuUM35cjmizTfOvp3ESfjzelxvxmc3vdQ5+j53Gr/vXW W9XhQzp2HXwBR0gN1EpO/2LjewdLjpMQT2PWXNdOQkYik4IF6Ykwu2p/BL0rny7b9ikD2lQ4E bWlYA+cry74mkXKFp4/s0hCiiUjqA7cohH7LAMPDHYxNq4/kyYCEi5orNKvi/24OSF1Datf8T 3WlafS1Q09XtvuqXfgIjsL5aWc82LlldrzHjyZNqtmY1t3g0WMDgJh2Zt97GH43aaeSV5V7Fy vZdbXj/evV9stop19YGVHN1wBpdnfxyxUIYcBW82A8CGBz3KerQT7lkoyiB2VZrDsSOh0vkoZ jZxMXYEu3Y1suwxz/8bFs9I23pY2gjo+1Pm1q45nR4byUasFBlQnXzfYBbjkYX8ppGSq1ZfZo zWinjrI/DSmhyBZNHdTxd08tA9dL35ny6D4uJqdmPP7DDkPEZlZBtxMFyk69MrdGr1reWQXxq jFoGzw7WCvHNZMQI+Ie3Rn9pxUGSTFI4gCDq2GgwyWU/Yit5x3/u4k9Hgjqm3RzhscluOD2Y+ LS757A4D8bGnBhjoKyvAYUnqHuAwxAz87Br0lgEBFwHoBSKLHGyRa1IdyQdCFAL8yBU9J8wO6 CX49hi+d5A2K6X6CArr4nMJJQQMsMxpOq7kxzaURoaBMiWEJe51d/zMAQ== X-Rspamd-Queue-Id: 4GrZKY393Rz4yws X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmx.net header.s=badeba3b8450 header.b=fzJXaBBY; dmarc=pass (policy=none) header.from=gmx.com; spf=pass (mx1.freebsd.org: domain of ben.rubson@gmx.com designates 212.227.15.19 as permitted sender) smtp.mailfrom=ben.rubson@gmx.com X-Spamd-Result: default: False [-4.46 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[212.227.15.19:from]; FREEMAIL_FROM(0.00)[gmx.com]; MV_CASE(0.50)[]; TO_DN_NONE(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:212.227.15.0/24]; DKIM_TRACE(0.00)[gmx.net:+]; DMARC_POLICY_ALLOW(-0.50)[gmx.com,none]; NEURAL_HAM_SHORT(-0.86)[-0.855]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmx.com]; ASN(0.00)[asn:8560, ipnet:212.227.0.0/16, country:DE]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[212.227.15.19:from]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[gmx.net:s=badeba3b8450]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[gmx.net:dkim]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N > On 19 Aug 2021, at 11:37, Peter Jeremy wrote: >=20 > (...) or a way to improve throughput doing "zfs recv" to a pool with a = high RTT. You should use zfs send/receive through mbuffer, which will allow to = sustain better throughput over high latency links. Feel free to play with its buffer size parameters to find the better = settings, depending on your link characteristics. Ben From nobody Fri Aug 20 15:23:57 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id B5405176E75D for ; Fri, 20 Aug 2021 15:24:00 +0000 (UTC) (envelope-from khng@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Grlm04q4dz4W1m for ; Fri, 20 Aug 2021 15:24:00 +0000 (UTC) (envelope-from khng@FreeBSD.org) Received: from Kas-MacBook-Pro.local (unknown [IPv6:2001:470:f816:0:1a9:413f:d3c5:1c26]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: khng/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 1D557B6C0 for ; Fri, 20 Aug 2021 15:23:59 +0000 (UTC) (envelope-from khng@FreeBSD.org) To: freebsd-fs@FreeBSD.org From: Ka Ho Ng Subject: fspacectl(2): result of rmsr.r_offset for a success and non-partial operation Message-ID: <87d4a87d-2ee3-74dd-3689-94dc0daf3983@FreeBSD.org> Date: Fri, 20 Aug 2021 23:23:57 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-ThisMailContainsUnwantedMimeParts: N There were some recent discussion in https://reviews.freebsd.org/D31604 around the returned value of rmsr.r_offset. For a complete and successful operation, rmsr.r_len is set to 0. Regarding rmsr.r_offset, the bottom line is to have rmsr.r_offset being no greater than current file size in case rmsr.r_offset is greater than file size, while leaving rmsr.r_offset to be within EOF in case rqsr.r_offset is not beyond EOF. With the current approach, rmsr.r_offset is loosely defined as file system is free to set it to some value as long as it is neither smaller than rqsr.r_offset (in case rqsr.r_offset is within EOF), or beyond EOF (in case rqsr.r_offset is beyond EOF). Do you think it is a good idea to make it stricter in case the call succeeds and rmsr.r_len == 0 (i.e. a complete operation)? If that is the case, what if we set rmsr.r_offset to be rqsr.r_offset + rqsr.r_len? Ka Ho From nobody Fri Aug 20 15:33:24 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 5D2B5176F625 for ; Fri, 20 Aug 2021 15:33:26 +0000 (UTC) (envelope-from khng@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Grlyt2DfHz4XXf for ; Fri, 20 Aug 2021 15:33:26 +0000 (UTC) (envelope-from khng@FreeBSD.org) Received: from Kas-MacBook-Pro.local (unknown [IPv6:2001:470:f816:0:1a9:413f:d3c5:1c26]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: khng/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id CBC2CB6CA for ; Fri, 20 Aug 2021 15:33:25 +0000 (UTC) (envelope-from khng@FreeBSD.org) Subject: Re: fspacectl(2): result of rmsr.r_offset for a success and non-partial operation From: Ka Ho Ng To: freebsd-fs@FreeBSD.org References: <87d4a87d-2ee3-74dd-3689-94dc0daf3983@FreeBSD.org> Message-ID: <29087ddc-9b46-15fa-4041-c3a50dcf99db@FreeBSD.org> Date: Fri, 20 Aug 2021 23:33:24 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.13.0 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 In-Reply-To: <87d4a87d-2ee3-74dd-3689-94dc0daf3983@FreeBSD.org> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-ThisMailContainsUnwantedMimeParts: N On 2021/8/20 11:23 PM, Ka Ho Ng wrote: > There were some recent discussion in https://reviews.freebsd.org/D31604 > around the returned value of rmsr.r_offset. For a complete and > successful operation, rmsr.r_len is set to 0. Regarding rmsr.r_offset, > the bottom line is to have rmsr.r_offset being no greater than current > file size in case rmsr.r_offset is greater than file size, while leaving > rmsr.r_offset to be within EOF in case rqsr.r_offset is not beyond EOF. > > With the current approach, rmsr.r_offset is loosely defined as file > system is free to set it to some value as long as it is neither smaller > than rqsr.r_offset (in case rqsr.r_offset is within EOF), or beyond EOF > (in case rqsr.r_offset is beyond EOF). Do you think it is a good idea to > make it stricter in case the call succeeds and rmsr.r_len == 0 (i.e. a > complete operation)? If that is the case, what if we set rmsr.r_offset > to be rqsr.r_offset + rqsr.r_len? > > Ka Ho > My another approach is to explicitly document that for a complete and successful operation (i.e. rmsr.r_len == 0) callers need not to consider the exact value of rmsr.r_offset, except if the operation range is not complete outside of EOF, rmsr.r_offset is not going to be outside of EOF either. Ka Ho From nobody Sat Aug 21 15:39:27 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 79B921784099 for ; Sat, 21 Aug 2021 15:39:36 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GsN3W1KjDz4nXf; Sat, 21 Aug 2021 15:39:35 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 17LFdRqs027344 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sat, 21 Aug 2021 18:39:30 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 17LFdRqs027344 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 17LFdRVc027343; Sat, 21 Aug 2021 18:39:27 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 21 Aug 2021 18:39:27 +0300 From: Konstantin Belousov To: Ka Ho Ng Cc: freebsd-fs@freebsd.org Subject: Re: fspacectl(2): result of rmsr.r_offset for a success and non-partial operation Message-ID: References: <87d4a87d-2ee3-74dd-3689-94dc0daf3983@FreeBSD.org> <29087ddc-9b46-15fa-4041-c3a50dcf99db@FreeBSD.org> List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <29087ddc-9b46-15fa-4041-c3a50dcf99db@FreeBSD.org> X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.5 X-Spam-Checker-Version: SpamAssassin 3.4.5 (2021-03-20) on tom.home X-Rspamd-Queue-Id: 4GsN3W1KjDz4nXf X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [-0.64 / 15.00]; ARC_NA(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_SPF_SOFTFAIL(0.00)[~all:c]; NEURAL_SPAM_MEDIUM(0.51)[0.513]; NEURAL_HAM_SHORT(-0.16)[-0.157]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none] X-ThisMailContainsUnwantedMimeParts: N On Fri, Aug 20, 2021 at 11:33:24PM +0800, Ka Ho Ng wrote: > On 2021/8/20 11:23 PM, Ka Ho Ng wrote: > > There were some recent discussion in https://reviews.freebsd.org/D31604 > > around the returned value of rmsr.r_offset. For a complete and > > successful operation, rmsr.r_len is set to 0. Regarding rmsr.r_offset, > > the bottom line is to have rmsr.r_offset being no greater than current > > file size in case rmsr.r_offset is greater than file size, while leaving > > rmsr.r_offset to be within EOF in case rqsr.r_offset is not beyond EOF. > > > > With the current approach, rmsr.r_offset is loosely defined as file > > system is free to set it to some value as long as it is neither smaller > > than rqsr.r_offset (in case rqsr.r_offset is within EOF), or beyond EOF > > (in case rqsr.r_offset is beyond EOF). Do you think it is a good idea to > > make it stricter in case the call succeeds and rmsr.r_len == 0 (i.e. a > > complete operation)? If that is the case, what if we set rmsr.r_offset > > to be rqsr.r_offset + rqsr.r_len? > > > > Ka Ho > > > > My another approach is to explicitly document that for a complete and > successful operation (i.e. rmsr.r_len == 0) callers need not to consider > the exact value of rmsr.r_offset, except if the operation range is not > complete outside of EOF, rmsr.r_offset is not going to be outside of EOF > either. I believe that for normal operation, rmsr.r_offset should be set to to rqsr.r_offset + rqsr.r_len, and for EOF case rqsr.r_offset + rqsr.r_len should be clipped at EOF. In both cases, rmsr.r_len should be set to zero. If rqsr.r_offset is beyond EOF, the statement above naturally implies that rmsr.r_offset is set to EOF, and rmsr.r_len is set to zero. This should interact well with a possibility that file is grown after the fspacectl(2) syscall is issued, and normal coding of the application using fspacectl(). From nobody Sun Aug 22 21:00:41 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id B829617850CA for ; Sun, 22 Aug 2021 21:00:42 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gt77Y43KHz3GmV for ; Sun, 22 Aug 2021 21:00:41 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 34DEA27A18 for ; Sun, 22 Aug 2021 21:00:41 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 17ML0f81053814 for ; Sun, 22 Aug 2021 21:00:41 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 17ML0f8L053813 for fs@FreeBSD.org; Sun, 22 Aug 2021 21:00:41 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202108222100.17ML0f8L053813@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: fs@FreeBSD.org Subject: Problem reports for fs@FreeBSD.org that need special attention Date: Sun, 22 Aug 2021 21:00:41 +0000 List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="16296660411.73eABB9c3.52715" Content-Transfer-Encoding: 7bit X-ThisMailContainsUnwantedMimeParts: Y --16296660411.73eABB9c3.52715 Date: Sun, 22 Aug 2021 21:00:41 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- Open | 211491 | System hangs after "Uptime" on reboot with ZFS Open | 221909 | [ZFS] Add a sysctl to toggle send_corrupt_data Open | 237067 | ZFS: Crash in vdev_dtl_reassess when using GELI w Open | 240831 | zfs: Panic during snapshot on 12.1-STABLE r352648 Open | 243973 | [zfs] rollback segmentation fault Open | 244656 | zfs: resilver doesn't provide enough information Open | 244692 | gjournal: Does not support TRIM Open | 244899 | [PATCH] zfs: xattr on a symlink target > 136 caus 8 problems total for which you should take action. --16296660411.73eABB9c3.52715-- From nobody Sun Aug 22 21:23:53 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 750521770AB9 for ; Sun, 22 Aug 2021 21:23:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gt7fK2dYTz3hqD for ; Sun, 22 Aug 2021 21:23:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 442C627F35 for ; Sun, 22 Aug 2021 21:23:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 17MLNrmB067957 for ; Sun, 22 Aug 2021 21:23:53 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 17MLNrAh067956 for fs@FreeBSD.org; Sun, 22 Aug 2021 21:23:53 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 257316] kstat.zfs sysctl tree unreachable for pools with "." in the name Date: Sun, 22 Aug 2021 21:23:53 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: commit-hook@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: asomers@FreeBSD.org X-Bugzilla-Flags: mfc-stable13? mfc-stable12? mfc-stable11? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D257316 --- Comment #2 from commit-hook@FreeBSD.org --- A commit in branch stable/13 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=3De7d99efb3638b71756da8033ef8cef742= 13c42fd commit e7d99efb3638b71756da8033ef8cef74213c42fd Author: Alan Somers AuthorDate: 2021-07-21 21:11:00 +0000 Commit: Alan Somers CommitDate: 2021-08-22 21:11:00 +0000 Escape any '.' characters in sysctl node names ZFS creates some sysctl nodes that include a pool name, and '.' is an allowed character in pool names. But it's the separator in the sysctl tree, so it can't be included in a sysctl name. Replace it with "%25". Handily, "%" is illegal in ZFS pool names, so there's no ambiguity there. PR: 257316 Sponsored by: Axcient Reviewed by: freqlabs Differential Revision: https://reviews.freebsd.org/D31265 (cherry picked from commit 6c9506559080da2914749bf611225d7c0a153609) sys/kern/kern_sysctl.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++= -- 1 file changed, 46 insertions(+), 2 deletions(-) --=20 You are receiving this mail because: You are on the CC list for the bug.= From nobody Sun Aug 22 23:04:11 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 58B4117748AB for ; Sun, 22 Aug 2021 23:04:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gt9t30w6rz4hRd for ; Sun, 22 Aug 2021 23:04:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 07F231378 for ; Sun, 22 Aug 2021 23:04:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 17MN4AWb016813 for ; Sun, 22 Aug 2021 23:04:10 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 17MN4AIj016812 for fs@FreeBSD.org; Sun, 22 Aug 2021 23:04:10 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 257316] kstat.zfs sysctl tree unreachable for pools with "." in the name Date: Sun, 22 Aug 2021 23:04:11 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: commit-hook@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: asomers@FreeBSD.org X-Bugzilla-Flags: mfc-stable13? mfc-stable12? mfc-stable11? X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D257316 --- Comment #3 from commit-hook@FreeBSD.org --- A commit in branch stable/12 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=3D98c467192082b3d4a6c91eeaa80868bb5= 231534c commit 98c467192082b3d4a6c91eeaa80868bb5231534c Author: Alan Somers AuthorDate: 2021-07-21 21:11:00 +0000 Commit: Alan Somers CommitDate: 2021-08-22 22:43:50 +0000 Escape any '.' characters in sysctl node names ZFS creates some sysctl nodes that include a pool name, and '.' is an allowed character in pool names. But it's the separator in the sysctl tree, so it can't be included in a sysctl name. Replace it with "%25". Handily, "%" is illegal in ZFS pool names, so there's no ambiguity there. PR: 257316 Sponsored by: Axcient Reviewed by: freqlabs Differential Revision: https://reviews.freebsd.org/D31265 (cherry picked from commit 6c9506559080da2914749bf611225d7c0a153609) sys/kern/kern_sysctl.c | 47 +++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 45 insertions(+), 2 deletions(-) --=20 You are receiving this mail because: You are on the CC list for the bug.= From nobody Sun Aug 22 23:05:12 2021 X-Original-To: fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 323DD177520A for ; Sun, 22 Aug 2021 23:05:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gt9vD0qsKz4hvm for ; Sun, 22 Aug 2021 23:05:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 04C4B12D9 for ; Sun, 22 Aug 2021 23:05:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 17MN5BtF016969 for ; Sun, 22 Aug 2021 23:05:11 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 17MN5BPB016968 for fs@FreeBSD.org; Sun, 22 Aug 2021 23:05:11 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 257316] kstat.zfs sysctl tree unreachable for pools with "." in the name Date: Sun, 22 Aug 2021 23:05:12 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 13.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: asomers@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: asomers@FreeBSD.org X-Bugzilla-Flags: mfc-stable13+ mfc-stable12+ mfc-stable11- X-Bugzilla-Changed-Fields: flagtypes.name bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 X-ThisMailContainsUnwantedMimeParts: N https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D257316 Alan Somers changed: What |Removed |Added ---------------------------------------------------------------------------- Flags|mfc-stable13?, |mfc-stable13+, |mfc-stable12?, |mfc-stable12+, |mfc-stable11? |mfc-stable11- Status|In Progress |Closed Resolution|--- |FIXED --- Comment #4 from Alan Somers --- Not MFCing to stable/11 because stable/11 doesn't create the kstat.zfs. sysctls. --=20 You are receiving this mail because: You are on the CC list for the bug.= From nobody Sun Aug 22 23:39:44 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 41925178AF9A for ; Sun, 22 Aug 2021 23:40:00 +0000 (UTC) (envelope-from ggm@algebras.org) Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com [IPv6:2a00:1450:4864:20::233]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GtBgL4y8Lz4rh6 for ; Sun, 22 Aug 2021 23:39:58 +0000 (UTC) (envelope-from ggm@algebras.org) Received: by mail-lj1-x233.google.com with SMTP id s3so28139012ljp.11 for ; Sun, 22 Aug 2021 16:39:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=algebras-org.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=MVB273vGrKdc9fN1PwqFUTe47V1aZIuurEqyzt6FFAM=; b=mic0vkm5A7B1d4ov2OtG+Oy5Qnqp4fliuQXZcL7dUr+Hbv3OL2+rtHQQBuc2/GawDV r9nuOje7kpnL+oGpt7Kw8hjjyHE9obfJ1SNHN9eABEtwYwJ7OfpnPvyml9CM1b7B7jJN UM4nNVj3Bpo1YcsMp4cyWZzzoP77obRyoBphHKW4sFwoMhiDL3vNyQ4jK4I8BUExkQjS 7U51yu3xuZaMLrT4ruea7j9FA+ykdwXgKsqbmadCgvGdyGo+fBrt5UXqOIh40AgBnDk0 z4z/WmmEGarnHR3PVOEU9nXAeViz8lntwSxnK42gVFt3KR6Z0Vy629Zo6XZHglna34mV dZbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=MVB273vGrKdc9fN1PwqFUTe47V1aZIuurEqyzt6FFAM=; b=t+s0lhsetNJlBA+M8oWxo1GJ8mXemTOL3L6TgvwaKEgi3OH5YznY7Ua9LDEONT0Jfa 8wT/YsaKFtUhaa0Q8u1+WK9HAJRjkt1zGXQt1CYsKpUw6+qqjogzSgLRHVqksNy2O6xe VX8bU78VUiAcyYjbdA/1fh9zEZBquKCnEJeBKuImtETW/UPGmPeiuekPW6LkDc+tu5kQ TkPZyCWb4zcvpNaiI38dcK/ZXfpjdhBEKzk/lSDfoEx58cxY14DpOqurDKABP9TEMv8A HOL3LZgVZXbNzWQ+BlfZbBa7R/pqJPtaFxFV7o2pWTL14EflVxyxF03AiDewoxU2JJF5 oa1w== X-Gm-Message-State: AOAM532tKOaeNmWukv3kJqnE/wq3C+DFjqPwnhHr8NHS3QHfJ8NWKN/1 AmZRSO9znG5AvJ3wLQ634PidzqKbn6WX1Em9XAmLhQ== X-Google-Smtp-Source: ABdhPJxBwjxLpdOvxAChGiQd+ucRaUKTZI41UiOVmNA6ln55gloMQ1+E5NODQjoNQ/rYVkxQItTisKFBQ/CWa4O1t78= X-Received: by 2002:a2e:8598:: with SMTP id b24mr20971226lji.209.1629675596807; Sun, 22 Aug 2021 16:39:56 -0700 (PDT) List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 References: <023225AD-2A97-47C5-9FE4-3ABF1BFD66F1@gmx.com> In-Reply-To: <023225AD-2A97-47C5-9FE4-3ABF1BFD66F1@gmx.com> From: George Michaelson Date: Mon, 23 Aug 2021 09:39:44 +1000 Message-ID: Subject: Re: ZFS on high-latency devices To: Ben RUBSON Cc: freebsd-fs@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4GtBgL4y8Lz4rh6 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=pass header.d=algebras-org.20150623.gappssmtp.com header.s=20150623 header.b=mic0vkm5; dmarc=none; spf=pass (mx1.freebsd.org: domain of ggm@algebras.org designates 2a00:1450:4864:20::233 as permitted sender) smtp.mailfrom=ggm@algebras.org X-Spamd-Result: default: False [0.36 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.88)[-0.885]; R_DKIM_ALLOW(-0.20)[algebras-org.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; NEURAL_SPAM_SHORT(1.00)[1.000]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; DMARC_NA(0.00)[algebras.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[algebras-org.20150623.gappssmtp.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::233:from]; NEURAL_SPAM_LONG(0.74)[0.740]; FREEMAIL_TO(0.00)[gmx.com]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-ThisMailContainsUnwantedMimeParts: N I don't want to abuse the subject line too much, but I can highly recommend the mbuffer approach, I've used this repeatedly, BSD-BSD and BSD-Linux. It definitely feels faster than SSH, since the 'no cipher' options were removed, and in the confusion of the HPC buffer changes. But, its not crypted on-the-wire. Mbuffer tuning is a bit of a black art: it would help enormously if there was some guidance on this, and personally I've never found the mbuffer -v option to work well: I get no real sense of how full or empty the buffer "is" or, if the use of sendmsg/recvmsg type buffer chains is better or worse. -G On Fri, Aug 20, 2021 at 6:19 PM Ben RUBSON wrote: > > > On 19 Aug 2021, at 11:37, Peter Jeremy wrote: > > > > (...) or a way to improve throughput doing "zfs recv" to a pool with a high RTT. > > You should use zfs send/receive through mbuffer, which will allow to sustain better throughput over high latency links. > Feel free to play with its buffer size parameters to find the better settings, depending on your link characteristics. > > Ben > > From nobody Sun Aug 22 23:48:13 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id D8AE7178E056 for ; Sun, 22 Aug 2021 23:48:31 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ot1-f41.google.com (mail-ot1-f41.google.com [209.85.210.41]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GtBsC4pvmz4tlN for ; Sun, 22 Aug 2021 23:48:31 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-ot1-f41.google.com with SMTP id g66-20020a9d12c8000000b0051aeba607f1so23165962otg.11 for ; Sun, 22 Aug 2021 16:48:31 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=bAr1argM5+pirInC5YXDXAMa2cimPSYBhy2mnwQj0fU=; b=mTnQQUeAQ3aaJpkP/yFlVqxh/TEaWqDV1wxLgQgDxdO6DlsrtN2V+8JW+MxAYN7+/b YtPq+mHh2jVSj4dFql+CDnwgEESVPhvFQWSRz98XxjBbwctgmivi9FXroEX9Cd2txvD3 t8EysHaoiO9YikALg+jUcArPmaT+H8h7TKvK7YFwNayk53gJ486d3JNUzP3BybiR5Aj1 IbyhBcIgc4P10itKO2ihf6EIrE4vwCJ9XDA4jip90sjwqbJbVaBT2mZ98B4UYz9xzh6+ fH2GBD/0Sni4T+vCTSy2Ddqj2WTDwnPU4j2SILzo//zEojE6mWyHJzfypzWVGl25i079 4rDw== X-Gm-Message-State: AOAM531F2VNHCWRw+EaxdKnHTj7REATOoyMBdSZI+DyWHwzybIYTX40B 9zYFVs9h2EmTPqU6iLDkGqj6tc/uuaFHyJWqIRywLH2e X-Google-Smtp-Source: ABdhPJxBAMZqXzUwlu+HMaGpXiaxICV4lpesUo2WFavvqhQqY/XTgkPkTkRNFMBMjqWqzyZ4HAgB+j8gbg9L85GB8Kw= X-Received: by 2002:a9d:d04:: with SMTP id 4mr26445297oti.251.1629676104826; Sun, 22 Aug 2021 16:48:24 -0700 (PDT) List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 References: <023225AD-2A97-47C5-9FE4-3ABF1BFD66F1@gmx.com> In-Reply-To: From: Alan Somers Date: Sun, 22 Aug 2021 17:48:13 -0600 Message-ID: Subject: Re: ZFS on high-latency devices To: George Michaelson Cc: Ben RUBSON , freebsd-fs Content-Type: multipart/alternative; boundary="00000000000053c29c05ca2e8922" X-Rspamd-Queue-Id: 4GtBsC4pvmz4tlN X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: Y --00000000000053c29c05ca2e8922 Content-Type: text/plain; charset="UTF-8" mbuffer is not going to help the OP. He's trying to create a pool on top of a networked block device. And if I understand correctly, he's connecting over a WAN, not a LAN. ZFS will never achieve decent performance in such a setup. It's designed as a local file system, and assumes it can quickly read metadata off of the disks at any time. The OP's best option is to go with "a": encrypt each dataset and send them with "zfs send --raw". I don't know why he thinks that it would be "very difficult". It's quite easy, if he doesn't care about old snapshots. Just: $ zfs create pool/new_dataset $ cp -a pool/old_dataset/* pool/new_dataset/ -Alan On Sun, Aug 22, 2021 at 5:40 PM George Michaelson wrote: > I don't want to abuse the subject line too much, but I can highly > recommend the mbuffer approach, I've used this repeatedly, BSD-BSD and > BSD-Linux. It definitely feels faster than SSH, since the 'no cipher' > options were removed, and in the confusion of the HPC buffer changes. > But, its not crypted on-the-wire. > > Mbuffer tuning is a bit of a black art: it would help enormously if > there was some guidance on this, and personally I've never found the > mbuffer -v option to work well: I get no real sense of how full or > empty the buffer "is" or, if the use of sendmsg/recvmsg type buffer > chains is better or worse. > > -G > > On Fri, Aug 20, 2021 at 6:19 PM Ben RUBSON wrote: > > > > > On 19 Aug 2021, at 11:37, Peter Jeremy wrote: > > > > > > (...) or a way to improve throughput doing "zfs recv" to a pool with a > high RTT. > > > > You should use zfs send/receive through mbuffer, which will allow to > sustain better throughput over high latency links. > > Feel free to play with its buffer size parameters to find the better > settings, depending on your link characteristics. > > > > Ben > > > > > > --00000000000053c29c05ca2e8922-- From nobody Sun Aug 22 23:54:39 2021 X-Original-To: freebsd-fs@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id BD5C3179051B for ; Sun, 22 Aug 2021 23:54:51 +0000 (UTC) (envelope-from ggm@algebras.org) Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com [IPv6:2a00:1450:4864:20::133]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GtC0W4Sp5z3CVT for ; Sun, 22 Aug 2021 23:54:51 +0000 (UTC) (envelope-from ggm@algebras.org) Received: by mail-lf1-x133.google.com with SMTP id u22so34102291lfq.13 for ; Sun, 22 Aug 2021 16:54:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=algebras-org.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=XdFsrim2L3LuBwNh362BPW3tLWGSnm6KW8cP86fTv+A=; b=f8APggIBAjzaGLYM8v1KF0JvCXTOUGhl3/dhd+xSwWgIW5flWpWV9DOIpYyBGB2cBN qoUBx8DgWPKcBFV4OtjS6WCkgAcxdZSkY8XNb1yZVepnxSA9Pn+ElqcsVeX49QIczjhD XRwiZG4DqTs4TGOcU5P45cfNGp5dlIdnSFb5E+T2LIL/vCLX9XH5mZo4VazeofHbIaal +JSVAWuRORALtHT8XqZb5rFg0NNq6nd6pvyXsqUgGFIgHeN/BJ0foT/XjayWmleByxWH kC9Ig9lUwJwQG8FSzfvgtuIiiZOOIh06dA6SOM4Yk2rYhLo9EjyWnGSengEFGoK3kYQY AN/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=XdFsrim2L3LuBwNh362BPW3tLWGSnm6KW8cP86fTv+A=; b=Th1Jp6MjTqmb/FPvFuepjpIbdoEFO4ImjeT5HfpFXjzrKWSPDwaeMuIxpQ2GQXI7dk Kvdci3VqBmEVOakjWJo5HPFoppa3V3zMkeUKoWiW7dRzG1pYcd3QHqvF1QBq4Mxd3YJA R496u7xuJ5si/9oRZ/CPnbxQz5vmbY+7oZyGkwg24pjBGIxcRHwlwjY9L8IBMsxemgxf zh4e1p9dXAcjMg0f02mEayDkXuOQKpKVIn9a73Q8v/eD4xdQs7KcFR6TyUhWNpJdZaR4 rwBFu703Bz4HJKJ4EeRTBPS8BvSoOhbaaNrlbbeXh4OlNr2q0A+gratqojlWmmJoXBfu dxfw== X-Gm-Message-State: AOAM530pCCO3jZOpuDOlEvRlGWRU9aC01e9OEi/M+q6P194uhIgsgK9Y Z60bwtuRrgQd1ZJjDmMvz8FTEc5U8stI2iD375J3nQ== X-Google-Smtp-Source: ABdhPJwMS6T5gscGAdu5wpTlqamoUUDNH8GNeieIe4tQGEIw3bwFPb/0XxJThHdWZsCH6HKDy7C1XEuOJ/yVv59HFy0= X-Received: by 2002:a05:6512:16a0:: with SMTP id bu32mr23133507lfb.322.1629676490187; Sun, 22 Aug 2021 16:54:50 -0700 (PDT) List-Id: Filesystems List-Archive: https://lists.freebsd.org/archives/freebsd-fs List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-fs@freebsd.org MIME-Version: 1.0 References: <023225AD-2A97-47C5-9FE4-3ABF1BFD66F1@gmx.com> In-Reply-To: From: George Michaelson Date: Mon, 23 Aug 2021 09:54:39 +1000 Message-ID: Subject: Re: ZFS on high-latency devices To: Alan Somers Cc: Ben RUBSON , freebsd-fs Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4GtC0W4Sp5z3CVT X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N I don't think its sensible to mesh long-delay file constructs into a pool. Maybe there is a model which permits this but I think higher abstractions like CEPH may be a better fit for the kind of distributed filestore this implies. I use ZFS send/receive to make "clones" of a zpool/zfs structure exist, and they are not bound in the delay problem for live FS delivery: you use mbuffer to make the transport of the entire ZFS datastate more effective. A long time ago, 25+ years ago I ran NFS over X25, as well as SMB. (ip over X25) and it was very painful. Long-haul, long delay networks are not a good fit for direct FS semantics of read/write if you care about speed. There isn't enough cacheing in the world to make the distance fully transparent. When I have to use FS-over- now it's typically to mount virtual ISO images for console-FS recovery of a host, and its bad enough over 150ms delay fibre links not to want to depend on it beyond the bootstrap phase. -G On Mon, Aug 23, 2021 at 9:48 AM Alan Somers wrote: > > mbuffer is not going to help the OP. He's trying to create a pool on top= of a networked block device. And if I understand correctly, he's connecti= ng over a WAN, not a LAN. ZFS will never achieve decent performance in suc= h a setup. It's designed as a local file system, and assumes it can quickl= y read metadata off of the disks at any time. The OP's best option is to g= o with "a": encrypt each dataset and send them with "zfs send --raw". I do= n't know why he thinks that it would be "very difficult". It's quite easy,= if he doesn't care about old snapshots. Just: > > $ zfs create pool/new_dataset > $ cp -a pool/old_dataset/* pool/new_dataset/ > > -Alan > > On Sun, Aug 22, 2021 at 5:40 PM George Michaelson wrot= e: >> >> I don't want to abuse the subject line too much, but I can highly >> recommend the mbuffer approach, I've used this repeatedly, BSD-BSD and >> BSD-Linux. It definitely feels faster than SSH, since the 'no cipher' >> options were removed, and in the confusion of the HPC buffer changes. >> But, its not crypted on-the-wire. >> >> Mbuffer tuning is a bit of a black art: it would help enormously if >> there was some guidance on this, and personally I've never found the >> mbuffer -v option to work well: I get no real sense of how full or >> empty the buffer "is" or, if the use of sendmsg/recvmsg type buffer >> chains is better or worse. >> >> -G >> >> On Fri, Aug 20, 2021 at 6:19 PM Ben RUBSON wrote: >> > >> > > On 19 Aug 2021, at 11:37, Peter Jeremy wrote: >> > > >> > > (...) or a way to improve throughput doing "zfs recv" to a pool with= a high RTT. >> > >> > You should use zfs send/receive through mbuffer, which will allow to s= ustain better throughput over high latency links. >> > Feel free to play with its buffer size parameters to find the better s= ettings, depending on your link characteristics. >> > >> > Ben >> > >> > >>