From owner-freebsd-fs@freebsd.org Sun May 12 21:00:55 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A3BE315A22EA for ; Sun, 12 May 2019 21:00:55 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 491FC8AE20 for ; Sun, 12 May 2019 21:00:55 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id 095D115A22CB; Sun, 12 May 2019 21:00:55 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C162B15A22C9 for ; Sun, 12 May 2019 21:00:54 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5E52F8AE14 for ; Sun, 12 May 2019 21:00:54 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id B70AA1F571 for ; Sun, 12 May 2019 21:00:53 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x4CL0roh041728 for ; Sun, 12 May 2019 21:00:53 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x4CL0r6C041727 for fs@FreeBSD.org; Sun, 12 May 2019 21:00:53 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201905122100.x4CL0r6C041727@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: fs@FreeBSD.org Subject: Problem reports for fs@FreeBSD.org that need special attention Date: Sun, 12 May 2019 21:00:53 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 12 May 2019 21:00:56 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 203492 | mount_unionfs -o below causes panic Open | 144447 | [zfs] sharenfs fsunshare() & fsshare_main() non f Open | 211491 | System hangs after "Uptime" on reboot with ZFS Open | 221909 | [ZFS] Add a sysctl to toggle send_corrupt_data Open | 237067 | ZFS: Crash in vdev_dtl_reassess when using GELI w 5 problems total for which you should take action. From owner-freebsd-fs@freebsd.org Mon May 13 11:07:34 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 837D7158AF08 for ; Mon, 13 May 2019 11:07:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 1C1A3745D5 for ; Mon, 13 May 2019 11:07:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id CD7B3158AF07; Mon, 13 May 2019 11:07:33 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB135158AF06 for ; Mon, 13 May 2019 11:07:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4A0DB745D3 for ; Mon, 13 May 2019 11:07:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 811407261 for ; Mon, 13 May 2019 11:07:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x4DB7WTo020574 for ; Mon, 13 May 2019 11:07:32 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x4DB7WtA020573 for fs@FreeBSD.org; Mon, 13 May 2019 11:07:32 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 237807] ZFS: ZVOL writes fast, ZVOL reads abysmal... Date: Mon, 13 May 2019 11:07:32 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: nbe@renzel.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 May 2019 11:07:34 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D237807 --- Comment #3 from Nils Beyer --- (In reply to sigsys from comment #2) the benchmark tool is completely sequential as it is a simple: dd if=3D/dev/zvol/morsleben-grube2/test of=3D/dev/null bs=3D1M ^C 375390208 bytes transferred in 15.330245 secs (24486902 bytes/sec) whereas the ZVOL has been filled before with random data using: dd if=3D/dev/urandom of=3D/dev/zvol/morsleben-grube2/test bs=3D1M ^C 8544845824 bytes transferred in 77.684268 secs (109994546 bytes/sec) "primarycache" and "secondarycache" are disabled for that ZVOL to get real = read speeds... Now, I've followed your advice and tried two identical instances of "dd" reading from the ZVOL. And you are right, I get twice the throughput: 40MB/s now. It still is abysmal if you ask me, but okay... The slow read throughput also exists via the iSCSI connection from a Windows Server (the ZPOOL "morsleben-grub2" is a backup target). So I have to convi= nce the Microsoft iSCSI initiator somehow to create more concurrent reads. Thank you for the pointers... --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon May 13 11:11:46 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D16A5158B218 for ; Mon, 13 May 2019 11:11:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 67B1874909 for ; Mon, 13 May 2019 11:11:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 2BA45158B217; Mon, 13 May 2019 11:11:46 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A02C158B216 for ; Mon, 13 May 2019 11:11:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AE8AF74907 for ; Mon, 13 May 2019 11:11:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id ECCB473AC for ; Mon, 13 May 2019 11:11:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x4DBBiEw030051 for ; Mon, 13 May 2019 11:11:44 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x4DBBiVS030049 for fs@FreeBSD.org; Mon, 13 May 2019 11:11:44 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 237807] ZFS: ZVOL writes fast, ZVOL reads abysmal... Date: Mon, 13 May 2019 11:11:44 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: crest@rlwinm.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 May 2019 11:11:47 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D237807 crest@rlwinm.de changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |crest@rlwinm.de --- Comment #4 from crest@rlwinm.de --- Can you retry with primarycache=3Dmetadata? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon May 13 11:51:41 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A461C158CDF1 for ; Mon, 13 May 2019 11:51:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 3A95F7608A for ; Mon, 13 May 2019 11:51:41 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id EEF7B158CDEF; Mon, 13 May 2019 11:51:40 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DD7A6158CDEE for ; Mon, 13 May 2019 11:51:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7E08D76087 for ; Mon, 13 May 2019 11:51:40 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id BD217797F for ; Mon, 13 May 2019 11:51:39 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x4DBpdSr012039 for ; Mon, 13 May 2019 11:51:39 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x4DBpdl6012038 for fs@FreeBSD.org; Mon, 13 May 2019 11:51:39 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 237807] ZFS: ZVOL writes fast, ZVOL reads abysmal... Date: Mon, 13 May 2019 11:51:39 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: nbe@renzel.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 May 2019 11:51:41 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D237807 --- Comment #5 from Nils Beyer --- (In reply to crest from comment #4) with "primarycache=3Dmetadata" and "secondarycache=3Dnone" much much better= ... Factor ten; get around 200MB/s now (no joke) with one single "dd". With two "dd" instances, I get around 400MB/s. What is the logic behind that "primarycache=3Dmetadata" is actually more performant than "primarycache=3Dall"? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon May 13 13:24:56 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA519158FFFD for ; Mon, 13 May 2019 13:24:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 2F6D1817CA for ; Mon, 13 May 2019 13:24:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id E724C158FFFC; Mon, 13 May 2019 13:24:55 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D4C0A158FFFB for ; Mon, 13 May 2019 13:24:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5B3C4817C7 for ; Mon, 13 May 2019 13:24:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 8BDA6867A for ; Mon, 13 May 2019 13:24:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x4DDOs1M031937 for ; Mon, 13 May 2019 13:24:54 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x4DDOscQ031936 for fs@FreeBSD.org; Mon, 13 May 2019 13:24:54 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 237807] ZFS: ZVOL writes fast, ZVOL reads abysmal... Date: Mon, 13 May 2019 13:24:54 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: crest@rlwinm.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 May 2019 13:24:56 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D237807 --- Comment #6 from crest@rlwinm.de --- There are three caching levels (all, metadata, none). At the default level = of all metadata and data get cached. This offers the best performance in most cases, but it can become a problem if it leads to double buffering (e.g. on= a VM host with data being cached once by the host and again by the guest OS). This can cause both caches to fight for resources. Disabling all caching requires ZFS to walk the B-tree on disk for every access which is very slow. Limiting caching to metadata is a compromise between the two extremes. Limi= ted to metadata caching ZFS caches everything but the actual data. This avoids = the need to (re-)read metadata from disk without double caching. The rule of thumb is to cache data just once per physical system as close to the user as possible (it's better to cache inside a VM than outside of it). Also keep in mind that the L2ARC is a pure victim cache. If the primary cac= he is limited to metadata it will never contain anything but metadata to be evicted to the secondary cache. Can you tell us more about your setup? What are the ZVOLs used for? What el= se is running on the system? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon May 13 13:58:34 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4224B1590C8D for ; Mon, 13 May 2019 13:58:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id D2FB882D02 for ; Mon, 13 May 2019 13:58:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 965151590C8C; Mon, 13 May 2019 13:58:33 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 73F2E1590C8B for ; Mon, 13 May 2019 13:58:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0EE0882CFF for ; Mon, 13 May 2019 13:58:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 344DA8AB4 for ; Mon, 13 May 2019 13:58:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x4DDwWde096911 for ; Mon, 13 May 2019 13:58:32 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x4DDwW1R096909 for fs@FreeBSD.org; Mon, 13 May 2019 13:58:32 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 237807] ZFS: ZVOL writes fast, ZVOL reads abysmal... Date: Mon, 13 May 2019 13:58:32 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: nbe@renzel.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 May 2019 13:58:34 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D237807 --- Comment #7 from Nils Beyer --- (In reply to crest from comment #6) the ZVOLs are used for ReFS-formatted (64kB cluster size) Veeam backup repositories connected via iSCSI and 10GBit: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D morsleben-grube2/dshyp02-veeam type volume = - morsleben-grube2/dshyp02-veeam creation Fri Mar 29 11:01 2019= - morsleben-grube2/dshyp02-veeam used 40.2T = - morsleben-grube2/dshyp02-veeam available 12.8T = - morsleben-grube2/dshyp02-veeam referenced 30.4T = - morsleben-grube2/dshyp02-veeam compressratio 1.00x = - morsleben-grube2/dshyp02-veeam reservation none=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20 default morsleben-grube2/dshyp02-veeam volsize 40T=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20 local morsleben-grube2/dshyp02-veeam volblocksize 64K = - morsleben-grube2/dshyp02-veeam checksum on=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20 default morsleben-grube2/dshyp02-veeam compression lz4=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20 inherited from morsleben-grube2 morsleben-grube2/dshyp02-veeam readonly off=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20 default morsleben-grube2/dshyp02-veeam createtxg 76 = - morsleben-grube2/dshyp02-veeam copies 1=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20=20=20 default morsleben-grube2/dshyp02-veeam refreservation 40.2T=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20 local morsleben-grube2/dshyp02-veeam guid 10198386066639651165 = - morsleben-grube2/dshyp02-veeam primarycache metadata=20=20=20=20= =20=20=20=20=20=20=20=20=20=20 local morsleben-grube2/dshyp02-veeam secondarycache none=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20 local morsleben-grube2/dshyp02-veeam usedbysnapshots 0 = - morsleben-grube2/dshyp02-veeam usedbydataset 30.4T = - morsleben-grube2/dshyp02-veeam usedbychildren 0 = - morsleben-grube2/dshyp02-veeam usedbyrefreservation 9.78T = - morsleben-grube2/dshyp02-veeam logbias latency=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20 default morsleben-grube2/dshyp02-veeam dedup off=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20 default morsleben-grube2/dshyp02-veeam mlslabel = - morsleben-grube2/dshyp02-veeam sync standard=20=20=20=20= =20=20=20=20=20=20=20=20=20=20 default morsleben-grube2/dshyp02-veeam refcompressratio 1.00x = - morsleben-grube2/dshyp02-veeam written 30.4T = - morsleben-grube2/dshyp02-veeam logicalused 30.5T = - morsleben-grube2/dshyp02-veeam logicalreferenced 30.5T = - morsleben-grube2/dshyp02-veeam volmode default=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20 default morsleben-grube2/dshyp02-veeam snapshot_limit none=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20 default morsleben-grube2/dshyp02-veeam snapshot_count none=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20 default morsleben-grube2/dshyp02-veeam redundant_metadata all=20=20=20=20=20=20= =20=20=20=20=20=20=20=20=20=20=20=20=20 default =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D Using CTL as the iSCSI target (4k blocksize) for the Windows Server 2016 ho= sts: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D lun 0 { path /dev/zvol/morsleben-grube2/dshyp02-veeam blocksize 4k } =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D Nothing else besides "ctld" is running on the storage system... --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed May 15 07:52:15 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AF65C15AE6E1 for ; Wed, 15 May 2019 07:52:15 +0000 (UTC) (envelope-from ganbold@gmail.com) Received: from mail-qk1-x735.google.com (mail-qk1-x735.google.com [IPv6:2607:f8b0:4864:20::735]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CC1B3803EE for ; Wed, 15 May 2019 07:52:14 +0000 (UTC) (envelope-from ganbold@gmail.com) Received: by mail-qk1-x735.google.com with SMTP id a132so885809qkb.13 for ; Wed, 15 May 2019 00:52:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=YgEIqFIXREDLFFQ4YHCmyhrnoAw0SCBWxz1J5D+08CQ=; b=uy4SyilFpBUyfA8fzKKcdlSopiJ8c40lcXkkKqWU16YY2vs/WcUqPPKBLZRAyuXuEI 3Lg1yAu9sqA5JFw4xOrzwVZd2Xzj+Wt1050ROtnE+x00DWvv5up5h0p3oFcNIxuH+NmK /svF6xAJda7CrQ0a47rOi+417yGARSDcD4YSH8PIk90ptZubTgjSYi3iaGU9X37Z+TIg MaF2B7K+bDpKEAelKACHvvQUxwFUt2IeTI7cv01T4FlN2GDNDUTtX1hsLYfs3MJ0+Wa4 YN/I6oyPHqN9eVF6VOnC+fVu0r3QIG82KdF055X2ZInje7aa2DzgBbiZDqW/ohTGsRpn bTOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=YgEIqFIXREDLFFQ4YHCmyhrnoAw0SCBWxz1J5D+08CQ=; b=O7+kmpp0vSJXER/fwDbV6EAPAe5uGJF/IGGwSoyzTBNvL9/pVraZfPlBXbzfq3ybdh Aes9kHTDR2zwk+NkWaRbMvr6CjSfTLqKaycWXUkj9Yt/H22sdoCvFFCxTZ5yq94L9DqT ayUoj4ZsEHqtt7GDgWJiF1y4CPfzpUvs2gUpC4CXvDRFxPor1A56tmMaVukYQMFBVQgl BstTgJEoaqCX3VB8LUKfG1ovowPrNlaul8/xaZuq5BOrRikZ3xJ//BQgQw1gU8ziErVd ZtOxfXdz8b021MAHDExqrYQt2Dr8KLehX7ZhHh6zp+JVvGM0hCeLaISFjrmWVEc19WSy GaHQ== X-Gm-Message-State: APjAAAWJyfAcSDbwZy85GzDk6d/cBXr0o0+EpphqS8yCjQC7Kjj25UgY zwPGUgE7d0COPghni8MenpjfQnXvf9aaIKF4kMxYl/KW X-Google-Smtp-Source: APXvYqxkRxIP+iWXdktdNrVq6RNbY35ZxmTOJenhbT/rU6JB+RHdgACSVQIXWUU/rzPZLt/KEJOGTHKN2X+kufEbVkU= X-Received: by 2002:a37:9cd5:: with SMTP id f204mr32025349qke.75.1557906734193; Wed, 15 May 2019 00:52:14 -0700 (PDT) MIME-Version: 1.0 From: Ganbold Tsagaankhuu Date: Wed, 15 May 2019 15:52:03 +0800 Message-ID: Subject: NFS file access log To: freebsd-fs@freebsd.org X-Rspamd-Queue-Id: CC1B3803EE X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=uy4SyilF; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of ganbold@gmail.com designates 2607:f8b0:4864:20::735 as permitted sender) smtp.mailfrom=ganbold@gmail.com X-Spamd-Result: default: False [-6.92 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[5.3.7.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; IP_SCORE(-2.99)[ip: (-9.36), ipnet: 2607:f8b0::/32(-3.26), asn: 15169(-2.26), country: US(-0.06)]; NEURAL_HAM_SHORT(-0.92)[-0.925,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 May 2019 07:52:16 -0000 Hi, Is there anyway to log file access in NFS client/server side? I'm experiencing following problem: vm_fault: pager read error, pid 75363 (php-fpm) ncl_getpages: error 70 and would like to know if there is possibility to find problematic files. thanks, Ganbold From owner-freebsd-fs@freebsd.org Wed May 15 12:06:48 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BA483158F79B for ; Wed, 15 May 2019 12:06:47 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: from mail-it1-x131.google.com (mail-it1-x131.google.com [IPv6:2607:f8b0:4864:20::131]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6137588FB3 for ; Wed, 15 May 2019 12:06:46 +0000 (UTC) (envelope-from kris@ixsystems.com) Received: by mail-it1-x131.google.com with SMTP id q132so4349686itc.5 for ; Wed, 15 May 2019 05:06:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ixsystems-com.20150623.gappssmtp.com; s=20150623; h=from:to:subject:date:message-id:mime-version:thread-index :content-language; bh=twPC9cPgXdedLetM6NoJDL4WnoRkOt3Im72xh4mP1eA=; b=Kp/aZlj+rcrWWFe1LP0Ux1vWvfWqFLCyRdvkwIdPwgl4/rfqaqbOBVt1eBnrMV8Ln5 zgogLDdGztjrx1kInJDdyGPlv4B5lNK/yKMMlDDku+7e2QJRa1juptOkXE0zpnX3vbne RaIUpeeJZIki8YBlc9N9ovbCZxNj8ilnd0QYXQlcv6wq+5e4tKJsXt+1+4dTercEB3lu aL1H1FS1tqrdCacA7JtS0fcYiUEsCnJ3FAbDm8jOipVVh9pc6BdAwftQFIvGz22vLx6i 0iuRuZMqqMcI/jAw9XYfSwCAHvaU4xSYGMbCLn8PBWqLfaay0Wrzmh9veVgyBC74yi5K O4mA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :thread-index:content-language; bh=twPC9cPgXdedLetM6NoJDL4WnoRkOt3Im72xh4mP1eA=; b=RMAsp0b1f/Tc4+kb/SWqmneEJtCmY3awWSyGkZG7omnv0h/lk3RNckYumAuNlJ00fJ QxlxN7buo6JjfjJ82pQzIcno/olqFFO6o1/wZhW8bdzF58n3+KEe9kx+XuEtaZxnKtPp TOHU4M7mpOWu/W5w3Z1ic6UgJJ4yd8pMGjvWtOpvYp8zDQPTZNYEyNiZAEBWn2eZxADf kHCaVNPj5vbsvWKXtT3A+GzjNsXetqlRaojSLJeuRE6iKturvj44TKRiul0cKf1ZIi/c viJPJf0Kz0LgnJkqmpjDism7vnCb2UI0eTB/T71wq7uZJhJz2RuTk/3rlGTpfIAexfrt jteQ== X-Gm-Message-State: APjAAAV+ptQRhzLKlJbn3FPSqmn5Qr02fVtnbIxsoB7CAAkmD/Bm7MSe m1+ZREWzg7cVLaTFG4/jTmZRNA== X-Google-Smtp-Source: APXvYqyN+3ZRbfKcwE9H7+dtYl6Q7oEkdyKX0QaytcUoxD5ui5L84bhZAgnEQAvh7hb4YpD3D+jC3g== X-Received: by 2002:a05:6638:606:: with SMTP id g6mr27541408jar.129.1557922005705; Wed, 15 May 2019 05:06:45 -0700 (PDT) Received: from KrisYogaC930 ([66.171.165.146]) by smtp.gmail.com with ESMTPSA id y97sm33353ita.19.2019.05.15.05.06.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 15 May 2019 05:06:44 -0700 (PDT) From: To: "'freebsd-fs'" , "'FreeBSD Current'" Subject: Updated Install Images with ZoL Date: Wed, 15 May 2019 08:06:43 -0400 Message-ID: <002301d50b16$a9b47660$fd1d6320$@ixsystems.com> MIME-Version: 1.0 X-Mailer: Microsoft Outlook 16.0 Thread-Index: AdULFj1bcJ5lug+tQoCBBaz0TCchsg== Content-Language: en-us X-Rspamd-Queue-Id: 6137588FB3 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ixsystems-com.20150623.gappssmtp.com header.s=20150623 header.b=Kp/aZlj+; dmarc=pass (policy=none) header.from=ixsystems.com; spf=pass (mx1.freebsd.org: domain of kris@ixsystems.com designates 2607:f8b0:4864:20::131 as permitted sender) smtp.mailfrom=kris@ixsystems.com X-Spamd-Result: default: False [-6.60 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[ixsystems-com.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; IP_SCORE(-2.63)[ip: (-7.59), ipnet: 2607:f8b0::/32(-3.25), asn: 15169(-2.26), country: US(-0.06)]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[ixsystems-com.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[cached: ALT3.ASPMX.L.GOOGLE.com]; FROM_NO_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[1.3.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.96)[-0.958,0]; DMARC_POLICY_ALLOW(-0.50)[ixsystems.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; MID_RHS_MATCH_FROM(0.00)[] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 May 2019 12:06:48 -0000 We're started rolling FreeBSD weekly HEAD images that include the most up to date ZoL ports, selectable at install time: http://pkg.trueos.org/iso/freebsd-pkgbase/ Feel free to give those a shot if you want to test ZoL. As usual, report issues to us via the GitHub issue tracker: https://github.com/zfsonfreebsd/ZoF/issues Thanks to everybody who's been helping us test so far! -- Kris Moore Vice President of Engineering iXsystems, Inc Ph: (408) 943-4100 Ph: (408) 943-4101 The Groundbreaking TrueNAS M-Series - Enterprise Storage & Servers Driven By Open Source From owner-freebsd-fs@freebsd.org Wed May 15 14:26:45 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 76F9C1592C35 for ; Wed, 15 May 2019 14:26:45 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B19918D3F8 for ; Wed, 15 May 2019 14:26:43 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Date: Wed, 15 May 2019 16:26:35 +0200 (CEST) From: Ronald Klop To: Ganbold Tsagaankhuu Cc: freebsd-fs@freebsd.org Message-ID: <290003690.37.1557930395569@localhost> In-Reply-To: References: Subject: Re: NFS file access log MIME-Version: 1.0 X-Mailer: Realworks (459.489.e46a8f03723) Importance: Normal X-Priority: 3 (Normal) X-Rspamd-Queue-Id: B19918D3F8 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of ronald-lists@klop.ws designates 194.109.157.24 as permitted sender) smtp.mailfrom=ronald-lists@klop.ws X-Spamd-Result: default: False [-0.64 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.85)[-0.848,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:194.109.157.0/24]; NEURAL_HAM_LONG(-0.96)[-0.962,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[klop.ws]; URI_COUNT_ODD(1.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.03)[0.031,0]; MX_GOOD(-0.01)[mx2.greenhost.nl,mx1.greenhost.nl]; RCPT_COUNT_TWO(0.00)[2]; HAS_X_PRIO_THREE(0.00)[3]; RCVD_IN_DNSWL_NONE(0.00)[24.157.109.194.list.dnswl.org : 127.0.15.0]; IP_SCORE(-0.05)[ipnet: 194.109.0.0/16(-0.14), asn: 3265(-0.12), country: NL(0.01)]; FREEMAIL_TO(0.00)[gmail.com]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MID_RHS_NOT_FQDN(0.50)[]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_ALL(0.00)[] Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 May 2019 14:26:45 -0000 DTrace might help. https://books.google.nl/books?id=jseJ56fUjJgC&lpg=PA163&ots=3M_bkEb6pj&dq=freebsd%20dtrace%20nfs&hl=nl&pg=PA163#v=onepage&q=freebsd%20dtrace%20nfs&f=false Never used it for NFS, so I'm not sure. Ronald. Van: Ganbold Tsagaankhuu Datum: woensdag, 15 mei 2019 09:52 Aan: freebsd-fs@freebsd.org Onderwerp: NFS file access log > > Hi, > > Is there anyway to log file access in NFS client/server side? > I'm experiencing following problem: > > vm_fault: pager read error, pid 75363 (php-fpm) > ncl_getpages: error 70 > > and would like to know if there is possibility to find problematic files. > > thanks, > > Ganbold > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > > From owner-freebsd-fs@freebsd.org Wed May 15 14:57:39 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3E63A1594A1E for ; Wed, 15 May 2019 14:57:39 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0F8DA8F0C0 for ; Wed, 15 May 2019 14:57:38 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: by mail-io1-xd32.google.com with SMTP id s20so2458146ioj.7 for ; Wed, 15 May 2019 07:57:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=r9SsyBsu+C/bwz6QcvLfGiTIPSDNEybhHVqO3Kdp04o=; b=CPUgnbeQXqsgzq6kAv0RBbNSdfUDW2wkyJLFMhh5PIH351wuYJyKCOq/LTvQ9hpqwX GsQbJ59qlchO08ecksNy2gcHWVvUDbO7zCMnS1IP8ymSWrgoFb902mLVlQLdDQYWaDx3 lOufc1rmWPma00dBU/T9zcL9/ueMpGJyioHJLxRy9rkVqOS0oMIaAXa1d0x0bKIdnLjO abIPlkcSNIobW91veh5eX8Jy3tCv151z3+JbaqcizPTEMXGqrf0/ho5KLu3zYLVJLG+C nl8r9vTDOE+Lzi+ijSSDgNIjVpox221uccXYjWxOuMwq03z5O+egeOtt+gYjGJkDDAWl c3Tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=r9SsyBsu+C/bwz6QcvLfGiTIPSDNEybhHVqO3Kdp04o=; b=DzxKnhQtNMCPZ0Ud8rAyDZtEF9Q0xc9QmpvP1BNoJdsUHzerWQu5HLn9ldmqyVoapQ 3fVUHQATZBo3NEIuIQsW4cPtX0t/FyXLO+YBkTFcTiUIWUCJu1fwwV+EpB9xKHAHAlp/ VYck6VL6pLJlqZysVJlmT+aIOmdFL1l1hfVvLGVlX5z3zXeCOJadyYomgFWKCGGhRu+v l0kjCUC7vzFreojC4aaZrber83PkPHdXkR2K1rtcORB07VKcLFus+nXZvdAsHVvkeWN1 PKbnjcyST/f9cDzl54ecjUxpY3kdFjr4JdAUVk4YWVCJs4O1V6B1ff2oqZax141inFXs QXDQ== X-Gm-Message-State: APjAAAURhk2E2KtUpywaGQvxuc7Z30vJ6pyKOMn+8nXJYcrCRjJct9kw RbAUjq/rXI5qGlKXOkJwC0iqeBe6kkzjGZkEoBg= X-Google-Smtp-Source: APXvYqy3jFBDkjh25EJVyYEr9l6S7bfxu8Yly8H1imthtHQJFPkhxlrrbNAOGAWPTOpOJr/MFPPsYLyCqAS4DpyGJCs= X-Received: by 2002:a6b:fe0d:: with SMTP id x13mr23902476ioh.1.1557932257365; Wed, 15 May 2019 07:57:37 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Adam Date: Wed, 15 May 2019 09:57:25 -0500 Message-ID: Subject: Re: NFS file access log To: Ganbold Tsagaankhuu Cc: freebsd-fs X-Rspamd-Queue-Id: 0F8DA8F0C0 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=CPUgnbeQ; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of amvandemore@gmail.com designates 2607:f8b0:4864:20::d32 as permitted sender) smtp.mailfrom=amvandemore@gmail.com X-Spamd-Result: default: False [-6.78 / 15.00]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_SHORT(-1.00)[-0.998,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2.3.d.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; IP_SCORE(-2.77)[ip: (-8.26), ipnet: 2607:f8b0::/32(-3.26), asn: 15169(-2.27), country: US(-0.06)]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 May 2019 14:57:39 -0000 On Wed, May 15, 2019 at 2:53 AM Ganbold Tsagaankhuu wrote: > Hi, > > Is there anyway to log file access in NFS client/server side? > I'm experiencing following problem: > > vm_fault: pager read error, pid 75363 (php-fpm) > ncl_getpages: error 70 > > and would like to know if there is possibility to find problematic files. > https://www.freebsd.org/doc/handbook/audit.html could used, but seems easier just to add some logging to the php app. And make sure file locking is working as desired. -- Adam From owner-freebsd-fs@freebsd.org Wed May 15 15:06:24 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3CAC41594E62 for ; Wed, 15 May 2019 15:06:24 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-QB1-obe.outbound.protection.outlook.com (mail-eopbgr660058.outbound.protection.outlook.com [40.107.66.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2AB918F6EE for ; Wed, 15 May 2019 15:06:21 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YQXPR0101MB2262.CANPRD01.PROD.OUTLOOK.COM (52.132.80.151) by YQXPR0101MB2102.CANPRD01.PROD.OUTLOOK.COM (52.132.77.155) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1878.22; Wed, 15 May 2019 15:06:19 +0000 Received: from YQXPR0101MB2262.CANPRD01.PROD.OUTLOOK.COM ([fe80::44dc:5f27:855:3fde]) by YQXPR0101MB2262.CANPRD01.PROD.OUTLOOK.COM ([fe80::44dc:5f27:855:3fde%3]) with mapi id 15.20.1900.010; Wed, 15 May 2019 15:06:19 +0000 From: Rick Macklem To: Ganbold Tsagaankhuu , "freebsd-fs@freebsd.org" Subject: Re: NFS file access log Thread-Topic: NFS file access log Thread-Index: AQHVCvMq7+IVJ0Zr30q7KplvIoFLsaZsRxZg Date: Wed, 15 May 2019 15:06:19 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: ea269d1b-9259-4b57-af67-08d6d946e2ca x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600141)(711020)(4605104)(2017052603328)(7193020); SRVR:YQXPR0101MB2102; x-ms-traffictypediagnostic: YQXPR0101MB2102: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:5236; x-forefront-prvs: 0038DE95A2 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(396003)(346002)(366004)(39860400002)(136003)(376002)(199004)(189003)(102836004)(66476007)(66946007)(76116006)(73956011)(64756008)(6506007)(486006)(305945005)(66556008)(66446008)(76176011)(476003)(478600001)(99286004)(446003)(25786009)(3480700005)(110136005)(86362001)(46003)(14454004)(53936002)(81166006)(81156014)(186003)(6246003)(2906002)(8936002)(8676002)(68736007)(316002)(786003)(74482002)(71200400001)(71190400001)(229853002)(2501003)(11346002)(33656002)(7696005)(4744005)(55016002)(6436002)(52536014)(256004)(5660300002)(14444005)(9686003)(74316002); DIR:OUT; SFP:1101; SCL:1; SRVR:YQXPR0101MB2102; H:YQXPR0101MB2262.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: uXJzuiTXONMIg/lAwmKMl9VlBrSPPPW8P8qjCdFQn7WyWHkolEEWqONnaWwHzFuft0oDyBnTrBU46iP6+TOZSSukXKA/bkhfYrNbaBnrDXKzFQBH0g04vZv796AD8nGzf9rCb9qdq2XFKN4gEd8uwPW9NCv3+PLK19ByS+qN2EVeGJffoSMHPJqvL8glRJLKE4jlnIIUOZMCq40mozg1WT9X1BHq75t4IJA4h5rS8yb1iL27oDNiRC/PpMwfePJkLmXJzCxr9oX4UYu2OkRT/Rbn9Fm46HLnNyULPcJc7QWQWvXSsDXuhIQJ3WwFX4TRap6f1cyFP4cbzj4vlZeq0L1BSBDiiszDZlA4Q/PDv+5Nrsnh+DZTzin1lsbHJqypukHZYTBbB4ntS9t6zeyHzD45eyVsV+ZpVwp0n0tl/hU= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-Network-Message-Id: ea269d1b-9259-4b57-af67-08d6d946e2ca X-MS-Exchange-CrossTenant-originalarrivaltime: 15 May 2019 15:06:19.4806 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQXPR0101MB2102 X-Rspamd-Queue-Id: 2AB918F6EE X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 40.107.66.58 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [-2.76 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.981,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; NEURAL_HAM_LONG(-0.99)[-0.985,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[uoguelph.ca]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com,mx2.hc184-76.ca.iphmx.com,mx1.hc184-76.ca.iphmx.com]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[58.66.107.40.list.dnswl.org : 127.0.3.0]; NEURAL_HAM_SHORT(-0.49)[-0.485,0]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 May 2019 15:06:24 -0000 Ganbold Tsagaankhuu wrote: >Is there anyway to log file access in NFS client/server side? >I'm experiencing following problem: > >vm_fault: pager read error, pid 75363 (php-fpm) >ncl_getpages: error 70 > >and would like to know if there is possibility to find problematic files. I haven't used DTrace, so I can't help with that. However, "70" is ESTALE and that means that the file has been deleted on th= e server, so the client can't read it. Look for a case where an application running on another client (or locally = on the server) would delete a file that would be accessed by php-fpm on this clien= t. (If you know about when it happens, you could do a raw packet capture at th= e server using "tcpdump -s 0 -w find.pcap" and then look at the packet captu= re in wireshark, looking for where file(s) get removed. It won't be easy for = a busy server.) rick From owner-freebsd-fs@freebsd.org Wed May 15 21:13:28 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CDB3215A2629 for ; Wed, 15 May 2019 21:13:28 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.org (uucp.dinoex.sub.de [IPv6:2001:1440:5001:1::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "uucp.dinoex.sub.de", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 11AE171E57 for ; Wed, 15 May 2019 21:13:27 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [194.45.71.2]) by uucp.dinoex.org (8.16.0.41/8.16.0.41) with ESMTPS id x4FLD6kP047430 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Wed, 15 May 2019 23:13:06 +0200 (CEST) (envelope-from pmc@citylink.dinoex.sub.org) X-MDaemon-Deliver-To: Received: from citylink.dinoex.sub.org (uucp@localhost) by uucp.dinoex.sub.de (8.16.0.41/8.16.0.41/Submit) with UUCP id x4FLD655047429 for freebsd-fs@freebsd.org; Wed, 15 May 2019 23:13:06 +0200 (CEST) (envelope-from pmc@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.15.2/8.15.2) with ESMTP id x4FKjQ90068751 for ; Wed, 15 May 2019 22:45:26 +0200 (CEST) (envelope-from peter@gate.oper.dinoex.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by gate.oper.dinoex.org (8.15.2/8.15.2) with ESMTP id x4FKgiKK068323 for ; Wed, 15 May 2019 22:42:44 +0200 (CEST) (envelope-from peter@gate.oper.dinoex.org) Received: (from peter@localhost) by gate.oper.dinoex.org (8.15.2/8.15.2/Submit) id x4FKgiN6068322 for freebsd-fs@freebsd.org; Wed, 15 May 2019 22:42:44 +0200 (CEST) (envelope-from peter) Date: Wed, 15 May 2019 22:42:44 +0200 From: Peter To: freebsd-fs@freebsd.org Subject: Waht is the minimum free space between GPT partitions? Message-ID: <20190515204243.GA67445@gate.oper.dinoex.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.11.4 (2019-03-13) X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 194.45.71.2; Sender-helo: uucp.dinoex.sub.de; ) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (uucp.dinoex.org [194.45.71.2]); Wed, 15 May 2019 23:13:09 +0200 (CEST) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 May 2019 21:13:29 -0000 It appears I can't find an answeer anywhere: What is the recommendend safety distance (free space) between two GPT partitions? I found, if I put partitions directly together (so that another starts immediately after one ends), under certain circumstances the volumes become inaccessible and the system (11.2) does crash. Obviousely there is a safety distance required - but how big should it be? From owner-freebsd-fs@freebsd.org Wed May 15 21:16:24 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AE27115A26C0 for ; Wed, 15 May 2019 21:16:24 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 523B771F3B for ; Wed, 15 May 2019 21:16:24 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (ip68-1-57-197.pn.at.cox.net [68.1.57.197]) by colo1.denninger.net (Postfix) with ESMTP id EB695211080 for ; Wed, 15 May 2019 17:15:46 -0400 (EDT) Received: from [192.168.10.14] (D4.Denninger.Net [192.168.10.14]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id 46B19F0A83 for ; Wed, 15 May 2019 16:15:46 -0500 (CDT) Subject: Re: Waht is the minimum free space between GPT partitions? To: freebsd-fs@freebsd.org References: <20190515204243.GA67445@gate.oper.dinoex.org> From: Karl Denninger Openpgp: preference=signencrypt Autocrypt: addr=karl@denninger.net; prefer-encrypt=mutual; keydata= mQINBFIX1zsBEADRcJfsQUl9oFeoMfLPJ1kql+3sIaYx0MfJAUhV9LnbWxr0fsWCskM1O4cV tHm5dqPkuPM4Ztc0jLotD1i9ubWvCHOlkLGxFOL+pFbjA+XZ7VKsC/xWmhMwJ3cM8HavK2OV SzEWQ/AEYtMi04IzGSwsxh/5/5R0mPHrsIomV5SbuiI0vjLuDj7fo6146AABI1ULzge4hBYW i/SHrqUrLORmUNBs6bxek79/B0Dzk5cIktD3LOfbT9EAa5J/osVkstMBhToJgQttaMIGv8SG CzpR/HwEokE+7DP+k2mLHnLj6H3kfugOF9pJH8Za4yFmw//s9cPXV8WwtZ2SKfVzn1unpKqf wmJ1PwJoom/d4fGvQDkgkGKRa6RGC6tPmXnqnx+YX4iCOdFfbP8L9rmk2sewDDVzHDU3I3ZZ 8hFIjMYM/QXXYszRatK0LCV0QPZuF7LCf4uQVKw1/oyJInsnH7+6a3c0h21x+CmSja9QJ+y0 yzgEN/nM89d6YTakfR+1xkYgodVmMy/bS8kmXbUUZG/CyeqCqc95RUySjKT2ECrf9GhhoQkl +D8n2MsrAUSMGB4GQSN+TIq9OBTpNuvATGSRuF9wnQcs1iSry+JNCpfRTyWp83uCNApe6oHU EET4Et6KDO3AvjvBMAX0TInTRGW2SQlJMuFKpc7Dg7tHK8zzqQARAQABtCNLYXJsIERlbm5p bmdlciA8a2FybEBkZW5uaW5nZXIubmV0PokCPAQTAQIAJgUCUhfXOwIbIwUJCWYBgAYLCQgH AwIEFQIIAwQWAgMBAh4BAheAAAoJEG6/sivc5s0PLxQP/i6x/QFx9G4Cw7C+LthhLXIm7NSH AtNbz2UjySEx2qkoQQjtsK6mcpEEaky4ky6t8gz0/SifIfJmSmyAx0UhUQ0WBv1vAXwtNrQQ jJd9Bj6l4c2083WaXyHPjt2u2Na6YFowyb4SaQb83hu/Zs25vkPQYJVVE0JX409MFVPUa6E3 zFbd1OTr3T4yNUy4gNeQZfzDqDS8slbIks2sXeoJrZ6qqXVI0ionoivOlaN4T6Q0UYyXtigj dQvvhMt0aNowKFjRqrmSDRpdz+o6yg7Mp7qEZ1V6EZk8KqQTH6htpCTQ8i79ttK4LG6bstSF Re6Fwq52nbrcANrcdmtZXqjo+SGbUqJ8b1ggrxAsJ5MEhRh2peKrCgI/TjQo+ZxfnqEoR4AI 46Cyiz+/lcVvlvmf2iPifS3EEdaH3Itfwt7MxFm6mQORYs6skHDw3tOYB2/AdCW6eRVYs2hB RMAG4uwApZfZDKgRoE95PJmQjeTBiGmRPcsQZtNESe7I7EjHtCDLwtJqvD4HkDDQwpzreT6W XkyIJ7ns7zDfA1E+AQhFR6rsTFGgQZRZKsVeov3SbhYKkCnVDCvb/PKQCAGkSZM9SvYG5Yax 8CMry3AefKktf9fqBFg8pWqtVxDwJr56dhi0GHXRu3jVI995rMGo1fLUG5fSxiZ8L5sAtokh 9WFmQpyl Message-ID: Date: Wed, 15 May 2019 16:15:46 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <20190515204243.GA67445@gate.oper.dinoex.org> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms050007060501000405020203" X-Rspamd-Queue-Id: 523B771F3B X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.983,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[] X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 May 2019 21:16:24 -0000 This is a cryptographically signed message in MIME format. --------------ms050007060501000405020203 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 T24gNS8xNS8yMDE5IDE1OjQyLCBQZXRlciB3cm90ZToNCj4gSXQgYXBwZWFycyBJIGNhbid0 IGZpbmQgYW4gYW5zd2VlciBhbnl3aGVyZToNCj4NCj4gV2hhdCBpcyB0aGUgcmVjb21tZW5k ZW5kIHNhZmV0eSBkaXN0YW5jZSAoZnJlZSBzcGFjZSkgYmV0d2VlbiB0d28NCj4gR1BUIHBh cnRpdGlvbnM/DQo+DQo+IEkgZm91bmQsIGlmIEkgcHV0IHBhcnRpdGlvbnMgZGlyZWN0bHkg dG9nZXRoZXIgKHNvIHRoYXQgYW5vdGhlciBzdGFydHMNCj4gaW1tZWRpYXRlbHkgYWZ0ZXIg b25lIGVuZHMpLCB1bmRlciBjZXJ0YWluIGNpcmN1bXN0YW5jZXMgdGhlIHZvbHVtZXMNCj4g YmVjb21lIGluYWNjZXNzaWJsZSBhbmQgdGhlIHN5c3RlbSAoMTEuMikgZG9lcyBjcmFzaC4g T2J2aW91c2VseSB0aGVyZQ0KPiBpcyBhIHNhZmV0eSBkaXN0YW5jZSByZXF1aXJlZCAtIGJ1 dCBob3cgYmlnIHNob3VsZCBpdCBiZT8NCg0KSHVoP8KgIFRoZXJlIGlzIG5vIHN1Y2ggdGhp bmcgcmVxdWlyZWQuDQoNCiQgZ3BhcnQgc2hvdw0KPT7CoMKgwqDCoMKgIDQwwqAgNTI0Mjg3 MjDCoCB2dGJkMMKgIEdQVMKgICgyNUcpDQrCoMKgwqDCoMKgwqDCoCA0MMKgwqDCoMKgwqDC oCA1MTLCoMKgwqDCoMKgIDHCoCBmcmVlYnNkLWJvb3TCoCAoMjU2SykNCsKgwqDCoMKgwqDC oCA1NTLCoMKgIDQxOTQzMDTCoMKgwqDCoMKgIDLCoCBmcmVlYnNkLXN3YXDCoCAoMi4wRykN CsKgwqAgNDE5NDg1NsKgIDQ4MjMzOTA0wqDCoMKgwqDCoCAzwqAgZnJlZWJzZC11ZnPCoCAo MjNHKQ0KDQpUaGF0IHNwZWNpZmljIG1hY2hpbmUgaGFzIGJlZW4gdXAgZm9yIDQ0OSBkYXlz IGF0IHRoaXMgcG9pbnQuLi4gaXQncw0KcnVubmluZyAxMS4xLCBidXQgc3RpbGwuLi4uwqAg Oy0pDQoNCi0tIA0KS2FybCBEZW5uaW5nZXINCmthcmxAZGVubmluZ2VyLm5ldCA8bWFpbHRv OmthcmxAZGVubmluZ2VyLm5ldD4NCi9UaGUgTWFya2V0IFRpY2tlci8NCi9bUy9NSU1FIGVu Y3J5cHRlZCBlbWFpbCBwcmVmZXJyZWRdLw0K --------------ms050007060501000405020203 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTkwNTE1MjExNTQ2 WjBPBgkqhkiG9w0BCQQxQgRA3KEYi9K7W0dwwWwR6fc02Ty4FZlp2N+2qMA9W2jLFMiGgvMp VWSYdcTHwuGzHgdBA6wHhR+djLWEt9cmLfpxwjBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgCkNbVtT9ke5CdcC7HLJ/6SNnSPu8zrT02hLe3AOHF1OGMdJwSJ80rIhp1n6+BUQ8NF LlCn3iUonV7/V4VAE8pn3tAGM+hAdth3ph3ltmdjtmiJ07BNSCYGeV08czDVoGne6G2vzHLe Gv10GDR1baLDIHoCFamDT6zT7jcvMvT478Lh3TUPL4dQ8mncfBryOFIrtEvWV0ClJYbkI8F1 wbqKsH3loPJob09PDnQyL2jzsu2e8y35lj630AluUfiPW9oOJVXDc6ktTS0mq21LjX0Qoni9 KPH2x/fRTZ3GW0DtuZh/J+eWLsfezeL2CcX07BNrI3h+JA5myRLET3MKiOZ29XpPXMen4HaJ JcbkIxi7Ei5TH0DTkJzGwJvpIZICFIm/OzZu6jJw7G3UmQA985BNg/6hbVfyECDW7sziVzN6 EXyb8zsEBx/5Inbz+qj2FGMSr08SWv9Ht+eJ/kXBId/jcC5LJW4iLWyfjOrJISftbr2o9WCV 5mlskc8cInSXA31CxPSO2Tor1EBynb7vLxIvWsKNuQttkIxEaIBlooBCPopTLT8qQL9Ku8zR +lw9N7BQoZC0GvxXoe0n/ZHixnnYvKtb4hxQHmQ7R1PLV57LNVz/Gqf0H/5ic15gnYaWHGF9 YiT4aJyaOTTUsz6Xf3ybLvMA1WRGn+0R21HPLOjCxgAAAAAAAA== --------------ms050007060501000405020203-- From owner-freebsd-fs@freebsd.org Wed May 15 21:26:11 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 41F3815A2A22 for ; Wed, 15 May 2019 21:26:11 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: from mail-it1-x131.google.com (mail-it1-x131.google.com [IPv6:2607:f8b0:4864:20::131]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D1E1B72529 for ; Wed, 15 May 2019 21:26:10 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: by mail-it1-x131.google.com with SMTP id p18so6509874itm.1 for ; Wed, 15 May 2019 14:26:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=B7Zqv6hVZXzxGyu42h03GFV4UsdZ6w2ReP5nQTwo/E8=; b=YD9x/71Thogh7jf1C607YMxO3giPu/Quq1urM0z+G4zUKf5kJAxlFWk0x0R1xlhkV+ 40aGrqHUbaYPZs7QX90pT7wp58HmMr07eZdDqrCzNs35n+izHav36sk/L1XIgbFbgxM5 3XmiQq3ChwaiW49EMXfxmIWeDmrR9jKEvfVOCcD/T9lP/ut9sO7caRz5eFhwDT5XzYie PUwQfr4eI+DIXS0dwJyFib1bbF099OK/OqSQN+VGawFlNMxgBqwROREgDfnGE0PThInF EpWf4yACuTwtOeTqQwbVctbyxIb4ItRTGshC6NkJy09kBSuJGfSqHkyZcq9sIa0dxSog /KOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=B7Zqv6hVZXzxGyu42h03GFV4UsdZ6w2ReP5nQTwo/E8=; b=GeZnXxlIS0NyYxp92TJP1ogH3GjRiCfoeUzgDVYfJOmLKLY9tw2uHwLCmTmukDAMJP DUKitnGvSYwcltxLuavfe+xDBBhRvAhW68Wq4+NNYj8MW2QMQqvEj5uN7XyE4LRcBaSr HeVvRpNQ/zuF7ybTNENL/thgk1TFLwLD85cJEgYtIUgyifn5Jp2/RThlj/cmi86YvVjg MnqzdjJz3+tqwR0W6ac1yz2HtD5IMwFPxaYBcOL3KBpJ3KGK3NObf6kM5pjlmW4Abe4s fdhZeS4b4uAvCtme+afm5s3mNSj7CMHLnlxlrw9k+A7vc7O49ldkFrBnteUJK4vrPeUb uJlg== X-Gm-Message-State: APjAAAV9MBlxAYE1bpBj/FFWURbiXWRmjU0SN4UnlNMEZkby2bDCKmMv gcK4OfoaqixzI5OWiUNMYMcHMefW6A7yPC/hqmWJJg== X-Google-Smtp-Source: APXvYqyYaMMxsWzs89rMQFQLYLA8nrR5c2dDKWbKacPVYxqM+AMGyuKnnvtBlLxC14UmkgDJG9UFhk21ZM5WMTndOoc= X-Received: by 2002:a24:b342:: with SMTP id z2mr8889467iti.121.1557955569935; Wed, 15 May 2019 14:26:09 -0700 (PDT) MIME-Version: 1.0 References: <20190515204243.GA67445@gate.oper.dinoex.org> In-Reply-To: <20190515204243.GA67445@gate.oper.dinoex.org> From: Adam Date: Wed, 15 May 2019 16:25:58 -0500 Message-ID: Subject: Re: Waht is the minimum free space between GPT partitions? To: Peter Cc: freebsd-fs X-Rspamd-Queue-Id: D1E1B72529 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.97 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.97)[-0.969,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 May 2019 21:26:11 -0000 On Wed, May 15, 2019 at 4:14 PM Peter wrote: > It appears I can't find an answeer anywhere: > > What is the recommendend safety distance (free space) between two > GPT partitions? > > I found, if I put partitions directly together (so that another starts > immediately after one ends), under certain circumstances the volumes > become inaccessible and the system (11.2) does crash. Obviousely there > is a safety distance required - but how big should it be? > Sounds more like an off-by-1 error. However partitions start should be 4k aligned for widest performance compatibility. -- Adam From owner-freebsd-fs@freebsd.org Wed May 15 22:29:22 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9D96015A3F7C for ; Wed, 15 May 2019 22:29:22 +0000 (UTC) (envelope-from SRS0=G7iZ=TP=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3349574350 for ; Wed, 15 May 2019 22:29:22 +0000 (UTC) (envelope-from SRS0=G7iZ=TP=quip.cz=000.fbsd@elsa.codelab.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id B169528417; Thu, 16 May 2019 00:29:14 +0200 (CEST) Received: from illbsd.quip.test (ip-62-24-92-232.net.upcbroadband.cz [62.24.92.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 4B7BE28411; Thu, 16 May 2019 00:29:13 +0200 (CEST) Subject: Re: Waht is the minimum free space between GPT partitions? To: Peter , freebsd-fs@freebsd.org References: <20190515204243.GA67445@gate.oper.dinoex.org> From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <60d57363-eb5c-e985-82ad-30f03b06a4c6@quip.cz> Date: Thu, 16 May 2019 00:29:16 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Firefox/52.0 SeaMonkey/2.49.3 MIME-Version: 1.0 In-Reply-To: <20190515204243.GA67445@gate.oper.dinoex.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 3349574350 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.98)[-0.982,0] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 May 2019 22:29:22 -0000 Peter wrote on 2019/05/15 22:42: > It appears I can't find an answeer anywhere: > > What is the recommendend safety distance (free space) between two > GPT partitions? > > I found, if I put partitions directly together (so that another starts > immediately after one ends), under certain circumstances the volumes > become inaccessible and the system (11.2) does crash. Obviousely there > is a safety distance required - but how big should it be? I read your post on forum https://forums.freebsd.org/threads/create-degraded-raid-5-with-2-disks-on-freebsd.70750/#post-426756 There is no requirement to have empty space between partitions. These 2 disks 6T each are partitioned with GPT and each partition is mirrored by gmirror: ~/> gpart show => 34 11721045101 ada0 GPT (5.5T) 34 6 - free - (3.0K) 40 1024 1 freebsd-boot (512K) 1064 984 - free - (492K) 2048 2097152 2 freebsd-ufs (1.0G) 2099200 16777216 3 freebsd-swap (8.0G) 18876416 16777216 4 freebsd-ufs (8.0G) 35653632 16777216 5 freebsd-ufs (8.0G) 52430848 6291456 6 freebsd-ufs (3.0G) 58722304 11628767232 7 freebsd-ufs (5.4T) 11687489536 33555599 - free - (16G) => 34 11721045101 ada1 GPT (5.5T) 34 6 - free - (3.0K) 40 1024 1 freebsd-boot (512K) 1064 984 - free - (492K) 2048 2097152 2 freebsd-ufs (1.0G) 2099200 16777216 3 freebsd-swap (8.0G) 18876416 16777216 4 freebsd-ufs (8.0G) 35653632 16777216 5 freebsd-ufs (8.0G) 52430848 6291456 6 freebsd-ufs (3.0G) 58722304 11628767232 7 freebsd-ufs (5.4T) 11687489536 33555599 - free - (16G) No problems for years. I think your case is somewhat different if you split disk in to 3 partitions later used as 3 devices for one ZFS pool, so maybe there is some coincidence with expanding ZFS... and then it is a bug which should be fixed. Can you prepare some simple testcase (scriptable) which make a panic on your host? I will try it in some spare VM. Miroslav Lachman From owner-freebsd-fs@freebsd.org Thu May 16 01:13:40 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62E5315A8397 for ; Thu, 16 May 2019 01:13:40 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.org (uucp.dinoex.sub.de [IPv6:2001:1440:5001:1::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "uucp.dinoex.sub.de", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 79C2781A0D for ; Thu, 16 May 2019 01:13:38 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [194.45.71.2]) by uucp.dinoex.org (8.16.0.41/8.16.0.41) with ESMTPS id x4G1D5vs066355 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 16 May 2019 03:13:06 +0200 (CEST) (envelope-from pmc@citylink.dinoex.sub.org) Received: from citylink.dinoex.sub.org (uucp@localhost) by uucp.dinoex.sub.de (8.16.0.41/8.16.0.41/Submit) with UUCP id x4G1D5ZW066354; Thu, 16 May 2019 03:13:05 +0200 (CEST) (envelope-from pmc@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.15.2/8.15.2) with ESMTP id x4G0aQBC002082; Thu, 16 May 2019 02:36:26 +0200 (CEST) (envelope-from peter@gate.oper.dinoex.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by gate.oper.dinoex.org (8.15.2/8.15.2) with ESMTP id x4G0a7X7002001; Thu, 16 May 2019 02:36:07 +0200 (CEST) (envelope-from peter@gate.oper.dinoex.org) Received: (from peter@localhost) by gate.oper.dinoex.org (8.15.2/8.15.2/Submit) id x4G0a70c001998; Thu, 16 May 2019 02:36:07 +0200 (CEST) (envelope-from peter) Date: Thu, 16 May 2019 02:36:07 +0200 From: Peter To: Miroslav Lachman <000.fbsd@quip.cz> Cc: freebsd-fs@freebsd.org Subject: Re: Waht is the minimum free space between GPT partitions? Message-ID: <20190516003607.GA93284@gate.oper.dinoex.org> References: <20190515204243.GA67445@gate.oper.dinoex.org> <60d57363-eb5c-e985-82ad-30f03b06a4c6@quip.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <60d57363-eb5c-e985-82ad-30f03b06a4c6@quip.cz> User-Agent: Mutt/1.11.4 (2019-03-13) X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 194.45.71.2; Sender-helo: uucp.dinoex.sub.de; ) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (uucp.dinoex.org [194.45.71.2]); Thu, 16 May 2019 03:13:09 +0200 (CEST) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 May 2019 01:13:40 -0000 On Thu, May 16, 2019 at 12:29:16AM +0200, Miroslav Lachman wrote: ! > I found, if I put partitions directly together (so that another starts ! > immediately after one ends), under certain circumstances the volumes ! > become inaccessible and the system (11.2) does crash. Obviousely there ! > is a safety distance required - but how big should it be? ! ! I read your post on forum ! https://forums.freebsd.org/threads/create-degraded-raid-5-with-2-disks-on-freebsd.70750/#post-426756 Hi, great, that should explain how to make it happen. ! No problems for years. Me neither with MBR/packlabels, but only recently switched to GPT. I suppose either GPT or ZFS-autoexpand seems to go out-of-bounds; I couldn't determine which. ! I think your case is somewhat different if you split disk in to 3 ! partitions later used as 3 devices for one ZFS pool, so maybe there is ! some coincidence with expanding ZFS... and then it is a bug which should ! be fixed. If we could fix it that would be even better! Agreed, it's an ugly operation, but I love to do ugly things with ZFS, and usually it stands it. ;) ! Can you prepare some simple testcase (scriptable) which make a panic on ! your host? I will try it in some spare VM. The description in mentioned forum-post is pretty much what I did. At first I did it on my router, as there is empty space on a disk, and when that had gone by-by, I tried it on the desktop with an (otherwise empty) USB stick. Takes an eternity to create ZFS-raidz even on USB-3 stick - they are not designed for that - but the outcome was the same. Procedure is: 1. create new GPT scheme on stick. 2. add 3x 1G freebsd-zfs partitions with 1G -free- in between. 3. zpool create test raidz da0p1 da0p2 da0p3 4. resize 3x partitions to 2G each 5. zpool set autoexpand=on test 6. export the pool 7. zpool online At that point it will start to complain that (some of) the pool isn't readable. Now resize the partitions back to 1G. -> kernel crash And after going thru that and having all partitions back at 1G, the pool works again. :) I'll try to reproduce it from a script, as soon as my toolchain is done with building from the recent patches. Cheerio, PMc From owner-freebsd-fs@freebsd.org Thu May 16 08:05:22 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4A7ED15AFFB9 for ; Thu, 16 May 2019 08:05:22 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from forward106j.mail.yandex.net (forward106j.mail.yandex.net [IPv6:2a02:6b8:0:801:2::109]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8B66E8E4BD for ; Thu, 16 May 2019 08:05:21 +0000 (UTC) (envelope-from bu7cher@yandex.ru) Received: from mxback21o.mail.yandex.net (mxback21o.mail.yandex.net [IPv6:2a02:6b8:0:1a2d::72]) by forward106j.mail.yandex.net (Yandex) with ESMTP id C0F4C11A146E; Thu, 16 May 2019 11:05:17 +0300 (MSK) Received: from smtp2o.mail.yandex.net (smtp2o.mail.yandex.net [2a02:6b8:0:1a2d::26]) by mxback21o.mail.yandex.net (nwsmtp/Yandex) with ESMTP id E7jIc4Fl1U-5H7G3rZc; Thu, 16 May 2019 11:05:17 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1557993917; bh=c0HcPhGw8z36kIY7ZsnvocBA4rLX/Pk980qjJBDxv2E=; h=In-Reply-To:From:To:Subject:Cc:Date:References:Message-ID; b=mWAq9vwE/cM87Y7tcZt0n7Fumd3zqNZ2vsWZHdnyJ5Vnn8pFdJ775U2ANZy1n9IWS EGPS3cGuZaTZQiWpStNGGsczMrU9oqXNWCURnfqSp+JDt8BHF2Puy+CVhjfThT+wS5 KcYXU/5QSg5ONgX6+RMiAvAnC48tOHYgoCWiskFk= Received: by smtp2o.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id CqvR6SPyJu-5GLWtowH; Thu, 16 May 2019 11:05:16 +0300 (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (Client certificate not present) Subject: Re: Waht is the minimum free space between GPT partitions? To: Peter , Miroslav Lachman <000.fbsd@quip.cz> Cc: freebsd-fs@freebsd.org References: <20190515204243.GA67445@gate.oper.dinoex.org> <60d57363-eb5c-e985-82ad-30f03b06a4c6@quip.cz> <20190516003607.GA93284@gate.oper.dinoex.org> From: "Andrey V. Elsukov" Openpgp: id=E6591E1B41DA1516F0C9BC0001C5EA0410C8A17A Autocrypt: addr=bu7cher@yandex.ru; prefer-encrypt=mutual; keydata= mQENBEwBF1kBCADB9sXFhBEUy8qQ4X63Y8eBatYMHGEFWN9ypS5lI3RE6qQW2EYbxNk7qUC5 21YIIS1mMFVBEfvR7J9uc7yaYgFCEb6Sce1RSO4ULN2mRKGHP3/Sl0ijZEjWHV91hY1YTHEF ZW/0GYinDf56sYpDDehaBF5wkWIo1+QK5nmj3vl0DIDCMNd7QEiWpyLVwECgLX2eOAXByT8B bCqVhJGcG6iFP7/B9Ll6uX5gb8thM9LM+ibwErDBVDGiOgvfxqidab7fdkh893IBCXa82H9N CNwnEtcgzh+BSKK5BgvPohFMgRwjti37TSxwLu63QejRGbZWSz3OK3jMOoF63tCgn7FvABEB AAG0JUFuZHJleSBWLiBFbHN1a292IDxidTdjaGVyQHlhbmRleC5ydT6JATgEEwECACIFAkwB F1kCGwMGCwkIBwMCBhUIAgkKCwQWAgMBAh4BAheAAAoJEAHF6gQQyKF6qmYIAI6ekfm1VA4T vqankI1ISE6ku4jV7UlpIQlEbE7/8n3Zd6teJ+pGOQhN5qk8QE7utdPdbktAzi+x7LIJVzUw 4TywZLXGrkP7VKYkfg6oyCGyzITghefQeJtr2TN4hYCkzPWpylkue8MtmqfZv/6royqwTbN+ +E09FQNvTgRUYJYTeQ1qOsxNRycwvw3dr2rOfuxShbzaHBB1pBIjGrMg8fC5pd65ACH5zuFV A0CoTNGMDrEZSfBkTW604UUHFFXeCoC3dwDZRKOWJ3GmMXns65Ai5YkA63BSHEE1Qle3VBhd cG1w0CB5FBV3pB27UVnf0jEbysrDqW4qN7XMRFSWNAy5AQ0ETAEXWQEIAJ2p6l9LBoqdH/0J PEFDY2t2gTvAuzz+8zs3R03dFuHcNbOwjvWCG0aOmVpAzkRa8egn5JB4sZaFUtKPYJEQ1Iu+ LUBwgvtXf4vWpzC67zs2dDuiW4LamH5p6xkTD61aHR7mCB3bg2TUjrDWn2Jt44cvoYxj3dz4 S49U1rc9ZPgD5axCNv45j72tggWlZvpefThP7xT1OlNTUqye2gAwQravXpZkl5JG4eOqJVIU X316iE3qso0iXRUtO7OseBf0PiVmk+wCahdreHOeOxK5jMhYkPKVn7z1sZiB7W2H2TojbmcK HZC22sz7Z/H36Lhg1+/RCnGzdEcjGc8oFHXHCxUAEQEAAYkBHwQYAQIACQUCTAEXWQIbDAAK CRABxeoEEMihegkYCAC3ivGYNe2taNm/4Nx5GPdzuaAJGKWksV+w9mo7dQvU+NmI2az5w8vw 98OmX7G0OV9snxMW+6cyNqBrVFTu33VVNzz9pnqNCHxGvj5dL5ltP160JV2zw2bUwJBYsgYQ WfyJJIM7l3gv5ZS3DGqaGIm9gOK1ANxfrR5PgPzvI9VxDhlr2juEVMZYAqPLEJe+SSxbwLoz BcFCNdDAyXcaAzXsx/E02YWm1hIWNRxanAe7Vlg7OL+gvLpdtrYCMg28PNqKNyrQ87LQ49O9 50IIZDOtNFeR0FGucjcLPdS9PiEqCoH7/waJxWp6ydJ+g4OYRBYNM0EmMgy1N85JJrV1mi5i Message-ID: <1f98b517-e5f9-42a6-d900-246d83972b55@yandex.ru> Date: Thu, 16 May 2019 11:03:16 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <20190516003607.GA93284@gate.oper.dinoex.org> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="9KTc7VChSqok7dg6pfdMDtudqnBpBLyYe" X-Rspamd-Queue-Id: 8B66E8E4BD X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-7.00 / 15.00]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-1.00)[-0.997,0]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 May 2019 08:05:22 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --9KTc7VChSqok7dg6pfdMDtudqnBpBLyYe Content-Type: multipart/mixed; boundary="E2vuBo1t11Ma6gyusTMj4pUypHS0GcXh7"; protected-headers="v1" From: "Andrey V. Elsukov" To: Peter , Miroslav Lachman <000.fbsd@quip.cz> Cc: freebsd-fs@freebsd.org Message-ID: <1f98b517-e5f9-42a6-d900-246d83972b55@yandex.ru> Subject: Re: Waht is the minimum free space between GPT partitions? References: <20190515204243.GA67445@gate.oper.dinoex.org> <60d57363-eb5c-e985-82ad-30f03b06a4c6@quip.cz> <20190516003607.GA93284@gate.oper.dinoex.org> In-Reply-To: <20190516003607.GA93284@gate.oper.dinoex.org> --E2vuBo1t11Ma6gyusTMj4pUypHS0GcXh7 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 16.05.2019 03:36, Peter wrote: > On Thu, May 16, 2019 at 12:29:16AM +0200, Miroslav Lachman wrote: > ! > I found, if I put partitions directly together (so that another sta= rts > ! > immediately after one ends), under certain circumstances the volume= s > ! > become inaccessible and the system (11.2) does crash. Obviousely th= ere > ! > is a safety distance required - but how big should it be? > !=20 > ! I read your post on forum=20 > ! https://forums.freebsd.org/threads/create-degraded-raid-5-with-2-disk= s-on-freebsd.70750/#post-426756 >=20 > Hi, great, that should explain how to make it happen. >=20 > ! No problems for years. >=20 > Me neither with MBR/packlabels, but only recently switched to GPT. >=20 > I suppose either GPT or ZFS-autoexpand seems to go out-of-bounds; I > couldn't determine which. >=20 > ! I think your case is somewhat different if you split disk in to 3=20 > ! partitions later used as 3 devices for one ZFS pool, so maybe there i= s=20 > ! some coincidence with expanding ZFS... and then it is a bug which sho= uld=20 > ! be fixed. >=20 > If we could fix it that would be even better! Agreed, it's an ugly > operation, but I love to do ugly things with ZFS, and usually it > stands it. ;) >=20 > ! Can you prepare some simple testcase (scriptable) which make a panic = on=20 > ! your host? I will try it in some spare VM. >=20 > The description in mentioned forum-post is pretty much what I did. >=20 > At first I did it on my router, as there is empty space on a disk, > and when that had gone by-by, I tried it on the desktop with an > (otherwise empty) USB stick. Takes an eternity to create ZFS-raidz > even on USB-3 stick - they are not designed for that - but the outcome > was the same. >=20 > Procedure is: > 1. create new GPT scheme on stick. > 2. add 3x 1G freebsd-zfs partitions with 1G -free- in between. > 3. zpool create test raidz da0p1 da0p2 da0p3 > 4. resize 3x partitions to 2G each > 5. zpool set autoexpand=3Don test > 6. export the pool > 7. zpool online When you exported the pool, ZFS is able to find its labels on the entire da0 disk. This is probably leads to panic, and not because there are no free space between partitions. When panic happens, just make a photo of panic screen, in most cases it can say where is the problem. Even better, if you add debug options to the kernel and then you will be able get core dump. --=20 WBR, Andrey V. Elsukov --E2vuBo1t11Ma6gyusTMj4pUypHS0GcXh7-- --9KTc7VChSqok7dg6pfdMDtudqnBpBLyYe Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQEzBAEBCAAdFiEE5lkeG0HaFRbwybwAAcXqBBDIoXoFAlzdGUQACgkQAcXqBBDI oXoboAf/US7SnrGfFwIBGDriHHNL+nn+kXFqQ6Z7IPycpKqYp0iF4sfUuWNa3BeB oFq1iSVNHAF1GSYGLFB2tjFjqrFurQSH9dDaYK6eoGjc8vCFsEt7Q8bX8bQTF/vJ MDo4f26Zy+eNZalh4SuuHHjli5pGRonr9VzI8Eaia6b/xp6u7CV/+vEOFo7Asr4/ Yult7uaA4h+mBjrsD3SCo2x6ssGkfuT7HTxFWdshgcdn4svwkRv2ounfzpamluIn lRvBfz3YCEF9Xb+j5GrtolkZPp8HhUZZfz0i6f1GWirfBoLroazy0HsHVg89J9Cm i4FQkMyKUvM9cNXG6TfDUsFHlIByHQ== =EJrx -----END PGP SIGNATURE----- --9KTc7VChSqok7dg6pfdMDtudqnBpBLyYe-- From owner-freebsd-fs@freebsd.org Thu May 16 08:24:19 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 373C9158874D for ; Thu, 16 May 2019 08:24:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id BF4088F1FD for ; Thu, 16 May 2019 08:24:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 80BC2158874C; Thu, 16 May 2019 08:24:18 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6E90E158874B for ; Thu, 16 May 2019 08:24:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0C2BD8F1FC for ; Thu, 16 May 2019 08:24:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 43A63C853 for ; Thu, 16 May 2019 08:24:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x4G8OHWD044432 for ; Thu, 16 May 2019 08:24:17 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x4G8OHai044431 for fs@FreeBSD.org; Thu, 16 May 2019 08:24:17 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 237807] ZFS: ZVOL writes fast, ZVOL reads abysmal... Date: Thu, 16 May 2019 08:24:16 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: nbe@renzel.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 May 2019 08:24:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D237807 --- Comment #8 from Nils Beyer --- Okay, played around a little bit but regardless what I'm trying I only get around 250MB/s read speed from the ZVOL via iSCSI. It's okayish but a little disappointing to have such many platters and only= get so low read speeds... --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu May 16 08:27:51 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 928961588A45 for ; Thu, 16 May 2019 08:27:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 267078F623 for ; Thu, 16 May 2019 08:27:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id DEFEC1588A3D; Thu, 16 May 2019 08:27:50 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CCBC61588A3B for ; Thu, 16 May 2019 08:27:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 66F028F61D for ; Thu, 16 May 2019 08:27:50 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id B15BFC856 for ; Thu, 16 May 2019 08:27:49 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x4G8Rnp5048429 for ; Thu, 16 May 2019 08:27:49 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x4G8RnfZ048428 for fs@FreeBSD.org; Thu, 16 May 2019 08:27:49 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 237807] ZFS: ZVOL writes fast, ZVOL reads abysmal... Date: Thu, 16 May 2019 08:27:49 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 12.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: nbe@renzel.net X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 May 2019 08:27:51 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D237807 --- Comment #9 from Nils Beyer --- (In reply to Nils Beyer from comment #8) and it's no network problem as I get around 700MB/s write speeds to the same ZVOL via iSCSI... --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Fri May 17 01:13:43 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0AFB015AC187 for ; Fri, 17 May 2019 01:13:43 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.org (uucp.dinoex.sub.de [IPv6:2001:1440:5001:1::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "uucp.dinoex.sub.de", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C41CE84C81 for ; Fri, 17 May 2019 01:13:41 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [194.45.71.2]) by uucp.dinoex.org (8.16.0.41/8.16.0.41) with ESMTPS id x4H1D69D090697 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 17 May 2019 03:13:06 +0200 (CEST) (envelope-from pmc@citylink.dinoex.sub.org) Received: from citylink.dinoex.sub.org (uucp@localhost) by uucp.dinoex.sub.de (8.16.0.41/8.16.0.41/Submit) with UUCP id x4H1D6RT090696; Fri, 17 May 2019 03:13:06 +0200 (CEST) (envelope-from pmc@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.15.2/8.15.2) with ESMTP id x4H13HOK037361; Fri, 17 May 2019 03:03:17 +0200 (CEST) (envelope-from peter@gate.oper.dinoex.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by gate.oper.dinoex.org (8.15.2/8.15.2) with ESMTP id x4H12eJc037298; Fri, 17 May 2019 03:02:40 +0200 (CEST) (envelope-from peter@gate.oper.dinoex.org) Received: (from peter@localhost) by gate.oper.dinoex.org (8.15.2/8.15.2/Submit) id x4H12dJH037297; Fri, 17 May 2019 03:02:39 +0200 (CEST) (envelope-from peter) Date: Fri, 17 May 2019 03:02:39 +0200 From: Peter To: Miroslav Lachman <000.fbsd@quip.cz> Cc: freebsd-fs@freebsd.org Subject: Re: Waht is the minimum free space ... (Full Report) Message-ID: <20190517010239.GA34758@gate.oper.dinoex.org> References: <20190515204243.GA67445@gate.oper.dinoex.org> <60d57363-eb5c-e985-82ad-30f03b06a4c6@quip.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: <60d57363-eb5c-e985-82ad-30f03b06a4c6@quip.cz> User-Agent: Mutt/1.11.4 (2019-03-13) X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 194.45.71.2; Sender-helo: uucp.dinoex.sub.de; ) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (uucp.dinoex.org [194.45.71.2]); Fri, 17 May 2019 03:13:09 +0200 (CEST) X-Spam: Yes X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 May 2019 01:13:43 -0000 Alright, now I was able to reproduce the failure under test conditions. And in addition to this, I was able to produce two deadlocks, a bunch of GPT errors and a ZFS assertion failure. Here we go: ABSTRACT =3D=3D=3D=3D=3D=3D=3D=3D The original idea was to check if ZFS can grow a raid5. (This is technically easy, but not all volume managers are willing to do it, so I decided to give it a try.) Therefore I created three partitions on some spare diskspace, with free-space interleave between, then created a ZFS raidz pool on them, enlarged the partitions, and tried to autoexpand the ZFS pool. While the procedure appeared to work in theory, I got strange results in practice, leading even to kernel crashes. Now I conducted five test-cases with protocols: 1. try with an USB-stick on a core-i consumer machine. The chosen procedure did now lead into deadlock long before even expanding the pool. 2. try with a spinning drive on the core-i. The chosen procedure was fully successful. 3. same as 2., but without exporting/importing the pool before expansion. This procedure led into deadlock. 4. try with spinning drive on ancient pentium server machine, with the exact same procedure as recently. The procedure did not lead to kernel crash this time; instead it produced GPT errors and ZFS assertion failure, which may provide further insight. 5. try the same procedure as in 4. now on the core-i machine. The procedure did lead to immediate reboot once, and to "trap 12: page fault" the other times. In both cases 1. and 3. it was not possible to execute "sync" anymore, so death of the system would be just a matter of time. See the protocols of the testcases below and some commentary at the end. Summary of the procedures: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D Testcase 1: * create partitions * create pool * export pool -> hangs Testcase 2: * create partitions * create pool * export pool * import pool * enlarge partitions * export pool * import pool * set autoexpand=3Don * online -e -> success! Testcase 3: * create partitions * create pool * enlarge partitions * set autoexpand=3Don * online -e -> hangs Testcase 4: * create partitions * create pool * enlarge partitions * set autoexpand=3Don * export pool * -> cannot import anymore * shrink back partitions * -> ZFS assertion failed * destroy pool and partitions * -> devices still present! Testcase 5: * create partitions * create pool * enlarge partitions * set autoexpand=3Don * export pool * -> cannot import anymore * shrink back partition -> crash! * shrink back partition -> crash! * shrink back partition -> crash! * import successful. --------------- TESTCASE 1 BEGIN ------------------------ Script started on Thu May 16 20:47:21 2019 root@disp:~ # camcontrol devlist at scbus1 target 0 lun 0 (pass0,ada0) at scbus2 target 0 lun 0 (pass1,ada1) at scbus3 target 0 lun 0 (pass2,cd0) at scbus4 target 0 lun 0 (pass3,ses0) at scbus5 target 0 lun 0 (da0,pass4) at scbus6 target 0 lun 0 (da1,pass5) root@disp:~ # gpart create -s GPT da1 da1 created root@disp:~ # gpart add -t freebsd-zfs -s 1G da1 da1p1 added root@disp:~ # gpart add -t freebsd-zfs -s 1G -b 4194344 da1 da1p2 added root@disp:~ # gpart add -t freebsd-zfs -s 1G -b 8388648 da1 da1p3 added root@disp:~ # gpart show da1 =3D> 40 60555184 da1 GPT (29G) 40 2097152 1 freebsd-zfs (1.0G) 2097192 2097152 - free - (1.0G) 4194344 2097152 2 freebsd-zfs (1.0G) 6291496 2097152 - free - (1.0G) 8388648 2097152 3 freebsd-zfs (1.0G) 10485800 50069424 - free - (24G) root@disp:~ # zpool create -f testraid raidz da1p1 da1p2 da1p3 root@disp:~ # zpool list testraid NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEAL= TH ALTROOT testraid 2.75G 552K 2.75G - - 0% 0% 1.00x ONLI= NE - root@disp:~ # zpool status testraid pool: testraid state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM testraid ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 da1p1 ONLINE 0 0 0 da1p2 ONLINE 0 0 0 da1p3 ONLINE 0 0 0 errors: No known data errors root@disp:~ # zpool export testraid ## At this point the command hung forever and we were dead-in-the-water, ## no disk-I/O happening. Output ps shows "D"eadlock in "connec": 0 5640 4647 0 20 0 7760 3784 connec D+ 10 0:00.01 zpool No "df", no "sync", no "reboot". --------------- TESTCASE 1 END ------------------------ --------------- TESTCASE 2 BEGIN ------------------------ Script started on Thu May 16 21:26:14 2019 root@disp:~ # camcontrol devlist at scbus1 target 0 lun 0 (pass0,ada0)=0D at scbus2 target 0 lun 0 (pass1,ada1)=0D at scbus3 target 0 lun 0 (pass2,ada2)=0D at scbus4 target 0 lun 0 (pass3,cd0)=0D at scbus5 target 0 lun 0 (pass4,ses0)=0D at scbus6 target 0 lun 0 (da0,pass5)=0D root@disp:~ # gpart show ada2 gpart: No such geom: ada2. root@disp:~ # gpart create -s GPT ada2 ada2 created root@disp:~ # gpart add -t freebsd-zfs -s 1G ada2 ada2p1 added root@disp:~ # gpart add -t freebsd-zfs -s 1G -b 4194344 ada2 ada2p2 added root@disp:~ # gpart add -t freebsd-zfs -s 1G -b 8388648 ada2 ada2p3 added root@disp:~ # gpart show ada2 =3D> 40 976773088 ada2 GPT (466G) 40 2097152 1 freebsd-zfs (1.0G) 2097192 2097152 - free - (1.0G) 4194344 2097152 2 freebsd-zfs (1.0G) 6291496 2097152 - free - (1.0G) 8388648 2097152 3 freebsd-zfs (1.0G) 10485800 966287328 - free - (461G) root@disp:~ # zpool create testraid raidz ada2p1 ada2p2 ada2p3 root@disp:~ # zpool list testraid NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEAL= TH ALTROOT testraid 2.75G 696K 2.75G - - 0% 0% 1.00x ONLI= NE - root@disp:~ # zpool status testraid pool: testraid state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM testraid ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 ada2p1 ONLINE 0 0 0 ada2p2 ONLINE 0 0 0 ada2p3 ONLINE 0 0 0 errors: No known data errors root@disp:~ # zpool export testraid root@disp:~ # zpool import pool: testraid id: 3885773658779285422 state: ONLINE action: The pool can be imported using its name or numeric identifier. config: testraid ONLINE raidz1-0 ONLINE ada2p1 ONLINE ada2p2 ONLINE ada2p3 ONLINE root@disp:~ # zpool import testraid root@disp:~ # zpool status testraid pool: testraid state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM testraid ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 ada2p1 ONLINE 0 0 0 ada2p2 ONLINE 0 0 0 ada2p3 ONLINE 0 0 0 errors: No known data errors root@disp:~ # gpart resize -s 4194304 -i 1 ada2 ada2p1 resized root@disp:~ # gpart resize -s 4194304 -i 2 ada2 ada2p2 resized root@disp:~ # gpart resize -s 4194304 -i 3 ada2 ada2p3 resized root@disp:~ # gpart show ada2 =3D> 40 976773088 ada2 GPT (466G) 40 4194304 1 freebsd-zfs (2.0G) 4194344 4194304 2 freebsd-zfs (2.0G) 8388648 4194304 3 freebsd-zfs (2.0G) 12582952 964190176 - free - (460G) root@disp:~ # zpool status testraid pool: testraid state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM testraid ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 ada2p1 ONLINE 0 0 0 ada2p2 ONLINE 0 0 0 ada2p3 ONLINE 0 0 0 errors: No known data errors root@disp:~ # zpool export testraid root@disp:~ # zpool import pool: testraid id: 3885773658779285422 state: ONLINE action: The pool can be imported using its name or numeric identifier. config: testraid ONLINE raidz1-0 ONLINE ada2p1 ONLINE ada2p2 ONLINE ada2p3 ONLINE root@disp:~ # zpool import testraid root@disp:~ # zpool list testraid NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEAL= TH ALTROOT testraid 2.75G 1.02M 2.75G - 3G 0% 0% 1.00x ONLI= NE - root@disp:~ # zpool set autoexpand=3Don testraid root@disp:~ # zpool list testraid NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEAL= TH ALTROOT testraid 2.75G 1.32M 2.75G - 3G 0% 0% 1.00x ONLI= NE - root@disp:~ # zpool status testraid pool: testraid state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM testraid ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 ada2p1 ONLINE 0 0 0 ada2p2 ONLINE 0 0 0 ada2p3 ONLINE 0 0 0 errors: No known data errors root@disp:~ # zpool online -e testraid ada2p1 root@disp:~ # zpool status testraid pool: testraid state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM testraid ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 ada2p1 ONLINE 0 0 0 ada2p2 ONLINE 0 0 0 ada2p3 ONLINE 0 0 0 errors: No known data errors root@disp:~ # zpool list testraid NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEAL= TH ALTROOT testraid 5.75G 1.20M 5.75G - - 0% 0% 1.00x ONLI= NE - root@disp:~ #=20 root@disp:~ # zpool destroy testraid root@disp:~ # gpart destroy -F ada2 ada2 destroyed --------------- TESTCASE 2 END ------------------------ --------------- TESTCASE 3 BEGIN ------------------------ ## continuation of script from testcase 2 ## root@disp:~ # gpart create -s GPT ada2 ada2 created root@disp:~ # gpart add -t freebsd-zfs -s 1G ada2 ada2p1 added root@disp:~ # gpart add -t freebsd-zfs -s 1G -b 4194344 ada2 ada2p2 added root@disp:~ # gpart add -t freebsd-zfs -s 1G -b 8388648 ada2 ada2p3 added root@disp:~ # zpool create testraid raidz ada2p1 ada2p2 ada2p3 root@disp:~ # zpool status testraid pool: testraid state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM testraid ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 ada2p1 ONLINE 0 0 0 ada2p2 ONLINE 0 0 0 ada2p3 ONLINE 0 0 0 errors: No known data errors root@disp:~ # gpart resize -s 4194304 -i 1 ada2 ada2p1 resized root@disp:~ # gpart resize -s 4194304 -i 2 ada2 ada2p2 resized root@disp:~ # gpart resize -s 4194304 -i 3 ada2 ada2p3 resized root@disp:~ # zpool set autoexpand=3Dyes testraid root@disp:~ # zpool online -e testraid ada2p1 ## At this point the command hung forever and we were dead-in-the-water, ## no disk-I/O happening. Output ps shows "D"eadlock in "tx->tx_s": 0 4433 1224 0 20 0 7760 3764 tx->tx_s D+ 9 0:00.01 zpool No "df", no "sync", no "reboot". --------------- TESTCASE 3 END ------------------------ --------------- TESTCASE 4 BEGIN ------------------------ ## DON'T DARE TO COMMENT ON THE AGE OF THIS HARDWARE! ## (it is a precious antique) ## Script started on Thu May 16 22:07:04 2019 root@edge:~ # camcontrol devlist at scbus0 target 0 lun 0 (ada0,pass0) at scbus1 target 0 lun 0 (pass1,cd0) at scbus2 target 0 lun 0 (da0,pass2) at scbus2 target 2 lun 0 (da1,pass3) at scbus2 target 4 lun 0 (da2,pass4) at scbus4 target 0 lun 0 (ada1,pass5) at scbus5 target 0 lun 0 (ada2,pass6) at scbus7 target 0 lun 0 (ada3,pass7) at scbus8 target 0 lun 0 (ada4,pass8) at scbus10 target 0 lun 0 (da3,pass9) root@edge:~ # gpart add -t freebsd-zfs -s 1G ada2 ada2p5 added root@edge:~ # gpart show ada2 =3D> 40 5860533088 ada2 GPT (2.7T) 40 209715200 1 freebsd (100G) 209715240 1687971896 2 freebsd-zfs (805G) 1897687136 1924953472 4 freebsd-zfs (918G) 3822640608 55838024 3 freebsd-zfs (27G) 3878478632 2097152 5 freebsd-zfs (1.0G) 3880575784 1979957344 - free - (944G) root@edge:~ # gpart add -t freebsd-zfs -s 1G -b 3882672936 ada2 ada2p6 added root@edge:~ # gpart add -t freebsd-zfs -s 1G -b 3886867240 ada2 ada2p7 added root@edge:~ # gpart show ada2 =3D> 40 5860533088 ada2 GPT (2.7T) 40 209715200 1 freebsd (100G) 209715240 1687971896 2 freebsd-zfs (805G) 1897687136 1924953472 4 freebsd-zfs (918G) 3822640608 55838024 3 freebsd-zfs (27G) 3878478632 2097152 5 freebsd-zfs (1.0G) 3880575784 2097152 - free - (1.0G) 3882672936 2097152 6 freebsd-zfs (1.0G) 3884770088 2097152 - free - (1.0G) 3886867240 2097152 7 freebsd-zfs (1.0G) 3888964392 1971568736 - free - (940G) root@edge:~ # zpool create testraid raidz ada2p5 ada2p6 ada2p7 root@edge:~ # zpool list testraid NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEAL= TH ALTROOT testraid 2.75G 656K 2.75G - - 0% 0% 1.00x ONLI= NE - root@edge:~ # zpool status testraid pool: testraid state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM testraid ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 ada2p5 ONLINE 0 0 0 ada2p6 ONLINE 0 0 0 ada2p7 ONLINE 0 0 0 errors: No known data errors root@edge:~ # gpart resize -s 4194304 -i 5 ada2 ada2p5 resized root@edge:~ # gpart resize -s 4194304 -i 6 ada2 ada2p6 resized root@edge:~ # gpart resize -s 4194304 -i 7 ada2 ada2p7 resized root@edge:~ # gpart show ada2 =3D> 40 5860533088 ada2 GPT (2.7T) 40 209715200 1 freebsd (100G) 209715240 1687971896 2 freebsd-zfs (805G) 1897687136 1924953472 4 freebsd-zfs (918G) 3822640608 55838024 3 freebsd-zfs (27G) 3878478632 4194304 5 freebsd-zfs (2.0G) 3882672936 4194304 6 freebsd-zfs (2.0G) 3886867240 4194304 7 freebsd-zfs (2.0G) 3891061544 1969471584 - free - (939G) root@edge:~ # zpool set autoexpand=3Don testraid root@edge:~ # zpool list testraid NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEAL= TH ALTROOT testraid 2.75G 848K 2.75G - - 0% 0% 1.00x ONLI= NE - root@edge:~ # zpool export testraid root@edge:~ # zpool import pool: testraid id: 12608152059619624422 state: UNAVAIL status: One or more devices are missing from the system. action: The pool cannot be imported. Attach the missing devices and try again. see: http://illumos.org/msg/ZFS-8000-3C config: testraid UNAVAIL insufficient replicas raidz1-0 UNAVAIL insufficient replicas 13692722988113028666 UNAVAIL cannot open 10312580954503443965 UNAVAIL cannot open 16943054157341459289 UNAVAIL cannot open root@edge:~ # zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH= ALTROOT backup 918G 461G 456G - - 12% 50% 1.00x ONLINE= - bm 800G 581G 219G - - 13% 72% 1.00x ONLINE= - gr 50.5G 17.2G 33.3G - - 15% 34% 1.00x ONLINE= - im 26.5G 11.5G 15.0G - - 47% 43% 1.00x ONLINE= - root@edge:~ # sync root@edge:~ # gpart show ada2 =3D> 40 5860533088 ada2 GPT (2.7T) 40 209715200 1 freebsd (100G) 209715240 1687971896 2 freebsd-zfs (805G) 1897687136 1924953472 4 freebsd-zfs (918G) 3822640608 55838024 3 freebsd-zfs (27G) 3878478632 4194304 5 freebsd-zfs (2.0G) 3882672936 4194304 6 freebsd-zfs (2.0G) 3886867240 4194304 7 freebsd-zfs (2.0G) 3891061544 1969471584 - free - (939G) root@edge:~ # gpart resize -s 2097152 -i 5 ada2 ada2p5 resized root@edge:~ # gpart resize -s 2097152 -i 6 ada2 ada2p6 resized root@edge:~ # gpart resize -s 2097152 -i 7 ada2 ada2p7 resized root@edge:~ # gpart show ada2 =3D> 40 5860533088 ada2 GPT (2.7T) 40 209715200 1 freebsd (100G) 209715240 1687971896 2 freebsd-zfs (805G) 1897687136 1924953472 4 freebsd-zfs (918G) 3822640608 55838024 3 freebsd-zfs (27G) 3878478632 2097152 5 freebsd-zfs (1.0G) 3880575784 2097152 - free - (1.0G) 3882672936 2097152 6 freebsd-zfs (1.0G) 3884770088 2097152 - free - (1.0G) 3886867240 2097152 7 freebsd-zfs (1.0G) 3888964392 1971568736 - free - (940G) root@edge:~ # zpool import Assertion failed: (avl_find() succeeded inside avl_add()), file /usr/src/sy= s/cddl/contrib/opensolaris/common/avl/avl.c, line 649. Abort (core dumped) root@edge:~ # zpool import Assertion failed: (avl_find() succeeded inside avl_add()), file /usr/src/sy= s/cddl/contrib/opensolaris/common/avl/avl.c, line 649. Abort (core dumped) root@edge:~ # gpart delete -i 7 ada2 ada2p7 deleted root@edge:~ # gpart delete -i 6 ada2 ada2p6 deleted root@edge:~ # gpart delete -i 5 ada2 ada2p5 deleted root@edge:~ # zpool import root@edge:~ # ls -la /dev/gptid/ total 1 dr-xr-xr-x 2 root wheel 512 May 16 20:54 . dr-xr-xr-x 9 root wheel 512 May 16 20:54 .. crw-r----- 1 root operator 0xed May 16 22:22 4ea3d975-7816-11e9-a104-00e= 01836f13c crw-r----- 1 root operator 0xef May 16 22:22 934b476f-7816-11e9-a104-00e= 01836f13c crw-r----- 1 root operator 0x94 May 16 20:54 ac0d2fe4-4b3b-11e9-8d45-00e= 01836f13c crw-r----- 1 root operator 0xf1 May 16 22:22 b71d41a1-7816-11e9-a104-00e= 01836f13c ## ## Ignore the one timestamped 20:54! - that one belongs to the regular ## installation (and I didnt yet find a way to get rid of it). ## root@edge:~ #=20 root@edge:~ # exit exit Script done on Thu May 16 23:09:43 2019 ## ## During this procedure the following errors appeared in the syslog: May 16 22:22:31 edge kernel: g_access(944): provider gptid/4ea3= d975-7816-11e9-a104-00e01836f13c has error 6 set May 16 22:22:31 edge kernel: g_access(944): provider gptid/4ea3= d975-7816-11e9-a104-00e01836f13c has error 6 set May 16 22:22:31 edge kernel: g_dev_taste: make_dev_p() failed (= gp->name=3Dgptid/4ea3d975-7816-11e9-a104-00e01836f13c, error=3D17) May 16 22:22:48 edge kernel: g_access(944): provider gptid/934b= 476f-7816-11e9-a104-00e01836f13c has error 6 set May 16 22:22:48 edge kernel: g_access(944): provider gptid/934b= 476f-7816-11e9-a104-00e01836f13c has error 6 set May 16 22:22:48 edge kernel: g_dev_taste: make_dev_p() failed (= gp->name=3Dgptid/934b476f-7816-11e9-a104-00e01836f13c, error=3D17) May 16 22:22:55 edge kernel: g_access(944): provider gptid/b71d= 41a1-7816-11e9-a104-00e01836f13c has error 6 set May 16 22:22:55 edge kernel: g_access(944): provider gptid/b71d= 41a1-7816-11e9-a104-00e01836f13c has error 6 set May 16 22:22:55 edge kernel: g_dev_taste: make_dev_p() failed (= gp->name=3Dgptid/b71d41a1-7816-11e9-a104-00e01836f13c, error=3D17) May 16 22:23:21 edge kernel: pid 11779 (zpool), uid 0: exited o= n signal 6 (core dumped) May 16 22:27:35 edge kernel: pid 12342 (zpool), uid 0: exited o= n signal 6 (core dumped) May 16 22:42:36 edge kernel: g_access(944): provider gptid/934b= 476f-7816-11e9-a104-00e01836f13c has error 6 set May 16 22:42:36 edge kernel: g_access(944): provider gptid/b71d= 41a1-7816-11e9-a104-00e01836f13c has error 6 set May 16 22:42:36 edge kernel: g_access(944): provider gptid/4ea3= d975-7816-11e9-a104-00e01836f13c has error 6 set ## ## --------------- TESTCASE 4 END ------------------------ --------------- TESTCASE 5 BEGIN ------------------------ Script started on Thu May 16 22:45:58 2019 root@disp:~ # camcontrol devlist at scbus1 target 0 lun 0 (pass0,ada0) at scbus2 target 0 lun 0 (pass1,ada1) at scbus3 target 0 lun 0 (pass2,ada2) at scbus4 target 0 lun 0 (pass3,cd0) at scbus5 target 0 lun 0 (pass4,ses0) at scbus6 target 0 lun 0 (da0,pass5) root@disp:~ # gpart create -s GPT ada2 ada2 created root@disp:~ # gpart add -t freebsd-zfs -s 1G ada2 ada2p1 added root@disp:~ # gpart add -t freebsd-zfs -s 1G -b 4194344 ada2 ada2p2 added root@disp:~ # gpart add -t freebsd-zfs -s 1G -b 8388648 ada2 ada2p3 added root@disp:~ # gpart show ada2 =3D> 40 976773088 ada2 GPT (466G) 40 2097152 1 freebsd-zfs (1.0G) 2097192 2097152 - free - (1.0G) 4194344 2097152 2 freebsd-zfs (1.0G) 6291496 2097152 - free - (1.0G) 8388648 2097152 3 freebsd-zfs (1.0G) 10485800 966287328 - free - (461G) root@disp:~ # zpool create -f testraid raidz ada2p1 ada2p2 ada2p3 root@disp:~ # zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEAL= TH ALTROOT build 78G 34.0G 44.0G - - 35% 43% 1.00x ONLI= NE - media 464G 418G 46.5G - - 27% 89% 1.00x ONLI= NE - testraid 2.75G 552K 2.75G - - 0% 0% 1.00x ONLI= NE - zdesk 39.5G 11.9G 27.6G - - 16% 30% 1.00x ONLI= NE - root@disp:~ # gpart resize -s 4194304 -i 1 ada2 ada2p1 resized root@disp:~ # gpart resize -s 4194304 -i 2 ada2 ada2p2 resized root@disp:~ # gpart resize -s 4194304 -i 3 ada2 ada2p3 resized root@disp:~ # zpool set autoexpand=3Don testraid root@disp:~ # zpool export testraid root@disp:~ # zpool import pool: testraid id: 9285999494183920856 state: UNAVAIL status: One or more devices are missing from the system. action: The pool cannot be imported. Attach the missing devices and try again. see: http://illumos.org/msg/ZFS-8000-3C config: testraid UNAVAIL insufficient replicas raidz1-0 UNAVAIL insufficient replicas 5467198674063294812 UNAVAIL cannot open 16413066309772469567 UNAVAIL cannot open 10976529604851394099 UNAVAIL cannot open root@disp:~ # ## Here the typescript ends due to system crash. ## The next command entered was ## # gpart resize -s 2097152 -i 2 ada2 ## ## The system showed some messages for 1/4 second and then rebooted. ## Nevertheless, the resize itself had successed, as was visible ## after reboot. ## Resizing of partition #2 gave this output (transcript from photo): @$ gpart resize -s 2097152 -i 2 ada2 Fatal trap 12: page fault while in kernel mode cpuid =3D 3; apic id =3D 06 fault virtual address =3D 0x8 fault code =3D supervisor read data, page not present instruction pointer =3D 0x20:0xffffffff8076103e stack pointer =3D 0x28:0xfffffe02299209e0 frame pointer =3D 0x28:0xfffffe02299209f0 code segment =3D base 0x0, limit 0xfffff, tyoe 0x1b =3D DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 13 (g_event) [ thread pid 13 tid 100025 ] Stooped at g_dev_orphan+0x2e: cmpb $0,0x8(%r14) db> ## ## Resizing of partition #3 then delivered the same page fault. ## ## After this, the pool could be imported again: Script started on Thu May 16 23:28:14 2019 root@disp:~ # gpart show ada2 =3D> 40 976773088 ada2 GPT (466G) 40 2097152 1 freebsd-zfs (1.0G) 2097192 2097152 - free - (1.0G) 4194344 2097152 2 freebsd-zfs (1.0G) 6291496 2097152 - free - (1.0G) 8388648 2097152 3 freebsd-zfs (1.0G) 10485800 966287328 - free - (461G) root@disp:~ # zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH = ALTROOT build 78G 34.0G 44.0G - - 35% 43% 1.00x ONLINE = - media 464G 418G 46.5G - - 27% 89% 1.00x ONLINE = - zdesk 39.5G 11.9G 27.6G - - 16% 30% 1.00x ONLINE = - root@disp:~ # zpool import pool: testraid id: 9285999494183920856 state: ONLINE action: The pool can be imported using its name or numeric identifier. config: testraid ONLINE raidz1-0 ONLINE ada2p1 ONLINE ada2p2 ONLINE ada2p3 ONLINE root@disp:~ # zpool import testraid root@disp:~ # zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEAL= TH ALTROOT build 78G 34.0G 44.0G - - 35% 43% 1.00x ONLI= NE - media 464G 418G 46.5G - - 27% 89% 1.00x ONLI= NE - testraid 2.75G 968K 2.75G - - 0% 0% 1.00x ONLI= NE - zdesk 39.5G 11.9G 27.6G - - 16% 30% 1.00x ONLI= NE - root@disp:~ # zpool stat =08us testraid pool: testraid state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM testraid ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 ada2p1 ONLINE 0 0 0 ada2p2 ONLINE 0 0 0 ada2p3 ONLINE 0 0 0 errors: No known data errors root@disp:~ # exit Script done on Thu May 16 23:29:29 2019 --------------- TESTCASE 5 END ------------------------ COMMENTARY ---------- Case 1: The USB stick showed seek times of 10'000 ms and more during the operations and became hot. It seems the device is rather unfit for such operation and/or not capable to handle FLUSH commands properly. Case 4: It appears that the device nodes are STILL PRESENT in /dev/gptid at the end of the procedure even AFTER they had been destroyed. Might it be they are created somehow DUPLICATE and that being the reason for the failures? The main difference in the -successful- testcase 2 seems to be that an export+import is done AFTER resizing the partitions and BEFORE trying to grow the pool. This way ZFS can properly adjust the "expandsize" attribute, before actually doing the grow. So the issue seems mainly a matter of (not) following proper procedures (which, to my knowledge, aren't documented anywhere). cheerio, PMc From owner-freebsd-fs@freebsd.org Fri May 17 05:31:31 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 39D2D15B0133 for ; Fri, 17 May 2019 05:31:31 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vtr.rulingia.com (unknown [IPv6:2001:19f0:5801:ebe:59df:1a4a:8731:11c1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vtr.rulingia.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E850C8B4E2 for ; Fri, 17 May 2019 05:31:29 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from server.rulingia.com (ppp59-167-167-3.static.internode.on.net [59.167.167.3]) by vtr.rulingia.com (8.15.2/8.15.2) with ESMTPS id x4H5UnZ7014031 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 17 May 2019 15:30:54 +1000 (AEST) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.15.2/8.15.2) with ESMTPS id x4H5Uhhv024910 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 17 May 2019 15:30:43 +1000 (AEST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.15.2/8.15.2/Submit) id x4H5Uh7M024909; Fri, 17 May 2019 15:30:43 +1000 (AEST) (envelope-from peter) Date: Fri, 17 May 2019 15:30:43 +1000 From: Peter Jeremy To: Peter Cc: Miroslav Lachman <000.fbsd@quip.cz>, freebsd-fs@freebsd.org Subject: Re: Waht is the minimum free space ... (Full Report) Message-ID: <20190517053043.GF98005@server.rulingia.com> References: <20190515204243.GA67445@gate.oper.dinoex.org> <60d57363-eb5c-e985-82ad-30f03b06a4c6@quip.cz> <20190517010239.GA34758@gate.oper.dinoex.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="jousvV0MzM2p6OtC" Content-Disposition: inline In-Reply-To: <20190517010239.GA34758@gate.oper.dinoex.org> X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.11.4 (2019-03-13) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 May 2019 05:31:31 -0000 --jousvV0MzM2p6OtC Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2019-May-17 03:02:39 +0200, Peter wrote: >The original idea was to check if ZFS can grow a raid5. I've done this (see https://bugs.au.freebsd.org/dokuwiki/zfsraid), though I also migrated from RAIDZ1 to RAIDZ2 in the process. If this process no longer works (that page is 4 years old), it would seem that there has been an unfortunate regression. --=20 Peter Jeremy --jousvV0MzM2p6OtC Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEE7rKYbDBnHnTmXCJ+FqWXoOSiCzQFAlzeRwJfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEVF QjI5ODZDMzA2NzFFNzRFNjVDMjI3RTE2QTU5N0EwRTRBMjBCMzQACgkQFqWXoOSi CzRrmRAAg+1Radr8U4sSu9GsI+YDe9ISFBeFT2lc3AVPtZ3pf4Ku8OKWmMBq/EPX WmNCRmmjz7HMTOBoELbHyZLvB2s+6xRn3fSf8/3zTPa4uQRHQY78QBbv8Lbx8uyo s65VVQFqluUK1FYIqZPSFd1/+EBZB1PgUnEzY1Kot/KStnJOezzUlaMpqd2Pd5UC vmZ4amucaI0iQvqwc+0txBhkpjDXhQF/82KyEhzJMBd/DFmuk+iwA5ImDVpU5fx0 3wGMTaBQPhpsiyuuHvbO8UWJXIqmbJ2KWKK9GqxNfqO5YEBErYLMn0DSmQkQFKSj l0yx2GlOhxmS0jDQ7R83O38UweJp+aQHRsNDduTFyaI0WsOzb59z51JnNtziJTKE hnP9FTqWbhGFNgLJGGlJwCaEbAQV460V67w6e3/R1yJeMr8Lb02xiXJochP7FCr3 w71v8xlv+c+jNIiIRMDvpm+m0FiH84O18/Dr78HhxpeCQrFyV+tTwDdBJn688690 lNYSuaZvBzOBQIYzojpntwmO7Y+KYdlhoOBkWXiZaN7zNe/ImN366rDu2g2R/sRr kYgvTNeJ72Ha0nuDkHH8jfNgtt68uiHNrkjjnprB5X7i//4sLdjwqzZgW29AG8gb 0WmMwmC4x8J1E3lT8uuT+XLMYzWqc8FoRLw5+Vy7QrVRshLroww= =VR9i -----END PGP SIGNATURE----- --jousvV0MzM2p6OtC-- From owner-freebsd-fs@freebsd.org Fri May 17 13:13:12 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 911A4159518F for ; Fri, 17 May 2019 13:13:12 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.org (uucp.dinoex.sub.de [IPv6:2001:1440:5001:1::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "uucp.dinoex.sub.de", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8D6F47336B for ; Fri, 17 May 2019 13:13:11 +0000 (UTC) (envelope-from pmc@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [194.45.71.2]) by uucp.dinoex.org (8.16.0.41/8.16.0.41) with ESMTPS id x4HDD4jG027547 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Fri, 17 May 2019 15:13:05 +0200 (CEST) (envelope-from pmc@citylink.dinoex.sub.org) Received: from citylink.dinoex.sub.org (uucp@localhost) by uucp.dinoex.sub.de (8.16.0.41/8.16.0.41/Submit) with UUCP id x4HDD4up027546; Fri, 17 May 2019 15:13:04 +0200 (CEST) (envelope-from pmc@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.15.2/8.15.2) with ESMTP id x4HCpJL6023404; Fri, 17 May 2019 14:51:19 +0200 (CEST) (envelope-from peter@gate.oper.dinoex.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by gate.oper.dinoex.org (8.15.2/8.15.2) with ESMTP id x4HCnbtQ022849; Fri, 17 May 2019 14:49:38 +0200 (CEST) (envelope-from peter@gate.oper.dinoex.org) Received: (from peter@localhost) by gate.oper.dinoex.org (8.15.2/8.15.2/Submit) id x4HCnbvA022848; Fri, 17 May 2019 14:49:37 +0200 (CEST) (envelope-from peter) Date: Fri, 17 May 2019 14:49:37 +0200 From: Peter To: Peter Jeremy Cc: Miroslav Lachman <000.fbsd@quip.cz>, freebsd-fs@freebsd.org Subject: Re: Waht is the minimum free space ... (Full Report) Message-ID: <20190517124937.GA11835@gate.oper.dinoex.org> References: <20190515204243.GA67445@gate.oper.dinoex.org> <60d57363-eb5c-e985-82ad-30f03b06a4c6@quip.cz> <20190517010239.GA34758@gate.oper.dinoex.org> <20190517053043.GF98005@server.rulingia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190517053043.GF98005@server.rulingia.com> User-Agent: Mutt/1.11.4 (2019-03-13) X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 194.45.71.2; Sender-helo: uucp.dinoex.sub.de; ) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (uucp.dinoex.org [194.45.71.2]); Fri, 17 May 2019 15:13:08 +0200 (CEST) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 May 2019 13:13:12 -0000 On Fri, May 17, 2019 at 03:30:43PM +1000, Peter Jeremy wrote: ! On 2019-May-17 03:02:39 +0200, Peter wrote: ! >The original idea was to check if ZFS can grow a raid5. ! ! I've done this (see https://bugs.au.freebsd.org/dokuwiki/zfsraid), though I ! also migrated from RAIDZ1 to RAIDZ2 in the process. If this process no ! longer works (that page is 4 years old), it would seem that there has been ! an unfortunate regression. You don't mention setting "autoexpand=on" - I suppose it would not work without that. What we have here is most likely not a problem with the raid or it's growth, but a kind of "autority conflict" between ZFS and GPT on who is going to manage the underlying partitions. (Which doesn't surprize me - if I were ZFS, I would be quite frustrated to run under GPT.)