From owner-freebsd-fs@freebsd.org Sun Jan 29 08:15:07 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 855A5CC6C9C for ; Sun, 29 Jan 2017 08:15:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 74DEED5 for ; Sun, 29 Jan 2017 08:15:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v0T8F7BS004833 for ; Sun, 29 Jan 2017 08:15:07 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216368] [samba] Network group, machine, and share browsing does not work correctly. Date: Sun, 29 Jan 2017 08:15:07 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Ports & Packages X-Bugzilla-Component: Individual Port(s) X-Bugzilla-Version: Latest X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: ronald-lists@klop.ws X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2017 08:15:07 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216368 Ronald Klop changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |ronald-lists@klop.ws --- Comment #2 from Ronald Klop --- (In reply to Andi Maulana from comment #0) This looks like the start of /usr/ports/www/firefox/pkg-message. Is this a real bug report? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sun Jan 29 13:13:14 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4EB68CC6D38 for ; Sun, 29 Jan 2017 13:13:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3B131A9A for ; Sun, 29 Jan 2017 13:13:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v0TDDDNg015205 for ; Sun, 29 Jan 2017 13:13:14 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216364] ZFS ARC cache is duplicating data? The cache size gets bigger then the pool. Date: Sun, 29 Jan 2017 13:13:14 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: misc X-Bugzilla-Version: 11.0-RELEASE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: k_georgiev@deltanews.bg X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2017 13:13:14 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216364 --- Comment #2 from k_georgiev@deltanews.bg --- Hello,=20 I agree, looks a lot like it's related. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sun Jan 29 21:00:51 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 04779CC65BE for ; Sun, 29 Jan 2017 21:00:51 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E95F66AF for ; Sun, 29 Jan 2017 21:00:50 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v0TL01s5057301 for ; Sun, 29 Jan 2017 21:00:50 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201701292100.v0TL01s5057301@kenobi.freebsd.org> From: bugzilla-noreply@FreeBSD.org To: freebsd-fs@FreeBSD.org Subject: Problem reports for freebsd-fs@FreeBSD.org that need special attention Date: Sun, 29 Jan 2017 21:00:50 +0000 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Jan 2017 21:00:51 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 203492 | mount_unionfs -o below causes panic Open | 136470 | [nfs] Cannot mount / in read-only, over NFS Open | 139651 | [nfs] mount(8): read-only remount of NFS volume d Open | 140068 | [smbfs] [patch] smbfs does not allow semicolon in Open | 144447 | [zfs] sharenfs fsunshare() & fsshare_main() non f Open | 203419 | solaris assert: (dn->dn_phys->dn_nlevels == 0 && Open | 211491 | System hangs after "Uptime" on reboot with ZFS 7 problems total for which you should take action. From owner-freebsd-fs@freebsd.org Mon Jan 30 11:46:47 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 540A8CC6273 for ; Mon, 30 Jan 2017 11:46:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 43FBD1563 for ; Mon, 30 Jan 2017 11:46:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v0UBkkUe033805 for ; Mon, 30 Jan 2017 11:46:47 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216380] mv /[dir == mountpoint] causes kernel panic Date: Mon, 30 Jan 2017 11:46:46 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2017 11:46:47 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216380 --- Comment #2 from commit-hook@freebsd.org --- A commit references this bug: Author: pho Date: Mon Jan 30 11:46:06 UTC 2017 New revision: 312985 URL: https://svnweb.freebsd.org/changeset/base/312985 Log: Added a regression test. PR: 216380 Sponsored by: Dell EMC Isilon Changes: user/pho/stress2/misc/rename13.sh --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon Jan 30 11:50:46 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A8EC6CC65F5 for ; Mon, 30 Jan 2017 11:50:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 988FC191C for ; Mon, 30 Jan 2017 11:50:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v0UBojMD040113 for ; Mon, 30 Jan 2017 11:50:46 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216586] zfs panic: sa.sa_magic == 0x2F505A in zfs_space_delta_cb() Date: Mon, 30 Jan 2017 11:50:46 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2017 11:50:46 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216586 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- CC|freebsd-amd64@FreeBSD.org | Assignee|freebsd-bugs@FreeBSD.org |freebsd-fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon Jan 30 15:35:54 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CADEDCC7FF2 for ; Mon, 30 Jan 2017 15:35:54 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id B38B71513; Mon, 30 Jan 2017 15:35:53 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id RAA12880; Mon, 30 Jan 2017 17:35:50 +0200 (EET) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1cYDzi-000HdL-IN; Mon, 30 Jan 2017 17:35:50 +0200 Subject: Re: Could somebody look at PR216178? To: lev@FreeBSD.org, freebsd-fs@FreeBSD.org References: <2cadd04c-7b7c-9963-489d-d211215a1d3f@FreeBSD.org> Cc: Alexander Motin , George Wilson From: Andriy Gapon Message-ID: <76d1dab3-2c68-0b06-433c-1ee2aed7d10d@FreeBSD.org> Date: Mon, 30 Jan 2017 17:34:55 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.6.0 MIME-Version: 1.0 In-Reply-To: <2cadd04c-7b7c-9963-489d-d211215a1d3f@FreeBSD.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2017 15:35:54 -0000 On 18/01/2017 18:27, Lev Serebryakov wrote: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=216178 > > I could reproduce it on 10-STABLE and 11-STABLE and I'm afraid, that > this ruin ARC and L2ARC efficiency completely. > Lev, I haven't forgotten about this issue. Unfortunately, I can not devote as much time as I would like to it. I revisited it again and I couldn't spot anything new besides what my old patch was supposed to fix: https://docs.freebsd.org/cgi/getmsg.cgi?fetch=24428+0+archive/2016/freebsd-fs/20161106.freebsd-fs I know that George [cc-ed] was going to work on this problem. Perhaps, he has something for you to test or debug. -- Andriy Gapon From owner-freebsd-fs@freebsd.org Mon Jan 30 15:36:28 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E9105CC803B for ; Mon, 30 Jan 2017 15:36:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D8CDB15C5 for ; Mon, 30 Jan 2017 15:36:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v0UFaSXn053298 for ; Mon, 30 Jan 2017 15:36:28 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216178] ZFS ARC and L2ARC are unrealistically large, maybe after r307265 Date: Mon, 30 Jan 2017 15:36:29 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2017 15:36:29 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216178 --- Comment #2 from Andriy Gapon --- (In reply to Lev A. Serebryakov from comment #0) Lev, I haven't forgotten about this issue. Unfortunately, I can not devote as much time as I would like to it. I revisited it again and I couldn't spot anything new besides what my old p= atch was supposed to fix: https://docs.freebsd.org/cgi/getmsg.cgi?fetch=3D24428+0+archive/2016/freebs= d-fs/20161106.freebsd-fs I know that George Wilson was going to work on this problem. Perhaps, he h= as something for you to test or debug. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon Jan 30 16:16:07 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F1908CC8A6E for ; Mon, 30 Jan 2017 16:16:07 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from onlyone.not-for.work (onlyone.not-for.work [IPv6:2a01:4f8:201:6350::2]) by mx1.freebsd.org (Postfix) with ESMTP id BEF58DBE; Mon, 30 Jan 2017 16:16:07 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from [192.168.17.133] (unknown [89.113.128.32]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.not-for.work (Postfix) with ESMTPSA id 9E82BC8B; Mon, 30 Jan 2017 19:15:58 +0300 (MSK) Reply-To: lev@FreeBSD.org Subject: Re: Could somebody look at PR216178? References: <2cadd04c-7b7c-9963-489d-d211215a1d3f@FreeBSD.org> <76d1dab3-2c68-0b06-433c-1ee2aed7d10d@FreeBSD.org> To: Andriy Gapon , freebsd-fs@FreeBSD.org Cc: Alexander Motin , George Wilson From: Lev Serebryakov Organization: FreeBSD Message-ID: <9965c55c-0061-1836-1628-a32246c7a33e@FreeBSD.org> Date: Mon, 30 Jan 2017 19:15:58 +0300 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: <76d1dab3-2c68-0b06-433c-1ee2aed7d10d@FreeBSD.org> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2017 16:16:08 -0000 On 30.01.2017 18:34, Andriy Gapon wrote: >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=216178 >> >> I could reproduce it on 10-STABLE and 11-STABLE and I'm afraid, that >> this ruin ARC and L2ARC efficiency completely. >> > > Lev, > > I haven't forgotten about this issue. > Unfortunately, I can not devote as much time as I would like to it. > I revisited it again and I couldn't spot anything new besides what my old patch > was supposed to fix: > https://docs.freebsd.org/cgi/getmsg.cgi?fetch=24428+0+archive/2016/freebsd-fs/20161106.freebsd-fs > > I know that George [cc-ed] was going to work on this problem. Perhaps, he has > something for you to test or debug. Now here is second installation with same problem (PR#216364), looks like it is not my local problem. I'm ready to test any patches & provide any additional information. -- // Lev Serebryakov From owner-freebsd-fs@freebsd.org Mon Jan 30 21:47:02 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47E89CC8EDF for ; Mon, 30 Jan 2017 21:47:02 +0000 (UTC) (envelope-from ler@FreeBSD.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2859810B5 for ; Mon, 30 Jan 2017 21:47:02 +0000 (UTC) (envelope-from ler@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:Subject:To:From:Date:Content-Transfer-Encoding: Content-Type:MIME-Version:Sender:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: In-Reply-To:References:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=WA5pqSVCGNmrFndZ+CMn9/QOr2LkWLYY/e7LAp+WL8Y=; b=rfSLN5STgyJvGYx/lB3dfE6IPF bHBmKbEIHoMbvi+tVUMMV+kvKTOXEYWcmeZ7LUe6Mz7/EcmLsWsPzf+q0N3B8JWXbeddRCOlAg5qm X0RPXG06skKDxYdSnMT87vPz0Mw/tfuRxOv54TcCRAKTu1Ca8e/cVOz5tz9RnKy7VCnE=; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]:59734 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.88 (FreeBSD)) (envelope-from ) id 1cYJmv-000K0k-HZ for freebsd-fs@freebsd.org; Mon, 30 Jan 2017 15:47:01 -0600 Received: from proxy.na.alcatel-lucent.com ([135.245.48.75]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Mon, 30 Jan 2017 15:47:01 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Mon, 30 Jan 2017 15:47:01 -0600 From: Larry Rosenman To: Freebsd fs Subject: 16.0E ExpandSize? -- New Server Message-ID: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> X-Sender: ler@FreeBSD.org User-Agent: Roundcube Webmail/1.2.3 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2017 21:47:02 -0000 Greetings, Have a new server with a recent -CURRENT on it. I see an expandsize of 16.0E, which is insane because there are only 6 2TB disks. borg-new /home/ler $ zpool status pool: zroot state: ONLINE scan: scrub repaired 0 in 0h7m with 0 errors on Mon Jan 30 15:43:27 2017 config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 mfid4p4 ONLINE 0 0 0 mfid0p4 ONLINE 0 0 0 mfid1p4 ONLINE 0 0 0 mfid3p4 ONLINE 0 0 0 mfid2p4 ONLINE 0 0 0 mfid5p4 ONLINE 0 0 0 errors: No known data errors borg-new /home/ler $ zpool list NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT zroot 10.8T 94.2G 10.7T 16.0E 0% 0% 1.00x ONLINE - borg-new /home/ler $ zpool status pool: zroot state: ONLINE scan: scrub repaired 0 in 0h7m with 0 errors on Mon Jan 30 15:43:27 2017 config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 mfid4p4 ONLINE 0 0 0 mfid0p4 ONLINE 0 0 0 mfid1p4 ONLINE 0 0 0 mfid3p4 ONLINE 0 0 0 mfid2p4 ONLINE 0 0 0 mfid5p4 ONLINE 0 0 0 errors: No known data errors borg-new /home/ler $ zpool history History for 'zroot': cannot show history for pool 'zroot': permission denied borg-new /home/ler $ sudo zpool history Password: History for 'zroot': 2017-01-25.19:02:15 zpool create -o altroot=/mnt -O compress=lz4 -O atime=off -m none -f zroot raidz1 mfid0p4 mfid1p4 mfid2p4 mfid3p4 mfid4p4 mfid5p4 2017-01-25.19:02:15 zfs create -o mountpoint=none zroot/ROOT 2017-01-25.19:02:15 zfs create -o mountpoint=/ zroot/ROOT/default 2017-01-25.19:02:15 zfs create -o mountpoint=/tmp -o exec=on -o setuid=off zroot/tmp 2017-01-25.19:02:15 zfs create -o mountpoint=/usr -o canmount=off zroot/usr 2017-01-25.19:02:15 zfs create zroot/usr/home 2017-01-25.19:02:15 zfs create -o setuid=off zroot/usr/ports 2017-01-25.19:02:15 zfs create zroot/usr/src 2017-01-25.19:02:15 zfs create -o mountpoint=/var -o canmount=off zroot/var 2017-01-25.19:02:15 zfs create -o exec=off -o setuid=off zroot/var/audit 2017-01-25.19:02:15 zfs create -o exec=off -o setuid=off zroot/var/crash 2017-01-25.19:02:15 zfs create -o exec=off -o setuid=off zroot/var/log 2017-01-25.19:02:15 zfs create -o atime=on zroot/var/mail 2017-01-25.19:02:15 zfs create -o setuid=off zroot/var/tmp 2017-01-25.19:02:15 zfs set mountpoint=/zroot zroot 2017-01-25.19:02:15 zpool set bootfs=zroot/ROOT/default zroot 2017-01-25.19:02:15 zpool set cachefile=/mnt/boot/zfs/zpool.cache zroot 2017-01-25.19:02:19 zfs set canmount=noauto zroot/ROOT/default 2017-01-26.19:51:23 zpool scrub zroot 2017-01-29.12:50:56 zpool replace zroot 6435283924725851469 /dev/mfid5p4 2017-01-29.12:52:44 zpool scrub zroot 2017-01-29.13:11:50 zfs create zroot/usr/obj 2017-01-29.13:26:11 zfs create zroot/ler 2017-01-29.13:27:58 zfs set mountpoint=/home/ler zroot/ler 2017-01-29.16:22:07 zpool replace zroot 8173792548235216166 /dev/mfid4p4 2017-01-29.16:27:18 zpool scrub zroot 2017-01-29.16:32:15 zfs create -o mountpoint=/home zroot/home 2017-01-30.15:36:34 zpool scrub zroot borg-new /home/ler $ gpart show => 40 3905945520 mfid0 GPT (1.8T) 40 1600 1 efi (800K) 1640 1024 2 freebsd-boot (512K) 2664 1432 - free - (716K) 4096 16777216 3 freebsd-swap (8.0G) 16781312 3889162240 4 freebsd-zfs (1.8T) 3905943552 2008 - free - (1.0M) => 40 3905945520 mfid1 GPT (1.8T) 40 1600 1 efi (800K) 1640 1024 2 freebsd-boot (512K) 2664 1432 - free - (716K) 4096 16777216 3 freebsd-swap (8.0G) 16781312 3889162240 4 freebsd-zfs (1.8T) 3905943552 2008 - free - (1.0M) => 40 3905945520 mfid2 GPT (1.8T) 40 1600 1 efi (800K) 1640 1024 2 freebsd-boot (512K) 2664 1432 - free - (716K) 4096 16777216 3 freebsd-swap (8.0G) 16781312 3889162240 4 freebsd-zfs (1.8T) 3905943552 2008 - free - (1.0M) => 40 3905945520 mfid3 GPT (1.8T) 40 1600 1 efi (800K) 1640 1024 2 freebsd-boot (512K) 2664 16777216 3 freebsd-swap (8.0G) 16779880 3889165680 4 freebsd-zfs (1.8T) => 40 3905945520 mfid5 GPT (1.8T) 40 1600 1 efi (800K) 1640 1024 2 freebsd-boot (512K) 2664 1432 - free - (716K) 4096 16777216 3 freebsd-swap (8.0G) 16781312 3889162240 4 freebsd-zfs (1.8T) 3905943552 2008 - free - (1.0M) => 40 3905945520 mfid4 GPT (1.8T) 40 1600 1 efi (800K) 1640 1024 2 freebsd-boot (512K) 2664 1432 - free - (716K) 4096 16777216 3 freebsd-swap (8.0G) 16781312 3889160192 4 freebsd-zfs (1.8T) 3905941504 4056 - free - (2.0M) borg-new /home/ler $ borg-new /home/ler $ uname -aKU FreeBSD borg-new 12.0-CURRENT FreeBSD 12.0-CURRENT #1 r312893: Fri Jan 27 12:39:05 CST 2017 root@borg-new:/usr/obj/usr/src/sys/GENERIC amd64 1200020 1200020 borg-new /home/ler $ Ideas? -- Larry Rosenman http://people.freebsd.org/~ler Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 From owner-freebsd-fs@freebsd.org Mon Jan 30 21:54:47 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4C6ADCC8219 for ; Mon, 30 Jan 2017 21:54:47 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-it0-x22b.google.com (mail-it0-x22b.google.com [IPv6:2607:f8b0:4001:c0b::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1C4A21697 for ; Mon, 30 Jan 2017 21:54:46 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-it0-x22b.google.com with SMTP id c7so203497111itd.1 for ; Mon, 30 Jan 2017 13:54:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:from:date:message-id:subject:to; bh=sPMy76SAbiEixdx0G2nj8CUA9HG8hyz7yuKjzBAkqZY=; b=JovVSunmsOWCY+Coth9tIeswrShklF+Pi14IAG3OrLFAFvCFMCyy01B/X76n5SrKjq mpg5poMrOAhC5GPAI1l8tfQ5QDF1zLaw2sJX9EjvbbjTD8Z0us+f+X7M0a8sDOeLQB9a M8q/SN04aT4FY3kG8Lve1tQNn15REdObQ4gvA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=sPMy76SAbiEixdx0G2nj8CUA9HG8hyz7yuKjzBAkqZY=; b=OdNdjfmZNLDlGIVt9iggzxVu88T1RBXKBQ54C8I8LF3e77CeVARGW5M7eNMrE00SmI XWro52nW21fKPnmPU9HUi/FDGA+ufh4XBeVsgubt6DNtaPPu5ZNBkLXApSM7zKUWis6B hna05OxmkvyA2fqtOpepoGOUd87ADLHdHB14owxygVsU89urOYDyn/0OR6m986xDCRt7 pmHes0NzSZm/D939ztt/Vn3CpQb1PBv8zAzyyy1quFCDRzmcT5MCa7CDtakYSzPrxSyv TWwothygp2YpZWpq3koC3PHVki6io8e0gfYWmp55fbP3ZVjHnwRZSXERayxOfhmfN5bR EOkA== X-Gm-Message-State: AIkVDXLnn5dkt5A1W0kI2279od6ya595PstOgZRpMATw2r52KkQipiiNNePfqghyPWvyXe7/V6oGMU0m/oucCHnG X-Received: by 10.36.250.201 with SMTP id v192mr18010509ith.43.1485813286205; Mon, 30 Jan 2017 13:54:46 -0800 (PST) MIME-Version: 1.0 Received: by 10.36.65.94 with HTTP; Mon, 30 Jan 2017 13:54:45 -0800 (PST) From: Matthew Ahrens Date: Mon, 30 Jan 2017 13:54:45 -0800 Message-ID: Subject: First ZFS User Conference To: "developer@lists.illumos.org" , illumos-zfs , freebsd-fs , zfs-discuss , Illumos Discussion Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2017 21:54:47 -0000 I will be speaking at the first ZFS User Conference, along with other ZFS developers including Brian Behlendorf, Tom Caputi, and Don Brady. The conference is March 16-17 in Norwalk CT. I would invite anyone using or working with ZFS to attend! This conference will focus on the deployment, administration, and tuning of the ZFS filesystem. As the name implies, it is more oriented towards end-users of ZFS than the OpenZFS Developer Summit, but we will still cover some next-gen features and a few technical details. More info here: http://zfs.datto.com/ --matt From owner-freebsd-fs@freebsd.org Mon Jan 30 23:40:31 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4891FCC8345 for ; Mon, 30 Jan 2017 23:40:31 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x233.google.com (mail-wm0-x233.google.com [IPv6:2a00:1450:400c:c09::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CEDCAE28 for ; Mon, 30 Jan 2017 23:40:30 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wm0-x233.google.com with SMTP id b65so56132871wmf.0 for ; Mon, 30 Jan 2017 15:40:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=QLRLn68OxDaaBgWL/XrWF4CaFPZPnWCs1d0dXg1+SdI=; b=pjKZA4zlq86n0EPGRiEZFi3YviGfuMq0I9obV/zjJeBwkJuI09UMzsQbsjC4Tv3uTQ rfLyNdCU1oNWJe66kmxn1IQNYIpdWMdkWRaBlcT7EXFoiOByuT7Zve77dyB5DSP1tluD z+bRZy1THyHCgo9WEmroJrLog+0qE7obdU2QMYomqoQJeNqCgQNlziOAeO6tpehZHGkD KGtP3ndMs2rOmELcTc0658jQY4MRiMZX5yM8RRa0s8gjNVVx1HsM6UPPufGmBCO0y+OS 0+dlDv1KcA4ODkOLl81UIYcCxcGrpk5FmZNQe1gdoBdHnfA+0y9eZX7MUnNUnxXruFtn FT/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=QLRLn68OxDaaBgWL/XrWF4CaFPZPnWCs1d0dXg1+SdI=; b=gFTtLtLoEAW4NE1bRiWyD+H8z8v36WikfrjRGw8b0gw5gKP5b4Vn3XhuM3eQ60N9Wh 81+nVR8CU4d2VPwsIYnGAcTyw8A2PnqLHWveZ832Vb8ipICHv4VeUaHnvoLke5FU2oTg r+sJwL2rmwbZeNHPLRv0qLHirAhOCsCF0OmOOObRUOE1TIFNWIHr1/I2X3F3ZMR3Bmt4 jGZ1yVU4XMGSDp/TUb/7b4NBuJ4o7RO5bRQi4GKz0ZDbj7g8Vo54purpkoml+u7V7GVy pHxzpRoWqNb2sW7Lx4fcwFzeTSi6QTNyFngnYZqJOb0gVdrNYexpny6Bm8LfiO4fC1eJ R7RA== X-Gm-Message-State: AIkVDXIrR77rLz942weAjAXABpoOTggMwCrbO+4ubTmZzgzPc/Jh28FG4VYcvgH80mUJzrGc X-Received: by 10.28.62.204 with SMTP id l195mr15126653wma.88.1485819628713; Mon, 30 Jan 2017 15:40:28 -0800 (PST) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id y145sm21000040wmc.17.2017.01.30.15.40.27 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 30 Jan 2017 15:40:28 -0800 (PST) Subject: Re: 16.0E ExpandSize? -- New Server To: freebsd-fs@freebsd.org References: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> From: Steven Hartland Message-ID: <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> Date: Mon, 30 Jan 2017 23:40:28 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 30 Jan 2017 23:40:31 -0000 What do you get from: zpool get expandsize On 30/01/2017 21:47, Larry Rosenman wrote: > Greetings, > Have a new server with a recent -CURRENT on it. I see an > expandsize of 16.0E, which is insane > because there are only 6 2TB disks. > > borg-new /home/ler $ zpool status > pool: zroot > state: ONLINE > scan: scrub repaired 0 in 0h7m with 0 errors on Mon Jan 30 15:43:27 > 2017 > config: > > NAME STATE READ WRITE CKSUM > zroot ONLINE 0 0 0 > raidz1-0 ONLINE 0 0 0 > mfid4p4 ONLINE 0 0 0 > mfid0p4 ONLINE 0 0 0 > mfid1p4 ONLINE 0 0 0 > mfid3p4 ONLINE 0 0 0 > mfid2p4 ONLINE 0 0 0 > mfid5p4 ONLINE 0 0 0 > > errors: No known data errors > borg-new /home/ler $ zpool list > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > zroot 10.8T 94.2G 10.7T 16.0E 0% 0% 1.00x ONLINE - > borg-new /home/ler $ zpool status > pool: zroot > state: ONLINE > scan: scrub repaired 0 in 0h7m with 0 errors on Mon Jan 30 15:43:27 > 2017 > config: > > NAME STATE READ WRITE CKSUM > zroot ONLINE 0 0 0 > raidz1-0 ONLINE 0 0 0 > mfid4p4 ONLINE 0 0 0 > mfid0p4 ONLINE 0 0 0 > mfid1p4 ONLINE 0 0 0 > mfid3p4 ONLINE 0 0 0 > mfid2p4 ONLINE 0 0 0 > mfid5p4 ONLINE 0 0 0 > > errors: No known data errors > borg-new /home/ler $ zpool history > History for 'zroot': > cannot show history for pool 'zroot': permission denied > borg-new /home/ler $ sudo zpool history > Password: > History for 'zroot': > 2017-01-25.19:02:15 zpool create -o altroot=/mnt -O compress=lz4 -O > atime=off -m none -f zroot raidz1 mfid0p4 mfid1p4 mfid2p4 mfid3p4 > mfid4p4 mfid5p4 > 2017-01-25.19:02:15 zfs create -o mountpoint=none zroot/ROOT > 2017-01-25.19:02:15 zfs create -o mountpoint=/ zroot/ROOT/default > 2017-01-25.19:02:15 zfs create -o mountpoint=/tmp -o exec=on -o > setuid=off zroot/tmp > 2017-01-25.19:02:15 zfs create -o mountpoint=/usr -o canmount=off > zroot/usr > 2017-01-25.19:02:15 zfs create zroot/usr/home > 2017-01-25.19:02:15 zfs create -o setuid=off zroot/usr/ports > 2017-01-25.19:02:15 zfs create zroot/usr/src > 2017-01-25.19:02:15 zfs create -o mountpoint=/var -o canmount=off > zroot/var > 2017-01-25.19:02:15 zfs create -o exec=off -o setuid=off zroot/var/audit > 2017-01-25.19:02:15 zfs create -o exec=off -o setuid=off zroot/var/crash > 2017-01-25.19:02:15 zfs create -o exec=off -o setuid=off zroot/var/log > 2017-01-25.19:02:15 zfs create -o atime=on zroot/var/mail > 2017-01-25.19:02:15 zfs create -o setuid=off zroot/var/tmp > 2017-01-25.19:02:15 zfs set mountpoint=/zroot zroot > 2017-01-25.19:02:15 zpool set bootfs=zroot/ROOT/default zroot > 2017-01-25.19:02:15 zpool set cachefile=/mnt/boot/zfs/zpool.cache zroot > 2017-01-25.19:02:19 zfs set canmount=noauto zroot/ROOT/default > 2017-01-26.19:51:23 zpool scrub zroot > 2017-01-29.12:50:56 zpool replace zroot 6435283924725851469 /dev/mfid5p4 > 2017-01-29.12:52:44 zpool scrub zroot > 2017-01-29.13:11:50 zfs create zroot/usr/obj > 2017-01-29.13:26:11 zfs create zroot/ler > 2017-01-29.13:27:58 zfs set mountpoint=/home/ler zroot/ler > 2017-01-29.16:22:07 zpool replace zroot 8173792548235216166 /dev/mfid4p4 > 2017-01-29.16:27:18 zpool scrub zroot > 2017-01-29.16:32:15 zfs create -o mountpoint=/home zroot/home > 2017-01-30.15:36:34 zpool scrub zroot > > borg-new /home/ler $ gpart show > => 40 3905945520 mfid0 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid1 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid2 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid3 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 16777216 3 freebsd-swap (8.0G) > 16779880 3889165680 4 freebsd-zfs (1.8T) > > => 40 3905945520 mfid5 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid4 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889160192 4 freebsd-zfs (1.8T) > 3905941504 4056 - free - (2.0M) > > borg-new /home/ler $ > > borg-new /home/ler $ uname -aKU > FreeBSD borg-new 12.0-CURRENT FreeBSD 12.0-CURRENT #1 r312893: Fri Jan > 27 12:39:05 CST 2017 root@borg-new:/usr/obj/usr/src/sys/GENERIC amd64 > 1200020 1200020 > borg-new /home/ler $ > > Ideas? > > From owner-freebsd-fs@freebsd.org Tue Jan 31 00:57:05 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 15489CC7E99 for ; Tue, 31 Jan 2017 00:57:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0530AB9B for ; Tue, 31 Jan 2017 00:57:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v0V0v4KP045457 for ; Tue, 31 Jan 2017 00:57:04 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216586] zfs panic: sa.sa_magic == 0x2F505A in zfs_space_delta_cb() Date: Tue, 31 Jan 2017 00:57:05 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: johannes@jo-t.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2017 00:57:05 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216586 --- Comment #2 from johannes@jo-t.de --- Haven't done any zfs/zpool upgrades in a long time. zfs version is 5 (for this and all other fs in the pool). zpool version is: > NAME PROPERTY VALUE S= OURCE > [...snip...] > freddata version - d= efault > freddata feature@async_destroy enabled l= ocal > freddata feature@empty_bpobj active l= ocal > freddata feature@lz4_compress active l= ocal > freddata feature@multi_vdev_crash_dump enabled l= ocal > freddata feature@spacemap_histogram active l= ocal > freddata feature@enabled_txg active l= ocal > freddata feature@hole_birth active l= ocal > freddata feature@extensible_dataset enabled l= ocal > freddata feature@embedded_data disabled l= ocal > freddata feature@bookmarks enabled l= ocal > freddata feature@filesystem_limits disabled l= ocal > freddata feature@large_blocks disabled l= ocal --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Tue Jan 31 03:36:59 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DBCEBC98811 for ; Tue, 31 Jan 2017 03:36:59 +0000 (UTC) (envelope-from francois.dion@gmail.com) Received: from mail-wm0-x230.google.com (mail-wm0-x230.google.com [IPv6:2a00:1450:400c:c09::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8B32F1768 for ; Tue, 31 Jan 2017 03:36:59 +0000 (UTC) (envelope-from francois.dion@gmail.com) Received: by mail-wm0-x230.google.com with SMTP id c85so241591782wmi.1 for ; Mon, 30 Jan 2017 19:36:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=/Vt5o9wn0kxBZMr3zE2xxzfHcDnNtPXuDOVqpM9eUx4=; b=MDJy/3/bxK1gC6GJQQ3EcxPeuRHPM42I1ZoKfYGzFnbEIofizY0VuAeBG1bJPgWK5K IxUk9o+vW0vpdymaeh/lIeC9KBbQjS+iSdxtERXgIcx9ZkLFzhtRi5nbJpeYhVHgaNXy zQmdT+zZoj9gZwWyYZSCJSpBpxiqgDQ1LNQWwLkgZxBawgosO3yEFOXoOcQXQAZagNop sefAsOLfToRMK3e1s+xr39vLrJYYTsX5gAnkA0euQcznrsNHlmTmqr/PQRSSVBYiBxzF LzwI1NoSE027oGFrNF8fzvhNq8+geDMlBnySQX0bNN8MxQH0ehI5t/ZgX5YRU1Byrvp/ Kw5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=/Vt5o9wn0kxBZMr3zE2xxzfHcDnNtPXuDOVqpM9eUx4=; b=jcoXgmXbPM/h7oeBixiKeRPVpHBZdnxD08Z+2iFn3kkPTfdDlAGBLKeUbS9X+Z/DOG ZURjaEoIHVuXVlpS95bs5dTg1pTn4gUp28cLzIj6BcvnMRn9du6KJqSH8NNo1Dwyw0h4 iJUcbHGK1Q04s3se3Hb9bqpvjFlpGq/ByqMcwL8LL/y9FPS5Svt50Ea6+BuDnQGfKKaH egAZGkbJmusgzeK5DfMXgggB6xww1+jaRFLzrRXP6nsqSP7kkDvdbt0QUM1BXgn3p524 9MoSWMS9WyurH0buU82EmkNzsFuNWBVuHGBnqamZmgRVekxf4L+oKsapw8opNu3y0D4a V34Q== X-Gm-Message-State: AIkVDXKHoerc31I9B6CQBaeJNkiVaxkUV+o9aAfIBSRfBZPjl6hNTZz1nepGRctQ88pMnPe87BxUa1ump2mltw== X-Received: by 10.28.57.131 with SMTP id g125mr15786512wma.33.1485833816873; Mon, 30 Jan 2017 19:36:56 -0800 (PST) MIME-Version: 1.0 Received: by 10.28.127.216 with HTTP; Mon, 30 Jan 2017 19:36:56 -0800 (PST) In-Reply-To: References: From: Francois Dion Date: Mon, 30 Jan 2017 22:36:56 -0500 Message-ID: Subject: Re: [discuss] First ZFS User Conference To: discuss Cc: "developer@lists.illumos.org" , illumos-zfs , freebsd-fs , zfs-discuss Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2017 03:37:00 -0000 And if you are wondering how ZFS fits in a data science or reproducible research framework, I'll be covering that, including Big Data. Francois @f_dion On Mon, Jan 30, 2017 at 4:54 PM, Matthew Ahrens wrote: > I will be speaking at the first ZFS User Conference, along with other ZFS > developers including Brian Behlendorf, Tom Caputi, and Don Brady. The > conference is March 16-17 in Norwalk CT. I would invite anyone using or > working with ZFS to attend! > > This conference will focus on the deployment, administration, and tuning > of the ZFS filesystem. As the name implies, it is more oriented towards > end-users of ZFS than the OpenZFS Developer Summit, but we will still cover > some next-gen features and a few technical details. > > More info here: http://zfs.datto.com/ > > --matt > *illumos-discuss* | Archives > > | > Modify > > Your Subscription > From owner-freebsd-fs@freebsd.org Tue Jan 31 13:04:48 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2DD13CC9D32 for ; Tue, 31 Jan 2017 13:04:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1D9E11896 for ; Tue, 31 Jan 2017 13:04:48 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v0VD4lxk087963 for ; Tue, 31 Jan 2017 13:04:47 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216586] zfs panic: sa.sa_magic == 0x2F505A in zfs_space_delta_cb() Date: Tue, 31 Jan 2017 13:04:48 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2017 13:04:48 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216586 --- Comment #3 from Andriy Gapon --- Could it be that you have some old files that were created before your last= zfs upgrade (even if that was a long time ago)? Could it be that rsync updates those files? You might find this issue (close, but no associated commit) to be interesti= ng: https://github.com/zfsonlinux/zfs/issues/2025 I see that in your case the wrong magic values also look like Unix timestam= ps, both from 25 April 2013. Could you please check if you still have more files from that era? P.S. Some other similar reports in ZoL repository: https://github.com/zfsonlinux/zfs/issues/1303 https://github.com/zfsonlinux/zfs/issues/3968 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Tue Jan 31 14:51:18 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EA7DBCC9C87 for ; Tue, 31 Jan 2017 14:51:18 +0000 (UTC) (envelope-from ler@FreeBSD.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C8B8510EF for ; Tue, 31 Jan 2017 14:51:18 +0000 (UTC) (envelope-from ler@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date: Content-Transfer-Encoding:Content-Type:MIME-Version:Sender:Reply-To: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=ChI7uspW2prnQDxiPG9flLfP+X3kl+/rtInAZ5rhteE=; b=q+WI6wLUktEzSqQRS0xNe8+YSs 2RETmTQhE0L+8jyJLkKAHJ5NhOXLG/6AIL3QSlbb3GubrZRNDwSYncTpBVkqha9XeUCavxHEZEFiZ FWDIO5LFhrnREyzRr35YZd7d7isHxqy3juuHFUBxRjVvBT9d6HOv9ZJM64pX/d31FlNs=; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]:17049 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.88 (FreeBSD)) (envelope-from ) id 1cYZm9-000Pdd-4W; Tue, 31 Jan 2017 08:51:17 -0600 Received: from proxy.na.alcatel-lucent.com ([135.245.48.75]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Tue, 31 Jan 2017 08:51:17 -0600 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Tue, 31 Jan 2017 08:51:17 -0600 From: Larry Rosenman To: Steven Hartland Cc: Freebsd fs Subject: Re: 16.0E ExpandSize? -- New Server In-Reply-To: <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> References: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> Message-ID: X-Sender: ler@FreeBSD.org User-Agent: Roundcube Webmail/1.2.3 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2017 14:51:19 -0000 borg-new /home/ler $ zpool get expandsize NAME PROPERTY VALUE SOURCE zroot expandsize 16.0E - borg-new /home/ler $ On 01/30/2017 5:40 pm, Steven Hartland wrote: > What do you get from: > zpool get expandsize > > On 30/01/2017 21:47, Larry Rosenman wrote: >> Greetings, >> Have a new server with a recent -CURRENT on it. I see an >> expandsize of 16.0E, which is insane >> because there are only 6 2TB disks. >> >> borg-new /home/ler $ zpool status >> pool: zroot >> state: ONLINE >> scan: scrub repaired 0 in 0h7m with 0 errors on Mon Jan 30 15:43:27 >> 2017 >> config: >> >> NAME STATE READ WRITE CKSUM >> zroot ONLINE 0 0 0 >> raidz1-0 ONLINE 0 0 0 >> mfid4p4 ONLINE 0 0 0 >> mfid0p4 ONLINE 0 0 0 >> mfid1p4 ONLINE 0 0 0 >> mfid3p4 ONLINE 0 0 0 >> mfid2p4 ONLINE 0 0 0 >> mfid5p4 ONLINE 0 0 0 >> >> errors: No known data errors >> borg-new /home/ler $ zpool list >> NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH >> ALTROOT >> zroot 10.8T 94.2G 10.7T 16.0E 0% 0% 1.00x ONLINE - >> borg-new /home/ler $ zpool status >> pool: zroot >> state: ONLINE >> scan: scrub repaired 0 in 0h7m with 0 errors on Mon Jan 30 15:43:27 >> 2017 >> config: >> >> NAME STATE READ WRITE CKSUM >> zroot ONLINE 0 0 0 >> raidz1-0 ONLINE 0 0 0 >> mfid4p4 ONLINE 0 0 0 >> mfid0p4 ONLINE 0 0 0 >> mfid1p4 ONLINE 0 0 0 >> mfid3p4 ONLINE 0 0 0 >> mfid2p4 ONLINE 0 0 0 >> mfid5p4 ONLINE 0 0 0 >> >> errors: No known data errors >> borg-new /home/ler $ zpool history >> History for 'zroot': >> cannot show history for pool 'zroot': permission denied >> borg-new /home/ler $ sudo zpool history >> Password: >> History for 'zroot': >> 2017-01-25.19:02:15 zpool create -o altroot=/mnt -O compress=lz4 -O >> atime=off -m none -f zroot raidz1 mfid0p4 mfid1p4 mfid2p4 mfid3p4 >> mfid4p4 mfid5p4 >> 2017-01-25.19:02:15 zfs create -o mountpoint=none zroot/ROOT >> 2017-01-25.19:02:15 zfs create -o mountpoint=/ zroot/ROOT/default >> 2017-01-25.19:02:15 zfs create -o mountpoint=/tmp -o exec=on -o >> setuid=off zroot/tmp >> 2017-01-25.19:02:15 zfs create -o mountpoint=/usr -o canmount=off >> zroot/usr >> 2017-01-25.19:02:15 zfs create zroot/usr/home >> 2017-01-25.19:02:15 zfs create -o setuid=off zroot/usr/ports >> 2017-01-25.19:02:15 zfs create zroot/usr/src >> 2017-01-25.19:02:15 zfs create -o mountpoint=/var -o canmount=off >> zroot/var >> 2017-01-25.19:02:15 zfs create -o exec=off -o setuid=off >> zroot/var/audit >> 2017-01-25.19:02:15 zfs create -o exec=off -o setuid=off >> zroot/var/crash >> 2017-01-25.19:02:15 zfs create -o exec=off -o setuid=off zroot/var/log >> 2017-01-25.19:02:15 zfs create -o atime=on zroot/var/mail >> 2017-01-25.19:02:15 zfs create -o setuid=off zroot/var/tmp >> 2017-01-25.19:02:15 zfs set mountpoint=/zroot zroot >> 2017-01-25.19:02:15 zpool set bootfs=zroot/ROOT/default zroot >> 2017-01-25.19:02:15 zpool set cachefile=/mnt/boot/zfs/zpool.cache >> zroot >> 2017-01-25.19:02:19 zfs set canmount=noauto zroot/ROOT/default >> 2017-01-26.19:51:23 zpool scrub zroot >> 2017-01-29.12:50:56 zpool replace zroot 6435283924725851469 >> /dev/mfid5p4 >> 2017-01-29.12:52:44 zpool scrub zroot >> 2017-01-29.13:11:50 zfs create zroot/usr/obj >> 2017-01-29.13:26:11 zfs create zroot/ler >> 2017-01-29.13:27:58 zfs set mountpoint=/home/ler zroot/ler >> 2017-01-29.16:22:07 zpool replace zroot 8173792548235216166 >> /dev/mfid4p4 >> 2017-01-29.16:27:18 zpool scrub zroot >> 2017-01-29.16:32:15 zfs create -o mountpoint=/home zroot/home >> 2017-01-30.15:36:34 zpool scrub zroot >> >> borg-new /home/ler $ gpart show >> => 40 3905945520 mfid0 GPT (1.8T) >> 40 1600 1 efi (800K) >> 1640 1024 2 freebsd-boot (512K) >> 2664 1432 - free - (716K) >> 4096 16777216 3 freebsd-swap (8.0G) >> 16781312 3889162240 4 freebsd-zfs (1.8T) >> 3905943552 2008 - free - (1.0M) >> >> => 40 3905945520 mfid1 GPT (1.8T) >> 40 1600 1 efi (800K) >> 1640 1024 2 freebsd-boot (512K) >> 2664 1432 - free - (716K) >> 4096 16777216 3 freebsd-swap (8.0G) >> 16781312 3889162240 4 freebsd-zfs (1.8T) >> 3905943552 2008 - free - (1.0M) >> >> => 40 3905945520 mfid2 GPT (1.8T) >> 40 1600 1 efi (800K) >> 1640 1024 2 freebsd-boot (512K) >> 2664 1432 - free - (716K) >> 4096 16777216 3 freebsd-swap (8.0G) >> 16781312 3889162240 4 freebsd-zfs (1.8T) >> 3905943552 2008 - free - (1.0M) >> >> => 40 3905945520 mfid3 GPT (1.8T) >> 40 1600 1 efi (800K) >> 1640 1024 2 freebsd-boot (512K) >> 2664 16777216 3 freebsd-swap (8.0G) >> 16779880 3889165680 4 freebsd-zfs (1.8T) >> >> => 40 3905945520 mfid5 GPT (1.8T) >> 40 1600 1 efi (800K) >> 1640 1024 2 freebsd-boot (512K) >> 2664 1432 - free - (716K) >> 4096 16777216 3 freebsd-swap (8.0G) >> 16781312 3889162240 4 freebsd-zfs (1.8T) >> 3905943552 2008 - free - (1.0M) >> >> => 40 3905945520 mfid4 GPT (1.8T) >> 40 1600 1 efi (800K) >> 1640 1024 2 freebsd-boot (512K) >> 2664 1432 - free - (716K) >> 4096 16777216 3 freebsd-swap (8.0G) >> 16781312 3889160192 4 freebsd-zfs (1.8T) >> 3905941504 4056 - free - (2.0M) >> >> borg-new /home/ler $ >> >> borg-new /home/ler $ uname -aKU >> FreeBSD borg-new 12.0-CURRENT FreeBSD 12.0-CURRENT #1 r312893: Fri Jan >> 27 12:39:05 CST 2017 root@borg-new:/usr/obj/usr/src/sys/GENERIC amd64 >> 1200020 1200020 >> borg-new /home/ler $ >> >> Ideas? >> >> > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" -- Larry Rosenman http://people.freebsd.org/~ler Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 From owner-freebsd-fs@freebsd.org Tue Jan 31 16:56:38 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8DE42CCAA5E for ; Tue, 31 Jan 2017 16:56:38 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wj0-x229.google.com (mail-wj0-x229.google.com [IPv6:2a00:1450:400c:c01::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1D2F516DA for ; Tue, 31 Jan 2017 16:56:37 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wj0-x229.google.com with SMTP id b20so17173711wjs.2 for ; Tue, 31 Jan 2017 08:56:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to; bh=siMIQJm/QxJXUh4jybY+k49DwlTrFJ6vImaMFSjVU58=; b=w3a5Ydt4rezFH71xSHbn2Nk9LTxBBaHCNPB8HEsAVgUeX5Hstqh691qP7n4khJeGQ5 OdAqD0gqkgYfZ8mg5rHyccRv34MaN+1F6LSFRdAR4VhDX4AOiixs2T03gDOiwkMthxCz feRb+kABEdWXU4nSY85PBkWC618ilkgjh+dHSJFKp7DrHkE95pZlOIBLRU+Ttg0qpQow 2vx81hJH1biyw/i0s9esOpvQhFJ3XCDn5gRtETJjLlBhWOqnZMokVapikEXXWTr+4urj O6zGT7PHMVLILVqtiHW/DmpX6Jdj2kKTWf+psed2CEAj8QPM/gzf8h1YIK2XA8yYuysf Uc5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=siMIQJm/QxJXUh4jybY+k49DwlTrFJ6vImaMFSjVU58=; b=PiQLx2jvw2P8haYzUP1s4vBC/jgxIMgEVrCE+VesGB+V5OuowzhpG3Ed/fY5+Rdokt bwgr+m+lbnorJoGLK+z3kImn65Yez6Pjq94mGnmTgfHyz7ihJCKYar3o4W5NjBD3cWyj 3xurPxmf78Zj7o218WudvhrXaqELMGg9+Ar1PVoOjyYJurOKAkBAjL9rKjdEVY9RElsd jlxdVwWGCDY10s5s3Npny25sZUZ+kgQT622MyjkjcUfoZJ1ALlsOUJrG0Eyd0d2AtHGE jaQzFwxINUF8R7i+5MQ8fyv6QZAXMVa6/Aj+DjbPUC7IYkOHcoYo2aLP0wmj/ZvRvcKx ZyGA== X-Gm-Message-State: AIkVDXKSDUFBM4/9APt9M/jFByRM++DJS+rWy1jkmuiyryBCg8dRf7Q0hUtZEXMThgsOvyR9 X-Received: by 10.223.128.77 with SMTP id 71mr24623939wrk.163.1485881796121; Tue, 31 Jan 2017 08:56:36 -0800 (PST) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id 186sm24678511wmw.24.2017.01.31.08.56.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 31 Jan 2017 08:56:35 -0800 (PST) Subject: Re: 16.0E ExpandSize? -- New Server To: Larry Rosenman References: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> Cc: Freebsd fs From: Steven Hartland Message-ID: Date: Tue, 31 Jan 2017 16:56:36 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2017 16:56:38 -0000 What does "zdb" (no params) report? On 31/01/2017 14:51, Larry Rosenman wrote: > borg-new /home/ler $ zpool get expandsize > NAME PROPERTY VALUE SOURCE > zroot expandsize 16.0E - > borg-new /home/ler $ > > On 01/30/2017 5:40 pm, Steven Hartland wrote: >> What do you get from: >> zpool get expandsize >> >> On 30/01/2017 21:47, Larry Rosenman wrote: >>> Greetings, >>> Have a new server with a recent -CURRENT on it. I see an >>> expandsize of 16.0E, which is insane >>> because there are only 6 2TB disks. >>> >>> borg-new /home/ler $ zpool status >>> pool: zroot >>> state: ONLINE >>> scan: scrub repaired 0 in 0h7m with 0 errors on Mon Jan 30 >>> 15:43:27 2017 >>> config: >>> >>> NAME STATE READ WRITE CKSUM >>> zroot ONLINE 0 0 0 >>> raidz1-0 ONLINE 0 0 0 >>> mfid4p4 ONLINE 0 0 0 >>> mfid0p4 ONLINE 0 0 0 >>> mfid1p4 ONLINE 0 0 0 >>> mfid3p4 ONLINE 0 0 0 >>> mfid2p4 ONLINE 0 0 0 >>> mfid5p4 ONLINE 0 0 0 >>> >>> errors: No known data errors >>> borg-new /home/ler $ zpool list >>> NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH >>> ALTROOT >>> zroot 10.8T 94.2G 10.7T 16.0E 0% 0% 1.00x ONLINE - >>> borg-new /home/ler $ zpool status >>> pool: zroot >>> state: ONLINE >>> scan: scrub repaired 0 in 0h7m with 0 errors on Mon Jan 30 >>> 15:43:27 2017 >>> config: >>> >>> NAME STATE READ WRITE CKSUM >>> zroot ONLINE 0 0 0 >>> raidz1-0 ONLINE 0 0 0 >>> mfid4p4 ONLINE 0 0 0 >>> mfid0p4 ONLINE 0 0 0 >>> mfid1p4 ONLINE 0 0 0 >>> mfid3p4 ONLINE 0 0 0 >>> mfid2p4 ONLINE 0 0 0 >>> mfid5p4 ONLINE 0 0 0 >>> >>> errors: No known data errors >>> borg-new /home/ler $ zpool history >>> History for 'zroot': >>> cannot show history for pool 'zroot': permission denied >>> borg-new /home/ler $ sudo zpool history >>> Password: >>> History for 'zroot': >>> 2017-01-25.19:02:15 zpool create -o altroot=/mnt -O compress=lz4 -O >>> atime=off -m none -f zroot raidz1 mfid0p4 mfid1p4 mfid2p4 mfid3p4 >>> mfid4p4 mfid5p4 >>> 2017-01-25.19:02:15 zfs create -o mountpoint=none zroot/ROOT >>> 2017-01-25.19:02:15 zfs create -o mountpoint=/ zroot/ROOT/default >>> 2017-01-25.19:02:15 zfs create -o mountpoint=/tmp -o exec=on -o >>> setuid=off zroot/tmp >>> 2017-01-25.19:02:15 zfs create -o mountpoint=/usr -o canmount=off >>> zroot/usr >>> 2017-01-25.19:02:15 zfs create zroot/usr/home >>> 2017-01-25.19:02:15 zfs create -o setuid=off zroot/usr/ports >>> 2017-01-25.19:02:15 zfs create zroot/usr/src >>> 2017-01-25.19:02:15 zfs create -o mountpoint=/var -o canmount=off >>> zroot/var >>> 2017-01-25.19:02:15 zfs create -o exec=off -o setuid=off >>> zroot/var/audit >>> 2017-01-25.19:02:15 zfs create -o exec=off -o setuid=off >>> zroot/var/crash >>> 2017-01-25.19:02:15 zfs create -o exec=off -o setuid=off zroot/var/log >>> 2017-01-25.19:02:15 zfs create -o atime=on zroot/var/mail >>> 2017-01-25.19:02:15 zfs create -o setuid=off zroot/var/tmp >>> 2017-01-25.19:02:15 zfs set mountpoint=/zroot zroot >>> 2017-01-25.19:02:15 zpool set bootfs=zroot/ROOT/default zroot >>> 2017-01-25.19:02:15 zpool set cachefile=/mnt/boot/zfs/zpool.cache zroot >>> 2017-01-25.19:02:19 zfs set canmount=noauto zroot/ROOT/default >>> 2017-01-26.19:51:23 zpool scrub zroot >>> 2017-01-29.12:50:56 zpool replace zroot 6435283924725851469 >>> /dev/mfid5p4 >>> 2017-01-29.12:52:44 zpool scrub zroot >>> 2017-01-29.13:11:50 zfs create zroot/usr/obj >>> 2017-01-29.13:26:11 zfs create zroot/ler >>> 2017-01-29.13:27:58 zfs set mountpoint=/home/ler zroot/ler >>> 2017-01-29.16:22:07 zpool replace zroot 8173792548235216166 >>> /dev/mfid4p4 >>> 2017-01-29.16:27:18 zpool scrub zroot >>> 2017-01-29.16:32:15 zfs create -o mountpoint=/home zroot/home >>> 2017-01-30.15:36:34 zpool scrub zroot >>> >>> borg-new /home/ler $ gpart show >>> => 40 3905945520 mfid0 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid1 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid2 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid3 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 16777216 3 freebsd-swap (8.0G) >>> 16779880 3889165680 4 freebsd-zfs (1.8T) >>> >>> => 40 3905945520 mfid5 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid4 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889160192 4 freebsd-zfs (1.8T) >>> 3905941504 4056 - free - (2.0M) >>> >>> borg-new /home/ler $ >>> >>> borg-new /home/ler $ uname -aKU >>> FreeBSD borg-new 12.0-CURRENT FreeBSD 12.0-CURRENT #1 r312893: Fri >>> Jan 27 12:39:05 CST 2017 root@borg-new:/usr/obj/usr/src/sys/GENERIC >>> amd64 1200020 1200020 >>> borg-new /home/ler $ >>> >>> Ideas? >>> >>> >> >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Tue Jan 31 17:07:45 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DA809CCAD30 for ; Tue, 31 Jan 2017 17:07:45 +0000 (UTC) (envelope-from ler@FreeBSD.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B60E01C71 for ; Tue, 31 Jan 2017 17:07:45 +0000 (UTC) (envelope-from ler@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date: Content-Type:MIME-Version:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=uh0mtxeLeA/Rvht7vOYwj+UJsJrQD8GNpIxx7/ZiCmk=; b=oTTXrVKstaa0nk5L3z5v3pFPKu qmgdR4OVEnnfQWQRFuHGRqFIh3lWIvbTVM5qqVrXM0tP6ixgxzV2tVYQqveEB0oc7wyk+L5ts3vlV H29AKvF0nSh62vkyyNA/5IoTRaM5DJi1q/2wvmwJoC4Yg8S0K9vYsTxI7z1309g5azCI=; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]:40314 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.88 (FreeBSD)) (envelope-from ) id 1cYbuC-0004Ed-UH; Tue, 31 Jan 2017 11:07:45 -0600 Received: from proxy.na.alcatel-lucent.com ([135.245.48.75]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Tue, 31 Jan 2017 11:07:44 -0600 MIME-Version: 1.0 Date: Tue, 31 Jan 2017 11:07:44 -0600 From: Larry Rosenman To: Steven Hartland Cc: Freebsd fs Subject: Re: 16.0E ExpandSize? -- New Server In-Reply-To: References: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> Message-ID: <35a9034f91542bb1329ac5104bf3b773@FreeBSD.org> X-Sender: ler@FreeBSD.org User-Agent: Roundcube Webmail/1.2.3 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2017 17:07:46 -0000 borg-new /home/ler $ sudo zdb Password: zroot: version: 5000 name: 'zroot' state: 0 txg: 58729 pool_guid: 11945658884309024932 hostid: 3619181042 hostname: 'borg-new' com.delphix:has_per_vdev_zaps vdev_children: 1 vdev_tree: type: 'root' id: 0 guid: 11945658884309024932 create_txg: 4 children[0]: type: 'raidz' id: 0 guid: 7596925654112466913 nparity: 1 metaslab_array: 42 metaslab_shift: 36 ashift: 12 asize: 11947478089728 is_log: 0 create_txg: 4 com.delphix:vdev_zap_top: 35 children[0]: type: 'disk' id: 0 guid: 1443238581175429852 path: '/dev/mfid4p4' whole_disk: 1 DTL: 137 create_txg: 4 com.delphix:vdev_zap_leaf: 131 children[1]: type: 'disk' id: 1 guid: 1865792721003775978 path: '/dev/mfid0p4' whole_disk: 1 DTL: 133 create_txg: 4 com.delphix:vdev_zap_leaf: 37 children[2]: type: 'disk' id: 2 guid: 12541720522827927342 path: '/dev/mfid1p4' whole_disk: 1 DTL: 132 create_txg: 4 com.delphix:vdev_zap_leaf: 38 children[3]: type: 'disk' id: 3 guid: 13053934791777776444 path: '/dev/mfid3p4' whole_disk: 1 DTL: 136 create_txg: 4 com.delphix:vdev_zap_leaf: 135 children[4]: type: 'disk' id: 4 guid: 4432707573898874857 path: '/dev/mfid2p4' whole_disk: 1 DTL: 130 create_txg: 4 com.delphix:vdev_zap_leaf: 40 children[5]: type: 'disk' id: 5 guid: 5106293125005422556 path: '/dev/mfid5p4' whole_disk: 1 DTL: 129 create_txg: 4 com.delphix:vdev_zap_leaf: 41 features_for_read: com.delphix:hole_birth com.delphix:embedded_data borg-new /home/ler $ On 01/31/2017 10:56 am, Steven Hartland wrote: > What does "zdb" (no params) report? > > On 31/01/2017 14:51, Larry Rosenman wrote: borg-new /home/ler $ zpool get expandsize > NAME PROPERTY VALUE SOURCE > zroot expandsize 16.0E - > borg-new /home/ler $ > > On 01/30/2017 5:40 pm, Steven Hartland wrote: > What do you get from: > zpool get expandsize > > On 30/01/2017 21:47, Larry Rosenman wrote: > Greetings, > Have a new server with a recent -CURRENT on it. I see an expandsize of 16.0E, which is insane > because there are only 6 2TB disks. > > borg-new /home/ler $ zpool status > pool: zroot > state: ONLINE > scan: scrub repaired 0 in 0h7m with 0 errors on Mon Jan 30 15:43:27 2017 > config: > > NAME STATE READ WRITE CKSUM > zroot ONLINE 0 0 0 > raidz1-0 ONLINE 0 0 0 > mfid4p4 ONLINE 0 0 0 > mfid0p4 ONLINE 0 0 0 > mfid1p4 ONLINE 0 0 0 > mfid3p4 ONLINE 0 0 0 > mfid2p4 ONLINE 0 0 0 > mfid5p4 ONLINE 0 0 0 > > errors: No known data errors > borg-new /home/ler $ zpool list > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > zroot 10.8T 94.2G 10.7T 16.0E 0% 0% 1.00x ONLINE - > borg-new /home/ler $ zpool status > pool: zroot > state: ONLINE > scan: scrub repaired 0 in 0h7m with 0 errors on Mon Jan 30 15:43:27 2017 > config: > > NAME STATE READ WRITE CKSUM > zroot ONLINE 0 0 0 > raidz1-0 ONLINE 0 0 0 > mfid4p4 ONLINE 0 0 0 > mfid0p4 ONLINE 0 0 0 > mfid1p4 ONLINE 0 0 0 > mfid3p4 ONLINE 0 0 0 > mfid2p4 ONLINE 0 0 0 > mfid5p4 ONLINE 0 0 0 > > errors: No known data errors > borg-new /home/ler $ zpool history > History for 'zroot': > cannot show history for pool 'zroot': permission denied > borg-new /home/ler $ sudo zpool history > Password: > History for 'zroot': > 2017-01-25.19:02:15 zpool create -o altroot=/mnt -O compress=lz4 -O atime=off -m none -f zroot raidz1 mfid0p4 mfid1p4 mfid2p4 mfid3p4 mfid4p4 mfid5p4 > 2017-01-25.19:02:15 zfs create -o mountpoint=none zroot/ROOT > 2017-01-25.19:02:15 zfs create -o mountpoint=/ zroot/ROOT/default > 2017-01-25.19:02:15 zfs create -o mountpoint=/tmp -o exec=on -o setuid=off zroot/tmp > 2017-01-25.19:02:15 zfs create -o mountpoint=/usr -o canmount=off zroot/usr > 2017-01-25.19:02:15 zfs create zroot/usr/home > 2017-01-25.19:02:15 zfs create -o setuid=off zroot/usr/ports > 2017-01-25.19:02:15 zfs create zroot/usr/src > 2017-01-25.19:02:15 zfs create -o mountpoint=/var -o canmount=off zroot/var > 2017-01-25.19:02:15 zfs create -o exec=off -o setuid=off zroot/var/audit > 2017-01-25.19:02:15 zfs create -o exec=off -o setuid=off zroot/var/crash > 2017-01-25.19:02:15 zfs create -o exec=off -o setuid=off zroot/var/log > 2017-01-25.19:02:15 zfs create -o atime=on zroot/var/mail > 2017-01-25.19:02:15 zfs create -o setuid=off zroot/var/tmp > 2017-01-25.19:02:15 zfs set mountpoint=/zroot zroot > 2017-01-25.19:02:15 zpool set bootfs=zroot/ROOT/default zroot > 2017-01-25.19:02:15 zpool set cachefile=/mnt/boot/zfs/zpool.cache zroot > 2017-01-25.19:02:19 zfs set canmount=noauto zroot/ROOT/default > 2017-01-26.19:51:23 zpool scrub zroot > 2017-01-29.12:50:56 zpool replace zroot 6435283924725851469 /dev/mfid5p4 > 2017-01-29.12:52:44 zpool scrub zroot > 2017-01-29.13:11:50 zfs create zroot/usr/obj > 2017-01-29.13:26:11 zfs create zroot/ler > 2017-01-29.13:27:58 zfs set mountpoint=/home/ler zroot/ler > 2017-01-29.16:22:07 zpool replace zroot 8173792548235216166 /dev/mfid4p4 > 2017-01-29.16:27:18 zpool scrub zroot > 2017-01-29.16:32:15 zfs create -o mountpoint=/home zroot/home > 2017-01-30.15:36:34 zpool scrub zroot > > borg-new /home/ler $ gpart show > => 40 3905945520 mfid0 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid1 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid2 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid3 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 16777216 3 freebsd-swap (8.0G) > 16779880 3889165680 4 freebsd-zfs (1.8T) > > => 40 3905945520 mfid5 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid4 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889160192 4 freebsd-zfs (1.8T) > 3905941504 4056 - free - (2.0M) > > borg-new /home/ler $ > > borg-new /home/ler $ uname -aKU > FreeBSD borg-new 12.0-CURRENT FreeBSD 12.0-CURRENT #1 r312893: Fri Jan 27 12:39:05 CST 2017 root@borg-new:/usr/obj/usr/src/sys/GENERIC amd64 1200020 1200020 > borg-new /home/ler $ > > Ideas? > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" -- Larry Rosenman http://people.freebsd.org/~ler Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 From owner-freebsd-fs@freebsd.org Tue Jan 31 17:29:02 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 12EFFCCA282 for ; Tue, 31 Jan 2017 17:29:02 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x22c.google.com (mail-wm0-x22c.google.com [IPv6:2a00:1450:400c:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8D5EE9C1 for ; Tue, 31 Jan 2017 17:29:01 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wm0-x22c.google.com with SMTP id c85so270613799wmi.1 for ; Tue, 31 Jan 2017 09:29:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to; bh=9XpOzGLizjdoflGH2hC4hY/gvPaQ3/cPwG8S9S+TExw=; b=WN1mwig/G6wsdu9H08czBNscQ66OijfQOojH7vCO44PzCJdh3032GVfR0HqFqL7cnw Nf6zb5/zh0y1kfPJi0/iiGYvxK4Nhm/B5pht6+eWbv2qLVs+HgySzGknjjEoy2zz0NjE GFRWsKug5UEn1n1bRKpF59fbqzt8K3jv1MzjDt3CeU2t/2XuB7VaiCPB3qv6AyO5bg/h r1JGSXBlRzZeqa58eTPofQmJTOCe9IOIz8RnCktJ5nBMIRUj5FjFBm/hzOORiRqMpDTn bqnf5PVDA/Gp7zm7yi607y7/+C1xwn61oJ5+CZhB9yyGu8cisQem81MW21tHgfHDky+b D5Bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=9XpOzGLizjdoflGH2hC4hY/gvPaQ3/cPwG8S9S+TExw=; b=dw1gpBJRe4coVbVG7aRVio6fOrF/IU9wHb9B3cYYaFU4RNJYX03llMby15YokNlhSJ ftCbDbi0zG9oDMgyl3NZWe78dH0276NzudFjhDClDq85zwC2iHJvoQGQhKpdvw354sHV MQ9aYzE7UWdqunEohhv4bQf3WsPubFkD9nbew3uPbqw8mzzR1OsuvDxHv6HEUjf9xX9/ 2owFxlbSKJneqYR5983HyW1ldg+EYqNLvC7kgg3z2kXXSAmS59eheVO/1mKAMxKEHfDL j0NmPzEs8YnWDyvkrY8LlEnfftEWTma9Iv36qJ0MSfeY9Lcg5GGUk5GqBQgpgNSQ0B9e nNoQ== X-Gm-Message-State: AIkVDXJrYgBdR4UoqDfOtRjk3+rorb6sGWSwzPxWRLRbN0cdWHMxUJd8I2k6CQHIEQlYI9HV X-Received: by 10.28.169.197 with SMTP id s188mr12276974wme.24.1485883739507; Tue, 31 Jan 2017 09:28:59 -0800 (PST) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id x39sm29470921wrb.3.2017.01.31.09.28.58 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 31 Jan 2017 09:28:58 -0800 (PST) Subject: Re: 16.0E ExpandSize? -- New Server To: Larry Rosenman References: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> <35a9034f91542bb1329ac5104bf3b773@FreeBSD.org> Cc: Freebsd fs From: Steven Hartland Message-ID: <76fc9505-f681-0de0-fe0c-5624b29de321@multiplay.co.uk> Date: Tue, 31 Jan 2017 17:28:59 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: <35a9034f91542bb1329ac5104bf3b773@FreeBSD.org> Content-Type: multipart/mixed; boundary="------------A7B22764671E113789477CB5" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2017 17:29:02 -0000 This is a multi-part message in MIME format. --------------A7B22764671E113789477CB5 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit What does the attached dtrace script output when you run zpool list -v while its running? On 31/01/2017 17:07, Larry Rosenman wrote: > > borg-new /home/ler $ sudo zdb > Password: > zroot: > version: 5000 > name: 'zroot' > state: 0 > txg: 58729 > pool_guid: 11945658884309024932 > hostid: 3619181042 > hostname: 'borg-new' > com.delphix:has_per_vdev_zaps > vdev_children: 1 > vdev_tree: > type: 'root' > id: 0 > guid: 11945658884309024932 > create_txg: 4 > children[0]: > type: 'raidz' > id: 0 > guid: 7596925654112466913 > nparity: 1 > metaslab_array: 42 > metaslab_shift: 36 > ashift: 12 > asize: 11947478089728 > is_log: 0 > create_txg: 4 > com.delphix:vdev_zap_top: 35 > children[0]: > type: 'disk' > id: 0 > guid: 1443238581175429852 > path: '/dev/mfid4p4' > whole_disk: 1 > DTL: 137 > create_txg: 4 > com.delphix:vdev_zap_leaf: 131 > children[1]: > type: 'disk' > id: 1 > guid: 1865792721003775978 > path: '/dev/mfid0p4' > whole_disk: 1 > DTL: 133 > create_txg: 4 > com.delphix:vdev_zap_leaf: 37 > children[2]: > type: 'disk' > id: 2 > guid: 12541720522827927342 > path: '/dev/mfid1p4' > whole_disk: 1 > DTL: 132 > create_txg: 4 > com.delphix:vdev_zap_leaf: 38 > children[3]: > type: 'disk' > id: 3 > guid: 13053934791777776444 > path: '/dev/mfid3p4' > whole_disk: 1 > DTL: 136 > create_txg: 4 > com.delphix:vdev_zap_leaf: 135 > children[4]: > type: 'disk' > id: 4 > guid: 4432707573898874857 > path: '/dev/mfid2p4' > whole_disk: 1 > DTL: 130 > create_txg: 4 > com.delphix:vdev_zap_leaf: 40 > children[5]: > type: 'disk' > id: 5 > guid: 5106293125005422556 > path: '/dev/mfid5p4' > whole_disk: 1 > DTL: 129 > create_txg: 4 > com.delphix:vdev_zap_leaf: 41 > features_for_read: > com.delphix:hole_birth > com.delphix:embedded_data > borg-new /home/ler $ > > > On 01/31/2017 10:56 am, Steven Hartland wrote: > >> What does "zdb" (no params) report? >> >> On 31/01/2017 14:51, Larry Rosenman wrote: >>> borg-new /home/ler $ zpool get expandsize >>> NAME PROPERTY VALUE SOURCE >>> zroot expandsize 16.0E - >>> borg-new /home/ler $ >>> >>> On 01/30/2017 5:40 pm, Steven Hartland wrote: >>>> What do you get from: >>>> zpool get expandsize >>>> >>>> On 30/01/2017 21:47, Larry Rosenman wrote: >>>>> Greetings, >>>>> Have a new server with a recent -CURRENT on it. I see an >>>>> expandsize of 16.0E, which is insane >>>>> because there are only 6 2TB disks. >>>>> >>>>> borg-new /home/ler $ zpool status >>>>> pool: zroot >>>>> state: ONLINE >>>>> scan: scrub repaired 0 in 0h7m with 0 errors on Mon Jan 30 >>>>> 15:43:27 2017 >>>>> config: >>>>> >>>>> NAME STATE READ WRITE CKSUM >>>>> zroot ONLINE 0 0 0 >>>>> raidz1-0 ONLINE 0 0 0 >>>>> mfid4p4 ONLINE 0 0 0 >>>>> mfid0p4 ONLINE 0 0 0 >>>>> mfid1p4 ONLINE 0 0 0 >>>>> mfid3p4 ONLINE 0 0 0 >>>>> mfid2p4 ONLINE 0 0 0 >>>>> mfid5p4 ONLINE 0 0 0 >>>>> >>>>> errors: No known data errors >>>>> borg-new /home/ler $ zpool list >>>>> NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH >>>>> ALTROOT >>>>> zroot 10.8T 94.2G 10.7T 16.0E 0% 0% 1.00x ONLINE - >>>>> borg-new /home/ler $ zpool status >>>>> pool: zroot >>>>> state: ONLINE >>>>> scan: scrub repaired 0 in 0h7m with 0 errors on Mon Jan 30 >>>>> 15:43:27 2017 >>>>> config: >>>>> >>>>> NAME STATE READ WRITE CKSUM >>>>> zroot ONLINE 0 0 0 >>>>> raidz1-0 ONLINE 0 0 0 >>>>> mfid4p4 ONLINE 0 0 0 >>>>> mfid0p4 ONLINE 0 0 0 >>>>> mfid1p4 ONLINE 0 0 0 >>>>> mfid3p4 ONLINE 0 0 0 >>>>> mfid2p4 ONLINE 0 0 0 >>>>> mfid5p4 ONLINE 0 0 0 >>>>> >>>>> errors: No known data errors >>>>> borg-new /home/ler $ zpool history >>>>> History for 'zroot': >>>>> cannot show history for pool 'zroot': permission denied >>>>> borg-new /home/ler $ sudo zpool history >>>>> Password: >>>>> History for 'zroot': >>>>> 2017-01-25.19:02:15 zpool create -o altroot=/mnt -O compress=lz4 >>>>> -O atime=off -m none -f zroot raidz1 mfid0p4 mfid1p4 mfid2p4 >>>>> mfid3p4 mfid4p4 mfid5p4 >>>>> 2017-01-25.19:02:15 zfs create -o mountpoint=none zroot/ROOT >>>>> 2017-01-25.19:02:15 zfs create -o mountpoint=/ zroot/ROOT/default >>>>> 2017-01-25.19:02:15 zfs create -o mountpoint=/tmp -o exec=on -o >>>>> setuid=off zroot/tmp >>>>> 2017-01-25.19:02:15 zfs create -o mountpoint=/usr -o canmount=off >>>>> zroot/usr >>>>> 2017-01-25.19:02:15 zfs create zroot/usr/home >>>>> 2017-01-25.19:02:15 zfs create -o setuid=off zroot/usr/ports >>>>> 2017-01-25.19:02:15 zfs create zroot/usr/src >>>>> 2017-01-25.19:02:15 zfs create -o mountpoint=/var -o canmount=off >>>>> zroot/var >>>>> 2017-01-25.19:02:15 zfs create -o exec=off -o setuid=off >>>>> zroot/var/audit >>>>> 2017-01-25.19:02:15 zfs create -o exec=off -o setuid=off >>>>> zroot/var/crash >>>>> 2017-01-25.19:02:15 zfs create -o exec=off -o setuid=off >>>>> zroot/var/log >>>>> 2017-01-25.19:02:15 zfs create -o atime=on zroot/var/mail >>>>> 2017-01-25.19:02:15 zfs create -o setuid=off zroot/var/tmp >>>>> 2017-01-25.19:02:15 zfs set mountpoint=/zroot zroot >>>>> 2017-01-25.19:02:15 zpool set bootfs=zroot/ROOT/default zroot >>>>> 2017-01-25.19:02:15 zpool set cachefile=/mnt/boot/zfs/zpool.cache >>>>> zroot >>>>> 2017-01-25.19:02:19 zfs set canmount=noauto zroot/ROOT/default >>>>> 2017-01-26.19:51:23 zpool scrub zroot >>>>> 2017-01-29.12:50:56 zpool replace zroot 6435283924725851469 >>>>> /dev/mfid5p4 >>>>> 2017-01-29.12:52:44 zpool scrub zroot >>>>> 2017-01-29.13:11:50 zfs create zroot/usr/obj >>>>> 2017-01-29.13:26:11 zfs create zroot/ler >>>>> 2017-01-29.13:27:58 zfs set mountpoint=/home/ler zroot/ler >>>>> 2017-01-29.16:22:07 zpool replace zroot 8173792548235216166 >>>>> /dev/mfid4p4 >>>>> 2017-01-29.16:27:18 zpool scrub zroot >>>>> 2017-01-29.16:32:15 zfs create -o mountpoint=/home zroot/home >>>>> 2017-01-30.15:36:34 zpool scrub zroot >>>>> >>>>> borg-new /home/ler $ gpart show >>>>> => 40 3905945520 mfid0 GPT (1.8T) >>>>> 40 1600 1 efi (800K) >>>>> 1640 1024 2 freebsd-boot (512K) >>>>> 2664 1432 - free - (716K) >>>>> 4096 16777216 3 freebsd-swap (8.0G) >>>>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>>>> 3905943552 2008 - free - (1.0M) >>>>> >>>>> => 40 3905945520 mfid1 GPT (1.8T) >>>>> 40 1600 1 efi (800K) >>>>> 1640 1024 2 freebsd-boot (512K) >>>>> 2664 1432 - free - (716K) >>>>> 4096 16777216 3 freebsd-swap (8.0G) >>>>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>>>> 3905943552 2008 - free - (1.0M) >>>>> >>>>> => 40 3905945520 mfid2 GPT (1.8T) >>>>> 40 1600 1 efi (800K) >>>>> 1640 1024 2 freebsd-boot (512K) >>>>> 2664 1432 - free - (716K) >>>>> 4096 16777216 3 freebsd-swap (8.0G) >>>>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>>>> 3905943552 2008 - free - (1.0M) >>>>> >>>>> => 40 3905945520 mfid3 GPT (1.8T) >>>>> 40 1600 1 efi (800K) >>>>> 1640 1024 2 freebsd-boot (512K) >>>>> 2664 16777216 3 freebsd-swap (8.0G) >>>>> 16779880 3889165680 4 freebsd-zfs (1.8T) >>>>> >>>>> => 40 3905945520 mfid5 GPT (1.8T) >>>>> 40 1600 1 efi (800K) >>>>> 1640 1024 2 freebsd-boot (512K) >>>>> 2664 1432 - free - (716K) >>>>> 4096 16777216 3 freebsd-swap (8.0G) >>>>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>>>> 3905943552 2008 - free - (1.0M) >>>>> >>>>> => 40 3905945520 mfid4 GPT (1.8T) >>>>> 40 1600 1 efi (800K) >>>>> 1640 1024 2 freebsd-boot (512K) >>>>> 2664 1432 - free - (716K) >>>>> 4096 16777216 3 freebsd-swap (8.0G) >>>>> 16781312 3889160192 4 freebsd-zfs (1.8T) >>>>> 3905941504 4056 - free - (2.0M) >>>>> >>>>> borg-new /home/ler $ >>>>> >>>>> borg-new /home/ler $ uname -aKU >>>>> FreeBSD borg-new 12.0-CURRENT FreeBSD 12.0-CURRENT #1 r312893: Fri >>>>> Jan 27 12:39:05 CST 2017 >>>>> root@borg-new:/usr/obj/usr/src/sys/GENERIC amd64 1200020 1200020 >>>>> borg-new /home/ler $ >>>>> >>>>> Ideas? >>>>> >>>> >>>> _______________________________________________ >>>> freebsd-fs@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > > -- > Larry Rosenman http://people.freebsd.org/~ler > > Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org > > US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 --------------A7B22764671E113789477CB5 Content-Type: text/plain; charset=UTF-8; name="vdev-stats.d" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="vdev-stats.d" IyEvdXNyL3NiaW4vZHRyYWNlIC1zCgojcHJhZ21hIEQgb3B0aW9uIHF1aWV0CgpmYnQ6OnZk ZXZfZ2V0X3N0YXRzOmVudHJ5CnsKCXZkID0gKHZkZXZfdCAqKWFyZzA7CiAgICAgICAgcHJp bnRmKCJ2ZGV2X3BhdGg6ICVzLCB2ZGV2X21heF9hc2l6ZTogJWx1LCB2ZGV2X2FzaXplOiAl bHVcbiIsCgkJdmQtPnZkZXZfcGF0aCA/IHN0cmluZ29mKHZkLT52ZGV2X3BhdGgpIDogIm4v YSIsIHZkLT52ZGV2X21heF9hc2l6ZSwgdmQtPnZkZXZfYXNpemUpOwp9Cg== --------------A7B22764671E113789477CB5-- From owner-freebsd-fs@freebsd.org Tue Jan 31 17:39:15 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 83BAFCCA619 for ; Tue, 31 Jan 2017 17:39:15 +0000 (UTC) (envelope-from ler@FreeBSD.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5E3A8FAF for ; Tue, 31 Jan 2017 17:39:15 +0000 (UTC) (envelope-from ler@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date: Content-Type:MIME-Version:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=XD6grvZ8kxHE+fT+FwIYF20oBQQW1TQt/wnXYNoUgPg=; b=txFVt99XS0xhcxomKFahxS4WHl XzrPMpXLgq7DAOhvU+sOvZnxnhhOSrW3GwJTCOOe/W5ZNaIYqmZ2dxo+7GduJaGQpAZso/hKgsKAc +N2TgBxIzuIEOH8K9IZ06NwE1Vw+BzKnxInZMk6TAZ/G/ljMc6P1kV8j/5hj9mcKUB9Q=; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]:18074 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.88 (FreeBSD)) (envelope-from ) id 1cYcOg-00058j-Ih; Tue, 31 Jan 2017 11:39:14 -0600 Received: from proxy.na.alcatel-lucent.com ([135.245.48.75]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Tue, 31 Jan 2017 11:39:14 -0600 MIME-Version: 1.0 Date: Tue, 31 Jan 2017 11:39:14 -0600 From: Larry Rosenman To: Steven Hartland Cc: Freebsd fs Subject: Re: 16.0E ExpandSize? -- New Server In-Reply-To: <76fc9505-f681-0de0-fe0c-5624b29de321@multiplay.co.uk> References: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> <35a9034f91542bb1329ac5104bf3b773@FreeBSD.org> <76fc9505-f681-0de0-fe0c-5624b29de321@multiplay.co.uk> Message-ID: <22e1bfc5840d972cf93643733682cda1@FreeBSD.org> X-Sender: ler@FreeBSD.org User-Agent: Roundcube Webmail/1.2.3 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2017 17:39:15 -0000 borg-new /home/ler $ sudo ./vdev-stats.d vdev_path: n/a, vdev_max_asize: 0, vdev_asize: 0 vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: 11947478089728 vdev_path: /dev/mfid4p4, vdev_max_asize: 1991245299712, vdev_asize: 1991245299712 vdev_path: /dev/mfid0p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288 vdev_path: /dev/mfid1p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288 vdev_path: /dev/mfid3p4, vdev_max_asize: 1991247921152, vdev_asize: 1991247921152 vdev_path: /dev/mfid2p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288 vdev_path: /dev/mfid5p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288 ^C borg-new /home/ler $ borg-new /home/ler $ zpool list -v NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT zroot 10.8T 94.3G 10.7T 16.0E 0% 0% 1.00x ONLINE - raidz1 10.8T 94.3G 10.7T 16.0E 0% 0% mfid4p4 - - - - - - mfid0p4 - - - - - - mfid1p4 - - - - - - mfid3p4 - - - - - - mfid2p4 - - - - - - mfid5p4 - - - - - - borg-new /home/ler $ On 01/31/2017 11:28 am, Steven Hartland wrote: > What does the attached dtrace script output when you run zpool list -v while its running? > > On 31/01/2017 17:07, Larry Rosenman wrote: > > borg-new /home/ler $ sudo zdb > Password: > zroot: > version: 5000 > name: 'zroot' > state: 0 > txg: 58729 > pool_guid: 11945658884309024932 > hostid: 3619181042 > hostname: 'borg-new' > com.delphix:has_per_vdev_zaps > vdev_children: 1 > vdev_tree: > type: 'root' > id: 0 > guid: 11945658884309024932 > create_txg: 4 > children[0]: > type: 'raidz' > id: 0 > guid: 7596925654112466913 > nparity: 1 > metaslab_array: 42 > metaslab_shift: 36 > ashift: 12 > asize: 11947478089728 > is_log: 0 > create_txg: 4 > com.delphix:vdev_zap_top: 35 > children[0]: > type: 'disk' > id: 0 > guid: 1443238581175429852 > path: '/dev/mfid4p4' > whole_disk: 1 > DTL: 137 > create_txg: 4 > com.delphix:vdev_zap_leaf: 131 > children[1]: > type: 'disk' > id: 1 > guid: 1865792721003775978 > path: '/dev/mfid0p4' > whole_disk: 1 > DTL: 133 > create_txg: 4 > com.delphix:vdev_zap_leaf: 37 > children[2]: > type: 'disk' > id: 2 > guid: 12541720522827927342 > path: '/dev/mfid1p4' > whole_disk: 1 > DTL: 132 > create_txg: 4 > com.delphix:vdev_zap_leaf: 38 > children[3]: > type: 'disk' > id: 3 > guid: 13053934791777776444 > path: '/dev/mfid3p4' > whole_disk: 1 > DTL: 136 > create_txg: 4 > com.delphix:vdev_zap_leaf: 135 > children[4]: > type: 'disk' > id: 4 > guid: 4432707573898874857 > path: '/dev/mfid2p4' > whole_disk: 1 > DTL: 130 > create_txg: 4 > com.delphix:vdev_zap_leaf: 40 > children[5]: > type: 'disk' > id: 5 > guid: 5106293125005422556 > path: '/dev/mfid5p4' > whole_disk: 1 > DTL: 129 > create_txg: 4 > com.delphix:vdev_zap_leaf: 41 > features_for_read: > com.delphix:hole_birth > com.delphix:embedded_data > borg-new /home/ler $ > > On 01/31/2017 10:56 am, Steven Hartland wrote: What does "zdb" (no params) report? > > On 31/01/2017 14:51, Larry Rosenman wrote: borg-new /home/ler $ zpool get expandsize > NAME PROPERTY VALUE SOURCE > zroot expandsize 16.0E - > borg-new /home/ler $ > > On 01/30/2017 5:40 pm, Steven Hartland wrote: > What do you get from: > zpool get expandsize > > On 30/01/2017 21:47, Larry Rosenman wrote: > Greetings, > Have a new server with a recent -CURRENT on it. I see an expandsize of 16.0E, which is insane > because there are only 6 2TB disks. > > borg-new /home/ler $ zpool status > pool: zroot > state: ONLINE > scan: scrub repaired 0 in 0h7m with 0 errors on Mon Jan 30 15:43:27 2017 > config: > > NAME STATE READ WRITE CKSUM > zroot ONLINE 0 0 0 > raidz1-0 ONLINE 0 0 0 > mfid4p4 ONLINE 0 0 0 > mfid0p4 ONLINE 0 0 0 > mfid1p4 ONLINE 0 0 0 > mfid3p4 ONLINE 0 0 0 > mfid2p4 ONLINE 0 0 0 > mfid5p4 ONLINE 0 0 0 > > errors: No known data errors > borg-new /home/ler $ zpool list > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > zroot 10.8T 94.2G 10.7T 16.0E 0% 0% 1.00x ONLINE - > borg-new /home/ler $ zpool status > pool: zroot > state: ONLINE > scan: scrub repaired 0 in 0h7m with 0 errors on Mon Jan 30 15:43:27 2017 > config: > > NAME STATE READ WRITE CKSUM > zroot ONLINE 0 0 0 > raidz1-0 ONLINE 0 0 0 > mfid4p4 ONLINE 0 0 0 > mfid0p4 ONLINE 0 0 0 > mfid1p4 ONLINE 0 0 0 > mfid3p4 ONLINE 0 0 0 > mfid2p4 ONLINE 0 0 0 > mfid5p4 ONLINE 0 0 0 > > errors: No known data errors > borg-new /home/ler $ zpool history > History for 'zroot': > cannot show history for pool 'zroot': permission denied > borg-new /home/ler $ sudo zpool history > Password: > History for 'zroot': > 2017-01-25.19:02:15 zpool create -o altroot=/mnt -O compress=lz4 -O atime=off -m none -f zroot raidz1 mfid0p4 mfid1p4 mfid2p4 mfid3p4 mfid4p4 mfid5p4 > 2017-01-25.19:02:15 zfs create -o mountpoint=none zroot/ROOT > 2017-01-25.19:02:15 zfs create -o mountpoint=/ zroot/ROOT/default > 2017-01-25.19:02:15 zfs create -o mountpoint=/tmp -o exec=on -o setuid=off zroot/tmp > 2017-01-25.19:02:15 zfs create -o mountpoint=/usr -o canmount=off zroot/usr > 2017-01-25.19:02:15 zfs create zroot/usr/home > 2017-01-25.19:02:15 zfs create -o setuid=off zroot/usr/ports > 2017-01-25.19:02:15 zfs create zroot/usr/src > 2017-01-25.19:02:15 zfs create -o mountpoint=/var -o canmount=off zroot/var > 2017-01-25.19:02:15 zfs create -o exec=off -o setuid=off zroot/var/audit > 2017-01-25.19:02:15 zfs create -o exec=off -o setuid=off zroot/var/crash > 2017-01-25.19:02:15 zfs create -o exec=off -o setuid=off zroot/var/log > 2017-01-25.19:02:15 zfs create -o atime=on zroot/var/mail > 2017-01-25.19:02:15 zfs create -o setuid=off zroot/var/tmp > 2017-01-25.19:02:15 zfs set mountpoint=/zroot zroot > 2017-01-25.19:02:15 zpool set bootfs=zroot/ROOT/default zroot > 2017-01-25.19:02:15 zpool set cachefile=/mnt/boot/zfs/zpool.cache zroot > 2017-01-25.19:02:19 zfs set canmount=noauto zroot/ROOT/default > 2017-01-26.19:51:23 zpool scrub zroot > 2017-01-29.12:50:56 zpool replace zroot 6435283924725851469 /dev/mfid5p4 > 2017-01-29.12:52:44 zpool scrub zroot > 2017-01-29.13:11:50 zfs create zroot/usr/obj > 2017-01-29.13:26:11 zfs create zroot/ler > 2017-01-29.13:27:58 zfs set mountpoint=/home/ler zroot/ler > 2017-01-29.16:22:07 zpool replace zroot 8173792548235216166 /dev/mfid4p4 > 2017-01-29.16:27:18 zpool scrub zroot > 2017-01-29.16:32:15 zfs create -o mountpoint=/home zroot/home > 2017-01-30.15:36:34 zpool scrub zroot > > borg-new /home/ler $ gpart show > => 40 3905945520 mfid0 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid1 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid2 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid3 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 16777216 3 freebsd-swap (8.0G) > 16779880 3889165680 4 freebsd-zfs (1.8T) > > => 40 3905945520 mfid5 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid4 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889160192 4 freebsd-zfs (1.8T) > 3905941504 4056 - free - (2.0M) > > borg-new /home/ler $ > > borg-new /home/ler $ uname -aKU > FreeBSD borg-new 12.0-CURRENT FreeBSD 12.0-CURRENT #1 r312893: Fri Jan 27 12:39:05 CST 2017 root@borg-new:/usr/obj/usr/src/sys/GENERIC amd64 1200020 1200020 > borg-new /home/ler $ > > Ideas? > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" -- Larry Rosenman http://people.freebsd.org/~ler [1] Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 -- Larry Rosenman http://people.freebsd.org/~ler Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 Links: ------ [1] http://people.freebsd.org/%7Eler From owner-freebsd-fs@freebsd.org Tue Jan 31 18:30:34 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5E4CCCC9A77 for ; Tue, 31 Jan 2017 18:30:34 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x236.google.com (mail-wm0-x236.google.com [IPv6:2a00:1450:400c:c09::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 13E3F101C for ; Tue, 31 Jan 2017 18:30:33 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wm0-x236.google.com with SMTP id r141so544659wmg.1 for ; Tue, 31 Jan 2017 10:30:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to; bh=+ffx5WybUGxLjd+96UH0FIpyC/mPQEzIu4u/r1FmqkE=; b=oeakjSnqPK7or7ucQfHLMJJa7ICbn5wn3+mncgoORtpg5UkpzCIU1huuha8lqQyUIo 5+wcxXxRISzu8u9ipmqOrMLCzec2VLNK1CvDKZjZB7KG16sU0tlvICEOP66uXvX8Jm2Q 0SO4PwpGa7b9RbvYgkGQA2O2FcqgiT4BDbCiOUQY+53YBd9baEiBMGNicC7HPsXBoFe9 Cd2AUTtKLBlhc867XM0l07oYxAbsVTooMJ2HaE1ysfc4P0V7b1pdGh5aCYpRS3PkXtCJ NDG8XtXYMxr8SUTLmDkbHX0eSjoRRNvEcZ2GFuhMqPKclk+SV3Dq9DZmdIfQyQGp4y8O tykw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=+ffx5WybUGxLjd+96UH0FIpyC/mPQEzIu4u/r1FmqkE=; b=LVu9xTdv9qd6WndL9Y5TenHz7yZb+FnGokqQtwgg9sHwl65tQxE2akxYkxD981yl2s 7toCGNPxAmK/ugAphWynNlbDrmlB+4zsOB3bhDc7SmaV0+lcD7FCLThAh0xdP3jn5NhX oNsHLoEVHi7Fx83drqqm+ADEBr0NBl4wgg6IvjxUoMqAt7JmBOrAJ/TKUH7r9hEwQZsr szlkuHsVMBmijdufyiOviJ/ORxZydW4gZCwwCPhNLShteyd6XtLTnTX2NNLxS9y6X5Yn heLx17qZbt7s1enRD1sLgUxCyCfcHjrQM0lFfH61b2NNc66o9dLDSKjO9OJK5yt84xPH uNIg== X-Gm-Message-State: AIkVDXJYsPYa4IywcJKauJU7T5ej8agecf6q9JUJsx+s3GSfWqcj7kHEHGWIGbvCs+JfI2VF X-Received: by 10.28.58.204 with SMTP id h195mr19587695wma.116.1485887431269; Tue, 31 Jan 2017 10:30:31 -0800 (PST) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id 18sm29701725wrb.14.2017.01.31.10.30.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 31 Jan 2017 10:30:30 -0800 (PST) Subject: Re: 16.0E ExpandSize? -- New Server To: Larry Rosenman References: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> <35a9034f91542bb1329ac5104bf3b773@FreeBSD.org> <76fc9505-f681-0de0-fe0c-5624b29de321@multiplay.co.uk> <22e1bfc5840d972cf93643733682cda1@FreeBSD.org> Cc: Freebsd fs From: Steven Hartland Message-ID: Date: Tue, 31 Jan 2017 18:30:31 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: <22e1bfc5840d972cf93643733682cda1@FreeBSD.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2017 18:30:34 -0000 Your issue is the reported vdev_max_asize > vdev_asize: vdev_max_asize: 11947471798272 vdev_asize: 11947478089728 max asize is smaller than asize by 6291456 For raidz1 Xsize should be the smallest disk Xsize * disks so: 1991245299712 * 6 = 11947471798272 So your max asize looks right but asize looks too big Expand Size is calculated by: if (vd->vdev_aux == NULL && tvd != NULL && vd->vdev_max_asize != 0) { vs->vs_esize = P2ALIGN(vd->vdev_max_asize - vd->vdev_asize, 1ULL << tvd->vdev_ms_shift); } So the question is why is asize too big? Given you seem to have some random disk sizes do you have auto expand turned on? On 31/01/2017 17:39, Larry Rosenman wrote: > vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: 11947478089728 From owner-freebsd-fs@freebsd.org Tue Jan 31 19:05:12 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 48CCBCCA6D4 for ; Tue, 31 Jan 2017 19:05:12 +0000 (UTC) (envelope-from ler@FreeBSD.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 23354AAD for ; Tue, 31 Jan 2017 19:05:12 +0000 (UTC) (envelope-from ler@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date: Content-Type:MIME-Version:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=fuIjALahvk8fKW2VrddoCTm2MD+2RIormkOgHkuwHVU=; b=VUizlxhNFDgT9TqNLMz7LZ/6nF T6as6h9sCbMwV4oWN+L+GUAbLKlVtsjkpDK7dX7rS60F73ARo6lwRKWLUynaVB8J7zsORJF2fXRTm JvwNQaeiymHdnXQq7jX3h2zM9bgKV7vXvjZmYzdKRzpGMJgXf3J0R//BrKg8HV1YggkU=; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]:64954 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.88 (FreeBSD)) (envelope-from ) id 1cYdjr-0009le-5U; Tue, 31 Jan 2017 13:05:11 -0600 Received: from proxy.na.alcatel-lucent.com ([135.245.48.75]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Tue, 31 Jan 2017 13:05:11 -0600 MIME-Version: 1.0 Date: Tue, 31 Jan 2017 13:05:11 -0600 From: Larry Rosenman To: Steven Hartland Cc: Freebsd fs Subject: Re: 16.0E ExpandSize? -- New Server In-Reply-To: References: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> <35a9034f91542bb1329ac5104bf3b773@FreeBSD.org> <76fc9505-f681-0de0-fe0c-5624b29de321@multiplay.co.uk> <22e1bfc5840d972cf93643733682cda1@FreeBSD.org> Message-ID: <8a710dc75c129f58b0372eeaeca575b5@FreeBSD.org> X-Sender: ler@FreeBSD.org User-Agent: Roundcube Webmail/1.2.3 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2017 19:05:12 -0000 I've replaced some disks due to failure, and some of the pariition sizes are different. autoexpand is off: borg-new /home/ler $ zpool get all zroot NAME PROPERTY VALUE SOURCE zroot size 10.8T - zroot capacity 0% - zroot altroot - default zroot health ONLINE - zroot guid 11945658884309024932 default zroot version - default zroot bootfs zroot/ROOT/default local zroot delegation on default zroot autoreplace off default zroot cachefile - default zroot failmode wait default zroot listsnapshots off default zroot autoexpand off default zroot dedupditto 0 default zroot dedupratio 1.00x - zroot free 10.7T - zroot allocated 94.3G - zroot readonly off - zroot comment - default zroot expandsize 16.0E - zroot freeing 0 default zroot fragmentation 0% - zroot leaked 0 default zroot feature@async_destroy enabled local zroot feature@empty_bpobj active local zroot feature@lz4_compress active local zroot feature@multi_vdev_crash_dump enabled local zroot feature@spacemap_histogram active local zroot feature@enabled_txg active local zroot feature@hole_birth active local zroot feature@extensible_dataset enabled local zroot feature@embedded_data active local zroot feature@bookmarks enabled local zroot feature@filesystem_limits enabled local zroot feature@large_blocks enabled local zroot feature@sha512 enabled local zroot feature@skein enabled local borg-new /home/ler $ borg-new /home/ler $ gpart show => 40 3905945520 mfid0 GPT (1.8T) 40 1600 1 efi (800K) 1640 1024 2 freebsd-boot (512K) 2664 1432 - free - (716K) 4096 16777216 3 freebsd-swap (8.0G) 16781312 3889162240 4 freebsd-zfs (1.8T) 3905943552 2008 - free - (1.0M) => 40 3905945520 mfid1 GPT (1.8T) 40 1600 1 efi (800K) 1640 1024 2 freebsd-boot (512K) 2664 1432 - free - (716K) 4096 16777216 3 freebsd-swap (8.0G) 16781312 3889162240 4 freebsd-zfs (1.8T) 3905943552 2008 - free - (1.0M) => 40 3905945520 mfid2 GPT (1.8T) 40 1600 1 efi (800K) 1640 1024 2 freebsd-boot (512K) 2664 1432 - free - (716K) 4096 16777216 3 freebsd-swap (8.0G) 16781312 3889162240 4 freebsd-zfs (1.8T) 3905943552 2008 - free - (1.0M) => 40 3905945520 mfid3 GPT (1.8T) 40 1600 1 efi (800K) 1640 1024 2 freebsd-boot (512K) 2664 16777216 3 freebsd-swap (8.0G) 16779880 3889165680 4 freebsd-zfs (1.8T) => 40 3905945520 mfid5 GPT (1.8T) 40 1600 1 efi (800K) 1640 1024 2 freebsd-boot (512K) 2664 1432 - free - (716K) 4096 16777216 3 freebsd-swap (8.0G) 16781312 3889162240 4 freebsd-zfs (1.8T) 3905943552 2008 - free - (1.0M) => 40 3905945520 mfid4 GPT (1.8T) 40 1600 1 efi (800K) 1640 1024 2 freebsd-boot (512K) 2664 1432 - free - (716K) 4096 16777216 3 freebsd-swap (8.0G) 16781312 3889160192 4 freebsd-zfs (1.8T) 3905941504 4056 - free - (2.0M) borg-new /home/ler $ this system was built last week, and I **CAN** rebuild it if necessary, but I didn't do anything strange (so I thought :) ) On 01/31/2017 12:30 pm, Steven Hartland wrote: > Your issue is the reported vdev_max_asize > vdev_asize: > vdev_max_asize: 11947471798272 > vdev_asize: 11947478089728 > > max asize is smaller than asize by 6291456 > > For raidz1 Xsize should be the smallest disk Xsize * disks so: > 1991245299712 * 6 = 11947471798272 > > So your max asize looks right but asize looks too big > > Expand Size is calculated by: > if (vd->vdev_aux == NULL && tvd != NULL && vd->vdev_max_asize != 0) { > vs->vs_esize = P2ALIGN(vd->vdev_max_asize - vd->vdev_asize, > 1ULL << tvd->vdev_ms_shift); > } > > So the question is why is asize too big? > > Given you seem to have some random disk sizes do you have auto expand turned on? > > On 31/01/2017 17:39, Larry Rosenman wrote: > >> vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: 11947478089728 -- Larry Rosenman http://people.freebsd.org/~ler Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 From owner-freebsd-fs@freebsd.org Tue Jan 31 20:37:16 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A914CCCA888 for ; Tue, 31 Jan 2017 20:37:16 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x22f.google.com (mail-wm0-x22f.google.com [IPv6:2a00:1450:400c:c09::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 558A0EE6 for ; Tue, 31 Jan 2017 20:37:16 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wm0-x22f.google.com with SMTP id 196so19653740wmm.1 for ; Tue, 31 Jan 2017 12:37:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to; bh=7dzmzWdF3F6V4hIEJtOWa4t7QG48jkyis8b5/eTMON4=; b=jN4v/8i2A3yIlPXFFd2TiZxBZPPHUtAhHGM+BHR0/aWAhbRhDYbL2s2jJiAasaMdx8 CextaQHoJS89SxCdCdmWhvM9xfAeNCLEa+mi/UFL46E+5IwaC6EsG0Ymap1sKsIB0P88 UgAuLqQrGOIrQ67cRNDkx3/XMqvySoEGa3i7/DgEVIO1frDystvz+7ZiH+iBAwgj2cSe L2QKTOojnNvlIg7MuTkUqPOkq4WtFBjVdY4NISCc604l+FBCjFdRsmmhO0EI/16wgRtX +6yayYgbfpg81wnjItOe0cIdMKAsubC01cwr5+wGXVC0pciwpfJllnB6o0x2Zl6PZwVt kPSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=7dzmzWdF3F6V4hIEJtOWa4t7QG48jkyis8b5/eTMON4=; b=no1X6v5E3r+H612H9A/Ae5jNOiojykkIZ5vw00S+QT0/mD0TYA8X2vUli9fYw6FWTN edaVKaBkVZwEedbKqcWgO9qTr0KHrzA190JGxxjD2RPa6r8w9EE02oiYB9FyvL6e22TB cGq7jZfJykJ9gSGVuLSfKET9QR6CvSZ0jg/x0s49fobZe5kKlrFfYhBW1RVCXuU1wMbf VrZDjWiXhpvcJR46Qb//8hinrCey59b3YBhzn+T9ntadQM9nNpfe9nAO80Wb0za5SWjF +TodzKlJVy4Cb+gWLLu/JE0d+c2CpsWG3SPysL/7YRQf76fV1l5KxR0n7i3ZilFyxhlf 3ljg== X-Gm-Message-State: AIkVDXJrcO2pKQB7Qv45HHZQ0Iiu9Bsk10CX7x2FAWjvxZ+ie6RiAHNY0qJPYRKu/rbZO407 X-Received: by 10.223.171.12 with SMTP id q12mr5831546wrc.74.1485895034517; Tue, 31 Jan 2017 12:37:14 -0800 (PST) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id l37sm30231742wrc.41.2017.01.31.12.37.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 31 Jan 2017 12:37:13 -0800 (PST) Subject: Re: 16.0E ExpandSize? -- New Server To: Larry Rosenman References: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> <35a9034f91542bb1329ac5104bf3b773@FreeBSD.org> <76fc9505-f681-0de0-fe0c-5624b29de321@multiplay.co.uk> <22e1bfc5840d972cf93643733682cda1@FreeBSD.org> <8a710dc75c129f58b0372eeaeca575b5@FreeBSD.org> Cc: Freebsd fs From: Steven Hartland Message-ID: Date: Tue, 31 Jan 2017 20:37:14 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: <8a710dc75c129f58b0372eeaeca575b5@FreeBSD.org> Content-Type: multipart/mixed; boundary="------------E4FD68EB3E67CE5F2306839B" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2017 20:37:16 -0000 This is a multi-part message in MIME format. --------------E4FD68EB3E67CE5F2306839B Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit In that case based on your zpool history I suspect that the original mfid4p4 was the same size as mfid0p4 (1991246348288) but its been replaced with a drive which is (1991245299712), slightly smaller. This smaller size results in a max_asize of 1991245299712 * 6 instead of original 1991246348288* 6. Now given the way min_asize (the value used to check if the device size is acceptable) is rounded to the the nearest metaslab I believe that replace would be allowed. https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c#L4947 Now the problem is that on open the calculated asize is only updated if its expanding: https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c#L1424 The updated dtrace file outputs vdev_min_asize which should confirm my suspicion about why the replace was allowed. On 31/01/2017 19:05, Larry Rosenman wrote: > > I've replaced some disks due to failure, and some of the pariition > sizes are different. > > > autoexpand is off: > > borg-new /home/ler $ zpool get all zroot > NAME PROPERTY VALUE SOURCE > zroot size 10.8T - > zroot capacity 0% - > zroot altroot - default > zroot health ONLINE - > zroot guid 11945658884309024932 default > zroot version - default > zroot bootfs zroot/ROOT/default local > zroot delegation on default > zroot autoreplace off default > zroot cachefile - default > zroot failmode wait default > zroot listsnapshots off default > zroot autoexpand off default > zroot dedupditto 0 default > zroot dedupratio 1.00x - > zroot free 10.7T - > zroot allocated 94.3G - > zroot readonly off - > zroot comment - default > zroot expandsize 16.0E - > zroot freeing 0 default > zroot fragmentation 0% - > zroot leaked 0 default > zroot feature@async_destroy enabled local > zroot feature@empty_bpobj active local > zroot feature@lz4_compress active local > zroot feature@multi_vdev_crash_dump enabled local > zroot feature@spacemap_histogram active local > zroot feature@enabled_txg active local > zroot feature@hole_birth active local > zroot feature@extensible_dataset enabled local > zroot feature@embedded_data active local > zroot feature@bookmarks enabled local > zroot feature@filesystem_limits enabled local > zroot feature@large_blocks enabled local > zroot feature@sha512 enabled local > zroot feature@skein enabled local > borg-new /home/ler $ > > > borg-new /home/ler $ gpart show > => 40 3905945520 mfid0 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid1 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid2 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid3 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 16777216 3 freebsd-swap (8.0G) > 16779880 3889165680 4 freebsd-zfs (1.8T) > > => 40 3905945520 mfid5 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid4 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889160192 4 freebsd-zfs (1.8T) > 3905941504 4056 - free - (2.0M) > > borg-new /home/ler $ > > > this system was built last week, and I **CAN** rebuild it if > necessary, but I didn't do anything strange (so I thought :) ) > > > > > On 01/31/2017 12:30 pm, Steven Hartland wrote: > >> Your issue is the reported vdev_max_asize > vdev_asize: >> vdev_max_asize: 11947471798272 >> vdev_asize: 11947478089728 >> >> max asize is smaller than asize by 6291456 >> >> For raidz1 Xsize should be the smallest disk Xsize * disks so: >> 1991245299712 * 6 = 11947471798272 >> >> So your max asize looks right but asize looks too big >> >> Expand Size is calculated by: >> if (vd->vdev_aux == NULL && tvd != NULL && vd->vdev_max_asize != 0) { >> vs->vs_esize = P2ALIGN(vd->vdev_max_asize - vd->vdev_asize, >> 1ULL << tvd->vdev_ms_shift); >> } >> >> So the question is why is asize too big? >> >> Given you seem to have some random disk sizes do you have auto expand >> turned on? >> >> On 31/01/2017 17:39, Larry Rosenman wrote: >>> vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: >>> 11947478089728 > > > -- > Larry Rosenman http://people.freebsd.org/~ler > > Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org > > US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 --------------E4FD68EB3E67CE5F2306839B Content-Type: text/plain; charset=UTF-8; name="vdev-stats.d" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="vdev-stats.d" IyEvdXNyL3NiaW4vZHRyYWNlIC1zCgojcHJhZ21hIEQgb3B0aW9uIHF1aWV0CgpmYnQ6OnZk ZXZfZ2V0X3N0YXRzOmVudHJ5CnsKCXZkID0gKHZkZXZfdCAqKWFyZzA7CiAgICAgICAgcHJp bnRmKCJ2ZGV2X3BhdGg6ICVzLCB2ZGV2X21heF9hc2l6ZTogJWx1LCB2ZGV2X2FzaXplOiAl bHUsIHZkZXZfbWluX2FzaXplOiAlbHVcbiIsCgkJdmQtPnZkZXZfcGF0aCA/IHN0cmluZ29m KHZkLT52ZGV2X3BhdGgpIDogIm4vYSIsIHZkLT52ZGV2X21heF9hc2l6ZSwgdmQtPnZkZXZf YXNpemUsIHZkLT52ZGV2X21pbl9hc2l6ZSk7Cn0K --------------E4FD68EB3E67CE5F2306839B-- From owner-freebsd-fs@freebsd.org Tue Jan 31 21:00:36 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5A398CCA03E for ; Tue, 31 Jan 2017 21:00:36 +0000 (UTC) (envelope-from ler@FreeBSD.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 330A51E71 for ; Tue, 31 Jan 2017 21:00:36 +0000 (UTC) (envelope-from ler@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date: Content-Type:MIME-Version:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=bZXERdbP8C5bX4gag+xYtxuZvHjSHg/qSI8bONX9c0M=; b=oiKO9Ga5iLgy/E+SoI1LYOZmHp dqv9uXXdL2yHyHnWZvLlfNHHkq2a75TkLgzKJIy/lqbVdQxaeq5xRPFW426iyjJZ1hFOehTHEAH5u ZkVMxipZqBz7tfpWIqh6wOPcleFSJsRTbXL3yyXs9flWfa7JPFrFwx1BOsOrzz/VLsHY=; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]:49342 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.88 (FreeBSD)) (envelope-from ) id 1cYfXX-000F3O-As; Tue, 31 Jan 2017 15:00:35 -0600 Received: from proxy.na.alcatel-lucent.com ([135.245.48.75]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Tue, 31 Jan 2017 15:00:35 -0600 MIME-Version: 1.0 Date: Tue, 31 Jan 2017 15:00:35 -0600 From: Larry Rosenman To: Steven Hartland Cc: Freebsd fs Subject: Re: 16.0E ExpandSize? -- New Server In-Reply-To: References: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> <35a9034f91542bb1329ac5104bf3b773@FreeBSD.org> <76fc9505-f681-0de0-fe0c-5624b29de321@multiplay.co.uk> <22e1bfc5840d972cf93643733682cda1@FreeBSD.org> <8a710dc75c129f58b0372eeaeca575b5@FreeBSD.org> Message-ID: X-Sender: ler@FreeBSD.org User-Agent: Roundcube Webmail/1.2.3 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2017 21:00:36 -0000 borg-new /home/ler $ sudo ./vdev-stats.d Password: vdev_path: n/a, vdev_max_asize: 0, vdev_asize: 0, vdev_min_asize: 0 vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: 11947478089728, vdev_min_asize: 11888469475328 vdev_path: /dev/mfid4p4, vdev_max_asize: 1991245299712, vdev_asize: 1991245299712, vdev_min_asize: 1981411579221 vdev_path: /dev/mfid0p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 vdev_path: /dev/mfid1p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 vdev_path: /dev/mfid3p4, vdev_max_asize: 1991247921152, vdev_asize: 1991247921152, vdev_min_asize: 1981411579221 vdev_path: /dev/mfid2p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 vdev_path: /dev/mfid5p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 ^C borg-new /home/ler $ borg-new /home/ler $ sudo zpool list -v Password: NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT zroot 10.8T 94.3G 10.7T 16.0E 0% 0% 1.00x ONLINE - raidz1 10.8T 94.3G 10.7T 16.0E 0% 0% mfid4p4 - - - - - - mfid0p4 - - - - - - mfid1p4 - - - - - - mfid3p4 - - - - - - mfid2p4 - - - - - - mfid5p4 - - - - - - borg-new /home/ler $ On 01/31/2017 2:37 pm, Steven Hartland wrote: > In that case based on your zpool history I suspect that the original mfid4p4 was the same size as mfid0p4 (1991246348288) but its been replaced with a drive which is (1991245299712), slightly smaller. > > This smaller size results in a max_asize of 1991245299712 * 6 instead of original 1991246348288* 6. > > Now given the way min_asize (the value used to check if the device size is acceptable) is rounded to the the nearest metaslab I believe that replace would be allowed. > https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c#L4947 > > Now the problem is that on open the calculated asize is only updated if its expanding: > https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c#L1424 > > The updated dtrace file outputs vdev_min_asize which should confirm my suspicion about why the replace was allowed. > > On 31/01/2017 19:05, Larry Rosenman wrote: > > I've replaced some disks due to failure, and some of the pariition sizes are different. > > autoexpand is off: > > borg-new /home/ler $ zpool get all zroot > NAME PROPERTY VALUE SOURCE > zroot size 10.8T - > zroot capacity 0% - > zroot altroot - default > zroot health ONLINE - > zroot guid 11945658884309024932 default > zroot version - default > zroot bootfs zroot/ROOT/default local > zroot delegation on default > zroot autoreplace off default > zroot cachefile - default > zroot failmode wait default > zroot listsnapshots off default > zroot autoexpand off default > zroot dedupditto 0 default > zroot dedupratio 1.00x - > zroot free 10.7T - > zroot allocated 94.3G - > zroot readonly off - > zroot comment - default > zroot expandsize 16.0E - > zroot freeing 0 default > zroot fragmentation 0% - > zroot leaked 0 default > zroot feature@async_destroy enabled local > zroot feature@empty_bpobj active local > zroot feature@lz4_compress active local > zroot feature@multi_vdev_crash_dump enabled local > zroot feature@spacemap_histogram active local > zroot feature@enabled_txg active local > zroot feature@hole_birth active local > zroot feature@extensible_dataset enabled local > zroot feature@embedded_data active local > zroot feature@bookmarks enabled local > zroot feature@filesystem_limits enabled local > zroot feature@large_blocks enabled local > zroot feature@sha512 enabled local > zroot feature@skein enabled local > borg-new /home/ler $ > > borg-new /home/ler $ gpart show > => 40 3905945520 mfid0 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid1 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid2 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid3 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 16777216 3 freebsd-swap (8.0G) > 16779880 3889165680 4 freebsd-zfs (1.8T) > > => 40 3905945520 mfid5 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid4 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889160192 4 freebsd-zfs (1.8T) > 3905941504 4056 - free - (2.0M) > > borg-new /home/ler $ > > this system was built last week, and I **CAN** rebuild it if necessary, but I didn't do anything strange (so I thought :) ) > > On 01/31/2017 12:30 pm, Steven Hartland wrote: Your issue is the reported vdev_max_asize > vdev_asize: > vdev_max_asize: 11947471798272 > vdev_asize: 11947478089728 > > max asize is smaller than asize by 6291456 > > For raidz1 Xsize should be the smallest disk Xsize * disks so: > 1991245299712 * 6 = 11947471798272 > > So your max asize looks right but asize looks too big > > Expand Size is calculated by: > if (vd->vdev_aux == NULL && tvd != NULL && vd->vdev_max_asize != 0) { > vs->vs_esize = P2ALIGN(vd->vdev_max_asize - vd->vdev_asize, > 1ULL << tvd->vdev_ms_shift); > } > > So the question is why is asize too big? > > Given you seem to have some random disk sizes do you have auto expand turned on? > > On 31/01/2017 17:39, Larry Rosenman wrote: vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: 11947478089728 -- Larry Rosenman http://people.freebsd.org/~ler [1] Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 -- Larry Rosenman http://people.freebsd.org/~ler Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 Links: ------ [1] http://people.freebsd.org/%7Eler From owner-freebsd-fs@freebsd.org Tue Jan 31 21:17:24 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 95571CCA5D3 for ; Tue, 31 Jan 2017 21:17:24 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x22c.google.com (mail-wm0-x22c.google.com [IPv6:2a00:1450:400c:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 18B90AD4 for ; Tue, 31 Jan 2017 21:17:24 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wm0-x22c.google.com with SMTP id r141so7704291wmg.1 for ; Tue, 31 Jan 2017 13:17:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to; bh=5JgueV3Cay2BhZSBrs1Ttj9gfCu5UekH7v7IRwwDkqo=; b=x+1vuf7Faju3/Pl0vCsJgucA/YlYb8uQwFX9mH1SKLhPks9F+THbefz1ip26zAE0Qp RHORgeYsT8TkseXWa5yfc9SVSqp0AJB5Zmx5AHIKRS1/dE/zMjHBkjsZ3q90zJFzWeOd +PtIe3l0P0bpK24ZapWBwS/mWGdzhwGUsfvt5trIpweCKytjl8de3Y2QCui7BZuEdZkl rMcLoPv7xvjgwogmD/qCZKhTgmsADhtmmfLoY20Hi/mxvGe51p0FX/quqtQV8Hiatdkc WUKoeoa/v/QSc1tfJEiS+sgy+t+J7IuwPyOFHGJ3dx+HF9hjCRM1NO5ToCw3ewB5stkQ kf8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=5JgueV3Cay2BhZSBrs1Ttj9gfCu5UekH7v7IRwwDkqo=; b=rjhgaqymUL1NluyNidYQaFI68enXLBQqdTkMPVsTWjKNbFcuJiiJGcZG01Hw9MUES2 Tq04it+QwrrHhsJ9g5LXZuaK4XVc3yq99DDgmEgWtozspA/hi2Hj5ZOz2eUw60XbILkH xL2RDQS3iKKFo4xGYMSlUA+bd0H4XDPne/FAw/cudxANUyjlSM4jM/BxA0djMZsJlIXD 0uzRyLJKcvwCgYoXFVUULJNIFVHZIrRQ1mNc+W//mE8ySyQNLU3vx2qq2JVpmRHE7yHh bwKlhr2F8mNDrU2fKDpmGQHSP6Qml1Yilkwj1+V9GF72zQ9/55tMW3fJQneuxz5ZB8x/ OivQ== X-Gm-Message-State: AIkVDXJMz0w7D7LY6Gnqhh7tmpFFLKYd72eJchZaTBpym/LBkfAyEzdeMzjoH6G7WXRSES4Q X-Received: by 10.223.169.140 with SMTP id b12mr25389695wrd.138.1485897441731; Tue, 31 Jan 2017 13:17:21 -0800 (PST) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id o143sm25949515wmd.3.2017.01.31.13.17.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 31 Jan 2017 13:17:20 -0800 (PST) Subject: Re: 16.0E ExpandSize? -- New Server To: Larry Rosenman References: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> <35a9034f91542bb1329ac5104bf3b773@FreeBSD.org> <76fc9505-f681-0de0-fe0c-5624b29de321@multiplay.co.uk> <22e1bfc5840d972cf93643733682cda1@FreeBSD.org> <8a710dc75c129f58b0372eeaeca575b5@FreeBSD.org> Cc: Freebsd fs From: Steven Hartland Message-ID: Date: Tue, 31 Jan 2017 21:17:21 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/mixed; boundary="------------612F8B147C213AC6D41C97A7" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2017 21:17:24 -0000 This is a multi-part message in MIME format. --------------612F8B147C213AC6D41C97A7 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Ok so that confirms it, try the attached patch (only a new kernel is needed) on a read only import of the pool and see if that fixes it. Regards Steve On 31/01/2017 21:00, Larry Rosenman wrote: > > borg-new /home/ler $ sudo ./vdev-stats.d > Password: > vdev_path: n/a, vdev_max_asize: 0, vdev_asize: 0, vdev_min_asize: 0 > vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: > 11947478089728, vdev_min_asize: 11888469475328 > vdev_path: /dev/mfid4p4, vdev_max_asize: 1991245299712, vdev_asize: > 1991245299712, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid0p4, vdev_max_asize: 1991246348288, vdev_asize: > 1991246348288, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid1p4, vdev_max_asize: 1991246348288, vdev_asize: > 1991246348288, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid3p4, vdev_max_asize: 1991247921152, vdev_asize: > 1991247921152, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid2p4, vdev_max_asize: 1991246348288, vdev_asize: > 1991246348288, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid5p4, vdev_max_asize: 1991246348288, vdev_asize: > 1991246348288, vdev_min_asize: 1981411579221 > ^C > > borg-new /home/ler $ > > > borg-new /home/ler $ sudo zpool list -v > Password: > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > zroot 10.8T 94.3G 10.7T 16.0E 0% 0% 1.00x ONLINE - > raidz1 10.8T 94.3G 10.7T 16.0E 0% 0% > mfid4p4 - - - - - - > mfid0p4 - - - - - - > mfid1p4 - - - - - - > mfid3p4 - - - - - - > mfid2p4 - - - - - - > mfid5p4 - - - - - - > borg-new /home/ler $ > > > On 01/31/2017 2:37 pm, Steven Hartland wrote: > >> In that case based on your zpool history I suspect that the original >> mfid4p4 was the same size as mfid0p4 (1991246348288) but its been >> replaced with a drive which is (1991245299712), slightly smaller. >> >> This smaller size results in a max_asize of 1991245299712 * 6 instead >> of original 1991246348288* 6. >> >> Now given the way min_asize (the value used to check if the device >> size is acceptable) is rounded to the the nearest metaslab I believe >> that replace would be allowed. >> https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c#L4947 >> >> Now the problem is that on open the calculated asize is only updated >> if its expanding: >> https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c#L1424 >> >> The updated dtrace file outputs vdev_min_asize which should confirm >> my suspicion about why the replace was allowed. >> >> On 31/01/2017 19:05, Larry Rosenman wrote: >>> >>> I've replaced some disks due to failure, and some of the pariition >>> sizes are different. >>> >>> >>> autoexpand is off: >>> >>> borg-new /home/ler $ zpool get all zroot >>> NAME PROPERTY VALUE SOURCE >>> zroot size 10.8T - >>> zroot capacity 0% - >>> zroot altroot - default >>> zroot health ONLINE - >>> zroot guid 11945658884309024932 default >>> zroot version - default >>> zroot bootfs zroot/ROOT/default local >>> zroot delegation on default >>> zroot autoreplace off default >>> zroot cachefile - default >>> zroot failmode wait default >>> zroot listsnapshots off default >>> zroot autoexpand off default >>> zroot dedupditto 0 default >>> zroot dedupratio 1.00x - >>> zroot free 10.7T - >>> zroot allocated 94.3G - >>> zroot readonly off - >>> zroot comment - default >>> zroot expandsize 16.0E - >>> zroot freeing 0 default >>> zroot fragmentation 0% - >>> zroot leaked 0 default >>> zroot feature@async_destroy enabled local >>> zroot feature@empty_bpobj active local >>> zroot feature@lz4_compress active local >>> zroot feature@multi_vdev_crash_dump enabled local >>> zroot feature@spacemap_histogram active local >>> zroot feature@enabled_txg active local >>> zroot feature@hole_birth active local >>> zroot feature@extensible_dataset enabled local >>> zroot feature@embedded_data active local >>> zroot feature@bookmarks enabled local >>> zroot feature@filesystem_limits enabled local >>> zroot feature@large_blocks enabled local >>> zroot feature@sha512 enabled local >>> zroot feature@skein enabled local >>> borg-new /home/ler $ >>> >>> >>> borg-new /home/ler $ gpart show >>> => 40 3905945520 mfid0 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid1 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid2 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid3 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 16777216 3 freebsd-swap (8.0G) >>> 16779880 3889165680 4 freebsd-zfs (1.8T) >>> >>> => 40 3905945520 mfid5 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid4 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889160192 4 freebsd-zfs (1.8T) >>> 3905941504 4056 - free - (2.0M) >>> >>> borg-new /home/ler $ >>> >>> >>> this system was built last week, and I **CAN** rebuild it if >>> necessary, but I didn't do anything strange (so I thought :) ) >>> >>> >>> >>> >>> On 01/31/2017 12:30 pm, Steven Hartland wrote: >>> >>> Your issue is the reported vdev_max_asize > vdev_asize: >>> vdev_max_asize: 11947471798272 >>> vdev_asize: 11947478089728 >>> >>> max asize is smaller than asize by 6291456 >>> >>> For raidz1 Xsize should be the smallest disk Xsize * disks so: >>> 1991245299712 * 6 = 11947471798272 >>> >>> So your max asize looks right but asize looks too big >>> >>> Expand Size is calculated by: >>> if (vd->vdev_aux == NULL && tvd != NULL && vd->vdev_max_asize != >>> 0) { >>> vs->vs_esize = P2ALIGN(vd->vdev_max_asize - vd->vdev_asize, >>> 1ULL << tvd->vdev_ms_shift); >>> } >>> >>> So the question is why is asize too big? >>> >>> Given you seem to have some random disk sizes do you have auto >>> expand turned on? >>> >>> On 31/01/2017 17:39, Larry Rosenman wrote: >>> >>> vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: >>> 11947478089728 >>> >>> >>> -- >>> Larry Rosenman http://people.freebsd.org/~ler >>> >>> Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org >>> >>> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 > > > -- > Larry Rosenman http://people.freebsd.org/~ler > > Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org > > US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 --------------612F8B147C213AC6D41C97A7 Content-Type: text/plain; charset=UTF-8; name="expand-sz-16e.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="expand-sz-16e.patch" SW5kZXg6IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMv dmRldi5jCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT0KLS0tIHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMv dXRzL2NvbW1vbi9mcy96ZnMvdmRldi5jCShyZXZpc2lvbiAzMTMwMDMpCisrKyBzeXMvY2Rk bC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3ZkZXYuYwkod29ya2lu ZyBjb3B5KQpAQCAtMTM3Nyw3ICsxMzc3LDcgQEAKIAl2ZC0+dmRldl9wc2l6ZSA9IHBzaXpl OwogCiAJLyoKLQkgKiBNYWtlIHN1cmUgdGhlIGFsbG9jYXRhYmxlIHNpemUgaGFzbid0IHNo cnVuay4KKwkgKiBNYWtlIHN1cmUgdGhlIGFsbG9jYXRhYmxlIHNpemUgaGFzbid0IHNocnVu ayB0b28gbXVjaC4KIAkgKi8KIAlpZiAoYXNpemUgPCB2ZC0+dmRldl9taW5fYXNpemUpIHsK IAkJdmRldl9zZXRfc3RhdGUodmQsIEJfVFJVRSwgVkRFVl9TVEFURV9DQU5UX09QRU4sCkBA IC0xNDIwLDEwICsxNDIwLDE5IEBACiAJICogSWYgYWxsIGNoaWxkcmVuIGFyZSBoZWFsdGh5 IGFuZCB0aGUgYXNpemUgaGFzIGluY3JlYXNlZCwKIAkgKiB0aGVuIHdlJ3ZlIGV4cGVyaWVu Y2VkIGR5bmFtaWMgTFVOIGdyb3d0aC4gIElmIGF1dG9tYXRpYwogCSAqIGV4cGFuc2lvbiBp cyBlbmFibGVkIHRoZW4gdXNlIHRoZSBhZGRpdGlvbmFsIHNwYWNlLgorCSAqIAorCSAqIE90 aGVyd2lzZSBpZiBhc2l6ZSBoYXMgcmVkdWNlZCwgc2hyaW5rIHRvIGVuc3VyZSB0aGF0CisJ ICogY2FsY3VsYXRpb25zIGJhc2VkIG9mIG1heF9hc2l6ZSBhbmQgYXNpemUgZS5nLiBlc2l6 ZSBhcmUKKwkgKiBhbHdheXMgdmFsaWQuIFRoaXMgaXMgc2FmZSBhcyB3ZSd2ZSBhbHJlYWR5 IHZhbGlkYXRlZCB0aGF0CisJICogYXNpemUgaXMgbm90IGxlc3MgdGhhbiBtaW5fYXNpemUu CiAJICovCi0JaWYgKHZkLT52ZGV2X3N0YXRlID09IFZERVZfU1RBVEVfSEVBTFRIWSAmJiBh c2l6ZSA+IHZkLT52ZGV2X2FzaXplICYmCi0JICAgICh2ZC0+dmRldl9leHBhbmRpbmcgfHwg c3BhLT5zcGFfYXV0b2V4cGFuZCkpCi0JCXZkLT52ZGV2X2FzaXplID0gYXNpemU7CisJaWYg KHZkLT52ZGV2X3N0YXRlID09IFZERVZfU1RBVEVfSEVBTFRIWSkgeworCQlpZiAoYXNpemUg PiB2ZC0+dmRldl9hc2l6ZSAmJgorCQkgICAgKHZkLT52ZGV2X2V4cGFuZGluZyB8fCBzcGEt PnNwYV9hdXRvZXhwYW5kKSkKKwkJCXZkLT52ZGV2X2FzaXplID0gYXNpemU7CisJCWVsc2Ug aWYgKGFzaXplIDwgdmQtPnZkZXZfYXNpemUpCisJCQl2ZC0+dmRldl9hc2l6ZSA9IGFzaXpl OworCX0KIAogCXZkZXZfc2V0X21pbl9hc2l6ZSh2ZCk7CiAK --------------612F8B147C213AC6D41C97A7-- From owner-freebsd-fs@freebsd.org Tue Jan 31 21:40:54 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B859DCCAF8E for ; Tue, 31 Jan 2017 21:40:54 +0000 (UTC) (envelope-from ler@FreeBSD.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 895D71755 for ; Tue, 31 Jan 2017 21:40:54 +0000 (UTC) (envelope-from ler@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date: Content-Type:MIME-Version:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=dCVd5CuL5uJKXzalGkc9gkgByc5vqHj96530p/QFa/M=; b=K4LvjoAMJxjwkgeb7Sy3hsTsID fX+tJmUXlr+IIcLUdQhn9NKMzIwPwuajYPrvWBGWi9gSkBo2ydXL+NAVzEce0kUjUnitxf1ay09dz 8A+BhnD2zQXic5WEB4R7LRLuIHRX5LSN3DSRj0lwmLd2EGVDvlbyeJ4FJ/VxW+DMIwYg=; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]:39374 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.88 (FreeBSD)) (envelope-from ) id 1cYgAX-000GIN-DL; Tue, 31 Jan 2017 15:40:53 -0600 Received: from proxy.na.alcatel-lucent.com ([135.245.48.75]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Tue, 31 Jan 2017 15:40:53 -0600 MIME-Version: 1.0 Date: Tue, 31 Jan 2017 15:40:53 -0600 From: Larry Rosenman To: Steven Hartland Cc: Freebsd fs Subject: Re: 16.0E ExpandSize? -- New Server In-Reply-To: References: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> <35a9034f91542bb1329ac5104bf3b773@FreeBSD.org> <76fc9505-f681-0de0-fe0c-5624b29de321@multiplay.co.uk> <22e1bfc5840d972cf93643733682cda1@FreeBSD.org> <8a710dc75c129f58b0372eeaeca575b5@FreeBSD.org> Message-ID: <530ac81cbcdf268919a887f3ceff41d8@FreeBSD.org> X-Sender: ler@FreeBSD.org User-Agent: Roundcube Webmail/1.2.3 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2017 21:40:54 -0000 perfect: borg-new /home/ler $ zpool list NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT zroot 10.8T 94.3G 10.7T - 0% 0% 1.00x ONLINE - borg-new /home/ler $ zpool get all NAME PROPERTY VALUE SOURCE zroot size 10.8T - zroot capacity 0% - zroot altroot - default zroot health ONLINE - zroot guid 11945658884309024932 default zroot version - default zroot bootfs zroot/ROOT/default local zroot delegation on default zroot autoreplace off default zroot cachefile - default zroot failmode wait default zroot listsnapshots off default zroot autoexpand off default zroot dedupditto 0 default zroot dedupratio 1.00x - zroot free 10.7T - zroot allocated 94.3G - zroot readonly off - zroot comment - default zroot expandsize - - zroot freeing 0 default zroot fragmentation 0% - zroot leaked 0 default zroot feature@async_destroy enabled local zroot feature@empty_bpobj active local zroot feature@lz4_compress active local zroot feature@multi_vdev_crash_dump enabled local zroot feature@spacemap_histogram active local zroot feature@enabled_txg active local zroot feature@hole_birth active local zroot feature@extensible_dataset enabled local zroot feature@embedded_data active local zroot feature@bookmarks enabled local zroot feature@filesystem_limits enabled local zroot feature@large_blocks enabled local zroot feature@sha512 enabled local zroot feature@skein enabled local borg-new /home/ler $ On 01/31/2017 3:17 pm, Steven Hartland wrote: > Ok so that confirms it, try the attached patch (only a new kernel is needed) on a read only import of the pool and see if that fixes it. > > Regards > Steve > > On 31/01/2017 21:00, Larry Rosenman wrote: > > borg-new /home/ler $ sudo ./vdev-stats.d > Password: > vdev_path: n/a, vdev_max_asize: 0, vdev_asize: 0, vdev_min_asize: 0 > vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: 11947478089728, vdev_min_asize: 11888469475328 > vdev_path: /dev/mfid4p4, vdev_max_asize: 1991245299712, vdev_asize: 1991245299712, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid0p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid1p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid3p4, vdev_max_asize: 1991247921152, vdev_asize: 1991247921152, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid2p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid5p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 > ^C > > borg-new /home/ler $ > > borg-new /home/ler $ sudo zpool list -v > Password: > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > zroot 10.8T 94.3G 10.7T 16.0E 0% 0% 1.00x ONLINE - > raidz1 10.8T 94.3G 10.7T 16.0E 0% 0% > mfid4p4 - - - - - - > mfid0p4 - - - - - - > mfid1p4 - - - - - - > mfid3p4 - - - - - - > mfid2p4 - - - - - - > mfid5p4 - - - - - - > borg-new /home/ler $ > > On 01/31/2017 2:37 pm, Steven Hartland wrote: In that case based on your zpool history I suspect that the original mfid4p4 was the same size as mfid0p4 (1991246348288) but its been replaced with a drive which is (1991245299712), slightly smaller. > > This smaller size results in a max_asize of 1991245299712 * 6 instead of original 1991246348288* 6. > > Now given the way min_asize (the value used to check if the device size is acceptable) is rounded to the the nearest metaslab I believe that replace would be allowed. > https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c#L4947 > > Now the problem is that on open the calculated asize is only updated if its expanding: > https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c#L1424 > > The updated dtrace file outputs vdev_min_asize which should confirm my suspicion about why the replace was allowed. > > On 31/01/2017 19:05, Larry Rosenman wrote: > > I've replaced some disks due to failure, and some of the pariition sizes are different. > > autoexpand is off: > > borg-new /home/ler $ zpool get all zroot > NAME PROPERTY VALUE SOURCE > zroot size 10.8T - > zroot capacity 0% - > zroot altroot - default > zroot health ONLINE - > zroot guid 11945658884309024932 default > zroot version - default > zroot bootfs zroot/ROOT/default local > zroot delegation on default > zroot autoreplace off default > zroot cachefile - default > zroot failmode wait default > zroot listsnapshots off default > zroot autoexpand off default > zroot dedupditto 0 default > zroot dedupratio 1.00x - > zroot free 10.7T - > zroot allocated 94.3G - > zroot readonly off - > zroot comment - default > zroot expandsize 16.0E - > zroot freeing 0 default > zroot fragmentation 0% - > zroot leaked 0 default > zroot feature@async_destroy enabled local > zroot feature@empty_bpobj active local > zroot feature@lz4_compress active local > zroot feature@multi_vdev_crash_dump enabled local > zroot feature@spacemap_histogram active local > zroot feature@enabled_txg active local > zroot feature@hole_birth active local > zroot feature@extensible_dataset enabled local > zroot feature@embedded_data active local > zroot feature@bookmarks enabled local > zroot feature@filesystem_limits enabled local > zroot feature@large_blocks enabled local > zroot feature@sha512 enabled local > zroot feature@skein enabled local > borg-new /home/ler $ > > borg-new /home/ler $ gpart show > => 40 3905945520 mfid0 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid1 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid2 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid3 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 16777216 3 freebsd-swap (8.0G) > 16779880 3889165680 4 freebsd-zfs (1.8T) > > => 40 3905945520 mfid5 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid4 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889160192 4 freebsd-zfs (1.8T) > 3905941504 4056 - free - (2.0M) > > borg-new /home/ler $ > > this system was built last week, and I **CAN** rebuild it if necessary, but I didn't do anything strange (so I thought :) ) > > On 01/31/2017 12:30 pm, Steven Hartland wrote: Your issue is the reported vdev_max_asize > vdev_asize: > vdev_max_asize: 11947471798272 > vdev_asize: 11947478089728 > > max asize is smaller than asize by 6291456 > > For raidz1 Xsize should be the smallest disk Xsize * disks so: > 1991245299712 * 6 = 11947471798272 > > So your max asize looks right but asize looks too big > > Expand Size is calculated by: > if (vd->vdev_aux == NULL && tvd != NULL && vd->vdev_max_asize != 0) { > vs->vs_esize = P2ALIGN(vd->vdev_max_asize - vd->vdev_asize, > 1ULL << tvd->vdev_ms_shift); > } > > So the question is why is asize too big? > > Given you seem to have some random disk sizes do you have auto expand turned on? > > On 31/01/2017 17:39, Larry Rosenman wrote: vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: 11947478089728 -- Larry Rosenman http://people.freebsd.org/~ler [1] Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 -- Larry Rosenman http://people.freebsd.org/~ler [1] Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 -- Larry Rosenman http://people.freebsd.org/~ler Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 Links: ------ [1] http://people.freebsd.org/%7Eler From owner-freebsd-fs@freebsd.org Tue Jan 31 21:47:16 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BF579CCA2DD for ; Tue, 31 Jan 2017 21:47:16 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x22c.google.com (mail-wm0-x22c.google.com [IPv6:2a00:1450:400c:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4052C1C42 for ; Tue, 31 Jan 2017 21:47:16 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wm0-x22c.google.com with SMTP id r141so8851877wmg.1 for ; Tue, 31 Jan 2017 13:47:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to; bh=KtjSVuxS/F1T0pgh6geYC6EQ6w0GPJDmu58DLtkB/kQ=; b=0k8FWkX3dwK47uQ1Ce151DXSy2jlCWuk7dxd/+dWFkqwqvOBM0Y4OLTBu1AcyhxbvQ RC3PEgG5/+/QRz10UOnj6JclnRl2ZuxT8v4QPCLnx2raVgKVsYgHyUj3eILOOBSVRj9U HqWAinrP1fZ4CYhzFGfJrEtrRSrVbsdy2uFbAUdkKVyIN92hpsaFrmVjTYQRksyp31Ne yyaM7tqdvGy/DE56jav3P78SxRkGqTYIv5CC6tZ5MDUZrLnrHg+Z9USD3P/BgsNXYZPs 29hAOqcNhPGcvF3ijLcCAj7XK5pJXBtd810uuI6sEhdvVFmUEI3usw5M475PsXT4sqtF TJcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=KtjSVuxS/F1T0pgh6geYC6EQ6w0GPJDmu58DLtkB/kQ=; b=f/NJBxge313Ky60u4UgieU4dRPeKktj+BggOsON8keh7RLEGlxLIB6NvXTL0QOzL0T nzOg2Ky7dK9KSjyPNDEUfCicAZNfPyxcQvY+iikZMQ+AlK+SasLHwM0xuTGEIl0wQN5h ThJZY7UW+SHxSGJhH98ksMk6GmiFLbai9TpbCvGjBGAMoqPluJQQodbnekfPprfqlLFt Cq4EuUXKY/saORLDLeH6fbh3pY2I9dyw9xRX57HSZKooTAZMydz0yC+wVMdARafiA00P QaJXMnhjRbFA4ph2fnxtReYloRggCql/bcAnV1lirtn7VhN//sM5yHi2FLLvGndj6VfW LHDA== X-Gm-Message-State: AIkVDXIUnai42DpNV7bfMDxEDa02u2p499B/RWv6RydqqNvAg8wLxIGgPsrqhwNul/dw+O8H X-Received: by 10.223.171.149 with SMTP id s21mr25145187wrc.64.1485899234139; Tue, 31 Jan 2017 13:47:14 -0800 (PST) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id x39sm30518733wrb.3.2017.01.31.13.47.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 31 Jan 2017 13:47:13 -0800 (PST) Subject: Re: 16.0E ExpandSize? -- New Server To: Larry Rosenman References: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> <35a9034f91542bb1329ac5104bf3b773@FreeBSD.org> <76fc9505-f681-0de0-fe0c-5624b29de321@multiplay.co.uk> <22e1bfc5840d972cf93643733682cda1@FreeBSD.org> <8a710dc75c129f58b0372eeaeca575b5@FreeBSD.org> Cc: Freebsd fs From: Steven Hartland Message-ID: Date: Tue, 31 Jan 2017 21:47:14 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/mixed; boundary="------------3F69C0E438D93AD11208EDA7" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2017 21:47:16 -0000 This is a multi-part message in MIME format. --------------3F69C0E438D93AD11208EDA7 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Hmm, looks like there's also a bug in the way vdev_min_asize is calculated for raidz as it can and has resulted in child min_asize which won't provided enough space for the parent due to the use of unrounded integer division. 1981411579221 * 6 = 11888469475326 < 11888469475328 You should have vdev_min_asize: 1981411579222 for your children. Updated patch attached, however calculation still isn't 100% reversible so may need work, however it does now ensure that the children will provide enough capacity for min_asize even if all of them are shrunk to their individual min_asize, which I believe previously may not have been the case. This isn't related to the incorrect EXPANDSZ output, but would be good if you could confirm it doesn't cause any issues for your pool given its state. On 31/01/2017 21:00, Larry Rosenman wrote: > > borg-new /home/ler $ sudo ./vdev-stats.d > Password: > vdev_path: n/a, vdev_max_asize: 0, vdev_asize: 0, vdev_min_asize: 0 > vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: > 11947478089728, vdev_min_asize: 11888469475328 > vdev_path: /dev/mfid4p4, vdev_max_asize: 1991245299712, vdev_asize: > 1991245299712, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid0p4, vdev_max_asize: 1991246348288, vdev_asize: > 1991246348288, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid1p4, vdev_max_asize: 1991246348288, vdev_asize: > 1991246348288, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid3p4, vdev_max_asize: 1991247921152, vdev_asize: > 1991247921152, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid2p4, vdev_max_asize: 1991246348288, vdev_asize: > 1991246348288, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid5p4, vdev_max_asize: 1991246348288, vdev_asize: > 1991246348288, vdev_min_asize: 1981411579221 > ^C > > borg-new /home/ler $ > > > borg-new /home/ler $ sudo zpool list -v > Password: > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > zroot 10.8T 94.3G 10.7T 16.0E 0% 0% 1.00x ONLINE - > raidz1 10.8T 94.3G 10.7T 16.0E 0% 0% > mfid4p4 - - - - - - > mfid0p4 - - - - - - > mfid1p4 - - - - - - > mfid3p4 - - - - - - > mfid2p4 - - - - - - > mfid5p4 - - - - - - > borg-new /home/ler $ > > > On 01/31/2017 2:37 pm, Steven Hartland wrote: > >> In that case based on your zpool history I suspect that the original >> mfid4p4 was the same size as mfid0p4 (1991246348288) but its been >> replaced with a drive which is (1991245299712), slightly smaller. >> >> This smaller size results in a max_asize of 1991245299712 * 6 instead >> of original 1991246348288* 6. >> >> Now given the way min_asize (the value used to check if the device >> size is acceptable) is rounded to the the nearest metaslab I believe >> that replace would be allowed. >> https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c#L4947 >> >> Now the problem is that on open the calculated asize is only updated >> if its expanding: >> https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c#L1424 >> >> The updated dtrace file outputs vdev_min_asize which should confirm >> my suspicion about why the replace was allowed. >> >> On 31/01/2017 19:05, Larry Rosenman wrote: >>> >>> I've replaced some disks due to failure, and some of the pariition >>> sizes are different. >>> >>> >>> autoexpand is off: >>> >>> borg-new /home/ler $ zpool get all zroot >>> NAME PROPERTY VALUE SOURCE >>> zroot size 10.8T - >>> zroot capacity 0% - >>> zroot altroot - default >>> zroot health ONLINE - >>> zroot guid 11945658884309024932 default >>> zroot version - default >>> zroot bootfs zroot/ROOT/default local >>> zroot delegation on default >>> zroot autoreplace off default >>> zroot cachefile - default >>> zroot failmode wait default >>> zroot listsnapshots off default >>> zroot autoexpand off default >>> zroot dedupditto 0 default >>> zroot dedupratio 1.00x - >>> zroot free 10.7T - >>> zroot allocated 94.3G - >>> zroot readonly off - >>> zroot comment - default >>> zroot expandsize 16.0E - >>> zroot freeing 0 default >>> zroot fragmentation 0% - >>> zroot leaked 0 default >>> zroot feature@async_destroy enabled local >>> zroot feature@empty_bpobj active local >>> zroot feature@lz4_compress active local >>> zroot feature@multi_vdev_crash_dump enabled local >>> zroot feature@spacemap_histogram active local >>> zroot feature@enabled_txg active local >>> zroot feature@hole_birth active local >>> zroot feature@extensible_dataset enabled local >>> zroot feature@embedded_data active local >>> zroot feature@bookmarks enabled local >>> zroot feature@filesystem_limits enabled local >>> zroot feature@large_blocks enabled local >>> zroot feature@sha512 enabled local >>> zroot feature@skein enabled local >>> borg-new /home/ler $ >>> >>> >>> borg-new /home/ler $ gpart show >>> => 40 3905945520 mfid0 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid1 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid2 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid3 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 16777216 3 freebsd-swap (8.0G) >>> 16779880 3889165680 4 freebsd-zfs (1.8T) >>> >>> => 40 3905945520 mfid5 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid4 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889160192 4 freebsd-zfs (1.8T) >>> 3905941504 4056 - free - (2.0M) >>> >>> borg-new /home/ler $ >>> >>> >>> this system was built last week, and I **CAN** rebuild it if >>> necessary, but I didn't do anything strange (so I thought :) ) >>> >>> >>> >>> >>> On 01/31/2017 12:30 pm, Steven Hartland wrote: >>> >>> Your issue is the reported vdev_max_asize > vdev_asize: >>> vdev_max_asize: 11947471798272 >>> vdev_asize: 11947478089728 >>> >>> max asize is smaller than asize by 6291456 >>> >>> For raidz1 Xsize should be the smallest disk Xsize * disks so: >>> 1991245299712 * 6 = 11947471798272 >>> >>> So your max asize looks right but asize looks too big >>> >>> Expand Size is calculated by: >>> if (vd->vdev_aux == NULL && tvd != NULL && vd->vdev_max_asize != >>> 0) { >>> vs->vs_esize = P2ALIGN(vd->vdev_max_asize - vd->vdev_asize, >>> 1ULL << tvd->vdev_ms_shift); >>> } >>> >>> So the question is why is asize too big? >>> >>> Given you seem to have some random disk sizes do you have auto >>> expand turned on? >>> >>> On 31/01/2017 17:39, Larry Rosenman wrote: >>> >>> vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: >>> 11947478089728 >>> >>> >>> -- >>> Larry Rosenman http://people.freebsd.org/~ler >>> >>> Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org >>> >>> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 > > > -- > Larry Rosenman http://people.freebsd.org/~ler > > Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org > > US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 --------------3F69C0E438D93AD11208EDA7 Content-Type: text/plain; charset=UTF-8; name="expand-sz-16e.patch" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="expand-sz-16e.patch" SW5kZXg6IHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMvdXRzL2NvbW1vbi9mcy96ZnMv dmRldi5jCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT0KLS0tIHN5cy9jZGRsL2NvbnRyaWIvb3BlbnNvbGFyaXMv dXRzL2NvbW1vbi9mcy96ZnMvdmRldi5jCShyZXZpc2lvbiAzMTMwMDMpCisrKyBzeXMvY2Rk bC9jb250cmliL29wZW5zb2xhcmlzL3V0cy9jb21tb24vZnMvemZzL3ZkZXYuYwkod29ya2lu ZyBjb3B5KQpAQCAtMjI5LDcgKzIyOSw4IEBACiAJICogc28gZWFjaCBjaGlsZCBtdXN0IHBy b3ZpZGUgYXQgbGVhc3QgMS9OdGggb2YgaXRzIGFzaXplLgogCSAqLwogCWlmIChwdmQtPnZk ZXZfb3BzID09ICZ2ZGV2X3JhaWR6X29wcykKLQkJcmV0dXJuIChwdmQtPnZkZXZfbWluX2Fz aXplIC8gcHZkLT52ZGV2X2NoaWxkcmVuKTsKKwkJcmV0dXJuICgocHZkLT52ZGV2X21pbl9h c2l6ZSArIHB2ZC0+dmRldl9jaGlsZHJlbiAtIDEpIC8KKwkJICAgIHB2ZC0+dmRldl9jaGls ZHJlbik7CiAKIAlyZXR1cm4gKHB2ZC0+dmRldl9taW5fYXNpemUpOwogfQpAQCAtMTM3Nyw3 ICsxMzc4LDcgQEAKIAl2ZC0+dmRldl9wc2l6ZSA9IHBzaXplOwogCiAJLyoKLQkgKiBNYWtl IHN1cmUgdGhlIGFsbG9jYXRhYmxlIHNpemUgaGFzbid0IHNocnVuay4KKwkgKiBNYWtlIHN1 cmUgdGhlIGFsbG9jYXRhYmxlIHNpemUgaGFzbid0IHNocnVuayB0b28gbXVjaC4KIAkgKi8K IAlpZiAoYXNpemUgPCB2ZC0+dmRldl9taW5fYXNpemUpIHsKIAkJdmRldl9zZXRfc3RhdGUo dmQsIEJfVFJVRSwgVkRFVl9TVEFURV9DQU5UX09QRU4sCkBAIC0xNDIwLDEwICsxNDIxLDE5 IEBACiAJICogSWYgYWxsIGNoaWxkcmVuIGFyZSBoZWFsdGh5IGFuZCB0aGUgYXNpemUgaGFz IGluY3JlYXNlZCwKIAkgKiB0aGVuIHdlJ3ZlIGV4cGVyaWVuY2VkIGR5bmFtaWMgTFVOIGdy b3d0aC4gIElmIGF1dG9tYXRpYwogCSAqIGV4cGFuc2lvbiBpcyBlbmFibGVkIHRoZW4gdXNl IHRoZSBhZGRpdGlvbmFsIHNwYWNlLgorCSAqIAorCSAqIE90aGVyd2lzZSBpZiBhc2l6ZSBo YXMgcmVkdWNlZCwgc2hyaW5rIHRvIGVuc3VyZSB0aGF0CisJICogY2FsY3VsYXRpb25zIGJh c2VkIG9mIG1heF9hc2l6ZSBhbmQgYXNpemUgZS5nLiBlc2l6ZSBhcmUKKwkgKiBhbHdheXMg dmFsaWQuIFRoaXMgaXMgc2FmZSBhcyB3ZSd2ZSBhbHJlYWR5IHZhbGlkYXRlZCB0aGF0CisJ ICogYXNpemUgaXMgbm90IGxlc3MgdGhhbiBtaW5fYXNpemUuCiAJICovCi0JaWYgKHZkLT52 ZGV2X3N0YXRlID09IFZERVZfU1RBVEVfSEVBTFRIWSAmJiBhc2l6ZSA+IHZkLT52ZGV2X2Fz aXplICYmCi0JICAgICh2ZC0+dmRldl9leHBhbmRpbmcgfHwgc3BhLT5zcGFfYXV0b2V4cGFu ZCkpCi0JCXZkLT52ZGV2X2FzaXplID0gYXNpemU7CisJaWYgKHZkLT52ZGV2X3N0YXRlID09 IFZERVZfU1RBVEVfSEVBTFRIWSkgeworCQlpZiAoYXNpemUgPiB2ZC0+dmRldl9hc2l6ZSAm JgorCQkgICAgKHZkLT52ZGV2X2V4cGFuZGluZyB8fCBzcGEtPnNwYV9hdXRvZXhwYW5kKSkK KwkJCXZkLT52ZGV2X2FzaXplID0gYXNpemU7CisJCWVsc2UgaWYgKGFzaXplIDwgdmQtPnZk ZXZfYXNpemUpCisJCQl2ZC0+dmRldl9hc2l6ZSA9IGFzaXplOworCX0KIAogCXZkZXZfc2V0 X21pbl9hc2l6ZSh2ZCk7CiAK --------------3F69C0E438D93AD11208EDA7-- From owner-freebsd-fs@freebsd.org Tue Jan 31 21:49:18 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9270ACCA36D for ; Tue, 31 Jan 2017 21:49:18 +0000 (UTC) (envelope-from ler@FreeBSD.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6A57F1D3C for ; Tue, 31 Jan 2017 21:49:18 +0000 (UTC) (envelope-from ler@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date: Content-Type:MIME-Version:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=dK0NMaPyAg+H70Ewm9sd0RlzlPhkG44RdQHBEfShpYw=; b=YOV54223uMERott6SXV0/a76Dq +aA+znQJ98YAj5vNkSPNeuKj5DYNFvAMBIbOWoinVQRMFlIr/Qu6guEDcm5f5ZjF8PWZ/2RiymHj/ 3I9U1/zz2A6yFdru06u4n+I1qcde4daT2SAcvj5UxpDJTTpCtsODsxgxYFnwmHrTeP5M=; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]:21948 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.88 (FreeBSD)) (envelope-from ) id 1cYgIf-000Gi0-Oh; Tue, 31 Jan 2017 15:49:17 -0600 Received: from proxy.na.alcatel-lucent.com ([135.245.48.75]) by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Tue, 31 Jan 2017 15:49:17 -0600 MIME-Version: 1.0 Date: Tue, 31 Jan 2017 15:49:17 -0600 From: Larry Rosenman To: Steven Hartland Cc: Freebsd fs Subject: Re: 16.0E ExpandSize? -- New Server In-Reply-To: References: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> <35a9034f91542bb1329ac5104bf3b773@FreeBSD.org> <76fc9505-f681-0de0-fe0c-5624b29de321@multiplay.co.uk> <22e1bfc5840d972cf93643733682cda1@FreeBSD.org> <8a710dc75c129f58b0372eeaeca575b5@FreeBSD.org> Message-ID: X-Sender: ler@FreeBSD.org User-Agent: Roundcube Webmail/1.2.3 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2017 21:49:18 -0000 revert the other patch and apply this one? On 01/31/2017 3:47 pm, Steven Hartland wrote: > Hmm, looks like there's also a bug in the way vdev_min_asize is calculated for raidz as it can and has resulted in child min_asize which won't provided enough space for the parent due to the use of unrounded integer division. > > 1981411579221 * 6 = 11888469475326 < 11888469475328 > > You should have vdev_min_asize: 1981411579222 for your children. > > Updated patch attached, however calculation still isn't 100% reversible so may need work, however it does now ensure that the children will provide enough capacity for min_asize even if all of them are shrunk to their individual min_asize, which I believe previously may not have been the case. > > This isn't related to the incorrect EXPANDSZ output, but would be good if you could confirm it doesn't cause any issues for your pool given its state. > > On 31/01/2017 21:00, Larry Rosenman wrote: > > borg-new /home/ler $ sudo ./vdev-stats.d > Password: > vdev_path: n/a, vdev_max_asize: 0, vdev_asize: 0, vdev_min_asize: 0 > vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: 11947478089728, vdev_min_asize: 11888469475328 > vdev_path: /dev/mfid4p4, vdev_max_asize: 1991245299712, vdev_asize: 1991245299712, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid0p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid1p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid3p4, vdev_max_asize: 1991247921152, vdev_asize: 1991247921152, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid2p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid5p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 > ^C > > borg-new /home/ler $ > > borg-new /home/ler $ sudo zpool list -v > Password: > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > zroot 10.8T 94.3G 10.7T 16.0E 0% 0% 1.00x ONLINE - > raidz1 10.8T 94.3G 10.7T 16.0E 0% 0% > mfid4p4 - - - - - - > mfid0p4 - - - - - - > mfid1p4 - - - - - - > mfid3p4 - - - - - - > mfid2p4 - - - - - - > mfid5p4 - - - - - - > borg-new /home/ler $ > > On 01/31/2017 2:37 pm, Steven Hartland wrote: In that case based on your zpool history I suspect that the original mfid4p4 was the same size as mfid0p4 (1991246348288) but its been replaced with a drive which is (1991245299712), slightly smaller. > > This smaller size results in a max_asize of 1991245299712 * 6 instead of original 1991246348288* 6. > > Now given the way min_asize (the value used to check if the device size is acceptable) is rounded to the the nearest metaslab I believe that replace would be allowed. > https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c#L4947 > > Now the problem is that on open the calculated asize is only updated if its expanding: > https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c#L1424 > > The updated dtrace file outputs vdev_min_asize which should confirm my suspicion about why the replace was allowed. > > On 31/01/2017 19:05, Larry Rosenman wrote: > > I've replaced some disks due to failure, and some of the pariition sizes are different. > > autoexpand is off: > > borg-new /home/ler $ zpool get all zroot > NAME PROPERTY VALUE SOURCE > zroot size 10.8T - > zroot capacity 0% - > zroot altroot - default > zroot health ONLINE - > zroot guid 11945658884309024932 default > zroot version - default > zroot bootfs zroot/ROOT/default local > zroot delegation on default > zroot autoreplace off default > zroot cachefile - default > zroot failmode wait default > zroot listsnapshots off default > zroot autoexpand off default > zroot dedupditto 0 default > zroot dedupratio 1.00x - > zroot free 10.7T - > zroot allocated 94.3G - > zroot readonly off - > zroot comment - default > zroot expandsize 16.0E - > zroot freeing 0 default > zroot fragmentation 0% - > zroot leaked 0 default > zroot feature@async_destroy enabled local > zroot feature@empty_bpobj active local > zroot feature@lz4_compress active local > zroot feature@multi_vdev_crash_dump enabled local > zroot feature@spacemap_histogram active local > zroot feature@enabled_txg active local > zroot feature@hole_birth active local > zroot feature@extensible_dataset enabled local > zroot feature@embedded_data active local > zroot feature@bookmarks enabled local > zroot feature@filesystem_limits enabled local > zroot feature@large_blocks enabled local > zroot feature@sha512 enabled local > zroot feature@skein enabled local > borg-new /home/ler $ > > borg-new /home/ler $ gpart show > => 40 3905945520 mfid0 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid1 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid2 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid3 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 16777216 3 freebsd-swap (8.0G) > 16779880 3889165680 4 freebsd-zfs (1.8T) > > => 40 3905945520 mfid5 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid4 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889160192 4 freebsd-zfs (1.8T) > 3905941504 4056 - free - (2.0M) > > borg-new /home/ler $ > > this system was built last week, and I **CAN** rebuild it if necessary, but I didn't do anything strange (so I thought :) ) > > On 01/31/2017 12:30 pm, Steven Hartland wrote: Your issue is the reported vdev_max_asize > vdev_asize: > vdev_max_asize: 11947471798272 > vdev_asize: 11947478089728 > > max asize is smaller than asize by 6291456 > > For raidz1 Xsize should be the smallest disk Xsize * disks so: > 1991245299712 * 6 = 11947471798272 > > So your max asize looks right but asize looks too big > > Expand Size is calculated by: > if (vd->vdev_aux == NULL && tvd != NULL && vd->vdev_max_asize != 0) { > vs->vs_esize = P2ALIGN(vd->vdev_max_asize - vd->vdev_asize, > 1ULL << tvd->vdev_ms_shift); > } > > So the question is why is asize too big? > > Given you seem to have some random disk sizes do you have auto expand turned on? > > On 31/01/2017 17:39, Larry Rosenman wrote: vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: 11947478089728 -- Larry Rosenman http://people.freebsd.org/~ler [1] Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 -- Larry Rosenman http://people.freebsd.org/~ler [1] Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 -- Larry Rosenman http://people.freebsd.org/~ler Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 Links: ------ [1] http://people.freebsd.org/%7Eler From owner-freebsd-fs@freebsd.org Tue Jan 31 22:02:29 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C3654CCAA94 for ; Tue, 31 Jan 2017 22:02:29 +0000 (UTC) (envelope-from marieheleneka@gmail.com) Received: from mail-qt0-x231.google.com (mail-qt0-x231.google.com [IPv6:2607:f8b0:400d:c0d::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 78C1EB88; Tue, 31 Jan 2017 22:02:29 +0000 (UTC) (envelope-from marieheleneka@gmail.com) Received: by mail-qt0-x231.google.com with SMTP id v23so247475116qtb.0; Tue, 31 Jan 2017 14:02:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=a/aV3Nq2HBnEisz7zHYTsNy03RZcFNK7VLtfzH6mm2Y=; b=eMaJTKFlapPqOGgObCtvVv+/A2UbRhEiVn+FVDHZTYLhFTLBXbp/odhqGTmvIcdm0o amUulclXTYmuUk7YV0ftP0xK2Bhw4EAoIr2KQayyyt16UCKiDvbpsgpwAuW+BNK70TzO eWoKVIE1/NC+bZm8/fHlIt1FVAP7Tk5M5bFdgiSIAm+H8Clfrv/ZZGiZeOzHwV+v8ifu a/fMx0ZNxwcFFJBP3PQ9hbG2576bsIgQ/XmI6Zcex3CAZ7IUtgepX1K7np/eR4WW4P+L X3jK3bQHX+za9Z3N8/lhSfC6llhLx9GcmxoxNlTijE2KveWkWCEPN9McC13nVy4WrbpP cp/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=a/aV3Nq2HBnEisz7zHYTsNy03RZcFNK7VLtfzH6mm2Y=; b=pmfnC5KJovcqAtlVjuIwsyA0ALOrHP1MtVz6k5XRFHRbg+OcVhAl1iRfCjBWZIDLgB J4dy70aAUoQ0QIDIqCcr8boRcLMLS6PQF9/lSPS4sdXh1MaWdQZbhsKHJ3LnzCqDBy4k KnCjdiph4G9QvJF5pMEeiPEgLIm1FObQ+/A85gLOcPAOfYLkm5umhaVn7MpYJhqO83WF HpnSCCGCxQugSH7YWehwL58aJxXJAI0/ACmfQ3cCtl5AEmVKwmHRYwffzW8VRAq54K+j icFQEpbRRs+/Ixady1yjn/ZBPcAZLCq2Z3xzyMlzL3Lwnst9qYvUz6hAyINdpDmbuTzL lwKw== X-Gm-Message-State: AIkVDXI7EPII3NTZ/r4mmB5DbYPFOnCilyfqV15/hAcrtpvb1XKlLQ40pjjLT3Ezaz9x4beFxlhE6bDDX42xew== X-Received: by 10.200.2.8 with SMTP id k8mr26509838qtg.163.1485900148471; Tue, 31 Jan 2017 14:02:28 -0800 (PST) MIME-Version: 1.0 References: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> <35a9034f91542bb1329ac5104bf3b773@FreeBSD.org> <76fc9505-f681-0de0-fe0c-5624b29de321@multiplay.co.uk> <22e1bfc5840d972cf93643733682cda1@FreeBSD.org> <8a710dc75c129f58b0372eeaeca575b5@FreeBSD.org> In-Reply-To: From: Marie Helene Kvello-Aune Date: Tue, 31 Jan 2017 22:02:17 +0000 Message-ID: Subject: Re: 16.0E ExpandSize? -- New Server To: Larry Rosenman , Steven Hartland Cc: Freebsd fs Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2017 22:02:29 -0000 On Tue, Jan 31, 2017 at 10:49 PM Larry Rosenman wrote: > revert the other patch and apply this one? > > On 01/31/2017 3:47 pm, Steven Hartland wrote: > > > Hmm, looks like there's also a bug in the way vdev_min_asize is > calculated for raidz as it can and has resulted in child min_asize which > won't provided enough space for the parent due to the use of unrounded > integer division. > > > > 1981411579221 * 6 = 11888469475326 < 11888469475328 > > > > You should have vdev_min_asize: 1981411579222 for your children. > > > > Updated patch attached, however calculation still isn't 100% reversible > so may need work, however it does now ensure that the children will provide > enough capacity for min_asize even if all of them are shrunk to their > individual min_asize, which I believe previously may not have been the case. > > > > This isn't related to the incorrect EXPANDSZ output, but would be good > if you could confirm it doesn't cause any issues for your pool given its > state. > > > > On 31/01/2017 21:00, Larry Rosenman wrote: > > > > borg-new /home/ler $ sudo ./vdev-stats.d > > Password: > > vdev_path: n/a, vdev_max_asize: 0, vdev_asize: 0, vdev_min_asize: 0 > > vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: > 11947478089728, vdev_min_asize: 11888469475328 > > vdev_path: /dev/mfid4p4, vdev_max_asize: 1991245299712, vdev_asize: > 1991245299712, vdev_min_asize: 1981411579221 > > vdev_path: /dev/mfid0p4, vdev_max_asize: 1991246348288, vdev_asize: > 1991246348288, vdev_min_asize: 1981411579221 > > vdev_path: /dev/mfid1p4, vdev_max_asize: 1991246348288, vdev_asize: > 1991246348288, vdev_min_asize: 1981411579221 > > vdev_path: /dev/mfid3p4, vdev_max_asize: 1991247921152, vdev_asize: > 1991247921152, vdev_min_asize: 1981411579221 > > vdev_path: /dev/mfid2p4, vdev_max_asize: 1991246348288, vdev_asize: > 1991246348288, vdev_min_asize: 1981411579221 > > vdev_path: /dev/mfid5p4, vdev_max_asize: 1991246348288, vdev_asize: > 1991246348288, vdev_min_asize: 1981411579221 > > ^C > > > > borg-new /home/ler $ > > > > borg-new /home/ler $ sudo zpool list -v > > Password: > > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > > zroot 10.8T 94.3G 10.7T 16.0E 0% 0% 1.00x ONLINE - > > raidz1 10.8T 94.3G 10.7T 16.0E 0% 0% > > mfid4p4 - - - - - - > > mfid0p4 - - - - - - > > mfid1p4 - - - - - - > > mfid3p4 - - - - - - > > mfid2p4 - - - - - - > > mfid5p4 - - - - - - > > borg-new /home/ler $ > > > > On 01/31/2017 2:37 pm, Steven Hartland wrote: In that case based on your > zpool history I suspect that the original mfid4p4 was the same size as > mfid0p4 (1991246348288) but its been replaced with a drive which is > (1991245299712), slightly smaller. > > > > This smaller size results in a max_asize of 1991245299712 * 6 instead of > original 1991246348288* 6. > > > > Now given the way min_asize (the value used to check if the device size > is acceptable) is rounded to the the nearest metaslab I believe that > replace would be allowed. > > > https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c#L4947 > > > > Now the problem is that on open the calculated asize is only updated if > its expanding: > > > https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c#L1424 > > > > The updated dtrace file outputs vdev_min_asize which should confirm my > suspicion about why the replace was allowed. > > > > On 31/01/2017 19:05, Larry Rosenman wrote: > > > > I've replaced some disks due to failure, and some of the pariition sizes > are different. > > > > autoexpand is off: > > > > borg-new /home/ler $ zpool get all zroot > > NAME PROPERTY VALUE SOURCE > > zroot size 10.8T - > > zroot capacity 0% - > > zroot altroot - default > > zroot health ONLINE - > > zroot guid 11945658884309024932 default > > zroot version - default > > zroot bootfs zroot/ROOT/default local > > zroot delegation on default > > zroot autoreplace off default > > zroot cachefile - default > > zroot failmode wait default > > zroot listsnapshots off default > > zroot autoexpand off default > > zroot dedupditto 0 default > > zroot dedupratio 1.00x - > > zroot free 10.7T - > > zroot allocated 94.3G - > > zroot readonly off - > > zroot comment - default > > zroot expandsize 16.0E - > > zroot freeing 0 default > > zroot fragmentation 0% - > > zroot leaked 0 default > > zroot feature@async_destroy enabled local > > zroot feature@empty_bpobj active local > > zroot feature@lz4_compress active local > > zroot feature@multi_vdev_crash_dump enabled local > > zroot feature@spacemap_histogram active local > > zroot feature@enabled_txg active local > > zroot feature@hole_birth active local > > zroot feature@extensible_dataset enabled local > > zroot feature@embedded_data active local > > zroot feature@bookmarks enabled local > > zroot feature@filesystem_limits enabled local > > zroot feature@large_blocks enabled local > > zroot feature@sha512 enabled local > > zroot feature@skein enabled local > > borg-new /home/ler $ > > > > borg-new /home/ler $ gpart show > > => 40 3905945520 mfid0 GPT (1.8T) > > 40 1600 1 efi (800K) > > 1640 1024 2 freebsd-boot (512K) > > 2664 1432 - free - (716K) > > 4096 16777216 3 freebsd-swap (8.0G) > > 16781312 3889162240 4 freebsd-zfs (1.8T) > > 3905943552 2008 - free - (1.0M) > > > > => 40 3905945520 mfid1 GPT (1.8T) > > 40 1600 1 efi (800K) > > 1640 1024 2 freebsd-boot (512K) > > 2664 1432 - free - (716K) > > 4096 16777216 3 freebsd-swap (8.0G) > > 16781312 3889162240 4 freebsd-zfs (1.8T) > > 3905943552 2008 - free - (1.0M) > > > > => 40 3905945520 mfid2 GPT (1.8T) > > 40 1600 1 efi (800K) > > 1640 1024 2 freebsd-boot (512K) > > 2664 1432 - free - (716K) > > 4096 16777216 3 freebsd-swap (8.0G) > > 16781312 3889162240 4 freebsd-zfs (1.8T) > > 3905943552 2008 - free - (1.0M) > > > > => 40 3905945520 mfid3 GPT (1.8T) > > 40 1600 1 efi (800K) > > 1640 1024 2 freebsd-boot (512K) > > 2664 16777216 3 freebsd-swap (8.0G) > > 16779880 3889165680 4 freebsd-zfs (1.8T) > > > > => 40 3905945520 mfid5 GPT (1.8T) > > 40 1600 1 efi (800K) > > 1640 1024 2 freebsd-boot (512K) > > 2664 1432 - free - (716K) > > 4096 16777216 3 freebsd-swap (8.0G) > > 16781312 3889162240 4 freebsd-zfs (1.8T) > > 3905943552 2008 - free - (1.0M) > > > > => 40 3905945520 mfid4 GPT (1.8T) > > 40 1600 1 efi (800K) > > 1640 1024 2 freebsd-boot (512K) > > 2664 1432 - free - (716K) > > 4096 16777216 3 freebsd-swap (8.0G) > > 16781312 3889160192 4 freebsd-zfs (1.8T) > > 3905941504 4056 - free - (2.0M) > > > > borg-new /home/ler $ > > > > this system was built last week, and I **CAN** rebuild it if necessary, > but I didn't do anything strange (so I thought :) ) > > > > On 01/31/2017 12:30 pm, Steven Hartland wrote: Your issue is the > reported vdev_max_asize > vdev_asize: > > vdev_max_asize: 11947471798272 > > vdev_asize: 11947478089728 > > > > max asize is smaller than asize by 6291456 > > > > For raidz1 Xsize should be the smallest disk Xsize * disks so: > > 1991245299712 * 6 = 11947471798272 > > > > So your max asize looks right but asize looks too big > > > > Expand Size is calculated by: > > if (vd->vdev_aux == NULL && tvd != NULL && vd->vdev_max_asize != 0) { > > vs->vs_esize = P2ALIGN(vd->vdev_max_asize - vd->vdev_asize, > > 1ULL << tvd->vdev_ms_shift); > > } > > > > So the question is why is asize too big? > > > > Given you seem to have some random disk sizes do you have auto expand > turned on? > > > > On 31/01/2017 17:39, Larry Rosenman wrote: vdev_path: n/a, > vdev_max_asize: 11947471798272, vdev_asize: 11947478089728 > > -- > Larry Rosenman http://people.freebsd.org/~ler [1] > Phone: +1 214-642-9640 <(214)%20642-9640> E-Mail: > ler@FreeBSD.org > US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 > > -- > Larry Rosenman http://people.freebsd.org/~ler [1] > Phone: +1 214-642-9640 <(214)%20642-9640> E-Mail: > ler@FreeBSD.org > US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 > > -- > Larry Rosenman http://people.freebsd.org/~ler > Phone: +1 214-642-9640 <(214)%20642-9640> E-Mail: > ler@FreeBSD.org > US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 > > > Links: > ------ > [1] http://people.freebsd.org/%7Eler > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > I have the same observation on my home file server. I've not tried the patches (will try that once I get time next week), but the output of the dtrace script while doing 'zpool list -v' shows: # ./dtrace.sh vdev_path: n/a, vdev_max_asize: 0, vdev_asize: 0 vdev_path: n/a, vdev_max_asize: 23907502915584, vdev_asize: 23907504488448 vdev_path: /dev/gpt/Bay1.eli, vdev_max_asize: 3984583819264, vdev_asize: 3984583819264 vdev_path: /dev/gpt/Bay2.eli, vdev_max_asize: 3984583819264, vdev_asize: 3984583819264 vdev_path: /dev/gpt/Bay3.eli, vdev_max_asize: 3984583819264, vdev_asize: 3984583819264 vdev_path: /dev/gpt/Bay4.eli, vdev_max_asize: 3984583819264, vdev_asize: 3984583819264 vdev_path: /dev/gpt/Bay5.eli, vdev_max_asize: 3984583819264, vdev_asize: 3984583819264 vdev_path: /dev/gpt/Bay6.eli, vdev_max_asize: 3984583819264, vdev_asize: 3984583819264 The second line has the same discrepancy as above. This pool was created without geli encryption first, then while the pool was still empty, each drive was offlined and replaced with its .eli counterpart. IIRC geli leaves some metadata on the disk, shrinking available space ever so slightly, which seems to fit the proposed cause earlier in this thread. MH From owner-freebsd-fs@freebsd.org Tue Jan 31 22:37:31 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 50459CCA603 for ; Tue, 31 Jan 2017 22:37:31 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x22e.google.com (mail-wm0-x22e.google.com [IPv6:2a00:1450:400c:c09::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C244F1EB9 for ; Tue, 31 Jan 2017 22:37:30 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wm0-x22e.google.com with SMTP id b65so10864092wmf.0 for ; Tue, 31 Jan 2017 14:37:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to; bh=rl+JYiLppLmvQ3rPrl4KicYNfG8/1wBzhE5n2ncex/4=; b=S2xwPK6GHBQtCTZU1rlXk+55522pA2uxwSXuIJQo9K5Rd1VaFO/BBnH0FzbtV7RNQZ Qs+fGWsnDW7Wmf6NF1gkewNzW0GFdj9CD48qi7dYkLz9PlQTuJRkXIIo44qo53Xz5Tes FdTuaEBrcpitbExjdh3pUpaXd4O2jpJDhpGjEIsgQ2dNcu6/3/1TCpiEka8aUfXGq5fu 8USsc7C5oHAmlt371147BhtbY50uvpHo974B77PgXwfcWj3W+BSbiUGw7+FlanmnWP9w zxM4VN8aeFpcWs9z1E90ikhmQWO8LGulegj2FzC28xHxWze+WShMrlySxXI12/DMhRHE dE1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=rl+JYiLppLmvQ3rPrl4KicYNfG8/1wBzhE5n2ncex/4=; b=bJg9pet53I6nDUNERyCEGPsbfvw7TuSL5ChgBoonyFhFQfukS3ZxLoGx3lR4qpQTlG UBlMtjlac4VXn+Fyc9vRvfQuV2BhQVnEOKM5JlegbLl/C/i9ZaAHsfxhjVgW3Z+tmNaE m8fWZRLx+Sdju5/kiUYSjrjbtdiIGoBRvmQnoYJJd71Y606KEKNZN4f1IiIjmvmI0iLV r1anl+XTpAXfTzFMfuUfxUnmqGD6zcC1McQpFmtwpTBCNOBnIc/SNI41uGVDfh1zopcG NGAvkkgwFiAwxCej8mbFWZ6FSSTJ6w9fetvAd1tAm9bE3DIDbL9vixd+1v5aTh5L6d3P kS8A== X-Gm-Message-State: AIkVDXKccyL90m4zhKazojA3XhViQfLtaizhW4npti5YXxF+0F1an7ZJWN6E7o8jKwedrBfm X-Received: by 10.28.143.204 with SMTP id r195mr137582wmd.32.1485902248435; Tue, 31 Jan 2017 14:37:28 -0800 (PST) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id k70sm26233743wmc.3.2017.01.31.14.37.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 31 Jan 2017 14:37:27 -0800 (PST) Subject: Re: 16.0E ExpandSize? -- New Server To: Marie Helene Kvello-Aune , Larry Rosenman References: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> <35a9034f91542bb1329ac5104bf3b773@FreeBSD.org> <76fc9505-f681-0de0-fe0c-5624b29de321@multiplay.co.uk> <22e1bfc5840d972cf93643733682cda1@FreeBSD.org> <8a710dc75c129f58b0372eeaeca575b5@FreeBSD.org> Cc: Freebsd fs From: Steven Hartland Message-ID: Date: Tue, 31 Jan 2017 22:37:28 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2017 22:37:31 -0000 On 31/01/2017 22:02, Marie Helene Kvello-Aune wrote: > On Tue, Jan 31, 2017 at 10:49 PM Larry Rosenman > wrote: > > revert the other patch and apply this one? > > On 01/31/2017 3:47 pm, Steven Hartland wrote: > > > Hmm, looks like there's also a bug in the way vdev_min_asize is > calculated for raidz as it can and has resulted in child min_asize > which won't provided enough space for the parent due to the use of > unrounded integer division. > > > > 1981411579221 * 6 = 11888469475326 < 11888469475328 > > > > You should have vdev_min_asize: 1981411579222 for your children. > > > > Updated patch attached, however calculation still isn't 100% > reversible so may need work, however it does now ensure that the > children will provide enough capacity for min_asize even if all of > them are shrunk to their individual min_asize, which I believe > previously may not have been the case. > > > > This isn't related to the incorrect EXPANDSZ output, but would > be good if you could confirm it doesn't cause any issues for your > pool given its state. > > > > On 31/01/2017 21:00, Larry Rosenman wrote: > > > > borg-new /home/ler $ sudo ./vdev-stats.d > > Password: > > vdev_path: n/a, vdev_max_asize: 0, vdev_asize: 0, vdev_min_asize: 0 > > vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: > 11947478089728, vdev_min_asize: 11888469475328 > > vdev_path: /dev/mfid4p4, vdev_max_asize: 1991245299712, > vdev_asize: 1991245299712, vdev_min_asize: 1981411579221 > > vdev_path: /dev/mfid0p4, vdev_max_asize: 1991246348288, > vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 > > vdev_path: /dev/mfid1p4, vdev_max_asize: 1991246348288, > vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 > > vdev_path: /dev/mfid3p4, vdev_max_asize: 1991247921152, > vdev_asize: 1991247921152, vdev_min_asize: 1981411579221 > > vdev_path: /dev/mfid2p4, vdev_max_asize: 1991246348288, > vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 > > vdev_path: /dev/mfid5p4, vdev_max_asize: 1991246348288, > vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 > > ^C > > > > borg-new /home/ler $ > > > > borg-new /home/ler $ sudo zpool list -v > > Password: > > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > > zroot 10.8T 94.3G 10.7T 16.0E 0% 0% 1.00x ONLINE - > > raidz1 10.8T 94.3G 10.7T 16.0E 0% 0% > > mfid4p4 - - - - - - > > mfid0p4 - - - - - - > > mfid1p4 - - - - - - > > mfid3p4 - - - - - - > > mfid2p4 - - - - - - > > mfid5p4 - - - - - - > > borg-new /home/ler $ > > > > On 01/31/2017 2:37 pm, Steven Hartland wrote: In that case based > on your zpool history I suspect that the original mfid4p4 was the > same size as mfid0p4 (1991246348288) but its been replaced with a > drive which is (1991245299712), slightly smaller. > > > > This smaller size results in a max_asize of 1991245299712 * 6 > instead of original 1991246348288* 6. > > > > Now given the way min_asize (the value used to check if the > device size is acceptable) is rounded to the the nearest metaslab > I believe that replace would be allowed. > > > https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c#L4947 > > > > Now the problem is that on open the calculated asize is only > updated if its expanding: > > > https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c#L1424 > > > > The updated dtrace file outputs vdev_min_asize which should > confirm my suspicion about why the replace was allowed. > > > > On 31/01/2017 19:05, Larry Rosenman wrote: > > > > I've replaced some disks due to failure, and some of the > pariition sizes are different. > > > > autoexpand is off: > > > > borg-new /home/ler $ zpool get all zroot > > NAME PROPERTY VALUE SOURCE > > zroot size 10.8T - > > zroot capacity 0% - > > zroot altroot - default > > zroot health ONLINE - > > zroot guid 11945658884309024932 default > > zroot version - default > > zroot bootfs zroot/ROOT/default local > > zroot delegation on default > > zroot autoreplace off default > > zroot cachefile - default > > zroot failmode wait default > > zroot listsnapshots off default > > zroot autoexpand off default > > zroot dedupditto 0 default > > zroot dedupratio 1.00x - > > zroot free 10.7T - > > zroot allocated 94.3G - > > zroot readonly off - > > zroot comment - default > > zroot expandsize 16.0E - > > zroot freeing 0 default > > zroot fragmentation 0% - > > zroot leaked 0 default > > zroot feature@async_destroy enabled local > > zroot feature@empty_bpobj active local > > zroot feature@lz4_compress active local > > zroot feature@multi_vdev_crash_dump enabled local > > zroot feature@spacemap_histogram active local > > zroot feature@enabled_txg active local > > zroot feature@hole_birth active local > > zroot feature@extensible_dataset enabled local > > zroot feature@embedded_data active local > > zroot feature@bookmarks enabled local > > zroot feature@filesystem_limits enabled local > > zroot feature@large_blocks enabled local > > zroot feature@sha512 enabled local > > zroot feature@skein enabled local > > borg-new /home/ler $ > > > > borg-new /home/ler $ gpart show > > => 40 3905945520 mfid0 GPT (1.8T) > > 40 1600 1 efi (800K) > > 1640 1024 2 freebsd-boot (512K) > > 2664 1432 - free - (716K) > > 4096 16777216 3 freebsd-swap (8.0G) > > 16781312 3889162240 4 freebsd-zfs (1.8T) > > 3905943552 2008 - free - (1.0M) > > > > => 40 3905945520 mfid1 GPT (1.8T) > > 40 1600 1 efi (800K) > > 1640 1024 2 freebsd-boot (512K) > > 2664 1432 - free - (716K) > > 4096 16777216 3 freebsd-swap (8.0G) > > 16781312 3889162240 4 freebsd-zfs (1.8T) > > 3905943552 2008 - free - (1.0M) > > > > => 40 3905945520 mfid2 GPT (1.8T) > > 40 1600 1 efi (800K) > > 1640 1024 2 freebsd-boot (512K) > > 2664 1432 - free - (716K) > > 4096 16777216 3 freebsd-swap (8.0G) > > 16781312 3889162240 4 freebsd-zfs (1.8T) > > 3905943552 2008 - free - (1.0M) > > > > => 40 3905945520 mfid3 GPT (1.8T) > > 40 1600 1 efi (800K) > > 1640 1024 2 freebsd-boot (512K) > > 2664 16777216 3 freebsd-swap (8.0G) > > 16779880 3889165680 4 freebsd-zfs (1.8T) > > > > => 40 3905945520 mfid5 GPT (1.8T) > > 40 1600 1 efi (800K) > > 1640 1024 2 freebsd-boot (512K) > > 2664 1432 - free - (716K) > > 4096 16777216 3 freebsd-swap (8.0G) > > 16781312 3889162240 4 freebsd-zfs (1.8T) > > 3905943552 2008 - free - (1.0M) > > > > => 40 3905945520 mfid4 GPT (1.8T) > > 40 1600 1 efi (800K) > > 1640 1024 2 freebsd-boot (512K) > > 2664 1432 - free - (716K) > > 4096 16777216 3 freebsd-swap (8.0G) > > 16781312 3889160192 4 freebsd-zfs (1.8T) > > 3905941504 4056 - free - (2.0M) > > > > borg-new /home/ler $ > > > > this system was built last week, and I **CAN** rebuild it if > necessary, but I didn't do anything strange (so I thought :) ) > > > > On 01/31/2017 12:30 pm, Steven Hartland wrote: Your issue is the > reported vdev_max_asize > vdev_asize: > > vdev_max_asize: 11947471798272 > > vdev_asize: 11947478089728 > > > > max asize is smaller than asize by 6291456 > > > > For raidz1 Xsize should be the smallest disk Xsize * disks so: > > 1991245299712 * 6 = 11947471798272 > > > > So your max asize looks right but asize looks too big > > > > Expand Size is calculated by: > > if (vd->vdev_aux == NULL && tvd != NULL && vd->vdev_max_asize != > 0) { > > vs->vs_esize = P2ALIGN(vd->vdev_max_asize - vd->vdev_asize, > > 1ULL << tvd->vdev_ms_shift); > > } > > > > So the question is why is asize too big? > > > > Given you seem to have some random disk sizes do you have auto > expand turned on? > > > > On 31/01/2017 17:39, Larry Rosenman wrote: vdev_path: n/a, > vdev_max_asize: 11947471798272, vdev_asize: 11947478089728 > > -- > Larry Rosenman http://people.freebsd.org/~ler > [1] > Phone: +1 214-642-9640 > E-Mail: ler@FreeBSD.org > US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 > > -- > Larry Rosenman http://people.freebsd.org/~ler > [1] > Phone: +1 214-642-9640 > E-Mail: ler@FreeBSD.org > US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 > > -- > Larry Rosenman http://people.freebsd.org/~ler > > Phone: +1 214-642-9640 > E-Mail: ler@FreeBSD.org > US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 > > > Links: > ------ > [1] http://people.freebsd.org/%7Eler > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to > "freebsd-fs-unsubscribe@freebsd.org > " > > > I have the same observation on my home file server. I've not tried the > patches (will try that once I get time next week), but the output of > the dtrace script while doing 'zpool list -v' shows: > > # ./dtrace.sh > vdev_path: n/a, vdev_max_asize: 0, vdev_asize: 0 > vdev_path: n/a, vdev_max_asize: 23907502915584, vdev_asize: 23907504488448 > vdev_path: /dev/gpt/Bay1.eli, vdev_max_asize: 3984583819264, > vdev_asize: 3984583819264 > vdev_path: /dev/gpt/Bay2.eli, vdev_max_asize: 3984583819264, > vdev_asize: 3984583819264 > vdev_path: /dev/gpt/Bay3.eli, vdev_max_asize: 3984583819264, > vdev_asize: 3984583819264 > vdev_path: /dev/gpt/Bay4.eli, vdev_max_asize: 3984583819264, > vdev_asize: 3984583819264 > vdev_path: /dev/gpt/Bay5.eli, vdev_max_asize: 3984583819264, > vdev_asize: 3984583819264 > vdev_path: /dev/gpt/Bay6.eli, vdev_max_asize: 3984583819264, > vdev_asize: 3984583819264 > > The second line has the same discrepancy as above. This pool was > created without geli encryption first, then while the pool was still > empty, each drive was offlined and replaced with its .eli counterpart. > IIRC geli leaves some metadata on the disk, shrinking available space > ever so slightly, which seems to fit the proposed cause earlier in > this thread. > > MH Yes indeed it does. Regards Steve From owner-freebsd-fs@freebsd.org Tue Jan 31 23:22:25 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4199CCCA221 for ; Tue, 31 Jan 2017 23:22:25 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x231.google.com (mail-wm0-x231.google.com [IPv6:2a00:1450:400c:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D6EB11576 for ; Tue, 31 Jan 2017 23:22:24 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wm0-x231.google.com with SMTP id c85so12236567wmi.1 for ; Tue, 31 Jan 2017 15:22:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to; bh=KPHc/sURsu/jgWh8hoRRketAskb1gfmJYI1C92bClE4=; b=P7zIyAL7SExdZ9y5k9QZVlgQoEi3xNsk75DpILwsoXYjrtBbvXcI+tb0oUmjnG5zyJ iHkz4q/WiTJRE7EVQ/27LjAY9u3GF19pwUVVVdAxC2hswb0o430QBksXhq7QODtPQRbX 0lK5Z+2GTlN/QCQGr7gdet2jBGMiM85JyBv8fhTSKoEAuQYFypXP34XVaIM7T9Zux5Sl smCAA2UdWJ5qRpfqEMektHPx+/4/vE9tVE452vKYu8YzqjLnOkdc9n7zt7A/dOfE6FEF F+fHK0Wie78c1TnoFwyOmOA6ziScO/JwmPrB73fdmF8+wFW/LgBdQBDQZaavz+rn7opK rcag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=KPHc/sURsu/jgWh8hoRRketAskb1gfmJYI1C92bClE4=; b=rkSXta+8t7vwAAlifwHKr0ICzME/jFt2VuOgVxtGONAe3mwvni3hhv6WuRqh5BAm3E 3xZRcWRrUx1smMQj+7esVEdb57BYZ5dvYb2JogS80dNPLafFNqLLEKCzUrdslGzWmf55 jEz6pwBv35N9wG5nyYYhg6tXkyTHM1mdCO+LrFnYfi3xAJ+DDwwy7CYTQ9c6EMLznY53 0EVUQ/8G1C9ykNmiIjYH7MF+W8aXvhNZwIGQHJrR03BLtzohMAGc8F464TZicEcDBisE RkQ/deY7MERyFtMd4H+4b0KVzQH7rhH/1a/EAxzBzBi/M/EcUxNXEJDXwgfsOHNI8a6w oyrg== X-Gm-Message-State: AIkVDXLfjbrADTWX6TeXuMScsK2iRaHWwYOBNUDg6szPoL+X6h64RFJD1z577i/aV5Uaz0fe X-Received: by 10.28.227.133 with SMTP id a127mr19305399wmh.104.1485904942549; Tue, 31 Jan 2017 15:22:22 -0800 (PST) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id 36sm30771198wrz.8.2017.01.31.15.22.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 31 Jan 2017 15:22:21 -0800 (PST) Subject: Re: 16.0E ExpandSize? -- New Server To: Larry Rosenman References: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> <35a9034f91542bb1329ac5104bf3b773@FreeBSD.org> <76fc9505-f681-0de0-fe0c-5624b29de321@multiplay.co.uk> <22e1bfc5840d972cf93643733682cda1@FreeBSD.org> <8a710dc75c129f58b0372eeaeca575b5@FreeBSD.org> Cc: Freebsd fs From: Steven Hartland Message-ID: <96534515-4fcb-774e-a599-8d48aec930cd@multiplay.co.uk> Date: Tue, 31 Jan 2017 23:22:22 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Jan 2017 23:22:25 -0000 Yep On 31/01/2017 21:49, Larry Rosenman wrote: > > revert the other patch and apply this one? > > > > On 01/31/2017 3:47 pm, Steven Hartland wrote: > >> Hmm, looks like there's also a bug in the way vdev_min_asize is >> calculated for raidz as it can and has resulted in child min_asize >> which won't provided enough space for the parent due to the use of >> unrounded integer division. >> >> 1981411579221 * 6 = 11888469475326 < 11888469475328 >> >> You should have vdev_min_asize: 1981411579222 for your children. >> >> Updated patch attached, however calculation still isn't 100% >> reversible so may need work, however it does now ensure that the >> children will provide enough capacity for min_asize even if all of >> them are shrunk to their individual min_asize, which I believe >> previously may not have been the case. >> >> This isn't related to the incorrect EXPANDSZ output, but would be >> good if you could confirm it doesn't cause any issues for your pool >> given its state. >> >> On 31/01/2017 21:00, Larry Rosenman wrote: >>> >>> borg-new /home/ler $ sudo ./vdev-stats.d >>> Password: >>> vdev_path: n/a, vdev_max_asize: 0, vdev_asize: 0, vdev_min_asize: 0 >>> vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: >>> 11947478089728, vdev_min_asize: 11888469475328 >>> vdev_path: /dev/mfid4p4, vdev_max_asize: 1991245299712, vdev_asize: >>> 1991245299712, vdev_min_asize: 1981411579221 >>> vdev_path: /dev/mfid0p4, vdev_max_asize: 1991246348288, vdev_asize: >>> 1991246348288, vdev_min_asize: 1981411579221 >>> vdev_path: /dev/mfid1p4, vdev_max_asize: 1991246348288, vdev_asize: >>> 1991246348288, vdev_min_asize: 1981411579221 >>> vdev_path: /dev/mfid3p4, vdev_max_asize: 1991247921152, vdev_asize: >>> 1991247921152, vdev_min_asize: 1981411579221 >>> vdev_path: /dev/mfid2p4, vdev_max_asize: 1991246348288, vdev_asize: >>> 1991246348288, vdev_min_asize: 1981411579221 >>> vdev_path: /dev/mfid5p4, vdev_max_asize: 1991246348288, vdev_asize: >>> 1991246348288, vdev_min_asize: 1981411579221 >>> ^C >>> >>> borg-new /home/ler $ >>> >>> >>> borg-new /home/ler $ sudo zpool list -v >>> Password: >>> NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT >>> zroot 10.8T 94.3G 10.7T 16.0E 0% 0% 1.00x ONLINE - >>> raidz1 10.8T 94.3G 10.7T 16.0E 0% 0% >>> mfid4p4 - - - - - - >>> mfid0p4 - - - - - - >>> mfid1p4 - - - - - - >>> mfid3p4 - - - - - - >>> mfid2p4 - - - - - - >>> mfid5p4 - - - - - - >>> borg-new /home/ler $ >>> >>> >>> On 01/31/2017 2:37 pm, Steven Hartland wrote: >>> >>> In that case based on your zpool history I suspect that the >>> original mfid4p4 was the same size as mfid0p4 (1991246348288) >>> but its been replaced with a drive which is (1991245299712), >>> slightly smaller. >>> >>> This smaller size results in a max_asize of 1991245299712 * 6 >>> instead of original 1991246348288* 6. >>> >>> Now given the way min_asize (the value used to check if the >>> device size is acceptable) is rounded to the the nearest >>> metaslab I believe that replace would be allowed. >>> https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c#L4947 >>> >>> Now the problem is that on open the calculated asize is only >>> updated if its expanding: >>> https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c#L1424 >>> >>> The updated dtrace file outputs vdev_min_asize which should >>> confirm my suspicion about why the replace was allowed. >>> >>> On 31/01/2017 19:05, Larry Rosenman wrote: >>> >>> I've replaced some disks due to failure, and some of the >>> pariition sizes are different. >>> >>> >>> autoexpand is off: >>> >>> borg-new /home/ler $ zpool get all zroot >>> NAME PROPERTY VALUE SOURCE >>> zroot size 10.8T - >>> zroot capacity 0% - >>> zroot altroot - default >>> zroot health ONLINE - >>> zroot guid 11945658884309024932 default >>> zroot version - default >>> zroot bootfs zroot/ROOT/default local >>> zroot delegation on default >>> zroot autoreplace off default >>> zroot cachefile - default >>> zroot failmode wait default >>> zroot listsnapshots off default >>> zroot autoexpand off default >>> zroot dedupditto 0 default >>> zroot dedupratio 1.00x - >>> zroot free 10.7T - >>> zroot allocated 94.3G - >>> zroot readonly off - >>> zroot comment - default >>> zroot expandsize 16.0E - >>> zroot freeing 0 default >>> zroot fragmentation 0% - >>> zroot leaked 0 default >>> zroot feature@async_destroy enabled local >>> zroot feature@empty_bpobj active local >>> zroot feature@lz4_compress active local >>> zroot feature@multi_vdev_crash_dump enabled local >>> zroot feature@spacemap_histogram active local >>> zroot feature@enabled_txg active local >>> zroot feature@hole_birth active local >>> zroot feature@extensible_dataset enabled local >>> zroot feature@embedded_data active local >>> zroot feature@bookmarks enabled local >>> zroot feature@filesystem_limits enabled local >>> zroot feature@large_blocks enabled local >>> zroot feature@sha512 enabled local >>> zroot feature@skein enabled local >>> borg-new /home/ler $ >>> >>> >>> borg-new /home/ler $ gpart show >>> => 40 3905945520 mfid0 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid1 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid2 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid3 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 16777216 3 freebsd-swap (8.0G) >>> 16779880 3889165680 4 freebsd-zfs (1.8T) >>> >>> => 40 3905945520 mfid5 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid4 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889160192 4 freebsd-zfs (1.8T) >>> 3905941504 4056 - free - (2.0M) >>> >>> borg-new /home/ler $ >>> >>> >>> this system was built last week, and I **CAN** rebuild it if >>> necessary, but I didn't do anything strange (so I thought :) ) >>> >>> >>> >>> >>> On 01/31/2017 12:30 pm, Steven Hartland wrote: >>> >>> Your issue is the reported vdev_max_asize > vdev_asize: >>> vdev_max_asize: 11947471798272 >>> vdev_asize: 11947478089728 >>> >>> max asize is smaller than asize by 6291456 >>> >>> For raidz1 Xsize should be the smallest disk Xsize * >>> disks so: >>> 1991245299712 * 6 = 11947471798272 >>> >>> So your max asize looks right but asize looks too big >>> >>> Expand Size is calculated by: >>> if (vd->vdev_aux == NULL && tvd != NULL && >>> vd->vdev_max_asize != 0) { >>> vs->vs_esize = P2ALIGN(vd->vdev_max_asize - >>> vd->vdev_asize, >>> 1ULL << tvd->vdev_ms_shift); >>> } >>> >>> So the question is why is asize too big? >>> >>> Given you seem to have some random disk sizes do you >>> have auto expand turned on? >>> >>> On 31/01/2017 17:39, Larry Rosenman wrote: >>> >>> vdev_path: n/a, vdev_max_asize: 11947471798272, >>> vdev_asize: 11947478089728 >>> >>> >>> -- >>> Larry Rosenman http://people.freebsd.org/~ler >>> >>> Phone: +1 214-642-9640 E-Mail: >>> ler@FreeBSD.org >>> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 >>> >>> >>> -- >>> Larry Rosenman http://people.freebsd.org/~ler >>> >>> Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org >>> >>> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 > > > -- > Larry Rosenman http://people.freebsd.org/~ler > > Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org > > US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 From owner-freebsd-fs@freebsd.org Wed Feb 1 02:31:34 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DB02ACCB4D7 for ; Wed, 1 Feb 2017 02:31:34 +0000 (UTC) (envelope-from ler@FreeBSD.org) Received: from thebighonker.lerctr.org (thebighonker.lerctr.org [IPv6:2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "thebighonker.lerctr.org", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B1D18618 for ; Wed, 1 Feb 2017 02:31:34 +0000 (UTC) (envelope-from ler@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lerctr.org; s=lerami; h=Message-ID:References:In-Reply-To:Subject:Cc:To:From:Date: Content-Type:MIME-Version:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender: Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=7ys4sF/Nv8ddfja/AStVJpaSzn3u0Eg8jdsUuNboJEY=; b=RK28PLRSPI5amgpo1T7fBre8j7 VXD61396qWaYnGcUW7QzLhgqQEYSJdmYdZz6H9bpKiymwNDa78LSaXXft7Xg3EHwYv6SPmvh2G9FN 2kEsAzTiPea9QnkN6TB8BFBcDdkMh6BSk5FpsX7PjTjvHM1iniZ8e1jrDlzLOPhm6wy8=; Received: from thebighonker.lerctr.org ([2001:470:1f0f:3ad:223:7dff:fe9e:6e8a]:53683 helo=webmail.lerctr.org) by thebighonker.lerctr.org with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.88 (FreeBSD)) (envelope-from ) id 1cYkhp-000PNT-JU; Tue, 31 Jan 2017 20:31:33 -0600 Received: from 2001:470:1f0f:42c:6dbb:d5c1:36f8:9778 by webmail.lerctr.org with HTTP (HTTP/1.1 POST); Tue, 31 Jan 2017 20:31:33 -0600 MIME-Version: 1.0 Date: Tue, 31 Jan 2017 20:31:33 -0600 From: Larry Rosenman To: Steven Hartland Cc: Freebsd fs Subject: Re: 16.0E ExpandSize? -- New Server In-Reply-To: <96534515-4fcb-774e-a599-8d48aec930cd@multiplay.co.uk> References: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> <35a9034f91542bb1329ac5104bf3b773@FreeBSD.org> <76fc9505-f681-0de0-fe0c-5624b29de321@multiplay.co.uk> <22e1bfc5840d972cf93643733682cda1@FreeBSD.org> <8a710dc75c129f58b0372eeaeca575b5@FreeBSD.org> <96534515-4fcb-774e-a599-8d48aec930cd@multiplay.co.uk> Message-ID: X-Sender: ler@FreeBSD.org User-Agent: Roundcube Webmail/1.2.3 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Feb 2017 02:31:35 -0000 no grief that I can see: borg-new /home/ler $ sudo zdb Password: zroot: version: 5000 name: 'zroot' state: 0 txg: 96143 pool_guid: 11945658884309024932 hostid: 3619181042 hostname: '' com.delphix:has_per_vdev_zaps vdev_children: 1 vdev_tree: type: 'root' id: 0 guid: 11945658884309024932 create_txg: 4 children[0]: type: 'raidz' id: 0 guid: 7596925654112466913 nparity: 1 metaslab_array: 42 metaslab_shift: 36 ashift: 12 asize: 11947471798272 is_log: 0 create_txg: 4 com.delphix:vdev_zap_top: 35 children[0]: type: 'disk' id: 0 guid: 1443238581175429852 path: '/dev/mfid4p4' whole_disk: 1 DTL: 137 create_txg: 4 com.delphix:vdev_zap_leaf: 131 children[1]: type: 'disk' id: 1 guid: 1865792721003775978 path: '/dev/mfid0p4' whole_disk: 1 DTL: 133 create_txg: 4 com.delphix:vdev_zap_leaf: 37 children[2]: type: 'disk' id: 2 guid: 12541720522827927342 path: '/dev/mfid1p4' whole_disk: 1 DTL: 132 create_txg: 4 com.delphix:vdev_zap_leaf: 38 children[3]: type: 'disk' id: 3 guid: 13053934791777776444 path: '/dev/mfid3p4' whole_disk: 1 DTL: 136 create_txg: 4 com.delphix:vdev_zap_leaf: 135 children[4]: type: 'disk' id: 4 guid: 4432707573898874857 path: '/dev/mfid2p4' whole_disk: 1 DTL: 130 create_txg: 4 com.delphix:vdev_zap_leaf: 40 children[5]: type: 'disk' id: 5 guid: 5106293125005422556 path: '/dev/mfid5p4' whole_disk: 1 DTL: 129 create_txg: 4 com.delphix:vdev_zap_leaf: 41 features_for_read: com.delphix:hole_birth com.delphix:embedded_data borg-new /home/ler $ sudo zpool list -v NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT zroot 10.8T 94.3G 10.7T - 0% 0% 1.00x ONLINE - raidz1 10.8T 94.3G 10.7T - 0% 0% mfid4p4 - - - - - - mfid0p4 - - - - - - mfid1p4 - - - - - - mfid3p4 - - - - - - mfid2p4 - - - - - - mfid5p4 - - - - - - borg-new /home/ler $ sudo zpool get all NAME PROPERTY VALUE SOURCE zroot size 10.8T - zroot capacity 0% - zroot altroot - default zroot health ONLINE - zroot guid 11945658884309024932 default zroot version - default zroot bootfs zroot/ROOT/default local zroot delegation on default zroot autoreplace off default zroot cachefile - default zroot failmode wait default zroot listsnapshots off default zroot autoexpand off default zroot dedupditto 0 default zroot dedupratio 1.00x - zroot free 10.7T - zroot allocated 94.3G - zroot readonly off - zroot comment - default zroot expandsize - - zroot freeing 0 default zroot fragmentation 0% - zroot leaked 0 default zroot feature@async_destroy enabled local zroot feature@empty_bpobj active local zroot feature@lz4_compress active local zroot feature@multi_vdev_crash_dump enabled local zroot feature@spacemap_histogram active local zroot feature@enabled_txg active local zroot feature@hole_birth active local zroot feature@extensible_dataset enabled local zroot feature@embedded_data active local zroot feature@bookmarks enabled local zroot feature@filesystem_limits enabled local zroot feature@large_blocks enabled local zroot feature@sha512 enabled local zroot feature@skein enabled local borg-new /home/ler $ On 01/31/2017 5:22 pm, Steven Hartland wrote: > Yep > > On 31/01/2017 21:49, Larry Rosenman wrote: > > revert the other patch and apply this one? > > On 01/31/2017 3:47 pm, Steven Hartland wrote: Hmm, looks like there's also a bug in the way vdev_min_asize is calculated for raidz as it can and has resulted in child min_asize which won't provided enough space for the parent due to the use of unrounded integer division. > > 1981411579221 * 6 = 11888469475326 < 11888469475328 > > You should have vdev_min_asize: 1981411579222 for your children. > > Updated patch attached, however calculation still isn't 100% reversible so may need work, however it does now ensure that the children will provide enough capacity for min_asize even if all of them are shrunk to their individual min_asize, which I believe previously may not have been the case. > > This isn't related to the incorrect EXPANDSZ output, but would be good if you could confirm it doesn't cause any issues for your pool given its state. > > On 31/01/2017 21:00, Larry Rosenman wrote: > > borg-new /home/ler $ sudo ./vdev-stats.d > Password: > vdev_path: n/a, vdev_max_asize: 0, vdev_asize: 0, vdev_min_asize: 0 > vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: 11947478089728, vdev_min_asize: 11888469475328 > vdev_path: /dev/mfid4p4, vdev_max_asize: 1991245299712, vdev_asize: 1991245299712, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid0p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid1p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid3p4, vdev_max_asize: 1991247921152, vdev_asize: 1991247921152, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid2p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 > vdev_path: /dev/mfid5p4, vdev_max_asize: 1991246348288, vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 > ^C > > borg-new /home/ler $ > > borg-new /home/ler $ sudo zpool list -v > Password: > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > zroot 10.8T 94.3G 10.7T 16.0E 0% 0% 1.00x ONLINE - > raidz1 10.8T 94.3G 10.7T 16.0E 0% 0% > mfid4p4 - - - - - - > mfid0p4 - - - - - - > mfid1p4 - - - - - - > mfid3p4 - - - - - - > mfid2p4 - - - - - - > mfid5p4 - - - - - - > borg-new /home/ler $ > > On 01/31/2017 2:37 pm, Steven Hartland wrote: In that case based on your zpool history I suspect that the original mfid4p4 was the same size as mfid0p4 (1991246348288) but its been replaced with a drive which is (1991245299712), slightly smaller. > > This smaller size results in a max_asize of 1991245299712 * 6 instead of original 1991246348288* 6. > > Now given the way min_asize (the value used to check if the device size is acceptable) is rounded to the the nearest metaslab I believe that replace would be allowed. > https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c#L4947 > > Now the problem is that on open the calculated asize is only updated if its expanding: > https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c#L1424 > > The updated dtrace file outputs vdev_min_asize which should confirm my suspicion about why the replace was allowed. > > On 31/01/2017 19:05, Larry Rosenman wrote: > > I've replaced some disks due to failure, and some of the pariition sizes are different. > > autoexpand is off: > > borg-new /home/ler $ zpool get all zroot > NAME PROPERTY VALUE SOURCE > zroot size 10.8T - > zroot capacity 0% - > zroot altroot - default > zroot health ONLINE - > zroot guid 11945658884309024932 default > zroot version - default > zroot bootfs zroot/ROOT/default local > zroot delegation on default > zroot autoreplace off default > zroot cachefile - default > zroot failmode wait default > zroot listsnapshots off default > zroot autoexpand off default > zroot dedupditto 0 default > zroot dedupratio 1.00x - > zroot free 10.7T - > zroot allocated 94.3G - > zroot readonly off - > zroot comment - default > zroot expandsize 16.0E - > zroot freeing 0 default > zroot fragmentation 0% - > zroot leaked 0 default > zroot feature@async_destroy enabled local > zroot feature@empty_bpobj active local > zroot feature@lz4_compress active local > zroot feature@multi_vdev_crash_dump enabled local > zroot feature@spacemap_histogram active local > zroot feature@enabled_txg active local > zroot feature@hole_birth active local > zroot feature@extensible_dataset enabled local > zroot feature@embedded_data active local > zroot feature@bookmarks enabled local > zroot feature@filesystem_limits enabled local > zroot feature@large_blocks enabled local > zroot feature@sha512 enabled local > zroot feature@skein enabled local > borg-new /home/ler $ > > borg-new /home/ler $ gpart show > => 40 3905945520 mfid0 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid1 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid2 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid3 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 16777216 3 freebsd-swap (8.0G) > 16779880 3889165680 4 freebsd-zfs (1.8T) > > => 40 3905945520 mfid5 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889162240 4 freebsd-zfs (1.8T) > 3905943552 2008 - free - (1.0M) > > => 40 3905945520 mfid4 GPT (1.8T) > 40 1600 1 efi (800K) > 1640 1024 2 freebsd-boot (512K) > 2664 1432 - free - (716K) > 4096 16777216 3 freebsd-swap (8.0G) > 16781312 3889160192 4 freebsd-zfs (1.8T) > 3905941504 4056 - free - (2.0M) > > borg-new /home/ler $ > > this system was built last week, and I **CAN** rebuild it if necessary, but I didn't do anything strange (so I thought :) ) > > On 01/31/2017 12:30 pm, Steven Hartland wrote: Your issue is the reported vdev_max_asize > vdev_asize: > vdev_max_asize: 11947471798272 > vdev_asize: 11947478089728 > > max asize is smaller than asize by 6291456 > > For raidz1 Xsize should be the smallest disk Xsize * disks so: > 1991245299712 * 6 = 11947471798272 > > So your max asize looks right but asize looks too big > > Expand Size is calculated by: > if (vd->vdev_aux == NULL && tvd != NULL && vd->vdev_max_asize != 0) { > vs->vs_esize = P2ALIGN(vd->vdev_max_asize - vd->vdev_asize, > 1ULL << tvd->vdev_ms_shift); > } > > So the question is why is asize too big? > > Given you seem to have some random disk sizes do you have auto expand turned on? > > On 31/01/2017 17:39, Larry Rosenman wrote: vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: 11947478089728 -- Larry Rosenman http://people.freebsd.org/~ler [1] Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 -- Larry Rosenman http://people.freebsd.org/~ler [1] Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 -- Larry Rosenman http://people.freebsd.org/~ler [1] Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 -- Larry Rosenman http://people.freebsd.org/~ler Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 Links: ------ [1] http://people.freebsd.org/%7Eler From owner-freebsd-fs@freebsd.org Wed Feb 1 02:43:53 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B4DC8CCB898 for ; Wed, 1 Feb 2017 02:43:53 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x231.google.com (mail-wm0-x231.google.com [IPv6:2a00:1450:400c:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1FC0ADDD for ; Wed, 1 Feb 2017 02:43:53 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wm0-x231.google.com with SMTP id b65so17523636wmf.0 for ; Tue, 31 Jan 2017 18:43:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to; bh=Onjyfcowgphq4fg01yaHTyaRWptYUCtTfwO+WZ6Vte4=; b=vGQ+aLJqMWAZnCoGg21JyY4S19IZBe6vozGjF93hEckCepqqpW+KXOGfVxmtEALo+E yXE+tXRCgJ8vn/JZghAvLmbfjqg6couzYwEESZIr1Eezw9xtQpX9k5plriP+92Dy3MKc YHcBOYiQvmrNLxuGhQdya8C0/7V1GNfFU2aZH5ci9ZyG+IuW7NygJbjckHrB+dHVpmm8 Avh+shyUX6Mbd/O52nFGcsHYZguZruPBNA8bo0f4jlrIpM3V6PORMwan2xgdrz9mq6P4 OcznTg5MVrJ99T6uxS29mNT8Yv3o215Xjfcu7S1oPVG2oByzeq2N9kC0TbKfEqvKp7/D RAjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=Onjyfcowgphq4fg01yaHTyaRWptYUCtTfwO+WZ6Vte4=; b=sWqfWhnUlCMo/FXq6UhkSKmcuo1jr2dZrWboi1tsy53E59DhY8G22h9c4z9eZ5+9Sk apz1/yASg+xTJnOz6qR6meoctOeqJFJzprpSXacPLz/FoglKK2wvgyEMbtk/bKsz4a7/ NIs7GkTkgi1pVL5YojnMe/6t4F6u51ayzl373euYx/3bZPqFtZqMKjpa/0w1u8FXqqOR Jio6oxLZNHW4bQRDQtiL3QkSW7Hki0xKTps+1ORr6A04V5AbGTZs3bYJo8Rq8p9Rv0B2 6c4Y9+BLDV0y4o2mcYC3jL/kXCS2+iIU/IzLq7cCx3Yi6I7V6ObmIWxXVisvS2cXrSk5 sJTg== X-Gm-Message-State: AIkVDXKvHLT292/VldlTjGTradL3dn+MgBOWQF++Oj7JSEiYk9zFKMxCQhTnKQKd6uoK0TIx X-Received: by 10.223.135.163 with SMTP id b32mr353860wrb.184.1485917030756; Tue, 31 Jan 2017 18:43:50 -0800 (PST) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id b8sm31355833wrb.17.2017.01.31.18.43.49 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 31 Jan 2017 18:43:49 -0800 (PST) Subject: Re: 16.0E ExpandSize? -- New Server To: Larry Rosenman References: <00db0ab7243ce6368c246ae20f9c075a@FreeBSD.org> <1a69057c-dc59-9b78-9762-4f98a071105e@multiplay.co.uk> <35a9034f91542bb1329ac5104bf3b773@FreeBSD.org> <76fc9505-f681-0de0-fe0c-5624b29de321@multiplay.co.uk> <22e1bfc5840d972cf93643733682cda1@FreeBSD.org> <8a710dc75c129f58b0372eeaeca575b5@FreeBSD.org> <96534515-4fcb-774e-a599-8d48aec930cd@multiplay.co.uk> Cc: Freebsd fs From: Steven Hartland Message-ID: <8387d38f-3185-8c07-396b-602c708002a6@multiplay.co.uk> Date: Wed, 1 Feb 2017 02:43:51 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Feb 2017 02:43:53 -0000 Thanks I've put a PR in upstream to get some eyes on the fix. https://github.com/openzfs/openzfs/pull/296 If no objections are raised to the approach I've used I'll commit the fix to HEAD too. On 01/02/2017 02:31, Larry Rosenman wrote: > > no grief that I can see: > > borg-new /home/ler $ sudo zdb > Password: > zroot: > version: 5000 > name: 'zroot' > state: 0 > txg: 96143 > pool_guid: 11945658884309024932 > hostid: 3619181042 > hostname: '' > com.delphix:has_per_vdev_zaps > vdev_children: 1 > vdev_tree: > type: 'root' > id: 0 > guid: 11945658884309024932 > create_txg: 4 > children[0]: > type: 'raidz' > id: 0 > guid: 7596925654112466913 > nparity: 1 > metaslab_array: 42 > metaslab_shift: 36 > ashift: 12 > asize: 11947471798272 > is_log: 0 > create_txg: 4 > com.delphix:vdev_zap_top: 35 > children[0]: > type: 'disk' > id: 0 > guid: 1443238581175429852 > path: '/dev/mfid4p4' > whole_disk: 1 > DTL: 137 > create_txg: 4 > com.delphix:vdev_zap_leaf: 131 > children[1]: > type: 'disk' > id: 1 > guid: 1865792721003775978 > path: '/dev/mfid0p4' > whole_disk: 1 > DTL: 133 > create_txg: 4 > com.delphix:vdev_zap_leaf: 37 > children[2]: > type: 'disk' > id: 2 > guid: 12541720522827927342 > path: '/dev/mfid1p4' > whole_disk: 1 > DTL: 132 > create_txg: 4 > com.delphix:vdev_zap_leaf: 38 > children[3]: > type: 'disk' > id: 3 > guid: 13053934791777776444 > path: '/dev/mfid3p4' > whole_disk: 1 > DTL: 136 > create_txg: 4 > com.delphix:vdev_zap_leaf: 135 > children[4]: > type: 'disk' > id: 4 > guid: 4432707573898874857 > path: '/dev/mfid2p4' > whole_disk: 1 > DTL: 130 > create_txg: 4 > com.delphix:vdev_zap_leaf: 40 > children[5]: > type: 'disk' > id: 5 > guid: 5106293125005422556 > path: '/dev/mfid5p4' > whole_disk: 1 > DTL: 129 > create_txg: 4 > com.delphix:vdev_zap_leaf: 41 > features_for_read: > com.delphix:hole_birth > com.delphix:embedded_data > borg-new /home/ler $ sudo zpool list -v > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > zroot 10.8T 94.3G 10.7T - 0% 0% 1.00x ONLINE - > raidz1 10.8T 94.3G 10.7T - 0% 0% > mfid4p4 - - - - - - > mfid0p4 - - - - - - > mfid1p4 - - - - - - > mfid3p4 - - - - - - > mfid2p4 - - - - - - > mfid5p4 - - - - - - > borg-new /home/ler $ sudo zpool get all > NAME PROPERTY VALUE SOURCE > zroot size 10.8T - > zroot capacity 0% - > zroot altroot - default > zroot health ONLINE - > zroot guid 11945658884309024932 default > zroot version - default > zroot bootfs zroot/ROOT/default local > zroot delegation on default > zroot autoreplace off default > zroot cachefile - default > zroot failmode wait default > zroot listsnapshots off default > zroot autoexpand off default > zroot dedupditto 0 default > zroot dedupratio 1.00x - > zroot free 10.7T - > zroot allocated 94.3G - > zroot readonly off - > zroot comment - default > zroot expandsize - - > zroot freeing 0 default > zroot fragmentation 0% - > zroot leaked 0 default > zroot feature@async_destroy enabled local > zroot feature@empty_bpobj active local > zroot feature@lz4_compress active local > zroot feature@multi_vdev_crash_dump enabled local > zroot feature@spacemap_histogram active local > zroot feature@enabled_txg active local > zroot feature@hole_birth active local > zroot feature@extensible_dataset enabled local > zroot feature@embedded_data active local > zroot feature@bookmarks enabled local > zroot feature@filesystem_limits enabled local > zroot feature@large_blocks enabled local > zroot feature@sha512 enabled local > zroot feature@skein enabled local > borg-new /home/ler $ > > > > On 01/31/2017 5:22 pm, Steven Hartland wrote: > >> Yep >> >> On 31/01/2017 21:49, Larry Rosenman wrote: >>> >>> revert the other patch and apply this one? >>> >>> >>> >>> On 01/31/2017 3:47 pm, Steven Hartland wrote: >>> >>> Hmm, looks like there's also a bug in the way vdev_min_asize is >>> calculated for raidz as it can and has resulted in child >>> min_asize which won't provided enough space for the parent due >>> to the use of unrounded integer division. >>> >>> 1981411579221 * 6 = 11888469475326 < 11888469475328 >>> >>> You should have vdev_min_asize: 1981411579222 for your children. >>> >>> Updated patch attached, however calculation still isn't 100% >>> reversible so may need work, however it does now ensure that the >>> children will provide enough capacity for min_asize even if all >>> of them are shrunk to their individual min_asize, which I >>> believe previously may not have been the case. >>> >>> This isn't related to the incorrect EXPANDSZ output, but would >>> be good if you could confirm it doesn't cause any issues for >>> your pool given its state. >>> >>> On 31/01/2017 21:00, Larry Rosenman wrote: >>> >>> borg-new /home/ler $ sudo ./vdev-stats.d >>> Password: >>> vdev_path: n/a, vdev_max_asize: 0, vdev_asize: 0, >>> vdev_min_asize: 0 >>> vdev_path: n/a, vdev_max_asize: 11947471798272, vdev_asize: >>> 11947478089728, vdev_min_asize: 11888469475328 >>> vdev_path: /dev/mfid4p4, vdev_max_asize: 1991245299712, >>> vdev_asize: 1991245299712, vdev_min_asize: 1981411579221 >>> vdev_path: /dev/mfid0p4, vdev_max_asize: 1991246348288, >>> vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 >>> vdev_path: /dev/mfid1p4, vdev_max_asize: 1991246348288, >>> vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 >>> vdev_path: /dev/mfid3p4, vdev_max_asize: 1991247921152, >>> vdev_asize: 1991247921152, vdev_min_asize: 1981411579221 >>> vdev_path: /dev/mfid2p4, vdev_max_asize: 1991246348288, >>> vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 >>> vdev_path: /dev/mfid5p4, vdev_max_asize: 1991246348288, >>> vdev_asize: 1991246348288, vdev_min_asize: 1981411579221 >>> ^C >>> >>> borg-new /home/ler $ >>> >>> >>> borg-new /home/ler $ sudo zpool list -v >>> Password: >>> NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT >>> zroot 10.8T 94.3G 10.7T 16.0E 0% 0% 1.00x ONLINE - >>> raidz1 10.8T 94.3G 10.7T 16.0E 0% 0% >>> mfid4p4 - - - - - - >>> mfid0p4 - - - - - - >>> mfid1p4 - - - - - - >>> mfid3p4 - - - - - - >>> mfid2p4 - - - - - - >>> mfid5p4 - - - - - - >>> borg-new /home/ler $ >>> >>> >>> On 01/31/2017 2:37 pm, Steven Hartland wrote: >>> >>> In that case based on your zpool history I suspect that >>> the original mfid4p4 was the same size as mfid0p4 >>> (1991246348288) but its been replaced with a drive which >>> is (1991245299712), slightly smaller. >>> >>> This smaller size results in a max_asize of >>> 1991245299712 * 6 instead of original 1991246348288* 6. >>> >>> Now given the way min_asize (the value used to check if >>> the device size is acceptable) is rounded to the the >>> nearest metaslab I believe that replace would be allowed. >>> https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c#L4947 >>> >>> Now the problem is that on open the calculated asize is >>> only updated if its expanding: >>> https://github.com/freebsd/freebsd/blob/master/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c#L1424 >>> >>> The updated dtrace file outputs vdev_min_asize which >>> should confirm my suspicion about why the replace was >>> allowed. >>> >>> On 31/01/2017 19:05, Larry Rosenman wrote: >>> >>> I've replaced some disks due to failure, and some of >>> the pariition sizes are different. >>> >>> >>> autoexpand is off: >>> >>> borg-new /home/ler $ zpool get all zroot >>> NAME PROPERTY VALUE SOURCE >>> zroot size 10.8T - >>> zroot capacity 0% - >>> zroot altroot - default >>> zroot health ONLINE - >>> zroot guid 11945658884309024932 default >>> zroot version - default >>> zroot bootfs zroot/ROOT/default local >>> zroot delegation on default >>> zroot autoreplace off default >>> zroot cachefile - default >>> zroot failmode wait default >>> zroot listsnapshots off default >>> zroot autoexpand off default >>> zroot dedupditto 0 default >>> zroot dedupratio 1.00x - >>> zroot free 10.7T - >>> zroot allocated 94.3G - >>> zroot readonly off - >>> zroot comment - default >>> zroot expandsize 16.0E - >>> zroot freeing 0 default >>> zroot fragmentation 0% - >>> zroot leaked 0 default >>> zroot feature@async_destroy enabled local >>> zroot feature@empty_bpobj active local >>> zroot feature@lz4_compress active local >>> zroot feature@multi_vdev_crash_dump enabled local >>> zroot feature@spacemap_histogram active local >>> zroot feature@enabled_txg active local >>> zroot feature@hole_birth active local >>> zroot feature@extensible_dataset enabled local >>> zroot feature@embedded_data active local >>> zroot feature@bookmarks enabled local >>> zroot feature@filesystem_limits enabled local >>> zroot feature@large_blocks enabled local >>> zroot feature@sha512 enabled local >>> zroot feature@skein enabled local >>> borg-new /home/ler $ >>> >>> >>> borg-new /home/ler $ gpart show >>> => 40 3905945520 mfid0 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid1 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid2 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid3 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 16777216 3 freebsd-swap (8.0G) >>> 16779880 3889165680 4 freebsd-zfs (1.8T) >>> >>> => 40 3905945520 mfid5 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889162240 4 freebsd-zfs (1.8T) >>> 3905943552 2008 - free - (1.0M) >>> >>> => 40 3905945520 mfid4 GPT (1.8T) >>> 40 1600 1 efi (800K) >>> 1640 1024 2 freebsd-boot (512K) >>> 2664 1432 - free - (716K) >>> 4096 16777216 3 freebsd-swap (8.0G) >>> 16781312 3889160192 4 freebsd-zfs (1.8T) >>> 3905941504 4056 - free - (2.0M) >>> >>> borg-new /home/ler $ >>> >>> >>> this system was built last week, and I **CAN** >>> rebuild it if necessary, but I didn't do anything >>> strange (so I thought :) ) >>> >>> >>> >>> >>> On 01/31/2017 12:30 pm, Steven Hartland wrote: >>> >>> Your issue is the reported vdev_max_asize > >>> vdev_asize: >>> vdev_max_asize: 11947471798272 >>> vdev_asize: 11947478089728 >>> >>> max asize is smaller than asize by 6291456 >>> >>> For raidz1 Xsize should be the smallest disk >>> Xsize * disks so: >>> 1991245299712 * 6 = 11947471798272 >>> >>> So your max asize looks right but asize looks >>> too big >>> >>> Expand Size is calculated by: >>> if (vd->vdev_aux == NULL && tvd != NULL && >>> vd->vdev_max_asize != 0) { >>> vs->vs_esize = P2ALIGN(vd->vdev_max_asize - >>> vd->vdev_asize, >>> 1ULL << tvd->vdev_ms_shift); >>> } >>> >>> So the question is why is asize too big? >>> >>> Given you seem to have some random disk sizes do >>> you have auto expand turned on? >>> >>> On 31/01/2017 17:39, Larry Rosenman wrote: >>> >>> vdev_path: n/a, vdev_max_asize: >>> 11947471798272, vdev_asize: 11947478089728 >>> >>> >>> -- >>> Larry Rosenman http://people.freebsd.org/~ler >>> >>> Phone: +1 214-642-9640 E-Mail: >>> ler@FreeBSD.org >>> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 >>> >>> >>> -- >>> Larry Rosenman http://people.freebsd.org/~ler >>> >>> Phone: +1 214-642-9640 E-Mail: >>> ler@FreeBSD.org >>> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 >>> >>> >>> -- >>> Larry Rosenman http://people.freebsd.org/~ler >>> >>> Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org >>> >>> US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 > > > -- > Larry Rosenman http://people.freebsd.org/~ler > > Phone: +1 214-642-9640 E-Mail: ler@FreeBSD.org > > US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281 From owner-freebsd-fs@freebsd.org Wed Feb 1 09:22:09 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B2489CCB46E for ; Wed, 1 Feb 2017 09:22:09 +0000 (UTC) (envelope-from paul.hargreaves@technowizardry.co.uk) Received: from mail-wm0-x22e.google.com (mail-wm0-x22e.google.com [IPv6:2a00:1450:400c:c09::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 539F1C6D for ; Wed, 1 Feb 2017 09:22:09 +0000 (UTC) (envelope-from paul.hargreaves@technowizardry.co.uk) Received: by mail-wm0-x22e.google.com with SMTP id b65so29442068wmf.0 for ; Wed, 01 Feb 2017 01:22:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=technowizardry.co.uk; s=google; h=from:content-transfer-encoding:mime-version:subject:message-id:date :to; bh=WAZZ7LcgsxEKa3zHPutIRvHc9T7/niMQawJRFD+P/Rk=; b=oaEIE3hxznO+fAK0xl+97j6w+MM5lKKfIjH1UjYTlwhXL9C7L0n1pKpY5IwrELWvoi rU9ur9sLdQBX4B+URA9rR+eUT4tffQBKEqL26pBzXDcZHSYyjwOciSswiW0zkibF5bfd fOrab4OUhhgjKus/91x1BOYM7FtMvJIMHNn3g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:content-transfer-encoding:mime-version :subject:message-id:date:to; bh=WAZZ7LcgsxEKa3zHPutIRvHc9T7/niMQawJRFD+P/Rk=; b=s7IUPV441azMUk0ULfD+eZ6YOh7O4osk2rXXFv0lMc77AfOrKQ+hW2Y4h0XHWyi458 dMZh8GoUcg7cQLoo1tCvs5K/TfUDKe5dbq8pSubR3sTJ08rvuasW9G3rpZ66Dbiu4I7/ AyrsbrJnbojb6L5ciYx2D7xGADg4v57XwwkHRyz1qXVm+Yy1nAQtePKhA8xGVue5D7Dg zS3wXW5VL4rD1CodkUYcZOinCyc8C7NoTG1Zi5wTBs1MlpHn3FXoRXkqNWsIiPN5FWeA YgQtwCjCzDKU5pmdtvFtxXxhtG/HczutavH4qlzvKgFFD2jtGylMsYS3DvnFbn7T5mqA hxmw== X-Gm-Message-State: AIkVDXL+xTJxiPyCZphfbwF/oBpDy1tK++HnuhYUak6k/OU+l0afvSq/E2kFjxoSw/3ZXg== X-Received: by 10.28.58.204 with SMTP id h195mr22005963wma.116.1485940926072; Wed, 01 Feb 2017 01:22:06 -0800 (PST) Received: from [6.6.6.151] (host5-81-123-215.range5-81.btcentralplus.com. [5.81.123.215]) by smtp.gmail.com with ESMTPSA id o81sm28379072wmb.14.2017.02.01.01.22.05 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 Feb 2017 01:22:05 -0800 (PST) From: Paul Hargreaves Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Subject: zpool upgrade instructions aren't complete enough - gpart operation not permitted Message-Id: Date: Wed, 1 Feb 2017 09:22:04 +0000 To: freebsd-fs@freebsd.org X-Mailer: Apple Mail (2.3259) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Feb 2017 09:22:09 -0000 Hi there, Just upgraded from 10.2 to 11.0. As part of that, did zpool upgrade: root@zfsbackup:~ # zpool upgrade zroot This system supports ZFS pool feature flags. Enabled the following features on 'zroot': sha512 skein If you boot from pool 'zroot', don't forget to update boot code. Assuming you use GPT partitioning and da0 is your boot disk the following command will do it: gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0 Ok, I think I=E2=80=99m using GPT partitioning and I think I boot from = da0 (specifically, zpool show shows zroot is da0s1d), but how to tell? root@zfsbackup:~ # gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 = da0 gpart: /dev/da0s1: Operation not permitted Hmm=E2=80=A6 ok. Google-fu only found this: https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot which doesn=E2=80=99t help since it=E2=80=99s out of date. Is there a different magic incantation I should be using? Also tried using the da0s1d but then get part: No such geom: da0s1d. Regards Paul= From owner-freebsd-fs@freebsd.org Wed Feb 1 09:30:47 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 51BDFCCB5D5 for ; Wed, 1 Feb 2017 09:30:47 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: from mail-wm0-x229.google.com (mail-wm0-x229.google.com [IPv6:2a00:1450:400c:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 00269E4D for ; Wed, 1 Feb 2017 09:30:46 +0000 (UTC) (envelope-from amvandemore@gmail.com) Received: by mail-wm0-x229.google.com with SMTP id b65so29782872wmf.0 for ; Wed, 01 Feb 2017 01:30:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=XyNX9t+hxITbikvmcY4pcZVs5L1/RsZ/YUIb6gwtwac=; b=qYsag/oF17N6SeBTmqLv2TNEfLdKIMsamb/nXO25Ih2SSVBU2yusOLBKPPlDzj7I7q 6i0BChyGh9LGYe4e1U9VxP8LiqVkieMIEqhjqqPFyGhDUU/xeC1ZHNuknfgP5PMtqUd4 5ZKb4q+dxzFJJe+WBVVr1UHUq957dc27D0LCbCeXnkMS5uD1K41OsNXptTBkFX8ilbMR LiHyvEXLmopZ/lrMdhKEVxd5F/9omIKZ6hkakrm4XA51w/C53KFoFMEhmEY0Un2edRlu fgc550iv2Xc7umNmcjB/BUtNxJuU+jIpk6V315dG2h4DSFrWwcw5wLWlIYNAUA5g1Ggi siuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=XyNX9t+hxITbikvmcY4pcZVs5L1/RsZ/YUIb6gwtwac=; b=HTvIYv+9c+bUfuZdl3YDkIVBObZapZmu+aVwmSdkHU7NODs1vdRlhvbvo6UB1uXXv9 hI4eJVSRhOvGduou7JoPBz+42DZIRldmPTDrOQ57vg8M6UEW4YUt4TjARW5+3vt/PO3a 6qZTzcIMHNffMOeKi78kOwOt/lhTMLi5OP7fiFNOwBp9ckERwfskfs4rL0A73e9flFns FDZd/k7z2bZEKqaJgb5FLdTNcRQ1EtfEIQ3+R8CSiNRW/d4LY6cPw1ZKh57/K0v986Nh 29NMrAY1ORgd4FE5AvF4R/VzEHed8b8NXJkWgjNEWe+MitSVhQ6yEu8R7E5Oa7sdh5uq voqA== X-Gm-Message-State: AIkVDXJo3HtV73Ni7fbC0FfNp5+wrYPph1wyNxYJ+JHQ/SKNnVqYuzE6tQb+A/VdgJgF2PsXHHKPme8UKESCHQ== X-Received: by 10.28.19.207 with SMTP id 198mr1883372wmt.70.1485941445256; Wed, 01 Feb 2017 01:30:45 -0800 (PST) MIME-Version: 1.0 Received: by 10.80.164.65 with HTTP; Wed, 1 Feb 2017 01:30:44 -0800 (PST) In-Reply-To: References: From: Adam Vande More Date: Wed, 1 Feb 2017 03:30:44 -0600 Message-ID: Subject: Re: zpool upgrade instructions aren't complete enough - gpart operation not permitted To: Paul Hargreaves Cc: freebsd-fs Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Feb 2017 09:30:47 -0000 On Wed, Feb 1, 2017 at 3:22 AM, Paul Hargreaves < paul.hargreaves@technowizardry.co.uk> wrote: > Hi there, > > Just upgraded from 10.2 to 11.0. > As part of that, did zpool upgrade: > > root@zfsbackup:~ # zpool upgrade zroot > This system supports ZFS pool feature flags. > > Enabled the following features on 'zroot': > sha512 > skein > > If you boot from pool 'zroot', don't forget to update boot code. > Assuming you use GPT partitioning and da0 is your boot disk > the following command will do it: > > gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0 > > > Ok, I think I=E2=80=99m using GPT partitioning and I think I boot from da= 0 > (specifically, zpool show shows zroot is da0s1d), but how to tell? > > root@zfsbackup:~ # gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 > da0 > gpart: /dev/da0s1: Operation not permitted > What is the output of: # gpart list da0 --=20 Adam From owner-freebsd-fs@freebsd.org Wed Feb 1 16:37:25 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C9AE2CCB02A for ; Wed, 1 Feb 2017 16:37:25 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-qt0-x234.google.com (mail-qt0-x234.google.com [IPv6:2607:f8b0:400d:c0d::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 86BDF1F60 for ; Wed, 1 Feb 2017 16:37:25 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: by mail-qt0-x234.google.com with SMTP id w20so201858441qtb.1 for ; Wed, 01 Feb 2017 08:37:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=5v7C2IieeEa0qCa+id8WMV+1QtLV2oDECfseZLAS2KA=; b=A0//CuS09F78CxQ1snbBTCRBc85btPekf8A1GEo2sNRZUk2BQybafHn5HpacYsYcUt Z+6k0QF+5YXkksJlYC1THXZdzHG8CeYQvsBzJ8xZ7BxdZzp7M+eBSE4AUXMyP/1aODUO Vv+GVQTDYXNN7TQc8+WzOuHoyG5t8SXNtmnjuv5Qq/ko33OdkqxOyFJJz/jticduA0tQ eXCbCqy+yOf3UN1clqBsmaj6semvesyYp6e5BFXkeVpGEefaJJMBvQD/eOw9JDO31ic/ pOqI766vQjroPWksTEPNqMR6Czpoqjt9ai+/pLYQd7pP76Kis+/oEuZLIXsLC+0vS11x kt6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=5v7C2IieeEa0qCa+id8WMV+1QtLV2oDECfseZLAS2KA=; b=VMpjcaErVmB5U2P4cYLCTdYOIFVDGKdKbILRtHAp+jq66LcFNNddmO1l94QQDMh36u /N9cX9DRJYfsNZ/zJyGSOEvwv0gAd+kw5Wnynr5/TCOFj2GNPGda5zVBRzNDDEYaEm6l h/TSPXibBwGkOWlRciOPXVpxbm7S5s5aqZZ0zk+cJN+f0VuCEtlZhk4+SEiqrBu0WoOy tth/Q7rCDCO82LjHkfZWHcYbhBHhTwgSGXbq2NMvZGrmRc8+ZuIFhkBzDhHcXZu2wEPf Myu6vxhaqfofHUDlhtM174VFMZXcFr2f8+FfkKuAGwhTG/pSebg3y3ByOnSzh5MLFbG6 aVyA== X-Gm-Message-State: AMke39kL8hboE6QuZGZ8niVR/mqh9zM+0wcucmxXr+xyXURJnHW4uovjX/0N2HIaZUOIVy2052pd2M1mtWIBzw== X-Received: by 10.55.66.67 with SMTP id p64mr3459852qka.273.1485967043270; Wed, 01 Feb 2017 08:37:23 -0800 (PST) MIME-Version: 1.0 Received: by 10.140.28.202 with HTTP; Wed, 1 Feb 2017 08:37:22 -0800 (PST) In-Reply-To: References: From: Freddie Cash Date: Wed, 1 Feb 2017 08:37:22 -0800 Message-ID: Subject: Re: zpool upgrade instructions aren't complete enough - gpart operation not permitted To: Paul Hargreaves Cc: FreeBSD Filesystems Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Feb 2017 16:37:25 -0000 On Wed, Feb 1, 2017 at 1:22 AM, Paul Hargreaves < paul.hargreaves@technowizardry.co.uk> wrote: > Hi there, > > Just upgraded from 10.2 to 11.0. > As part of that, did zpool upgrade: > > root@zfsbackup:~ # zpool upgrade zroot > This system supports ZFS pool feature flags. > > Enabled the following features on 'zroot': > sha512 > skein > > If you boot from pool 'zroot', don't forget to update boot code. > Assuming you use GPT partitioning and da0 is your boot disk > the following command will do it: > > gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0 > > Ok, I think I=E2=80=99m using GPT partitioning and I think I boot from da= 0 > (specifically, zpool show shows zroot is da0s1d), but how to tell? > =E2=80=8BGPT partitioned disks will have device nodes like da0p1 (meaning p= artition 1). MBR partitioned disks will have device nodes like da0s1d (meaning partitiong d in slice 1). You can use "gpart show da0" to get more information. IOW, you have MBR partitioned disks, and should not use the suggested command as it only applies to GPT partitioned disks. --=20 Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@freebsd.org Wed Feb 1 21:38:05 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 569D6CCB117 for ; Wed, 1 Feb 2017 21:38:05 +0000 (UTC) (envelope-from paul.hargreaves@technowizardry.co.uk) Received: from mail-wm0-x235.google.com (mail-wm0-x235.google.com [IPv6:2a00:1450:400c:c09::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0A17E187F for ; Wed, 1 Feb 2017 21:38:04 +0000 (UTC) (envelope-from paul.hargreaves@technowizardry.co.uk) Received: by mail-wm0-x235.google.com with SMTP id r141so58879823wmg.1 for ; Wed, 01 Feb 2017 13:38:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=technowizardry.co.uk; s=google; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=YYCrl8NfW7zujhDJj1UTcG0l2Nov961INtze4aaKQ3s=; b=gyl5RcNyf2uAODI0f9C8Y+nkdJjxhOPgDtmlecvQA0axPqpktTiJjNinh75ohzflen 6kdQWHz4rOP2Fg8QMfERFpfWQAdWrtvfaufNxZnRS4TUBvN+mYU7h8pjPLonnbyMDKLL zcW5wsIxSY6uDDycIxYRyzP9sww0kSXftmVAU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=YYCrl8NfW7zujhDJj1UTcG0l2Nov961INtze4aaKQ3s=; b=cCF6rZgjXmSZhnGKCC7LNXxsj8HkF/5JdITtoikFpV5F0qnBYNITJCWE+Lm34gTMH7 8gyFB/H1BU8PuG3Oyi+gRMfE2Gethyj5utcLPynK/Bh+X3/+iWA7XoUR5E56nPpTd3gL CC2kfO8ml+SUev9+kGwhY6Yn+P4CrC5PR8u7uxnAAF+r85rEHUHHnMpgHwBR6bC3ioTh y6YJGnhm/p7DZfGumDYaeR6h1Da+utRH+GO6EPnpzzN2eSQVltdb2M6WGvUIw9RSi27b cnyxeCbyDpro+K/eGF+u2XRe4RnYmKiq33gmd1VphkKdZRJDG0uAnL7WVpoAxurTDA6C mENw== X-Gm-Message-State: AIkVDXILEwVplIq2f9mgDzrw82kde1FwWcGppz6TNG0KWRsWzHDo6intb1LWYhrtflhz6w== X-Received: by 10.223.161.194 with SMTP id v2mr4252896wrv.144.1485985082418; Wed, 01 Feb 2017 13:38:02 -0800 (PST) Received: from [6.6.6.151] (host5-81-123-215.range5-81.btcentralplus.com. [5.81.123.215]) by smtp.gmail.com with ESMTPSA id e6sm36160982wrc.30.2017.02.01.13.38.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 01 Feb 2017 13:38:01 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\)) Subject: Re: zpool upgrade instructions aren't complete enough - gpart operation not permitted From: Paul Hargreaves In-Reply-To: Date: Wed, 1 Feb 2017 21:37:59 +0000 Cc: FreeBSD Filesystems Content-Transfer-Encoding: quoted-printable Message-Id: <1CDA43BF-107B-4C67-90A6-18475FEF16CB@technowizardry.co.uk> References: To: Freddie Cash X-Mailer: Apple Mail (2.3259) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 01 Feb 2017 21:38:05 -0000 Machine =E2=80=98a=E2=80=99 $ gpart show da0 =3D> 63 41942977 da0 MBR (20G) 63 41942943 1 freebsd [active] (20G) 41943006 34 - free - (17K) $=20 So yes, that looks like MBR, so I=E2=80=99ll ignore the message on this = machine. Machine =E2=80=98b=E2=80=99 (joy of home labs) $ dmesg | grep GEOM GEOM: da4: the primary GPT table is corrupt or invalid. GEOM: da4: using the secondary instead -- recovery strongly advised. GEOM: da5: the primary GPT table is corrupt or invalid. GEOM: da5: using the secondary instead -- recovery strongly advised. GEOM: diskid/DISK-S21DNSAG229555L%20%20%20%20%20: the primary GPT table = is corrupt or invalid. GEOM: diskid/DISK-S21DNSAG229555L%20%20%20%20%20: using the secondary = instead -- recovery strongly advised. GEOM: diskid/DISK-S21DNSAG225964H%20%20%20%20%20: the primary GPT table = is corrupt or invalid. GEOM: diskid/DISK-S21DNSAG225964H%20%20%20%20%20: using the secondary = instead -- recovery strongly advised. GEOM: da4: the primary GPT table is corrupt or invalid. GEOM: da4: using the secondary instead -- recovery strongly advised. GEOM: da5: the primary GPT table is corrupt or invalid. GEOM: da5: using the secondary instead -- recovery strongly advised. GEOM: diskid/DISK-S21DNSAG229555L%20%20%20%20%20: the primary GPT table = is corrupt or invalid. GEOM: diskid/DISK-S21DNSAG229555L%20%20%20%20%20: using the secondary = instead -- recovery strongly advised. GEOM: diskid/DISK-S21DNSAG225964H%20%20%20%20%20: the primary GPT table = is corrupt or invalid. GEOM: diskid/DISK-S21DNSAG225964H%20%20%20%20%20: using the secondary = instead -- recovery strongly advised. $ gpart show da4 gpart: No such geom: da4. $ gpart show da5 gpart: No such geom: da5. $=20 $ gpart show =3D> 34 41942973 da0 GPT (20G) 34 6 - free - (3.0K) 40 1024 1 freebsd-boot (512K) 1064 984 - free - (492K) 2048 4194304 2 freebsd-swap (2.0G) 4196352 37744640 3 freebsd-zfs (18G) 41940992 2015 - free - (1.0M) $=20 $ gpart list Geom name: da0 modified: false state: OK fwheads: 255 fwsectors: 63 last: 41943006 first: 34 entries: 128 scheme: GPT Providers: 1. Name: da0p1 Mediasize: 524288 (512K) Sectorsize: 512 Stripesize: 0 Stripeoffset: 20480 Mode: r0w0e0 rawuuid: c07ec6ba-6523-11e5-9018-000c29e22a2a rawtype: 83bd6b9d-7f41-11dc-be0b-001560b84f0f label: gptboot0 length: 524288 offset: 20480 type: freebsd-boot index: 1 end: 1063 start: 40 2. Name: da0p2 Mediasize: 2147483648 (2.0G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 1048576 Mode: r1w1e0 rawuuid: c081b168-6523-11e5-9018-000c29e22a2a rawtype: 516e7cb5-6ecf-11d6-8ff8-00022d09712b label: swap0 length: 2147483648 offset: 1048576 type: freebsd-swap index: 2 end: 4196351 start: 2048 3. Name: da0p3 Mediasize: 19325255680 (18G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 2148532224 Mode: r1w1e1 rawuuid: c0846dd6-6523-11e5-9018-000c29e22a2a rawtype: 516e7cba-6ecf-11d6-8ff8-00022d09712b label: zfs0 length: 19325255680 offset: 2148532224 type: freebsd-zfs index: 3 end: 41940991 start: 4196352 Consumers: 1. Name: da0 Mediasize: 21474836480 (20G) Sectorsize: 512 Mode: r2w2e3 $=20 $ zpool status fast pool: fast state: ONLINE status: Some supported features are not enabled on the pool. The pool = can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not = support the features. See zpool-features(7) for details. scan: scrub repaired 0 in 0h4m with 0 errors on Tue Jan 31 22:18:25 = 2017 config: NAME STATE READ = WRITE CKSUM fast ONLINE 0 = 0 0 mirror-0 ONLINE 0 = 0 0 diskid/DISK-S21DNSAG225964H%20%20%20%20%20 ONLINE 0 = 0 0 diskid/DISK-S21DNSAG229555L%20%20%20%20%20 ONLINE 0 = 0 0 errors: No known data errors $=20 Those two SSDs aren=E2=80=99t zroot and are part of an zpool so for = those also should I ignore the GEOM errors? I haven=E2=80=99t bothered upgrading this one yet but am about to. Regards Paul > On 1 Feb 2017, at 16:37, Freddie Cash wrote: >=20 > On Wed, Feb 1, 2017 at 1:22 AM, Paul Hargreaves = wrote: > Hi there, >=20 > Just upgraded from 10.2 to 11.0. > As part of that, did zpool upgrade: >=20 > root@zfsbackup:~ # zpool upgrade zroot > This system supports ZFS pool feature flags. >=20 > Enabled the following features on 'zroot': > sha512 > skein >=20 > If you boot from pool 'zroot', don't forget to update boot code. > Assuming you use GPT partitioning and da0 is your boot disk > the following command will do it: >=20 > gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0 >=20 > Ok, I think I=E2=80=99m using GPT partitioning and I think I boot from = da0 (specifically, zpool show shows zroot is da0s1d), but how to tell? >=20 > =E2=80=8BGPT partitioned disks will have device nodes like da0p1 = (meaning partition 1). >=20 > MBR partitioned disks will have device nodes like da0s1d (meaning = partitiong d in slice 1). >=20 > You can use "gpart show da0" to get more information. >=20 > IOW, you have MBR partitioned disks, and should not use the suggested = command as it only applies to GPT partitioned disks. >=20 > --=20 > Freddie Cash > fjwcash@gmail.com From owner-freebsd-fs@freebsd.org Thu Feb 2 11:53:09 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5A604CCC8D8 for ; Thu, 2 Feb 2017 11:53:09 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2086BA1A for ; Thu, 2 Feb 2017 11:53:09 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1cZFwm-000HOo-TP for freebsd-fs@freebsd.org; Thu, 02 Feb 2017 14:53:04 +0300 Date: Thu, 2 Feb 2017 14:53:04 +0300 From: Slawa Olhovchenkov To: freebsd-fs@freebsd.org Subject: ZFS: mkdir: File too large Message-ID: <20170202115304.GA26493@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Feb 2017 11:53:09 -0000 I am got strange error on stable/11: # mkdir -p /tank/1 mkdir: /tank/1: File too large What is wrong? # zfs list tank NAME USED AVAIL REFER MOUNTPOINT tank 107T 4.96T 171K /tank # zpool list -v NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT tank 130T 120T 9.65T - 11% 92% 1.00x ONLINE - raidz1 32.5T 28.8T 3.68T - 8% 88% ada2 - - - - - - ada3 - - - - - - ada4 - - - - - - ada5 - - - - - - da0 - - - - - - da1 - - - - - - da30 - - - - - - da31 - - - - - - ada0 - - - - - - raidz1 32.5T 30.6T 1.92T - 13% 94% ada1 - - - - - - da2 - - - - - - da3 - - - - - - da4 - - - - - - da5 - - - - - - da6 - - - - - - da7 - - - - - - da8 - - - - - - da9 - - - - - - raidz1 32.5T 30.6T 1.91T - 13% 94% da10 - - - - - - da11 - - - - - - da12 - - - - - - da13 - - - - - - da14 - - - - - - da15 - - - - - - da16 - - - - - - da17 - - - - - - da18 - - - - - - raidz1 32.5T 30.4T 2.13T - 12% 93% da19 - - - - - - da20 - - - - - - da21 - - - - - - da22 - - - - - - da23 - - - - - - da24 - - - - - - da25 - - - - - - da26 - - - - - - da27 - - - - - - # zpool status -v pool: tank state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(7) for details. scan: scrub repaired 0 in 7h7m with 0 errors on Sun Jul 31 04:39:17 2016 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 ada2 ONLINE 0 0 0 ada3 ONLINE 0 0 0 ada4 ONLINE 0 0 0 ada5 ONLINE 0 0 0 da0 ONLINE 0 0 0 da1 ONLINE 0 0 0 da30 ONLINE 0 0 0 da31 ONLINE 0 0 0 ada0 ONLINE 0 0 0 raidz1-1 ONLINE 0 0 0 ada1 ONLINE 0 0 0 da2 ONLINE 0 0 0 da3 ONLINE 0 0 0 da4 ONLINE 0 0 0 da5 ONLINE 0 0 0 da6 ONLINE 0 0 0 da7 ONLINE 0 0 0 da8 ONLINE 0 0 0 da9 ONLINE 0 0 0 raidz1-2 ONLINE 0 0 0 da10 ONLINE 0 0 0 da11 ONLINE 0 0 0 da12 ONLINE 0 0 0 da13 ONLINE 0 0 0 da14 ONLINE 0 0 0 da15 ONLINE 0 0 0 da16 ONLINE 0 0 0 da17 ONLINE 0 0 0 da18 ONLINE 0 0 0 raidz1-3 ONLINE 0 0 0 da19 ONLINE 0 0 0 da20 ONLINE 0 0 0 da21 ONLINE 0 0 0 da22 ONLINE 0 0 0 da23 ONLINE 0 0 0 da24 ONLINE 0 0 0 da25 ONLINE 0 0 0 da26 ONLINE 0 0 0 da27 ONLINE 0 0 0 errors: No known data errors From owner-freebsd-fs@freebsd.org Thu Feb 2 12:58:54 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ABC3BCCD98B for ; Thu, 2 Feb 2017 12:58:54 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wj0-x22c.google.com (mail-wj0-x22c.google.com [IPv6:2a00:1450:400c:c01::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 493B1F90 for ; Thu, 2 Feb 2017 12:58:54 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wj0-x22c.google.com with SMTP id n2so1675594wjq.3 for ; Thu, 02 Feb 2017 04:58:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=1vFJ7QM86Hh4sOwE3UsdKTqGynTz3ioLSOOkrb50cNc=; b=gqfj4p3o/Mk0NbzSgVp6fQEfoy0s2Un+4iVW053zxWBMPmUA2NEtkm0UBuRspENxQk cXgLPu8fYnHsWelFwuQTu/3CYOlVaqLCor45+cBe3sCNlDlSSfdBLYFwf1cTsgZDrC4+ NyWbJwNRRz6vJi4JP9ZB8APZi60NahKGYhHAlFJxqBNXV/dJ9OaO6EMjhHJ39Ct0qmOy I9bx/JgdqxnNhDTB+OLT/n8Q1e0tV0oRt469Ng/i6JoYeVPlauT20tbimZqjRgaeQ9zA ne87fxdybYcz/WL6ciM2DR4Jcwk4mr/aN3ocbMcSKfm2W5fL1gzFt/062kDz/DLeDmGY 4gqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=1vFJ7QM86Hh4sOwE3UsdKTqGynTz3ioLSOOkrb50cNc=; b=F3WUvIUoHyjMFW5XX1sUW9M3IV8n5RIHaweZznf2/XhVLUsO3ZRcwyhTRqCWIhKsxY Dq/qYI2TBPoD+n+K7mKv5eiuikxGep0DTCWjSJvlGm8UHQ6mA0aFzL0kM8ABu5wwr7RR Z0ZTxG4HJ+yXauLVkpRj0SQ0DALEP0hDkcO5DP9dEPD3/Ih+PL6gf6QBMM5M541gfiV2 dMSZN1lD0x+q8QAgsg60FwOg4nAtDjjPZs0s8HMTwJB8kVjnuCOEcFgPY3S0xIpp7hJv hTet6lBKgqwyGcDpD5HoNtr+WxkCQPIr3LS9TjEQzbQmux0xEOz70DmoIwLEtCVJ36Se Jmeg== X-Gm-Message-State: AIkVDXI4Q0t+hlZq3cuyp2eIaqEzmLeuq6AuR2IDECOwvpT4mfZbG905UUH/UusAe2rB2AMF X-Received: by 10.223.165.87 with SMTP id j23mr8883246wrb.79.1486040330673; Thu, 02 Feb 2017 04:58:50 -0800 (PST) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id i15sm21416403wmf.21.2017.02.02.04.58.49 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 02 Feb 2017 04:58:49 -0800 (PST) Subject: Re: ZFS: mkdir: File too large To: freebsd-fs@freebsd.org References: <20170202115304.GA26493@zxy.spb.ru> From: Steven Hartland Message-ID: <6dfcaec8-cf5e-1611-7b5a-854f64118fb3@multiplay.co.uk> Date: Thu, 2 Feb 2017 12:58:50 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: <20170202115304.GA26493@zxy.spb.ru> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Feb 2017 12:58:54 -0000 Try tracing SET_ERROR for EFBIG with dtrace to determine underlying cause. On 02/02/2017 11:53, Slawa Olhovchenkov wrote: > File too large From owner-freebsd-fs@freebsd.org Thu Feb 2 14:25:00 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 51768CCAB4E for ; Thu, 2 Feb 2017 14:25:00 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0FEF814AB for ; Thu, 2 Feb 2017 14:25:00 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1cZIJj-000Lit-9P; Thu, 02 Feb 2017 17:24:55 +0300 Date: Thu, 2 Feb 2017 17:24:55 +0300 From: Slawa Olhovchenkov To: freebsd-fs@freebsd.org, killing@multiplay.co.uk Subject: Re: ZFS: mkdir: File too large Message-ID: <20170202142455.GB26493@zxy.spb.ru> References: <20170202115304.GA26493@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170202115304.GA26493@zxy.spb.ru> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Feb 2017 14:25:00 -0000 > Try tracing SET_ERROR for EFBIG with dtrace to determine underlying > cause. 0 37281 none:set-error set-error 27 zfs.ko`zfs_freebsd_mkdir+0x3a3 kernel`VOP_MKDIR_APV+0x8a kernel`kern_mkdirat+0x1e1 kernel`amd64_syscall+0x50e kernel`0xffffffff8079499b (kgdb) x zfs_freebsd_mkdir+0x3a3 0xffffffff81171813 : 0xb4b78b45 Current language: auto; currently minimal (kgdb) info line *0xffffffff81171813 Line 2145 of "/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c" starts at address 0xffffffff81171813 and ends at 0xffffffff8117181a . 2138 /* 2139 * Add a new entry to the directory. 2140 */ 2141 getnewvnode_reserve(1); 2142 tx = dmu_tx_create(zfsvfs->z_os); 2143 dmu_tx_hold_zap(tx, dzp->z_id, TRUE, dirname); 2144 dmu_tx_hold_zap(tx, DMU_NEW_OBJECT, FALSE, NULL); 2145 fuid_dirtied = zfsvfs->z_fuid_dirty; 2146 if (fuid_dirtied) 2147 zfs_fuid_txhold(zfsvfs, tx); 2148 if (!zfsvfs->z_use_sa && acl_ids.z_aclp->z_acl_bytes > ZFS_ACE_SPACE) { 2149 dmu_tx_hold_write(tx, DMU_NEW_OBJECT, 0, 2150 acl_ids.z_aclp->z_acl_bytes); 2151 } PS: Please, CC me From owner-freebsd-fs@freebsd.org Thu Feb 2 14:50:05 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3FDCECCD419 for ; Thu, 2 Feb 2017 14:50:05 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x22c.google.com (mail-wm0-x22c.google.com [IPv6:2a00:1450:400c:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CF58A32E for ; Thu, 2 Feb 2017 14:50:04 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wm0-x22c.google.com with SMTP id r141so92477932wmg.1 for ; Thu, 02 Feb 2017 06:50:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=b+954QHzErQNfiVkfYBc7qQHNlP49kpgAGpbHi24HXE=; b=xUtSVtgRM84LSFzTrSksuzjQoMMj7TFoeUcAF8jckJeRW3FW2N0BXv7evTyZbf842/ YMOawkpvVFPXUMfa//R+Efuw0/2DQxmJfUGmUg9F+Cr4wJgBcZQ81+MnWMEMZ7jw+2lx JYtT0PV0HM60CmZsQXIPdAtWx7Ar8GIffjSrLiPMdfuiOXjmZ6xW4bhwbMZIsjGDUh5e SIKfxjGUePZ3Rg4sbbmNnyi2Ss5YvSUWYZu0ZaGmBpfxeJZGhXIpGCmbDQGYIxmyR0JZ 94BlWY9mGrJy7/5ottda8NLHWD7VwQXcX14/Y3APeS3ukWDs6+9boBPa461zGesWO1lL AEvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=b+954QHzErQNfiVkfYBc7qQHNlP49kpgAGpbHi24HXE=; b=pkVaf4ftJwgjHtdMGplDA223s91WXL7uFSFGQ6CLlxJktpQKJdfpeWl7COCAtK5Wbf ZQsJ0J4Wj5Y12ZoXblvLYFe56XdpdLxXF9NWDUe7c8nwwAgQmXI93I8YYitb1HdBxjaN UMpctsT/GFLVedrnAkK+nyHiadDVDBXFM1QMrVrT31Ua4pJzmhknHQ3f+AS5AFMrIh4l 2gI+vkG7tlOXZ+deJyNuYsbqql+o2wV2hOfs+9DRRl2Z+TuuT83se8cT9sT0ytfNVYP8 Ay08Z/h8qDbZTIXsDFUWDRx6xPR10Do5+VtyQoCbJZomuklD3z7AzVJ9ICMa8c1LNvdB y9rA== X-Gm-Message-State: AIkVDXIVSo4RYYwYpHtYXbdCwKNz6fh/RnoLmk3oen2hVo0P3PhqZ5EIVv+XDylUAlnD+8ay X-Received: by 10.28.45.197 with SMTP id t188mr8749880wmt.15.1486047001468; Thu, 02 Feb 2017 06:50:01 -0800 (PST) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id v102sm40287338wrb.11.2017.02.02.06.50.00 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 02 Feb 2017 06:50:00 -0800 (PST) Subject: Re: ZFS: mkdir: File too large To: Slawa Olhovchenkov , freebsd-fs@freebsd.org References: <20170202115304.GA26493@zxy.spb.ru> <20170202142455.GB26493@zxy.spb.ru> From: Steven Hartland Message-ID: <397541e0-2327-0591-e05c-bdf09cb2d923@multiplay.co.uk> Date: Thu, 2 Feb 2017 14:50:01 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: <20170202142455.GB26493@zxy.spb.ru> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Feb 2017 14:50:05 -0000 On 02/02/2017 14:24, Slawa Olhovchenkov wrote: >> Try tracing SET_ERROR for EFBIG with dtrace to determine underlying >> cause. > 0 37281 none:set-error set-error 27 > zfs.ko`zfs_freebsd_mkdir+0x3a3 > kernel`VOP_MKDIR_APV+0x8a > kernel`kern_mkdirat+0x1e1 > kernel`amd64_syscall+0x50e > kernel`0xffffffff8079499b > > (kgdb) x zfs_freebsd_mkdir+0x3a3 > 0xffffffff81171813 : 0xb4b78b45 > Current language: auto; currently minimal > (kgdb) info line *0xffffffff81171813 > Line 2145 of "/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c" starts at address 0xffffffff81171813 and ends at 0xffffffff8117181a . > > 2138 /* > 2139 * Add a new entry to the directory. > 2140 */ > 2141 getnewvnode_reserve(1); > 2142 tx = dmu_tx_create(zfsvfs->z_os); > 2143 dmu_tx_hold_zap(tx, dzp->z_id, TRUE, dirname); > 2144 dmu_tx_hold_zap(tx, DMU_NEW_OBJECT, FALSE, NULL); > 2145 fuid_dirtied = zfsvfs->z_fuid_dirty; > 2146 if (fuid_dirtied) > 2147 zfs_fuid_txhold(zfsvfs, tx); > 2148 if (!zfsvfs->z_use_sa && acl_ids.z_aclp->z_acl_bytes > ZFS_ACE_SPACE) { > 2149 dmu_tx_hold_write(tx, DMU_NEW_OBJECT, 0, > 2150 acl_ids.z_aclp->z_acl_bytes); > 2151 } > > PS: Please, CC me What OS ver as those offsets seem odd? From owner-freebsd-fs@freebsd.org Thu Feb 2 14:52:53 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 43A56CCD5F0 for ; Thu, 2 Feb 2017 14:52:53 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0658095C for ; Thu, 2 Feb 2017 14:52:53 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1cZIkl-000MP6-2t; Thu, 02 Feb 2017 17:52:51 +0300 Date: Thu, 2 Feb 2017 17:52:51 +0300 From: Slawa Olhovchenkov To: Steven Hartland Cc: freebsd-fs@freebsd.org Subject: Re: ZFS: mkdir: File too large Message-ID: <20170202145251.GC26493@zxy.spb.ru> References: <20170202115304.GA26493@zxy.spb.ru> <20170202142455.GB26493@zxy.spb.ru> <397541e0-2327-0591-e05c-bdf09cb2d923@multiplay.co.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <397541e0-2327-0591-e05c-bdf09cb2d923@multiplay.co.uk> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Feb 2017 14:52:53 -0000 On Thu, Feb 02, 2017 at 02:50:01PM +0000, Steven Hartland wrote: > On 02/02/2017 14:24, Slawa Olhovchenkov wrote: > >> Try tracing SET_ERROR for EFBIG with dtrace to determine underlying > >> cause. > > 0 37281 none:set-error set-error 27 > > zfs.ko`zfs_freebsd_mkdir+0x3a3 > > kernel`VOP_MKDIR_APV+0x8a > > kernel`kern_mkdirat+0x1e1 > > kernel`amd64_syscall+0x50e > > kernel`0xffffffff8079499b > > > > (kgdb) x zfs_freebsd_mkdir+0x3a3 > > 0xffffffff81171813 : 0xb4b78b45 > > Current language: auto; currently minimal > > (kgdb) info line *0xffffffff81171813 > > Line 2145 of "/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c" starts at address 0xffffffff81171813 and ends at 0xffffffff8117181a . > > > > 2138 /* > > 2139 * Add a new entry to the directory. > > 2140 */ > > 2141 getnewvnode_reserve(1); > > 2142 tx = dmu_tx_create(zfsvfs->z_os); > > 2143 dmu_tx_hold_zap(tx, dzp->z_id, TRUE, dirname); > > 2144 dmu_tx_hold_zap(tx, DMU_NEW_OBJECT, FALSE, NULL); > > 2145 fuid_dirtied = zfsvfs->z_fuid_dirty; > > 2146 if (fuid_dirtied) > > 2147 zfs_fuid_txhold(zfsvfs, tx); > > 2148 if (!zfsvfs->z_use_sa && acl_ids.z_aclp->z_acl_bytes > ZFS_ACE_SPACE) { > > 2149 dmu_tx_hold_write(tx, DMU_NEW_OBJECT, 0, > > 2150 acl_ids.z_aclp->z_acl_bytes); > > 2151 } > > > > PS: Please, CC me > What OS ver as those offsets seem odd? I am try trace every call in zfs_freebsd_mkdir. This is dmu_tx_assign: 3 29599 dmu_tx_assign:return ret 27 2153 dmu_tx_hold_sa_create(tx, acl_ids.z_aclp->z_acl_bytes + 2154 ZFS_SA_BASE_ATTR_SIZE); 2155 2156 error = dmu_tx_assign(tx, TXG_WAIT); 2157 if (error) { 2158 zfs_acl_ids_free(&acl_ids); 2159 dmu_tx_abort(tx); 2160 getnewvnode_drop_reserve(); 2161 ZFS_EXIT(zfsvfs); 2162 return (error); 2163 } From owner-freebsd-fs@freebsd.org Thu Feb 2 15:04:19 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AD0B9CCD9B6 for ; Thu, 2 Feb 2017 15:04:19 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 72368F14 for ; Thu, 2 Feb 2017 15:04:19 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1cZIvp-000Mj5-FE; Thu, 02 Feb 2017 18:04:17 +0300 Date: Thu, 2 Feb 2017 18:04:17 +0300 From: Slawa Olhovchenkov To: Steven Hartland Cc: freebsd-fs@freebsd.org Subject: Re: ZFS: mkdir: File too large Message-ID: <20170202150417.GD26493@zxy.spb.ru> References: <20170202115304.GA26493@zxy.spb.ru> <20170202142455.GB26493@zxy.spb.ru> <397541e0-2327-0591-e05c-bdf09cb2d923@multiplay.co.uk> <20170202145251.GC26493@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170202145251.GC26493@zxy.spb.ru> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Feb 2017 15:04:19 -0000 On Thu, Feb 02, 2017 at 05:52:51PM +0300, Slawa Olhovchenkov wrote: > On Thu, Feb 02, 2017 at 02:50:01PM +0000, Steven Hartland wrote: > > > On 02/02/2017 14:24, Slawa Olhovchenkov wrote: > > >> Try tracing SET_ERROR for EFBIG with dtrace to determine underlying > > >> cause. > > > 0 37281 none:set-error set-error 27 > > > zfs.ko`zfs_freebsd_mkdir+0x3a3 > > > kernel`VOP_MKDIR_APV+0x8a > > > kernel`kern_mkdirat+0x1e1 > > > kernel`amd64_syscall+0x50e > > > kernel`0xffffffff8079499b > > > > > > (kgdb) x zfs_freebsd_mkdir+0x3a3 > > > 0xffffffff81171813 : 0xb4b78b45 > > > Current language: auto; currently minimal > > > (kgdb) info line *0xffffffff81171813 > > > Line 2145 of "/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c" starts at address 0xffffffff81171813 and ends at 0xffffffff8117181a . > > > > > > 2138 /* > > > 2139 * Add a new entry to the directory. > > > 2140 */ > > > 2141 getnewvnode_reserve(1); > > > 2142 tx = dmu_tx_create(zfsvfs->z_os); > > > 2143 dmu_tx_hold_zap(tx, dzp->z_id, TRUE, dirname); > > > 2144 dmu_tx_hold_zap(tx, DMU_NEW_OBJECT, FALSE, NULL); > > > 2145 fuid_dirtied = zfsvfs->z_fuid_dirty; > > > 2146 if (fuid_dirtied) > > > 2147 zfs_fuid_txhold(zfsvfs, tx); > > > 2148 if (!zfsvfs->z_use_sa && acl_ids.z_aclp->z_acl_bytes > ZFS_ACE_SPACE) { > > > 2149 dmu_tx_hold_write(tx, DMU_NEW_OBJECT, 0, > > > 2150 acl_ids.z_aclp->z_acl_bytes); > > > 2151 } > > > > > > PS: Please, CC me > > What OS ver as those offsets seem odd? Offset seem true: dmu_tx_assign failed at start by tx->tx_err == EFBIG. tx->tx_err is EFBIG at dmu_tx_hold_zap:entry : # dtrace -n 'fbt:zfs:dmu_tx_hold_zap:entry { printf("err %d id %x", args[0]->tx_err, args[1]); }' dtrace: description 'fbt:zfs:dmu_tx_hold_zap:entry ' matched 1 probe CPU ID FUNCTION:NAME 0 31349 dmu_tx_hold_zap:entry err 0 id 4 0 31349 dmu_tx_hold_zap:entry err 0 id ffffffffffffffff 0 31349 dmu_tx_hold_zap:entry err 27 id 6 > I am try trace every call in zfs_freebsd_mkdir. > This is dmu_tx_assign: > > 3 29599 dmu_tx_assign:return ret 27 > > 2153 dmu_tx_hold_sa_create(tx, acl_ids.z_aclp->z_acl_bytes + > 2154 ZFS_SA_BASE_ATTR_SIZE); > 2155 > 2156 error = dmu_tx_assign(tx, TXG_WAIT); > 2157 if (error) { > 2158 zfs_acl_ids_free(&acl_ids); > 2159 dmu_tx_abort(tx); > 2160 getnewvnode_drop_reserve(); > 2161 ZFS_EXIT(zfsvfs); > 2162 return (error); > 2163 } From owner-freebsd-fs@freebsd.org Thu Feb 2 15:58:34 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3D474CCDD77 for ; Thu, 2 Feb 2017 15:58:34 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-wm0-x22a.google.com (mail-wm0-x22a.google.com [IPv6:2a00:1450:400c:c09::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DCE75110A for ; Thu, 2 Feb 2017 15:58:33 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-wm0-x22a.google.com with SMTP id 196so6434723wmm.1 for ; Thu, 02 Feb 2017 07:58:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=bQ9CtRQoDsoorlbLnd6ZYcIcApbG1yjR9VmOdPvW3YA=; b=EI1NK+aH3tp1Id7h5xg5j+TXzCUToiCoVpuM+vWI8X7QiSTaV3Y3GYic5ZtK6cMftB EdvhZxtxmH/1idhJiA0XZTnpp/o96CSg78gaaTJOoZSs9x4vg58fI2dA7z9nU6ZSnr1b AgbFIBDToNX90xtX0y+26hH1UYyt4MwOMNgYdroFrZR5n9tXstRq6n41mJG0/1suhmWC 0eUs5iE8jFFuVKNm8qrtaNrc2tIbep2grmd33OkvfeovYrSV1xm6Ig8El6m1iQjtyaVk DwDwnqTDshDXRkNAn/ExmPoe8HP+HJ6UDsL2W4h8W3Lz3h+zrWuf+4ld4Ah7IxcDVndm FNQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=bQ9CtRQoDsoorlbLnd6ZYcIcApbG1yjR9VmOdPvW3YA=; b=uj/1fjmrNn2JfeT47Myqm7HJUpKMh5XOHb3IHNCCiWagmL6nCbodBkAhWsVbVu758f KQA10y6mwKxRmXhlmHf2MjO9j/b8uHtkP/q33r22DuMDwQ0nMR3Dgj75Mw4Cf4fyisom d/mgSm2XV+XjBCeANHdOzonBLxMJ2AX7CuZTac97bV4ZELaYH/WrA0goYwlPY6AIXaSY rVkMxNlr/zSiEq04jYxo9IjQyFZnBGesF0VUaw9blwXC9O/3TZo2BFs0hNla8tNMUtuO kVjEM+2AFRFkljWBgOrUAbTXPqPTc0+WouWclvmoRCSzNnQgFVeKXSLNJbjOF83BMTwy tblA== X-Gm-Message-State: AIkVDXLCFGllG54+QQjut6nr6SZaT3katQrqQBELfEln50ThNcb+fAmy3vSV3cP7Hzy74ZDr X-Received: by 10.28.153.196 with SMTP id b187mr26848309wme.53.1486051110768; Thu, 02 Feb 2017 07:58:30 -0800 (PST) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id v128sm3559136wmv.2.2017.02.02.07.58.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 02 Feb 2017 07:58:29 -0800 (PST) Subject: Re: ZFS: mkdir: File too large To: Slawa Olhovchenkov , freebsd-fs@freebsd.org References: <20170202115304.GA26493@zxy.spb.ru> <20170202142455.GB26493@zxy.spb.ru> From: Steven Hartland Message-ID: Date: Thu, 2 Feb 2017 15:58:30 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: <20170202142455.GB26493@zxy.spb.ru> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Feb 2017 15:58:34 -0000 On 02/02/2017 14:24, Slawa Olhovchenkov wrote: >> Try tracing SET_ERROR for EFBIG with dtrace to determine underlying >> cause. > 0 37281 none:set-error set-error 27 > zfs.ko`zfs_freebsd_mkdir+0x3a3 > kernel`VOP_MKDIR_APV+0x8a > kernel`kern_mkdirat+0x1e1 > kernel`amd64_syscall+0x50e > kernel`0xffffffff8079499b > > (kgdb) x zfs_freebsd_mkdir+0x3a3 > 0xffffffff81171813 : 0xb4b78b45 > Current language: auto; currently minimal > (kgdb) info line *0xffffffff81171813 > Line 2145 of "/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c" starts at address 0xffffffff81171813 and ends at 0xffffffff8117181a . > > 2138 /* > 2139 * Add a new entry to the directory. > 2140 */ > 2141 getnewvnode_reserve(1); > 2142 tx = dmu_tx_create(zfsvfs->z_os); > 2143 dmu_tx_hold_zap(tx, dzp->z_id, TRUE, dirname); > 2144 dmu_tx_hold_zap(tx, DMU_NEW_OBJECT, FALSE, NULL); > 2145 fuid_dirtied = zfsvfs->z_fuid_dirty; > 2146 if (fuid_dirtied) > 2147 zfs_fuid_txhold(zfsvfs, tx); > 2148 if (!zfsvfs->z_use_sa && acl_ids.z_aclp->z_acl_bytes > ZFS_ACE_SPACE) { > 2149 dmu_tx_hold_write(tx, DMU_NEW_OBJECT, 0, > 2150 acl_ids.z_aclp->z_acl_bytes); > 2151 } > > PS: Please, CC me Having a cursory look I'm going to guess -> zfs_freebsd_mkdir dmu_tx_assign dmu_tx_try_assign dmu_tx_count_write if (txh->txh_space_towrite + txh->txh_space_tooverwrite > 2 * DMU_MAX_ACCESS) err = SET_ERROR(EFBIG); Regards Steve From owner-freebsd-fs@freebsd.org Thu Feb 2 16:29:40 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EBF87CCE561 for ; Thu, 2 Feb 2017 16:29:40 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B03F06DB for ; Thu, 2 Feb 2017 16:29:40 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1cZKGP-000OrR-RH; Thu, 02 Feb 2017 19:29:37 +0300 Date: Thu, 2 Feb 2017 19:29:37 +0300 From: Slawa Olhovchenkov To: Steven Hartland Cc: freebsd-fs@freebsd.org Subject: Re: ZFS: mkdir: File too large Message-ID: <20170202162937.GE26493@zxy.spb.ru> References: <20170202115304.GA26493@zxy.spb.ru> <20170202142455.GB26493@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Feb 2017 16:29:41 -0000 On Thu, Feb 02, 2017 at 03:58:30PM +0000, Steven Hartland wrote: > On 02/02/2017 14:24, Slawa Olhovchenkov wrote: > >> Try tracing SET_ERROR for EFBIG with dtrace to determine underlying > >> cause. > > 0 37281 none:set-error set-error 27 > > zfs.ko`zfs_freebsd_mkdir+0x3a3 > > kernel`VOP_MKDIR_APV+0x8a > > kernel`kern_mkdirat+0x1e1 > > kernel`amd64_syscall+0x50e > > kernel`0xffffffff8079499b > > > > (kgdb) x zfs_freebsd_mkdir+0x3a3 > > 0xffffffff81171813 : 0xb4b78b45 > > Current language: auto; currently minimal > > (kgdb) info line *0xffffffff81171813 > > Line 2145 of "/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c" starts at address 0xffffffff81171813 and ends at 0xffffffff8117181a . > > > > 2138 /* > > 2139 * Add a new entry to the directory. > > 2140 */ > > 2141 getnewvnode_reserve(1); > > 2142 tx = dmu_tx_create(zfsvfs->z_os); > > 2143 dmu_tx_hold_zap(tx, dzp->z_id, TRUE, dirname); > > 2144 dmu_tx_hold_zap(tx, DMU_NEW_OBJECT, FALSE, NULL); > > 2145 fuid_dirtied = zfsvfs->z_fuid_dirty; > > 2146 if (fuid_dirtied) > > 2147 zfs_fuid_txhold(zfsvfs, tx); > > 2148 if (!zfsvfs->z_use_sa && acl_ids.z_aclp->z_acl_bytes > ZFS_ACE_SPACE) { > > 2149 dmu_tx_hold_write(tx, DMU_NEW_OBJECT, 0, > > 2150 acl_ids.z_aclp->z_acl_bytes); > > 2151 } > > > > PS: Please, CC me > Having a cursory look I'm going to guess -> > zfs_freebsd_mkdir > dmu_tx_assign > dmu_tx_try_assign > dmu_tx_count_write > if (txh->txh_space_towrite + txh->txh_space_tooverwrite > > 2 * DMU_MAX_ACCESS) > err = SET_ERROR(EFBIG); zfs_freebsd_mkdir dmu_tx_hold_zap(tx, DMU_NEW_OBJECT, FALSE, NULL); dmu_tx_count_write return err = EFBIG Not sure about check txh->txh_space_towrite/txh->txh_space_tooverwrite: dtrace: failed to compile script /mnt/big.d: line 30: printf( ) argument #3 is incompatible with conversion #2 prototype: conversion: %d prototype: char, short, int, long, or long long argument: refcount_t out: if (refcount_count(&txh->txh_space_towrite) + refcount_count(&txh->txh_space_tooverwrite) > 2 * DMU_MAX_ACCESS) err = SET_ERROR(EFBIG); From owner-freebsd-fs@freebsd.org Thu Feb 2 16:37:16 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 02983CCE75D for ; Thu, 2 Feb 2017 16:37:16 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BAEF2BEF for ; Thu, 2 Feb 2017 16:37:15 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1cZKNl-000P3h-Ng; Thu, 02 Feb 2017 19:37:13 +0300 Date: Thu, 2 Feb 2017 19:37:13 +0300 From: Slawa Olhovchenkov To: Steven Hartland Cc: freebsd-fs@freebsd.org Subject: Re: ZFS: mkdir: File too large Message-ID: <20170202163713.GF26493@zxy.spb.ru> References: <20170202115304.GA26493@zxy.spb.ru> <20170202142455.GB26493@zxy.spb.ru> <20170202162937.GE26493@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170202162937.GE26493@zxy.spb.ru> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Feb 2017 16:37:16 -0000 On Thu, Feb 02, 2017 at 07:29:37PM +0300, Slawa Olhovchenkov wrote: > On Thu, Feb 02, 2017 at 03:58:30PM +0000, Steven Hartland wrote: > > > On 02/02/2017 14:24, Slawa Olhovchenkov wrote: > > >> Try tracing SET_ERROR for EFBIG with dtrace to determine underlying > > >> cause. > > > 0 37281 none:set-error set-error 27 > > > zfs.ko`zfs_freebsd_mkdir+0x3a3 > > > kernel`VOP_MKDIR_APV+0x8a > > > kernel`kern_mkdirat+0x1e1 > > > kernel`amd64_syscall+0x50e > > > kernel`0xffffffff8079499b > > > > > > (kgdb) x zfs_freebsd_mkdir+0x3a3 > > > 0xffffffff81171813 : 0xb4b78b45 > > > Current language: auto; currently minimal > > > (kgdb) info line *0xffffffff81171813 > > > Line 2145 of "/usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c" starts at address 0xffffffff81171813 and ends at 0xffffffff8117181a . > > > > > > 2138 /* > > > 2139 * Add a new entry to the directory. > > > 2140 */ > > > 2141 getnewvnode_reserve(1); > > > 2142 tx = dmu_tx_create(zfsvfs->z_os); > > > 2143 dmu_tx_hold_zap(tx, dzp->z_id, TRUE, dirname); > > > 2144 dmu_tx_hold_zap(tx, DMU_NEW_OBJECT, FALSE, NULL); > > > 2145 fuid_dirtied = zfsvfs->z_fuid_dirty; > > > 2146 if (fuid_dirtied) > > > 2147 zfs_fuid_txhold(zfsvfs, tx); > > > 2148 if (!zfsvfs->z_use_sa && acl_ids.z_aclp->z_acl_bytes > ZFS_ACE_SPACE) { > > > 2149 dmu_tx_hold_write(tx, DMU_NEW_OBJECT, 0, > > > 2150 acl_ids.z_aclp->z_acl_bytes); > > > 2151 } > > > > > > PS: Please, CC me > > Having a cursory look I'm going to guess -> > > zfs_freebsd_mkdir > > dmu_tx_assign > > dmu_tx_try_assign > > dmu_tx_count_write > > if (txh->txh_space_towrite + txh->txh_space_tooverwrite > > > 2 * DMU_MAX_ACCESS) > > err = SET_ERROR(EFBIG); > > > zfs_freebsd_mkdir > dmu_tx_hold_zap(tx, DMU_NEW_OBJECT, FALSE, NULL); > dmu_tx_count_write > return err = EFBIG > > Not sure about check > txh->txh_space_towrite/txh->txh_space_tooverwrite: Ah, check it now: 1 28010 dmu_tx_count_write:entry err 0 txh_space_towrite 114688 txh_space_tooverwrite 0 1 28011 dmu_tx_count_write:return err 27 txh_space_towrite 79020032 txh_space_tooverwrite 0 yes, this is relay > 2 * DMU_MAX_ACCESS. > dtrace: failed to compile script /mnt/big.d: line 30: printf( ) > argument #3 is incompatible with conversion #2 prototype: > conversion: %d > prototype: char, short, int, long, or long long > argument: refcount_t > > out: > if (refcount_count(&txh->txh_space_towrite) + > refcount_count(&txh->txh_space_tooverwrite) > > 2 * DMU_MAX_ACCESS) > err = SET_ERROR(EFBIG); > From owner-freebsd-fs@freebsd.org Fri Feb 3 02:31:59 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CBDA2CCDB27; Fri, 3 Feb 2017 02:31:59 +0000 (UTC) (envelope-from ultima1252@gmail.com) Received: from mail-yb0-x22d.google.com (mail-yb0-x22d.google.com [IPv6:2607:f8b0:4002:c09::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8E1FF1B07; Fri, 3 Feb 2017 02:31:59 +0000 (UTC) (envelope-from ultima1252@gmail.com) Received: by mail-yb0-x22d.google.com with SMTP id o65so2045961ybo.2; Thu, 02 Feb 2017 18:31:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=dButyUbQ5iTvPrQO/TVCBj8PpIAxBy/Ic44tS/rHWbo=; b=JbexhR1vcawix1HvD9Rr+gmlorncrCHAsQJPWCQCGebMcvHLHEnWuv24bST8TP66Pr dRVCyyUTFnSyjNOMPY5FtOlpIySA0N4x1abQHKLKmXSWIu1vD9MMOuOZc1gUsKj6Knfz fqi31n/tVDL16mesu0kD0Otejx6hAaX5xA/qs1bkq5OK5qG5OuweZpvLfk4y0eeS9Ib3 m/0uaMeuWJ0ooEfZWEfCIvehwMYqlGiqkZwrt9JQWWv0yO1ISy3Wqd61qPHZ3W+1TJEN 2KpBr5luLixmTG7Ju26d8en1WIlq6hQTDK37A5Uc8kwii8oWTP6Su0cZsvodw7gNZqid nhEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=dButyUbQ5iTvPrQO/TVCBj8PpIAxBy/Ic44tS/rHWbo=; b=OXcehSapoTSaSoYUNNa3py+UmZP8C+IX3cKVLPS7lVPvw+S9ni5Tsq8MwGXrVTra9V s8k3qflxQLh4q6AIayjGc14hG6T7msg/NDv4S/UbnyM2Jdpc7FXbA0ZRgtkkDM9TJgbN Ra0KwffdVFyRHvyot1GV1VZmxSurq7zOSFY+sHXE7JjUTW7acYqMyMMgjqsN7iP2cMYY 9Rauw1taa85Yetc1iQEKTcT8RDCTR/udFCPj3L4KYXiZIq8ou/satldcdr90fBVqfZc/ nJtebppGt1hl18iUCGSs2uh8g2nuESrahLUzzmCMtGRU8WmxWpinpTHkQM6KdQyk9SH6 kO+Q== X-Gm-Message-State: AIkVDXLzwkF0keeiaxseUK6JkdpRwsYIyZWdEUUJ9m/+AuLXKHZN0PgeOwFBDLz6eI9QM/oTqp1ge5W/SMrEUw== X-Received: by 10.37.248.2 with SMTP id u2mr4670497ybd.52.1486089118503; Thu, 02 Feb 2017 18:31:58 -0800 (PST) MIME-Version: 1.0 Received: by 10.129.52.65 with HTTP; Thu, 2 Feb 2017 18:31:58 -0800 (PST) From: Ultima Date: Thu, 2 Feb 2017 21:31:58 -0500 Message-ID: Subject: zfs snapshot_limit is not respected To: freebsd-stable@freebsd.org, freebsd-current@freebsd.org, freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Feb 2017 02:31:59 -0000 I recently moved some data on a box with limited space. I decided I should limit the snapshots so that space would not become an issue. I just check back a week later to find out the box is hitting the borderline. Doing I quick check I realized that the snapshot_limit is not being respected. # uname -a FreeBSD R1 11.0-STABLE FreeBSD 11.0-STABLE #17 r312232: Sun Jan 15 10:59:10 EST 2017 root@S1:/usr/src/11-STABLE/obj/usr/src/11-STABLE/src/sys/MYKERNEL amd64 # zfs create zroot/bhyve/test # zfs set snapshot_limit=0 zroot/bhyve/test # zfs snapshot zroot/bhyve/test@1 # zfs snapshot zroot/bhyve/test@2 # zfs snapshot zroot/bhyve/test@3 # zfs list -t snapshot | grep zroot/bhyve/test zroot/bhyve/test@1 0 - 88K - zroot/bhyve/test@2 0 - 88K - zroot/bhyve/test@3 0 - 88K - # zfs get all zroot/bhyve/test | grep snapshot zroot/bhyve/test usedbysnapshots 0 - zroot/bhyve/test snapshot_limit 0 local zroot/bhyve/test snapshot_count 3 local Also wanted to verify 0 was not being mistaken for none. # for snapshot in `zfs list -t snapshot | grep zroot/bhyve/test | awk '{print $1}'`; do zfs destroy $snapshot ; done # zfs get all zroot/bhyve/test | grep snapshot zroot/bhyve/test usedbysnapshots 0 - zroot/bhyve/test snapshot_limit 0 local zroot/bhyve/test snapshot_count 0 local # zfs set snapshot_limit=1 zroot/bhyve/test # zfs snapshot zroot/bhyve/test@1 # zfs snapshot zroot/bhyve/test@2 # zfs snapshot zroot/bhyve/test@3 # zfs get all zroot/bhyve/test | grep snapshot zroot/bhyve/test usedbysnapshots 0 - zroot/bhyve/test snapshot_limit 1 local zroot/bhyve/test snapshot_count 3 local Also tested on head FreeBSD S1 12.0-CURRENT FreeBSD 12.0-CURRENT #26 r312388: Wed Jan 18 12:38:52 EST 2017 root@S1:/usr/src/head/obj/usr/src/head/src/sys/MYKERNEL-NODEBUG amd64 From owner-freebsd-fs@freebsd.org Fri Feb 3 12:42:53 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C2478CCD0D4; Fri, 3 Feb 2017 12:42:53 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from mail.in-addr.com (mail.in-addr.com [IPv6:2a01:4f8:191:61e8::2525:2525]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8E9EE16DE; Fri, 3 Feb 2017 12:42:53 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from gjp by mail.in-addr.com with local (Exim 4.88 (FreeBSD)) (envelope-from ) id 1cZdCV-0004Kq-NV; Fri, 03 Feb 2017 12:42:51 +0000 Date: Fri, 3 Feb 2017 12:42:51 +0000 From: Gary Palmer To: Ultima Cc: freebsd-stable@freebsd.org, freebsd-current@freebsd.org, freebsd-fs@freebsd.org Subject: Re: zfs snapshot_limit is not respected Message-ID: <20170203124251.GA68015@in-addr.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on mail.in-addr.com); SAEximRunCond expanded to false X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Feb 2017 12:42:53 -0000 On Thu, Feb 02, 2017 at 09:31:58PM -0500, Ultima wrote: > I recently moved some data on a box with limited space. I decided I should > limit the snapshots so that space would not become an issue. I just check > back a week later to find out the box is hitting the borderline. Doing I > quick check I realized that the snapshot_limit is not being respected. > > # uname -a > FreeBSD R1 11.0-STABLE FreeBSD 11.0-STABLE #17 r312232: Sun Jan 15 10:59:10 > EST 2017 root@S1:/usr/src/11-STABLE/obj/usr/src/11-STABLE/src/sys/MYKERNEL > amd64 > > # zfs create zroot/bhyve/test > # zfs set snapshot_limit=0 zroot/bhyve/test > # zfs snapshot zroot/bhyve/test@1 > > > # zfs snapshot zroot/bhyve/test@2 > # zfs snapshot zroot/bhyve/test@3 > # zfs list -t snapshot | grep zroot/bhyve/test > zroot/bhyve/test@1 0 - > 88K - > zroot/bhyve/test@2 0 - > 88K - > zroot/bhyve/test@3 0 - > 88K - > # zfs get all zroot/bhyve/test | grep snapshot > zroot/bhyve/test usedbysnapshots 0 - > zroot/bhyve/test snapshot_limit 0 local > zroot/bhyve/test snapshot_count 3 local > > Also wanted to verify 0 was not being mistaken for none. > > # for snapshot in `zfs list -t snapshot | grep zroot/bhyve/test | awk > '{print $1}'`; do zfs destroy $snapshot ; done > > # zfs get all zroot/bhyve/test | grep snapshot > zroot/bhyve/test usedbysnapshots 0 - > zroot/bhyve/test snapshot_limit 0 local > zroot/bhyve/test snapshot_count 0 local > > # zfs set snapshot_limit=1 zroot/bhyve/test > # zfs snapshot zroot/bhyve/test@1 > # zfs snapshot zroot/bhyve/test@2 > # zfs snapshot zroot/bhyve/test@3 > # zfs get all zroot/bhyve/test | grep snapshot > zroot/bhyve/test usedbysnapshots 0 - > zroot/bhyve/test snapshot_limit 1 local > zroot/bhyve/test snapshot_count 3 local > > > Also tested on head > FreeBSD S1 12.0-CURRENT FreeBSD 12.0-CURRENT #26 r312388: Wed Jan 18 > 12:38:52 EST 2017 > root@S1:/usr/src/head/obj/usr/src/head/src/sys/MYKERNEL-NODEBUG > amd64 Hi, I suspect this line from the manpage is key: The limit is not enforced if the user is allowed to change the limit Regards, Gary From owner-freebsd-fs@freebsd.org Fri Feb 3 15:27:49 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E2221CCE701; Fri, 3 Feb 2017 15:27:49 +0000 (UTC) (envelope-from ultima1252@gmail.com) Received: from mail-yb0-x22c.google.com (mail-yb0-x22c.google.com [IPv6:2607:f8b0:4002:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8F80FC59; Fri, 3 Feb 2017 15:27:49 +0000 (UTC) (envelope-from ultima1252@gmail.com) Received: by mail-yb0-x22c.google.com with SMTP id 123so6869657ybe.3; Fri, 03 Feb 2017 07:27:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=90z5R+cAE8kRp+53gjHy5SjNSxArbYP8OHsumqA/KhI=; b=kN/HAPFFp2mxF40sPpRcerux4+folvEhDEEJ+JKf6eE2TV7/W8MbG9S5ui5l6JvwlM wRawdPuksTlE3kHOh8FlK2RqaZn/UJjL0iHR0BaJRHwxUT7maWa/rHi2D2ZnBIKYlCf3 8DGg/wCXVfl/lnZpeiKO8gq2SMTM+6xYY1vgnWygPNG7omWxJdi/jsrcbTr4CV37XdF+ xcbVPXAjsSJvwmwgrpBtX1dKsoAkZ+b9fgsadZH+i3v8jksgHEt+5SlSeF/Ldj9/OMps uk2Oc5uYMFECkp7Z6yZg2hm55ytVZDxAKeTZfQUfreUE8bY2onzvpreKBuZeC/ubzFUi GRUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=90z5R+cAE8kRp+53gjHy5SjNSxArbYP8OHsumqA/KhI=; b=MEz5yiywi1GIDNQpmQ8OTVNyGtnZDz1RqUDOjoe0LiChF5SJJrglxeQbgMsdM8TEmd wsIiiRdDn6SeTtCGm9t10q6ZV9Htjjfz/TyD8+RjCOMrIgS+KpyHWi1Cvhj7zzSvziSq cjhmYbrbVjmCiROHI74KI3QcosVeB3ZLMPjqbzKYIB40wjdyGydkkX5bg8FWPI9jUZwx Il6kBBMUcdRZXBM5zDSslqYFGU9u4m5wbCVhzRYMaGM5YPvParbPDh2V5V2fWnhU8R3n diihIRL4KRN5ZDfeJA2PlIJNwWzXoZ3ndx0OxyEOlJkYxoXh2jYDsCOcmCg6jWQJHKZy hJDQ== X-Gm-Message-State: AIkVDXJKncecFJDMjmkV9241CdcxuTYc5V6h4lizevSjsCF+oYUhI7c6ig7km9Xwwv4dV9rJH8vApkZGOthrAA== X-Received: by 10.37.248.2 with SMTP id u2mr6440810ybd.52.1486135668573; Fri, 03 Feb 2017 07:27:48 -0800 (PST) MIME-Version: 1.0 Received: by 10.129.52.65 with HTTP; Fri, 3 Feb 2017 07:27:48 -0800 (PST) In-Reply-To: <20170203124251.GA68015@in-addr.com> References: <20170203124251.GA68015@in-addr.com> From: Ultima Date: Fri, 3 Feb 2017 10:27:48 -0500 Message-ID: Subject: Re: zfs snapshot_limit is not respected To: Gary Palmer Cc: freebsd-stable@freebsd.org, freebsd-current@freebsd.org, freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Feb 2017 15:27:50 -0000 Hey Gary, You are probably right. Do you know how to "lock" this property by chance? I'v read this exact line several times trying to understand the exact meaning. The "user is allowed to change the limit" I *think* is referring to the zfs allow command. The problem is that I checked the dataset and it is showing no permissions granted to a user. So I guess user in this case is also including the root user, but how does one lock the property from root? I keep going through the manpage looking for something I may have missed but keep coming up empty. Thanks for replying, Ultima On Fri, Feb 3, 2017 at 7:42 AM, Gary Palmer wrote: > On Thu, Feb 02, 2017 at 09:31:58PM -0500, Ultima wrote: > > I recently moved some data on a box with limited space. I decided I > should > > limit the snapshots so that space would not become an issue. I just check > > back a week later to find out the box is hitting the borderline. Doing I > > quick check I realized that the snapshot_limit is not being respected. > > > > # uname -a > > FreeBSD R1 11.0-STABLE FreeBSD 11.0-STABLE #17 r312232: Sun Jan 15 > 10:59:10 > > EST 2017 root@S1:/usr/src/11-STABLE/obj/usr/src/11-STABLE/src/sys/ > MYKERNEL > > amd64 > > > > # zfs create zroot/bhyve/test > > # zfs set snapshot_limit=0 zroot/bhyve/test > > # zfs snapshot zroot/bhyve/test@1 > > > > > > # zfs snapshot zroot/bhyve/test@2 > > # zfs snapshot zroot/bhyve/test@3 > > # zfs list -t snapshot | grep zroot/bhyve/test > > zroot/bhyve/test@1 0 - > > 88K - > > zroot/bhyve/test@2 0 - > > 88K - > > zroot/bhyve/test@3 0 - > > 88K - > > # zfs get all zroot/bhyve/test | grep snapshot > > zroot/bhyve/test usedbysnapshots 0 - > > zroot/bhyve/test snapshot_limit 0 local > > zroot/bhyve/test snapshot_count 3 local > > > > Also wanted to verify 0 was not being mistaken for none. > > > > # for snapshot in `zfs list -t snapshot | grep zroot/bhyve/test | awk > > '{print $1}'`; do zfs destroy $snapshot ; done > > > > # zfs get all zroot/bhyve/test | grep snapshot > > zroot/bhyve/test usedbysnapshots 0 - > > zroot/bhyve/test snapshot_limit 0 local > > zroot/bhyve/test snapshot_count 0 local > > > > # zfs set snapshot_limit=1 zroot/bhyve/test > > # zfs snapshot zroot/bhyve/test@1 > > # zfs snapshot zroot/bhyve/test@2 > > # zfs snapshot zroot/bhyve/test@3 > > # zfs get all zroot/bhyve/test | grep snapshot > > zroot/bhyve/test usedbysnapshots 0 - > > zroot/bhyve/test snapshot_limit 1 local > > zroot/bhyve/test snapshot_count 3 local > > > > > > Also tested on head > > FreeBSD S1 12.0-CURRENT FreeBSD 12.0-CURRENT #26 r312388: Wed Jan 18 > > 12:38:52 EST 2017 > > root@S1:/usr/src/head/obj/usr/src/head/src/sys/MYKERNEL-NODEBUG > > amd64 > > Hi, > > I suspect this line from the manpage is key: > > The limit is not enforced if the user is allowed to change the limit > > Regards, > > Gary > From owner-freebsd-fs@freebsd.org Fri Feb 3 16:12:10 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A6E0CCF536; Fri, 3 Feb 2017 16:12:10 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from mail.in-addr.com (mail.in-addr.com [IPv6:2a01:4f8:191:61e8::2525:2525]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DA64E8C7; Fri, 3 Feb 2017 16:12:09 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from gjp by mail.in-addr.com with local (Exim 4.88 (FreeBSD)) (envelope-from ) id 1cZgT2-0003hV-22; Fri, 03 Feb 2017 16:12:08 +0000 Date: Fri, 3 Feb 2017 16:12:07 +0000 From: Gary Palmer To: Ultima Cc: freebsd-stable@freebsd.org, freebsd-current@freebsd.org, freebsd-fs@freebsd.org Subject: Re: zfs snapshot_limit is not respected Message-ID: <20170203161207.GB68015@in-addr.com> References: <20170203124251.GA68015@in-addr.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on mail.in-addr.com); SAEximRunCond expanded to false X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Feb 2017 16:12:10 -0000 On Fri, Feb 03, 2017 at 10:27:48AM -0500, Ultima wrote: > Hey Gary, > > You are probably right. Do you know how to "lock" this property by chance? > I'v read this exact line several times trying to understand the exact > meaning. The "user is allowed to change the limit" I *think* is referring > to the zfs allow command. The problem is that I checked the dataset and it > is showing no permissions granted to a user. So I guess user in this case > is also including the root user, but how does one lock the property from > root? I keep going through the manpage looking for something I may have > missed but keep coming up empty. Hi, I suspect you can't lock root out, but you could allow a different user to create snapshots. If you use that user to create snapshots, they should be subject to the snapshot limit, assuming you don't let them change that property. Regards, Gary > Thanks for replying, > Ultima > > On Fri, Feb 3, 2017 at 7:42 AM, Gary Palmer wrote: > > > On Thu, Feb 02, 2017 at 09:31:58PM -0500, Ultima wrote: > > > I recently moved some data on a box with limited space. I decided I > > should > > > limit the snapshots so that space would not become an issue. I just check > > > back a week later to find out the box is hitting the borderline. Doing I > > > quick check I realized that the snapshot_limit is not being respected. > > > > > > # uname -a > > > FreeBSD R1 11.0-STABLE FreeBSD 11.0-STABLE #17 r312232: Sun Jan 15 > > 10:59:10 > > > EST 2017 root@S1:/usr/src/11-STABLE/obj/usr/src/11-STABLE/src/sys/ > > MYKERNEL > > > amd64 > > > > > > # zfs create zroot/bhyve/test > > > # zfs set snapshot_limit=0 zroot/bhyve/test > > > # zfs snapshot zroot/bhyve/test@1 > > > > > > > > > # zfs snapshot zroot/bhyve/test@2 > > > # zfs snapshot zroot/bhyve/test@3 > > > # zfs list -t snapshot | grep zroot/bhyve/test > > > zroot/bhyve/test@1 0 - > > > 88K - > > > zroot/bhyve/test@2 0 - > > > 88K - > > > zroot/bhyve/test@3 0 - > > > 88K - > > > # zfs get all zroot/bhyve/test | grep snapshot > > > zroot/bhyve/test usedbysnapshots 0 - > > > zroot/bhyve/test snapshot_limit 0 local > > > zroot/bhyve/test snapshot_count 3 local > > > > > > Also wanted to verify 0 was not being mistaken for none. > > > > > > # for snapshot in `zfs list -t snapshot | grep zroot/bhyve/test | awk > > > '{print $1}'`; do zfs destroy $snapshot ; done > > > > > > # zfs get all zroot/bhyve/test | grep snapshot > > > zroot/bhyve/test usedbysnapshots 0 - > > > zroot/bhyve/test snapshot_limit 0 local > > > zroot/bhyve/test snapshot_count 0 local > > > > > > # zfs set snapshot_limit=1 zroot/bhyve/test > > > # zfs snapshot zroot/bhyve/test@1 > > > # zfs snapshot zroot/bhyve/test@2 > > > # zfs snapshot zroot/bhyve/test@3 > > > # zfs get all zroot/bhyve/test | grep snapshot > > > zroot/bhyve/test usedbysnapshots 0 - > > > zroot/bhyve/test snapshot_limit 1 local > > > zroot/bhyve/test snapshot_count 3 local > > > > > > > > > Also tested on head > > > FreeBSD S1 12.0-CURRENT FreeBSD 12.0-CURRENT #26 r312388: Wed Jan 18 > > > 12:38:52 EST 2017 > > > root@S1:/usr/src/head/obj/usr/src/head/src/sys/MYKERNEL-NODEBUG > > > amd64 > > > > Hi, > > > > I suspect this line from the manpage is key: > > > > The limit is not enforced if the user is allowed to change the limit > > > > Regards, > > > > Gary > > > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Fri Feb 3 16:44:02 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 84919CCE050; Fri, 3 Feb 2017 16:44:02 +0000 (UTC) (envelope-from ultima1252@gmail.com) Received: from mail-yb0-x231.google.com (mail-yb0-x231.google.com [IPv6:2607:f8b0:4002:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 35655177B; Fri, 3 Feb 2017 16:44:02 +0000 (UTC) (envelope-from ultima1252@gmail.com) Received: by mail-yb0-x231.google.com with SMTP id w194so7664274ybe.0; Fri, 03 Feb 2017 08:44:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=DSwEZUv9I7ly4VSc/iYaj5I+vRlU7Wvz5KkTTdiRjdg=; b=pHg+aUedD3UxK6K17pXg8f2p/Uv+DID12DY4TagUnfI9+a40B0z8b+E17K+PCTqin0 l+Hu0oqMTwxnftRS0nEDBQqTazOmbmxt9uUTODJMO09rjAi9Z2EnViEcUB8wd2KW1qQF 4DWueGjrMCIP1msYSFVDLlMTmTscYA9yIkpKJVhNhXrNIH+04UodIJQZwhNuahIeJsyK BfMPBFXn9Yvug029zLhnE6jysptop+OGqZi6ytW+p5BulXjP1eTvYLlnR+HuPYEWmydz likhWGGe5P0RCA5lAaHKqiP3VoV/IHZmyvTrZzdTkkSHah2GQnR/6UqB5g92W3KRkMXl z9nA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=DSwEZUv9I7ly4VSc/iYaj5I+vRlU7Wvz5KkTTdiRjdg=; b=Mol+9YYXydtn643+HbS7w3VHf7SJjJddNthiHwk3EVHelbk9uNx4WdoxyTHvtSJC0J ZD/twrBJ7uTkcyJv8abciuGhuR8hHFUnX/9znQA+6mvBVxhp9U3Y/f81TcPpC3G7zf+1 inEkGQijCarhv4AMB+wM88g2Lg0hAI+Ne0jJ5uBfvdSr8wK55N67yzx3CqulfJwvm044 cZrqxeOVaeZjCdMYBz6sulKgWaboVOH+aOhTnrSYkxsKqXqUClCzlOZYNsIYJre2kEyT PMzMVvNSL2DUUTqQYTYRM34fKMx/NK464tiJDidrtS1nT7s/kG5PPgSVNX7GDDdsHrxs VLgg== X-Gm-Message-State: AIkVDXKuo/GLw/VC5oZiZowefaQT4ZonC2CJwruT/c6ZdnjLpUNXYtR7wgep2HFu7voru55DEIGEwl1N27nAUg== X-Received: by 10.37.56.208 with SMTP id f199mr6592142yba.94.1486140241182; Fri, 03 Feb 2017 08:44:01 -0800 (PST) MIME-Version: 1.0 Received: by 10.129.52.65 with HTTP; Fri, 3 Feb 2017 08:44:00 -0800 (PST) In-Reply-To: <20170203161207.GB68015@in-addr.com> References: <20170203124251.GA68015@in-addr.com> <20170203161207.GB68015@in-addr.com> From: Ultima Date: Fri, 3 Feb 2017 11:44:00 -0500 Message-ID: Subject: Re: zfs snapshot_limit is not respected To: Gary Palmer Cc: freebsd-stable@freebsd.org, freebsd-current@freebsd.org, freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Feb 2017 16:44:02 -0000 Yes just tested this and it is how it works. Thanks for the explanation. Ultima From owner-freebsd-fs@freebsd.org Fri Feb 3 17:44:51 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8604BCCF81C for ; Fri, 3 Feb 2017 17:44:51 +0000 (UTC) (envelope-from stephen.joseph@united-email.info) Received: from mserver.united-email.info (mail.united-email.info [137.59.52.162]) by mx1.freebsd.org (Postfix) with ESMTP id DD9C3785 for ; Fri, 3 Feb 2017 17:44:50 +0000 (UTC) (envelope-from stephen.joseph@united-email.info) Received: from UnitedPC (unknown [202.133.79.30]) by mserver.united-email.info (Postfix) with ESMTPA id 330BB193EC3 for ; Fri, 3 Feb 2017 23:16:08 +0530 (IST) Reply-To: From: "Stephen Joseph" To: Subject: Ask a question Date: Fri, 3 Feb 2017 12:44:43 -0500 Message-ID: MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: AdJ+RGgnnbjJcY1rT9m4KbzJF0x6Tw== Content-Language: en-us x-cr-hashedpuzzle: K5o= AnBf CRCe EBQH ENYB E6Zf FGvh FX5q G1jT HUfl HV7q IcfQ IdTI Igy4 KF7F KHof; 1; ZgByAGUAZQBiAHMAZAAtAGYAcwBAAGYAcgBlAGUAYgBzAGQALgBvAHIAZwA=; Sosha1_v1; 7; {E2262D18-24FB-405A-B012-F93BA4785ED8}; cwB0AGUAcABoAGUAbgAuAGoAbwBzAGUAcABoAEAAdQBuAGkAdABlAGQALQBlAG0AYQBpAGwALgBpAG4AZgBvAA==; Fri, 03 Feb 2017 17:44:06 GMT; QQBzAGsAIABhACAAcQB1AGUAcwB0AGkAbwBuAA== x-cr-puzzleid: {E2262D18-24FB-405A-B012-F93BA4785ED8} Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Feb 2017 17:44:51 -0000 Hi, My name is Stephen Joseph, the Business Development Manager at one of = the leading email database providing companies.=20 Our area of expertise lies specifically in the major industrial lists = like: Agriculture, Business Services, Chambers of Commerce, Cities, Towns & Municipalities, Construction, Consumer Services, Cultural, Education, Energy, Utilities & Waste Treatment, Finance, Government, Healthcare, Hospitality, Insurance, Law Firms & Legal Services, Manufacturing, Media = & Internet, Metals & Mining, Organizations, Real Estate, Retail, Software, Telecommunications, Transportation, etc. Building long term relationships with our clients is something we = strongly believe in and we go to great lengths to ensure our valuable clients = receive first rate service. Some of the most popular services we offer include: =FC List service #1 - B2B & B2C Email Lists =FC List service #2 -Data Processing & Email Appending =FC List service #3 - Email Campaigning =20 We=92d welcome the opportunity to discuss your email list needs.=20 Best Regards, =20 Stephen Joseph | Business Development Associate | T: +1 610 572 4885 Global Business Data =96 Email Append =96 Data Append. =20 If you are not interested in receiving further emails, please reply back with =93LEAVEOUT=94 in the subject line=94. =20 From owner-freebsd-fs@freebsd.org Fri Feb 3 23:02:17 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 18F30CCFBCF for ; Fri, 3 Feb 2017 23:02:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 01DBCC5F for ; Fri, 3 Feb 2017 23:02:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v13N2GmN038492 for ; Fri, 3 Feb 2017 23:02:16 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216586] zfs panic: sa.sa_magic == 0x2F505A in zfs_space_delta_cb() Date: Fri, 03 Feb 2017 23:02:17 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: johannes@jo-t.de X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 Feb 2017 23:02:17 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216586 --- Comment #4 from johannes@jo-t.de --- Thanks Andriy. There are indeed lots of (very) old files there. I had an earlier snapshot = from a few years ago (that refs all these files). Deleted all files in the fs, a= nd rsync'd everything anew. No panic. I've also tried (first rolling back to the old snapshot with all old files present) rsync with either --xattrs present or not. Panics for both cases. But the time until panic varies a lot. So looks like a race condition. Question is now, is there bogus data (old format that wasn't properly upgra= ded on-the-fly) with the correct checksum on the disk, and the current code tri= ps over it. Or is the race condition still present when it tries to on-the-fly-upgrade = old files? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat Feb 4 12:36:05 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DE686CCD9A0 for ; Sat, 4 Feb 2017 12:36:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CE49033E for ; Sat, 4 Feb 2017 12:36:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v14Ca4nR001038 for ; Sat, 4 Feb 2017 12:36:05 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 216586] zfs panic: sa.sa_magic == 0x2F505A in zfs_space_delta_cb() Date: Sat, 04 Feb 2017 12:36:05 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 10.3-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 04 Feb 2017 12:36:06 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D216586 --- Comment #5 from Andriy Gapon --- (In reply to johannes from comment #4) I believe that it is the race condition. https://github.com/zfsonlinux/zfs/issues/2025#issuecomment-40459546 I think that what I wrote there still holds and there hasn't been any fix in this area. --=20 You are receiving this mail because: You are the assignee for the bug.=