From owner-freebsd-fs@freebsd.org Mon Aug 3 16:10:31 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8335437EE6B for ; Mon, 3 Aug 2020 16:10:31 +0000 (UTC) (envelope-from swills@FreeBSD.org) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BL2sz1c9Nz42Tj for ; Mon, 3 Aug 2020 16:10:30 +0000 (UTC) (envelope-from swills@FreeBSD.org) Received: from lrrr.mouf.net (2606-a000-40c4-3e01-e9b5-43e9-cd72-ab13.inf6.spectrum.com [IPv6:2606:a000:40c4:3e01:e9b5:43e9:cd72:ab13]) (authenticated bits=0) by mouf.net (8.14.9/8.14.9) with ESMTP id 073GAIAJ041441 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT) for ; Mon, 3 Aug 2020 16:10:23 GMT (envelope-from swills@FreeBSD.org) To: freebsd-fs@freebsd.org From: Steve Wills Subject: zfs scrub enable by default Message-ID: Date: Mon, 3 Aug 2020 12:10:10 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]); Mon, 03 Aug 2020 16:10:23 +0000 (UTC) X-Spam-Status: No, score=0.2 required=4.5 tests=KHOP_HELO_FCRDNS autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mouf.net X-Virus-Scanned: clamav-milter 0.99.2 at mouf.net X-Virus-Status: Clean X-Rspamd-Queue-Id: 4BL2sz1c9Nz42Tj X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; ASN(0.00)[asn:36236, ipnet:2607:fc50::/36, country:US] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Aug 2020 16:10:31 -0000 Hi, I wonder why we don't enable zfs periodic scrub by default? https://svnweb.freebsd.org/base/head/usr.sbin/periodic/periodic.conf?view=markup#l162 Anyone happen to know? Thanks, Steve From owner-freebsd-fs@freebsd.org Mon Aug 3 16:35:38 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1709A37FC78 for ; Mon, 3 Aug 2020 16:35:38 +0000 (UTC) (envelope-from wfc@mintsol.com) Received: from scully.mintsol.com (scully.mintsol.com [199.182.77.206]) by mx1.freebsd.org (Postfix) with ESMTP id 4BL3Qx4vTWz44Jm; Mon, 3 Aug 2020 16:35:37 +0000 (UTC) (envelope-from wfc@mintsol.com) Received: from mintsol.com (officecc.mintsol.com [96.85.114.33]) by scully.mintsol.com with esmtp; Mon, 03 Aug 2020 12:35:30 -0400 id 00B0D069.000000005F283CD2.0000FF9C Received: from localhost (localhost [127.0.0.1]) (IDENT: uid 1002) by mintsol.com with esmtp; Mon, 03 Aug 2020 12:35:30 -0400 id 00000AAC.5F283CD2.0000E630 Date: Mon, 3 Aug 2020 12:35:30 -0400 (EDT) From: Walter Cramer To: Steve Wills cc: freebsd-fs@freebsd.org Subject: Re: zfs scrub enable by default In-Reply-To: Message-ID: <20200803121444.J57111@mulder.mintsol.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4BL3Qx4vTWz44Jm X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:22768, ipnet:199.182.77.0/24, country:US] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Aug 2020 16:35:38 -0000 Assuming a fairly-idle system, or one with a tiny zpool - scrubs from 'periodic daily' can work fine. In cases where that assumption is false - the resources consumed by an hours-long scrub, slowing down a system which people expect to be "as responsive as it usually it"... Our defaults should try to minimize the nasty surprises. -Walter On Mon, 3 Aug 2020, Steve Wills wrote: > Hi, > > I wonder why we don't enable zfs periodic scrub by default? > > https://svnweb.freebsd.org/base/head/usr.sbin/periodic/periodic.conf?view=markup#l162 > > Anyone happen to know? > > Thanks, > Steve > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Mon Aug 3 18:05:10 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 71EDF3A396F for ; Mon, 3 Aug 2020 18:05:10 +0000 (UTC) (envelope-from pen@lysator.liu.se) Received: from mail.lysator.liu.se (mail.lysator.liu.se [130.236.254.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BL5QD6QXLz4CMw for ; Mon, 3 Aug 2020 18:05:08 +0000 (UTC) (envelope-from pen@lysator.liu.se) Received: from mail.lysator.liu.se (localhost [127.0.0.1]) by mail.lysator.liu.se (Postfix) with ESMTP id 523B640007 for ; Mon, 3 Aug 2020 20:05:05 +0200 (CEST) Received: by mail.lysator.liu.se (Postfix, from userid 1004) id 374504000A; Mon, 3 Aug 2020 20:05:05 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on bernadotte.lysator.liu.se X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED, AWL, HTML_MESSAGE autolearn=disabled version=3.4.2 X-Spam-Score: -1.0 Received: from [IPv6:2001:9b1:28ff:d901:5d72:52a7:c896:6e29] (unknown [IPv6:2001:9b1:28ff:d901:5d72:52a7:c896:6e29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.lysator.liu.se (Postfix) with ESMTPSA id 679FB40007 for ; Mon, 3 Aug 2020 20:05:04 +0200 (CEST) From: Peter Eriksson Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.80.23.2.2\)) Subject: Re: zfs scrub enable by default Date: Mon, 3 Aug 2020 20:05:01 +0200 References: <20200803121444.J57111@mulder.mintsol.com> To: freebsd-fs In-Reply-To: <20200803121444.J57111@mulder.mintsol.com> Message-Id: <0BF85C14-5671-4779-928A-34C5FD3B82C1@lysator.liu.se> X-Mailer: Apple Mail (2.3608.80.23.2.2) X-Virus-Scanned: ClamAV using ClamSMTP X-Rspamd-Queue-Id: 4BL5QD6QXLz4CMw X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=liu.se; spf=pass (mx1.freebsd.org: domain of pen@lysator.liu.se designates 130.236.254.3 as permitted sender) smtp.mailfrom=pen@lysator.liu.se X-Spamd-Result: default: False [-3.26 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.03)[-1.028]; FROM_HAS_DN(0.00)[]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; NEURAL_HAM_LONG(-1.01)[-1.010]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_ALL(0.00)[]; RCVD_IN_DNSWL_MED(-0.20)[130.236.254.3:from]; DMARC_POLICY_ALLOW(-0.50)[liu.se,none]; NEURAL_HAM_SHORT(-0.73)[-0.725]; R_SPF_ALLOW(-0.20)[+a:mail.lysator.liu.se]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:2843, ipnet:130.236.0.0/16, country:SE]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Aug 2020 18:05:10 -0000 On our pretty big ZFS fileservers (home directories) we run scrubs = manually but every now and then - but I have a script that pauses the = scrubs during =E2=80=9Cprime time=E2=80=9D since a full scrub run takes = a day or so... Crontab: # zpool-scrub-pause 0 8 * * 1-5 /sbin/zpool-scrub pause all >/var/liu/log/zpool-scrub-pause = 2>&1 # zpool-scrub-resume 0 21 * * 1-5 /sbin/zpool-scrub resume all = >/var/liu/log/zpool-scrub-resume 2>&1 The script can be downloaded from: https://www.grebo.net/~peter/zfs/ Perhaps something similar could be used with configurable =E2=80=9Cprime = time hours=E2=80=9D)? - Peter > On 3 Aug 2020, at 18:35, Walter Cramer wrote: >=20 > Assuming a fairly-idle system, or one with a tiny zpool - scrubs from = 'periodic daily' can work fine. >=20 > In cases where that assumption is false - the resources consumed by an = hours-long scrub, slowing down a system which people expect to be "as = responsive as it usually it"... >=20 > Our defaults should try to minimize the nasty surprises. >=20 > -Walter >=20 >=20 > On Mon, 3 Aug 2020, Steve Wills wrote: >=20 >> Hi, >>=20 >> I wonder why we don't enable zfs periodic scrub by default? >>=20 >> = https://svnweb.freebsd.org/base/head/usr.sbin/periodic/periodic.conf?view=3D= markup#l162 >>=20 >> Anyone happen to know? >>=20 >> Thanks, >> Steve >>=20 >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>=20 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Mon Aug 3 18:11:14 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A112B3A3BE6 for ; Mon, 3 Aug 2020 18:11:14 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BL5YF5JXVz4DDb for ; Mon, 3 Aug 2020 18:11:13 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (096-033-205-208.res.spectrum.com [96.33.205.208]) by colo1.denninger.net (Postfix) with ESMTP id 244F32110B3 for ; Mon, 3 Aug 2020 14:11:06 -0400 (EDT) Received: from [192.168.10.25] (D15.Denninger.Net [192.168.10.25]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id B717923D41F for ; Mon, 3 Aug 2020 14:11:06 -0400 (EDT) Subject: Re: zfs scrub enable by default To: freebsd-fs@freebsd.org References: <20200803121444.J57111@mulder.mintsol.com> <0BF85C14-5671-4779-928A-34C5FD3B82C1@lysator.liu.se> From: Karl Denninger Message-ID: Date: Mon, 3 Aug 2020 14:11:06 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.1.0 MIME-Version: 1.0 In-Reply-To: <0BF85C14-5671-4779-928A-34C5FD3B82C1@lysator.liu.se> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms090202070506070700040702" X-Rspamd-Queue-Id: 4BL5YF5JXVz4DDb X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=denninger.net; spf=pass (mx1.freebsd.org: domain of karl@denninger.net designates 104.236.120.189 as permitted sender) smtp.mailfrom=karl@denninger.net X-Spamd-Result: default: False [-5.11 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.01)[-1.008]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.03)[-1.028]; RCVD_COUNT_THREE(0.00)[3]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; NEURAL_HAM_SHORT(-0.17)[-0.169]; DMARC_POLICY_ALLOW(-0.50)[denninger.net,none]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~,4:~]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Aug 2020 18:11:14 -0000 This is a cryptographically signed message in MIME format. --------------ms090202070506070700040702 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable On 8/3/2020 14:05, Peter Eriksson wrote: > On our pretty big ZFS fileservers (home directories) we run scrubs manu= ally but every now and then - but I have a script that pauses the scrubs = during =E2=80=9Cprime time=E2=80=9D since a full scrub run takes a day or= so... > > > Crontab: > > # zpool-scrub-pause > 0 8 * * 1-5 /sbin/zpool-scrub pause all >/var/liu/log/zpool-scrub-pause= 2>&1 > # zpool-scrub-resume > 0 21 * * 1-5 /sbin/zpool-scrub resume all >/var/liu/log/zpool-scrub-res= ume 2>&1 > > > The script can be downloaded from: > > https://www.grebo.net/~peter/zfs/ > > > Perhaps something similar could be used with configurable =E2=80=9Cprim= e time hours=E2=80=9D)? > > - Peter > > >> On 3 Aug 2020, at 18:35, Walter Cramer wrote: >> >> Assuming a fairly-idle system, or one with a tiny zpool - scrubs from = 'periodic daily' can work fine. >> >> In cases where that assumption is false - the resources consumed by an= hours-long scrub, slowing down a system which people expect to be "as re= sponsive as it usually it"... >> >> Our defaults should try to minimize the nasty surprises. >> >> -Walter >> >> >> On Mon, 3 Aug 2020, Steve Wills wrote: >> >>> Hi, >>> >>> I wonder why we don't enable zfs periodic scrub by default? >>> >>> https://svnweb.freebsd.org/base/head/usr.sbin/periodic/periodic.conf?= view=3Dmarkup#l162 >>> >>> Anyone happen to know? >>> >>> Thanks, >>> Steve I will pipe up AGAINST making this a default "do on its own" thing. It is trivially automated in the cron, and this allows it to be tailored = by the load pattern and expectations of the individual configuration.=C2=A0= =C2=A0=20 A documentation entry explaining the value and need to do it on *some*=20 sort of schedule makes sense, but automating it is IMHO just begging for = trouble. The "very small" ZFS system (where there's no actual enhancement to=20 redundancy) is one degenerate case where a noted error can only tell you = that you need to find the impacted file(s) and restore them from backup, = since there is no ability to work around the fault.=C2=A0 On the other ha= nd=20 larger systems where the time required to complete a scan impacts=20 "working hours" (whatever that means in the context of the usual load)=20 will be materially impacted by an automated scan in most cases, and thus = should be hand-tuned anyway. --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms090202070506070700040702 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMjAwODAzMTgxMTA2 WjBPBgkqhkiG9w0BCQQxQgRAli6k5884FGeaqd1tZvQvscalhUk4B2fKX6uue6bzWTOEPqVn U8UpY4VL4BiZ1n3mzUht9VhrUNSxUmTCXaO9TTBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgBGwpsjUP+xtRd5iL9cP0ttYYCMbIuYnpX2GouxqcfSrWnouA+NYf1URHxz1cHvDnyV 9Nvkm23t7wlkdkvxUZKV867bS9bkaoTmXCFT2jF62qLFpJacO6wJK4QPhujbPzF/g3X4BpnS rPkAS19BKr3ndvHOj5XErxDaX5bBCfeDBrpr15mFmdQM8/zvndeLSzB/K3JQ1YiDYBQ4gWEY 9PgI8jUc/TDRSaPmkatH1dPJsQDjq0v7WNlwQoT32JzNzWgYnpobEalA7Pry38eEHW9MBipO s4irde4XXqSsAfPz3rBv0RtC7/J2eQynrCsBU9aES7hvHrbOFmhnCsUQo/1t4T/QZFgwAaVr iSTY8ixBfKjNXGyXzHnnmRGQYnfK8PbcxSnBckRRNB8RbRYV7IeKFE4Oo6ygGxekFK5q3YCQ re0lw3hWTetzm7kJ2uOQAV1VgwNiLOK0vEASOW71Jvbx+x1Ch6BiV11fUCa2cZSh+1YfGPqw iBwcZ/Y7pkKo1FT8uxqFaqPHp4qQP0xMYYHonp+8mN6dDVGTgdjNFV8DabXcX0SH0yjywS05 hOHJzQpf+W4dK7RSxZC+IHBOffpcFS837bVNXfvUgApUVQyfzO+iBNBxyPhgfpii4/ndH2Zy gab9/Y/Dm4vgUAtwCI1YJePl4nUInW+iYFhY9W4GZQAAAAAAAA== --------------ms090202070506070700040702-- From owner-freebsd-fs@freebsd.org Mon Aug 3 20:26:00 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 736E23A6BC9 for ; Mon, 3 Aug 2020 20:26:00 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from tor1-11.mx.scaleengine.net (tor1-11.mx.scaleengine.net [209.51.186.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BL8Xm23fhz4MVs for ; Mon, 3 Aug 2020 20:26:00 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by tor1-11.mx.scaleengine.net (Postfix) with ESMTPSA id 38B521A211 for ; Mon, 3 Aug 2020 20:25:59 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.10.3 tor1-11.mx.scaleengine.net 38B521A211 To: freebsd-fs@freebsd.org References: From: Allan Jude Autocrypt: addr=allanjude@freebsd.org; prefer-encrypt=mutual; keydata= xsFNBFVwZcYBEADwrZDH0xe0ZVjc9ORCc6PcBLwS/RTXA6NkvpD6ea02pZ8lPOVgteuuugFc D34LdDbiWr+479vfrKBh+Y38GL0oZ0/13j10tIlDMHSa5BU0y6ACtnhupFvVlQ57+XaJAb/q 7qkfSiuxVwQ3FY3PL3cl1RrIP5eGHLA9hu4eVbu+FOX/q/XVKz49HaeIaxzo2Q54572VzIo6 C28McX9m65UL5fXMUGJDDLCItLmehZlHsQQ+uBxvODLFpVV2lUgDR/0rDa0B9zHZX8jY8qQ7 ZdCSy7CwClXI054CkXZCaBzgxYh/CotdI8ezmaw7NLs5vWNTxaDEFXaFMQtMVhvqQBpHkfOD 7rjjOmFw00nJL4FuPE5Yut0CPyx8vLjVmNJSt/Y8WxxmhutsqJYFgYfWl/vaWkrFLur/Zcmz IklwLw35HLsCZytCN5A3rGKdRbQjD6QPXOTJu0JPrJF6t2xFkWAT7oxnSV0ELhl2g+JfMMz2 Z1PDmS3NRnyEdqEm7NoRGXJJ7bgxDbN+9SXTyOletqGNXj/bSrBvhvZ0RQrzdHAPwQUfVSU2 qBhQEi2apSZstgVNMan0GUPqCdbE2zpysg+zT7Yhvf9EUQbzPL4LpdK1llT9fZbrdMzEXvEF oSvwJFdV3sqKmZc7b+E3PuxK6GTsKqaukd/3Cj8aLHG1T1im1QARAQABzSJBbGxhbiBKdWRl IDxhbGxhbmp1ZGVAZnJlZWJzZC5vcmc+wsF/BBMBAgApBQJVcGXGAhsjBQkSzAMABwsJCAcD AgEGFQgCCQoLBBYCAwECHgECF4AACgkQGZU1PhKYC34Muw/+JOKpSfhhysWFYiRXynGRDe07 Z6pVsn7DzrPUMRNZfHu8Uujmmy3p2nx9FelIY9yjd2UKHhug+whM54MiIFs90eCRVa4XEsPR 4FFAm0DAWrrb7qhZFcE/GhHdRWpZ341WAElWf6Puj2devtRjfYbikvj5+1V1QmDbju7cEw5D mEET44pTuD2VMRJpu2yZZzkM0i+wKFuPxlhqreufA1VNkZXI/rIfkYWK+nkXd9Efw3YdCyCQ zUgTUCb88ttSqcyhik/li1CDbXBpkzDCKI6I/8fAb7jjOC9LAtrZJrdgONywcVFoyK9ZN7EN AVA+xvYCmuYhR/3zHWH1g4hAm1v1+gIsufhajhfo8/wY1SetlzPaYkSkVQLqD8T6zZyhf+AN bC7ci44UsiKGAplB3phAXrtSPUEqM86kbnHg3fSx37kWKUiYNOnx4AC2VXvEiKsOBlpyt3dw WQbOtOYM+vkfbBwDtoGOOPYAKxc4LOIt9r+J8aD+gTooi9Eo5tvphATf9WkCpl9+aaGbSixB tUpvQMRnSMqTqq4Z7DeiG6VMRQIjsXDSLJEUqcfhnLFo0Ko/RiaHd5xyAQ4DhQ9QpkyQjjNf /3f/dYG7JAtoD30txaQ5V8uHrz210/77DRRX+HJjEj6xCxWUGvQgvEZf5XXyxeePvqZ+zQyT DX61bYw6w6bOwU0EVXBlxgEQAMy7YVnCCLN4oAOBVLZ5nUbVPvpUhsdA94/0/P+uqCIh28Cz ar56OCX0X19N/nAWecxL4H32zFbIRyDB2V/MEh4p9Qvyu/j4i1r3Ex5GhOT2hnit43Ng46z5 29Es4TijrHJP4/l/rB2VOqMKBS7Cq8zk1cWqaI9XZ59imxDNjtLLPPM+zQ1yE3OAMb475QwN UgWxTMw8rkA7CEaqeIn4sqpTSD5C7kT1Bh26+rbgJDZ77D6Uv1LaCZZOaW52okW3bFbdozV8 yM2u+xz2Qs8bHz67p+s+BlygryiOyYytpkiK6Iy4N7FTolyj5EIwCuqzfk0SaRHeOKX2ZRjC qatkgoD/t13PNT38V9tw3qZVOJDS0W6WM8VSg+F+bkM9LgJ8CmKV+Hj0k3pfGfYPOZJ/v18i +SmZmL/Uw2RghnwDWGAsPCKu4uZR777iw7n9Io6Vfxndw2dcS0e9klvFYoaGS6H2F13Asygr WBzFNGFQscN4mUW+ZYBzpTOcHkdT7w8WS55BmXYLna+dYer9/HaAuUrONjujukN4SPS1fMJ2 /CS/idAUKyyVVX5vozoNK2JVC1h1zUAVsdnmhEzNPsvBoqcVNfyqBFROEVLIPwq+lQMGNVjH ekLTKRWf59MEhUC2ztjSKkGmwdg73d6xSXMuq45EgIJV2wPvOgWQonoHH/kxABEBAAHCwWUE GAECAA8FAlVwZcYCGwwFCRLMAwAACgkQGZU1PhKYC34w5A//YViBtZyDV5O+SJT9FFO3lb9x Zdxf0trA3ooCt7gdBkdnBM6T5EmjgVZ3KYYyFfwXZVkteuCCycMF/zVw5eE9FL1+zz9gg663 nY9q2F77TZTKXVWOLlOV2bY+xaK94U4ytogOGhh9b4UnQ/Ct3+6aviCF78Go608BXbmF/GVT 7uhddemk7ItxM1gE5Hscx3saxGKlayaOsdPKeGTVJCDEtHDuOc7/+jGh5Zxpk/Hpi+DUt1ot 8e6hPYLIQa4uVx4f1xxxV858PQ7QysSLr9pTV7FAQ18JclCaMc7JWIa3homZQL/MNKOfST0S 2e+msuRwQo7AnnfFKBUtb02KwpA4GhWryhkjUh/kbVc1wmGxaU3DgXYQ5GV5+Zf4kk/wqr/7 KG0dkTz6NLCVLyDlmAzuFhf66DJ3zzz4yIo3pbDYi3HB/BwJXVSKB3Ko0oUo+6/qMrOIS02L s++QE/z7K12CCcs7WwOjfCYHK7VtE0Sr/PfybBdTbuDncOuAyAIeIKxdI2nmQHzl035hhvQX s4CSghsP319jAOQiIolCeSbTMD4QWMK8RL/Pe1FI1jC3Nw9s+jq8Dudtbcj2UwAP/STUEbJ9 5rznzuuhPjE0e++EU/RpWmcaIMK/z1zZDMN+ce2v1qzgV936ZhJ3iaVzyqbEE81gDxg3P+IM kiYh4ZtPB4Q= Subject: Re: zfs scrub enable by default Message-ID: <24edb075-155c-439d-1ef5-541893036429@freebsd.org> Date: Mon, 3 Aug 2020 16:25:55 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="dUDSsrYzWePLsuMJ6BLynjBl8cuScCtBf" X-Rspamd-Queue-Id: 4BL8Xm23fhz4MVs X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; ASN(0.00)[asn:6939, ipnet:209.51.160.0/19, country:US] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Aug 2020 20:26:00 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --dUDSsrYzWePLsuMJ6BLynjBl8cuScCtBf Content-Type: multipart/mixed; boundary="k6fK7soGuFtXcTR5VEmoLkcvVDV0EkrAL"; protected-headers="v1" From: Allan Jude To: freebsd-fs@freebsd.org Message-ID: <24edb075-155c-439d-1ef5-541893036429@freebsd.org> Subject: Re: zfs scrub enable by default References: In-Reply-To: --k6fK7soGuFtXcTR5VEmoLkcvVDV0EkrAL Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 2020-08-03 12:10, Steve Wills wrote: > Hi, >=20 > I wonder why we don't enable zfs periodic scrub by default? >=20 > https://svnweb.freebsd.org/base/head/usr.sbin/periodic/periodic.conf?vi= ew=3Dmarkup#l162 >=20 >=20 > Anyone happen to know? >=20 > Thanks, > Steve >=20 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" I think switching this to on-by-default is a good thing. To be clear, which the check is part of 'periodic daily', it only triggers a scrub if it has been more than 35 days since the last scrub. FreeNAS already has does this, and that accounts for a large number of FreeBSD ZFS deployments. There is tuning you can do in ZFS to try to lessen the impact of a scrub on your production workloads. The periodic script lets you select which pools to include (defaults to all), so you can tune it to only scrub your root pool every 35 days, and not the large pool that might take too long to scrub or whatever. It also lets you set a different threshold for each pool. So I don't see any reason not to enable it by default, and just document how to adjust it if people really need to disable it. Honestly, I think those who are disabling it are doing themselves a disservice. --=20 Allan Jude --k6fK7soGuFtXcTR5VEmoLkcvVDV0EkrAL-- --dUDSsrYzWePLsuMJ6BLynjBl8cuScCtBf Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJfKHLWAAoJEBmVNT4SmAt+CNEQALkMvi5q03PoXcueqA7sxejH +fAe5dNtvqDYKRY0J8MQJ44hMJFR3tpoKvwpb8f5JX9ALDIleiM/8o2bdTCYr3lf Sa7lN6ZVGPXD3gSXuF+p2Dra1u1mt3hA+qrIKXrPeToC8hUQ+SOUFfPIu2s0tZWd gKDUfaYHzOVgRbPyfrTCAHlZhAOIUy014xhRDBjrG9seH7aXjlaN1X+soGDqtG1P CXfIp67KY8XepZjM9Ig6o7K3j7XoGhtY9n+xPS+Bbaq8VhG7yJD8CR3EaxHpdkCI AgS5RDL/VRNtzHF43BdQvJKFVl1KR61JNFvdthkPy9k10HXEmgSBpiH312iDJAFY sIKVGTI/F0Xil/KutdVcxY8px1FaJkwb1NHN3khdNUv8AaPvKVzkpli+5BTY6B+C NzCQzaJZffwkaDMOer9/63zHVoIt22Ia+EGHVaHmm79J8zxFX37JNGoV1NUlKIKu pMrXarTY3ksYNVcMwm7tNMshT9kPQSReRV0xhsVbRtDWDzsZvtmFJEYpniY5gDEn lHjHnNH5Ad/ya/KQrYdak+uKGTjToMNL//xvrEfL4AFeZ93+On3DjQ/M9VvdONUJ xzj98Ke+mPN07492Nr/BMlAZ9iwDLKaIUngq2dJfmpoxUVDDryKYQoRl5+76DiNk 4T5CHQNvCfAXKLxdlxlu =91oc -----END PGP SIGNATURE----- --dUDSsrYzWePLsuMJ6BLynjBl8cuScCtBf-- From owner-freebsd-fs@freebsd.org Mon Aug 3 21:12:11 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A617A3A7556 for ; Mon, 3 Aug 2020 21:12:11 +0000 (UTC) (envelope-from spork@bway.net) Received: from smtp1.bway.net (smtp1.bway.net [216.220.96.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BL9Z30nB6z4PYM; Mon, 3 Aug 2020 21:12:10 +0000 (UTC) (envelope-from spork@bway.net) Received: from gaseousweiner.sporklab.com (pool-173-70-93-30.nwrknj.fios.verizon.net [173.70.93.30]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) (Authenticated sender: spork@bway.net) by smtp1.bway.net (Postfix) with ESMTPSA id EBD5895897; Mon, 3 Aug 2020 17:12:09 -0400 (EDT) Content-Type: multipart/signed; boundary="Apple-Mail=_71218212-C9A8-468D-BD83-5DA65A9CB9FD"; protocol="application/pgp-signature"; micalg=pgp-sha256 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.15\)) Subject: Re: zfs scrub enable by default From: Charles Sprickman In-Reply-To: <24edb075-155c-439d-1ef5-541893036429@freebsd.org> Date: Mon, 3 Aug 2020 17:12:09 -0400 Cc: freebsd-fs@freebsd.org X-Mao-Original-Outgoing-Id: 618181929.277581-d530c6a94d4ae897b9d6e7e9501f221d Message-Id: <0E22A84A-DAAF-4651-865B-0AE038C7C3F4@bway.net> References: <24edb075-155c-439d-1ef5-541893036429@freebsd.org> To: Allan Jude X-Mailer: Apple Mail (2.3445.104.15) X-Rspamd-Queue-Id: 4BL9Z30nB6z4PYM X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; ASN(0.00)[asn:8059, ipnet:216.220.96.0/19, country:US]; REPLY(-4.00)[] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Aug 2020 21:12:11 -0000 --Apple-Mail=_71218212-C9A8-468D-BD83-5DA65A9CB9FD Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Aug 3, 2020, at 4:25 PM, Allan Jude wrote: >=20 > On 2020-08-03 12:10, Steve Wills wrote: >> Hi, >>=20 >> I wonder why we don't enable zfs periodic scrub by default? >>=20 >> = https://svnweb.freebsd.org/base/head/usr.sbin/periodic/periodic.conf?view=3D= markup#l162 >>=20 >>=20 >> Anyone happen to know? >>=20 >> Thanks, >> Steve >>=20 >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >=20 > I think switching this to on-by-default is a good thing. >=20 > To be clear, which the check is part of 'periodic daily', it only > triggers a scrub if it has been more than 35 days since the last = scrub. >=20 > FreeNAS already has does this, and that accounts for a large number of > FreeBSD ZFS deployments. >=20 > There is tuning you can do in ZFS to try to lessen the impact of a = scrub > on your production workloads. >=20 >=20 > The periodic script lets you select which pools to include (defaults = to > all), so you can tune it to only scrub your root pool every 35 days, = and > not the large pool that might take too long to scrub or whatever. It > also lets you set a different threshold for each pool. >=20 > So I don't see any reason not to enable it by default, and just = document > how to adjust it if people really need to disable it. Honestly, I = think > those who are disabling it are doing themselves a disservice. I 100% agree. I think often FreeBSD defaults tend to favor the = experienced user and throw the newbies under the bus. In this case there = were arguments against that amounted to =E2=80=9CWhat about people = running large production systems=E2=80=9D, to which my answer would be, = =E2=80=9CWhat about them? They are experienced, read all the mailing = lists, would know right away what was happening when they saw a scrub = running, and already know how to disable it and could make the case for = disabling or swapping in their own solution=E2=80=9D. Contrast with the random home user, noob, casual user who would likely = benefit from whatever data protection, pre-emptive failure notification, = etc. this would provide (for example I=E2=80=99ve had a scrub show me a = failing drive before SMART did). Charles >=20 > -- > Allan Jude >=20 --Apple-Mail=_71218212-C9A8-468D-BD83-5DA65A9CB9FD Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQEzBAEBCAAdFiEECbwhUg0jlYPK5QaKiZUhnP6GpPYFAl8ofakACgkQiZUhnP6G pPbhfwgAsVMetrmx5pgpt4V0oRElZffcMtUdvSYh5gWEW6vESu+rBdScR2tqEXwG pjlmhg9rZBnRdHKWe4NZQ3S7GHTSnr4jK60tkpoVzy4zdPNiFOkl1MKOsyZzQPfC 6fGT1bfEfcMPKA5QrwGU6hJ4PoB7XnLPeFwZ91Ilp+tnj9odfSo0tbAAjHCWe+rR mgN1Ey5MGOnhNw3ey+cn6YKqffGYsns32dwTaLnV77yRFkQS1Tj+DMnkpFfTQf43 CocG1ooPnr7+1LrtR26qzy8yWF1fyxxwDbP/MGXsfvIv/cHKJuD30KxXT1Q2qDND xOno5tnFtlNLBLIs9EakWFXv0uHTfQ== =yjDH -----END PGP SIGNATURE----- --Apple-Mail=_71218212-C9A8-468D-BD83-5DA65A9CB9FD-- From owner-freebsd-fs@freebsd.org Mon Aug 3 21:34:12 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DD76F3A8220 for ; Mon, 3 Aug 2020 21:34:12 +0000 (UTC) (envelope-from codeblue@inbox.lv) Received: from shark3.inbox.lv (shark3.inbox.lv [194.152.32.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLB3R40wwz4Qj1 for ; Mon, 3 Aug 2020 21:34:11 +0000 (UTC) (envelope-from codeblue@inbox.lv) Received: from shark3.inbox.lv (localhost [127.0.0.1]) by shark3-out.inbox.lv (Postfix) with ESMTP id 91961280353 for ; Tue, 4 Aug 2020 00:34:08 +0300 (EEST) Received: from localhost (localhost [127.0.0.1]) by shark3-in.inbox.lv (Postfix) with ESMTP id 8C6F9280342 for ; Tue, 4 Aug 2020 00:34:08 +0300 (EEST) Received: from shark3.inbox.lv ([127.0.0.1]) by localhost (shark3.inbox.lv [127.0.0.1]) (spamfilter, port 35) with ESMTP id CMvCmptBFe_d for ; Tue, 4 Aug 2020 00:34:08 +0300 (EEST) Received: from mail.inbox.lv (pop1 [127.0.0.1]) by shark3-in.inbox.lv (Postfix) with ESMTP id 0E75D280336 for ; Tue, 4 Aug 2020 00:34:08 +0300 (EEST) Received: from [10.0.0.2] (bzq-109-65-93-146.red.bezeqint.net [109.65.93.146]) (Authenticated sender: codeblue@inbox.lv) by mail.inbox.lv (Postfix) with ESMTPA id 1DD453D6A1E for ; Tue, 4 Aug 2020 00:34:05 +0300 (EEST) Subject: Re: zfs scrub enable by default To: freebsd-fs@freebsd.org References: <20200803121444.J57111@mulder.mintsol.com> <0BF85C14-5671-4779-928A-34C5FD3B82C1@lysator.liu.se> From: John Long Message-ID: <7ac1b087-b879-69bd-a29a-a64d2bf0c1a0@inbox.lv> Date: Mon, 3 Aug 2020 21:33:53 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Virus-Scanned: OK X-ESPOL: EZqEOwUB+w1LucO6K4Ru5uDswcrAKFdj4mfmt9096HJQqLXErdd2bm6RHZiNag+4bg== X-Rspamd-Queue-Id: 4BLB3R40wwz4Qj1 X-Spamd-Bar: ----- X-Spamd-Result: default: False [-5.28 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_FIVE(0.00)[6]; RECEIVED_SPAMHAUS_PBL(0.00)[109.65.93.146:received]; FROM_HAS_DN(0.00)[]; R_DKIM_ALLOW(-0.20)[inbox.lv:s=30062014]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:194.152.32.83]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[inbox.lv:dkim]; NEURAL_HAM_MEDIUM(-1.03)[-1.029]; DKIM_TRACE(0.00)[inbox.lv:+]; DMARC_POLICY_ALLOW(-0.50)[inbox.lv,quarantine]; NEURAL_HAM_SHORT(-1.16)[-1.163]; NEURAL_HAM_LONG(-0.99)[-0.991]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:12993, ipnet:194.152.32.0/23, country:LV]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_VERYGOOD(0.00)[194.152.32.83:from]; RCVD_IN_DNSWL_LOW(-0.10)[194.152.32.83:from] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Aug 2020 21:34:12 -0000 On 03/08/2020 18:11, Karl Denninger wrote: > On 8/3/2020 14:05, Peter Eriksson wrote: >>> In cases where that assumption is false - the resources consumed by >>> an hours-long scrub, slowing down a system which people expect to be >>> "as responsive as it usually it"... >>> >>> Our defaults should try to minimize the nasty surprises. >>> >>> -Walter >>> >>> >>> On Mon, 3 Aug 2020, Steve Wills wrote: >>> >>>> Hi, >>>> >>>> I wonder why we don't enable zfs periodic scrub by default? >>>> >>>> https://svnweb.freebsd.org/base/head/usr.sbin/periodic/periodic.conf?view=markup#l162 >>>> >>>> >>>> Anyone happen to know? >>>> >>>> Thanks, >>>> Steve > > I will pipe up AGAINST making this a default "do on its own" thing. Agreed. Please don't do this. That goes beyond the bounds of providing a useful system and crosses the red line of doing things we didn't ask for, don't reasonably expect, and could lead to problems that are impossible to anticipate. It is up to the user to configure his system the way he wants. The as-installed OS should be as unsurprising, benign, and vanilla as possible. Windows 10 boxes come preconfigured to defrag the hard drive automatically on some eerily disruptive schedule. I have to turn it off on every new box to avoid performance problems. I had to uninstall large sets of packages on Fedora Linux to stop the system indexing file contents in the name of uber-search capability which I don't want, a la Microsoft. Surely, FreeBSD can avoid these kinds of problems. Thank you. /jl From owner-freebsd-fs@freebsd.org Mon Aug 3 21:41:43 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D997B3A853B for ; Mon, 3 Aug 2020 21:41:43 +0000 (UTC) (envelope-from codeblue@inbox.lv) Received: from shark4.inbox.lv (shark4.inbox.lv [194.152.32.84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLBD70JnBz4QvK for ; Mon, 3 Aug 2020 21:41:42 +0000 (UTC) (envelope-from codeblue@inbox.lv) Received: from shark4.inbox.lv (localhost [127.0.0.1]) by shark4-out.inbox.lv (Postfix) with ESMTP id CC1DE61002 for ; Tue, 4 Aug 2020 00:41:40 +0300 (EEST) Received: from localhost (localhost [127.0.0.1]) by shark4-in.inbox.lv (Postfix) with ESMTP id B31A260FF6 for ; Tue, 4 Aug 2020 00:41:40 +0300 (EEST) Received: from shark4.inbox.lv ([127.0.0.1]) by localhost (shark4.inbox.lv [127.0.0.1]) (spamfilter, port 35) with ESMTP id 2PcaT119a0Gr for ; Tue, 4 Aug 2020 00:41:40 +0300 (EEST) Received: from mail.inbox.lv (pop1 [127.0.0.1]) by shark4-in.inbox.lv (Postfix) with ESMTP id BA58A60B83 for ; Tue, 4 Aug 2020 00:41:16 +0300 (EEST) Received: from [10.0.0.2] (bzq-109-65-93-146.red.bezeqint.net [109.65.93.146]) (Authenticated sender: codeblue@inbox.lv) by mail.inbox.lv (Postfix) with ESMTPA id 5D0523D6A1E for ; Tue, 4 Aug 2020 00:41:13 +0300 (EEST) Subject: Re: zfs scrub enable by default To: freebsd-fs@freebsd.org References: <24edb075-155c-439d-1ef5-541893036429@freebsd.org> From: John Long Message-ID: <0d1d14c9-64ba-368c-b2f4-56efa741cc5e@inbox.lv> Date: Mon, 3 Aug 2020 21:41:01 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <24edb075-155c-439d-1ef5-541893036429@freebsd.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Virus-Scanned: OK X-ESPOL: AJqEQ2Vk+XRHuMmgLYFy7uD6x9K1SilQsF6G0s5O5DIG9q+jx9N0c2uVG4buGgq+bg== X-Rspamd-Queue-Id: 4BLBD70JnBz4QvK X-Spamd-Bar: ----- X-Spamd-Result: default: False [-5.21 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[inbox.lv:s=30062014]; RCVD_COUNT_FIVE(0.00)[6]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:194.152.32.84:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[inbox.lv:dkim]; RECEIVED_SPAMHAUS_PBL(0.00)[109.65.93.146:received]; NEURAL_HAM_MEDIUM(-1.02)[-1.022]; DKIM_TRACE(0.00)[inbox.lv:+]; DMARC_POLICY_ALLOW(-0.50)[inbox.lv,quarantine]; NEURAL_HAM_SHORT(-1.10)[-1.101]; NEURAL_HAM_LONG(-0.99)[-0.988]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:12993, ipnet:194.152.32.0/23, country:LV]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_VERYGOOD(0.00)[194.152.32.84:from]; RCVD_IN_DNSWL_LOW(-0.10)[194.152.32.84:from] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Aug 2020 21:41:43 -0000 On 03/08/2020 20:25, Allan Jude wrote: > On 2020-08-03 12:10, Steve Wills wrote: >> Hi, >> >> I wonder why we don't enable zfs periodic scrub by default? >> >> https://svnweb.freebsd.org/base/head/usr.sbin/periodic/periodic.conf?view=markup#l162 >> >> >> Anyone happen to know? >> >> Thanks, >> Steve >> >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > I think switching this to on-by-default is a good thing. > > To be clear, which the check is part of 'periodic daily', it only > triggers a scrub if it has been more than 35 days since the last scrub. > > FreeNAS already has does this, and that accounts for a large number of > FreeBSD ZFS deployments. There is a difference. FreeNAS is an appliance. It should minimize the need for management. FreeBSD is the base OS. It should do as little as possible so people can set it up the way we want rather than spend days or weeks unbreaking surprising, non-standard behavior and hundreds of tweaks for everybody who requested them. > There is tuning you can do in ZFS to try to lessen the impact of a scrub > on your production workloads. That's up to the guy running the box to do or not to do. > The periodic script lets you select which pools to include (defaults to > all), so you can tune it to only scrub your root pool every 35 days, and > not the large pool that might take too long to scrub or whatever. It > also lets you set a different threshold for each pool. A NAS appliance could benefit from something like this. A generic OS cannot. > So I don't see any reason not to enable it by default, and just document > how to adjust it if people really need to disable it. Honestly, I think > those who are disabling it are doing themselves a disservice. It's offensive for you to presume to think you know better what the other guy needs than he does himself. Thank you for clarifying it though, I think it's valuable to understand the thinking of people who want to coopt the OS into their personal playground. /jl From owner-freebsd-fs@freebsd.org Mon Aug 3 22:20:12 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 16B173A88F6 for ; Mon, 3 Aug 2020 22:20:12 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from msa1.earth.yoonka.com (yoonka.com [88.98.225.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "msa1.earth.yoonka.com", Issuer "msa1.earth.yoonka.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLC4W0VNbz4SX8 for ; Mon, 3 Aug 2020 22:20:10 +0000 (UTC) (envelope-from list1@gjunka.com) Received: from venus.yoonka.com ([10.70.7.24]) (authenticated bits=0) by msa1.earth.yoonka.com (8.15.2/8.15.2) with ESMTPSA id 073MK3Nh035743 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO) for ; Mon, 3 Aug 2020 22:20:03 GMT (envelope-from list1@gjunka.com) X-Authentication-Warning: msa1.earth.yoonka.com: Host [10.70.7.24] claimed to be venus.yoonka.com Subject: Re: zfs scrub enable by default To: freebsd-fs@freebsd.org References: <24edb075-155c-439d-1ef5-541893036429@freebsd.org> <0d1d14c9-64ba-368c-b2f4-56efa741cc5e@inbox.lv> From: Grzegorz Junka Message-ID: Date: Mon, 3 Aug 2020 22:20:03 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.8.0 MIME-Version: 1.0 In-Reply-To: <0d1d14c9-64ba-368c-b2f4-56efa741cc5e@inbox.lv> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-US X-Rspamd-Queue-Id: 4BLC4W0VNbz4SX8 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of list1@gjunka.com designates 88.98.225.149 as permitted sender) smtp.mailfrom=list1@gjunka.com X-Spamd-Result: default: False [-2.02 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:88.98.225.149]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; HAS_XAW(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-0.90)[-0.903]; TO_DN_NONE(0.00)[]; NEURAL_SPAM_SHORT(0.02)[0.019]; DMARC_NA(0.00)[gjunka.com]; NEURAL_HAM_MEDIUM(-0.84)[-0.837]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:56478, ipnet:88.98.192.0/18, country:GB]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Aug 2020 22:20:12 -0000 On 03/08/2020 21:41, John Long via freebsd-fs wrote: > On 03/08/2020 20:25, Allan Jude wrote: >> On 2020-08-03 12:10, Steve Wills wrote: >>> Hi, >>> >>> I wonder why we don't enable zfs periodic scrub by default? >>> >>> https://svnweb.freebsd.org/base/head/usr.sbin/periodic/periodic.conf?view=markup#l162 >>> >>> >>> >>> Anyone happen to know? >>> >>> Thanks, >>> Steve >>> >>> _______________________________________________ >>> freebsd-fs@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> >> I think switching this to on-by-default is a good thing. >> >> To be clear, which the check is part of 'periodic daily', it only >> triggers a scrub if it has been more than 35 days since the last scrub. >> >> FreeNAS already has does this, and that accounts for a large number of >> FreeBSD ZFS deployments. > > There is a difference. FreeNAS is an appliance. It should minimize the > need for management. > > FreeBSD is the base OS. It should do as little as possible so people > can set it up the way we want rather than spend days or weeks > unbreaking surprising, non-standard behavior and hundreds of tweaks > for everybody who requested them. > >> There is tuning you can do in ZFS to try to lessen the impact of a scrub >> on your production workloads. > > That's up to the guy running the box to do or not to do. > >> The periodic script lets you select which pools to include (defaults to >> all), so you can tune it to only scrub your root pool every 35 days, and >> not the large pool that might take too long to scrub or whatever. It >> also lets you set a different threshold for each pool. > > A NAS appliance could benefit from something like this. A generic OS > cannot. > >> So I don't see any reason not to enable it by default, and just document >> how to adjust it if people really need to disable it. Honestly, I think >> those who are disabling it are doing themselves a disservice. > > It's offensive for you to presume to think you know better what the > other guy needs than he does himself. Thank you for clarifying it > though, I think it's valuable to understand the thinking of people who > want to coopt the OS into their personal playground. > Thanks John, I couldn't agree more. If I wanted to let my OS decide what's better for me I would run my server on Windows or Mac. That having said, FreeBSD does come with a few nice defaults that I wouldn't know how to set up myself otherwise. For example, once the sendmail is configured it automatically sends daily weekly and monthly health reports for the host and all jails, including disk usage, configuration changes, security checks, cleaning runs, failed su login attempts, etc. One might say they are similar to the scrubbing, i.e. if I don't configure sendmail the system will be collecting them in the host's mailbox eventually potentially impacting the server. I am not advocating for making scrub the default, just pointing out that FreeBSD does come with some potentially impacting defaults already. GrzegorzJ From owner-freebsd-fs@freebsd.org Mon Aug 3 23:34:32 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DCE953A9A21 for ; Mon, 3 Aug 2020 23:34:32 +0000 (UTC) (envelope-from josh@tcbug.org) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLDkH5Wffz4Wk2 for ; Mon, 3 Aug 2020 23:34:31 +0000 (UTC) (envelope-from josh@tcbug.org) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id EEC475C0148 for ; Mon, 3 Aug 2020 19:34:29 -0400 (EDT) Received: from imap2 ([10.202.2.52]) by compute2.internal (MEProxy); Mon, 03 Aug 2020 19:34:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=4YPj9K ii6qor+NDgjdSJMml0wHHwC1E6EggSDRblrm4=; b=OWBitnl+bIp/b7j9Jwuslk iyJlXl4XYWADg7rLO8kscS3ivfuRedOFj3uFxvXb/mHLFMAvimDNBG3YN1VhYhH5 XiCZLFeRGw2DkEWnDMa0X9s3CFQhrB3+pzrGneg0+OL/R6KCXPJNg7Y+3WzgJKw2 xqTR+NlOG6XXJd5WICBrTZDStj/bV09ODG/xMbPrF8SBiEI2JYJ7BHl7xvwIY+cE cw10Ltx3tJVNLrwinHnMJN2MORp0o1joGSJ3PbN+v+MELr3rEirHCw7XTZyQc77u Nu2VNppXDps5jqkzpl+gG8wadfYQSvLGSFTjlUERKIMdJib0RhDpePbdfOhwF5/w == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduiedrjeehgddvhecutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefofgggkfgjfhffhffvufgtsehttdertderredtnecuhfhrohhmpedflfhoshhh ucfrrggvthiivghlfdcuoehjohhshhesthgtsghughdrohhrgheqnecuggftrfgrthhtvg hrnhepffeigfevteegleeuhfdvkeelhfekieehtdffleekgeegkeekgffhjeefgfeiudfh necuffhomhgrihhnpehfrhgvvggsshgurdhorhhgnecuvehluhhsthgvrhfuihiivgeptd enucfrrghrrghmpehmrghilhhfrhhomhepjhhoshhhsehttggsuhhgrdhorhhg X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 982E7E00DD; Mon, 3 Aug 2020 19:34:29 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.3.0-128-gd51a832-fm-20200728.001-gd51a8328 Mime-Version: 1.0 Message-Id: In-Reply-To: <0d1d14c9-64ba-368c-b2f4-56efa741cc5e@inbox.lv> References: <24edb075-155c-439d-1ef5-541893036429@freebsd.org> <0d1d14c9-64ba-368c-b2f4-56efa741cc5e@inbox.lv> Date: Mon, 03 Aug 2020 18:33:49 -0500 From: "Josh Paetzel" To: "Paul Pathiakis via freebsd-fs" Subject: Re: zfs scrub enable by default Content-Type: text/plain X-Rspamd-Queue-Id: 4BLDkH5Wffz4Wk2 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=OWBitnl+; dmarc=none; spf=none (mx1.freebsd.org: domain of josh@tcbug.org has no SPF policy when checking 66.111.4.29) smtp.mailfrom=josh@tcbug.org X-Spamd-Result: default: False [-1.95 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.96)[-0.962]; R_DKIM_ALLOW(-0.20)[messagingengine.com:s=fm3]; XM_UA_NO_VERSION(0.01)[]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[66.111.4.29:from]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; DMARC_NA(0.00)[tcbug.org]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.00)[-0.996]; RCVD_COUNT_THREE(0.00)[4]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[messagingengine.com:+]; NEURAL_HAM_SHORT(-0.60)[-0.599]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:11403, ipnet:66.111.4.0/24, country:US]; MID_RHS_WWW(0.50)[]; RCVD_IN_DNSWL_LOW(-0.10)[66.111.4.29:from] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 03 Aug 2020 23:34:32 -0000 On Mon, Aug 3, 2020, at 4:41 PM, John Long via freebsd-fs wrote: > On 03/08/2020 20:25, Allan Jude wrote: > > On 2020-08-03 12:10, Steve Wills wrote: > >> Hi, > >> > >> I wonder why we don't enable zfs periodic scrub by default? > >> > >> https://svnweb.freebsd.org/base/head/usr.sbin/periodic/periodic.conf?view=markup#l162 > >> > >> > >> Anyone happen to know? > >> > >> Thanks, > >> Steve > >> > >> _______________________________________________ > >> freebsd-fs@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs > >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > > > I think switching this to on-by-default is a good thing. > > > > To be clear, which the check is part of 'periodic daily', it only > > triggers a scrub if it has been more than 35 days since the last scrub. > > > > FreeNAS already has does this, and that accounts for a large number of > > FreeBSD ZFS deployments. > > There is a difference. FreeNAS is an appliance. It should minimize the > need for management. > > FreeBSD is the base OS. It should do as little as possible so people can > set it up the way we want rather than spend days or weeks unbreaking > surprising, non-standard behavior and hundreds of tweaks for everybody > who requested them. > > > There is tuning you can do in ZFS to try to lessen the impact of a scrub > > on your production workloads. > > That's up to the guy running the box to do or not to do. > > > The periodic script lets you select which pools to include (defaults to > > all), so you can tune it to only scrub your root pool every 35 days, and > > not the large pool that might take too long to scrub or whatever. It > > also lets you set a different threshold for each pool. > > A NAS appliance could benefit from something like this. A generic OS cannot. > > > So I don't see any reason not to enable it by default, and just document > > how to adjust it if people really need to disable it. Honestly, I think > > those who are disabling it are doing themselves a disservice. > > It's offensive for you to presume to think you know better what the > other guy needs than he does himself. Thank you for clarifying it > though, I think it's valuable to understand the thinking of people who > want to coopt the OS into their personal playground. > > /jl > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > ZFS needs scrub in the same way that UFS needs fsck. (in that i you don't scrub a ZFS volume for long enough and you don't fsck a UFS volume for long enough they will stop being useful filesystems) If you've ever had the joy of running UFS on a volume so large you couldn't fsck it or run say XFS on a volume so large you couldn't run xfs_repair you'll know that you can get away with that for a while, but eventually the chickens come home to roost and you'll be restoring from tape or failing over to your DR site, or complaining about lost data. (Note that XFS is journaled and is far more resilient than UFS to this sort of thing but it's still an issue) (Thankfully UFS added background fsck years ago but there was a time when a FreeBSD system wouldn't go multiuser until fsck had completed and really large volumes (say greater than 2TB, (and remember this was 15+ years ago before you start talking about your laptop has a 2TB SSD) weren't feasible unless you disabled fsck or had 9+ hours to spare every time you rebooted. When FreeBSD grew ZFS support the periodic scrub script didn't exist. Thankfully we don't have a "Allow perfect to be the enemy of the good" stance and in spite of that shortcoming (and many others) ZFS went in. Over time the FreeBSD ZFS support was iterated on, a scrub periodic script was added...but it was left disabled by default: POLA violation and all that. The reality is having scrub off is (in the vast majority of cases) the wrong thing to do. I say that as someone who has nearly an exabyte in ZFS right now, and lead the team that brought ZFS based FreeNAS to the world. So the suggestion to "fix the glitch" and enable periodic scrubs makes perfect sense to me. Those who don't know any better will just magically receive the benefit of something they should've been doing all along. Those who do know better have a lot of time on their hands (at least judging by this thread!) and with way less typing than the length of these emails can disable the periodic script. Sounds win-win to me. As far as the personal playground comment....we're all just trying to do the right thing for the biggest group of people while knowing all along that perfection for everyone is an unachievable goal (however it is possible that while chasing perfection we may achieve greatness). As an interesting data point, no version of FreeBSD ever shipped with fsck disabled even if it was an unusable feature for some of the users.... (Just my ten cents, my two cents is free) -- Thanks, Josh Paetzel From owner-freebsd-fs@freebsd.org Tue Aug 4 00:08:51 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 18A003AA2B5 for ; Tue, 4 Aug 2020 00:08:51 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smarthost1.greenhost.nl (smarthost1.greenhost.nl [195.190.28.88]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLFTs4cbYz4Y8T for ; Tue, 4 Aug 2020 00:08:49 +0000 (UTC) (envelope-from ronald-lists@klop.ws) Received: from smtp.greenhost.nl ([213.108.110.112]) by smarthost1.greenhost.nl with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1k2kVZ-0002sZ-HN; Tue, 04 Aug 2020 02:08:45 +0200 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=klop.ws; s=mail; h=In-Reply-To:Message-ID:From:Content-Transfer-Encoding:MIME-Version: Date:References:Subject:To:Content-Type:Sender:Reply-To:Cc:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=ux6XhcAUCXlut6jFPDCvynYxYKJmc1ZUv0rsLxScuDQ=; b=JXVBMlqF4QMINVXIqzivnWsZXz Fj+VMZybR1k4nCT2/dWxIu7vxeYfVGdVrit0yNNz8jb8GxjJyG+mhhVOeRF48dO2fWRxLRvmALNik l7hIMedwn0Nbgd9ZqaHaX/D3s6aC3QXNfMVN04YAmPS2SgpOUO+zG7hXno+60ppnWwMU=; Content-Type: text/plain; charset=utf-8; format=flowed; delsp=yes To: "Paul Pathiakis via freebsd-fs" , "Josh Paetzel" Subject: Re: zfs scrub enable by default References: <24edb075-155c-439d-1ef5-541893036429@freebsd.org> <0d1d14c9-64ba-368c-b2f4-56efa741cc5e@inbox.lv> Date: Tue, 04 Aug 2020 02:08:43 +0200 MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: "Ronald Klop" Message-ID: In-Reply-To: User-Agent: Opera Mail/12.16 (FreeBSD) X-Authenticated-As-Hash: 398f5522cb258ce43cb679602f8cfe8b62a256d1 X-Virus-Scanned: by clamav at smarthost1.samage.net X-Spam-Level: / X-Spam-Score: -0.4 X-Spam-Status: No, score=-0.4 required=5.0 tests=ALL_TRUSTED, BAYES_50, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, DKIM_VALID_EF autolearn=disabled version=3.4.2 X-Scan-Signature: 8c99c374a8d12108ecfc332d37a9628a X-Rspamd-Queue-Id: 4BLFTs4cbYz4Y8T X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=klop.ws header.s=mail header.b=JXVBMlqF; dmarc=none; spf=pass (mx1.freebsd.org: domain of ronald-lists@klop.ws designates 195.190.28.88 as permitted sender) smtp.mailfrom=ronald-lists@klop.ws X-Spamd-Result: default: False [-2.98 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.02)[-1.019]; R_DKIM_ALLOW(-0.20)[klop.ws:s=mail]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[195.190.28.88:from]; R_SPF_ALLOW(-0.20)[+ip4:195.190.28.64/27]; NEURAL_HAM_LONG(-1.04)[-1.043]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[klop.ws]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[klop.ws:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[195.190.28.88:from]; NEURAL_HAM_SHORT(-0.92)[-0.916]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; MID_RHS_NOT_FQDN(0.50)[]; ASN(0.00)[asn:47172, ipnet:195.190.28.0/24, country:NL]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2020 00:08:51 -0000 On Tue, 04 Aug 2020 01:33:49 +0200, Josh Paetzel wrote: > > > On Mon, Aug 3, 2020, at 4:41 PM, John Long via freebsd-fs wrote: >> On 03/08/2020 20:25, Allan Jude wrote: >> > On 2020-08-03 12:10, Steve Wills wrote: >> >> Hi, >> >> >> >> I wonder why we don't enable zfs periodic scrub by default? >> >> >> >> >> https://svnweb.freebsd.org/base/head/usr.sbin/periodic/periodic.conf?view=markup#l162 >> >> >> >> >> >> Anyone happen to know? >> >> >> >> Thanks, >> >> Steve >> >> >> >> _______________________________________________ >> >> freebsd-fs@freebsd.org mailing list >> >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> > >> > I think switching this to on-by-default is a good thing. >> > >> > To be clear, which the check is part of 'periodic daily', it only >> > triggers a scrub if it has been more than 35 days since the last >> scrub. >> > >> > FreeNAS already has does this, and that accounts for a large number of >> > FreeBSD ZFS deployments. >> >> There is a difference. FreeNAS is an appliance. It should minimize the >> need for management. >> >> FreeBSD is the base OS. It should do as little as possible so people can >> set it up the way we want rather than spend days or weeks unbreaking >> surprising, non-standard behavior and hundreds of tweaks for everybody >> who requested them. >> >> > There is tuning you can do in ZFS to try to lessen the impact of a >> scrub >> > on your production workloads. >> >> That's up to the guy running the box to do or not to do. >> >> > The periodic script lets you select which pools to include (defaults >> to >> > all), so you can tune it to only scrub your root pool every 35 days, >> and >> > not the large pool that might take too long to scrub or whatever. It >> > also lets you set a different threshold for each pool. >> >> A NAS appliance could benefit from something like this. A generic OS >> cannot. >> >> > So I don't see any reason not to enable it by default, and just >> document >> > how to adjust it if people really need to disable it. Honestly, I >> think >> > those who are disabling it are doing themselves a disservice. >> >> It's offensive for you to presume to think you know better what the >> other guy needs than he does himself. Thank you for clarifying it >> though, I think it's valuable to understand the thinking of people who >> want to coopt the OS into their personal playground. >> >> /jl >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> > > ZFS needs scrub in the same way that UFS needs fsck. (in that i you > don't scrub a ZFS volume for long enough and you don't fsck a UFS volume > for long enough they will stop being useful filesystems) If you've ever > had the joy of running UFS on a volume so large you couldn't fsck it or > run say XFS on a volume so large you couldn't run xfs_repair you'll know > that you can get away with that for a while, but eventually the chickens > come home to roost and you'll be restoring from tape or failing over to > your DR site, or complaining about lost data. (Note that XFS is > journaled and is far more resilient than UFS to this sort of thing but > it's still an issue) > > (Thankfully UFS added background fsck years ago but there was a time > when a FreeBSD system wouldn't go multiuser until fsck had completed and > really large volumes (say greater than 2TB, (and remember this was 15+ > years ago before you start talking about your laptop has a 2TB SSD) > weren't feasible unless you disabled fsck or had 9+ hours to spare every > time you rebooted. > > When FreeBSD grew ZFS support the periodic scrub script didn't exist. > Thankfully we don't have a "Allow perfect to be the enemy of the good" > stance and in spite of that shortcoming (and many others) ZFS went in. > > Over time the FreeBSD ZFS support was iterated on, a scrub periodic > script was added...but it was left disabled by default: POLA violation > and all that. > > The reality is having scrub off is (in the vast majority of cases) the > wrong thing to do. I say that as someone who has nearly an exabyte in > ZFS right now, and lead the team that brought ZFS based FreeNAS to the > world. So the suggestion to "fix the glitch" and enable periodic scrubs > makes perfect sense to me. Those who don't know any better will just > magically receive the benefit of something they should've been doing all > along. Those who do know better have a lot of time on their hands (at > least judging by this thread!) and with way less typing than the length > of these emails can disable the periodic script. Sounds win-win to me. > > As far as the personal playground comment....we're all just trying to do > the right thing for the biggest group of people while knowing all along > that perfection for everyone is an unachievable goal (however it is > possible that while chasing perfection we may achieve greatness). As an > interesting data point, no version of FreeBSD ever shipped with fsck > disabled even if it was an unusable feature for some of the users.... > > (Just my ten cents, my two cents is free) > +1 to enable by default (All my ZFS systems scrub, except my FreeBSD I found out in this mailinglist today.) From owner-freebsd-fs@freebsd.org Tue Aug 4 00:14:14 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 710073AA7CF for ; Tue, 4 Aug 2020 00:14:14 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLFc62QTQz4YYX for ; Tue, 4 Aug 2020 00:14:14 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: from mail-lf1-f53.google.com (mail-lf1-f53.google.com [209.85.167.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) (Authenticated sender: mmacy) by smtp.freebsd.org (Postfix) with ESMTPSA id 31D882E7E0 for ; Tue, 4 Aug 2020 00:14:14 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: by mail-lf1-f53.google.com with SMTP id c15so4097535lfi.3 for ; Mon, 03 Aug 2020 17:14:14 -0700 (PDT) X-Gm-Message-State: AOAM53280dPXfEZduRc3DCj+so6BqS3SgkpS13L/jYMRfgpHQ1ukrFFn iLdR1XXvda6R3rttRyYBz2d+lg8h9bPZaa8tW8o= X-Google-Smtp-Source: ABdhPJz8r917PNnuyezthGmXMF/2JJroBpdgdsMPlBAneOUyJfHyabB23pJ5QVw5N2xMWRmrUUoGyav4Xzxm/EQgBEE= X-Received: by 2002:ac2:4c05:: with SMTP id t5mr9769289lfq.89.1596500052754; Mon, 03 Aug 2020 17:14:12 -0700 (PDT) MIME-Version: 1.0 References: <20200727154651.37a7c23e711da9218cee61dd@yamagi.org> <20200730152825.c246a854a1763883b2f165d4@yamagi.org> In-Reply-To: From: Matthew Macy Date: Mon, 3 Aug 2020 17:14:01 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: CFT for vendor openzfs - week 3 reminder To: Yamagi Cc: freebsd-fs Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2020 00:14:14 -0000 > I'll rebase the OpenZFS projects/openzfs_vendor when the fix is merged: > https://github.com/openzfs/zfs/pull/10658 Should be fixed now. From owner-freebsd-fs@freebsd.org Tue Aug 4 00:24:58 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6BDA23AAB0C; Tue, 4 Aug 2020 00:24:58 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLFrV202Hz4Z5g; Tue, 4 Aug 2020 00:24:58 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: from mail-lj1-f171.google.com (mail-lj1-f171.google.com [209.85.208.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) (Authenticated sender: mmacy) by smtp.freebsd.org (Postfix) with ESMTPSA id 22EF12D9BB; Tue, 4 Aug 2020 00:24:58 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: by mail-lj1-f171.google.com with SMTP id v9so7142414ljk.6; Mon, 03 Aug 2020 17:24:58 -0700 (PDT) X-Gm-Message-State: AOAM532SYEzVxv4cpg+N8Eicu2Lwtfcmj/kAwiOSlTQZ/Rb/1zoZrsA3 JgVVYIXVmt7U4cYHxz7sG8jQz/sVa2PC4qOVs6s= X-Google-Smtp-Source: ABdhPJwFOeuJmIr/AK29TNzeiUK22+qGXeOoe9ZHThgpLIG3qtx4OKeXBKSkYnD2B2JR0co9BXFKm3WeiML0//zORww= X-Received: by 2002:a05:651c:307:: with SMTP id a7mr8245161ljp.297.1596500696675; Mon, 03 Aug 2020 17:24:56 -0700 (PDT) MIME-Version: 1.0 From: Matthew Macy Date: Mon, 3 Aug 2020 17:24:45 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: CFT for vendor openzfs - week 5 reminder + memdisk images To: freebsd-current , freebsd-fs , freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2020 00:24:58 -0000 On Wednesday, July 8th I issued the initial call for testing for the update to HEAD to vendored openzfs. We'd like to give users roughly a month to test before merging. The tentative merge date is August 17th. Again, I hope it's not terribly controversial to point out that it really rests with users of non amd64 platforms to test to avoid any unpleasant surprises the next time they update their trees following the merge. amd64, i386, and aarch64 memdisk images can be found at: https://people.freebsd.org/~freqlabs/freebsd-openzfs/latest/ If you're using a platform not listed above and would be inclined to test if you had an image to work with, let us know. Alternatively, you can still build following the instructions below. The review for merging in to base can be found at: https://reviews.freebsd.org/D25872 ========================================================== NB: Do NOT zpool upgrade unless you are willing to live without the ability to ever rollback to the legacy zfs kmod. Checkout updated HEAD: % git clone https://github.com/mattmacy/networking.git -b projects/openzfs_vendor freebsd Checkout updated openzfs in to sys/contrib: % git clone https://github.com/zfsonfreebsd/ZoF.git -b projects/openzfs_vendor freebsd/sys/contrib/openzfs Build world and kernel with whatever your usual configuration is. Where possible the openzfs kmod is backward compatible with the cmd utils in HEAD so common operations work with existing tools and the new kmod. In the projects/openzfs_vendor branch of ZoF ozfs libraries are backward compatible with the zfs kmod in HEAD. Although ideally one would test this in a separate boot environment, the interoperability should allow one to rollback without too much difficulty. NB: The patch updates /etc/rc.d/zfs - so if you skip mergemaster pools other than the root pool will not be imported at boot. Thanks in advance for your time. -M From owner-freebsd-fs@freebsd.org Tue Aug 4 00:48:19 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E96F93AB96A for ; Tue, 4 Aug 2020 00:48:19 +0000 (UTC) (envelope-from grant@gray.id.au) Received: from mail.agc.net.au (mail.agc.net.au [167.86.119.123]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLGMQ48mFz4bG0 for ; Tue, 4 Aug 2020 00:48:17 +0000 (UTC) (envelope-from grant@gray.id.au) Received: from localhost (localhost [127.0.0.1]) by mail.agc.net.au (Postfix) with ESMTP id 6E1F028B4E1 for ; Tue, 4 Aug 2020 10:48:08 +1000 (AEST) Received: from mail.agc.net.au ([127.0.0.1]) by localhost (mail.agc.net.au [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id T_e5P6SsLoD2 for ; Tue, 4 Aug 2020 10:48:02 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by mail.agc.net.au (Postfix) with ESMTP id E123328B2E0 for ; Tue, 4 Aug 2020 10:48:01 +1000 (AEST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.agc.net.au E123328B2E0 X-Virus-Scanned: amavisd-new at mail.agc.net.au Received: from mail.agc.net.au ([127.0.0.1]) by localhost (mail.agc.net.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id J2uH865ff6WY for ; Tue, 4 Aug 2020 10:48:01 +1000 (AEST) Received: from mail.agc.net.au (mail.agc.net.au [167.86.119.123]) by mail.agc.net.au (Postfix) with ESMTP id 8F3F228AFEE for ; Tue, 4 Aug 2020 10:48:01 +1000 (AEST) Date: Tue, 4 Aug 2020 10:47:50 +1000 (AEST) From: Grant Gray To: freebsd-fs Message-ID: <2038290194.249397.1596502070473.JavaMail.zimbra@gray.id.au> In-Reply-To: References: <24edb075-155c-439d-1ef5-541893036429@freebsd.org> <0d1d14c9-64ba-368c-b2f4-56efa741cc5e@inbox.lv> Subject: Re: zfs scrub enable by default MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [167.86.119.123] X-Mailer: Zimbra 8.8.15_GA_3945 (ZimbraWebClient - FF80 (Linux)/8.8.15_GA_3953) Thread-Topic: zfs scrub enable by default Thread-Index: G9gVllcTPr6nLKdyqgdP3Vb3Nq+vIA== X-Rspamd-Queue-Id: 4BLGMQ48mFz4bG0 X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.36 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.03)[-1.034]; RCVD_COUNT_FIVE(0.00)[6]; HAS_XOIP(0.00)[]; FROM_HAS_DN(0.00)[]; R_DKIM_ALLOW(-0.20)[gray.id.au:s=30D3F2DC-EB1E-11E9-943D-4BC9C5489560]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; NEURAL_HAM_LONG(-1.01)[-1.010]; RCPT_COUNT_ONE(0.00)[1]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gray.id.au:+]; DMARC_POLICY_ALLOW(-0.50)[gray.id.au,quarantine]; NEURAL_HAM_SHORT(-0.31)[-0.313]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:51167, ipnet:167.86.118.0/23, country:DE]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2020 00:48:20 -0000 ----- On 4 Aug, 2020, at 9:33 AM, Josh Paetzel josh@tcbug.org wrote: > On Mon, Aug 3, 2020, at 4:41 PM, John Long via freebsd-fs wrote: >> On 03/08/2020 20:25, Allan Jude wrote: >> > On 2020-08-03 12:10, Steve Wills wrote: >> >> Hi, >> >> >> >> I wonder why we don't enable zfs periodic scrub by default? >> >> >> >> https://svnweb.freebsd.org/base/head/usr.sbin/periodic/periodic.conf?view=markup#l162 >> >> >> >> >> >> Anyone happen to know? >> >> >> >> Thanks, >> >> Steve >> >> >> >> _______________________________________________ >> >> freebsd-fs@freebsd.org mailing list >> >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> > >> > I think switching this to on-by-default is a good thing. >> > >> > To be clear, which the check is part of 'periodic daily', it only >> > triggers a scrub if it has been more than 35 days since the last scrub. >> > >> > FreeNAS already has does this, and that accounts for a large number of >> > FreeBSD ZFS deployments. >> >> There is a difference. FreeNAS is an appliance. It should minimize the >> need for management. >> >> FreeBSD is the base OS. It should do as little as possible so people can >> set it up the way we want rather than spend days or weeks unbreaking >> surprising, non-standard behavior and hundreds of tweaks for everybody >> who requested them. >> >> > There is tuning you can do in ZFS to try to lessen the impact of a scrub >> > on your production workloads. >> >> That's up to the guy running the box to do or not to do. >> >> > The periodic script lets you select which pools to include (defaults to >> > all), so you can tune it to only scrub your root pool every 35 days, and >> > not the large pool that might take too long to scrub or whatever. It >> > also lets you set a different threshold for each pool. >> >> A NAS appliance could benefit from something like this. A generic OS cannot. >> >> > So I don't see any reason not to enable it by default, and just document >> > how to adjust it if people really need to disable it. Honestly, I think >> > those who are disabling it are doing themselves a disservice. >> >> It's offensive for you to presume to think you know better what the >> other guy needs than he does himself. Thank you for clarifying it >> though, I think it's valuable to understand the thinking of people who >> want to coopt the OS into their personal playground. >> >> /jl >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> > > ZFS needs scrub in the same way that UFS needs fsck. (in that i you don't scrub > a ZFS volume for long enough and you don't fsck a UFS volume for long enough > they will stop being useful filesystems) If you've ever had the joy of running > UFS on a volume so large you couldn't fsck it or run say XFS on a volume so > large you couldn't run xfs_repair you'll know that you can get away with that > for a while, but eventually the chickens come home to roost and you'll be > restoring from tape or failing over to your DR site, or complaining about lost > data. (Note that XFS is journaled and is far more resilient than UFS to this > sort of thing but it's still an issue) > These are some fun (and real) historical anecdotes but, other than bit-rot, they do not represent current reality as these problems have been largely solved or the momentum has shifted to better filesystems (such as ZFS and XFS). > The reality is having scrub off is (in the vast majority of cases) the wrong > thing to do. I say that as someone who has nearly an exabyte in ZFS right now, > and lead the team that brought ZFS based FreeNAS to the world. So the > suggestion to "fix the glitch" and enable periodic scrubs makes perfect sense > to me. Those who don't know any better will just magically receive the benefit > of something they should've been doing all along. Those who do know better > have a lot of time on their hands (at least judging by this thread!) and with > way less typing than the length of these emails can disable the periodic > script. Sounds win-win to me. > > -- > > Thanks, > > Josh Paetzel I think you are projecting a false characterisation of the average FreeBSD user and workload. FreeBSD is not an appliance and is not, in my experience, deployed or used by inexperienced users for any serious use-case. Who exactly is the user/use-case you are advocating for? There seems to be a conflation of purpose of scrubbing and general disk health; the point of a scrub is to catch bit errors in files that are infrequently accessed, that would otherwise not be detected in real-time. The value of that is largely dictated by whether you have RAID, and your backup strategy. A scrub does not tell you anything about the state of blocks that are not in use (or the overall health of a disk; ref smartd). Other people in this thread have already alluded to the fact that scrubbing can be detrimental, or even disastrous, for many of the workloads FreeBSD is used for. Excess scrubbing is a waste of energy and potentially reduces the working life of the disks. What I (and others in this thread) are getting at is setting a scrub schedule is so context specific that getting it right for the general case is near impossible, because there is no general case. Not scrubbing by default does not lead to unexpected behaviour. Scrubbing by default can and does lead to unexpected behaviour. People who care about their data use RAID and will determine a scrub schedule that suits their context. I take your point that, for a specific type of user, scrubbing by default makes sense. I'm just not sure that type of user is the benchmark for deciding FreeBSD system policies. If a user won't bear the cognitive load of understanding how to protect their own data, I would suggest FreeBSD is not for them and there are better options that will protect them better, by default (such as Ubuntu). From owner-freebsd-fs@freebsd.org Tue Aug 4 09:28:37 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4DDEB3B6067 for ; Tue, 4 Aug 2020 09:28:37 +0000 (UTC) (envelope-from demis@yandex.ru) Received: from forward105o.mail.yandex.net (forward105o.mail.yandex.net [IPv6:2a02:6b8:0:1a2d::608]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLTvl6VJjz46bt for ; Tue, 4 Aug 2020 09:28:35 +0000 (UTC) (envelope-from demis@yandex.ru) Received: from forward102q.mail.yandex.net (forward102q.mail.yandex.net [IPv6:2a02:6b8:c0e:1ba:0:640:516:4e7d]) by forward105o.mail.yandex.net (Yandex) with ESMTP id E03F94201E30; Tue, 4 Aug 2020 12:28:30 +0300 (MSK) Received: from mxback4q.mail.yandex.net (mxback4q.mail.yandex.net [IPv6:2a02:6b8:c0e:6d:0:640:ed15:d2bd]) by forward102q.mail.yandex.net (Yandex) with ESMTP id DC7AE7F20002; Tue, 4 Aug 2020 12:28:30 +0300 (MSK) Received: from localhost (localhost [::1]) by mxback4q.mail.yandex.net (mxback/Yandex) with ESMTP id akr1Wl83jI-SToSA6Vi; Tue, 04 Aug 2020 12:28:30 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yandex.ru; s=mail; t=1596533310; bh=f9k6wcoTFRnvyTktpVK5rYr8cWMf6AAVa8Kr6H79wcQ=; h=References:Date:Message-Id:Subject:In-Reply-To:To:From; b=vK8wvGsq8bz7Obc3M1GvdUEX5nDBUkB+1chVLj7DcMzNXP+pz7BD6AI9AMpXacI1T GNcDtK7EGLUeC8yvja4KNFF7lJ6S6Pi2l+uW/a8hR6I2dTN69xXdOwPMMPH6M2qup8 QfLRVgvzmY5sBNoSe+hXduDqF7lHOxapbasE7MWY= Received: by vla5-e043431e7e8d.qloud-c.yandex.net with HTTP; Tue, 04 Aug 2020 12:28:29 +0300 From: DemIS To: Grant Gray , freebsd-fs In-Reply-To: <2038290194.249397.1596502070473.JavaMail.zimbra@gray.id.au> References: <24edb075-155c-439d-1ef5-541893036429@freebsd.org> <0d1d14c9-64ba-368c-b2f4-56efa741cc5e@inbox.lv> <2038290194.249397.1596502070473.JavaMail.zimbra@gray.id.au> Subject: Re: zfs scrub enable by default X-Mailer: Yamail [ http://yandex.ru ] 5.0 Date: Tue, 04 Aug 2020 12:28:29 +0300 Message-Id: <937221596532639@mail.yandex.ru> X-Rspamd-Queue-Id: 4BLTvl6VJjz46bt X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yandex.ru header.s=mail header.b=vK8wvGsq; dmarc=pass (policy=none) header.from=yandex.ru; spf=pass (mx1.freebsd.org: domain of demis@yandex.ru designates 2a02:6b8:0:1a2d::608 as permitted sender) smtp.mailfrom=demis@yandex.ru X-Spamd-Result: default: False [-3.51 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.04)[-1.041]; R_DKIM_ALLOW(-0.20)[yandex.ru:s=mail]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[yandex.ru]; R_SPF_ALLOW(-0.20)[+ip6:2a02:6b8:0:1000::/52]; NEURAL_HAM_LONG(-1.00)[-1.001]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yandex.ru:+]; MIME_BASE64_TEXT(0.10)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2a02:6b8:0:1a2d::608:from]; MIME_HTML_ONLY(0.20)[]; NEURAL_HAM_SHORT(-0.87)[-0.865]; DMARC_POLICY_ALLOW(-0.50)[yandex.ru,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:~]; FREEMAIL_ENVFROM(0.00)[yandex.ru]; ASN(0.00)[asn:13238, ipnet:2a02:6b8::/32, country:RU]; RCVD_TLS_LAST(0.00)[]; DWL_DNSWL_NONE(0.00)[yandex.ru:dkim] MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2020 09:28:37 -0000 From owner-freebsd-fs@freebsd.org Tue Aug 4 10:53:46 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 305EA3B7C12 for ; Tue, 4 Aug 2020 10:53:46 +0000 (UTC) (envelope-from lists@yamagi.org) Received: from mail1.yamagi.org (mail1.yamagi.org [IPv6:2001:19f0:b001:853::3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLWp15wY0z4CM5; Tue, 4 Aug 2020 10:53:45 +0000 (UTC) (envelope-from lists@yamagi.org) Received: from [212.48.125.108] (helo=aka.server.knhmn.de) by mail1.yamagi.org with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.93.0.4 (FreeBSD)) (envelope-from ) id 1k2uZZ-000Gi1-Fg; Tue, 04 Aug 2020 12:53:38 +0200 Date: Tue, 4 Aug 2020 12:53:22 +0200 From: Yamagi To: mmacy@freebsd.org Cc: freebsd-fs@freebsd.org Subject: Re: CFT for vendor openzfs - week 3 reminder Message-Id: <20200804125322.37024484356ed4833fde2e76@yamagi.org> In-Reply-To: References: <20200727154651.37a7c23e711da9218cee61dd@yamagi.org> <20200730152825.c246a854a1763883b2f165d4@yamagi.org> X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.32; amd64-portbld-freebsd12.1) Mime-Version: 1.0 Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg="PGP-SHA256"; boundary="Signature=_Tue__4_Aug_2020_12_53_22_+0200_yjO3hTZDMm+yd_ql" X-Rspamd-Queue-Id: 4BLWp15wY0z4CM5 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:20473, ipnet:2001:19f0:b000::/38, country:US] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2020 10:53:46 -0000 --Signature=_Tue__4_Aug_2020_12_53_22_+0200_yjO3hTZDMm+yd_ql Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, 3 Aug 2020 17:14:01 -0700 Matthew Macy wrote: > > I'll rebase the OpenZFS projects/openzfs_vendor when the fix is merged: > > https://github.com/openzfs/zfs/pull/10658 >=20 > Should be fixed now. I can confirm that it's working now. Thank you for fixing this! --=20 Homepage: https://www.yamagi.org Github: https://github.com/yamagi GPG: 0x1D502515 --Signature=_Tue__4_Aug_2020_12_53_22_+0200_yjO3hTZDMm+yd_ql Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEOXu/lxyufwz0gC5x6xRy5x1QJRUFAl8pPiIACgkQ6xRy5x1Q JRVqDRAApj2hoNRvglQaDmXJ0An87+BTy9rEdgndG5+C9o4rC9WgsB/bXDdhrWYC TZ/NhxEAiI9jKSUqaI6h9okTE5Op8j76hNqqOzIBCkzxJozvNzaCf2qfYczgj3Bb XKov0WqXJfgGoHHfiLGS2qfqSo5RFLeJgsYir4cHqV61go2s6vu/eQ0xM538g4XJ 6PgIqSZu0y8MI5qzwcvPCno+djCV/410WrTXtOzMJxKP6+vpHPPMcp1uPMOOyyfU P/zZFpu+CEDZVon+UWA3Nsfs1ORjTwj/0M5mFqgAcDjdureeMSX1KWhwV9Or5yum nGv//0HfVabbfDx3ZAFLGLNH5eTdHRO61GuS2/Jcxe0EZjDFkeQfLXL4COHGU4rN kCf7jxYWY0Ee7mvXbd0+pELus0XUQjfUeVIqbFEm05CEHvs/eBh02jO+i3XbutZ+ w/pyVIjt4sjuBQ1sIyrR8J3eWoI8uV2Gq6oBDAaZqDXF+wSyxOShxtoR9ELbBpag E76MylzwuDOYIpqyGdUfEY18CKfD5IY2ZjJe4Ww0siXQqN32f0vrwSMxicqc3DJO WQBPvJTOlJSL/72WmsaDNc4YFnGPYOYDE8calNdiIY8eESJaUw5CraOrU4+DMVgn 4WagEYdluDRBk9KIaB1wDilpVsq2ZCFAW+NSLBjFqlX1TkztRI8= =lNBU -----END PGP SIGNATURE----- --Signature=_Tue__4_Aug_2020_12_53_22_+0200_yjO3hTZDMm+yd_ql-- From owner-freebsd-fs@freebsd.org Tue Aug 4 12:25:26 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 370DD372D41 for ; Tue, 4 Aug 2020 12:25:26 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from mail.in-addr.com (mail.in-addr.com [IPv6:2a01:4f8:191:61e8::2525:2525]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLYqn64Hdz4Hqt for ; Tue, 4 Aug 2020 12:25:25 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from gjp by mail.in-addr.com with local (Exim 4.93.0.4 (FreeBSD)) (envelope-from ) id 1k2w0L-000MCE-JG for freebsd-fs@freebsd.org; Tue, 04 Aug 2020 13:25:17 +0100 Date: Tue, 4 Aug 2020 13:25:17 +0100 From: Gary Palmer To: FreeBSD FS Subject: Re: zfs scrub enable by default Message-ID: <20200804122517.GA39788@in-addr.com> References: <24edb075-155c-439d-1ef5-541893036429@freebsd.org> <0d1d14c9-64ba-368c-b2f4-56efa741cc5e@inbox.lv> <2038290194.249397.1596502070473.JavaMail.zimbra@gray.id.au> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2038290194.249397.1596502070473.JavaMail.zimbra@gray.id.au> X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on mail.in-addr.com); SAEximRunCond expanded to false X-Rspamd-Queue-Id: 4BLYqn64Hdz4Hqt X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; ASN(0.00)[asn:24940, ipnet:2a01:4f8::/29, country:DE] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2020 12:25:26 -0000 On Tue, Aug 04, 2020 at 10:47:50AM +1000, Grant Gray via freebsd-fs wrote: > > > ----- On 4 Aug, 2020, at 9:33 AM, Josh Paetzel josh@tcbug.org wrote: > > > On Mon, Aug 3, 2020, at 4:41 PM, John Long via freebsd-fs wrote: > >> On 03/08/2020 20:25, Allan Jude wrote: > >> > On 2020-08-03 12:10, Steve Wills wrote: > >> >> Hi, > >> >> > >> >> I wonder why we don't enable zfs periodic scrub by default? > >> >> > >> >> https://svnweb.freebsd.org/base/head/usr.sbin/periodic/periodic.conf?view=markup#l162 > >> >> > >> >> > >> >> Anyone happen to know? > >> >> > >> >> Thanks, > >> >> Steve > >> >> > >> >> _______________________________________________ > >> >> freebsd-fs@freebsd.org mailing list > >> >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs > >> >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > >> > > >> > I think switching this to on-by-default is a good thing. > >> > > >> > To be clear, which the check is part of 'periodic daily', it only > >> > triggers a scrub if it has been more than 35 days since the last scrub. > >> > > >> > FreeNAS already has does this, and that accounts for a large number of > >> > FreeBSD ZFS deployments. > >> > >> There is a difference. FreeNAS is an appliance. It should minimize the > >> need for management. > >> > >> FreeBSD is the base OS. It should do as little as possible so people can > >> set it up the way we want rather than spend days or weeks unbreaking > >> surprising, non-standard behavior and hundreds of tweaks for everybody > >> who requested them. > >> > >> > There is tuning you can do in ZFS to try to lessen the impact of a scrub > >> > on your production workloads. > >> > >> That's up to the guy running the box to do or not to do. > >> > >> > The periodic script lets you select which pools to include (defaults to > >> > all), so you can tune it to only scrub your root pool every 35 days, and > >> > not the large pool that might take too long to scrub or whatever. It > >> > also lets you set a different threshold for each pool. > >> > >> A NAS appliance could benefit from something like this. A generic OS cannot. > >> > >> > So I don't see any reason not to enable it by default, and just document > >> > how to adjust it if people really need to disable it. Honestly, I think > >> > those who are disabling it are doing themselves a disservice. > >> > >> It's offensive for you to presume to think you know better what the > >> other guy needs than he does himself. Thank you for clarifying it > >> though, I think it's valuable to understand the thinking of people who > >> want to coopt the OS into their personal playground. > >> > >> /jl > >> _______________________________________________ > >> freebsd-fs@freebsd.org mailing list > >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs > >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > >> > > > > ZFS needs scrub in the same way that UFS needs fsck. (in that i you don't scrub > > a ZFS volume for long enough and you don't fsck a UFS volume for long enough > > they will stop being useful filesystems) If you've ever had the joy of running > > UFS on a volume so large you couldn't fsck it or run say XFS on a volume so > > large you couldn't run xfs_repair you'll know that you can get away with that > > for a while, but eventually the chickens come home to roost and you'll be > > restoring from tape or failing over to your DR site, or complaining about lost > > data. (Note that XFS is journaled and is far more resilient than UFS to this > > sort of thing but it's still an issue) > > > > These are some fun (and real) historical anecdotes but, other than bit-rot, they do not represent current reality as these problems have been largely solved or the momentum has shifted to better filesystems (such as ZFS and XFS). > > > The reality is having scrub off is (in the vast majority of cases) the wrong > > thing to do. I say that as someone who has nearly an exabyte in ZFS right now, > > and lead the team that brought ZFS based FreeNAS to the world. So the > > suggestion to "fix the glitch" and enable periodic scrubs makes perfect sense > > to me. Those who don't know any better will just magically receive the benefit > > of something they should've been doing all along. Those who do know better > > have a lot of time on their hands (at least judging by this thread!) and with > > way less typing than the length of these emails can disable the periodic > > script. Sounds win-win to me. > > > > -- > > > > Thanks, > > > > Josh Paetzel > > I think you are projecting a false characterisation of the average FreeBSD user and workload. FreeBSD is not an appliance and is not, in my experience, deployed or used by inexperienced users for any serious use-case. Who exactly is the user/use-case you are advocating for? > > There seems to be a conflation of purpose of scrubbing and general disk health; the point of a scrub is to catch bit errors in files that are infrequently accessed, that would otherwise not be detected in real-time. The value of that is largely dictated by whether you have RAID, and your backup strategy. A scrub does not tell you anything about the state of blocks that are not in use (or the overall health of a disk; ref smartd). > > Other people in this thread have already alluded to the fact that scrubbing can be detrimental, or even disastrous, for many of the workloads FreeBSD is used for. Excess scrubbing is a waste of energy and potentially reduces the working life of the disks. What I (and others in this thread) are getting at is setting a scrub schedule is so context specific that getting it right for the general case is near impossible, because there is no general case. > > Not scrubbing by default does not lead to unexpected behaviour. Scrubbing by default can and does lead to unexpected behaviour. People who care about their data use RAID and will determine a scrub schedule that suits their context. > > I take your point that, for a specific type of user, scrubbing by default makes sense. I'm just not sure that type of user is the benchmark for deciding FreeBSD system policies. If a user won't bear the cognitive load of understanding how to protect their own data, I would suggest FreeBSD is not for them and there are better options that will protect them better, by default (such as Ubuntu). Rather than assume a default which works for everyone, why not put an option screen in the installer if the user selects ZFS. They can choose from daily, weekly, monthly or never with a warning about possible (but rare) corruption on the latter option. Whatever the outcome of this discussion it should only be set on new installs as I think a lot of people would be upset if a new default scrub schedule were forced on them on upgrade, especially if they had one already defined. Regards, Gary From owner-freebsd-fs@freebsd.org Tue Aug 4 13:13:08 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 288EA3738F5 for ; Tue, 4 Aug 2020 13:13:08 +0000 (UTC) (envelope-from codeblue@inbox.lv) Received: from shark1.inbox.lv (shark1.inbox.lv [194.152.32.81]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLZtp72Hzz4LWM for ; Tue, 4 Aug 2020 13:13:06 +0000 (UTC) (envelope-from codeblue@inbox.lv) Received: from shark1.inbox.lv (localhost [127.0.0.1]) by shark1-out.inbox.lv (Postfix) with ESMTP id D559711181C4 for ; Tue, 4 Aug 2020 16:13:03 +0300 (EEST) Received: from localhost (localhost [127.0.0.1]) by shark1-in.inbox.lv (Postfix) with ESMTP id CF892111807D for ; Tue, 4 Aug 2020 16:13:03 +0300 (EEST) Received: from shark1.inbox.lv ([127.0.0.1]) by localhost (shark1.inbox.lv [127.0.0.1]) (spamfilter, port 35) with ESMTP id D3-nvk-5CSRC for ; Tue, 4 Aug 2020 16:13:03 +0300 (EEST) Received: from mail.inbox.lv (pop1 [127.0.0.1]) by shark1-in.inbox.lv (Postfix) with ESMTP id 216C811181C4 for ; Tue, 4 Aug 2020 16:13:03 +0300 (EEST) Received: from [10.0.0.2] (bzq-109-65-93-146.red.bezeqint.net [109.65.93.146]) (Authenticated sender: codeblue@inbox.lv) by mail.inbox.lv (Postfix) with ESMTPA id BC07F3DB126 for ; Tue, 4 Aug 2020 16:13:02 +0300 (EEST) Subject: Re: zfs scrub enable by default To: freebsd-fs@freebsd.org References: <24edb075-155c-439d-1ef5-541893036429@freebsd.org> <0d1d14c9-64ba-368c-b2f4-56efa741cc5e@inbox.lv> <2038290194.249397.1596502070473.JavaMail.zimbra@gray.id.au> From: John Long Message-ID: Date: Tue, 4 Aug 2020 13:12:59 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <2038290194.249397.1596502070473.JavaMail.zimbra@gray.id.au> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Virus-Scanned: OK X-ESPOL: EZqEOwUB6gdL+J/+N+Yf6uL4pKK+V1knvSf4z7447m4g1r7As9drdmqLEofxGAa/bg== X-Rspamd-Queue-Id: 4BLZtp72Hzz4LWM X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.67 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[inbox.lv:s=30062014]; RCVD_COUNT_FIVE(0.00)[6]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[194.152.32.81:from]; R_SPF_ALLOW(-0.20)[+ip4:194.152.32.81:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[inbox.lv:dkim]; RECEIVED_SPAMHAUS_PBL(0.00)[109.65.93.146:received]; NEURAL_HAM_MEDIUM(-1.02)[-1.019]; DKIM_TRACE(0.00)[inbox.lv:+]; DMARC_POLICY_ALLOW(-0.50)[inbox.lv,quarantine]; NEURAL_HAM_SHORT(-0.58)[-0.580]; NEURAL_HAM_LONG(-0.97)[-0.972]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:12993, ipnet:194.152.32.0/23, country:LV]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[194.152.32.81:from] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2020 13:13:08 -0000 On 04/08/2020 00:47, Grant Gray via freebsd-fs wrote: > > > ----- On 4 Aug, 2020, at 9:33 AM, Josh Paetzel josh@tcbug.org wrote: > >> On Mon, Aug 3, 2020, at 4:41 PM, John Long via freebsd-fs wrote: >>> On 03/08/2020 20:25, Allan Jude wrote: >>>> On 2020-08-03 12:10, Steve Wills wrote: >>>>> Hi, >>>>> >>>>> I wonder why we don't enable zfs periodic scrub by default? >>>>> >>>>> https://svnweb.freebsd.org/base/head/usr.sbin/periodic/periodic.conf?view=markup#l162 >>>>> >>>>> >>>>> Anyone happen to know? >>>>> >>>>> Thanks, >>>>> Steve >>>>> >>>>> _______________________________________________ >>>>> freebsd-fs@freebsd.org mailing list >>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>>> >>>> I think switching this to on-by-default is a good thing. >>>> >>>> To be clear, which the check is part of 'periodic daily', it only >>>> triggers a scrub if it has been more than 35 days since the last scrub. >>>> >>>> FreeNAS already has does this, and that accounts for a large number of >>>> FreeBSD ZFS deployments. >>> >>> There is a difference. FreeNAS is an appliance. It should minimize the >>> need for management. >>> >>> FreeBSD is the base OS. It should do as little as possible so people can >>> set it up the way we want rather than spend days or weeks unbreaking >>> surprising, non-standard behavior and hundreds of tweaks for everybody >>> who requested them. >>> >>>> There is tuning you can do in ZFS to try to lessen the impact of a scrub >>>> on your production workloads. >>> >>> That's up to the guy running the box to do or not to do. >>> >>>> The periodic script lets you select which pools to include (defaults to >>>> all), so you can tune it to only scrub your root pool every 35 days, and >>>> not the large pool that might take too long to scrub or whatever. It >>>> also lets you set a different threshold for each pool. >>> >>> A NAS appliance could benefit from something like this. A generic OS cannot. >>> >>>> So I don't see any reason not to enable it by default, and just document >>>> how to adjust it if people really need to disable it. Honestly, I think >>>> those who are disabling it are doing themselves a disservice. >>> >>> It's offensive for you to presume to think you know better what the >>> other guy needs than he does himself. Thank you for clarifying it >>> though, I think it's valuable to understand the thinking of people who >>> want to coopt the OS into their personal playground. >>> >>> /jl >>> _______________________________________________ >>> freebsd-fs@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>> >> >> ZFS needs scrub in the same way that UFS needs fsck. (in that i you don't scrub >> a ZFS volume for long enough and you don't fsck a UFS volume for long enough >> they will stop being useful filesystems) If you've ever had the joy of running >> UFS on a volume so large you couldn't fsck it or run say XFS on a volume so >> large you couldn't run xfs_repair you'll know that you can get away with that >> for a while, but eventually the chickens come home to roost and you'll be >> restoring from tape or failing over to your DR site, or complaining about lost >> data. (Note that XFS is journaled and is far more resilient than UFS to this >> sort of thing but it's still an issue) >> > > These are some fun (and real) historical anecdotes but, other than bit-rot, they do not represent current reality as these problems have been largely solved or the momentum has shifted to better filesystems (such as ZFS and XFS). That's correct. And furthermore, conflating UFS fsck and ZFS scrub as was done upthread is incorrect and unhelpful. > >> The reality is having scrub off is (in the vast majority of cases) the wrong >> thing to do. I say that as someone who has nearly an exabyte in ZFS right now, >> and lead the team that brought ZFS based FreeNAS to the world. So the >> suggestion to "fix the glitch" and enable periodic scrubs makes perfect sense >> to me. Those who don't know any better will just magically receive the benefit >> of something they should've been doing all along. Those who do know better >> have a lot of time on their hands (at least judging by this thread!) and with >> way less typing than the length of these emails can disable the periodic >> script. Sounds win-win to me. The reality is, scrub not running automagically is the default from the guys at Sun who brought us ZFS in the first place. It's a failure to rewrite history on your part that you suggest otherwise and it doesn't matter if it makes sense to you or not, unless you are talking about your own box. Do what you want. Don't expect us to like your ill-thought out, selfish demands for changes. It's not broken, don't try to fix it. If you want hand-holding or a turnkey system, use an appliance that somebody else built or (gasp!) build one yourself. >> >> -- >> >> Thanks, >> >> Josh Paetzel > > I think you are projecting a false characterisation of the average FreeBSD user and workload. FreeBSD is not an appliance and is not, in my experience, deployed or used by inexperienced users for any serious use-case. Who exactly is the user/use-case you are advocating for? > > There seems to be a conflation of purpose of scrubbing and general disk health; the point of a scrub is to catch bit errors in files that are infrequently accessed, that would otherwise not be detected in real-time. The value of that is largely dictated by whether you have RAID, and your backup strategy. A scrub does not tell you anything about the state of blocks that are not in use (or the overall health of a disk; ref smartd). > > Other people in this thread have already alluded to the fact that scrubbing can be detrimental, or even disastrous, for many of the workloads FreeBSD is used for. Excess scrubbing is a waste of energy and potentially reduces the working life of the disks. What I (and others in this thread) are getting at is setting a scrub schedule is so context specific that getting it right for the general case is near impossible, because there is no general case. > > Not scrubbing by default does not lead to unexpected behaviour. Scrubbing by default can and does lead to unexpected behaviour. People who care about their data use RAID and will determine a scrub schedule that suits their context. > > I take your point that, for a specific type of user, scrubbing by default makes sense. I'm just not sure that type of user is the benchmark for deciding FreeBSD system policies. If a user won't bear the cognitive load of understanding how to protect their own data, I would suggest FreeBSD is not for them and there are better options that will protect them better, by default (such as Ubuntu). And this is exactly the dividing line. But it's politically incorrect to say it. Thanks for your post, I agree with your points. By the way, my Solaris SPARC and Intel boxes don't scrub ZFS by default and over 15 years later the fact it has not been automated does not cause any problems. I take responsibility for managing my boxes and I'm categorically opposed to black magic and people or code intending to outsmart the guy running the box. /jl From owner-freebsd-fs@freebsd.org Tue Aug 4 15:54:54 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 49196378907 for ; Tue, 4 Aug 2020 15:54:54 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com [IPv6:2a00:1450:4864:20::532]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLfTT3zmlz4XBD for ; Tue, 4 Aug 2020 15:54:53 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-ed1-x532.google.com with SMTP id c10so546268edk.6 for ; Tue, 04 Aug 2020 08:54:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=+WKfzwdEuQVYvogm5R/DGKpS4AN6sUFuQ4F9BftYFB0=; b=ouFAclIFo40etlwWSsH8LMZGrnCS+cvpLeqI5nKHifA3eIMt+J7H96721o2SKlvf9l 8nVPsevDxZuSt57CRcyC515iouAhXqnyNzlQ9C56SexCNsHt5J1lh1V/UAi6EImR/PFb pvP1JsiUWx1UehzktUkmF7jKK4JEZKD4/dFDK19cIHfnFwKevNjU7buJqQ/kduEb4x1d 9xf9bLgSiMQiCsOf2XW+wxUwLsxZ4JIimj+PcEK88ulqY7ZbWbfDwbXrVMsD+faAvjkB i90gqQNqgDw8W/dbQso4mPsE1Yfk/NKFOcd31VmM47v622oh8ZZ0SOYiEXLSX46GfV3D 1PMg== X-Gm-Message-State: AOAM532jZtpJ7wXAj1R3CgaAUD0jCzINcRDQGgF02umIrXrGE/0pf+Cw xO9/zt/dk86Uu6XI9SjH01mqs3wTgAZ450qeBoAPsQ== X-Google-Smtp-Source: ABdhPJzlnTS/lGZ0fGYmYn7GJvS4ywSg7gjftMmezBdaBmx0GdKHXYshLYR2IHJPkAzh/OR0HyUsoeREKuECWc20nNQ= X-Received: by 2002:a05:6402:1282:: with SMTP id w2mr20340189edv.183.1596556491751; Tue, 04 Aug 2020 08:54:51 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Tue, 4 Aug 2020 08:54:41 -0700 Message-ID: Subject: Re: zfs scrub enable by default To: Steve Wills Cc: freebsd-fs , George Wilson X-Rspamd-Queue-Id: 4BLfTT3zmlz4XBD X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.02 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.01)[-1.011]; R_DKIM_ALLOW(-0.20)[delphix.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; NEURAL_HAM_LONG(-1.01)[-1.010]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[delphix.com:+]; DMARC_POLICY_ALLOW(-0.50)[delphix.com,quarantine]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::532:from]; NEURAL_HAM_SHORT(-0.29)[-0.295]; FORGED_SENDER(0.30)[matt@delphix.com,matthew.ahrens@delphix.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[matt@delphix.com,matthew.ahrens@delphix.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2020 15:54:54 -0000 This question was raised elsewhere, and I agree with this reply from George Wilson, my colleague and an expert in the i/o subsystems of ZFS as well as having lots of experience with customers: Having scrubs enabled by default is a great idea but at Sun (and Delphix > too) we found that the impact was often too much for some > workloads/customers. This is the challenge we faced and why there was never > a policy to enable it everywhere. We did explore ideas to make the impact > less and to be able to always scrub. Some of those ideas included periodic > or continuous scrubs where the impact could be reduced by only scrubbing > portions of the pool at a time, at a reduced i/o rate. At Delphix, we have > investigated similar concepts and one of our interns prototyped one of the > ideas.Much has changed since the early scrub days and revisiting some of > the earlier ideas and investigating new ones is probably a good topic for > the community. I do think that just enabling scrub by default without > further enhancements would still be too impactful for some customers but > the concept definitely has merit. --matt On Mon, Aug 3, 2020 at 9:10 AM Steve Wills wrote: > Hi, > > I wonder why we don't enable zfs periodic scrub by default? > > > https://svnweb.freebsd.org/base/head/usr.sbin/periodic/periodic.conf?view=markup#l162 > > Anyone happen to know? > > Thanks, > Steve > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Tue Aug 4 16:55:08 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B9C39379E82 for ; Tue, 4 Aug 2020 16:55:08 +0000 (UTC) (envelope-from swills@FreeBSD.org) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLgq01ZWWz4Zv4 for ; Tue, 4 Aug 2020 16:55:07 +0000 (UTC) (envelope-from swills@FreeBSD.org) Received: from lrrr.mouf.net (cpe-76-182-16-135.nc.res.rr.com [76.182.16.135]) (authenticated bits=0) by mouf.net (8.14.9/8.14.9) with ESMTP id 074Gsqd4070359 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 4 Aug 2020 16:54:58 GMT (envelope-from swills@FreeBSD.org) Subject: Re: zfs scrub enable by default To: Matthew Ahrens Cc: freebsd-fs , George Wilson References: From: Steve Wills Message-ID: Date: Tue, 4 Aug 2020 12:54:47 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:68.0) Gecko/20100101 Thunderbird/68.11.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Tue, 04 Aug 2020 16:54:58 +0000 (UTC) X-Spam-Status: No, score=-0.7 required=4.5 tests=KHOP_HELO_FCRDNS, NICE_REPLY_A autolearn=unavailable autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mouf.net X-Virus-Scanned: clamav-milter 0.99.2 at mouf.net X-Virus-Status: Clean X-Rspamd-Queue-Id: 4BLgq01ZWWz4Zv4 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; ASN(0.00)[asn:36236, ipnet:2607:fc50::/36, country:US]; local_wl_from(0.00)[FreeBSD.org] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2020 16:55:08 -0000 Hi, On 8/4/20 11:54 AM, Matthew Ahrens wrote: > This question was raised elsewhere, and I agree with this reply from > George Wilson, my colleague and an expert in the i/o subsystems of ZFS > as well as having lots of experience with customers: > > Having scrubs enabled by default is a great idea but at Sun (and > Delphix too) we found that the impact was often too much for some > workloads/customers. This is the challenge we faced and why there > was never a policy to enable it everywhere. We did explore ideas to > make the impact less and to be able to always scrub. Some of those > ideas included periodic or continuous scrubs where the impact could > be reduced by only scrubbing portions of the pool at a time, at a > reduced i/o rate. At Delphix, we have investigated similar concepts > and one of our interns prototyped one of the ideas.Much has changed > since the early scrub days and revisiting some of the earlier ideas > and investigating new ones is probably a good topic for the > community. I do think that just enabling scrub by default without > further enhancements would still be too impactful for some customers > but the concept definitely has merit. > Thanks for that! Very informative. I thought the Fishworks storage appliances had it on by default, but maybe I'm mistaken or maybe it changed over time. I wonder what "some" means, that is, is it 80% of people? 50%? 20? And what percent would mean "too many" to have it on and expect them to tune it if needed. I suppose there's no way to know. There are definitely some interesting ideas for how to limit the impact of scrub, but those would definitely have to be built and proven, of course. Thanks again, really helpful! Steve From owner-freebsd-fs@freebsd.org Tue Aug 4 17:25:25 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0417637A7BE for ; Tue, 4 Aug 2020 17:25:25 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-qv1-xf2c.google.com (mail-qv1-xf2c.google.com [IPv6:2607:f8b0:4864:20::f2c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLhTw0cqQz4cg4 for ; Tue, 4 Aug 2020 17:25:23 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-qv1-xf2c.google.com with SMTP id y11so16430595qvl.4 for ; Tue, 04 Aug 2020 10:25:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=eY9OYgECdx+XJXUTYI6sB1+vnP0qH76lBiHhwxGg8b0=; b=eJIbWHavRD3Bps0nA3lO1jyI4vetWP++17RI1dVCr/0nzWr0OexDpvcePfSKIua80W oIoHsd6NoGEzC6UXOPgorfqLJHn5h5yhKhRfQIx7h8DHHE8KXq2tosuS+CHMLII4KFN3 AVKQg5c/KFCWuLBFQ1J+csEKSx5H3ZldN6qhtQFan1wNVne0iGlP/SQLH0SUW21p3KnV Tcnckmmi6H7gcoYbkcfpCK/5SvxSYmdA1sIe8XhoJdnRX2zKqGSB+lE7trRVk+qli2Zt X2FL3AqRV42U4V3UY1o9e+nEnCLtahUMJQ1NlFh6O1k1TeSL1oxXWsI5PFIwbQgQIMGQ j+0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=eY9OYgECdx+XJXUTYI6sB1+vnP0qH76lBiHhwxGg8b0=; b=CxTliYxYLGzuIMSwVWmCbzJ4j6EVJOrSusaedFWgMMCtmt7viigR8/L7LCtdqUF7qj YM7oErbYnrqypho6K+ugzbVS67D54sjduD6YtT7gS5Pv6hk+xH9eWlVEP+vNr4eVVYtH oih+jpshbN2jo4lFGdS+gkEZX1rzIWIMnv7yszsmkoiA+y+dURS4MlcK4IspDf3I8CTf YVh7yFQ04zQXLNHNFGhRqP4PXOsRoYce3kjIyYapzYH27K3Fgc797HBSnS/uYH1tZkIn VEJAtc/XK8yA3mjpelHdihsOqL9eHONjf00IMBj4fqOSyEhmqcMP8MHXoL1jLs2gv4ul jBDg== X-Gm-Message-State: AOAM533Db7GtjcU/K+XeABjmzyjegWQQXeRJZS224Pn0ZMK382/EbCvP OiEDH23PjaB3voJnIdo1k1dasmcKHYc+TYOO7c+gBw== X-Google-Smtp-Source: ABdhPJzN45qp1ex7Y3A4I55sxU6EleH/ESD/Q8GrxQn2Ez1HbOLe0dIXYci83mtDKJOIkywKZim2Zx6ZEXPX7otG3Ug= X-Received: by 2002:a0c:b599:: with SMTP id g25mr23597538qve.118.1596561922449; Tue, 04 Aug 2020 10:25:22 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Warner Losh Date: Tue, 4 Aug 2020 11:25:11 -0600 Message-ID: Subject: Re: zfs scrub enable by default To: Steve Wills Cc: Matthew Ahrens , freebsd-fs , George Wilson X-Rspamd-Queue-Id: 4BLhTw0cqQz4cg4 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20150623.gappssmtp.com header.s=20150623 header.b=eJIbWHav; dmarc=none; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2607:f8b0:4864:20::f2c) smtp.mailfrom=wlosh@bsdimp.com X-Spamd-Result: default: False [-2.36 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.94)[-0.940]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-0.93)[-0.928]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; DMARC_NA(0.00)[bsdimp.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20150623.gappssmtp.com:+]; NEURAL_HAM_SHORT(-0.49)[-0.489]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::f2c:from]; R_SPF_NA(0.00)[no SPF record]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2020 17:25:25 -0000 On Tue, Aug 4, 2020 at 10:55 AM Steve Wills wrote: > Hi, > > On 8/4/20 11:54 AM, Matthew Ahrens wrote: > > This question was raised elsewhere, and I agree with this reply from > > George Wilson, my colleague and an expert in the i/o subsystems of ZFS > > as well as having lots of experience with customers: > > > > Having scrubs enabled by default is a great idea but at Sun (and > > Delphix too) we found that the impact was often too much for some > > workloads/customers. This is the challenge we faced and why there > > was never a policy to enable it everywhere. We did explore ideas to > > make the impact less and to be able to always scrub. Some of those > > ideas included periodic or continuous scrubs where the impact could > > be reduced by only scrubbing portions of the pool at a time, at a > > reduced i/o rate. At Delphix, we have investigated similar concepts > > and one of our interns prototyped one of the ideas.Much has changed > > since the early scrub days and revisiting some of the earlier ideas > > and investigating new ones is probably a good topic for the > > community. I do think that just enabling scrub by default without > > further enhancements would still be too impactful for some customers > > but the concept definitely has merit. > > > > Thanks for that! Very informative. I thought the Fishworks storage > appliances had it on by default, but maybe I'm mistaken or maybe it > changed over time. > > I wonder what "some" means, that is, is it 80% of people? 50%? 20? And > what percent would mean "too many" to have it on and expect them to tune > it if needed. I suppose there's no way to know. > > There are definitely some interesting ideas for how to limit the impact > of scrub, but those would definitely have to be built and proven, of > course. > Yea, without numbers, it's unclear what to do with this advice since it says both "do it" and "don't do it" depending on how you read it. Like Steve said, if "some" is 80% it's a clear case for not enabling by default. If it is 5% or 10%, then the case is clear to enable it by default... I'm in the 'enable by default' camp *NOW* and keep a close eye out for the next six months. If there's only a couple of issues, leave it for the release. If there's all kinds of issues, then turn it back off. Better scrubbing is always possible, depending on the workload. We have a much better scrubber than before, and I think we should at least try it by default absent data indicating a big issue. Warner From owner-freebsd-fs@freebsd.org Tue Aug 4 17:28:53 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0518737A4BA for ; Tue, 4 Aug 2020 17:28:53 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLhYw1CpHz4cv6 for ; Tue, 4 Aug 2020 17:28:51 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (096-033-205-208.res.spectrum.com [96.33.205.208]) by colo1.denninger.net (Postfix) with ESMTP id EEC132110BD for ; Tue, 4 Aug 2020 13:28:49 -0400 (EDT) Received: from [192.168.10.25] (D15.Denninger.Net [192.168.10.25]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id 8C4DA132557 for ; Tue, 4 Aug 2020 13:28:50 -0400 (EDT) Subject: Re: zfs scrub enable by default To: freebsd-fs@freebsd.org References: From: Karl Denninger Message-ID: <5b5b179b-f478-3561-a3ea-3b2022cd9215@denninger.net> Date: Tue, 4 Aug 2020 13:28:50 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.1.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms050209010502070702020003" X-Rspamd-Queue-Id: 4BLhYw1CpHz4cv6 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=denninger.net; spf=pass (mx1.freebsd.org: domain of karl@denninger.net designates 104.236.120.189 as permitted sender) smtp.mailfrom=karl@denninger.net X-Spamd-Result: default: False [-5.46 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.003]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.03)[-1.027]; RCVD_COUNT_THREE(0.00)[3]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; NEURAL_HAM_SHORT(-0.53)[-0.533]; DMARC_POLICY_ALLOW(-0.50)[denninger.net,none]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~,4:~]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2020 17:28:53 -0000 This is a cryptographically signed message in MIME format. --------------ms050209010502070702020003 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable On 8/4/2020 13:25, Warner Losh wrote: > On Tue, Aug 4, 2020 at 10:55 AM Steve Wills wrote:= > >> Hi, >> >> On 8/4/20 11:54 AM, Matthew Ahrens wrote: >>> This question was raised elsewhere, and I agree with this reply from >>> George Wilson, my colleague and an expert in the i/o subsystems of ZF= S >>> as well as having lots of experience with customers: >>> >>> Having scrubs enabled by default is a great idea but at Sun (and= >>> Delphix too) we found that the impact was often too much for som= e >>> workloads/customers. This is the challenge we faced and why ther= e >>> was never a policy to enable it everywhere. We did explore ideas= to >>> make the impact less and to be able to always scrub. Some of tho= se >>> ideas included periodic or continuous scrubs where the impact co= uld >>> be reduced by only scrubbing portions of the pool at a time, at = a >>> reduced i/o rate. At Delphix, we have investigated similar conce= pts >>> and one of our interns prototyped one of the ideas.Much has chan= ged >>> since the early scrub days and revisiting some of the earlier id= eas >>> and investigating new ones is probably a good topic for the >>> community. I do think that just enabling scrub by default withou= t >>> further enhancements would still be too impactful for some custo= mers >>> but the concept definitely has merit. >>> >> Thanks for that! Very informative. I thought the Fishworks storage >> appliances had it on by default, but maybe I'm mistaken or maybe it >> changed over time. >> >> I wonder what "some" means, that is, is it 80% of people? 50%? 20? And= >> what percent would mean "too many" to have it on and expect them to tu= ne >> it if needed. I suppose there's no way to know. >> >> There are definitely some interesting ideas for how to limit the impac= t >> of scrub, but those would definitely have to be built and proven, of >> course. >> > Yea, without numbers, it's unclear what to do with this advice since it= > says both "do it" and "don't do it" depending on how you read it. Like > Steve said, if "some" is 80% it's a clear case for not enabling by defa= ult. > If it is 5% or 10%, then the case is clear to enable it by default... > > I'm in the 'enable by default' camp *NOW* and keep a close eye out for = the > next six months. If there's only a couple of issues, leave it for the > release. If there's all kinds of issues, then turn it back off. > > Better scrubbing is always possible, depending on the workload. We have= a > much better scrubber than before, and I think we should at least try it= by > default absent data indicating a big issue. > > Warner > _______________________________________________ It hammers the living daylights out of performance on a RaidZx=20 filesystem. On a mirror set not nearly so much. If you have an actual workload on that RaidZ system while it's running=20 you are very likely to see material and objectionable performance=20 impact.=C2=A0 If you can start it when there's no material load and it=20 finishes before there is some you won't notice it, but if not, well..... --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms050209010502070702020003 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMjAwODA0MTcyODUw WjBPBgkqhkiG9w0BCQQxQgRAMpMeG5X/83pKbgQHPkB97JwgUfKi4yeiIvLh5iaUDLFu36A0 v6kGs0GJF3Cy1xt55HLV5NZB1qS/YZQaxhLznTBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgCpuzxm4E9atCMQlr+O66oj7kL2R+MTW8Zrxx3Z2f3APd7f8Uco3iKt2FU6LgIVFEJ5 8bhGA82Ezyn0SOsN4idxPeYmYnitjChq03+zW8KTs/1Rr+wTPAejKaOTOF/OS4Vgpu69c8SH svUEdZdUnRTkCMqZMA5CNJXDxCUeLIhk1LvhnhgKVblSPYDIqA4N3G1G1pm3gIB/LrBll4Qh Ygbng37ZrpEvYUP9IGu6ApOLU+5zQxVDsChsCCBf/ZhscH59zjHdbBuwgk46CJz38sKjzkmc J3ym574ZbwqY2l8vObdb+K6XjqqIYBVD9S0uHgR+3J3Avup/H5nohuCqTa14dTq4MQOh8XjZ eF/DVk/K/6up6QrUwsj76A/mQQwEWGxvLFLrhUDubC6ERPrq8UgI0On41Tuts3n3zMuhZuGX IlnFciXtDX9ZjItSTe8IZ1Wmu4cgWl8NuUja7yqHCq9iXuzZHyO6QFs9FFUikM5eO3n8sHPt rUr77+VXk6OQrfctaytUkGjg3nrnEk24zV1rlww8+i8/jphJ+J0fwq1NPlTJnixrdVrenvKU bfkNVajJPeWHQICAYRVGqEIvA6YchNMMTO2QlT9ZWffdFSh66DQVEPUSmzZHQnr+SmUmoaF1 l6YpRLix/is4EgaCuAYg44Sf9Y4CT9sPig22kp9AiwAAAAAAAA== --------------ms050209010502070702020003-- From owner-freebsd-fs@freebsd.org Tue Aug 4 20:13:06 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D78E637E273 for ; Tue, 4 Aug 2020 20:13:06 +0000 (UTC) (envelope-from ryan@ixsystems.com) Received: from mail-ed1-x541.google.com (mail-ed1-x541.google.com [IPv6:2a00:1450:4864:20::541]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLmCQ13W2z3ZHM for ; Tue, 4 Aug 2020 20:13:05 +0000 (UTC) (envelope-from ryan@ixsystems.com) Received: by mail-ed1-x541.google.com with SMTP id c15so21152772edj.3 for ; Tue, 04 Aug 2020 13:13:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ixsystems.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=/KAP0zIv+BgdNlYUAUkXqaEkjgWHVrD6JvfbhUTDaRA=; b=iJgOxZgp3Jcm8e0ClcTqp9YgSfoTjdtp8gtpsHUakg5Y4a/4vzHPZsb6mCoDMBevT1 p+ED00d5DdJ3USmynQt2hkeT6cV4VYrv1ErDcMRHb4kK1MGBJLlnWqmxWhaulHJws3Be TXWLT+sYVCSu75LsFKVsmGHRveF1jpkqLFGuWr3u0ZeliRDJyMsppskdsuGag1eoxOm5 BK0jR3Yg8RsqAFM4U/5P9pL0PhaLcmRWrra50fTsa9N4G+d0x90iOpya5fIJa3pHCNwx 6OkF1OIYmZxAgVr6IV/bbSr1KQsq3zMWXfdhGPI3SO3spCNWKFQhzPE1Sc47V9XO1NIK CrFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=/KAP0zIv+BgdNlYUAUkXqaEkjgWHVrD6JvfbhUTDaRA=; b=iD4cKh229Y4nQjVlHty37aK552sUn0XvnwsDid9BsWpTuZdkYT38chsSjk7/Erf7Pm pl14K6ChqlAQ59dWBFocqcS5bf0P9r2bGQGww9EMFbMccKQVQjVRom8754yVdP5JlKRV zfJm2yMcHeNnzwFeXZuDyyM8V+HcDKsMkicyzjIzxQ6vJ1ShEhyFvcMJbQXpFRNsz7vm 5Yt0uVgh/1LhKeMdHh/xILeH7c3um/q0fwTpf7BP9MnJXsil1/bXH8fZexuelHQkv0yp 39Qv8HFDJjuoOXXFpL7gByjR6ZcXh6zEMf1cB1Gvq4EaofjkfFyZwYyf+/R0T/Nx/lsZ L5nw== X-Gm-Message-State: AOAM531gMppDWqpk+ivh1VZPs5xZVEtJaK/jOf8qGPopIi2J1q0RoLkB Jhpe/2v9E5ahqkrTq2xWf2YTTy2xJ3NkRkjOoaxpUA== X-Google-Smtp-Source: ABdhPJz3XERNFLPVbn7WLdiTUGeHJFic3z1njmT6Z2nLg6P22NY/R3JQePyWITrMw0mjd2hX1OlR2uab3Da5UZpIeGY= X-Received: by 2002:a05:6402:3121:: with SMTP id dd1mr22771547edb.72.1596571984602; Tue, 04 Aug 2020 13:13:04 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Ryan Moeller Date: Tue, 4 Aug 2020 16:12:53 -0400 Message-ID: Subject: Re: zfs scrub enable by default To: Warner Losh Cc: Steve Wills , freebsd-fs , George Wilson Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4BLmCQ13W2z3ZHM X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ixsystems.com header.s=google header.b=iJgOxZgp; dmarc=pass (policy=none) header.from=ixsystems.com; spf=pass (mx1.freebsd.org: domain of ryan@ixsystems.com designates 2a00:1450:4864:20::541 as permitted sender) smtp.mailfrom=ryan@ixsystems.com X-Spamd-Result: default: False [-3.53 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.01)[-1.014]; R_DKIM_ALLOW(-0.20)[ixsystems.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; NEURAL_HAM_LONG(-1.02)[-1.019]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[ixsystems.com:+]; DMARC_POLICY_ALLOW(-0.50)[ixsystems.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::541:from]; NEURAL_HAM_SHORT(-0.49)[-0.495]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Aug 2020 20:13:06 -0000 On Tue, Aug 4, 2020 at 1:25 PM Warner Losh wrote: > > On Tue, Aug 4, 2020 at 10:55 AM Steve Wills wrote: > > > Hi, > > > > On 8/4/20 11:54 AM, Matthew Ahrens wrote: > > > This question was raised elsewhere, and I agree with this reply from > > > George Wilson, my colleague and an expert in the i/o subsystems of ZFS > > > as well as having lots of experience with customers: > > > > > > Having scrubs enabled by default is a great idea but at Sun (and > > > Delphix too) we found that the impact was often too much for some > > > workloads/customers. This is the challenge we faced and why there > > > was never a policy to enable it everywhere. We did explore ideas to > > > make the impact less and to be able to always scrub. Some of those > > > ideas included periodic or continuous scrubs where the impact could > > > be reduced by only scrubbing portions of the pool at a time, at a > > > reduced i/o rate. At Delphix, we have investigated similar concepts > > > and one of our interns prototyped one of the ideas.Much has changed > > > since the early scrub days and revisiting some of the earlier ideas > > > and investigating new ones is probably a good topic for the > > > community. I do think that just enabling scrub by default without > > > further enhancements would still be too impactful for some customers > > > but the concept definitely has merit. > > > > > > > Thanks for that! Very informative. I thought the Fishworks storage > > appliances had it on by default, but maybe I'm mistaken or maybe it > > changed over time. > > > > I wonder what "some" means, that is, is it 80% of people? 50%? 20? And > > what percent would mean "too many" to have it on and expect them to tune > > it if needed. I suppose there's no way to know. > > > > There are definitely some interesting ideas for how to limit the impact > > of scrub, but those would definitely have to be built and proven, of > > course. > > > > Yea, without numbers, it's unclear what to do with this advice since it > says both "do it" and "don't do it" depending on how you read it. Like > Steve said, if "some" is 80% it's a clear case for not enabling by default. > If it is 5% or 10%, then the case is clear to enable it by default... > > I'm in the 'enable by default' camp *NOW* and keep a close eye out for the > next six months. If there's only a couple of issues, leave it for the > release. If there's all kinds of issues, then turn it back off. It's easy to recover from lost performance by stopping the scrub and reconfiguring the periodic script. It's harder to recover from data loss by changing your configuration after the fact. > Better scrubbing is always possible, depending on the workload. We have a > much better scrubber than before, and I think we should at least try it by > default absent data indicating a big issue. > > Warner > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" -- Ryan Moeller iXsystems, Inc. OS Developer Email: ryan@iXsystems.com From owner-freebsd-fs@freebsd.org Wed Aug 5 00:18:23 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3ABB63A4768 for ; Wed, 5 Aug 2020 00:18:23 +0000 (UTC) (envelope-from grant@gray.id.au) Received: from mail.agc.net.au (mail.agc.net.au [167.86.119.123]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLsfP5S0Gz44ft for ; Wed, 5 Aug 2020 00:18:20 +0000 (UTC) (envelope-from grant@gray.id.au) Received: from localhost (localhost [127.0.0.1]) by mail.agc.net.au (Postfix) with ESMTP id 56B862A64EA; Wed, 5 Aug 2020 10:18:17 +1000 (AEST) Received: from mail.agc.net.au ([127.0.0.1]) by localhost (mail.agc.net.au [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id lFwhWWZTB-n3; Wed, 5 Aug 2020 10:18:16 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by mail.agc.net.au (Postfix) with ESMTP id D664B2A65C0; Wed, 5 Aug 2020 10:18:15 +1000 (AEST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.agc.net.au D664B2A65C0 X-Virus-Scanned: amavisd-new at mail.agc.net.au Received: from mail.agc.net.au ([127.0.0.1]) by localhost (mail.agc.net.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 3TIuE6rYDvI8; Wed, 5 Aug 2020 10:18:15 +1000 (AEST) Received: from mail.agc.net.au (mail.agc.net.au [167.86.119.123]) by mail.agc.net.au (Postfix) with ESMTP id 869C32A6727; Wed, 5 Aug 2020 10:18:15 +1000 (AEST) Date: Wed, 5 Aug 2020 10:18:14 +1000 (AEST) From: Grant Gray To: Ryan Moeller Cc: Warner Losh , freebsd-fs , George Wilson Message-ID: <105090343.294898.1596586694925.JavaMail.zimbra@gray.id.au> In-Reply-To: References: Subject: Re: zfs scrub enable by default MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [167.86.119.123] X-Mailer: Zimbra 8.8.15_GA_3945 (ZimbraWebClient - FF80 (Linux)/8.8.15_GA_3953) Thread-Topic: zfs scrub enable by default Thread-Index: JfsIOSIu4bMWHWqq06BqiLLzYmP/4A== X-Rspamd-Queue-Id: 4BLsfP5S0Gz44ft X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.39 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.04)[-1.037]; RCVD_COUNT_FIVE(0.00)[6]; HAS_XOIP(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.01)[-1.007]; MIME_GOOD(-0.10)[text/plain]; R_DKIM_ALLOW(-0.20)[gray.id.au:s=30D3F2DC-EB1E-11E9-943D-4BC9C5489560]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gray.id.au:+]; DMARC_POLICY_ALLOW(-0.50)[gray.id.au,quarantine]; NEURAL_HAM_SHORT(-0.35)[-0.349]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:51167, ipnet:167.86.118.0/23, country:DE]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2020 00:18:23 -0000 > > It's easy to recover from lost performance by stopping the scrub and > reconfiguring the periodic script. It's harder to recover from data > loss by changing your configuration after the fact. > You are making a great example of the two core issue with this proposition. 1. That the side-effect of performance loss due to scrubbing is benign. I have a customer using hardware iSCSI HBA's attached to ZFS zvols. This is a notoriously bad case wrt to sync performance (yes this will get better 'soon' with pending patches). A scrub results in most/all of the attached systems BSOD'ing. This is just a personal example; others will suffer reduced transaction rates, poor customer experience, lost revenue etc. 2. That scrubbing PREVENTS data loss. Scrubbing can only tell you about data loss AFTER it happens. Yes, it could alert you to a problem that prevents further data loss, but it's already too late. Scrubbing is not a substitute for RAID, backups and proactive SMART testing. From owner-freebsd-fs@freebsd.org Wed Aug 5 01:21:35 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 57C293A579A for ; Wed, 5 Aug 2020 01:21:35 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from smtp.simplesystems.org (smtp.simplesystems.org [65.66.246.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLv3L2mrHz46wk for ; Wed, 5 Aug 2020 01:21:34 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from scrappy.simplesystems.org (scrappy.simplesystems.org [65.66.246.73]) by smtp.simplesystems.org (8.14.4+Sun/8.14.4) with ESMTP id 0751LPbI007218; Tue, 4 Aug 2020 20:21:26 -0500 (CDT) Date: Tue, 4 Aug 2020 20:21:25 -0500 (CDT) From: Bob Friesenhahn X-X-Sender: bfriesen@scrappy.simplesystems.org To: Grant Gray cc: freebsd-fs Subject: Re: zfs scrub enable by default In-Reply-To: <105090343.294898.1596586694925.JavaMail.zimbra@gray.id.au> Message-ID: References: <105090343.294898.1596586694925.JavaMail.zimbra@gray.id.au> User-Agent: Alpine 2.20 (GSO 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (smtp.simplesystems.org [65.66.246.90]); Tue, 04 Aug 2020 20:21:26 -0500 (CDT) X-Rspamd-Queue-Id: 4BLv3L2mrHz46wk X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of bfriesen@simple.dallas.tx.us designates 65.66.246.90 as permitted sender) smtp.mailfrom=bfriesen@simple.dallas.tx.us X-Spamd-Result: default: False [-2.97 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.01)[-1.009]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.04)[-1.039]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[dallas.tx.us]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.62)[-0.620]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:7018, ipnet:65.64.0.0/13, country:US]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2020 01:21:35 -0000 On Wed, 5 Aug 2020, Grant Gray via freebsd-fs wrote: > > 2. That scrubbing PREVENTS data loss. Scrubbing can only tell you about data loss AFTER it happens. Yes, it could alert you to a problem that prevents further data loss, but it's already too late. Scrubbing is not a substitute for RAID, backups and proactive SMART testing. Any reasonable zfs pool should have sufficient hardware redundancy to allow it sufficient time to work without losing data before a human is able to notice and replace the failing hardware. You make a good point about scrubbing reporting data loss after it happens, but usually there will not be actual data loss if the pool is designed properly with sufficient redundancy. In this case, the scrub failure may be an indication of decreased redundancy and need for attention from an attendant human. In some cases there might just be a minor media failure and a resilver will put a redundant copy of the data elsewhere. If the pool has more redundancy, then there is less need to scrub it often. Scrub simple duplex mirrors more often than raidz3. There is an old saying "If a tree falls in the forest and no one is there to hear the noise, does it make a sound?". This is somewhat applicable to zfs since if the data is so derelict that no one accesses it at all so that it can be detected to be going 'bad', is it really that important to protect? Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt From owner-freebsd-fs@freebsd.org Wed Aug 5 02:01:00 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 02E7F3A5F30 for ; Wed, 5 Aug 2020 02:01:00 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLvwq08J7z48Wv for ; Wed, 5 Aug 2020 02:00:58 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (096-033-205-208.res.spectrum.com [96.33.205.208]) by colo1.denninger.net (Postfix) with ESMTP id A62722110B5 for ; Tue, 4 Aug 2020 22:00:21 -0400 (EDT) Received: from [192.168.10.25] (D15.Denninger.Net [192.168.10.25]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id 3E0B313409F for ; Tue, 4 Aug 2020 22:00:22 -0400 (EDT) Subject: Re: zfs scrub enable by default To: freebsd-fs@freebsd.org References: <105090343.294898.1596586694925.JavaMail.zimbra@gray.id.au> From: Karl Denninger Message-ID: Date: Tue, 4 Aug 2020 22:00:23 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.1.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms090900040404050202040303" X-Rspamd-Queue-Id: 4BLvwq08J7z48Wv X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=denninger.net; spf=pass (mx1.freebsd.org: domain of karl@denninger.net designates 104.236.120.189 as permitted sender) smtp.mailfrom=karl@denninger.net X-Spamd-Result: default: False [-5.80 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.03)[-1.027]; FROM_HAS_DN(0.00)[]; SIGNED_SMIME(-2.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; TO_DN_NONE(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.02)[-1.023]; RCVD_COUNT_THREE(0.00)[3]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; NEURAL_HAM_SHORT(-0.85)[-0.849]; DMARC_POLICY_ALLOW(-0.50)[denninger.net,none]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~,4:~]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2020 02:01:00 -0000 This is a cryptographically signed message in MIME format. --------------ms090900040404050202040303 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable On 8/4/2020 21:21, Bob Friesenhahn wrote: > On Wed, 5 Aug 2020, Grant Gray via freebsd-fs wrote: >> >> 2. That scrubbing PREVENTS data loss. Scrubbing can only tell you=20 >> about data loss AFTER it happens. Yes, it could alert you to a=20 >> problem that prevents further data loss, but it's already too late.=20 >> Scrubbing is not a substitute for RAID, backups and proactive SMART=20 >> testing. > > Any reasonable zfs pool should have sufficient hardware redundancy to=20 > allow it sufficient time to work without losing data before a human is = > able to notice and replace the failing hardware. You assume much. Let me give you two allegedly "degenerate" cases that are actually not=20 degenerate at all. 1. A laptop or workstation.=C2=A0 It is backed up.=C2=A0 It uses ZFS beca= use it's=20 faster, and I can establish a filesystem for some project very easily=20 and quickly, it's segregated, I can work on it and destroy it trivially=20 when done.=C2=A0 I can set quotas on that, etc.=C2=A0 If I want to move i= ts=20 mountpoint, I can trivially do so. And so on.=C2=A0 Note that here there = is=20 no redundancy at all; no raidZx, no mirroring, etc.=C2=A0 I'm merely usin= g it=20 for convenience. 2. The same, but with multiple BEs.=C2=A0 Same idea, but now I can have=20 multiple OS versions and change between them for a given boot very=20 quickly while leaving my user data (and application software) alone.=C2=A0= =20 VERY useful if I'm doing code development on the OS (and I do that once=20 in a while) or application development and want to test against=20 different FreeBSD revisions (which I do a LOT.) Again, I don't get a wet = crap about redundancy, but I care very much about the convenience. My laptop is set up with ZFS and has exactly one storage device in it.=C2= =A0=20 I don't consider that foolish at all for the above reasons (and it has=20 kept me from hair-pulling exercises when drm-kmod problems bit people,=20 including me), but a scrub on an automated "schedule" basis for that=20 machine is IMHO cray-cray material. --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms090900040404050202040303 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMjAwODA1MDIwMDIz WjBPBgkqhkiG9w0BCQQxQgRAATLmUWlhwLKeSHQBoDZ0bpt4ouM25mONFVktFgrqZGRZKr0V cveaFmWFlHzEKHPYjYvr2KPT/pVTlqWcbskO2TBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgCScnNgpf7LQIveiPEZDbZR/BXqFQJ9F/ccu/BV4rcA+irSPm7sIGcv3SP7mHtjmttb Z4MIC0wwNvHzz5biyeEdsnLQ+cI3DX2sqFla5WhySoqsqccjUR2xUUvILGi49DroYjZ9jvM1 zmny1F7mLergq+VcgaV8lRr4oUNofOM1ZjR+Us7HEoyhPHpV+YKN9eneVYRICkUD1J0PIY37 7en8L0VarlAifu9f0VOOU4jZkL+nUY+VZNPY6TmoZFCGPARORoKvMnhi1CWxAaugn+vYQTAX ybMX6ITsZyAJzKFvQuxAK01Lm+J5Y9ujUHLbgpzKUfIIZr/j4t/2a6+Pqym4wWD3CjnQ+MDU HW8KQOle/fgWXdpVPGmV01N24PcO9TpZLEGg9BD2Azkh5CDbxq1A94OR+cQHPYmG6meoAQjC y/a/NODUb4YbX5XO2Ujme3KQmeJinBLyK+q6O/Xqtf6lWIxq7bl/8deKREB41otcScZXkfT3 ZZhUhIN1RCICxpChBUXYTzkrkMSfPWm3Fr+93cdK5y/QIpuXhQXeXHBzVfYbMhsrfDjHL+gm UvAsSVK04q61FFtPa085UBT/DGswk0HkddMKRjIcXWWnXTNGksLVVTTos4DWcdzIV9X872wT BqXwy+l3Tqf4tB5jSIOYvRr3q1K3YXmlyn/7r4rKbgAAAAAAAA== --------------ms090900040404050202040303-- From owner-freebsd-fs@freebsd.org Wed Aug 5 04:32:40 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 592793A9DFC for ; Wed, 5 Aug 2020 04:32:40 +0000 (UTC) (envelope-from ryan@ixsystems.com) Received: from mail-ej1-x62d.google.com (mail-ej1-x62d.google.com [IPv6:2a00:1450:4864:20::62d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLzHq3yX4z4Hyl for ; Wed, 5 Aug 2020 04:32:39 +0000 (UTC) (envelope-from ryan@ixsystems.com) Received: by mail-ej1-x62d.google.com with SMTP id f24so24383263ejx.6 for ; Tue, 04 Aug 2020 21:32:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ixsystems.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=TsWZbdJ+H0hfG5tLHMEM9Qkp+ORrHh3aru4c11sESLo=; b=kY6rMQAhx5I6vtIWkQQgeA5nAe1aENKjTyqjVOUQ6cN/3q+FCluiV6SvfGDxqJY2i6 DMcBTkeGmxQqjWCVjDV48bZ6tZW0r2gJ4NoJ35Y4kMpz8oIziJAKQLKXJ+X5UQjTtMsO 4MK5OdqXnqMcVjsXeX3q4VlQ7aTR2A8EeyO4YyGXqMjgEIdVL1TiewcqcVLwXG9ZKycg s0IOq13JeVISDmb+xKXIDX8bOGUo064TOj/kAvK8dllOpBGCrsY5K8fLeAIUGi40RipD pXodFzbgb4g+/9A1JTa10z7Ud0V9DX5v1AlLDoRSX8F3v+2IkCAvQ6wWYPpcIoUDCR6t Ym0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=TsWZbdJ+H0hfG5tLHMEM9Qkp+ORrHh3aru4c11sESLo=; b=gMmqbIuqpCOhZXo3sNHWx9x5hrm4DxM2B7QfUwKvm+x944L82eTlhAqZi93QGsPZkF 3hOdcjOuZDf4tExuMzb4XhOYsiQPGlhNii5YMo1TeyDDdv+NrSbHH93U1oFhfO8vbAjr pITlZpsoDGhuN50xyhtayvx8l2pyRAXHsMjodKXWJeBR1XAxhOOpFck17ms7gCFoXS79 8vFQTmQY3gIVNbtfKhEpiNkJVxW+QAmREOid6Ev/h6DQxBuIqf9h1w0SnWb1LICgwML4 3KrUSA5ijgnEJ3iW3xQM1uCacBrE96zSOV0Os2bFO01GDT7gIoVtvHyOymga96rNBXQj k6qg== X-Gm-Message-State: AOAM533rEhMPn5Y4aRjAUaxkqyvjAbfX0rxUJBCs4H0Dxfj9C2ywhaWo 1brgSMXDi9hEtCrga8hnobtP6c43mWLZvthOuykEFA== X-Google-Smtp-Source: ABdhPJyJTNRgYAbNXZs1XxBllMWngHbNh65JOa+p3LL5l4mdI2vUxXSaFLlNYor7xo/9BMyqBYDZcHh9ugJLSU94Ims= X-Received: by 2002:a17:906:82ca:: with SMTP id a10mr775198ejy.524.1596601957959; Tue, 04 Aug 2020 21:32:37 -0700 (PDT) MIME-Version: 1.0 References: <105090343.294898.1596586694925.JavaMail.zimbra@gray.id.au> In-Reply-To: From: Ryan Moeller Date: Wed, 5 Aug 2020 00:32:26 -0400 Message-ID: Subject: Re: zfs scrub enable by default To: "PK1048.COM" Cc: Grant Gray , freebsd-fs , George Wilson Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4BLzHq3yX4z4Hyl X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ixsystems.com header.s=google header.b=kY6rMQAh; dmarc=pass (policy=none) header.from=ixsystems.com; spf=pass (mx1.freebsd.org: domain of ryan@ixsystems.com designates 2a00:1450:4864:20::62d as permitted sender) smtp.mailfrom=ryan@ixsystems.com X-Spamd-Result: default: False [-2.87 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.991]; R_DKIM_ALLOW(-0.20)[ixsystems.com:s=google]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; NEURAL_HAM_LONG(-1.00)[-1.004]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; NEURAL_SPAM_SHORT(0.13)[0.130]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[ixsystems.com:+]; DMARC_POLICY_ALLOW(-0.50)[ixsystems.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62d:from]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2020 04:32:40 -0000 On Tue, Aug 4, 2020 at 11:36 PM PK1048.COM wrote: > > On Aug 4, 2020, at 8:18 PM, Grant Gray via freebsd-fs wrote: > > > 2. That scrubbing PREVENTS data loss. Scrubbing can only tell you about= data loss AFTER it happens. Yes, it could alert you to a problem that prev= ents further data loss, but it's already too late. Scrubbing is not a subst= itute for RAID, backups and proactive SMART testing. > > Scrubbing does not directly prevent data loss. Scrubbing identifies data = returned from the underlying device as corrupt or incorrect. If the zpool h= as sufficient redundancy, then no data is lost. > > But, normal ZFS operation also identifies incorrect date returned from th= e underlying device(s). > > While I have only managed a few hundred drives under ZFS, I cannot recall= one case where a proactive scrub found bad data _before_ normal operations= . Once a checksum error is reported via `zpool status` then I have found a = scrub useful to determine the extent of the drive failure. > > I know there are others with thousands of drives managed under ZFS and th= eir experience may differ. > > In terms of whether the periodic scrub should be enabled by default, I am= undecided, as I can see both sides of the argument. I would prefer to make= it a user choice during installation as one of the ZFS options (such as na= tive 4K drives, ashift=3D12 or mirrored swap). Make the default enabled, bu= t by putting it in the installation options, those who know they need it di= sabled can make sure it is off from day one and not have a rude surprise th= e first time it runs (35 days after installation). > I don't think it was mentioned on this list, but that seems to be the current direction this is heading. Providing an option in the installer, whether enabled by default or not, should address everyone's concerns. --=20 Ryan Moeller iXsystems, Inc. OS Developer Email: ryan@iXsystems.com From owner-freebsd-fs@freebsd.org Wed Aug 5 04:36:29 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9C0AD3AA078 for ; Wed, 5 Aug 2020 04:36:29 +0000 (UTC) (envelope-from grant@gray.id.au) Received: from mail.agc.net.au (mail.agc.net.au [167.86.119.123]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BLzNC0HTdz4Hrm for ; Wed, 5 Aug 2020 04:36:25 +0000 (UTC) (envelope-from grant@gray.id.au) Received: from localhost (localhost [127.0.0.1]) by mail.agc.net.au (Postfix) with ESMTP id 2F01118A5BC for ; Wed, 5 Aug 2020 14:35:49 +1000 (AEST) Received: from mail.agc.net.au ([127.0.0.1]) by localhost (mail.agc.net.au [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id D5fwG32m18sC for ; Wed, 5 Aug 2020 14:35:30 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by mail.agc.net.au (Postfix) with ESMTP id BF38318A4DD for ; Wed, 5 Aug 2020 14:35:29 +1000 (AEST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.agc.net.au BF38318A4DD X-Virus-Scanned: amavisd-new at mail.agc.net.au Received: from mail.agc.net.au ([127.0.0.1]) by localhost (mail.agc.net.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id Tb7rvuHfROVk for ; Wed, 5 Aug 2020 14:35:28 +1000 (AEST) Received: from mail.agc.net.au (mail.agc.net.au [167.86.119.123]) by mail.agc.net.au (Postfix) with ESMTP id D05AE18A4D3 for ; Wed, 5 Aug 2020 14:35:25 +1000 (AEST) Date: Wed, 5 Aug 2020 14:35:20 +1000 (AEST) From: Grant Gray To: freebsd-fs Message-ID: <1647249742.296540.1596602120083.JavaMail.zimbra@gray.id.au> Subject: Re: zfs scrub enable by default MIME-Version: 1.0 X-Originating-IP: [167.86.119.123] X-Mailer: Zimbra 8.8.15_GA_3945 (ZimbraWebClient - FF80 (Linux)/8.8.15_GA_3953) Thread-Index: 9N3dwL/pDlx/Oe/PmMJI7K8/UIgGHw== Thread-Topic: zfs scrub enable by default X-Rspamd-Queue-Id: 4BLzNC0HTdz4Hrm X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.15 / 15.00]; FAKE_REPLY(1.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_FIVE(0.00)[6]; HAS_XOIP(0.00)[]; FROM_HAS_DN(0.00)[]; R_DKIM_ALLOW(-0.20)[gray.id.au:s=30D3F2DC-EB1E-11E9-943D-4BC9C5489560]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; NEURAL_HAM_LONG(-1.01)[-1.014]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_MEDIUM(-0.98)[-0.980]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gray.id.au:+]; DMARC_POLICY_ALLOW(-0.50)[gray.id.au,quarantine]; NEURAL_HAM_SHORT(-0.15)[-0.153]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:51167, ipnet:167.86.118.0/23, country:DE]; MID_RHS_MATCH_FROM(0.00)[] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2020 04:36:29 -0000 On 5/8/20 1:36 pm, PK1048.COM wrote: > On Aug 4, 2020, at 8:18 PM, Grant Gray via freebsd-fs wrote: > >> 2. That scrubbing PREVENTS data loss. Scrubbing can only tell you about data loss AFTER it happens. Yes, it could alert you to a problem that prevents further data loss, but it's already too late. Scrubbing is not a substitute for RAID, backups and proactive SMART testing. > Scrubbing does not directly prevent data loss. Scrubbing identifies data returned from the underlying device as corrupt or incorrect. If the zpool has sufficient redundancy, then no data is lost. > > But, normal ZFS operation also identifies incorrect date returned from the underlying device(s). You've taken me out of context; I was replying to an implication that scrubbing helps prevent data loss. It obviously doesn't because disks don't care about your arbitrary scrub interval and will fail as they see fit. Your tolerance to their bit destroying whims is determined, as you rightly point out, by the level of redundancy at play. If you have no redundancy, scrubbing only tells you that you are already screwed. If you do have redundancy, scrubbing tells you how many N's you have left. If you are out of N's, see above. But this isn't a discussion about the value of scrubbing (my apologies to all for indulging in the digression). That was never in question. Regardless of if/how much redundancy we have, we should scrub within an interval that makes sense in the context of our backup strategy to ensure there is always one known-good copy of the data. > While I have only managed a few hundred drives under ZFS, I cannot recall one case where a proactive scrub found bad data _before_ normal operations. Once a checksum error is reported via `zpool status` then I have found a scrub useful to determine the extent of the drive failure. > > I know there are others with thousands of drives managed under ZFS and their experience may differ. > > In terms of whether the periodic scrub should be enabled by default, I am undecided, as I can see both sides of the argument. I would prefer to make it a user choice during installation as one of the ZFS options (such as native 4K drives, ashift=12 or mirrored swap). Make the default enabled, but by putting it in the installation options, those who know they need it disabled can make sure it is off from day one and not have a rude surprise the first time it runs (35 days after installation). > The suggestion of making periodic scrub an install-time option came up earlier in the thread and seems like a good compromise, though I'm not convinced it should be enabled by default because, as I started previously, not scrubbing does not produce unexpected behaviour, whereas scrubbing can. From owner-freebsd-fs@freebsd.org Wed Aug 5 13:15:52 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D2040376564 for ; Wed, 5 Aug 2020 13:15:52 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from smtp.simplesystems.org (smtp.simplesystems.org [65.66.246.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BMBvW2q8Nz4pvc for ; Wed, 5 Aug 2020 13:15:51 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from scrappy.simplesystems.org (scrappy.simplesystems.org [65.66.246.73]) by smtp.simplesystems.org (8.14.4+Sun/8.14.4) with ESMTP id 075DFisK000207; Wed, 5 Aug 2020 08:15:44 -0500 (CDT) Date: Wed, 5 Aug 2020 08:15:44 -0500 (CDT) From: Bob Friesenhahn X-X-Sender: bfriesen@scrappy.simplesystems.org To: Karl Denninger cc: freebsd-fs@freebsd.org Subject: Re: zfs scrub enable by default In-Reply-To: Message-ID: References: <105090343.294898.1596586694925.JavaMail.zimbra@gray.id.au> User-Agent: Alpine 2.20 (GSO 67 2015-01-07) MIME-Version: 1.0 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (smtp.simplesystems.org [65.66.246.90]); Wed, 05 Aug 2020 08:15:44 -0500 (CDT) X-Rspamd-Queue-Id: 4BMBvW2q8Nz4pvc X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of bfriesen@simple.dallas.tx.us designates 65.66.246.90 as permitted sender) smtp.mailfrom=bfriesen@simple.dallas.tx.us X-Spamd-Result: default: False [-1.57 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.003]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx:c]; NEURAL_HAM_LONG(-1.05)[-1.053]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[dallas.tx.us]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.21)[-0.210]; CTYPE_MIXED_BOGUS(1.00)[]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:7018, ipnet:65.64.0.0/13, country:US]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2020 13:15:52 -0000 On Tue, 4 Aug 2020, Karl Denninger wrote: > Let me give you two allegedly "degenerate" cases that are actually not > degenerate at all. > > 1. A laptop or workstation.  It is backed up.  It uses ZFS because it's > faster, and I can establish a filesystem for some project very easily and > quickly, it's segregated, I can work on it and destroy it trivially when > done.  I can set quotas on that, etc.  If I want to move its mountpoint, I > can trivially do so. And so on.  Note that here there is no redundancy at > all; no raidZx, no mirroring, etc.  I'm merely using it for convenience. Did you remember to set copies=2 or copies=3 for zfs filesystems where you hope not to experience data loss? It needs to be set as soon as possible since it only applies to new files. This is a way to get more media redundancy, although the whole drive may fail. Zfs scrub is not going to protect your precious data from loss given just one drive unless you increase the copies setting. Zfs itself already uses redundant copies for its own data structures. Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt From owner-freebsd-fs@freebsd.org Wed Aug 5 13:32:43 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D3774376A81 for ; Wed, 5 Aug 2020 13:32:43 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-f174.google.com (mail-oi1-f174.google.com [209.85.167.174]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BMCGz0XJhz4qmy for ; Wed, 5 Aug 2020 13:32:42 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi1-f174.google.com with SMTP id b22so13638293oic.8 for ; Wed, 05 Aug 2020 06:32:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Q/ZYvV2y8Fp05xtKOzSjycbgjSjnxNQy/0f5JPOsB50=; b=uYLoyHHI5YkcSVFXrpoMh3Y0cYZ6Xzo/oPrgqBXzxybXpUesVw40mRfjtbX8g/Yw6a E2YNNCfO+pTF4RZGsWoDkNjB4aDhxbKQigBGjsI4Ci4d/MfnS2fJZLv9GK1ZcNLv5od3 ZUs+sEd/oGKEVsPfpc0o2nX5vDnDCmFGAfRnDoGx61wOzKse++rIc/bdE9uI7DBOqxdW NFjW0AioKfCXUt1fVuQftvY5iOJuXtuzKuggAT1HKl6BSHvpsHeV3WWNwpX/80ukXrzf UHHTmhztphCJTBRvG/7xUvQ3ffb8Gg0yWzyulgsVRzOD66HLsx+jzt50RqopriJNnzhb 63fQ== X-Gm-Message-State: AOAM53113pqNqeliWCVpoFQQ1RwCzxZbVAUC2LIYWvgNhDD+kpnkRBMo rB902rGz5xAIO8vaG2GPQeVMyEsClTeQ46tEeQo= X-Google-Smtp-Source: ABdhPJwvXTQu6euZ12lszzAY00CYwMA152NcSYFsWqPPA9ULRI912BPW7/RuQe4AANOam8GEY6iYBT3Fkgy36+lYte0= X-Received: by 2002:aca:b884:: with SMTP id i126mr2827415oif.57.1596634361582; Wed, 05 Aug 2020 06:32:41 -0700 (PDT) MIME-Version: 1.0 References: <105090343.294898.1596586694925.JavaMail.zimbra@gray.id.au> In-Reply-To: From: Alan Somers Date: Wed, 5 Aug 2020 07:32:29 -0600 Message-ID: Subject: Re: zfs scrub enable by default To: Bob Friesenhahn Cc: Karl Denninger , freebsd-fs X-Rspamd-Queue-Id: 4BMCGz0XJhz4qmy X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.167.174 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-1.70 / 15.00]; RCVD_TLS_ALL(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-0.95)[-0.953]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; NEURAL_SPAM_SHORT(0.15)[0.148]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[209.85.167.174:from]; NEURAL_HAM_MEDIUM(-0.89)[-0.893]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.167.174:from]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_TWO(0.00)[2]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2020 13:32:43 -0000 On Wed, Aug 5, 2020 at 7:15 AM Bob Friesenhahn wrote: > On Tue, 4 Aug 2020, Karl Denninger wrote: > > > Let me give you two allegedly "degenerate" cases that are actually not > > degenerate at all. > > > > 1. A laptop or workstation. It is backed up. It uses ZFS because it's > > faster, and I can establish a filesystem for some project very easily > and > > quickly, it's segregated, I can work on it and destroy it trivially when > > done. I can set quotas on that, etc. If I want to move its mountpoint, > I > > can trivially do so. And so on. Note that here there is no redundancy > at > > all; no raidZx, no mirroring, etc. I'm merely using it for convenience. > > Did you remember to set copies=2 or copies=3 for zfs filesystems where > you hope not to experience data loss? It needs to be set as soon as > possible since it only applies to new files. This is a way to get > more media redundancy, although the whole drive may fail. > > Zfs scrub is not going to protect your precious data from loss given > just one drive unless you increase the copies setting. Zfs itself > already uses redundant copies for its own data structures. > -1. In my experience, based on many thousands of drives, a whole drive failure is more likely than the failure or silent corruption of a few sectors. The ZFS copies setting really isn't very useful with modern HDDs. -Alan From owner-freebsd-fs@freebsd.org Wed Aug 5 13:48:12 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id ACC51376A67 for ; Wed, 5 Aug 2020 13:48:12 +0000 (UTC) (envelope-from karl@denninger.net) Received: from colo1.denninger.net (colo1.denninger.net [104.236.120.189]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BMCcq5grbz4rPn for ; Wed, 5 Aug 2020 13:48:11 +0000 (UTC) (envelope-from karl@denninger.net) Received: from denninger.net (096-033-205-208.res.spectrum.com [96.33.205.208]) by colo1.denninger.net (Postfix) with ESMTP id A0CB02110B5; Wed, 5 Aug 2020 09:47:39 -0400 (EDT) Received: from [192.168.10.25] (D15.Denninger.Net [192.168.10.25]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by denninger.net (Postfix) with ESMTPSA id 491B61D89EF; Wed, 5 Aug 2020 09:47:40 -0400 (EDT) Subject: Re: zfs scrub enable by default To: Bob Friesenhahn Cc: freebsd-fs@freebsd.org References: <105090343.294898.1596586694925.JavaMail.zimbra@gray.id.au> From: Karl Denninger Message-ID: <180eeb29-3cd8-efbd-41fe-97df8fd78ccc@denninger.net> Date: Wed, 5 Aug 2020 09:47:40 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Thunderbird/78.1.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms000209070104010805030705" X-Rspamd-Queue-Id: 4BMCcq5grbz4rPn X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=denninger.net; spf=pass (mx1.freebsd.org: domain of karl@denninger.net designates 104.236.120.189 as permitted sender) smtp.mailfrom=karl@denninger.net X-Spamd-Result: default: False [-5.62 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.02)[-1.024]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; SIGNED_SMIME(-2.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/alternative,text/plain]; HAS_ATTACHMENT(0.00)[]; NEURAL_HAM_LONG(-1.03)[-1.034]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.66)[-0.660]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[denninger.net,none]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~,4:~]; ASN(0.00)[asn:14061, ipnet:104.236.64.0/18, country:US]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2020 13:48:12 -0000 This is a cryptographically signed message in MIME format. --------------ms000209070104010805030705 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable On 8/5/2020 09:15, Bob Friesenhahn wrote: > On Tue, 4 Aug 2020, Karl Denninger wrote: > >> Let me give you two allegedly "degenerate" cases that are actually=20 >> not degenerate at all. >> >> 1. A laptop or workstation.=C2=A0 It is backed up.=C2=A0 It uses ZFS b= ecause=20 >> it's faster, and I can establish a filesystem for some project very=20 >> easily and quickly, it's segregated, I can work on it and destroy it=20 >> trivially when done.=C2=A0 I can set quotas on that, etc.=C2=A0 If I w= ant to=20 >> move its mountpoint, I can trivially do so. And so on.=C2=A0 Note that= =20 >> here there is no redundancy at all; no raidZx, no mirroring, etc.=C2=A0= =20 >> I'm merely using it for convenience. > > Did you remember to set copies=3D2 or copies=3D3 for zfs filesystems wh= ere=20 > you hope not to experience data loss?=C2=A0 It needs to be set as soon = as=20 > possible since it only applies to new files.=C2=A0 This is a way to get= =20 > more media redundancy, although the whole drive may fail. > > Zfs scrub is not going to protect your precious data from loss given=20 > just one drive unless you increase the copies setting.=C2=A0 Zfs itself= =20 > already uses redundant copies for its own data structures. > copies=3D2 does nothing if the drive itself fails (as opposed to a handfu= l=20 of sectors failing on said drive.) SSDs rarely lose a block undetected by their own controller.=20 More-commonly when they fail they fail HARD and the entire volume=20 becomes inaccessible immediately.=C2=A0 The same is frequently (but less = frequently than with an SSD) true for spinning rust.=C2=A0 In my experien= ce=20 with various "non-removable" spinning rust going back to the original=20 5Mb winchester devices in a decent percentage of the instances the=20 device becomes either completely inaccessible all at once or it may as=20 well be since the unreadable sectors are numerous enough and the=20 timeouts on attempted reads long enough that recovery of the=20 non-impacted data will frequently take days (or worse.) Don't even get me started on controller/adapter failures.=C2=A0 I've had = two=20 in my career that scribbled on everything connected to them at once.=C2=A0= =20 Even ZFS in a RaidZx or mirrored configuration won't protect you from=20 that.=C2=A0 You either have backups or you have nothing. As I said, a workstation or laptop is, in such a configuration, backed=20 up to some other storage (external, either plugged in or on a network)=20 and if the drive fails it is replaced and restored. ZFS makes this easy=20 to do and also quite economical in terms of time and either storage or=20 network bandwidth with its snapshot capability. The latter is also a reason to use ZFS standing alone, irrespective of=20 redundancy of data storage. I've had scrub catch incipient errors (but you have to look at the stats = to know it found and fixed the blocks) before the volume failed and was=20 able to replace the bad volume before it puked entirely.=C2=A0 I apprecia= te=20 that when it happens but the point here is that turning on automated=20 scrubs is like automated patrol reads on an old-style RAID adapter; it=20 sounds great right up until it can cause trouble without benefit.=C2=A0 T= o=20 have it as an option in the installer is a good thing, but to silently=20 turn it on, especially if it's done during an upgrade where it wasn't on = before, is IMHO a bad thing as it has at least a decent probability of=20 producing unpleasant surprises. --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms000209070104010805030705 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC DdgwggagMIIEiKADAgECAhMA5EiKghDOXrvfxYxjITXYDdhIMA0GCSqGSIb3DQEBCwUAMIGL MQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJTmljZXZpbGxlMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExITAf BgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQTAeFw0xNzA4MTcxNjQyMTdaFw0yNzA4 MTUxNjQyMTdaMHsxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkwFwYDVQQKDBBD dWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5c3RlbXMgQ0ExJTAjBgNVBAMMHEN1 ZGEgU3lzdGVtcyBMTEMgMjAxNyBJbnQgQ0EwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIK AoICAQC1aJotNUI+W4jP7xQDO8L/b4XiF4Rss9O0B+3vMH7Njk85fZ052QhZpMVlpaaO+sCI KqG3oNEbuOHzJB/NDJFnqh7ijBwhdWutdsq23Ux6TvxgakyMPpT6TRNEJzcBVQA0kpby1DVD 0EKSK/FrWWBiFmSxg7qUfmIq/mMzgE6epHktyRM3OGq3dbRdOUgfumWrqHXOrdJz06xE9NzY vc9toqZnd79FUtE/nSZVm1VS3Grq7RKV65onvX3QOW4W1ldEHwggaZxgWGNiR/D4eosAGFxn uYeWlKEC70c99Mp1giWux+7ur6hc2E+AaTGh+fGeijO5q40OGd+dNMgK8Es0nDRw81lRcl24 SWUEky9y8DArgIFlRd6d3ZYwgc1DMTWkTavx3ZpASp5TWih6yI8ACwboTvlUYeooMsPtNa9E 6UQ1nt7VEi5syjxnDltbEFoLYcXBcqhRhFETJe9CdenItAHAtOya3w5+fmC2j/xJz29og1KH YqWHlo3Kswi9G77an+zh6nWkMuHs+03DU8DaOEWzZEav3lVD4u76bKRDTbhh0bMAk4eXriGL h4MUoX3Imfcr6JoyheVrAdHDL/BixbMH1UUspeRuqQMQ5b2T6pabXP0oOB4FqldWiDgJBGRd zWLgCYG8wPGJGYgHibl5rFiI5Ix3FQncipc6SdUzOQIDAQABo4IBCjCCAQYwHQYDVR0OBBYE FF3AXsKnjdPND5+bxVECGKtc047PMIHABgNVHSMEgbgwgbWAFBu1oRhUMNEzjODolDka5k4Q EDBioYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UEBwwJ TmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRhIFN5 c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYIJAKxAy1WBo2kY MBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgGGMA0GCSqGSIb3DQEBCwUAA4IC AQCB5686UCBVIT52jO3sz9pKuhxuC2npi8ZvoBwt/IH9piPA15/CGF1XeXUdu2qmhOjHkVLN gO7XB1G8CuluxofOIUce0aZGyB+vZ1ylHXlMeB0R82f5dz3/T7RQso55Y2Vog2Zb7PYTC5B9 oNy3ylsnNLzanYlcW3AAfzZcbxYuAdnuq0Im3EpGm8DoItUcf1pDezugKm/yKtNtY6sDyENj tExZ377cYA3IdIwqn1Mh4OAT/Rmh8au2rZAo0+bMYBy9C11Ex0hQ8zWcvPZBDn4v4RtO8g+K uQZQcJnO09LJNtw94W3d2mj4a7XrsKMnZKvm6W9BJIQ4Nmht4wXAtPQ1xA+QpxPTmsGAU0Cv HmqVC7XC3qxFhaOrD2dsvOAK6Sn3MEpH/YrfYCX7a7cz5zW3DsJQ6o3pYfnnQz+hnwLlz4MK 17NIA0WOdAF9IbtQqarf44+PEyUbKtz1r0KGeGLs+VGdd2FLA0e7yuzxJDYcaBTVwqaHhU2/ Fna/jGU7BhrKHtJbb/XlLeFJ24yvuiYKpYWQSSyZu1R/gvZjHeGb344jGBsZdCDrdxtQQcVA 6OxsMAPSUPMrlg9LWELEEYnVulQJerWxpUecGH92O06wwmPgykkz//UmmgjVSh7ErNvL0lUY UMfunYVO/O5hwhW+P4gviCXzBFeTtDZH259O7TCCBzAwggUYoAMCAQICEwCg0WvVwekjGFiO 62SckFwepz0wDQYJKoZIhvcNAQELBQAwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3Jp ZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBD QTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExMQyAyMDE3IEludCBDQTAeFw0xNzA4MTcyMTIx MjBaFw0yMjA4MTYyMTIxMjBaMFcxCzAJBgNVBAYTAlVTMRAwDgYDVQQIDAdGbG9yaWRhMRkw FwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRswGQYDVQQDDBJrYXJsQGRlbm5pbmdlci5uZXQw ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQC+HVSyxVtJhy3Ohs+PAGRuO//Dha9A 16l5FPATr6wude9zjX5f2lrkRyU8vhCXTZW7WbvWZKpcZ8r0dtZmiK9uF58Ec6hhvfkxJzbg 96WHBw5Fumd5ahZzuCJDtCAWW8R7/KN+zwzQf1+B3MVLmbaXAFBuKzySKhKMcHbK3/wjUYTg y+3UK6v2SBrowvkUBC+jxNg3Wy12GsTXcUS/8FYIXgVVPgfZZrbJJb5HWOQpvvhILpPCD3xs YJFNKEPltXKWHT7Qtc2HNqikgNwj8oqOb+PeZGMiWapsatKm8mxuOOGOEBhAoTVTwUHlMNTg 6QUCJtuWFCK38qOCyk9Haj+86lUU8RG6FkRXWgMbNQm1mWREQhw3axgGLSntjjnznJr5vsvX SYR6c+XKLd5KQZcS6LL8FHYNjqVKHBYM+hDnrTZMqa20JLAF1YagutDiMRURU23iWS7bA9tM cXcqkclTSDtFtxahRifXRI7Epq2GSKuEXe/1Tfb5CE8QsbCpGsfSwv2tZ/SpqVG08MdRiXxN 5tmZiQWo15IyWoeKOXl/hKxA9KPuDHngXX022b1ly+5ZOZbxBAZZMod4y4b4FiRUhRI97r9l CxsP/EPHuuTIZ82BYhrhbtab8HuRo2ofne2TfAWY2BlA7ExM8XShMd9bRPZrNTokPQPUCWCg CdIATQIDAQABo4IBzzCCAcswPAYIKwYBBQUHAQEEMDAuMCwGCCsGAQUFBzABhiBodHRwOi8v b2NzcC5jdWRhc3lzdGVtcy5uZXQ6ODg4ODAJBgNVHRMEAjAAMBEGCWCGSAGG+EIBAQQEAwIF oDAOBgNVHQ8BAf8EBAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMEMDMGCWCG SAGG+EIBDQQmFiRPcGVuU1NMIEdlbmVyYXRlZCBDbGllbnQgQ2VydGlmaWNhdGUwHQYDVR0O BBYEFLElmNWeVgsBPe7O8NiBzjvjYnpRMIHKBgNVHSMEgcIwgb+AFF3AXsKnjdPND5+bxVEC GKtc047PoYGRpIGOMIGLMQswCQYDVQQGEwJVUzEQMA4GA1UECAwHRmxvcmlkYTESMBAGA1UE BwwJTmljZXZpbGxlMRkwFwYDVQQKDBBDdWRhIFN5c3RlbXMgTExDMRgwFgYDVQQLDA9DdWRh IFN5c3RlbXMgQ0ExITAfBgNVBAMMGEN1ZGEgU3lzdGVtcyBMTEMgMjAxNyBDQYITAORIioIQ zl6738WMYyE12A3YSDAdBgNVHREEFjAUgRJrYXJsQGRlbm5pbmdlci5uZXQwDQYJKoZIhvcN AQELBQADggIBAJXboPFBMLMtaiUt4KEtJCXlHO/3ZzIUIw/eobWFMdhe7M4+0u3te0sr77QR dcPKR0UeHffvpth2Mb3h28WfN0FmJmLwJk+pOx4u6uO3O0E1jNXoKh8fVcL4KU79oEQyYkbu 2HwbXBU9HbldPOOZDnPLi0whi/sbFHdyd4/w/NmnPgzAsQNZ2BYT9uBNr+jZw4SsluQzXG1X lFL/qCBoi1N2mqKPIepfGYF6drbr1RnXEJJsuD+NILLooTNf7PMgHPZ4VSWQXLNeFfygoOOK FiO0qfxPKpDMA+FHa8yNjAJZAgdJX5Mm1kbqipvb+r/H1UAmrzGMbhmf1gConsT5f8KU4n3Q IM2sOpTQe7BoVKlQM/fpQi6aBzu67M1iF1WtODpa5QUPvj1etaK+R3eYBzi4DIbCIWst8MdA 1+fEeKJFvMEZQONpkCwrJ+tJEuGQmjoQZgK1HeloepF0WDcviiho5FlgtAij+iBPtwMuuLiL shAXA5afMX1hYM4l11JXntle12EQFP1r6wOUkpOdxceCcMVDEJBBCHW2ZmdEaXgAm1VU+fnQ qS/wNw/S0X3RJT1qjr5uVlp2Y0auG/eG0jy6TT0KzTJeR9tLSDXprYkN2l/Qf7/nT6Q03qyE QnnKiBXWAZXveafyU/zYa7t3PTWFQGgWoC4w6XqgPo4KV44OMYIFBzCCBQMCAQEwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBglghkgBZQMEAgMFAKCCAkUw GAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMjAwODA1MTM0NzQx WjBPBgkqhkiG9w0BCQQxQgRABj3x9JNkeILUUoSCCd5G7YZceSZ68Hy15dq6P9gZXssGCNld mAq3fUptJNscx0FT4I9tav1NGfK5GuV/ggsC3zBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFl AwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3 DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGjBgkrBgEEAYI3EAQxgZUwgZIwezEL MAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lzdGVtcyBM TEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0ZW1zIExM QyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTCBpQYLKoZIhvcNAQkQAgsxgZWg gZIwezELMAkGA1UEBhMCVVMxEDAOBgNVBAgMB0Zsb3JpZGExGTAXBgNVBAoMEEN1ZGEgU3lz dGVtcyBMTEMxGDAWBgNVBAsMD0N1ZGEgU3lzdGVtcyBDQTElMCMGA1UEAwwcQ3VkYSBTeXN0 ZW1zIExMQyAyMDE3IEludCBDQQITAKDRa9XB6SMYWI7rZJyQXB6nPTANBgkqhkiG9w0BAQEF AASCAgA00MTHTl6gy7xS6d8XS5rfcVPeJR6fZB/E4/nNZSheUDRS6Jd7CTLbwP/nVT7Fzbxi XKodKt1d+0zNHVo2rGwIqAqxKvJV6ikeVqZ5hVWKllWCCthiPZq6sTtXRBBNkDXtl6zH+78X W2Tvn/SEpbu3PrPeMwxgCx/qcRwrf8zgf9QekQvvesnUvheCk+NntCAkemLOqzZx0GZ3mvkr db8JUgWsnNR82oFF9T4AfbqLMG+4kaYUKOcgh/CtsDu877bC456BXK9SFGOapj+rU5NNugoz sAnxcXQ0169q334jOqALeRoIJ9E86/vLMiUyEE96EjmRf9Z7yHtthfQSiOUgdgXuT7xXYz3G abM/M+bCPlO69A4ITp5oIUIkW2TKL8barQ7RTs/EirDzKJuMl8PHxRrOwhNS7f0Cqik/l/aS yZ7MylY5YijV5niVhSLJoD1djrAqjaFkypk/fNsPeEQhcw4l4qdWvzQxB3SL/EEBhy8Iu9sR 3+YYy8p/F+wiTcZmPBEULFDSh9VSQpo009au9JVmlDvbTv8LUiYTV6e380Qdww0yD5PGRhAs wfb7haLSzBs6Vb8avK5UCTkzb/9ph+NUKM87jJZkgj+9l/lfryM7geK0DCAhHatsAjMM1Aty 5Z/uucVilbFWTtp7ulXmKVzwqosjWV5yqfahns+pCQAAAAAAAA== --------------ms000209070104010805030705-- From owner-freebsd-fs@freebsd.org Wed Aug 5 14:55:10 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A91A1378D55 for ; Wed, 5 Aug 2020 14:55:10 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from smtp.simplesystems.org (smtp.simplesystems.org [65.66.246.90]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BMF655VW5z3Rrd; Wed, 5 Aug 2020 14:55:09 +0000 (UTC) (envelope-from bfriesen@simple.dallas.tx.us) Received: from scrappy.simplesystems.org (scrappy.simplesystems.org [65.66.246.73]) by smtp.simplesystems.org (8.14.4+Sun/8.14.4) with ESMTP id 075Et644002897; Wed, 5 Aug 2020 09:55:07 -0500 (CDT) Date: Wed, 5 Aug 2020 09:55:06 -0500 (CDT) From: Bob Friesenhahn X-X-Sender: bfriesen@scrappy.simplesystems.org To: Alan Somers cc: freebsd-fs Subject: Re: zfs scrub enable by default In-Reply-To: Message-ID: References: <105090343.294898.1596586694925.JavaMail.zimbra@gray.id.au> User-Agent: Alpine 2.20 (GSO 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (smtp.simplesystems.org [65.66.246.90]); Wed, 05 Aug 2020 09:55:07 -0500 (CDT) X-Rspamd-Queue-Id: 4BMF655VW5z3Rrd X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of bfriesen@simple.dallas.tx.us designates 65.66.246.90 as permitted sender) smtp.mailfrom=bfriesen@simple.dallas.tx.us X-Spamd-Result: default: False [-2.51 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+mx:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[dallas.tx.us]; NEURAL_HAM_LONG(-1.03)[-1.031]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.18)[-0.184]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:7018, ipnet:65.64.0.0/13, country:US]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2020 14:55:10 -0000 On Wed, 5 Aug 2020, Alan Somers wrote: >> >> Zfs scrub is not going to protect your precious data from loss given >> just one drive unless you increase the copies setting. Zfs itself >> already uses redundant copies for its own data structures. > > -1. In my experience, based on many thousands of drives, a whole drive > failure is more likely than the failure or silent corruption of a few > sectors. The ZFS copies setting really isn't very useful with modern HDDs. This is something that I totally agree with. In this case a scrub will not help all that much other than make your 'bad day' occur earlier if the scrub pushes the device over the brink due to the increased loading. Scrub is more likely to successfully correct issues given sufficient physical redundancy but finding the issues right away is less important given sufficient physical redundancy since issues will be automatically corrected upon data access. Manual scrub of a new pool (after copying in the data) is a very wise idea in order to uncover system-level issues. Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt From owner-freebsd-fs@freebsd.org Wed Aug 5 15:22:05 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3728537986E for ; Wed, 5 Aug 2020 15:22:05 +0000 (UTC) (envelope-from codeblue@inbox.lv) Received: from shark4.inbox.lv (shark4.inbox.lv [194.152.32.84]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BMFj8042Tz3TV9 for ; Wed, 5 Aug 2020 15:22:03 +0000 (UTC) (envelope-from codeblue@inbox.lv) Received: from shark4.inbox.lv (localhost [127.0.0.1]) by shark4-out.inbox.lv (Postfix) with ESMTP id 007C86110F for ; Wed, 5 Aug 2020 18:22:01 +0300 (EEST) Received: from localhost (localhost [127.0.0.1]) by shark4-in.inbox.lv (Postfix) with ESMTP id E98766110D for ; Wed, 5 Aug 2020 18:22:00 +0300 (EEST) Received: from shark4.inbox.lv ([127.0.0.1]) by localhost (shark4.inbox.lv [127.0.0.1]) (spamfilter, port 35) with ESMTP id 1gja5zpP9Jgx for ; Wed, 5 Aug 2020 18:22:00 +0300 (EEST) Received: from mail.inbox.lv (pop1 [127.0.0.1]) by shark4-in.inbox.lv (Postfix) with ESMTP id 992E56110B for ; Wed, 5 Aug 2020 18:22:00 +0300 (EEST) Received: from [10.0.0.2] (bzq-109-65-93-146.red.bezeqint.net [109.65.93.146]) (Authenticated sender: codeblue@inbox.lv) by mail.inbox.lv (Postfix) with ESMTPA id 409A13D328B for ; Wed, 5 Aug 2020 18:22:00 +0300 (EEST) Subject: Re: zfs scrub enable by default To: freebsd-fs@freebsd.org References: <105090343.294898.1596586694925.JavaMail.zimbra@gray.id.au> From: John Long Message-ID: <1327e123-35df-1e27-af7a-7225dae91a21@inbox.lv> Date: Wed, 5 Aug 2020 15:21:57 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Virus-Scanned: OK X-ESPOL: EZeEAiZdhRs1tcGiUuRh5uPt2NC2SFMgoyb+yKFSnX9YtbfBs9x0c2eSB/ecFHvFbg== X-Rspamd-Queue-Id: 4BMFj8042Tz3TV9 X-Spamd-Bar: ----- X-Spamd-Result: default: False [-5.27 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_FIVE(0.00)[6]; RECEIVED_SPAMHAUS_PBL(0.00)[109.65.93.146:received]; FROM_HAS_DN(0.00)[]; R_DKIM_ALLOW(-0.20)[inbox.lv:s=30062014]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:194.152.32.84]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[inbox.lv:dkim]; NEURAL_HAM_MEDIUM(-1.01)[-1.008]; DKIM_TRACE(0.00)[inbox.lv:+]; DMARC_POLICY_ALLOW(-0.50)[inbox.lv,quarantine]; NEURAL_HAM_SHORT(-1.14)[-1.142]; NEURAL_HAM_LONG(-1.02)[-1.017]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:12993, ipnet:194.152.32.0/23, country:LV]; MID_RHS_MATCH_FROM(0.00)[]; RWL_MAILSPIKE_VERYGOOD(0.00)[194.152.32.84:from]; RCVD_IN_DNSWL_LOW(-0.10)[194.152.32.84:from] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2020 15:22:05 -0000 On 05/08/2020 13:15, Bob Friesenhahn wrote: > On Tue, 4 Aug 2020, Karl Denninger wrote: > >> Let me give you two allegedly "degenerate" cases that are actually not >> degenerate at all. >> >> 1. A laptop or workstation.  It is backed up.  It uses ZFS because >> it's faster, and I can establish a filesystem for some project very >> easily and quickly, it's segregated, I can work on it and destroy it >> trivially when done.  I can set quotas on that, etc.  If I want to >> move its mountpoint, I can trivially do so. And so on.  Note that here >> there is no redundancy at all; no raidZx, no mirroring, etc.  I'm >> merely using it for convenience. > > Did you remember to set copies=2 or copies=3 for zfs filesystems where > you hope not to experience data loss?  It needs to be set as soon as > possible since it only applies to new files.  This is a way to get more > media redundancy, although the whole drive may fail. Does copies=n actually create n-1 additional physical copies or is it copy-on-write, or something else yet? /jl From owner-freebsd-fs@freebsd.org Wed Aug 5 15:24:55 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6AECB3796E9 for ; Wed, 5 Aug 2020 15:24:55 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-ot1-f42.google.com (mail-ot1-f42.google.com [209.85.210.42]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BMFmP3bbkz3Tq9 for ; Wed, 5 Aug 2020 15:24:53 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-ot1-f42.google.com with SMTP id e11so9427012otk.4 for ; Wed, 05 Aug 2020 08:24:53 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=LsNlRDigcehWOqkeXXU4nY+Ib7F7NMRcU7FlgYymE40=; b=s331bI2jmmcZfnavzKARASKej7BEibz5ss9aKkppSm5qa3AZdykAreyMEyGb0PT9i1 1Vg9GfyHKRTUcmA8o1DpaJhmZKjfnu6aWoRemT2oKG4Uravp/KI6Dc35c46PBOpyR5+l DBzthtFB8c4USfY90LQd34Y3BEr46U1KokKX58xQD4wR/DL8IIhKEeMyWs27toZL3GFy 5E6FKeFAM7DvrDBTdmfh3z45q4uFVu9K0qJxdI4RK8RxgZD8xSjBeNgdc1aPoQcUvlMR Kh4vMTatQSUKt8p2VyZHavz0E+JkWWY0KX5Sn6NW3JgWgSYGdWOVOvxJTjsIKJ06rmiu pEKg== X-Gm-Message-State: AOAM530ELz29yAtJXptd2n/BfxdLG8UM+hi06vAki6/X5MmzNEmI4NcW 7VvRRR0Ra/E7Um1jAtXrOfWYXrOMjZUcdU1JkZI= X-Google-Smtp-Source: ABdhPJxBAYD7yIGtBalWpXcwlnPtOvCMmR9C0/6WxQpopT3O8JnwFWGTeDsJFnEf4WImTAOyPW07ri9PTjsfqQwxALQ= X-Received: by 2002:a9d:4b82:: with SMTP id k2mr3249262otf.18.1596641092143; Wed, 05 Aug 2020 08:24:52 -0700 (PDT) MIME-Version: 1.0 References: <105090343.294898.1596586694925.JavaMail.zimbra@gray.id.au> <1327e123-35df-1e27-af7a-7225dae91a21@inbox.lv> In-Reply-To: <1327e123-35df-1e27-af7a-7225dae91a21@inbox.lv> From: Alan Somers Date: Wed, 5 Aug 2020 09:24:40 -0600 Message-ID: Subject: Re: zfs scrub enable by default To: John Long Cc: freebsd-fs X-Rspamd-Queue-Id: 4BMFmP3bbkz3Tq9 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.210.42 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-2.21 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.71)[-0.708]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[209.85.210.42:from]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; NEURAL_HAM_LONG(-1.03)[-1.032]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.47)[-0.472]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[209.85.210.42:from]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2020 15:24:55 -0000 On Wed, Aug 5, 2020 at 9:22 AM John Long via freebsd-fs < freebsd-fs@freebsd.org> wrote: > On 05/08/2020 13:15, Bob Friesenhahn wrote: > > On Tue, 4 Aug 2020, Karl Denninger wrote: > > > >> Let me give you two allegedly "degenerate" cases that are actually not > >> degenerate at all. > >> > >> 1. A laptop or workstation. It is backed up. It uses ZFS because > >> it's faster, and I can establish a filesystem for some project very > >> easily and quickly, it's segregated, I can work on it and destroy it > >> trivially when done. I can set quotas on that, etc. If I want to > >> move its mountpoint, I can trivially do so. And so on. Note that here > >> there is no redundancy at all; no raidZx, no mirroring, etc. I'm > >> merely using it for convenience. > > > > Did you remember to set copies=2 or copies=3 for zfs filesystems where > > you hope not to experience data loss? It needs to be set as soon as > > possible since it only applies to new files. This is a way to get more > > media redundancy, although the whole drive may fail. > > Does copies=n actually create n-1 additional physical copies or is it > copy-on-write, or something else yet? > > /jl > Yes, copies=3 will actually create 3 physical copies of the data somewhere. It's basically mirroring at the DMU layer, rather than the block layer. -Alan From owner-freebsd-fs@freebsd.org Wed Aug 5 16:06:38 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1CCA137A471 for ; Wed, 5 Aug 2020 16:06:38 +0000 (UTC) (envelope-from codeblue@inbox.lv) Received: from shark2.inbox.lv (shark2.inbox.lv [194.152.32.82]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BMGhY15qsz3WpM for ; Wed, 5 Aug 2020 16:06:36 +0000 (UTC) (envelope-from codeblue@inbox.lv) Received: from shark2.inbox.lv (localhost [127.0.0.1]) by shark2-out.inbox.lv (Postfix) with ESMTP id DC5EA3BFE6 for ; Wed, 5 Aug 2020 19:06:33 +0300 (EEST) Received: from localhost (localhost [127.0.0.1]) by shark2-in.inbox.lv (Postfix) with ESMTP id CF1233BFCA for ; Wed, 5 Aug 2020 19:06:33 +0300 (EEST) Received: from shark2.inbox.lv ([127.0.0.1]) by localhost (shark2.inbox.lv [127.0.0.1]) (spamfilter, port 35) with ESMTP id 0-3T6FQcz3VB for ; Wed, 5 Aug 2020 19:06:33 +0300 (EEST) Received: from mail.inbox.lv (pop1 [127.0.0.1]) by shark2-in.inbox.lv (Postfix) with ESMTP id 7D97A3BE3B for ; Wed, 5 Aug 2020 19:06:33 +0300 (EEST) Received: from [10.0.0.2] (bzq-109-65-93-146.red.bezeqint.net [109.65.93.146]) (Authenticated sender: codeblue@inbox.lv) by mail.inbox.lv (Postfix) with ESMTPA id 1C7303D980D for ; Wed, 5 Aug 2020 19:06:32 +0300 (EEST) Subject: Re: zfs scrub enable by default To: freebsd-fs References: <105090343.294898.1596586694925.JavaMail.zimbra@gray.id.au> <1327e123-35df-1e27-af7a-7225dae91a21@inbox.lv> From: John Long Message-ID: <71d447e6-3f18-319a-ec98-9e35feec3180@inbox.lv> Date: Wed, 5 Aug 2020 16:06:29 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Virus-Scanned: OK X-ESPOL: EZeEAiZdhRs1tcGiSfBh5uDgxtaxV1krujuJvbBJlW4g1r7As9drdmqLEofxGAa/bg== X-Rspamd-Queue-Id: 4BMGhY15qsz3WpM X-Spamd-Bar: ----- X-Spamd-Result: default: False [-5.19 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[inbox.lv:s=30062014]; RCVD_COUNT_FIVE(0.00)[6]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[194.152.32.82:from]; R_SPF_ALLOW(-0.20)[+ip4:194.152.32.82:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; NEURAL_HAM_LONG(-1.03)[-1.030]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[inbox.lv:dkim]; RECEIVED_SPAMHAUS_PBL(0.00)[109.65.93.146:received]; NEURAL_HAM_MEDIUM(-0.99)[-0.987]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[inbox.lv:+]; DMARC_POLICY_ALLOW(-0.50)[inbox.lv,quarantine]; NEURAL_HAM_SHORT(-1.07)[-1.068]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:12993, ipnet:194.152.32.0/23, country:LV]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[194.152.32.82:from] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2020 16:06:38 -0000 On 05/08/2020 15:24, Alan Somers wrote: > On Wed, Aug 5, 2020 at 9:22 AM John Long via freebsd-fs > > wrote: > > On 05/08/2020 13:15, Bob Friesenhahn wrote: > > On Tue, 4 Aug 2020, Karl Denninger wrote: > > > >> Let me give you two allegedly "degenerate" cases that are > actually not > >> degenerate at all. > >> > >> 1. A laptop or workstation.  It is backed up.  It uses ZFS because > >> it's faster, and I can establish a filesystem for some project very > >> easily and quickly, it's segregated, I can work on it and > destroy it > >> trivially when done.  I can set quotas on that, etc.  If I want to > >> move its mountpoint, I can trivially do so. And so on.  Note > that here > >> there is no redundancy at all; no raidZx, no mirroring, etc.  I'm > >> merely using it for convenience. > > > > Did you remember to set copies=2 or copies=3 for zfs filesystems > where > > you hope not to experience data loss?  It needs to be set as soon as > > possible since it only applies to new files.  This is a way to > get more > > media redundancy, although the whole drive may fail. > > Does copies=n actually create n-1 additional physical copies or is it > copy-on-write, or something else yet? > > /jl > > > Yes, copies=3 will actually create 3 physical copies of the data > somewhere.  It's basically mirroring at the DMU layer, rather than the > block layer. > -Alan Thanks, I figured that must be the case but I thought it was better ask. So is it correct that aside from a single disk vdev, it would be a bad practice to specify additional copies? How does dedup deal with it? /jl From owner-freebsd-fs@freebsd.org Wed Aug 5 16:13:31 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C08CA37AC27 for ; Wed, 5 Aug 2020 16:13:31 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-oi1-f180.google.com (mail-oi1-f180.google.com [209.85.167.180]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BMGrV74yxz3X6X for ; Wed, 5 Aug 2020 16:13:30 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-oi1-f180.google.com with SMTP id h3so16484786oie.11 for ; Wed, 05 Aug 2020 09:13:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=clBbvIjE7rNA1MHdcIN7CDO0UQSI19lFgmQhr3jycXc=; b=q4dxcoJEX/pz+tuvqSW8wiAxLnkRZRlYLtvRr99Z9byb+5lHVbfdv4t14GsFIuQL3H I0jfmnQ1FXFdtHZCbowJCejtCtt9jpJ/b04fxdQpBD31X3MjCxHecIVdfLrXd7xFnRSi oWS9XIxB8avzy98ZtbA/rpuZJ3ldHZ2Hfva3a7kARNjpxuZJwbIKD4RxO2uMhbsz0Sb/ UFKzOoUXnn8zk43HiEDhOxGdJSHnbTeSyRQeJ9Hw9fIHXIVXFQ3sW3EN4UGx6GvFySYj 96l6M24PewhnovHiI+iIsGiK6aiYQpsMX0eBH2IeucVpxvLvod3vihggV+rlDPqIzYKM wDjQ== X-Gm-Message-State: AOAM531HAHvWt1pKXsFZ37etMmDSx6ice+mjwHbF09rjWK9ZvUoaNewC LI3b2IS5ge0IPjNoOUBg1MWP8Bx3NQsAtTXGT3E= X-Google-Smtp-Source: ABdhPJymQ/71pmlP3dEKAhoizAXDvTdmbXvdbZoYra8ASawCnK2TBmR2pxTWqaQC15njpbeHzEHB+GhDpVSSyMLwYi0= X-Received: by 2002:aca:1c0c:: with SMTP id c12mr1772215oic.73.1596644009792; Wed, 05 Aug 2020 09:13:29 -0700 (PDT) MIME-Version: 1.0 References: <105090343.294898.1596586694925.JavaMail.zimbra@gray.id.au> <1327e123-35df-1e27-af7a-7225dae91a21@inbox.lv> <71d447e6-3f18-319a-ec98-9e35feec3180@inbox.lv> In-Reply-To: <71d447e6-3f18-319a-ec98-9e35feec3180@inbox.lv> From: Alan Somers Date: Wed, 5 Aug 2020 10:13:18 -0600 Message-ID: Subject: Re: zfs scrub enable by default To: John Long Cc: freebsd-fs X-Rspamd-Queue-Id: 4BMGrV74yxz3X6X X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of asomers@gmail.com designates 209.85.167.180 as permitted sender) smtp.mailfrom=asomers@gmail.com X-Spamd-Result: default: False [-2.21 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.63)[-0.632]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-0.98)[-0.981]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; DMARC_NA(0.00)[freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.59)[-0.593]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[209.85.167.180:from]; FORGED_SENDER(0.30)[asomers@freebsd.org,asomers@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.167.180:from]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[asomers@freebsd.org,asomers@gmail.com]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2020 16:13:31 -0000 On Wed, Aug 5, 2020 at 10:06 AM John Long via freebsd-fs < freebsd-fs@freebsd.org> wrote: > On 05/08/2020 15:24, Alan Somers wrote: > > On Wed, Aug 5, 2020 at 9:22 AM John Long via freebsd-fs > > > wrote: > > > > On 05/08/2020 13:15, Bob Friesenhahn wrote: > > > On Tue, 4 Aug 2020, Karl Denninger wrote: > > > > > >> Let me give you two allegedly "degenerate" cases that are > > actually not > > >> degenerate at all. > > >> > > >> 1. A laptop or workstation. It is backed up. It uses ZFS > because > > >> it's faster, and I can establish a filesystem for some project > very > > >> easily and quickly, it's segregated, I can work on it and > > destroy it > > >> trivially when done. I can set quotas on that, etc. If I want > to > > >> move its mountpoint, I can trivially do so. And so on. Note > > that here > > >> there is no redundancy at all; no raidZx, no mirroring, etc. I'm > > >> merely using it for convenience. > > > > > > Did you remember to set copies=2 or copies=3 for zfs filesystems > > where > > > you hope not to experience data loss? It needs to be set as soon > as > > > possible since it only applies to new files. This is a way to > > get more > > > media redundancy, although the whole drive may fail. > > > > Does copies=n actually create n-1 additional physical copies or is it > > copy-on-write, or something else yet? > > > > /jl > > > > > > Yes, copies=3 will actually create 3 physical copies of the data > > somewhere. It's basically mirroring at the DMU layer, rather than the > > block layer. > -Alan > > Thanks, I figured that must be the case but I thought it was better ask. > > So is it correct that aside from a single disk vdev, it would be a bad > practice to specify additional copies? How does dedup deal with it? > > /jl > Yeah. I wouldn't recommend ever chaning the copies value from the default. If dedup is on (also not recommended), then it wouldn't make sense to use multiple copies anyway. And if zfs encryption is on, then you can't use multiple copies, because encryption repurposes some of the block pointer's fields. -Alan From owner-freebsd-fs@freebsd.org Wed Aug 5 16:47:54 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5283B37B073 for ; Wed, 5 Aug 2020 16:47:54 +0000 (UTC) (envelope-from codeblue@inbox.lv) Received: from shark1.inbox.lv (shark1.inbox.lv [194.152.32.81]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BMHc900CMz3Z1k for ; Wed, 5 Aug 2020 16:47:52 +0000 (UTC) (envelope-from codeblue@inbox.lv) Received: from shark1.inbox.lv (localhost [127.0.0.1]) by shark1-out.inbox.lv (Postfix) with ESMTP id 0411411182B4 for ; Wed, 5 Aug 2020 19:47:50 +0300 (EEST) Received: from localhost (localhost [127.0.0.1]) by shark1-in.inbox.lv (Postfix) with ESMTP id F2FF611182B3 for ; Wed, 5 Aug 2020 19:47:49 +0300 (EEST) Received: from shark1.inbox.lv ([127.0.0.1]) by localhost (shark1.inbox.lv [127.0.0.1]) (spamfilter, port 35) with ESMTP id zy6KfVGlQOUO for ; Wed, 5 Aug 2020 19:47:49 +0300 (EEST) Received: from mail.inbox.lv (pop1 [127.0.0.1]) by shark1-in.inbox.lv (Postfix) with ESMTP id B5B5F11182B1 for ; Wed, 5 Aug 2020 19:47:49 +0300 (EEST) Received: from [10.0.0.2] (bzq-109-65-93-146.red.bezeqint.net [109.65.93.146]) (Authenticated sender: codeblue@inbox.lv) by mail.inbox.lv (Postfix) with ESMTPA id 5DC8E3D9810 for ; Wed, 5 Aug 2020 19:47:49 +0300 (EEST) Subject: Re: zfs scrub enable by default To: freebsd-fs References: <105090343.294898.1596586694925.JavaMail.zimbra@gray.id.au> <1327e123-35df-1e27-af7a-7225dae91a21@inbox.lv> <71d447e6-3f18-319a-ec98-9e35feec3180@inbox.lv> From: John Long Message-ID: <249ab487-e8c6-6e8f-f966-976407d4d3d3@inbox.lv> Date: Wed, 5 Aug 2020 16:47:46 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Virus-Scanned: OK X-ESPOL: EZeEAiZdhQAmtcG+Ippq4v3txciyUlw/zlT3t8EsiwZUt7DFs953cWydHJiNag+4bg== X-Rspamd-Queue-Id: 4BMHc900CMz3Z1k X-Spamd-Bar: ----- X-Spamd-Result: default: False [-5.10 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[inbox.lv:s=30062014]; RCVD_COUNT_FIVE(0.00)[6]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_GOOD(0.00)[194.152.32.81:from]; R_SPF_ALLOW(-0.20)[+ip4:194.152.32.81:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; NEURAL_HAM_LONG(-1.03)[-1.030]; RCPT_COUNT_ONE(0.00)[1]; DWL_DNSWL_LOW(-1.00)[inbox.lv:dkim]; RECEIVED_SPAMHAUS_PBL(0.00)[109.65.93.146:received]; NEURAL_HAM_MEDIUM(-0.99)[-0.987]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[inbox.lv:+]; DMARC_POLICY_ALLOW(-0.50)[inbox.lv,quarantine]; NEURAL_HAM_SHORT(-0.99)[-0.986]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:12993, ipnet:194.152.32.0/23, country:LV]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[194.152.32.81:from] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2020 16:47:54 -0000 On 05/08/2020 16:13, Alan Somers wrote: > On Wed, Aug 5, 2020 at 10:06 AM John Long via freebsd-fs > > wrote: > > On 05/08/2020 15:24, Alan Somers wrote: > > On Wed, Aug 5, 2020 at 9:22 AM John Long via freebsd-fs > > > >> wrote: snipped > Yeah.  I wouldn't recommend ever chaning the copies value from the > default.  If dedup is on (also not recommended), then it wouldn't make > sense to use multiple copies anyway. Yeah that felt like a shoot self in ass opportunity but I never thought about it until this thread. >  And if zfs encryption is on, then > you can't use multiple copies, because encryption repurposes some of the > block pointer's fields. > -Alan Last I looked ZFS on FreeBSD does not support encryption. But it is helpful to know that there is a technical issue in doing both. Thank you for the info! /jl From owner-freebsd-fs@freebsd.org Wed Aug 5 17:57:37 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4D68737E744 for ; Wed, 5 Aug 2020 17:57:37 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from tor1-11.mx.scaleengine.net (tor1-11.mx.scaleengine.net [IPv6:2001:470:1:474::25]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BMK8c49tjz3ygB for ; Wed, 5 Aug 2020 17:57:36 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by tor1-11.mx.scaleengine.net (Postfix) with ESMTPSA id 2753D21336 for ; Wed, 5 Aug 2020 17:57:30 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.10.3 tor1-11.mx.scaleengine.net 2753D21336 Subject: Re: zfs scrub enable by default To: freebsd-fs@freebsd.org References: <105090343.294898.1596586694925.JavaMail.zimbra@gray.id.au> From: Allan Jude Autocrypt: addr=allanjude@freebsd.org; prefer-encrypt=mutual; keydata= xsFNBFVwZcYBEADwrZDH0xe0ZVjc9ORCc6PcBLwS/RTXA6NkvpD6ea02pZ8lPOVgteuuugFc D34LdDbiWr+479vfrKBh+Y38GL0oZ0/13j10tIlDMHSa5BU0y6ACtnhupFvVlQ57+XaJAb/q 7qkfSiuxVwQ3FY3PL3cl1RrIP5eGHLA9hu4eVbu+FOX/q/XVKz49HaeIaxzo2Q54572VzIo6 C28McX9m65UL5fXMUGJDDLCItLmehZlHsQQ+uBxvODLFpVV2lUgDR/0rDa0B9zHZX8jY8qQ7 ZdCSy7CwClXI054CkXZCaBzgxYh/CotdI8ezmaw7NLs5vWNTxaDEFXaFMQtMVhvqQBpHkfOD 7rjjOmFw00nJL4FuPE5Yut0CPyx8vLjVmNJSt/Y8WxxmhutsqJYFgYfWl/vaWkrFLur/Zcmz IklwLw35HLsCZytCN5A3rGKdRbQjD6QPXOTJu0JPrJF6t2xFkWAT7oxnSV0ELhl2g+JfMMz2 Z1PDmS3NRnyEdqEm7NoRGXJJ7bgxDbN+9SXTyOletqGNXj/bSrBvhvZ0RQrzdHAPwQUfVSU2 qBhQEi2apSZstgVNMan0GUPqCdbE2zpysg+zT7Yhvf9EUQbzPL4LpdK1llT9fZbrdMzEXvEF oSvwJFdV3sqKmZc7b+E3PuxK6GTsKqaukd/3Cj8aLHG1T1im1QARAQABzSJBbGxhbiBKdWRl IDxhbGxhbmp1ZGVAZnJlZWJzZC5vcmc+wsF/BBMBAgApBQJVcGXGAhsjBQkSzAMABwsJCAcD AgEGFQgCCQoLBBYCAwECHgECF4AACgkQGZU1PhKYC34Muw/+JOKpSfhhysWFYiRXynGRDe07 Z6pVsn7DzrPUMRNZfHu8Uujmmy3p2nx9FelIY9yjd2UKHhug+whM54MiIFs90eCRVa4XEsPR 4FFAm0DAWrrb7qhZFcE/GhHdRWpZ341WAElWf6Puj2devtRjfYbikvj5+1V1QmDbju7cEw5D mEET44pTuD2VMRJpu2yZZzkM0i+wKFuPxlhqreufA1VNkZXI/rIfkYWK+nkXd9Efw3YdCyCQ zUgTUCb88ttSqcyhik/li1CDbXBpkzDCKI6I/8fAb7jjOC9LAtrZJrdgONywcVFoyK9ZN7EN AVA+xvYCmuYhR/3zHWH1g4hAm1v1+gIsufhajhfo8/wY1SetlzPaYkSkVQLqD8T6zZyhf+AN bC7ci44UsiKGAplB3phAXrtSPUEqM86kbnHg3fSx37kWKUiYNOnx4AC2VXvEiKsOBlpyt3dw WQbOtOYM+vkfbBwDtoGOOPYAKxc4LOIt9r+J8aD+gTooi9Eo5tvphATf9WkCpl9+aaGbSixB tUpvQMRnSMqTqq4Z7DeiG6VMRQIjsXDSLJEUqcfhnLFo0Ko/RiaHd5xyAQ4DhQ9QpkyQjjNf /3f/dYG7JAtoD30txaQ5V8uHrz210/77DRRX+HJjEj6xCxWUGvQgvEZf5XXyxeePvqZ+zQyT DX61bYw6w6bOwU0EVXBlxgEQAMy7YVnCCLN4oAOBVLZ5nUbVPvpUhsdA94/0/P+uqCIh28Cz ar56OCX0X19N/nAWecxL4H32zFbIRyDB2V/MEh4p9Qvyu/j4i1r3Ex5GhOT2hnit43Ng46z5 29Es4TijrHJP4/l/rB2VOqMKBS7Cq8zk1cWqaI9XZ59imxDNjtLLPPM+zQ1yE3OAMb475QwN UgWxTMw8rkA7CEaqeIn4sqpTSD5C7kT1Bh26+rbgJDZ77D6Uv1LaCZZOaW52okW3bFbdozV8 yM2u+xz2Qs8bHz67p+s+BlygryiOyYytpkiK6Iy4N7FTolyj5EIwCuqzfk0SaRHeOKX2ZRjC qatkgoD/t13PNT38V9tw3qZVOJDS0W6WM8VSg+F+bkM9LgJ8CmKV+Hj0k3pfGfYPOZJ/v18i +SmZmL/Uw2RghnwDWGAsPCKu4uZR777iw7n9Io6Vfxndw2dcS0e9klvFYoaGS6H2F13Asygr WBzFNGFQscN4mUW+ZYBzpTOcHkdT7w8WS55BmXYLna+dYer9/HaAuUrONjujukN4SPS1fMJ2 /CS/idAUKyyVVX5vozoNK2JVC1h1zUAVsdnmhEzNPsvBoqcVNfyqBFROEVLIPwq+lQMGNVjH ekLTKRWf59MEhUC2ztjSKkGmwdg73d6xSXMuq45EgIJV2wPvOgWQonoHH/kxABEBAAHCwWUE GAECAA8FAlVwZcYCGwwFCRLMAwAACgkQGZU1PhKYC34w5A//YViBtZyDV5O+SJT9FFO3lb9x Zdxf0trA3ooCt7gdBkdnBM6T5EmjgVZ3KYYyFfwXZVkteuCCycMF/zVw5eE9FL1+zz9gg663 nY9q2F77TZTKXVWOLlOV2bY+xaK94U4ytogOGhh9b4UnQ/Ct3+6aviCF78Go608BXbmF/GVT 7uhddemk7ItxM1gE5Hscx3saxGKlayaOsdPKeGTVJCDEtHDuOc7/+jGh5Zxpk/Hpi+DUt1ot 8e6hPYLIQa4uVx4f1xxxV858PQ7QysSLr9pTV7FAQ18JclCaMc7JWIa3homZQL/MNKOfST0S 2e+msuRwQo7AnnfFKBUtb02KwpA4GhWryhkjUh/kbVc1wmGxaU3DgXYQ5GV5+Zf4kk/wqr/7 KG0dkTz6NLCVLyDlmAzuFhf66DJ3zzz4yIo3pbDYi3HB/BwJXVSKB3Ko0oUo+6/qMrOIS02L s++QE/z7K12CCcs7WwOjfCYHK7VtE0Sr/PfybBdTbuDncOuAyAIeIKxdI2nmQHzl035hhvQX s4CSghsP319jAOQiIolCeSbTMD4QWMK8RL/Pe1FI1jC3Nw9s+jq8Dudtbcj2UwAP/STUEbJ9 5rznzuuhPjE0e++EU/RpWmcaIMK/z1zZDMN+ce2v1qzgV936ZhJ3iaVzyqbEE81gDxg3P+IM kiYh4ZtPB4Q= Message-ID: <52faf0aa-2c91-97c0-e0b9-ce8c1b513cbd@freebsd.org> Date: Wed, 5 Aug 2020 13:57:25 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="aT8JEaXoPWqjghaFErxWVY9e6WSfUFRS8" X-Rspamd-Queue-Id: 4BMK8c49tjz3ygB X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2020 17:57:37 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --aT8JEaXoPWqjghaFErxWVY9e6WSfUFRS8 Content-Type: multipart/mixed; boundary="6pL0dlkpiRaE5F36b7jNbDtc2C74GyX9B"; protected-headers="v1" From: Allan Jude To: freebsd-fs@freebsd.org Message-ID: <52faf0aa-2c91-97c0-e0b9-ce8c1b513cbd@freebsd.org> Subject: Re: zfs scrub enable by default References: <105090343.294898.1596586694925.JavaMail.zimbra@gray.id.au> In-Reply-To: --6pL0dlkpiRaE5F36b7jNbDtc2C74GyX9B Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 2020-08-05 00:32, Ryan Moeller wrote: > On Tue, Aug 4, 2020 at 11:36 PM PK1048.COM wrote: >> >> On Aug 4, 2020, at 8:18 PM, Grant Gray via freebsd-fs wrote: >> >>> 2. That scrubbing PREVENTS data loss. Scrubbing can only tell you abo= ut data loss AFTER it happens. Yes, it could alert you to a problem that = prevents further data loss, but it's already too late. Scrubbing is not a= substitute for RAID, backups and proactive SMART testing. >> >> Scrubbing does not directly prevent data loss. Scrubbing identifies da= ta returned from the underlying device as corrupt or incorrect. If the zp= ool has sufficient redundancy, then no data is lost. >> >> But, normal ZFS operation also identifies incorrect date returned from= the underlying device(s). >> >> While I have only managed a few hundred drives under ZFS, I cannot rec= all one case where a proactive scrub found bad data _before_ normal opera= tions. Once a checksum error is reported via `zpool status` then I have f= ound a scrub useful to determine the extent of the drive failure. >> >> I know there are others with thousands of drives managed under ZFS and= their experience may differ. >> >=20 >> In terms of whether the periodic scrub should be enabled by default, I= am undecided, as I can see both sides of the argument. I would prefer to= make it a user choice during installation as one of the ZFS options (suc= h as native 4K drives, ashift=3D12 or mirrored swap). Make the default en= abled, but by putting it in the installation options, those who know they= need it disabled can make sure it is off from day one and not have a rud= e surprise the first time it runs (35 days after installation). >> >=20 > I don't think it was mentioned on this list, but that seems to be the > current direction this is heading. Providing an option in the > installer, whether enabled by default or not, should address > everyone's concerns. >=20 I don't think the installer is the place to define defaults though. So I'd lean towards: on by default, but offer it as an option in the installer, so they can choose to disable it. --=20 Allan Jude --6pL0dlkpiRaE5F36b7jNbDtc2C74GyX9B-- --aT8JEaXoPWqjghaFErxWVY9e6WSfUFRS8 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJfKvMJAAoJEBmVNT4SmAt+oPYP/j1xwimOgSWxNJyh3exXqnqs VTUOefDcfH8EJkpJ60SB01Rki51LATvBl3+4ZNwmjMGBCooi3IcsLPDyhcbQXUE0 B+PXOQj4tGWu5FiQ6MEAvO9MQit3/yLyYeeKMdf1heABlo4KE0APg/MbgPbXDmwU lJk1GHnFEBEMwI2j55sO034xs3AYNFsg2G0rA0UfIr6nsEnCGDACul8reUhqlJDk YSu23Sw7q1ubmy1tlcNm1dguQplBtecH0cvB6KHWEup6zEgvEJghRKxJaky5lQNR bSvI00fK28uEJiiw89XtwTz5bUGQ6uk5TEZXNODPjR7u2GWElZDXzqb0+W8L4vHW aKM5T6pq4og7EUmmF7IN1y4La3ciLURoRt8exqsr2leodFLGPTbUEHd9x8DfgD9e Fjd+YrVaCFIlQaW4i+C8FGoLrm5uXFuB1pePGTFFA8BwpzE5abHAedJxk4CtwHzS PCwwJuAWJpTdXKcYsXEWNRbMh/ScpGcD/udmiRCQ5ru1DD6uhPy88eOy46sZ24Rc HgRBkxlWpAR16z1Hoj07DLLR7FzQRdnvlfImVlC9OnlCjas0dOH4Nlumw5H0/NnQ aw489TQiWAVkrR7zo8PAfHlWdgumzxHX4dgQkJiDnLSeXFZuj79eE+1pwBNb/fEk fjXeVep/R1l/35FsnkOZ =zg85 -----END PGP SIGNATURE----- --aT8JEaXoPWqjghaFErxWVY9e6WSfUFRS8-- From owner-freebsd-fs@freebsd.org Wed Aug 5 18:21:03 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6B2303A5DB5 for ; Wed, 5 Aug 2020 18:21:03 +0000 (UTC) (envelope-from wfc@mintsol.com) Received: from scully.mintsol.com (scully.mintsol.com [199.182.77.206]) by mx1.freebsd.org (Postfix) with ESMTP id 4BMKgf3fbvz440l; Wed, 5 Aug 2020 18:21:02 +0000 (UTC) (envelope-from wfc@mintsol.com) Received: from mintsol.com (officecc.mintsol.com [96.85.114.33]) by scully.mintsol.com with esmtp; Wed, 05 Aug 2020 14:20:55 -0400 id 00B0D048.000000005F2AF887.00016CED Received: from localhost (localhost [127.0.0.1]) (IDENT: uid 1002) by mintsol.com with esmtp; Wed, 05 Aug 2020 14:20:56 -0400 id 00000ADF.5F2AF888.0001356E Date: Wed, 5 Aug 2020 14:20:56 -0400 (EDT) From: Walter Cramer To: Gary Palmer cc: FreeBSD FS Subject: Re: zfs scrub enable by default In-Reply-To: <20200804122517.GA39788@in-addr.com> Message-ID: <20200805123550.W71882@mulder.mintsol.com> References: <24edb075-155c-439d-1ef5-541893036429@freebsd.org> <0d1d14c9-64ba-368c-b2f4-56efa741cc5e@inbox.lv> <2038290194.249397.1596502070473.JavaMail.zimbra@gray.id.au> <20200804122517.GA39788@in-addr.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4BMKgf3fbvz440l X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of wfc@mintsol.com designates 199.182.77.206 as permitted sender) smtp.mailfrom=wfc@mintsol.com X-Spamd-Result: default: False [-2.08 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.995]; FROM_HAS_DN(0.00)[]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; R_SPF_ALLOW(-0.20)[+a:scully.mintsol.com]; DMARC_NA(0.00)[mintsol.com]; NEURAL_HAM_LONG(-1.01)[-1.011]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.38)[-0.378]; RCPT_COUNT_TWO(0.00)[2]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:22768, ipnet:199.182.77.0/24, country:US] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2020 18:21:03 -0000 On Tue, 4 Aug 2020, Gary Palmer wrote: > On Tue, Aug 04, 2020 at 10:47:50AM +1000, Grant Gray via freebsd-fs wrote: >> >> >> ----- On 4 Aug, 2020, at 9:33 AM, Josh Paetzel josh@tcbug.org wrote: >> >>> On Mon, Aug 3, 2020, at 4:41 PM, John Long via freebsd-fs wrote: >>>> On 03/08/2020 20:25, Allan Jude wrote: >>>>> On 2020-08-03 12:10, Steve Wills wrote: >>>>>> Hi, >>>>>> >>>>>> I wonder why we don't enable zfs periodic scrub by default? >>>>>> >>>>>> https://svnweb.freebsd.org/base/head/usr.sbin/periodic/periodic.conf?view=markup#l162 >>>>>> >>>>>> >>>>>> Anyone happen to know? >>>>>> >>>>>> Thanks, >>>>>> Steve >>>>>> >>>>>> _______________________________________________ >>>>>> freebsd-fs@freebsd.org mailing list >>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>>>> >>>>> I think switching this to on-by-default is a good thing. >>>>> >>>>> To be clear, which the check is part of 'periodic daily', it only >>>>> triggers a scrub if it has been more than 35 days since the last scrub. >>>>> >>>>> FreeNAS already has does this, and that accounts for a large number of >>>>> FreeBSD ZFS deployments. >>>> >>>> There is a difference. FreeNAS is an appliance. It should minimize the >>>> need for management. >>>> >>>> FreeBSD is the base OS. It should do as little as possible so people can >>>> set it up the way we want rather than spend days or weeks unbreaking >>>> surprising, non-standard behavior and hundreds of tweaks for everybody >>>> who requested them. >>>> >>>>> There is tuning you can do in ZFS to try to lessen the impact of a scrub >>>>> on your production workloads. >>>> >>>> That's up to the guy running the box to do or not to do. >>>> >>>>> The periodic script lets you select which pools to include (defaults to >>>>> all), so you can tune it to only scrub your root pool every 35 days, and >>>>> not the large pool that might take too long to scrub or whatever. It >>>>> also lets you set a different threshold for each pool. >>>> >>>> A NAS appliance could benefit from something like this. A generic OS cannot. >>>> >>>>> So I don't see any reason not to enable it by default, and just document >>>>> how to adjust it if people really need to disable it. Honestly, I think >>>>> those who are disabling it are doing themselves a disservice. >>>> >>>> It's offensive for you to presume to think you know better what the >>>> other guy needs than he does himself. Thank you for clarifying it >>>> though, I think it's valuable to understand the thinking of people who >>>> want to coopt the OS into their personal playground. >>>> >>>> /jl >>>> _______________________________________________ >>>> freebsd-fs@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>>> >>> >>> ZFS needs scrub in the same way that UFS needs fsck. (in that i you don't scrub >>> a ZFS volume for long enough and you don't fsck a UFS volume for long enough >>> they will stop being useful filesystems) If you've ever had the joy of running >>> UFS on a volume so large you couldn't fsck it or run say XFS on a volume so >>> large you couldn't run xfs_repair you'll know that you can get away with that >>> for a while, but eventually the chickens come home to roost and you'll be >>> restoring from tape or failing over to your DR site, or complaining about lost >>> data. (Note that XFS is journaled and is far more resilient than UFS to this >>> sort of thing but it's still an issue) >>> >> >> These are some fun (and real) historical anecdotes but, other than bit-rot, they do not represent current reality as these problems have been largely solved or the momentum has shifted to better filesystems (such as ZFS and XFS). >> >>> The reality is having scrub off is (in the vast majority of cases) the wrong >>> thing to do. I say that as someone who has nearly an exabyte in ZFS right now, >>> and lead the team that brought ZFS based FreeNAS to the world. So the >>> suggestion to "fix the glitch" and enable periodic scrubs makes perfect sense >>> to me. Those who don't know any better will just magically receive the benefit >>> of something they should've been doing all along. Those who do know better >>> have a lot of time on their hands (at least judging by this thread!) and with >>> way less typing than the length of these emails can disable the periodic >>> script. Sounds win-win to me. >>> >>> -- >>> >>> Thanks, >>> >>> Josh Paetzel >> >> I think you are projecting a false characterisation of the average FreeBSD user and workload. FreeBSD is not an appliance and is not, in my experience, deployed or used by inexperienced users for any serious use-case. Who exactly is the user/use-case you are advocating for? >> >> There seems to be a conflation of purpose of scrubbing and general disk health; the point of a scrub is to catch bit errors in files that are infrequently accessed, that would otherwise not be detected in real-time. The value of that is largely dictated by whether you have RAID, and your backup strategy. A scrub does not tell you anything about the state of blocks that are not in use (or the overall health of a disk; ref smartd). >> >> Other people in this thread have already alluded to the fact that scrubbing can be detrimental, or even disastrous, for many of the workloads FreeBSD is used for. Excess scrubbing is a waste of energy and potentially reduces the working life of the disks. What I (and others in this thread) are getting at is setting a scrub schedule is so context specific that getting it right for the general case is near impossible, because there is no general case. >> >> Not scrubbing by default does not lead to unexpected behaviour. Scrubbing by default can and does lead to unexpected behaviour. People who care about their data use RAID and will determine a scrub schedule that suits their context. >> >> I take your point that, for a specific type of user, scrubbing by default makes sense. I'm just not sure that type of user is the benchmark for deciding FreeBSD system policies. If a user won't bear the cognitive load of understanding how to protect their own data, I would suggest FreeBSD is not for them and there are better options that will protect them better, by default (such as Ubuntu). > > Rather than assume a default which works for everyone, why not put an > option screen in the installer if the user selects ZFS. They can choose > from daily, weekly, monthly or never with a warning about possible (but rare) > corruption on the latter option. > > Whatever the outcome of this discussion it should only be set on new > installs as I think a lot of people would be upset if a new default > scrub schedule were forced on them on upgrade, especially if they > had one already defined. > > Regards, > > Gary I don't recall any arguments for having scrub_zfs specifically in the daily - vs. weekly or monthly - periodic scripts. Nor mention of anyone fine-tuning its default_threshold, which is now exactly 5 weeks. Since (my bet) the great majority of systems have a strong, predictable weekly cycle of busier and more-idle periods, it seems logical to move the 800.scrub-zfs script to /etc/periodic/weekly - regardless of any decision we might reach about the default enable value being "YES", or "NO". (Personally, I favor putting very long, resource-intensive periodic tasks into their own /usr/local/etc/periodic/{descriptive_name_here} scripts. But that's adding complexity, and any sysadmin who understands cron jobs, 'man periodic', and small shell scripts can quickly do that for themselves.) Otherwise, +1 to Gary's ideas. -Walter From owner-freebsd-fs@freebsd.org Wed Aug 5 19:53:55 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7AC7A3A7A4A for ; Wed, 5 Aug 2020 19:53:55 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lj1-f177.google.com (mail-lj1-f177.google.com [209.85.208.177]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BMMkp4fx2z49MM; Wed, 5 Aug 2020 19:53:54 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lj1-f177.google.com with SMTP id g6so36446910ljn.11; Wed, 05 Aug 2020 12:53:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:openpgp:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=Da4C7dFKGkglmbwW/u+tG2nolr1zczovT64jTHowgts=; b=FHvVenFNWrdeebDIL44+DNeph80cvsYMXAo4QteOdb6koeIN1E2/KFKjjc4CIjLQnQ +CmPkisbZU7UKrrKRa+Vhg0/XagKfIsXjub7iUwDwzojYI7ZsKNU48jSImQyRyy4AzCi GlT5FlkSAccxlD/XTwF2IUCCNc4yPN7frYwnyiQjYiNpL1bgX9N25XAZc7J1GaFrfJpT vLz9TNGm1smQ5nnVO7wq5k2QRuTRvn77KARCZnZQPfuFHr1M5UjhWvm9yI3kDgPNoRiD YCNrOTDmmr0Ln7Ve8sz9Imv4/IWz839gmunR8rHyG7sAUUAzL/r3a3xadDzOYJQR5hVF 3kZg== X-Gm-Message-State: AOAM532e+empeQCZ2tlRNL7Zx6oWk39d2WLSfA6ChsDq/eNk7n9N2wxr 9wMFafaEPUxRfrbDvcNxjF4bksoZ X-Google-Smtp-Source: ABdhPJzklmspaQZeeMpqJSv4Se323f3qeP8eT/Q5x3V6abPbzZrMdOU580+3UMPz0kAWM/dh5Xu9Wg== X-Received: by 2002:a2e:7504:: with SMTP id q4mr339717ljc.41.1596657232508; Wed, 05 Aug 2020 12:53:52 -0700 (PDT) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id y136sm1547760lfa.79.2020.08.05.12.53.50 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 05 Aug 2020 12:53:51 -0700 (PDT) Subject: Re: zfs scrub enable by default To: Alan Somers Cc: freebsd-fs References: <105090343.294898.1596586694925.JavaMail.zimbra@gray.id.au> <1327e123-35df-1e27-af7a-7225dae91a21@inbox.lv> <71d447e6-3f18-319a-ec98-9e35feec3180@inbox.lv> From: Andriy Gapon Openpgp: preference=signencrypt Autocrypt: addr=avg@FreeBSD.org; prefer-encrypt=mutual; keydata= mQINBFm4LIgBEADNB/3lT7f15UKeQ52xCFQx/GqHkSxEdVyLFZTmY3KyNPQGBtyvVyBfprJ7 mAeXZWfhat6cKNRAGZcL5EmewdQuUfQfBdYmKjbw3a9GFDsDNuhDA2QwFt8BmkiVMRYyvI7l N0eVzszWCUgdc3qqM6qqcgBaqsVmJluwpvwp4ZBXmch5BgDDDb1MPO8AZ2QZfIQmplkj8Y6Z AiNMknkmgaekIINSJX8IzRzKD5WwMsin70psE8dpL/iBsA2cpJGzWMObVTtCxeDKlBCNqM1i gTXta1ukdUT7JgLEFZk9ceYQQMJJtUwzWu1UHfZn0Fs29HTqawfWPSZVbulbrnu5q55R4PlQ /xURkWQUTyDpqUvb4JK371zhepXiXDwrrpnyyZABm3SFLkk2bHlheeKU6Yql4pcmSVym1AS4 dV8y0oHAfdlSCF6tpOPf2+K9nW1CFA8b/tw4oJBTtfZ1kxXOMdyZU5fiG7xb1qDgpQKgHUX8 7Rd2T1UVLVeuhYlXNw2F+a2ucY+cMoqz3LtpksUiBppJhw099gEXehcN2JbUZ2TueJdt1FdS ztnZmsHUXLxrRBtGwqnFL7GSd6snpGIKuuL305iaOGODbb9c7ne1JqBbkw1wh8ci6vvwGlzx rexzimRaBzJxlkjNfMx8WpCvYebGMydNoeEtkWldtjTNVsUAtQARAQABtB5BbmRyaXkgR2Fw b24gPGF2Z0BGcmVlQlNELm9yZz6JAlQEEwEIAD4WIQS+LEO7ngQnXA4Bjr538m7TUc1yjwUC WbgsiAIbIwUJBaOagAULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAKCRB38m7TUc1yj+JAEACV l9AK/nOWAt/9cufV2fRj0hdOqB1aCshtSrwHk/exXsDa4/FkmegxXQGY+3GWX3deIyesbVRL rYdtdK0dqJyT1SBqXK1h3/at9rxr9GQA6KWOxTjUFURsU7ok/6SIlm8uLRPNKO+yq0GDjgaO LzN+xykuBA0FlhQAXJnpZLcVfPJdWv7sSHGedL5ln8P8rxR+XnmsA5TUaaPcbhTB+mG+iKFj GghASDSfGqLWFPBlX/fpXikBDZ1gvOr8nyMY9nXhgfXpq3B6QCRYKPy58ChrZ5weeJZ29b7/ QdEO8NFNWHjSD9meiLdWQaqo9Y7uUxN3wySc/YUZxtS0bhAd8zJdNPsJYG8sXgKjeBQMVGuT eCAJFEYJqbwWvIXMfVWop4+O4xB+z2YE3jAbG/9tB/GSnQdVSj3G8MS80iLS58frnt+RSEw/ psahrfh0dh6SFHttE049xYiC+cM8J27Aaf0i9RflyITq57NuJm+AHJoU9SQUkIF0nc6lfA+o JRiyRlHZHKoRQkIg4aiKaZSWjQYRl5Txl0IZUP1dSWMX4s3XTMurC/pnja45dge/4ESOtJ9R 8XuIWg45Oq6MeIWdjKddGhRj3OohsltKgkEU3eLKYtB6qRTQypHHUawCXz88uYt5e3w4V16H lCpSTZV/EVHnNe45FVBlvK7k7HFfDDkryLkCDQRZuCyIARAAlq0slcsVboY/+IUJdcbEiJRW be9HKVz4SUchq0z9MZPX/0dcnvz/gkyYA+OuM78dNS7Mbby5dTvOqfpLJfCuhaNYOhlE0wY+ 1T6Tf1f4c/uA3U/YiadukQ3+6TJuYGAdRZD5EqYFIkreARTVWg87N9g0fT9BEqLw9lJtEGDY EWUE7L++B8o4uu3LQFEYxcrb4K/WKmgtmFcm77s0IKDrfcX4doV92QTIpLiRxcOmCC/OCYuO jB1oaaqXQzZrCutXRK0L5XN1Y1PYjIrEzHMIXmCDlLYnpFkK+itlXwlE2ZQxkfMruCWdQXye syl2fynAe8hvp7Mms9qU2r2K9EcJiR5N1t1C2/kTKNUhcRv7Yd/vwusK7BqJbhlng5ZgRx0m WxdntU/JLEntz3QBsBsWM9Y9wf2V4tLv6/DuDBta781RsCB/UrU2zNuOEkSixlUiHxw1dccI 6CVlaWkkJBxmHX22GdDFrcjvwMNIbbyfQLuBq6IOh8nvu9vuItup7qemDG3Ms6TVwA7BD3j+ 3fGprtyW8Fd/RR2bW2+LWkMrqHffAr6Y6V3h5kd2G9Q8ZWpEJk+LG6Mk3fhZhmCnHhDu6CwN MeUvxXDVO+fqc3JjFm5OxhmfVeJKrbCEUJyM8ESWLoNHLqjywdZga4Q7P12g8DUQ1mRxYg/L HgZY3zfKOqcAEQEAAYkCPAQYAQgAJhYhBL4sQ7ueBCdcDgGOvnfybtNRzXKPBQJZuCyIAhsM BQkFo5qAAAoJEHfybtNRzXKPBVwQAKfFy9P7N3OsLDMB56A4Kf+ZT+d5cIx0Yiaf4n6w7m3i ImHHHk9FIetI4Xe54a2IXh4Bq5UkAGY0667eIs+Z1Ea6I2i27Sdo7DxGwq09Qnm/Y65ADvXs 3aBvokCcm7FsM1wky395m8xUos1681oV5oxgqeRI8/76qy0hD9WR65UW+HQgZRIcIjSel9vR XDaD2HLGPTTGr7u4v00UeTMs6qvPsa2PJagogrKY8RXdFtXvweQFz78NbXhluwix2Tb9ETPk LIpDrtzV73CaE2aqBG/KrboXT2C67BgFtnk7T7Y7iKq4/XvEdDWscz2wws91BOXuMMd4c/c4 OmGW9m3RBLufFrOag1q5yUS9QbFfyqL6dftJP3Zq/xe+mr7sbWbhPVCQFrH3r26mpmy841ym dwQnNcsbIGiBASBSKksOvIDYKa2Wy8htPmWFTEOPRpFXdGQ27awcjjnB42nngyCK5ukZDHi6 w0qK5DNQQCkiweevCIC6wc3p67jl1EMFY5+z+zdTPb3h7LeVnGqW0qBQl99vVFgzLxchKcl0 R/paSFgwqXCZhAKMuUHncJuynDOP7z5LirUeFI8qsBAJi1rXpQoLJTVcW72swZ42IdPiboqx NbTMiNOiE36GqMcTPfKylCbF45JNX4nF9ElM0E+Y8gi4cizJYBRr2FBJgay0b9Cp Message-ID: <160f945d-5d07-8bbb-f3f6-4d2fa5d40c63@FreeBSD.org> Date: Wed, 5 Aug 2020 22:53:50 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Firefox/60.0 Thunderbird/60.9.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4BMMkp4fx2z49MM X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of agapon@gmail.com designates 209.85.208.177 as permitted sender) smtp.mailfrom=agapon@gmail.com X-Spamd-Result: default: False [-0.56 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[FreeBSD.org]; NEURAL_HAM_LONG(-0.50)[-0.505]; NEURAL_SPAM_MEDIUM(0.07)[0.071]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.12)[-0.123]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[209.85.208.177:from]; FORGED_SENDER(0.30)[avg@FreeBSD.org,agapon@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.208.177:from]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[avg@FreeBSD.org,agapon@gmail.com]; FREEMAIL_ENVFROM(0.00)[gmail.com]; RECEIVED_SPAMHAUS_PBL(0.00)[93.72.151.96:received] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Aug 2020 19:53:55 -0000 On 05/08/2020 19:13, Alan Somers wrote: > If dedup is on (also not recommended), then it wouldn't make > sense to use multiple copies anyway. Even when dedup is enabled n separate copies are still saved. -- Andriy Gapon From owner-freebsd-fs@freebsd.org Thu Aug 6 02:49:57 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 921973AF87F for ; Thu, 6 Aug 2020 02:49:57 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from tor1-11.mx.scaleengine.net (tor1-11.mx.scaleengine.net [209.51.186.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BMXys1VdNz4Wr3 for ; Thu, 6 Aug 2020 02:49:56 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by tor1-11.mx.scaleengine.net (Postfix) with ESMTPSA id F07A122E63; Thu, 6 Aug 2020 02:49:49 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.10.3 tor1-11.mx.scaleengine.net F07A122E63 From: Allan Jude To: status-updates@freebsdfoundation.org, freebsd-fs , openzfs-developer References: <7b8842ad-d520-c575-22ee-2cd77244f2c6@freebsd.org> <708ec9f2-3c5c-6452-f6e6-bfb11a7f7eb2@freebsd.org> <528ca743-7889-d1fd-ca95-a17cd430725b@freebsd.org> <9d77cb73-c8e8-cca0-b4b8-28e6790268d6@freebsd.org> <327f4b10-9727-331e-2dc9-641dad96dd2a@freebsd.org> Autocrypt: addr=allanjude@freebsd.org; prefer-encrypt=mutual; keydata= xsFNBFVwZcYBEADwrZDH0xe0ZVjc9ORCc6PcBLwS/RTXA6NkvpD6ea02pZ8lPOVgteuuugFc D34LdDbiWr+479vfrKBh+Y38GL0oZ0/13j10tIlDMHSa5BU0y6ACtnhupFvVlQ57+XaJAb/q 7qkfSiuxVwQ3FY3PL3cl1RrIP5eGHLA9hu4eVbu+FOX/q/XVKz49HaeIaxzo2Q54572VzIo6 C28McX9m65UL5fXMUGJDDLCItLmehZlHsQQ+uBxvODLFpVV2lUgDR/0rDa0B9zHZX8jY8qQ7 ZdCSy7CwClXI054CkXZCaBzgxYh/CotdI8ezmaw7NLs5vWNTxaDEFXaFMQtMVhvqQBpHkfOD 7rjjOmFw00nJL4FuPE5Yut0CPyx8vLjVmNJSt/Y8WxxmhutsqJYFgYfWl/vaWkrFLur/Zcmz IklwLw35HLsCZytCN5A3rGKdRbQjD6QPXOTJu0JPrJF6t2xFkWAT7oxnSV0ELhl2g+JfMMz2 Z1PDmS3NRnyEdqEm7NoRGXJJ7bgxDbN+9SXTyOletqGNXj/bSrBvhvZ0RQrzdHAPwQUfVSU2 qBhQEi2apSZstgVNMan0GUPqCdbE2zpysg+zT7Yhvf9EUQbzPL4LpdK1llT9fZbrdMzEXvEF oSvwJFdV3sqKmZc7b+E3PuxK6GTsKqaukd/3Cj8aLHG1T1im1QARAQABzSJBbGxhbiBKdWRl IDxhbGxhbmp1ZGVAZnJlZWJzZC5vcmc+wsF/BBMBAgApBQJVcGXGAhsjBQkSzAMABwsJCAcD AgEGFQgCCQoLBBYCAwECHgECF4AACgkQGZU1PhKYC34Muw/+JOKpSfhhysWFYiRXynGRDe07 Z6pVsn7DzrPUMRNZfHu8Uujmmy3p2nx9FelIY9yjd2UKHhug+whM54MiIFs90eCRVa4XEsPR 4FFAm0DAWrrb7qhZFcE/GhHdRWpZ341WAElWf6Puj2devtRjfYbikvj5+1V1QmDbju7cEw5D mEET44pTuD2VMRJpu2yZZzkM0i+wKFuPxlhqreufA1VNkZXI/rIfkYWK+nkXd9Efw3YdCyCQ zUgTUCb88ttSqcyhik/li1CDbXBpkzDCKI6I/8fAb7jjOC9LAtrZJrdgONywcVFoyK9ZN7EN AVA+xvYCmuYhR/3zHWH1g4hAm1v1+gIsufhajhfo8/wY1SetlzPaYkSkVQLqD8T6zZyhf+AN bC7ci44UsiKGAplB3phAXrtSPUEqM86kbnHg3fSx37kWKUiYNOnx4AC2VXvEiKsOBlpyt3dw WQbOtOYM+vkfbBwDtoGOOPYAKxc4LOIt9r+J8aD+gTooi9Eo5tvphATf9WkCpl9+aaGbSixB tUpvQMRnSMqTqq4Z7DeiG6VMRQIjsXDSLJEUqcfhnLFo0Ko/RiaHd5xyAQ4DhQ9QpkyQjjNf /3f/dYG7JAtoD30txaQ5V8uHrz210/77DRRX+HJjEj6xCxWUGvQgvEZf5XXyxeePvqZ+zQyT DX61bYw6w6bOwU0EVXBlxgEQAMy7YVnCCLN4oAOBVLZ5nUbVPvpUhsdA94/0/P+uqCIh28Cz ar56OCX0X19N/nAWecxL4H32zFbIRyDB2V/MEh4p9Qvyu/j4i1r3Ex5GhOT2hnit43Ng46z5 29Es4TijrHJP4/l/rB2VOqMKBS7Cq8zk1cWqaI9XZ59imxDNjtLLPPM+zQ1yE3OAMb475QwN UgWxTMw8rkA7CEaqeIn4sqpTSD5C7kT1Bh26+rbgJDZ77D6Uv1LaCZZOaW52okW3bFbdozV8 yM2u+xz2Qs8bHz67p+s+BlygryiOyYytpkiK6Iy4N7FTolyj5EIwCuqzfk0SaRHeOKX2ZRjC qatkgoD/t13PNT38V9tw3qZVOJDS0W6WM8VSg+F+bkM9LgJ8CmKV+Hj0k3pfGfYPOZJ/v18i +SmZmL/Uw2RghnwDWGAsPCKu4uZR777iw7n9Io6Vfxndw2dcS0e9klvFYoaGS6H2F13Asygr WBzFNGFQscN4mUW+ZYBzpTOcHkdT7w8WS55BmXYLna+dYer9/HaAuUrONjujukN4SPS1fMJ2 /CS/idAUKyyVVX5vozoNK2JVC1h1zUAVsdnmhEzNPsvBoqcVNfyqBFROEVLIPwq+lQMGNVjH ekLTKRWf59MEhUC2ztjSKkGmwdg73d6xSXMuq45EgIJV2wPvOgWQonoHH/kxABEBAAHCwWUE GAECAA8FAlVwZcYCGwwFCRLMAwAACgkQGZU1PhKYC34w5A//YViBtZyDV5O+SJT9FFO3lb9x Zdxf0trA3ooCt7gdBkdnBM6T5EmjgVZ3KYYyFfwXZVkteuCCycMF/zVw5eE9FL1+zz9gg663 nY9q2F77TZTKXVWOLlOV2bY+xaK94U4ytogOGhh9b4UnQ/Ct3+6aviCF78Go608BXbmF/GVT 7uhddemk7ItxM1gE5Hscx3saxGKlayaOsdPKeGTVJCDEtHDuOc7/+jGh5Zxpk/Hpi+DUt1ot 8e6hPYLIQa4uVx4f1xxxV858PQ7QysSLr9pTV7FAQ18JclCaMc7JWIa3homZQL/MNKOfST0S 2e+msuRwQo7AnnfFKBUtb02KwpA4GhWryhkjUh/kbVc1wmGxaU3DgXYQ5GV5+Zf4kk/wqr/7 KG0dkTz6NLCVLyDlmAzuFhf66DJ3zzz4yIo3pbDYi3HB/BwJXVSKB3Ko0oUo+6/qMrOIS02L s++QE/z7K12CCcs7WwOjfCYHK7VtE0Sr/PfybBdTbuDncOuAyAIeIKxdI2nmQHzl035hhvQX s4CSghsP319jAOQiIolCeSbTMD4QWMK8RL/Pe1FI1jC3Nw9s+jq8Dudtbcj2UwAP/STUEbJ9 5rznzuuhPjE0e++EU/RpWmcaIMK/z1zZDMN+ce2v1qzgV936ZhJ3iaVzyqbEE81gDxg3P+IM kiYh4ZtPB4Q= Subject: Re: ZSTD Project Weekly Status Update Message-ID: Date: Wed, 5 Aug 2020 22:49:44 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <327f4b10-9727-331e-2dc9-641dad96dd2a@freebsd.org> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="R8vzuaKLhzyCjf7STmUzjL8AXKZS0tooK" X-Rspamd-Queue-Id: 4BMXys1VdNz4Wr3 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; ASN(0.00)[asn:6939, ipnet:209.51.160.0/19, country:US] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 06 Aug 2020 02:49:57 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --R8vzuaKLhzyCjf7STmUzjL8AXKZS0tooK Content-Type: multipart/mixed; boundary="UkN47Zpoz8FxP3sgMe7uV1jD4vlSCfYuM"; protected-headers="v1" From: Allan Jude To: status-updates@freebsdfoundation.org, freebsd-fs , openzfs-developer Message-ID: Subject: Re: ZSTD Project Weekly Status Update References: <7b8842ad-d520-c575-22ee-2cd77244f2c6@freebsd.org> <708ec9f2-3c5c-6452-f6e6-bfb11a7f7eb2@freebsd.org> <528ca743-7889-d1fd-ca95-a17cd430725b@freebsd.org> <9d77cb73-c8e8-cca0-b4b8-28e6790268d6@freebsd.org> <327f4b10-9727-331e-2dc9-641dad96dd2a@freebsd.org> In-Reply-To: <327f4b10-9727-331e-2dc9-641dad96dd2a@freebsd.org> --UkN47Zpoz8FxP3sgMe7uV1jD4vlSCfYuM Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable This is the seventh weekly status report on the project to integrate ZSTD into OpenZFS. The compatibility related changes I created last week were refined and marged into the mainline branch. Thanks to Brian Behlendorf for reviewing my proposed change for the zstd feature flag activation, and pointing out a better approach. I have reworked the patch based on his suggestion and prototype: https://github.com/allanjude/zfs/commit/2508dafcec0a05d61afc5fbd5da356e20= 1afbe97 - Activate the per-dataset ZSTD feature flag as soon as the property is set to ZSTD. Before, simply doing `zfs set compression=3Dzstd dataset` would not activate the feature flag. The feature flag would be activated when the first block that used ZSTD compression was written (see dsl_dataset_block_born()). This meant that if you set the property, exported the pool, the pool would import on systems with older versions of ZFS that did not support ZSTD, but would crash their userspace tools, because the property value was out of bounds. https://github.com/allanjude/zfs/commit/b8bec3fd2a8feb3a4de572eb15515d376= 4f92a35 - I created a test that ensures that the feature flag is activated by `zfs set compression=3Dzstd` and also ensures that the feature flag reverts to the 'enabled' state once the last dataset using zstd is destroyed. The next step is ensuring that ZSTD compression inter-operates properly with the L2ARC and Encryption etc. I've also been discussing ideas with Brian about future-proofing, to handle the case where a newer version of ZSTD might compression the same input differently (better ratio), and how that would impact L2ARC, nop-write, etc. One idea (originally from Pawel Dawidek) is to do something similar to what encryption does, and split the checksum field. Using half to checksum the original data, and half the compressed version. This would allow ZFS to detect when the same content compressed differently (combined with the ZSTD version header in the compressed data), giving better compatibility as we upgrade ZSTD. This project is sponsored by the FreeBSD Foundation. On 2020-07-29 21:10, Allan Jude wrote: > This is the sixth weekly status report on the project to integrate ZSTD= > into OpenZFS. >=20 > https://github.com/openzfs/zfs/pull/10631 - Improved the `zfs recv` > error handling when it receives an out-of-bounds property value. > Specifically, if a zfs send stream is created that supports a newer > compression or checksum type, the property will fail to be set on the > receiving system. This is fine, but `zfs recv` would abort() and create= > a core file, rather than reporting the error, because it did not > understand the EINVAL being returned for that property. In the case > where the property is outside the accepted range, we now return the new= > ZFS_ERR_BADPROP value, and the correct message is displayed to the user= =2E > I opted not to use ERANGE because that is used for 'this property value= > should not be used on a root pool'. The idea is to get this fix merged > into the 0.8.x branch for the next point release, to improve > compatibility with streams generated by OpenZFS 2.0 >=20 >=20 > https://github.com/openzfs/zfs/pull/10632 - General improvement to erro= r > handling when the error code is EZFS_UNKNOWN. >=20 >=20 > https://github.com/allanjude/zfs/commit/8f37c1ad8edaff20a550b3df07995da= b80c06492 > - ZFS replication compatibility improvements. As discussed on the > leadership call earlier this month, keep the compatibility simple. If > the -c flag is given, send blocks compressed with any compression > algorithm. The improved error handling will let the user know if their > system can't handle ZSTD. >=20 >=20 > https://github.com/allanjude/zfs/commit/0ffd80e281f79652973378599cd0332= 172f365bd > - per-dataset feature activation. This switches the ZSTD feature flag > from 'enabled' to 'active' as soon as the property is set, instead of > when the first block is written. This ensures that the pool can't be > imported on a system that does not support ZSTD that will cause the ZFS= > cli tools to panic. >=20 >=20 > I will be working on adding some tests for the feature activation. >=20 > I've been looking at ways to add tests for the replication changes, but= > it doesn't seem to be easy to test the results of a 'zfs recv' that doe= s > not know about ZSTD (where the values are outside of the valid range fo= r > the enum). If anyone has any ideas here, I'd be very interested. >=20 >=20 > On 2020-07-20 23:40, Allan Jude wrote: >> This is the fifth weekly status report on the project to integrate ZST= D >> into OpenZFS. >> >> https://github.com/c0d3z3r0/zfs/pull/14/commits/9807c99169e5931a754bb0= df68267ffa2f289474 >> - Created a new test case to ensure that ZSTD compressed blocks surviv= e >> replication with the -c flag. We wanted to make sure the on-disk >> compression header survived the trip. >> >> https://github.com/c0d3z3r0/zfs/pull/14/commits/94bef464fc304e9d6f5850= 391e41720c3955af11 >> - I split the zstd.c file into OS specific bits >> (module/os/{linux,freebsd}/zstd_os.c) and also split the .h file into >> zstd.h and zstd_impl.h. This was done so that FreeBSD can use the >> existing kmem_cache mechanism, while Linux can use the vmem_alloc pool= >> created in the earlier versions of this patch. I significantly changed= >> the FreeBSD implementation from my earlier work, to reuse the power of= 2 >> zio_data_buf_cache[]'s that already exist, only adding a few additiona= l >> kmem_caches for large blocks with high compression levels. This should= >> avoid creating as many unnecessary kmem caches. >> >> https://github.com/c0d3z3r0/zfs/pull/14/commits/3d48243b77e6c8c3bf562c= 7a2315dd6cc571f28c >> - Lastly, in my testing I was seeing a lot of hits on the new >> compression failure kstat I added. This was caused by the ZFS "early >> abort" feature, where we give the compressor an output buffer that is >> smaller than the input, so it will fail if the block will not compress= >> enough to be worth it. This helps avoid wasting CPU on uncompressible >> blocks. However, it seems the 'one-file' version of zstd we are using >> does not expose the ZSTD_ErrorCode enum. This needs to be investigated= >> further to avoid issues if the value changes (although it is apparentl= y >> stable after version 1.3.1). >> >> I am still working on a solution for zfs send stream compatibility. I = am >> leaning towards creating a new flag, --zstd, to enable ZSTD compressed= >> output. If the -e or -c flag are used without the --zstd flag, and the= >> dataset has the zstd feature active, the idea would be to emit a warni= ng >> but send the blocks uncompressed, so that the stream remains compatibl= e >> with older versions of ZFS. I will be discussing this on the OpenZFS >> Leadership call tomorrow, and am open to suggestions on how to best >> handle this. >> >> >> On 2020-07-14 22:26, Allan Jude wrote: >>> In my continuing effort to complete the integration of ZSTD into >>> OpenZFS, here is my fourth weekly status report: >>> >>> https://github.com/allanjude/zfs/commit/b0b1270d4e7835ecff41320830137= 5e3de2a4153 >>> - Create a new test case to make sure that the ZSTD header we write >>> along with the data is correct. Verify that the physical size of the >>> compressed data is less than the psize for the block pointer, and ver= ify >>> that the level matches. It uses a random level between 1 and 19 and t= hen >>> verifies with zdb that the block was compressed with that level. >>> >>> I am still working on a solution for setting the zstd feature flag to= >>> 'active' as soon as it is set, rather than only once a block is born.= As >>> well as fixing up compatibility around zfs send/recv with the embedde= d >>> block points flag. >>> >>> This project is sponsored by the FreeBSD Foundation. >>> >>> >> >=20 >=20 --=20 Allan Jude --UkN47Zpoz8FxP3sgMe7uV1jD4vlSCfYuM-- --R8vzuaKLhzyCjf7STmUzjL8AXKZS0tooK Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJfK2/NAAoJEBmVNT4SmAt+1NkQAIhyISCT+s2BErkniQp2h6vQ x0ecSB4zIJt6dpK80GJ5lBYpgzOSFnSyvtfOItt4nXTbpa2nklyKiuNiyAP17P1W zc/2A6M3hQBmCBBgZwEL8FA51RtEEZmosgMCF5XYI42WH0UFJZlmx1jogGZ9IOT8 AeK9UPGSwjOV+UZggJHqmnBkO4pAExnRZ4xBLM6154BT6vSBEG+eM7qO9L9uwCvZ 6NwSAroFclxA9TFjbVm1Tb+GLgipPsK64DI2yaPalML3C/uJ236LFWV7I4Ynvee6 GDQhR0Tf1XE1c7D+1SgXMQ0u3fPC703zJmJ3sUjZXhvFrhKsycaq3SzmGIJ9tchi Qu36SXJ3W8BKOQb+a1VyYt8yuRZcpp1FLndDHZw+gmQiGBkQ4cQ9kzPYkHNA1uPv XMOVVrF95jlYwJ8TOqd0S1uFReNvB3GwJHjm7aWYDbwhU2vReOWR5rYktS0moosG BG5tDT20JyXq7LcutH979yivRM5SFltMw3VF66nPWPwDEeYWalCqFmt38YJz2iep 4Tx1g8+PDGTUt18kiPLy8gGvSyJdTG0s87rfDKSbiQEW9mnt4VM/MF2PKMInH3+Q yyTlkXTrxv+19fdJvEybsRajrvO2gddG+Hg40FNPrwqBU0g24xxQFF/CIW4ijvNU BNMSQjp9zGGXUeVqiBNr =ryYQ -----END PGP SIGNATURE----- --R8vzuaKLhzyCjf7STmUzjL8AXKZS0tooK--