From owner-freebsd-fs@freebsd.org Fri Jan 4 10:52:51 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 966A0142598A for ; Fri, 4 Jan 2019 10:52:51 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from cu1176c.smtpx.saremail.com (cu1176c.smtpx.saremail.com [195.16.148.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 744E08AE9D for ; Fri, 4 Jan 2019 10:52:50 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from [172.16.8.5] (unknown [192.148.167.11]) by proxypop02.sare.net (Postfix) with ESMTPA id 05C289DC4E3; Fri, 4 Jan 2019 11:52:39 +0100 (CET) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Re: Interesting: ZFS scrub prefetch hurting sequential scrub performance? From: Borja Marcos In-Reply-To: <8ECF7513-9DFB-46EF-86BA-DB717D713792@sarenet.es> Date: Fri, 4 Jan 2019 11:52:37 +0100 Cc: freebsd-fs@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <8ECF7513-9DFB-46EF-86BA-DB717D713792@sarenet.es> To: Borja Marcos X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: 744E08AE9D X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of borjam@sarenet.es designates 195.16.148.151 as permitted sender) smtp.mailfrom=borjam@sarenet.es X-Spamd-Result: default: False [-0.66 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.63)[-0.631,0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:195.16.148.0/24]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[sarenet.es]; NEURAL_HAM_LONG(-0.79)[-0.791,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[smtp.sarenet.es,smtp.sarenet.es,smtp.sarenet.es]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[151.148.16.195.list.dnswl.org : 127.0.10.0]; NEURAL_HAM_SHORT(-0.44)[-0.441,0]; SUBJECT_ENDS_QUESTION(1.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:3262, ipnet:195.16.128.0/19, country:ES]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.01)[country: ES(0.05)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jan 2019 10:52:51 -0000 > On 3 Jan 2019, at 11:34, Borja Marcos wrote: >=20 >=20 > Hi, >=20 > I have noticed that my scrubs have become painfully slow. I am = wondering wether I=E2=80=99ve just hit some worst case or maybe > there is some interaction between the ZFS sequential scrub and scrub = prefetch. I don=E2=80=99t recall seeing this behavior > before the sequential scrub code was committed.=20 >=20 > Did I hit some worst case or should scrub prefetch be disabled with = the new sequential scrub code? I have done a test with the old scrub code (vfs.zfs.zfs_scan_legacy=3D1) = and I see a very similar behavior, with the=20 scrub stalling again. Once more, disabling prefetch for the scrub = (vfs.zfs.no_scrub_prefetch=3D1) solves the issue. I suffered this problem on 11 at some point but I attributed it = (wrongly!) to hardware problems at the time. Not I=E2=80=99ve just found a talk about a new prefetch mechanism for = the scrub by Tom Caputi. Could it be the problem? https://www.youtube.com/watch?v=3Dupn9tYh917s Borja.