From owner-freebsd-fs@freebsd.org Thu Apr 28 07:52:49 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BEBF8B1FF05 for ; Thu, 28 Apr 2016 07:52:49 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 803341E52 for ; Thu, 28 Apr 2016 07:52:48 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 60AA82840C; Thu, 28 Apr 2016 09:52:46 +0200 (CEST) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 98FFA28417; Thu, 28 Apr 2016 09:52:45 +0200 (CEST) Message-ID: <5721C14D.8@quip.cz> Date: Thu, 28 Apr 2016 09:52:45 +0200 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:35.0) Gecko/20100101 Firefox/35.0 SeaMonkey/2.32 MIME-Version: 1.0 To: Steven Hartland , freebsd-fs@freebsd.org, Adam Nowacki Subject: Re: How to speed up slow zpool scrub? References: <698816653.2698619.1461685653634.JavaMail.yahoo.ref@mail.yahoo.com> <698816653.2698619.1461685653634.JavaMail.yahoo@mail.yahoo.com> <571F9897.2070008@quip.cz> <571FEB34.7040305@andyit.com.au> <56C0A956-F134-4A8D-A8B6-B93DCA045BE4@pk1048.com> <084201d1a03e$d2158fe0$7640afa0$@andyit.com.au> <5720AAF8.4090900@quip.cz> <5720F890.3040600@platinum.linux.pl> <572100C9.8010606@quip.cz> <57211EFF.4000000@platinum.linux.pl> <572124CC.6050808@multiplay.co.uk> In-Reply-To: <572124CC.6050808@multiplay.co.uk> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Apr 2016 07:52:49 -0000 Steven Hartland wrote on 04/27/2016 22:45: > > > On 27/04/2016 21:20, Adam Nowacki wrote: [...] >>>> 2) some data blocks do not span all disks (metadata, small files, >>>> compression) >>>> End result is that zfs can't always read from all disks during scrub so >>>> disk utilization is going to be less than 100% even when going at full >>>> speed. >>> Thank you for the explanation. >> Try increasing vfs.zfs.top_maxinflight to 100 or more. >> > If your on a recent version vfs.zfs.vdev.scrub_max_active is usually the > most pertinent change to make. Thank you to both of you. It seems that vfs.zfs.top_maxinflight=128 did higher disk utilization (about 90%). Scrubbing is over after 4 days so I will play with these sysctls next time # zpool status -v tank0 pool: tank0 state: ONLINE scan: scrub repaired 0 in 96h27m with 0 errors on Thu Apr 28 03:29:21 2016 config: NAME STATE READ WRITE CKSUM tank0 ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 gpt/disk0tank0 ONLINE 0 0 0 gpt/disk1tank0 ONLINE 0 0 0 gpt/disk2tank0 ONLINE 0 0 0 gpt/disk3tank0 ONLINE 0 0 0 errors: No known data errors Thank you! Miroslav Lachman