From owner-freebsd-fs@freebsd.org Tue Apr 26 13:35:38 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D9D43B1D4B0 for ; Tue, 26 Apr 2016 13:35:38 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 65D731BC7 for ; Tue, 26 Apr 2016 13:35:37 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id CB4E328430; Tue, 26 Apr 2016 15:35:34 +0200 (CEST) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 114C428426; Tue, 26 Apr 2016 15:35:33 +0200 (CEST) Message-ID: <571F6EA4.90800@quip.cz> Date: Tue, 26 Apr 2016 15:35:32 +0200 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:35.0) Gecko/20100101 Firefox/35.0 SeaMonkey/2.32 MIME-Version: 1.0 To: jg@internetx.com, freebsd-fs@freebsd.org Subject: Re: How to speed up slow zpool scrub? References: <571F62AD.6080005@quip.cz> <571F687D.8040103@internetx.com> In-Reply-To: <571F687D.8040103@internetx.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Apr 2016 13:35:38 -0000 InterNetX - Juergen Gotteswinter wrote on 04/26/2016 15:09: > to speed up the scrub itself you can try > > sysctl -w vfs.zfs.scrub_delay = 4 (default, 0 means higher prio) I will try it in the idle times > but be careful as this can cause a serious performance impact, the value > can be changed on the fly > > your pool is raidz, mirror ? dedup is hopefully disabled? I forgot to mention it. Disks are partitioned to four partitions: # gpart show -l ada0 => 34 7814037101 ada0 GPT (3.6T) 34 6 - free - (3.0K) 40 1024 1 boot0 (512K) 1064 10485760 2 swap0 (5.0G) 10486824 31457280 3 disk0sys (15G) 41944104 7769948160 4 disk0tank0 (3.6T) 7811892264 2144871 - free - (1.0G) diskXsys partitions are used for base system pool which is 4-way mirror diskXtank0 partitions are used for data storage as RAIDZ # zpool list NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT sys 14.9G 11.0G 3.92G - 79% 73% 1.00x ONLINE - tank0 14.4T 10.8T 3.56T - 19% 75% 1.00x ONLINE - # zpool status -v pool: sys state: ONLINE scan: scrub repaired 0 in 1h2m with 0 errors on Sun Apr 24 04:03:54 2016 config: NAME STATE READ WRITE CKSUM sys ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gpt/disk0sys ONLINE 0 0 0 gpt/disk1sys ONLINE 0 0 0 gpt/disk2sys ONLINE 0 0 0 gpt/disk3sys ONLINE 0 0 0 errors: No known data errors pool: tank0 state: ONLINE scan: scrub in progress since Sun Apr 24 03:01:35 2016 7.63T scanned out of 10.6T at 36.7M/s, 23h32m to go 0 repaired, 71.98% done config: NAME STATE READ WRITE CKSUM tank0 ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 gpt/disk0tank0 ONLINE 0 0 0 gpt/disk1tank0 ONLINE 0 0 0 gpt/disk2tank0 ONLINE 0 0 0 gpt/disk3tank0 ONLINE 0 0 0 errors: No known data errors # zdb | grep ashift ashift: 12 ashift: 12 Thank you for your informations. > Am 4/26/2016 um 2:44 PM schrieb Miroslav Lachman: >> Hi, >> >> is there any way to make zpool scrub faster? >> We have one older machine with CPU Pentium(R) Dual E2160 @1.80GHz, 5GB >> of RAM and 4x 4TB HDDs. It is just a storage for backups for about 20 >> machines. >> Scrub is scheduled from periodic each 30 days but it takes about 4 days >> to complete and everything during scrub is slow. Backups takes 8 hours >> instead of 5 (made by rsync), deleting of old files is even more slower. >> >> The backups are made every night from the midnight to morning, the >> machine is idle for the rest of the day. >> >> Is there any tuning to make scrub faster in this idle time? >> Or is it better to do it other way - slower scrub with even lower >> priority taking for about one week but not affecting time of normal >> operations? (is it dangerous to have scrub running this long or reboot >> machine during the scrub?) >> >> I have a performance graphs of this machine and CPU is about 70% idle >> during scrub, but hard drives are busy 75% (according to iostat) >> >> FreeBSD 10.3-RELEASE amd64 GENERIC >> >> Miroslav Lachman >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >