Date: Tue, 26 Apr 2016 15:35:32 +0200 From: Miroslav Lachman <000.fbsd@quip.cz> To: jg@internetx.com, freebsd-fs@freebsd.org Subject: Re: How to speed up slow zpool scrub? Message-ID: <571F6EA4.90800@quip.cz> In-Reply-To: <571F687D.8040103@internetx.com> References: <571F62AD.6080005@quip.cz> <571F687D.8040103@internetx.com>
next in thread | previous in thread | raw e-mail | index | archive | help
InterNetX - Juergen Gotteswinter wrote on 04/26/2016 15:09:
> to speed up the scrub itself you can try
>
> sysctl -w vfs.zfs.scrub_delay = 4 (default, 0 means higher prio)
I will try it in the idle times
> but be careful as this can cause a serious performance impact, the value
> can be changed on the fly
>
> your pool is raidz, mirror ? dedup is hopefully disabled?
I forgot to mention it. Disks are partitioned to four partitions:
# gpart show -l ada0
=> 34 7814037101 ada0 GPT (3.6T)
34 6 - free - (3.0K)
40 1024 1 boot0 (512K)
1064 10485760 2 swap0 (5.0G)
10486824 31457280 3 disk0sys (15G)
41944104 7769948160 4 disk0tank0 (3.6T)
7811892264 2144871 - free - (1.0G)
diskXsys partitions are used for base system pool which is 4-way mirror
diskXtank0 partitions are used for data storage as RAIDZ
# zpool list
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
sys 14.9G 11.0G 3.92G - 79% 73% 1.00x ONLINE -
tank0 14.4T 10.8T 3.56T - 19% 75% 1.00x ONLINE -
# zpool status -v
pool: sys
state: ONLINE
scan: scrub repaired 0 in 1h2m with 0 errors on Sun Apr 24 04:03:54 2016
config:
NAME STATE READ WRITE CKSUM
sys ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
gpt/disk0sys ONLINE 0 0 0
gpt/disk1sys ONLINE 0 0 0
gpt/disk2sys ONLINE 0 0 0
gpt/disk3sys ONLINE 0 0 0
errors: No known data errors
pool: tank0
state: ONLINE
scan: scrub in progress since Sun Apr 24 03:01:35 2016
7.63T scanned out of 10.6T at 36.7M/s, 23h32m to go
0 repaired, 71.98% done
config:
NAME STATE READ WRITE CKSUM
tank0 ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
gpt/disk0tank0 ONLINE 0 0 0
gpt/disk1tank0 ONLINE 0 0 0
gpt/disk2tank0 ONLINE 0 0 0
gpt/disk3tank0 ONLINE 0 0 0
errors: No known data errors
# zdb | grep ashift
ashift: 12
ashift: 12
Thank you for your informations.
> Am 4/26/2016 um 2:44 PM schrieb Miroslav Lachman:
>> Hi,
>>
>> is there any way to make zpool scrub faster?
>> We have one older machine with CPU Pentium(R) Dual E2160 @1.80GHz, 5GB
>> of RAM and 4x 4TB HDDs. It is just a storage for backups for about 20
>> machines.
>> Scrub is scheduled from periodic each 30 days but it takes about 4 days
>> to complete and everything during scrub is slow. Backups takes 8 hours
>> instead of 5 (made by rsync), deleting of old files is even more slower.
>>
>> The backups are made every night from the midnight to morning, the
>> machine is idle for the rest of the day.
>>
>> Is there any tuning to make scrub faster in this idle time?
>> Or is it better to do it other way - slower scrub with even lower
>> priority taking for about one week but not affecting time of normal
>> operations? (is it dangerous to have scrub running this long or reboot
>> machine during the scrub?)
>>
>> I have a performance graphs of this machine and CPU is about 70% idle
>> during scrub, but hard drives are busy 75% (according to iostat)
>>
>> FreeBSD 10.3-RELEASE amd64 GENERIC
>>
>> Miroslav Lachman
>> _______________________________________________
>> freebsd-fs@freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?571F6EA4.90800>