Date: Wed, 27 Apr 2016 21:45:00 +0100 From: Steven Hartland <killing@multiplay.co.uk> To: freebsd-fs@freebsd.org Subject: Re: How to speed up slow zpool scrub? Message-ID: <572124CC.6050808@multiplay.co.uk> In-Reply-To: <57211EFF.4000000@platinum.linux.pl> References: <698816653.2698619.1461685653634.JavaMail.yahoo.ref@mail.yahoo.com> <698816653.2698619.1461685653634.JavaMail.yahoo@mail.yahoo.com> <571F9897.2070008@quip.cz> <571FEB34.7040305@andyit.com.au> <56C0A956-F134-4A8D-A8B6-B93DCA045BE4@pk1048.com> <084201d1a03e$d2158fe0$7640afa0$@andyit.com.au> <5720AAF8.4090900@quip.cz> <5720F890.3040600@platinum.linux.pl> <572100C9.8010606@quip.cz> <57211EFF.4000000@platinum.linux.pl>
next in thread | previous in thread | raw e-mail | index | archive | help
On 27/04/2016 21:20, Adam Nowacki wrote:
> On 2016-04-27 20:11, Miroslav Lachman wrote:
>> Adam Nowacki wrote on 04/27/2016 19:36:
>>> On 2016-04-27 14:05, Miroslav Lachman wrote:
>>>> Thank you for output of your zpool scrub. It is definitely faster than
>>>> mine.
>>>>
>>>> To: Paul pk1048
>>>> Mine scrub does not repair anything. Drives are OK (in SMART).
>>>> CPU is about 70%-90% idle during scrub + rsync backup and drives are
>>>> about 60%-70% busy according to iostat:
>>>>
>>>> root@kiwi ~/# iostat -x -w 10 ada0 ada1 ada2 ada3
>>>> device r/s w/s kr/s kw/s qlen svc_t %b
>>>> ada0 121.8 16.6 602.1 526.9 3 9.2 52
>>>> ada1 122.2 16.5 606.9 528.5 4 9.8 54
>>>> ada2 117.0 14.6 601.7 524.9 2 11.3 60
>>>> ada3 120.6 13.5 610.1 491.3 0 11.4 61
>>>>
>>>> I really don't know why it cannot go faster if nothing is loaded for
>>>> 100%.
>>> 1) zpool scrub is single threaded with prefetch,
>> Hmm, this can be the cause. Does it mean that ZFS is faster on CPU with
>> higher "per core" power and number of cores (threads) is not so
>> important? (in this case of scrub)
> No. Zpool scrub thread doesn't need much CPU time as disk I/O handling
> (including decompression and checkums) happens in other threads.
>
>>> 2) some data blocks do not span all disks (metadata, small files,
>>> compression)
>>> End result is that zfs can't always read from all disks during scrub so
>>> disk utilization is going to be less than 100% even when going at full
>>> speed.
>> Thank you for the explanation.
> Try increasing vfs.zfs.top_maxinflight to 100 or more.
>
If your on a recent version vfs.zfs.vdev.scrub_max_active is usually the
most pertinent change to make.
Regards
Steve
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?572124CC.6050808>