Date: Wed, 27 Apr 2016 22:20:15 +0200 From: Adam Nowacki <nowakpl@platinum.linux.pl> To: Miroslav Lachman <000.fbsd@quip.cz>, freebsd-fs@freebsd.org Subject: Re: How to speed up slow zpool scrub? Message-ID: <57211EFF.4000000@platinum.linux.pl> In-Reply-To: <572100C9.8010606@quip.cz> References: <698816653.2698619.1461685653634.JavaMail.yahoo.ref@mail.yahoo.com> <698816653.2698619.1461685653634.JavaMail.yahoo@mail.yahoo.com> <571F9897.2070008@quip.cz> <571FEB34.7040305@andyit.com.au> <56C0A956-F134-4A8D-A8B6-B93DCA045BE4@pk1048.com> <084201d1a03e$d2158fe0$7640afa0$@andyit.com.au> <5720AAF8.4090900@quip.cz> <5720F890.3040600@platinum.linux.pl> <572100C9.8010606@quip.cz>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2016-04-27 20:11, Miroslav Lachman wrote: > Adam Nowacki wrote on 04/27/2016 19:36: >> On 2016-04-27 14:05, Miroslav Lachman wrote: > >>> Thank you for output of your zpool scrub. It is definitely faster than >>> mine. >>> >>> To: Paul pk1048 >>> Mine scrub does not repair anything. Drives are OK (in SMART). >>> CPU is about 70%-90% idle during scrub + rsync backup and drives are >>> about 60%-70% busy according to iostat: >>> >>> root@kiwi ~/# iostat -x -w 10 ada0 ada1 ada2 ada3 > >>> device r/s w/s kr/s kw/s qlen svc_t %b >>> ada0 121.8 16.6 602.1 526.9 3 9.2 52 >>> ada1 122.2 16.5 606.9 528.5 4 9.8 54 >>> ada2 117.0 14.6 601.7 524.9 2 11.3 60 >>> ada3 120.6 13.5 610.1 491.3 0 11.4 61 >>> >>> I really don't know why it cannot go faster if nothing is loaded for >>> 100%. >> >> 1) zpool scrub is single threaded with prefetch, > > Hmm, this can be the cause. Does it mean that ZFS is faster on CPU with > higher "per core" power and number of cores (threads) is not so > important? (in this case of scrub) No. Zpool scrub thread doesn't need much CPU time as disk I/O handling (including decompression and checkums) happens in other threads. >> 2) some data blocks do not span all disks (metadata, small files, >> compression) >> End result is that zfs can't always read from all disks during scrub so >> disk utilization is going to be less than 100% even when going at full >> speed. > > Thank you for the explanation. Try increasing vfs.zfs.top_maxinflight to 100 or more.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?57211EFF.4000000>