Date: Wed, 08 Jul 2015 09:22:46 -0500 From: Karl Denninger <karl@denninger.net> To: freebsd-fs@freebsd.org Subject: Re: Speeding up resilvering Message-ID: <559D3236.1060102@denninger.net> In-Reply-To: <559D3167.1000705@infracaninophile.co.uk> References: <559D2AB6.5070007@FreeBSD.org> <559D2C7C.6060201@denninger.net> <559D3167.1000705@infracaninophile.co.uk>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] On 7/8/2015 09:19, Matthew Seaman wrote: > On 2015/07/08 14:58, Karl Denninger wrote: >> On 7/8/2015 08:50, Matthew Seaman wrote: >>> Hi, >>> >>> I've a zpool which is taking an inordinately long time to resilver and >>> replace a device. It had only got to about 8% completion after a day, >>> implying over a week to resilver about 6TB data. >>> >>> Now, I've applied the resilver performance tuning sysctls from >>> https://wiki.freebsd.org/ZFSTuningGuide: >>> >>> vfs.zfs.scrub_delay=0 >>> vfs.zfs.top_maxinflight=128 >>> vfs.zfs.resilver_min_time_ms=5000 >>> vfs.zfs.resilver_delay=0 >>> >>> but it doesn't seem to have made a great deal of difference. Most of >>> the benefit there would apparently come from reducing scrub_delay or >>> resilver_delay -- but this system is only booted to single user mode, so >>> the drives are otherwise idle and the resilver should have automatically >>> switched to running full throttle anyhow. >>> >>> Given the machine is not going to be doing anything else other than >>> resilvering for the time being, is there any more aggressive tuning or >>> tricks to get it to go faster that people would recommend? >>> >>> Cheers, >>> >>> Matthew >>> >> What OS version? >> > 10.1-RELEASE-p10 > > Cheers, > > Matthew Look at the IO saturation on the disk channel(s) involved with either systat -vm or iostat. If the channel is saturated then there's nothing you can do in terms of tuning; the question then turns to why actual I/O performance is so poor and has to be addressed there. -- Karl Denninger karl@denninger.net <mailto:karl@denninger.net> /The Market Ticker/ /[S/MIME encrypted email preferred]/ [-- Attachment #2 --] 0 *H 01 0 + 0 *H _0[0C)0 *H 01 0 UUS10UFlorida10U Niceville10U Cuda Systems LLC1 0UCuda Systems LLC CA1"0 *H Cuda Systems LLC CA0 150421022159Z 200419022159Z0Z1 0 UUS10UFlorida10U Cuda Systems LLC1 0 UKarl Denninger (OCSP)0"0 *H 0 X@vkY Tq/vE]5#֯MX \8LJ/V?5Da+ sJc*/r{ȼnS+ w")ąZ^DtdCOZ ~7Q '@a#ijc۴oZdB&!Ӝ-< ? HN5y 5}F|e f"V لio74zn">a1qWuɖbFeGE&3(KhixG3!#e_XƬ Ϝ/,$+;4y 'Bz<qT9_?rRUpn5 Jn&Rx/p Jyel*pN8/#9u/YP EC)TY>~/˘N[vyiDKˉ ,^" ?$T8 v& K%z8C @?K{9f`+@,| Mbia 007++0)0'+0http://cudasystems.net:88880 U 0 0 `HB0 U 0, `HB OpenSSL Generated Certificate0 U -h\Ff Y0 U #0$q}ݽʒm50 U 0karl@denninger.net0 *H Owbabɺx&Uk[ (Oj!%p MQ0I!#QH}.>~2&D}<wm_>V6 v ]f>=Nn+8;q wfΰ/RLyUG#b}n!Dր _up|_ǰc/%ۥ nN8:d;-UJd/m1~VނיnN I˾$tF1&}|?q?\đXԑ&\4V<lKۮ3%Am_(q -(cAeGX)f}-˥6cv~Kg8m~v ;|9:-iAPқ6ېn-.)<[$KJtt/L4ᖣ^Cmu4vb{+BG$M0c\[MR|0FԸP&78"4p# }D Z9;V9#>Sw"[UP710001 0 UUS10UFlorida10U Niceville10U Cuda Systems LLC1 0UCuda Systems LLC CA1"0 *H Cuda Systems LLC CA)0 + !0 *H 1 *H 0 *H 1 150708142246Z0# *H 1wZ; WT}H7x0l *H 1_0]0 `He*0 `He0 *H 0*H 0 *H @0+0 *H (0 +71001 0 UUS10UFlorida10U Niceville10U Cuda Systems LLC1 0UCuda Systems LLC CA1"0 *H Cuda Systems LLC CA)0 *H 101 0 UUS10UFlorida10U Niceville10U Cuda Systems LLC1 0UCuda Systems LLC CA1"0 *H Cuda Systems LLC CA)0 *H sCjeVDKUG懯C`} @"n#CXW4mC{dôILrDڕ/o 55oʆB;}efa P{'^DIg$ *7H8 `ͼ*2TAfu LE\PԺ]55@1 _RǁawIM#xXkdh,JYpO@%X[*da]}'\c0M" &OW,=;qAǡw!3 rssG3ğW< h "acꃅiJ͠}N꺯l ؟ @p^M܆s<+CZ]f`oQׄ S`MTr pH|+ӷ %hnX3N %Q7n9d2#ڝR }Oٗf]q:X͂[TƼb" 7`d4\qYhelp
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?559D3236.1060102>