Date: Thu, 14 Dec 2017 14:43:24 +0100 From: "O. Hartmann" <ohartmann@walstatt.org> To: Daniel Nebdal <dnebdal@gmail.com> Cc: "O. Hartmann" <ohartmann@walstatt.org>, FreeBSD CURRENT <freebsd-current@freebsd.org> Subject: Re: ZFS RAIDZ1: resilvering at <17.3M/s => abyssal slow ... Message-ID: <20171214144351.24a81faa@thor.intern.walstatt.dynvpn.de> In-Reply-To: <CA%2Bt49PJuznLxGnLERwAhVW2CjETJYWO6rKcWo3qOo5bZLQkqYA@mail.gmail.com> References: <20171214124900.64211bd9@thor.intern.walstatt.dynvpn.de> <CA%2Bt49PJuznLxGnLERwAhVW2CjETJYWO6rKcWo3qOo5bZLQkqYA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] Am Thu, 14 Dec 2017 14:09:39 +0100 Daniel Nebdal <dnebdal@gmail.com> schrieb: > On Thu, Dec 14, 2017 at 12:48 PM, O. Hartmann <ohartmann@walstatt.org> wrote: > > Hello out there, > > > > I had to replace a HDD on a RAIDZ1 pool comprised from 4 HDDs, each 3 TB (netto size > > of the pool is about 8-9 TB, the used amount is reported to be ~ 6TB). > > > > I just started the rebuild/resilvering process and watch the pool crwaling at ~ 18 > > MB/s. At the moment, there is no load on the array, the host is a IvyBridge XEON with > > 4 core/8 threads and 3,4 GHz and 16 GB of RAM. The HDDs are attached to a on-board > > SATA II (300 MB/s max) Intel chip - this just for the record. > > > > Recently, I switch on the "sync" attribute on most of the defined pools's zfs > > filesystems > > - I also use a SSD for ZIL/L2ARC caching, but it seems to be unused recently in > > FreeBSD CURRENT's ZFS - this from a observers perspective only. > > > > When scrubbing, I see recently also reduced performance on the pool, so I'm wondering > > about the low throughput at the very moment when resilvering is in progress. > > > > If the "perspective" of "zpool status" is correct, then I have to wait after two hours > > for another 100 hours - ~ 4 days? Ups ... I think there is something badly > > misconfigured or missing. > > > > The pool is not "tuned" in any very sophisticated way, since I trust the kernels > > ability to scale automatically - if I'm not wrong for amd64 ... > > > > Are there any aspects to look at? > > > > Thanks in advance, > > > > Oliver > > > > -- > > O. Hartmann > > > > Ich widerspreche der Nutzung oder Übermittlung meiner Daten für > > Werbezwecke oder für die Markt- oder Meinungsforschung (§ 28 Abs. 4 BDSG). > > > This is kind of to be expected - for whatever reason, resilvers seem > to go super slow at first and then speed up significantly. Just don't > ask me how long "at first" is - I'd give it several (more) hours. > I don't _think_ sync affects resilver speed (since that's a filesystem > level setting and the resilver happens at the block level), but that's > pure speculation. > The Tuning guide at https://wiki.freebsd.org/ZFSTuningGuide also > suggests a list of sysctls you can tweak, which could be worth a try: > > ### wiki > If you're getting horrible performance during a scrub or resilver, the > following sysctls can be set: > > vfs.zfs.scrub_delay=0 > vfs.zfs.top_maxinflight=128 > vfs.zfs.resilver_min_time_ms=5000 > vfs.zfs.resilver_delay=0 > > Setting those sysctls to those values increased my (Shawn Webb's) > resilver performance from 7MB/s to 230MB/s. > ### > > > I had already set those tuning parameters, shortly after I sent the mail to the list, but there wasn't any sign of increasing performance since then. Now, after roughly 4 hours of running resilvering, it seems that the speed/throughput is increasing: from initially 12 - 17 MB/s its now at ~ 90 MBytes/s - sounds more promising. I remember that I whitnessed this also when I checked the last time scrubbing: a non linear gradient in throughput while time shifts forward ... So, everything seems to be all right ... Kind regards, Oliver -- O. Hartmann Ich widerspreche der Nutzung oder Übermittlung meiner Daten für Werbezwecke oder für die Markt- oder Meinungsforschung (§ 28 Abs. 4 BDSG). [-- Attachment #2 --] -----BEGIN PGP SIGNATURE----- iLUEARMKAB0WIQQZVZMzAtwC2T/86TrS528fyFhYlAUCWjKAFwAKCRDS528fyFhY lKbnAf9kPSkE5QQPzBcIJIRC+CCQbf4Sggo8Te4DkufQR4z70UqaWlDGYZ+Rg1QE Pe9Xdfw2BUSuzYcKf2t1Kn8HjDDtAf9UubfzRat+MI96OqkwT4fmuDxfa3DIxURW xry6hFJ6/cKmSsrw0T//PYv9HoWofvVc/OUe+DplawUpwSjbdusB =6mYM -----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20171214144351.24a81faa>