Date: Tue, 26 Apr 2016 15:02:29 +0100 From: krad <kraduk@gmail.com> To: Miroslav Lachman <000.fbsd@quip.cz> Cc: jg@internetx.com, FreeBSD FS <freebsd-fs@freebsd.org> Subject: Re: How to speed up slow zpool scrub? Message-ID: <CALfReyc9ki3-wEhkvvv=4TR28Nk0H0SvH8FAW=ev7H7Zu0oQoA@mail.gmail.com> In-Reply-To: <571F6EA4.90800@quip.cz> References: <571F62AD.6080005@quip.cz> <571F687D.8040103@internetx.com> <571F6EA4.90800@quip.cz>
next in thread | previous in thread | raw e-mail | index | archive | help
Erk, i would try to move your system off those data disks as you have two pools competing for the disk spindles. This is never ideal. You can by all means backup your os to those data pools but keep them on separate physical mediums. A couple of small SSD would do the trick nicely and could probably be added with no down time. You would probably want to find a suitable window though to make sure the box reboots nicely though. On 26 April 2016 at 14:35, Miroslav Lachman <000.fbsd@quip.cz> wrote: > InterNetX - Juergen Gotteswinter wrote on 04/26/2016 15:09: > >> to speed up the scrub itself you can try >> >> sysctl -w vfs.zfs.scrub_delay = 4 (default, 0 means higher prio) >> > > I will try it in the idle times > > but be careful as this can cause a serious performance impact, the value >> can be changed on the fly >> >> your pool is raidz, mirror ? dedup is hopefully disabled? >> > > I forgot to mention it. Disks are partitioned to four partitions: > > # gpart show -l ada0 > => 34 7814037101 ada0 GPT (3.6T) > 34 6 - free - (3.0K) > 40 1024 1 boot0 (512K) > 1064 10485760 2 swap0 (5.0G) > 10486824 31457280 3 disk0sys (15G) > 41944104 7769948160 4 disk0tank0 (3.6T) > 7811892264 2144871 - free - (1.0G) > > diskXsys partitions are used for base system pool which is 4-way mirror > > diskXtank0 partitions are used for data storage as RAIDZ > > # zpool list > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > sys 14.9G 11.0G 3.92G - 79% 73% 1.00x ONLINE - > tank0 14.4T 10.8T 3.56T - 19% 75% 1.00x ONLINE - > > > # zpool status -v > pool: sys > state: ONLINE > scan: scrub repaired 0 in 1h2m with 0 errors on Sun Apr 24 04:03:54 2016 > config: > > NAME STATE READ WRITE CKSUM > sys ONLINE 0 0 0 > mirror-0 ONLINE 0 0 0 > gpt/disk0sys ONLINE 0 0 0 > gpt/disk1sys ONLINE 0 0 0 > gpt/disk2sys ONLINE 0 0 0 > gpt/disk3sys ONLINE 0 0 0 > > errors: No known data errors > > pool: tank0 > state: ONLINE > scan: scrub in progress since Sun Apr 24 03:01:35 2016 > 7.63T scanned out of 10.6T at 36.7M/s, 23h32m to go > 0 repaired, 71.98% done > config: > > NAME STATE READ WRITE CKSUM > tank0 ONLINE 0 0 0 > raidz1-0 ONLINE 0 0 0 > gpt/disk0tank0 ONLINE 0 0 0 > gpt/disk1tank0 ONLINE 0 0 0 > gpt/disk2tank0 ONLINE 0 0 0 > gpt/disk3tank0 ONLINE 0 0 0 > > errors: No known data errors > > > # zdb | grep ashift > ashift: 12 > ashift: 12 > > > Thank you for your informations. > > > > Am 4/26/2016 um 2:44 PM schrieb Miroslav Lachman: >> >>> Hi, >>> >>> is there any way to make zpool scrub faster? >>> We have one older machine with CPU Pentium(R) Dual E2160 @1.80GHz, 5GB >>> of RAM and 4x 4TB HDDs. It is just a storage for backups for about 20 >>> machines. >>> Scrub is scheduled from periodic each 30 days but it takes about 4 days >>> to complete and everything during scrub is slow. Backups takes 8 hours >>> instead of 5 (made by rsync), deleting of old files is even more slower. >>> >>> The backups are made every night from the midnight to morning, the >>> machine is idle for the rest of the day. >>> >>> Is there any tuning to make scrub faster in this idle time? >>> Or is it better to do it other way - slower scrub with even lower >>> priority taking for about one week but not affecting time of normal >>> operations? (is it dangerous to have scrub running this long or reboot >>> machine during the scrub?) >>> >>> I have a performance graphs of this machine and CPU is about 70% idle >>> during scrub, but hard drives are busy 75% (according to iostat) >>> >>> FreeBSD 10.3-RELEASE amd64 GENERIC >>> >>> Miroslav Lachman >>> _______________________________________________ >>> freebsd-fs@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >>> >> >> > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CALfReyc9ki3-wEhkvvv=4TR28Nk0H0SvH8FAW=ev7H7Zu0oQoA>