Date: Sat, 11 May 2019 00:53:38 +1000 From: Michelle Sullivan <michelle@sorbs.net> To: Miroslav Lachman <000.fbsd@quip.cz>, Alan Somers <asomers@freebsd.org> Cc: Dimitry Andric <dim@freebsd.org>, FreeBSD-STABLE Mailing List <freebsd-stable@freebsd.org> Subject: Re: ZFS... Message-ID: <85f44376-32eb-885e-1eb8-d3a2e204bc84@sorbs.net> In-Reply-To: <8e443083-1254-520b-014d-2f9a94008533@quip.cz> References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <2e4941bf-999a-7f16-f4fe-1a520f2187c0@sorbs.net> <20190430102024.E84286@mulder.mintsol.com> <41FA461B-40AE-4D34-B280-214B5C5868B5@punkt.de> <20190506080804.Y87441@mulder.mintsol.com> <08E46EBF-154F-4670-B411-482DCE6F395D@sorbs.net> <33D7EFC4-5C15-4FE0-970B-E6034EF80BEF@gromit.dlib.vt.edu> <A535026E-F9F6-4BBA-8287-87EFD02CF207@sorbs.net> <a82bfabe-a8c3-fd9a-55ec-52530d4eafff@denninger.net> <a1b78a63-0ef1-af51-4e33-a9a97a257c8b@sorbs.net> <CAMPTd_A7RYJ12pFyY4TzbXct82kWfr1hcEkSpDg7bjP25xjJGA@mail.gmail.com> <d91cf5@sorbs.net> <7D18A234-E7BF-4855-BD51-4AE2253DB1E4@sorbs.net> <E68600B3-F856-4909-AB6E-BDFCD8AAAB43@punkt.de> <805ee7f1-83f6-c59e-8107-4851ca9fce6e@quip.cz> <E980141F-48D9-4870-8FE1-9A5610F12826@FreeBSD.org> <5de7f3d3-b34c-0382-b7d4-b7e38339649b@quip.cz> <CAOtMX2ip_G0afh_4AaYcQt4=zX4qkCTfs0qb66DpCgcALZXS5g@mail.gmail.com> <8e443083-1254-520b-014d-2f9a94008533@quip.cz>
next in thread | previous in thread | raw e-mail | index | archive | help
Miroslav Lachman wrote: > Alan Somers wrote on 2019/05/09 14:50: > > [...] > >> On 11.3 and even much older releases, you can greatly speed up scrub >> and resilver by tweaking some sysctls. If you have spinning rust, >> raise vfs.zfs.top_maxinflight so they'll do fewer seeks. I used to >> set it to 8192 on machines with 32GB of RAM. Raising >> vfs.zfs.resilver_min_time_ms to 5000 helps a little, too. > > I have this in sysctl.conf > vfs.zfs.scrub_delay=0 > vfs.zfs.top_maxinflight=128 > vfs.zfs.resilver_min_time_ms=5000 > vfs.zfs.resilver_delay=0 > > I found it somewhere in the mailinglist discussing this issue in the > past. > > Isn't yours 8192 too much? The machine in question has 4x SATA drives > on very dump and slow controller and only 5GB of RAM. > > Even if I read this > vfs.zfs.top_maxinflight: Maximum I/Os per top-level vdev > I am still not sure what it really means and how I can "calculate" > optimal value. I calculated it by looking at the iops using gstat and then multiplied it by the spindles in use. That seemed to give the optimum. Much lower and the drives had idle time.... to much and it causes 'pauses' whilst the writes happen. "Tuning" for me had no fine tuning, it seems very sledgehammerish ... big changes are noticeable for better or worse.. small changes you cannot tell by eye, and I guess measuring known operations (such as a controlled environment scrub) might show differing results but I suspect with other things going on these negligible changes are likely to be useless. > > As Michelle pointed there is drawback when sysctls are optimized for > quick scrub, but this machines is only running nightly backup script > fetching data from other 20 machines so this scrip sets sysctl back to > sane defaults during backup not really.. optimize for scrub should only affect the system whilst the scrub and resilvers are actually happening,.. the rest of the time my systems were not affected (noticeably) my problem was a scrub would kick off and last a couple of weeks (1 week heavily preferrencing the scrub) causing the video streaming to become stuttery .. which watching a movie whilst this was happening was really not good... especially as it wasn't for just a few seconds/minutes/hours. > sysctl vfs.zfs.scrub_delay=4 > /dev/null > sysctl vfs.zfs.top_maxinflight=32 > /dev/null > sysctl vfs.zfs.resilver_min_time_ms=3000 > /dev/null > sysctl vfs.zfs.resilver_delay=2 > /dev/null > > At the and it reloads back optimized settings from sysctel.conf > > Miroslav Lachman > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" -- Michelle Sullivan http://www.mhix.org/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?85f44376-32eb-885e-1eb8-d3a2e204bc84>