Date: Wed, 06 Apr 2022 18:51:49 +0200
From: egoitz@ramattack.net
To: Eugene Grosbein <eugen@grosbein.net>
Cc: freebsd-fs@freebsd.org, freebsd-hackers@freebsd.org, Freebsd performance <freebsd-performance@freebsd.org>
Subject: Re: {* 05.00 *}Re: Desperate with 870 QVO and ZFS
Message-ID: <7aa95cb4bf1fd38b3fce93bc26826042@ramattack.net>
In-Reply-To: <ca3f86f2-94a1-be94-ad55-7bd1c9bc50ab@grosbein.net>
References: <4e98275152e23141eae40dbe7ba5571f@ramattack.net> <665236B1-8F61-4B0E-BD9B-7B501B8BD617@ultra-secure.de> <0ef282aee34b441f1991334e2edbcaec@ramattack.net> <28e11d7ec0ac5dbea45f9f271fc28f06@ramattack.net> <ca3f86f2-94a1-be94-ad55-7bd1c9bc50ab@grosbein.net>
next in thread | previous in thread | raw e-mail | index | archive | help
--=_be6199b7d5e868a9b11faca98e7631c5 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Hi Eugene!!! Thank you so much really again mate :) :) :) About your recommendations... Eugene, if some of them wouldn't be working as expected, could we revert some or all of them or perhaps some of your recommendations below need to be definitive?. I do answer below in green bold for better distinction :) :) El 2022-04-06 18:14, Eugene Grosbein escribió: > ATENCION > ATENCION > ATENCION!!! Este correo se ha enviado desde fuera de la organizacion. No pinche en los enlaces ni abra los adjuntos a no ser que reconozca el remitente y sepa que el contenido es seguro. > > 06.04.2022 22:30, egoitz@ramattack.net пишет: > >> One perhaps important note!! >> >> When this happens... almost all processes appear in top in the following state: >> >> txg state or >> >> txg-> >> >> bio.... >> >> perhaps should the the vfs.zfs.dirty_data_max, vfs.zfs.txg.timeout, vfs.zfs.vdev.async_write_active_max_dirty_percent be increased, decreased.... I'm afraid of doing some chage ana finally ending up with an inestable server.... I'm not an expert in handling these values.... >> >> Any recommendation?. > > 1) Make sure the pool has enough free space because ZFS can became crawling slow otherwise. > > THIS IS JUST AN EXAMPLE... BUT YOU CAN SEE ALL SIMILARLY.... > > ZPOOL LIST > NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > ZROOT 448G 2.27G 446G - - 1% 0% 1.00X ONLINE - > MAIL_DATASET 58.2T 19.4T 38.8T - - 32% 33% 1.00X ONLINE - > > 2) Increase recordsize upto 1MB for file systems located in the pool > so ZFS is allowed to use bigger request sizes for read/write operations > > WE HAVE THE DEFAULT... SO 128K... > > 3) If you use compression, look if achieved compressratio worth it and > if not (<1.4 f.e.) then better disable compression to avoid its overhead; > > WE DON'T USE COMPRESSION AS IT'S NOT SET BY DEFAULT. SOME PEOPLE SAY YOU SHOULD HAVE IT ENABLED.... BUT.... JUST FOR AVOID HAVING SOME DATA COMPRESSED SOME OTHER NOT (IN CASE YOU ENABLE AND LATER DISABLE) AND FINALLY FOR AVOID ACCESSING TO INFORMATION WITH DIFFERENT CPU COSTS OF HANDLING... WE HAVE NOT TOUCHED COMPRESSION.... > > WE SHOULD SAY WE HAVE LOTS OF CPU... > > 4) try "zfs set redundant_metadata=most" to decrease amount of small writes to the file systems; > > OK.... > > 5) If you have good power supply and stable (non-crashing) OS, try increasing > sysctl vfs.zfs.txg.timeout from defaule 5sec, but do not be extreme (f.e. upto 10sec). > Maybe it will increase amount of long writes and decrease amount of short writes, that is good. > > WELL I HAVE SYNC IN DISABLED IN THE DATASETS... DO YOU STILL THINK IT'S GOOD TO CHANGE IT?. JUST A QUESTION OF PERSON WANTING TO LEARN :) . > > WHAT ABOUT THE VFS.ZFS.DIRTY_DATA_MAX AND THE VFS.ZFS.DIRTY_DATA_MAX_MAX, WOULD YOU INCREASE THEM FROM 4GB IT'S SET NOW?. > > THANKS A LOT EUGENE!!!! > CHEERS!! --=_be6199b7d5e868a9b11faca98e7631c5 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=UTF-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; charset= =3DUTF-8" /></head><body style=3D'font-size: 10pt; font-family: Verdana,Gen= eva,sans-serif'> <p>Hi Eugene!!!</p> <p><br /></p> <p>Thank you so much really again mate :) :) :)</p> <p><br /></p> <p>About your recommendations... Eugene, if some of them wouldn't be workin= g as expected, could we revert some or all of them or perhaps some of your = recommendations below need to be definitive?.</p> <p><br /></p> <p>I do answer below in green bold for better distinction :) :)</p> <p><br /></p> <p><br /></p> <div> </div> <p><br /></p> <p>El 2022-04-06 18:14, Eugene Grosbein escribió:</p> <blockquote type=3D"cite" style=3D"padding: 0 0.4em; border-left: #1010ff 2= px solid; margin: 0"><!-- html ignored --><!-- head ignored --><!-- meta ig= nored --> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= ATENCION<br /> ATENCION<br /> ATENCION!!! Este correo se ha enviado desde f= uera de la organizacion. No pinche en los enlaces ni abra los adjuntos a no= ser que reconozca el remitente y sepa que el contenido es seguro.<br /> <b= r /> 06.04.2022 22:30, <a href=3D"mailto:egoitz@ramattack.net">egoitz@ramat= tack.net</a> =D0=BF=D0=B8=D1=88=D0=B5=D1=82: <blockquote type=3D"cite" style=3D"padding: 0 0.4em; border-left: #1010ff 2= px solid; margin: 0">One perhaps important note!!<br /> <br /> <br /> When = this happens... almost all processes appear in top in the following state:<= br /> <br /> <br /> txg state or<br /> <br /> txg-><br /> <br /> bio..= =2E.<br /> <br /> <br /> perhaps should the the vfs.zfs.dirty_data_max, vfs= =2Ezfs.txg.timeout, vfs.zfs.vdev.async_write_active_max_dirty_percent be in= creased, decreased.... I'm afraid of doing some chage ana finally ending up= with an inestable server.... I'm not an expert in handling these values.= =2E..<br /> <br /> <br /> Any recommendation?.</blockquote> <br /> 1) Make sure the pool has enough free space because ZFS can became c= rawling slow otherwise.</div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= </div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= <strong><span style=3D"color: #339966;">This is just an example... but you = can see all similarly....</span></strong></div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= </div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= <strong><span style=3D"color: #339966;">zpool list</span></strong><br /><st= rong><span style=3D"color: #339966;">NAME &nbs= p; SIZE ALLOC FREE&nb= sp; CKPOINT EXPANDSZ FRAG CAP DED= UP HEALTH ALTROOT</span></strong><br /><strong><span style=3D"c= olor: #339966;">zroot  = ; 448G 2.27G 446G &nbs= p; - - &nb= sp; 1% 0% 1.00x ONLINE = ; -</span></strong><br /><strong><span style=3D"color: #339966;">mail_datas= et 58.2T 19.4T 38.8T &= nbsp; - - = 32% 33% 1.00x ONLINE -</span></strong>= <br /><br /> <br /> 2) Increase recordsize upto 1MB for file systems locate= d in the pool<br /> so ZFS is allowed to use bigger request sizes for read/= write operations</div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= </div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= <span style=3D"color: #339966;"><strong>We have the default... so 128K...</= strong></span><br /> <br /> 3) If you use compression, look if achieved com= pressratio worth it and<br /> if not (<1.4 f.e.) then better disable com= pression to avoid its overhead;</div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= </div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= <strong><span style=3D"color: #339966;">We don't use compression as it's no= t set by default. Some people say you should have it enabled.... but.... ju= st for avoid having some data compressed some other not (in case you enable= and later disable) and finally for avoid accessing to information with dif= ferent cpu costs of handling... we have not touched compression....</span><= /strong></div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= </div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= <strong><span style=3D"color: #339966;">We should say we have lots of CPU= =2E..</span></strong></div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= <br /> <br /> 4) try "zfs set redundant_metadata=3Dmost" to decrease amount= of small writes to the file systems;</div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= </div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= <strong><span style=3D"color: #339966;">Ok....</span></strong></div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= <br /> <br /> 5) If you have good power supply and stable (non-crashing) OS= , try increasing<br /> sysctl vfs.zfs.txg.timeout from defaule 5sec, but do= not be extreme (f.e. upto 10sec).<br /> Maybe it will increase amount of l= ong writes and decrease amount of short writes, that is good.<br /> </div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= </div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= <strong><span style=3D"color: #339966;">Well I have sync in disabled in the= datasets... do you still think it's good to change it?. Just a question of= person wanting to learn :) .</span> </strong></div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= </div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= <strong><span style=3D"color: #339966;">What about the vfs.zfs.dirty_data_m= ax and the vfs.zfs.dirty_data_max_max, would you increase them from 4GB it'= s set now?.</span></strong></div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= </div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= </div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= </div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= </div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= </div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= </div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= </div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= </div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= </div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= </div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= </div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= </div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= <strong><span style=3D"color: #339966;">Thanks a lot Eugene!!!!</span></str= ong></div> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= <strong><span style=3D"color: #339966;">Cheers!!</span></strong></div> </blockquote> </body></html> --=_be6199b7d5e868a9b11faca98e7631c5--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7aa95cb4bf1fd38b3fce93bc26826042>