Date: Thu, 07 Apr 2022 15:53:06 +0200
From: egoitz@ramattack.net
To: Stefan Esser <se@freebsd.org>
Cc: Jan Bramkamp <crest@rlwinm.de>, performance@freebsd.org, freebsd-fs@freebsd.org, FreeBSD Hackers <freebsd-hackers@freebsd.org>
Subject: Re: {* 05.00 *}Re: Re: Desperate with 870 QVO and ZFS
Message-ID: <a508681e9e48df69d4cb393860791d6f@ramattack.net>
In-Reply-To: <49f43af5-e145-c793-959d-ab1596421d81@FreeBSD.org>
References: <4e98275152e23141eae40dbe7ba5571f@ramattack.net> <665236B1-8F61-4B0E-BD9B-7B501B8BD617@ultra-secure.de> <0ef282aee34b441f1991334e2edbcaec@ramattack.net> <28e11d7ec0ac5dbea45f9f271fc28f06@ramattack.net> <ca3f86f2-94a1-be94-ad55-7bd1c9bc50ab@grosbein.net> <7aa95cb4bf1fd38b3fce93bc26826042@ramattack.net> <alpine.GSO.2.20.2204061513180.9475@scrappy.simplesystems.org> <d8eec440-aff5-e2c7-35d5-04ac96939213@sentex.net> <d5e51e47-d24a-27bd-ac56-295bfa798de7@rlwinm.de> <49f43af5-e145-c793-959d-ab1596421d81@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] Hi Stefan, Thanks a lot again, mate. Answering below in bold blue... El 2022-04-07 09:59, Stefan Esser escribió: > I have not compared dovecot's zlib compression with zstd-2 on the file system, > but since I use the latter on all my ZFS file systems (excepts those that > exclusively hold compressed files and media), I'm using it for Dovecot mdbox > files, too. I get a compression ratio of 2,29 with ZFS zstd-2, maybe I should > copy the files over into a zlib compressed mdbox for comparison ... > > WE ARE RUNNING CYRUS HERE... ALTHOUGH THAT CHECK SOUNDS INTERESTING... > > One large advantage of the mdbox format in the context of the mail server > set-up at the start of this thread is that deletions are only registered in > an index file (while mbox needs a rewrite of potentially large parts of the > mail folder and mdir immediately deletes files (TRIM) and updates inodes and > directory entries, causing multiple writes per deleted message). > > I SEE... REALLY SAID... I LOVE CYRUS... IT'S REPLICATION IS EXTREMELY RELIABLE... > > SOME TIME NOW... DOVECOT DIDN'T HAD REPLICATION... AND WE HAVE SOME DEVELOPMENTS DONE FROM SOME TIME NOW FOR CYRUS IMAP... > > BUT GOOD TO KNOW TO ABOUT OTHER SOFTWARE'S ADVANTAGES... > > With mdbox you can delay all "expensive" file system operations to the > point of least load each day, for example. Such a compression run is also > well suited for SSDs, since it does not perform random updates that punch > holes in a large number of erase blocks (which then will need to be garbage > collected, causing write amplification to put further load and stress on > the SSD). > > WE DON'T DELETE MAIL DURING DAY HOURS. WE USE A FEATURE OF CYRUS CALLED EXPUNGE DELAYED. THE DELETED EMAIL IS DELETED FROM DISK AT 04AM (UNTIL THAT MOMENT IS JUST TAGGED AS DELETED IN A CYRUS DATABASE). THE EXCEPTION HAPPENS WHEN YOU RENAME A FOLDER OR DELETE AN ENTIRE FOLDER. IF YOU DELETE AN ENTIRE FOLDER, I THINK IT GETS COPIED TO A DELETED/..WHATEVER.. FOLDER AND THEN YES... IT COPIES AND LATER DELETES... > > APART FROM THAT, CYRUS DOES A LOT OF DATABASE CHECKPOINTING, CAUSING DATABASES TO BE COPIED TO A NEW CREATED ONE AND THE OLD ONE TO BE DELETED. THIS ARE THE ONLY REMOVALS WE DO DURING DAY TIME. THE REST IS DONE FROM 04AM TO 05AM, WHEN THERE'S NO LOAD. > > CHEERS!!! [-- Attachment #2 --] <html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" /></head><body style='font-size: 10pt; font-family: Verdana,Geneva,sans-serif'> <p>Hi Stefan,</p> <p><br /></p> <p>Thanks a lot again, mate. Answering below in bold blue...</p> <div> </div> <p><br /></p> <p>El 2022-04-07 09:59, Stefan Esser escribió:</p> <blockquote type="cite" style="padding: 0 0.4em; border-left: #1010ff 2px solid; margin: 0"><!-- html ignored --><!-- head ignored --><!-- meta ignored --> <div class="pre" style="margin: 0; padding: 0; font-family: monospace"><br /> I have not compared dovecot's zlib compression with zstd-2 on the file system,<br /> but since I use the latter on all my ZFS file systems (excepts those that<br /> exclusively hold compressed files and media), I'm using it for Dovecot mdbox<br /> files, too. I get a compression ratio of 2,29 with ZFS zstd-2, maybe I should<br /> copy the files over into a zlib compressed mdbox for comparison ...</div> <div class="pre" style="margin: 0; padding: 0; font-family: monospace"> </div> <div class="pre" style="margin: 0; padding: 0; font-family: monospace"><span style="color: #0000ff;"><strong>We are running Cyrus here... although that check sounds interesting...</strong></span><br /> <br /> One large advantage of the mdbox format in the context of the mail server<br /> set-up at the start of this thread is that deletions are only registered in<br /> an index file (while mbox needs a rewrite of potentially large parts of the<br /> mail folder and mdir immediately deletes files (TRIM) and updates inodes and<br /> directory entries, causing multiple writes per deleted message).</div> <div class="pre" style="margin: 0; padding: 0; font-family: monospace"> </div> <div class="pre" style="margin: 0; padding: 0; font-family: monospace"><strong><span style="color: #0000ff;">I see... really said... I love Cyrus... it's replication is extremely reliable...</span></strong></div> <div class="pre" style="margin: 0; padding: 0; font-family: monospace"> </div> <div class="pre" style="margin: 0; padding: 0; font-family: monospace"><strong><span style="color: #0000ff;">Some time now... Dovecot didn't had replication... and we have some developments done from some time now for Cyrus IMAP...</span></strong></div> <div class="pre" style="margin: 0; padding: 0; font-family: monospace"> </div> <div class="pre" style="margin: 0; padding: 0; font-family: monospace"><strong><span style="color: #0000ff;">But good to know to about other software's advantages...</span></strong><br /> <br /> With mdbox you can delay all "expensive" file system operations to the<br /> point of least load each day, for example. Such a compression run is also<br /> well suited for SSDs, since it does not perform random updates that punch<br /> holes in a large number of erase blocks (which then will need to be garbage<br /> collected, causing write amplification to put further load and stress on<br /> the SSD).</div> <div class="pre" style="margin: 0; padding: 0; font-family: monospace"> </div> <div class="pre" style="margin: 0; padding: 0; font-family: monospace"><strong><span style="color: #0000ff;">We don't delete mail during day hours. We use a feature of Cyrus called expunge delayed. The deleted email is deleted from disk at 04am (until that moment is just tagged as deleted in a Cyrus database). The exception happens when you rename a folder or delete an entire folder. If you delete an entire folder, I think it gets copied to a DELETED/..whatever.. folder and then yes... it copies and later deletes...</span></strong></div> <div class="pre" style="margin: 0; padding: 0; font-family: monospace"> </div> <div class="pre" style="margin: 0; padding: 0; font-family: monospace"><strong><span style="color: #0000ff;">Apart from that, Cyrus does a lot of database checkpointing, causing databases to be copied to a new created one and the old one to be deleted. This are the only removals we do during day time. The rest is done from 04am to 05am, when there's no load.</span></strong></div> <div class="pre" style="margin: 0; padding: 0; font-family: monospace"> </div> <div class="pre" style="margin: 0; padding: 0; font-family: monospace"><strong><span style="color: #0000ff;">Cheers!!!</span></strong></div> </blockquote> </body></html>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?a508681e9e48df69d4cb393860791d6f>