Date: Thu, 7 Apr 2022 09:59:27 +0200
From: Stefan Esser <se@FreeBSD.org>
To: egoitz@ramattack.net
Cc: Jan Bramkamp <crest@rlwinm.de>, performance@freebsd.org, "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org>, FreeBSD Hackers <freebsd-hackers@freebsd.org>
Subject: Re: {* 05.00 *}Re: Desperate with 870 QVO and ZFS
Message-ID: <49f43af5-e145-c793-959d-ab1596421d81@FreeBSD.org>
In-Reply-To: <d5e51e47-d24a-27bd-ac56-295bfa798de7@rlwinm.de>
References: <4e98275152e23141eae40dbe7ba5571f@ramattack.net> <665236B1-8F61-4B0E-BD9B-7B501B8BD617@ultra-secure.de> <0ef282aee34b441f1991334e2edbcaec@ramattack.net> <28e11d7ec0ac5dbea45f9f271fc28f06@ramattack.net> <ca3f86f2-94a1-be94-ad55-7bd1c9bc50ab@grosbein.net> <7aa95cb4bf1fd38b3fce93bc26826042@ramattack.net> <alpine.GSO.2.20.2204061513180.9475@scrappy.simplesystems.org> <d8eec440-aff5-e2c7-35d5-04ac96939213@sentex.net> <d5e51e47-d24a-27bd-ac56-295bfa798de7@rlwinm.de>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--------------o4K5Sngmd3lLPnmlzLRzMtEK
Content-Type: multipart/mixed; boundary="------------ce4Isv9Wdp6yhBkTY0YsttW0";
protected-headers="v1"
From: Stefan Esser <se@FreeBSD.org>
To: egoitz@ramattack.net
Cc: Jan Bramkamp <crest@rlwinm.de>, performance@freebsd.org,
"freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org>,
FreeBSD Hackers <freebsd-hackers@freebsd.org>
Message-ID: <49f43af5-e145-c793-959d-ab1596421d81@FreeBSD.org>
Subject: Re: {* 05.00 *}Re: Desperate with 870 QVO and ZFS
References: <4e98275152e23141eae40dbe7ba5571f@ramattack.net>
<665236B1-8F61-4B0E-BD9B-7B501B8BD617@ultra-secure.de>
<0ef282aee34b441f1991334e2edbcaec@ramattack.net>
<28e11d7ec0ac5dbea45f9f271fc28f06@ramattack.net>
<ca3f86f2-94a1-be94-ad55-7bd1c9bc50ab@grosbein.net>
<7aa95cb4bf1fd38b3fce93bc26826042@ramattack.net>
<alpine.GSO.2.20.2204061513180.9475@scrappy.simplesystems.org>
<d8eec440-aff5-e2c7-35d5-04ac96939213@sentex.net>
<d5e51e47-d24a-27bd-ac56-295bfa798de7@rlwinm.de>
In-Reply-To: <d5e51e47-d24a-27bd-ac56-295bfa798de7@rlwinm.de>
--------------ce4Isv9Wdp6yhBkTY0YsttW0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Am 06.04.22 um 23:19 schrieb Jan Bramkamp:
> On 06.04.22 22:43, mike tancsa wrote:
>> On 4/6/2022 4:18 PM, Bob Friesenhahn wrote:
>>> On Wed, 6 Apr 2022, egoitz@ramattack.net wrote:
>>>>>
>>>>> WE DON'T USE COMPRESSION AS IT'S NOT SET BY DEFAULT. SOME PEOPLE SA=
Y YOU
>>>>> SHOULD HAVE IT ENABLED.... BUT.... JUST FOR AVOID HAVING SOME DATA
>>>>> COMPRESSED SOME OTHER NOT (IN CASE YOU ENABLE AND LATER DISABLE) AN=
D
>>>>> FINALLY FOR AVOID ACCESSING TO INFORMATION WITH DIFFERENT CPU COSTS=
OF
>>>>> HANDLING... WE HAVE NOT TOUCHED COMPRESSION....
>>>
>>> There seems to be a problem with your caps-lock key.
>>>
>>> Since it seems that you said that you are using maildir for your mail=
>>> server, it is likely very useful if you do enable even rather mild
>>> compression (e.g. lz4) since this will reduce the write work-load and=
even
>>> short files will be stored more efficiently.
>>>
>> FYI, a couple of our big zfs=C2=A0 mailspools sees a 1.24x and 1.23x c=
ompress
>> ratio with lz4.=C2=A0 We use Maildir format as well.=C2=A0 They are no=
t RELENG_13 so
>> not sure how zstd would fair.
> I've found that Dovecot's mdbox format compresses a lot better than Mai=
ldir (or
> sdbox), because it stores multiple messages per file resulting in files=
large
> enough to contain enough exploitable reduncancy to compress down to the=
next
> smaller blocksize. In a corporate or education environment where users =
tend to
> send the same medium to large attachments multiple times to multiple re=
cipients
> on the same server Dovecot's single instance storage is a game changer.=
It
> reduced my IMAP storage requirements by a *factor* of 4.7 which allowed=
me to
> get rid of spinning disks for the mail servers instead of playing losin=
g games
> with hybrid storage. Dovecot also supports zlib compression in the appl=
ication
> instead of punting it to the file system. I don't know if Cyrus IMAP of=
fers
> similar features, but if it does I would recommend evaluating them inst=
ead of
> compressing or deduplicating at the file system level.
I have not compared dovecot's zlib compression with zstd-2 on the file sy=
stem,
but since I use the latter on all my ZFS file systems (excepts those that=
exclusively hold compressed files and media), I'm using it for Dovecot md=
box
files, too. I get a compression ratio of 2,29 with ZFS zstd-2, maybe I sh=
ould
copy the files over into a zlib compressed mdbox for comparison ...
One large advantage of the mdbox format in the context of the mail server=
set-up at the start of this thread is that deletions are only registered =
in
an index file (while mbox needs a rewrite of potentially large parts of t=
he
mail folder and mdir immediately deletes files (TRIM) and updates inodes =
and
directory entries, causing multiple writes per deleted message).
With mdbox you can delay all "expensive" file system operations to the
point of least load each day, for example. Such a compression run is also=
well suited for SSDs, since it does not perform random updates that punch=
holes in a large number of erase blocks (which then will need to be garba=
ge
collected, causing write amplification to put further load and stress on
the SSD).
--------------ce4Isv9Wdp6yhBkTY0YsttW0--
--------------o4K5Sngmd3lLPnmlzLRzMtEK
Content-Type: application/pgp-signature; name="OpenPGP_signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="OpenPGP_signature"
-----BEGIN PGP SIGNATURE-----
wsB5BAABCAAjFiEEo3HqZZwL7MgrcVMTR+u171r99UQFAmJOmd8FAwAAAAAACgkQR+u171r99UQz
fQgAj/0scy7zAbl1SoRPExnKQSTSk320RX81cVGflFCk2hDHRKeF9bScO22aZil0nYKaCKPR1Mps
kyujxrmFpTgWjrjhNe7noHe5sz3LlGplXB7YNMKr0eujF1VC9YrlSvQLGTDFJeJyIRcI7EjSAgoy
8aLZjMG8rI7XCiMo1y+bpJqyWsElxYFoiomi2h2fkZ5MFZtWfyqPNaCFd2e4YHW6x6WYS9/H9ZDc
k3E6GoFUaoRIuHC5dw2HbeUxBr72TYRLZgSH5pMBlk0cN60QNHkOBiBahbyzPVA/U1IJqCSVTWJP
7ZUNURO6nTn71io0o148Jho31IitSdxA4cuaJOOIVg==
=DqKu
-----END PGP SIGNATURE-----
--------------o4K5Sngmd3lLPnmlzLRzMtEK--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?49f43af5-e145-c793-959d-ab1596421d81>