From nobody Wed Apr 6 15:43:27 2022 X-Original-To: freebsd-performance@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 0B3581A873E2; Wed, 6 Apr 2022 15:43:37 +0000 (UTC) (envelope-from se@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KYTLw164tz4Rt7; Wed, 6 Apr 2022 15:43:36 +0000 (UTC) (envelope-from se@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1649259816; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=DszpBHls3d9k0db5g8HTfayx6XUbGeuJVLcEKqu7X00=; b=Gc5xuom5FY9xOpw9pbd1F6xAo9h6fFjjX6ZZa0srj87yA97Uu0xMLJwlwn29DiJqmS9lUW 6rBc9B/LUBjAL9JHluoyLecZG7WPTnLnLEPPGAvYvf3YpF6hdPZtLdyJnIacTrcTWqZecc jGl7MrQSGn3OkOQyrus9vri9gEPhUe698IJAZIKrSzcwCtA9Q78KvAXoAIRSu88kvfGct8 BeknFduPoU5/w4PknQGlUiHqCOszLY2/XqhVi2EZFXmUCMgfXRKj8tWsSxjC1cu0WihEtp 1PpW9FVAy00CMNWFZgSnHOoRBHw3G/TVcikzZf6ozAsF+20TZoWPEDas69Z0Lw== Received: from [IPV6:2003:cd:5f22:6f00:a5c2:9043:ac1c:2c06] (p200300cd5f226f00a5c29043ac1c2c06.dip0.t-ipconnect.de [IPv6:2003:cd:5f22:6f00:a5c2:9043:ac1c:2c06]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: se/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id C2DF02234; Wed, 6 Apr 2022 15:43:32 +0000 (UTC) (envelope-from se@FreeBSD.org) Message-ID: <dd9a55ac-053d-7802-169d-04c95c045ed2@FreeBSD.org> Date: Wed, 6 Apr 2022 17:43:27 +0200 List-Id: Performance/tuning <freebsd-performance.freebsd.org> List-Archive: https://lists.freebsd.org/archives/freebsd-performance List-Help: <mailto:performance+help@freebsd.org> List-Post: <mailto:performance@freebsd.org> List-Subscribe: <mailto:performance+subscribe@freebsd.org> List-Unsubscribe: <mailto:performance+unsubscribe@freebsd.org> Sender: owner-freebsd-performance@freebsd.org X-BeenThere: freebsd-performance@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0) Gecko/20100101 Thunderbird/91.7.0 Subject: Re: Desperate with 870 QVO and ZFS Content-Language: en-US To: egoitz@ramattack.net Cc: freebsd-fs@freebsd.org, freebsd-hackers@freebsd.org, freebsd-performance@freebsd.org, Rainer Duffner <rainer@ultra-secure.de> References: <4e98275152e23141eae40dbe7ba5571f@ramattack.net> <665236B1-8F61-4B0E-BD9B-7B501B8BD617@ultra-secure.de> <0ef282aee34b441f1991334e2edbcaec@ramattack.net> From: Stefan Esser <se@FreeBSD.org> In-Reply-To: <0ef282aee34b441f1991334e2edbcaec@ramattack.net> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------xWEDst9IZgYvLPukN2ga6r9b" ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1649259816; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=DszpBHls3d9k0db5g8HTfayx6XUbGeuJVLcEKqu7X00=; b=aN4rR0ArnFTHAcbxMwgavfJPkS5+OKQlPfEt1Et1uxmcF+JlZXt0XA3/T7la/p1horP/zj WOseXuHjUAkNWnLPN2aeh5cq+TTuB3/9XrhmOF3NDjuj59dF5ikvm0pvAhqZ6GfbWNxFTD zIPeu0BwVtRLTujVxtYvsWhk9syiPIrVfmjaL40GxcnIuwocG2gMs0ygkn9P5okXCX+ZTy ztX0bnRWvrPKSQjztLOY6TiTSagOw8H6L9InjiPlXdF3TsJxxuK+LRecN9+90HbGQRHzxb 8lyj0IQtmdFmSoZI7o0sHk0TkiBus9KSDJ1ITSaYRKyPkCiw3BZ9TQDw2YHISQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1649259816; a=rsa-sha256; cv=none; b=Kh0/OSy2P0jg8Ud2tGvmRxYZEOwkXWAVZpjm83vZci17DkOsyBf8dW34XMZ5BukYz2EVBV ogoweL4BZh0pL34FGqJ8cL0/CioGC8N/qydaux+MobxVFKyvDfX8fTOlR6QOclh225o63Z ubUAOVUytF5H+3ku//KvSceootMPYY1/UOOjPImCZsAjTiWXyz54Y2gKSyfAF0J7LG4bf0 fCiWAbNWnTWYlEfcUGE3SRnKQrPk+htaTT2APKWkRX5jGvDxlO3EYzHcQU7ie+qjRv8EuX 4kAmnhFgsgOiYSoCB2OSqti+/Y3ux1IFIZ5OV8+vv0n1muQusDn97SPRphb9Mw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------xWEDst9IZgYvLPukN2ga6r9b Content-Type: multipart/mixed; boundary="------------gUL8upCKsfSinXvRen5NU9ab"; protected-headers="v1" From: Stefan Esser <se@FreeBSD.org> To: egoitz@ramattack.net Cc: freebsd-fs@freebsd.org, freebsd-hackers@freebsd.org, freebsd-performance@freebsd.org, Rainer Duffner <rainer@ultra-secure.de> Message-ID: <dd9a55ac-053d-7802-169d-04c95c045ed2@FreeBSD.org> Subject: Re: Desperate with 870 QVO and ZFS References: <4e98275152e23141eae40dbe7ba5571f@ramattack.net> <665236B1-8F61-4B0E-BD9B-7B501B8BD617@ultra-secure.de> <0ef282aee34b441f1991334e2edbcaec@ramattack.net> In-Reply-To: <0ef282aee34b441f1991334e2edbcaec@ramattack.net> --------------gUL8upCKsfSinXvRen5NU9ab Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Am 06.04.22 um 16:36 schrieb egoitz@ramattack.net: > Hi Rainer! >=20 > Thank you so much for your help :) :) >=20 > Well I assume they are in a datacenter and should not be a power outage= =2E... >=20 > About dataset size... yes... our ones are big... they can be 3-4 TB eas= ily each > dataset..... >=20 > We bought them, because as they are for mailboxes and mailboxes grow an= d > grow.... for having space for hosting them... Which mailbox format (e.g. mbox, maildir, ...) do you use? > We knew they had some speed issues, but those speed issues, we thought = (as > Samsung explains in the QVO site) they started after exceeding the spee= ding > buffer this disks have. We though that meanwhile you didn't exceed it's= > capacity (the capacity of the speeding buffer) no speed problem arises.= Perhaps > we were wrong?. These drives are meant for small loads in a typical PC use case, i.e. some installations of software in the few GB range, else only files of a few MB being written, perhaps an import of media files that range from tens to a few hundred MB at a time, but less often than once a day. As the SSD fills, the space available for the single level write cache gets smaller (on many SSDs, I have no numbers for this particular device), and thus the amount of data that can be written at single cell speed shrinks as the SSD gets full. I have just looked up the size of the SLC cache, it is specified to be 78 GB for the empty SSD, 6 GB when it is full (for the 2 TB version, smaller models will have a smaller SLC cache). But after writing those few GB at a speed of some 500 MB/s (i.e. after 12 to 150 seconds), the drive will need several minutes to transfer those writes to the quad-level cells, and will operate at a fraction of the nominal performance during that time. (QLC writes max out at 80 MB/s for the 1 TB model, 160 MB/s for the 2 TB model.) And cheap SSDs often have no RAM cache (not checked, but I'd be surprised if the QVO had one) and thus cannot keep bookkeeping date in such a cache, further limiting the performance under load. And the resilience (max. amount of data written over its lifetime) is also quite low - I hope those drives are used in some kind of RAID configuration. The 870 QVO is specified for 370 full capacity writes, i.e. 370 TB for the 1 TB model. That's still a few hundred GB a day - but only if the write amplification stays in a reasonable range ... --------------gUL8upCKsfSinXvRen5NU9ab-- --------------xWEDst9IZgYvLPukN2ga6r9b Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsB5BAABCAAjFiEEo3HqZZwL7MgrcVMTR+u171r99UQFAmJNtR8FAwAAAAAACgkQR+u171r99UQX jggAh1PLi41CMsG6xbRvf9KA3JRSYjHGSCr3soAi5Su5VZmVNts3ocVUONOfR4yoTj/JGZ0HYvwi iQm4PxPLS7Fj69joQnernx6Dhem6yg8hJSwrU3HDZQ4lIDSQ2B220+uz9MrqImu21JvDxIRzmgH+ kQ7Q3+ZoxCi0BJX83yL8sh0wMA5tLrV1e8IKrpBR/mLiwQZRoaOPKXKx29eP4Q8St57UySGfGL13 O33jTgM8sAAGkImgAa2JzXRhYQ2KY5QnplYv1cxk6Zbpuq1TgovqGm3pzak0i1kTiK95K3tWYhMh 2ne2gmnXcO6CGu+QeUapXJN10sefE8gRNrBDLzpjJg== =HKc2 -----END PGP SIGNATURE----- --------------xWEDst9IZgYvLPukN2ga6r9b--