From owner-freebsd-fs@FreeBSD.ORG Fri Mar 28 12:12:12 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6DAA4718 for ; Fri, 28 Mar 2014 12:12:12 +0000 (UTC) Received: from fs.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "NewFS.denninger.net", Issuer "NewFS.denninger.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1C15933A for ; Fri, 28 Mar 2014 12:12:11 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by fs.denninger.net (8.14.8/8.14.8) with ESMTP id s2SCC6AU044313 for ; Fri, 28 Mar 2014 07:12:06 -0500 (CDT) (envelope-from karl@denninger.net) Received: from [127.0.0.1] (TLS/SSL) [192.168.1.40] by Spamblock-sys (LOCAL/AUTH); Fri Mar 28 07:12:06 2014 Message-ID: <53356711.8010509@denninger.net> Date: Fri, 28 Mar 2014 07:12:01 -0500 From: Karl Denninger User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.4.0 MIME-Version: 1.0 To: Joar Jegleim , freebsd-fs@freebsd.org Subject: Re: zfs l2arc warmup References: <20140328005911.GA30665@neutralgood.org> In-Reply-To: Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha1; boundary="------------ms000605000505070701090103" X-Antivirus: avast! (VPS 140328-0, 03/28/2014), Outbound message X-Antivirus-Status: Clean X-Content-Filtered-By: Mailman/MimeDel 2.1.17 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Mar 2014 12:12:12 -0000 This is a cryptographically signed message in MIME format. --------------ms000605000505070701090103 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable On 3/28/2014 4:23 AM, Joar Jegleim wrote: > On 28 March 2014 01:59, wrote: >> On Thu, Mar 27, 2014 at 11:10:48AM +0100, Joar Jegleim wrote: >>> But it's really not a problem for me how long it takes to warm up the= >>> l2arc, if it takes a week that's ok. After all I don't plan on >>> reboot'ing this setup very often + I have 2 servers so I have the >>> option to let the server warmup until i hook it into production again= >>> after maintenance / patch upgrade and so on . >>> >>> I'm just curious about wether or not the l2arc warmup itself, or if I= >>> would have to do that manual rsync to force l2arc warmup. >> Have you measured the difference in performance between a cold L2ARC a= nd >> a warm one? Even better, have you measured the performance with a cold= >> L2ARC to see if it meets your performance needs? > No I haven't. > I actually started using those 2 ssd's for l2arc the day before I sent > out this mail to the list . > I haven't done this the 'right' way by producing some numbers for > measurement, but I do know that the way this application work today is > that it will pull random jpegs from this dataset of about 1.6TB, > consisting of lots of millions of files ( more than 20 million). And > that today this pool is served from 20 SATA 7.2K disks which would be > the slowest solution for random read access. > Based on the huge performance gain by using ssd's simply by looking at > the spec., but also by looking at other peoples graphs from the net ( > people who have done this more thorough than me) I'm pretty confident > to say that if at any time when the application request a jpeg if it > was served from either ram or ssd it would be a substantial > performance gain compared from serving it from the 7.2k array of > disks. No, the simplest solution is IMHO to stop trying to RAM-back a 1.6TB=20 data set through various machinations. A cache is just that -- a cache. It's purpose is to make *frequently=20 accessed* data more-quickly available to an application. You have the=20 antithesis of cachable data in that you have a pure random access model=20 with no predictive or "frequently used" means to determine what is=20 likely to be requested next. IMHO the best and cheapest way to serve that data is to eliminate=20 rotational and positioning latency from the data path. If it is a=20 read-nearly-always (or read only) data set then redundancy is only=20 necessary to prevent downtime (not data loss) since it be easily backed u= p. For the model you describe I would buy however many SSD disks were=20 necessary to store said data set, design a means to back it up reliably=20 and be done with it. Backing the data store with L2ARC (and the RAM to manage it) is likely=20 self-defeating as you not only are paying for BOTH the spinning rust AND = the SSDs but you have doubled the number of devices that can fail and=20 interrupt service. --=20 -- Karl karl@denninger.net --------------ms000605000505070701090103 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFTzCC BUswggQzoAMCAQICAQgwDQYJKoZIhvcNAQEFBQAwgZ0xCzAJBgNVBAYTAlVTMRAwDgYDVQQI EwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkqhkiG9w0BCQEWIGN1c3Rv bWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0MB4XDTEzMDgyNDE5MDM0NFoXDTE4MDgyMzE5 MDM0NFowWzELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExFzAVBgNVBAMTDkthcmwg RGVubmluZ2VyMSEwHwYJKoZIhvcNAQkBFhJrYXJsQGRlbm5pbmdlci5uZXQwggIiMA0GCSqG SIb3DQEBAQUAA4ICDwAwggIKAoICAQC5n2KBrBmG22nVntVdvgKCB9UcnapNThrW1L+dq6th d9l4mj+qYMUpJ+8I0rTbY1dn21IXQBoBQmy8t1doKwmTdQ59F0FwZEPt/fGbRgBKVt3Quf6W 6n7kRk9MG6gdD7V9vPpFV41e+5MWYtqGWY3ScDP8SyYLjL/Xgr+5KFKkDfuubK8DeNqdLniV jHo/vqmIgO+6NgzPGPgmbutzFQXlxUqjiNAAKzF2+Tkddi+WKABrcc/EqnBb0X8GdqcIamO5 SyVmuM+7Zdns7D9pcV16zMMQ8LfNFQCDvbCuuQKMDg2F22x5ekYXpwjqTyfjcHBkWC8vFNoY 5aFMdyiN/Kkz0/kduP2ekYOgkRqcShfLEcG9SQ4LQZgqjMpTjSOGzBr3tOvVn5LkSJSHW2Z8 Q0dxSkvFG2/lsOWFbwQeeZSaBi5vRZCYCOf5tRd1+E93FyQfpt4vsrXshIAk7IK7f0qXvxP4 GDli5PKIEubD2Bn+gp3vB/DkfKySh5NBHVB+OPCoXRUWBkQxme65wBO02OZZt0k8Iq0i4Rci WV6z+lQHqDKtaVGgMsHn6PoeYhjf5Al5SP+U3imTjF2aCca1iDB5JOccX04MNljvifXgcbJN nkMgrzmm1ZgJ1PLur/ADWPlnz45quOhHg1TfUCLfI/DzgG7Z6u+oy4siQuFr9QT0MQIDAQAB o4HWMIHTMAkGA1UdEwQCMAAwEQYJYIZIAYb4QgEBBAQDAgWgMAsGA1UdDwQEAwIF4DAsBglg hkgBhvhCAQ0EHxYdT3BlblNTTCBHZW5lcmF0ZWQgQ2VydGlmaWNhdGUwHQYDVR0OBBYEFHw4 +LnuALyLA5Cgy7T5ZAX1WzKPMB8GA1UdIwQYMBaAFF3U3hpBZq40HB5VM7B44/gmXiI0MDgG CWCGSAGG+EIBAwQrFilodHRwczovL2N1ZGFzeXN0ZW1zLm5ldDoxMTQ0My9yZXZva2VkLmNy bDANBgkqhkiG9w0BAQUFAAOCAQEAZ0L4tQbBd0hd4wuw/YVqEBDDXJ54q2AoqQAmsOlnoxLO 31ehM/LvrTIP4yK2u1VmXtUumQ4Ao15JFM+xmwqtEGsh70RRrfVBAGd7KOZ3GB39FP2TgN/c L5fJKVxOqvEnW6cL9QtvUlcM3hXg8kDv60OB+LIcSE/P3/s+0tEpWPjxm3LHVE7JmPbZIcJ1 YMoZvHh0NSjY5D0HZlwtbDO7pDz9sZf1QEOgjH828fhtborkaHaUI46pmrMjiBnY6ujXMcWD pxtikki0zY22nrxfTs5xDWGxyrc/cmucjxClJF6+OYVUSaZhiiHfa9Pr+41okLgsRB0AmNwE f6ItY3TI8DGCBQowggUGAgEBMIGjMIGdMQswCQYDVQQGEwJVUzEQMA4GA1UECBMHRmxvcmlk YTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExDMRwwGgYD VQQDExNDdWRhIFN5c3RlbXMgTExDIENBMS8wLQYJKoZIhvcNAQkBFiBjdXN0b21lci1zZXJ2 aWNlQGN1ZGFzeXN0ZW1zLm5ldAIBCDAJBgUrDgMCGgUAoIICOzAYBgkqhkiG9w0BCQMxCwYJ KoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNDAzMjgxMjEyMDFaMCMGCSqGSIb3DQEJBDEW BBRJyFKgRFAhVuDYkburNBmHs7t/VzBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFlAwQBKjAL BglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFA MAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIG0BgkrBgEEAYI3EAQxgaYwgaMwgZ0xCzAJBgNV BAYTAlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoT EEN1ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExLzAtBgkq hkiG9w0BCQEWIGN1c3RvbWVyLXNlcnZpY2VAY3VkYXN5c3RlbXMubmV0AgEIMIG2BgsqhkiG 9w0BCRACCzGBpqCBozCBnTELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExEjAQBgNV BAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1zIExMQzEcMBoGA1UEAxMTQ3Vk YSBTeXN0ZW1zIExMQyBDQTEvMC0GCSqGSIb3DQEJARYgY3VzdG9tZXItc2VydmljZUBjdWRh c3lzdGVtcy5uZXQCAQgwDQYJKoZIhvcNAQEBBQAEggIAqndZ7mn02nRN0YG5eJ/8+eIsGA0t gD07Mf+Lts+gl2TBME9O0htaQlYEnHWr2MzBzsDsOG6sVmUI5DI0qfP2b2ho+SbOq2j0Y9nk Sm7ZfwSlm5BMOS91QQxJck5RI1iukQFaWgAozYiXyJRUR2xj8LZp/t19xuSc+aNfie6k+931 4CeD+hPnSVFP/VOolI9sCXT3tEhEPPoMyx5UOgNPFtUDdASM+qiC3/6vKKBx6eiBhZdb2e7n jnpFGyd/bqC1C2WGLR1Pn/Z/9TqjE0y/EEIVcr7cD6M4KZfkHtad1bH4INMxuUrWKwP6/0mr k/pacowNqnIwoBd4SsIin+y5spM/pAzFKDE+0w1uTHkwV5nsEzb1YHZilFzOjgIjhP/PgeGM H3O3EXKv4gu2TYarCd4M1oTQ7PGnafIN5XbcmTm6++RSUFtX6y7RS9EOHYcBYNZ+fvBF7eNL u1ZjYTyjGopXB3KETh2XJyMuEdJoIn36DBfGoTtYh63V0ghUWOZ7XLRMC3rqRtQIfql9sxfR 0/VqJJzv/GCGbUvTdsXxPBrhJzTUIJjbvjXZwQK+5OgeD1x/l1w/ymYh1AvndvKikKM543Wb 1k+kGa0iIMM4u6exB8/OaLoqXCAGS5wo9mURFJBWD2LHwKa/hCOFBG9e5ayZecNWK9S/x4JN jKIzQ/UAAAAAAAA= --------------ms000605000505070701090103--