Date: Mon, 22 Aug 2016 11:46:35 -0500 From: Karl Denninger <karl@denninger.net> To: freebsd-fs@freebsd.org Subject: Re: Recommended HBA for ZFS, contemporary Message-ID: <580a6ea8-9c84-3e99-5a7d-7b2434c84e67@denninger.net> In-Reply-To: <alpine.BSF.2.00.1608221926550.82304@woozle.rinet.ru> References: <alpine.BSF.2.00.1608221904270.82304@woozle.rinet.ru> <ba972848-140c-31a8-505e-2470ed750fc8@denninger.net> <alpine.BSF.2.00.1608221926550.82304@woozle.rinet.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a cryptographically signed message in MIME format. --------------ms020807000804010108050705 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 8/22/2016 11:30, Dmitry Morozovsky wrote: > Karl, > > On Mon, 22 Aug 2016, Karl Denninger wrote: > >> I'm still a fan of the LSI SAS-9211 cards along with (for higher >> density) a SAS port expander. I've been extremely happy with this >> combination with one caveat -- most systems must boot from the actual >> card and not a port on the expander(s) you connect to it due to BIOS >> constraints. They're also crazily cost-effective on-balance. >> >> In practice this means you have 4 ports on the base card available (2 >> used for mirrored boot drives, 2 for other things) and the rest of the= >> disks go on the expander, or you run more than one card. >> >> Whether you can actually saturate this combination depends on what you= >> attach. In large configurations, especially those stuffed with SSDs, >> you can -- at which point something faster such as the 93xx series whi= ch > which immediately leads me to the question: what is the driver supporti= ng 9300=20 > in HBA mode? Quick googling does not reveal interoperability, and 9300= is not=20 > in hardware list for FreeBSD yet. So much for LSI's claims eh? :-) >> are materially faster (but still in "host" mode as opposed to any sort= >> of "smart", "raid" or "buffered" mode) should be considered. >> >> The "smarter" the card the dumber the results tend to be when ZFS is i= n >> use, in my experience :) > Yes, I'm well aware of it ;) > > Thanks! Incidentally actually managing to saturate a 9211 (which comes up as a "SAS-2008") on the mps driver is non-trivial. If you THINK you are, make sure you really are. If you really are and have a second PCIe slot with the requisite lanes available that's probably the next thing to do. Yes, it's possible (especially with SSDs) to saturate these but not easy. You can get the 2xxx series cards in 16-port versions (4 x 4) which leaves you three ports that can each be attached to an expander.=20 The limiting factor usually winds up being the configuration of said plugs and the shelves for the drives themselves (2i/2e configurations work great if you're using expansion chassis with an expander in them, by the way.) I have a number of the Intel 6-port expanders in production servers and have never have had any trouble with them; they just plain work. The only note I'll make on the expanders (and these host cards) is to make sure you have sufficient airflow in the chassis; they're passive heat-sink cooled so airflow matters. In any sort of proper server chassis this is probably not going to be an issue but if you're going for something that doesn't sound like an old 707 jet on take-off you need to pay attention to make sure there's sufficient cooling airflow. I tend to avoid using the motherboard SATA ports; I've had my share of "fun" with them over time, especially if I need to hot-swap a drive (e.g. a failed member of the root mirror.) You only need to have a machine crash on you unnecessarily in this fashion once to be disabused of that sort of insanity in the future. --=20 Karl Denninger karl@denninger.net <mailto:karl@denninger.net> /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms020807000804010108050705 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjA4MjIxNjQ2MzVaME8GCSqGSIb3DQEJBDFCBEAs 8TNvA+Elyw/z6MtJuqDGeJs537cc2nIunj5GvivQCw1cnmCiBM+rsceoBGC/V8pvISDoJ3Ei HENN29GvVgvHMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIApwc440FT OxifqYGYVuCr4kmfZVGNl0oQ/oXjt8nEJOeUWq4CRe+MwJELzY+3/wHZjTuWVML3BIqeHpGO 4wrxm6fECCoLNeiAvW960wCuUOm5wlV+DDQ2q9TAgfOchM//syRjyW1xigw4+Mus2aUmYRtJ +1hB9IeMaYg5PgH16lfCz4hIV/0NhOlbPy9+qk8QPqasuZaJ1Nq+sEiJ8znZ+Z3SK2NwUhmN +QJW0x7byBsud1bm0Ay3OJAD5MUxkq1fwqRan/8bipAiAnNVN5PIWpBpVLD1AhckqxoLYecb WPbzJsGKfFqaiLzGKGOzqoHz8SDX7hP21ePoaRkQfjh/jp3wvBU6rMcJpRhzyWRhwgX+kcJm 2/UqbUxyqcBE5SC9gJyz+YgYDThlEuZ3QY1vxlpPtA5IMztaQo3g09RLCdF1NvCSbUWeM/u5 Fh9FEsXsYowQ3P65laA381RBzEFFXeSTKnbXDeoGUURcCe3/xHL2zPYthq7CLC/kCE7QhrSa QCt3SHuZ2wvh4g1UNrGCH5myx+YxkAecxRRemf77CAE2I+EeWVRD1J9S3Pv3UsYLYik2MPox TiL3Vt7/QrU5zOQi0Whi2HGKeg8JH9ozZpyciN+l1obukTxuxypHsRel1PmWI/+CNi8A0b8N V2KCEoqz3xbxE5mJEbvQiUrVJ3EAAAAAAAA= --------------ms020807000804010108050705--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?580a6ea8-9c84-3e99-5a7d-7b2434c84e67>