From owner-freebsd-fs@freebsd.org Mon Jan 21 06:34:44 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7B85A14920F3 for ; Mon, 21 Jan 2019 06:34:44 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [64.62.153.212]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "anubis.delphij.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1F65E70B6C for ; Mon, 21 Jan 2019 06:34:43 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from odin.corp.delphij.net (unknown [IPv6:2601:646:8882:37a:8c4c:9b29:ea1d:db2b]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by anubis.delphij.net (Postfix) with ESMTPSA id D4E8B3CD45 for ; Sun, 20 Jan 2019 22:34:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphij.net; s=m7e2; t=1548052475; x=1548066875; bh=JZTKD+Z0szzYTlJSC+SWyzt/Lz39Pw7HKvmwgFGRAXM=; h=Reply-To:Subject:To:References:From:Date:In-Reply-To; b=PKATH1XQJ/SxBLw/ZRR2nGT0+I/eB5mrEvZCXmHE1CiiOY+OvMtUM10KiQA6yFhws fGsje8olhuluc+879DIz9PoIYLFionyqfOxhaGm+c8Ul9TmNBi3EIasFuFrlsnEfhJ 3jk06QC4AzXGophQD00wF6CjOJ3x4s+qWIq9YlcVvovKw9648oPVcZHF/fo4UYrs2K y8YGAkjDGiLQ/Ul2Y7ciw8y8neQ5M3+nqD6XOL/r9aPma3OCIaJgUo0AUDmq+SBRmy wL3eT0QwjVytPTcufAK0OipfNV4RM7bhLgVEi032w5YRxCMIqatRe9VsZ5xrikKITD ZDEszcvwvPcwg== Reply-To: d@delphij.net Subject: Re: ZFS on Hardware RAID To: freebsd-fs@freebsd.org References: <1180280695.63420.1547910313494.JavaMail.zimbra@gausus.net> <92646202.63422.1547910433715.JavaMail.zimbra@gausus.net> <1691666278.63816.1547976245836.JavaMail.zimbra@gausus.net> From: Xin Li Openpgp: preference=signencrypt Autocrypt: addr=delphij@delphij.net; prefer-encrypt=mutual; keydata= mQINBFuSR4oBEACvvEgwRIHs6IcSP/yaDtySF78Ji3rP29qdiQsxhMsOtvtffdbS56VApIWO UFb3/iN2gA8HwLvrmjijN0HEoLVX7na1WARmxRYzQMtApsZIUTtx7hnUYlsi2F5odZa6CDW9 a954DLRzYxiUwYDcu5Zjl9bglK1H8e/N9uC0Vuigr4teWfh86brzOyf819QzwFVYfMIK4ihw QGwMvTzbyVuCFy+LENkmcVYni70oQy6rZ5ktSuYbuOFvu7inRRfhSWPHziV7k+bW88sJ7xhv lBlegcnhkSudWX2M8tZ3MO1PJOcyys0CJlsBY5Weiog2lIPi05h/E9pZ9mc1Vud17iqDaL6w RaggOUhuPfDGCdO5ro82W4BZGeQMRnRF5Ntk+t2ShIH4nn3xRLV0E5nziCiKlgiMqOrz/ZTL QTVbHrCuiwD+fSK14y0oHbkOLYTYLlgh1JbwfY2Ty7elOYiWzyeJ7sJh2dF91NSEneWIOys3 mBpuvtU3nSzzTvAB48VV+Nbg1CpIOgNlPjj7uhIum/Z/VjUaJEyaLpTIRh0MVJVcbP7hXSqZ NA35EEZZVnWEOYdycm4CmEdeNPWkrAf2Ya77iR5VLGypwMlsUMQPh+sKVWDD38M8stFGBBNm d01Hi74Bsq5hKan654dOqMt5eYklrVj0ucMzFQtus7oE502UswARAQABtBxYaW4gTEkgPGRl bHBoaWpAZGVscGhpai5uZXQ+iQJUBBMBCgA+FiEEceNg5NEMZIki80nQQHl/fJX0g08FAluS R/YCGwMFCQmuhAAFCwkIBwMFFQoJCAsFFgIDAQACHgECF4AACgkQQHl/fJX0g0+2Og//bWpE F2V5/M5l6YW1T8oLcT9rIOH6oq9M0LMNRgFeiNNnilGIeeIgtOGBRueG4CZiZAvsRPJkrO70 1R2SrdkCIvwGUzUAxx1NfBWb+vgm4fgkW/MotGonceM5v0qfSKKXasWvDctkK28aG+IoQzmi FjXNW4+ju4zeQFYwD4ZDWqw9MqO0hVb24uW3dxtQhbfmOLgJ/PEDMQaFuANbW1c+iR0BQA3D Go/EeMY4kpN8on6Aqt/S/4JVltudfQ9OXdjQsC7netSaB9K3mHGt9aKAAB7RzlRY00DKkYS/ /eQwLzGPmK7yX13M68mMDjBs6mIR8t/E1S5OdBNhHRPNPlEbwugR4KaiCsN5yqzJoSV99fKY z2VyxjWPaG8yhHE+jmKUgIBKTfFUQEfkriQR4EASoeJ+soaMTiFDBij1Zw5n3ndLRFMB1ZCl fZLER36mAgW4m4kP83TWnDiJLxOxSOxifV8HpTFjff902H85cybg9KMwrfPDr6W19GGk5Vo1 fkza5krRMGbKWb7+74Evusi0ZxJLIOFwp5Y8eVqUMZaAD3f1ZX1M3pgXOp20QgAy+2KvMHij rLa4q+tMGRzYYD1BnFVSVdXAX5VOoTmHBcDz67DkuRwk2Byp1sgd407oEOmSwrNJlKS0TPCm xUJ2fdSQF+1/MMSRfee49vtMvz7cOrC5Ag0EW5JHigEQANiBmIFAfRNH3nzYNWC0yC+tfx3z sUwAsH1VaBM/cTib+yKtbBOSIlXWjJZWX3MHwoI/1LeGghB2mxkkX1L0pJ/vj1eXNR+sFZ32 0pYcl61Fxg/5fioG4QDTM4i3i7NR5PxDnc6UVaynSlII93DedRhZ1ROtdn4vyMgzsDiqhbL7 BthDOt5KxjqdRk4qRPSw7BovEqZLOcG5IJtf/zZUzRbM7SBljEbOAfekDGx1Br+RrYSD7/Ef Pwwzou9T8315IpBpIHyQF/dZNk3iFiB9Ed5CA71ZRYV5YoLWE9lL0j9kxOLQ5vHnX3mVq7QZ Bc7nzwZ6UhQgYmrG5+RWvuiPpGwvDRIsugJUGXucYkAQh5kuNblmkwpv6u9rNMjCNbzAylOa qdogra5EW+RUSbRz0b4iIr8nnZeAlh7BihCe7JjOwbDjoBEEEtSfVc4hD/LENqpcYVrChphf aOLB9YIXhnVDTVvMc9OklWT/81HzAaDQqOQCzEfY92199Ct9/CwRoQ2OpO8TO5+8A7b9Nb33 nmxMn09mb48ruRacMrfHxCWbgU4w9SEfbip4GcS5wGG6yTC+hw55Iwnnwus40NrJ0GEr8a4r cdsLbkvlyoNHB8ZGgyJ4aFCQ1V4qE1BnlTk7Z8BYBUkJM1odPSkVvHpCnMUjVpJ3hEOC+73Z YH1dh7lZABEBAAGJAjwEGAEKACYWIQRx42Dk0QxkiSLzSdBAeX98lfSDTwUCW5JHigIbDAUJ Ca6EAAAKCRBAeX98lfSDTz8DEACMh3poeUb+gWNF4RWFZuLteZVo0+E1JLYXQkmtrRBLXviP +Qy0pXyFAVxLM4hNIBoIDYfK9BcwrBYf7AwSKrH0GiNwFpgHCkbZd6qoZy2gB+adTnCpVCTJ KJetsH/8awkrChJWMK0ckGf3EeWMPvawG7kW7FBz70NYEZ0pOMiaEZNVtzD3wwbYWUiDFYth 83XGglOExg+1ShTW5XjQPRrdyJAO+aUW4o3lVjfyUJXMgI4rmhMiLVm06GuNrbpKIF0s+4Vd jQAjhrDQjfoXi9CkfsA/cONseuHNv1JGj3RqHiqHJq1dbrpodXp925zGDAnUGxCOBPoFopAH gVzR89GTut059GpwqsddZmU6y7rqifuam/ekJ+QRwc16vgt7pHqCrTY8WPxRZr2UpFU1wlTo COdeiFep1gq1F9jzFjJnoMaAdmC6k7bgAA+RQusOgIhJL0jIej7DoAHxmxFFCfRy+lDtpXwF gQ8HMvzHI65QWmQnMo7s6SQH/ZH5s1yR6SJq8+3lDz+dCuT42qJVqIPVvxd10LW0FNN+t7HF eLadU6ekSgD13/EYMYXlvNHkw7dAItSDxIzgRyykLz0bCU9xwNWoS4Z43+ifF9anJ+uR0ltW El1j++h6ZrD3LLuCgJIt1so0m49GzdcSpOI7LCwMlacyvafiEyjUn+tSNDsnfw== Message-ID: <7e496a87-9c94-63e7-e48e-7f719c2e7874@delphij.net> Date: Sun, 20 Jan 2019 22:34:31 -0800 User-Agent: Thunderbird MIME-Version: 1.0 In-Reply-To: <1691666278.63816.1547976245836.JavaMail.zimbra@gausus.net> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="ZgN8q2zxDFUkdMnT3BGlRx1UpT5ovRvhq" X-Rspamd-Queue-Id: 1F65E70B6C X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphij.net header.s=m7e2 header.b=PKATH1XQ; dmarc=pass (policy=none) header.from=delphij.net; spf=pass (mx1.freebsd.org: domain of delphij@delphij.net designates 64.62.153.212 as permitted sender) smtp.mailfrom=delphij@delphij.net X-Spamd-Result: default: False [-6.78 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_REPLYTO(0.00)[d@delphij.net]; XM_UA_NO_VERSION(0.01)[]; R_SPF_ALLOW(-0.20)[+mx]; HAS_ATTACHMENT(0.00)[]; TO_DN_NONE(0.00)[]; DKIM_TRACE(0.00)[delphij.net:+]; MX_GOOD(-0.01)[cached: anubis.delphij.net]; DMARC_POLICY_ALLOW(-0.50)[delphij.net,none]; NEURAL_HAM_SHORT(-1.00)[-0.996,0]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+]; IP_SCORE(-0.68)[asn: 6939(-3.35), country: US(-0.08)]; ASN(0.00)[asn:6939, ipnet:64.62.128.0/18, country:US]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[delphij.net:s=m7e2]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; REPLYTO_DOM_EQ_FROM_DOM(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jan 2019 06:34:44 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --ZgN8q2zxDFUkdMnT3BGlRx1UpT5ovRvhq Content-Type: multipart/mixed; boundary="Ff9DMMrcplC0uSkPEXxyawPmsOV379354"; protected-headers="v1" From: Xin Li Reply-To: d@delphij.net To: freebsd-fs@freebsd.org Message-ID: <7e496a87-9c94-63e7-e48e-7f719c2e7874@delphij.net> Subject: Re: ZFS on Hardware RAID References: <1180280695.63420.1547910313494.JavaMail.zimbra@gausus.net> <92646202.63422.1547910433715.JavaMail.zimbra@gausus.net> <1691666278.63816.1547976245836.JavaMail.zimbra@gausus.net> In-Reply-To: <1691666278.63816.1547976245836.JavaMail.zimbra@gausus.net> --Ff9DMMrcplC0uSkPEXxyawPmsOV379354 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 1/20/19 01:24, Maciej Jan Broniarz wrote: > Hi, >=20 > I am thinking about the scenario with ZFS on single disks configured to= RAID0 by hw raid. > Please correct me, if i'm wrong, but HW Raid uses a dedicated unit to p= rocess all RAID related work (eg. parity checks). > With ZFS the job is done by CPU. How significant is the performance los= s in that particular case? But you don't really skip that cost when layering ZFS on top of a hardware RAID. ZFS still does checksumming, etc., and you really want it to do its job because a) it would tell you which files were corrupted in the worst case scenario, and b) when doing scrubs, ZFS only scans areas with data, while in hardware RAID case, the filesystem is opaque and it has to scan everything. >=20 > mjb >=20 >=20 > ----- Oryginalna wiadomo=C5=9B=C4=87 ----- > Od: "andy thomas" > Do: "Rich" > DW: "Maciej Jan Broniarz" , "freebsd-fs" > Wys=C5=82ane: niedziela, 20 stycze=C5=84 2019 9:45:21 > Temat: Re: ZFS on Hardware RAID >=20 > I have to agree with your comment that hardware RAID controllers add=20 > another layer of opaque complexity but for what it's worth, I have to=20 > admit ZFS on h/w RAID does work and can work well in practice. >=20 > I run a number of very busy webservers (Dell PowerEdge 2950 with LSI=20 > MegaRAID SAS 1078 controllers) with the first two disks in RAID 1 as th= e=20 > FreeBSD system disk and the remaining 4 disks configured as RAID 0 virt= ual=20 > disks making up a ZFS RAIDz1 pool with 3 disks plus one hot spare.=20 > With 6-10 jails running on each server, these have been running for=20 > years with no problems at all. >=20 > Andy >=20 > On Sat, 19 Jan 2019, Rich wrote: >=20 >> The two caveats I'd offer are: >> - RAID controllers add an opaque complexity layer if you have problems= >> - e.g. if you're using single-disk RAID0s to make a RAID controller >> pretend to be an HBA, if the disk starts misbehaving, you have an >> additional layer of behavior (how the RAID controller interprets >> drives misbehaving and shows that to the OS) to figure out whether the= >> drive is bad, the connection is loose, the controller is bad, ... >> - abstracting the redundancy away from ZFS means that ZFS can't >> recover if it knows there's a problem but the underlying RAID >> controller doesn't - that is, say you made a RAID-6, and ZFS sees some= >> block fail checksum. There's not a way to say "hey that block was >> wrong, try that read again with different disks" to the controller, so= >> you're just sad at data loss on your nominally "redundant" array. >> >> - Rich >> >> On Sat, Jan 19, 2019 at 11:44 AM Maciej Jan Broniarz wrote: >>> >>> Hi, >>> >>> I want to use ZFS on a hardware-raid array. I have no option of makin= g it JBOD. I know it is best to use ZFS on JBOD, but >>> that possible in that particular case. My question is - how bad of an= idea is it. I have read very different opinions on that subject, but non= e of them seems conclusive. >>> >>> Any comments and especially case studies are most welcome. >>> All best, >>> mjb >>> _______________________________________________ >>> freebsd-fs@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"= >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> >=20 >=20 > ---------------------------- > Andy Thomas, > Time Domain Systems >=20 > Tel: +44 (0)7866 556626 > Fax: +44 (0)20 8372 2582 > http://www.time-domain.co.uk > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >=20 --Ff9DMMrcplC0uSkPEXxyawPmsOV379354-- --ZgN8q2zxDFUkdMnT3BGlRx1UpT5ovRvhq Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJcRWf7AAoJEEB5f3yV9INPj24P/R3MJYhg1QG9hydGuqitIWWZ nhK8zPfZ09YAt/PnHr7VWP6zMIPsLMP1NHYxmJd2erC2GdsI7GOM9ZgNBz+zb/u4 uYQdbZz3ObFVQIayGoHpuxTxJnpPXmjj4LkwoVPY/5SIcGplGvEDNBVn1aPDWbwl PXu9IMjekgQ4Ih4a4X2o5Vb3CY51FuIVNNzr+G5AM6kA47bXjSwO4SDHeFUJvwsu f44XZECeaa2VjCCzNwoWVlVUT5M8p5VnAmGDhBYD4/wpJvhffhUANc+HAsl2ozaH SkeRfCitbFLKWKtXQLwtb2rvONQMSAh9qLFv1q5FyVuRWqvHDJd7s7DoK998UgJg n2hck3UG4/kiDE76kgchi7YISioUgd081cGPmY4mtTJg8NEq5kVdASQxxTqtWpNM 9UdJ1fz8TpxB9X8D/szx330WunGoiFEz15Ia71k7tEaB1hqYvwrpfwvTCEqLFbRC 5gHq9yKlpD5T4gJLYiLsH+R2Ceh5YI1mlNttzn86Ukb2nqOh3r4KzEvXuW1ZGZnL bG4d1L0hEmJJkWdTz6XXnw6W0xZDzNr317lUfey35ASFgR6d4mRSdgN19FTTmkVg +6zxOsrZ0PS69vTaUHVjYAq3sbVGXrsyzNkBN9o5RU0O65SvimGujr0aBaIDmXkw 9O2TcnTEjP4j7NRT63tn =yj9f -----END PGP SIGNATURE----- --ZgN8q2zxDFUkdMnT3BGlRx1UpT5ovRvhq--