From owner-freebsd-fs@freebsd.org Sun Jan 20 09:14:42 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D4F3514ABE2C for ; Sun, 20 Jan 2019 09:14:41 +0000 (UTC) (envelope-from andy@time-domain.co.uk) Received: from mail0.time-domain.co.uk (host81-142-251-212.in-addr.btopenworld.com [81.142.251.212]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail0", Issuer "mail0" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 042338D580 for ; Sun, 20 Jan 2019 09:14:39 +0000 (UTC) (envelope-from andy@time-domain.co.uk) Received: from localhost (localhost [127.0.0.1]) by mail0.time-domain.co.uk (8.15.2/8.15.2) with ESMTPS id x0K8jL9L015634 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 20 Jan 2019 08:45:24 GMT (envelope-from andy@time-domain.co.uk) Date: Sun, 20 Jan 2019 08:45:21 +0000 (GMT) From: andy thomas X-X-Sender: andy-tds@mail0.time-domain.co.uk To: Rich cc: Maciej Jan Broniarz , freebsd-fs Subject: Re: ZFS on Hardware RAID In-Reply-To: Message-ID: References: <1180280695.63420.1547910313494.JavaMail.zimbra@gausus.net> <92646202.63422.1547910433715.JavaMail.zimbra@gausus.net> User-Agent: Alpine 2.21 (BSF 202 2017-01-01) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Rspamd-Queue-Id: 042338D580 X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [2.46 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.77)[-0.771,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_SPAM_SHORT(0.88)[0.884,0]; HFILTER_HOSTNAME_4(2.50)[host81-142-251-212.in-addr.btopenworld.com]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[time-domain.co.uk]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-0.90)[-0.903,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[smtp0.time-domain.co.uk]; IP_SCORE(-0.14)[asn: 2856(-0.62), country: GB(-0.09)]; R_SPF_NA(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:2856, ipnet:81.128.0.0/12, country:GB]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jan 2019 09:14:42 -0000 I have to agree with your comment that hardware RAID controllers add another layer of opaque complexity but for what it's worth, I have to admit ZFS on h/w RAID does work and can work well in practice. I run a number of very busy webservers (Dell PowerEdge 2950 with LSI MegaRAID SAS 1078 controllers) with the first two disks in RAID 1 as the FreeBSD system disk and the remaining 4 disks configured as RAID 0 virtual disks making up a ZFS RAIDz1 pool with 3 disks plus one hot spare. With 6-10 jails running on each server, these have been running for years with no problems at all. Andy On Sat, 19 Jan 2019, Rich wrote: > The two caveats I'd offer are: > - RAID controllers add an opaque complexity layer if you have problems > - e.g. if you're using single-disk RAID0s to make a RAID controller > pretend to be an HBA, if the disk starts misbehaving, you have an > additional layer of behavior (how the RAID controller interprets > drives misbehaving and shows that to the OS) to figure out whether the > drive is bad, the connection is loose, the controller is bad, ... > - abstracting the redundancy away from ZFS means that ZFS can't > recover if it knows there's a problem but the underlying RAID > controller doesn't - that is, say you made a RAID-6, and ZFS sees some > block fail checksum. There's not a way to say "hey that block was > wrong, try that read again with different disks" to the controller, so > you're just sad at data loss on your nominally "redundant" array. > > - Rich > > On Sat, Jan 19, 2019 at 11:44 AM Maciej Jan Broniarz wrote: >> >> Hi, >> >> I want to use ZFS on a hardware-raid array. I have no option of making it JBOD. I know it is best to use ZFS on JBOD, but >> that possible in that particular case. My question is - how bad of an idea is it. I have read very different opinions on that subject, but none of them seems conclusive. >> >> Any comments and especially case studies are most welcome. >> All best, >> mjb >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > ---------------------------- Andy Thomas, Time Domain Systems Tel: +44 (0)7866 556626 Fax: +44 (0)20 8372 2582 http://www.time-domain.co.uk