From owner-freebsd-fs@freebsd.org Sun Jan 20 09:59:29 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5BB2D14ACD63 for ; Sun, 20 Jan 2019 09:59:29 +0000 (UTC) (envelope-from andy@time-domain.co.uk) Received: from mail0.time-domain.co.uk (host81-142-251-212.in-addr.btopenworld.com [81.142.251.212]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail0", Issuer "mail0" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DCC58E72E for ; Sun, 20 Jan 2019 09:59:28 +0000 (UTC) (envelope-from andy@time-domain.co.uk) Received: from localhost (localhost [127.0.0.1]) by mail0.time-domain.co.uk (8.15.2/8.15.2) with ESMTPS id x0K9xPh4016016 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 20 Jan 2019 09:59:25 GMT (envelope-from andy@time-domain.co.uk) Date: Sun, 20 Jan 2019 09:59:25 +0000 (GMT) From: andy thomas X-X-Sender: andy-tds@mail0.time-domain.co.uk To: Maciej Jan Broniarz cc: Rich , freebsd-fs Subject: Re: ZFS on Hardware RAID In-Reply-To: <1691666278.63816.1547976245836.JavaMail.zimbra@gausus.net> Message-ID: References: <1180280695.63420.1547910313494.JavaMail.zimbra@gausus.net> <92646202.63422.1547910433715.JavaMail.zimbra@gausus.net> <1691666278.63816.1547976245836.JavaMail.zimbra@gausus.net> User-Agent: Alpine 2.21 (BSF 202 2017-01-01) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Rspamd-Queue-Id: 4DCC58E72E X-Spamd-Bar: + Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [1.84 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.79)[-0.794,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_SPAM_SHORT(0.30)[0.295,0]; HFILTER_HOSTNAME_4(2.50)[host81-142-251-212.in-addr.btopenworld.com]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[time-domain.co.uk]; AUTH_NA(1.00)[]; NEURAL_HAM_LONG(-0.91)[-0.912,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: smtp0.time-domain.co.uk]; IP_SCORE(-0.14)[asn: 2856(-0.61), country: GB(-0.09)]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:2856, ipnet:81.128.0.0/12, country:GB]; FREEMAIL_CC(0.00)[gmail.com]; RCVD_TLS_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 20 Jan 2019 09:59:29 -0000 I don't h/w RAID controllers do any parity checking, etc for RAID 0 virtual disks containing only one disk? I know ZFS on h/w raid can't possibly be optimal and use of JBOD, pass-thru or plain HBA is to be preferred at all times but older RAID controllers don't support non-RAID operation - after all, h/w RAID controller design is primarily intended for the Windows Server market where Windows didn't support any kind of software RAID scheme at that time (it might do know, I don't know). All I can say is ZFS on h/w RAID does work and we've been using it in production for years. I also have a system at home that has FreeBSD installed as ZFS on root on a LSI SAS RAID controller with four RAID 0 virtual disks, again with no problems. Below is the output from one of our servers: root@penguin14:~ # uname -a FreeBSD penguin14 9.3-RELEASE FreeBSD 9.3-RELEASE #0 r268512: Thu Jul 10 23:44:39 UTC 2014 root@snap.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 root@penguin14:~ # uptime 9:39AM up 1575 days, 17:52, 2 users, load averages: 0.27, 0.26, 0.28 root@penguin14:~ # zpool status pool: penguin14_tank state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM penguin14_tank ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 mfid1p1 ONLINE 0 0 0 mfid2p1 ONLINE 0 0 0 mfid3p1 ONLINE 0 0 0 spares mfid4p1 AVAIL errors: No known data errors root@penguin14:~ # jls -h jid host.hostname jid host.hostname 19 penguin14web4 20 penguin14web3 21 penguin14web2 23 penguin14ssl 24 p14mysql55 25 noticeboard_3pb Andy On Sun, 20 Jan 2019, Maciej Jan Broniarz wrote: > Hi, > > I am thinking about the scenario with ZFS on single disks configured to RAID0 by hw raid. > Please correct me, if i'm wrong, but HW Raid uses a dedicated unit to process all RAID related work (eg. parity checks). > With ZFS the job is done by CPU. How significant is the performance loss in that particular case? > > mjb > > > ----- Oryginalna wiadomo?? ----- > Od: "andy thomas" > Do: "Rich" > DW: "Maciej Jan Broniarz" , "freebsd-fs" > Wys?ane: niedziela, 20 stycze? 2019 9:45:21 > Temat: Re: ZFS on Hardware RAID > > I have to agree with your comment that hardware RAID controllers add > another layer of opaque complexity but for what it's worth, I have to > admit ZFS on h/w RAID does work and can work well in practice. > > I run a number of very busy webservers (Dell PowerEdge 2950 with LSI > MegaRAID SAS 1078 controllers) with the first two disks in RAID 1 as the > FreeBSD system disk and the remaining 4 disks configured as RAID 0 virtual > disks making up a ZFS RAIDz1 pool with 3 disks plus one hot spare. > With 6-10 jails running on each server, these have been running for > years with no problems at all. > > Andy > > On Sat, 19 Jan 2019, Rich wrote: > >> The two caveats I'd offer are: >> - RAID controllers add an opaque complexity layer if you have problems >> - e.g. if you're using single-disk RAID0s to make a RAID controller >> pretend to be an HBA, if the disk starts misbehaving, you have an >> additional layer of behavior (how the RAID controller interprets >> drives misbehaving and shows that to the OS) to figure out whether the >> drive is bad, the connection is loose, the controller is bad, ... >> - abstracting the redundancy away from ZFS means that ZFS can't >> recover if it knows there's a problem but the underlying RAID >> controller doesn't - that is, say you made a RAID-6, and ZFS sees some >> block fail checksum. There's not a way to say "hey that block was >> wrong, try that read again with different disks" to the controller, so >> you're just sad at data loss on your nominally "redundant" array. >> >> - Rich >> >> On Sat, Jan 19, 2019 at 11:44 AM Maciej Jan Broniarz wrote: >>> >>> Hi, >>> >>> I want to use ZFS on a hardware-raid array. I have no option of making it JBOD. I know it is best to use ZFS on JBOD, but >>> that possible in that particular case. My question is - how bad of an idea is it. I have read very different opinions on that subject, but none of them seems conclusive. >>> >>> Any comments and especially case studies are most welcome. >>> All best, >>> mjb >>> _______________________________________________ >>> freebsd-fs@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> > > > ---------------------------- > Andy Thomas, > Time Domain Systems > > Tel: +44 (0)7866 556626 > Fax: +44 (0)20 8372 2582 > http://www.time-domain.co.uk > > ---------------------------- Andy Thomas, Time Domain Systems Tel: +44 (0)7866 556626 Fax: +44 (0)20 8372 2582 http://www.time-domain.co.uk