From owner-freebsd-fs@freebsd.org Tue Jan 22 11:53:06 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E3E1614B9423 for ; Tue, 22 Jan 2019 11:53:05 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from cu01176b.smtpx.saremail.com (cu01176b.smtpx.saremail.com [195.16.151.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0B8BE880C2 for ; Tue, 22 Jan 2019 11:53:04 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from [172.16.8.5] (unknown [192.148.167.11]) by proxypop01.sare.net (Postfix) with ESMTPA id C50069DD42B; Tue, 22 Jan 2019 12:53:00 +0100 (CET) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Re: ZFS on Hardware RAID From: Borja Marcos In-Reply-To: <335e44ec-7c76-8dbd-f587-46e6a9266efc@multiplay.co.uk> Date: Tue, 22 Jan 2019 12:52:59 +0100 Cc: jdelisle , freebsd-fs Content-Transfer-Encoding: quoted-printable Message-Id: References: <1180280695.63420.1547910313494.JavaMail.zimbra@gausus.net> <92646202.63422.1547910433715.JavaMail.zimbra@gausus.net> <1691666278.63816.1547976245836.JavaMail.zimbra@gausus.net> <92746659-4B3F-415C-BB6A-6C99837AFAF2@sarenet.es> <335e44ec-7c76-8dbd-f587-46e6a9266efc@multiplay.co.uk> To: Steven Hartland X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: 0B8BE880C2 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of borjam@sarenet.es designates 195.16.151.151 as permitted sender) smtp.mailfrom=borjam@sarenet.es X-Spamd-Result: default: False [-1.24 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-0.34)[-0.340,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:195.16.150.0/23]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[sarenet.es]; NEURAL_HAM_LONG(-0.98)[-0.983,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: smtp.sarenet.es]; NEURAL_HAM_SHORT(-0.12)[-0.118,0]; RCVD_IN_DNSWL_NONE(0.00)[151.151.16.195.list.dnswl.org : 127.0.10.0]; IP_SCORE(0.01)[country: ES(0.05)]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:3262, ipnet:195.16.128.0/19, country:ES]; FREEMAIL_CC(0.00)[gmail.com]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Jan 2019 11:53:06 -0000 > On 21 Jan 2019, at 16:40, Steven Hartland = wrote: >=20 >>=20 > Typically yes it does. >> The same could be said of other functionalities available when you = talk directly >> to the disks. >>=20 >>=20 >>=20 >>> That's why I did as I describe above. I didn't have the money to = buy an >>> HBA until a year or two later, and I had good backups, so I used the = PERC >>> 6/e. >>>=20 >> Depending on the LSI model used in that card you can make it behave = like an HBA.=20 >>=20 >>=20 > Indeed you can for most, but its not an easy or error free option, = requires force flashing from IR to IT firmware, and if it goes wrong you = can end up with a bricked controller. >=20 > We've used LSI's in IR mode (RAID) but as individual disks JBOD with = ZFS raid2z or mirroring on top for a long time and on lots of machines, = not had any issues to report. So while I would never use hardware RAID = from those controllers with ZFS, JBOD on them is an easy and reliable = option in our experience. Sorry to insist, but maybe we are talking of different stuff? My = apologies in advance if I am confused and/or my information is outdated.=20 JBOD mode: a mode offered by some LSI Logic IR controllers to create = virtual devices mapped so that each one is effectively. For example, with the mfi driver the =E2=80=9Cdisks=E2=80=9D appear as = =E2=80=9Cmfisyspd=E2=80=9D devices while the =E2=80=9Cconventional=E2=80=9D= logical volumes appear as=20 =E2=80=9Cmfid=E2=80=9D. My information could be outdated though?=20 HBA mode: when the card is in IT mode *or* it does expose the actual = targets to the CAM layer. In the past I did it by manually patching the drivers and I=E2=80=99ve kept systems running = smoothly for many years despite using IR cards.=20 Currently you can use hw.mfi.allow_cam_disk_passthrough tunable to = achieve the same effect without ugly tinkering.=20 Unless I am terribly wrong, when I tried the JBOD mode as defined above = (with LSI SAS3 cards and IR firmware) the mfisyspd devices were not actual CAM devices.=20 Actually I find =E2=80=9CJBOD=E2=80=9D an artificial and confusing term = because it can mean so many things. I would have rather preferred LSI to avoid it and provide either real, transparent access to the = actual targets insteaf of somewhat =E2=80=9Cvirtualizing=E2=80=9D them. Again, I may be confused? I haven=E2=80=99t tried to configure a LSI IR = card as =E2=80=9CJBOD=E2=80=9D for a long time. Maybe now they do expose the targets instead of defining logical volumes? Thanks! Borja.