From owner-freebsd-stable@freebsd.org Fri May 3 10:09:14 2019 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D214A158B0D2 for ; Fri, 3 May 2019 10:09:13 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from cu1176c.smtpx.saremail.com (cu1176c.smtpx.saremail.com [195.16.148.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 81E2F6A59B for ; Fri, 3 May 2019 10:09:12 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from [172.16.8.250] (unknown [192.148.167.11]) by proxypop02.sare.net (Postfix) with ESMTPA id 686249DC8E0; Fri, 3 May 2019 12:09:01 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.8\)) Subject: Re: ZFS... From: Borja Marcos In-Reply-To: Date: Fri, 3 May 2019 12:09:00 +0200 Cc: freebsd-stable@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <30506b3d-64fb-b327-94ae-d9da522f3a48@sorbs.net> <56833732-2945-4BD3-95A6-7AF55AB87674@sorbs.net> <3d0f6436-f3d7-6fee-ed81-a24d44223f2f@netfence.it> <17B373DA-4AFC-4D25-B776-0D0DED98B320@sorbs.net> <70fac2fe3f23f85dd442d93ffea368e1@ultra-secure.de> <70C87D93-D1F9-458E-9723-19F9777E6F12@sorbs.net> <58DA896C-5312-47BC-8887-7680941A9AF2@sarenet.es> To: Pete French X-Mailer: Apple Mail (2.3445.104.8) X-Rspamd-Queue-Id: 81E2F6A59B X-Spamd-Bar: ----- X-Spamd-Result: default: False [-5.15 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:195.16.148.0/24]; MV_CASE(0.50)[]; MX_GOOD(-0.01)[cached: smtp.sarenet.es]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[sarenet.es,reject]; SUBJ_ALL_CAPS(0.45)[6]; NEURAL_HAM_SHORT(-0.90)[-0.901,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:3262, ipnet:195.16.128.0/19, country:ES]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[151.148.16.195.list.dnswl.org : 127.0.10.0]; IP_SCORE(-2.39)[ip: (-6.15), ipnet: 195.16.128.0/19(-3.35), asn: 3262(-2.48), country: ES(0.04)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 03 May 2019 10:09:14 -0000 > On 3 May 2019, at 11:55, Pete French = wrote: >=20 >=20 >=20 > On 03/05/2019 08:09, Borja Marcos via freebsd-stable wrote: >=20 >> The right way to use disks is to give ZFS access to the plain CAM = devices, not thorugh some so-called JBOD on a RAID >> controller which, at least for a long time, has been a *logical* = =E2=80=9CRAID0=E2=80=9D volume on a single disk. That additional layer = can >> completely break the semantics of transaction writes and cache = flushes. >> With some older cards it can be tricky to achieve, from patching = source drivers to enabling a sysctl tunable or even >> flashing the card to turn it into a plain HBA with no RAID features = (or minimal ones). >=20 > Oddly enough I got bitten by something like this yesteray. I have a = machine containing an HP P400 RAID controller, which is nice enough, but = I run ZFS so I have made the drives all into RAID-0 as being as close as = I can get to accessing the raw SAS drives. >=20 > BSD seems them as da0, da1, da2, da3 - but the RAID controller oly = presents one of them to the BIOS, so my booting has to be all from that = drive. This has been da0 for as long as I can remember, but yesteday it = decided to start using what BSD sees as da1. Of course this is very hard = to recognise as da0 and da1 are pretty much mirrors of each other. Spent = a long time trying to work out why the fixes I was applying to da0 were = not being used at boot time. Hmm What happens when you do a =E2=80=9Ccamcontrol devlist=E2=80=9D? Camcontrol tags da0 -v? How is the controller recognized by FreeBSD? For some of them it=E2=80=99s= possible to instruct the controller to present the physical devices to = CAM. Of course you need to be careful to avoid any logical volume configuration in that = case.=20 But I would only tinker with this at system installation time, making = such a change on a running system with valid data can be disastrous. For mfi recognized cards there is a tunable: = hw.mfi.allow_cam_disk_passthrough For aac cards it was a matter of commenting a couple of source code = lines in the driver (at your own risk of course). I=E2=80=99ve been = running a=20 server for years doing that. Borja.