From owner-freebsd-scsi@freebsd.org Sun Aug 6 08:48:01 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9CB2DDCC317 for ; Sun, 6 Aug 2017 08:48:01 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 06ADF7D663 for ; Sun, 6 Aug 2017 08:48:00 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from [IPv6:2a02:2698:26:9b0e:d7a:6d28:b07e:5ba6] (dynamic-2a02-2698-26-0-0.perm.ertelecom.ru [IPv6:2a02:2698:26:9b0e:d7a:6d28:b07e:5ba6] (may be forged)) (authenticated bits=0) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPSA id v768lsbN035507 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Sun, 6 Aug 2017 13:47:54 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1502009274; bh=13yyWfGKbkVJlk7l+8nQolBKYwy15eGiLCEVfJCUCvM=; h=Subject:Cc:References:From:Date:In-Reply-To; b=aJcI1k7lh3ztkBtwLgZztAzKRp7RA82dyM7l+7jEXniaEyAPXpuoYtJYyQVk5W8Ww htf6JpQNoYGne4Sd1o19nqdV207Mwy+KoeFYLvf7W0FC5KsK4gw3c+8SoC+oX6DNwu MdQcMiuYx3t223ixItyNzId/gnfWoBzbIgxt/mjU= Subject: Re: camcontrol sanitize Cc: "freebsd-scsi@freebsd.org" References: <22536136-e8c0-17c9-5c8d-e8fed11022b7@norma.perm.ru> <20170727152207.GA91629@mithlond.kdm.org> From: "Eugene M. Zheganin" Message-ID: Date: Sun, 6 Aug 2017 13:47:54 +0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <20170727152207.GA91629@mithlond.kdm.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-GB X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Aug 2017 08:48:01 -0000 Hi, On 27.07.2017 20:22, Kenneth D. Merry wrote: > > CAM status 0xe is a bus reset. Do you see anything in dmesg that might > shed more light on it? If you use the -v switch, camcontrol will print out > more verbose error information, and should provide text for the error > number. > > What kind of controller are you using? I'm guessing you are using a 12Gb > SAS controller given what I see above. What firmware revision is on the > controller? > > The drive is a SATA drive, and SANITIZE is a SCSI command. There are > translations defined in the SAT spec to translate SANITIZE to an ATA > command. I don't know whether your controller supports those translations > or whether the drive supports the underlying ATA BLOCK ERASE EXT command. > It's an LSI3008-based HBA (SAS9300-4i4e): # mprutil show adapter mpr0 Adapter: Board Name: SAS9300-4i4e Board Assembly: H3-25515-00G Chip Name: LSISAS3008 Chip Revision: ALL BIOS Revision: 8.11.00.00 Firmware Revision: 14.00.00.00 Integrated RAID: no PhyNum CtlrHandle DevHandle Disabled Speed Min Max Device 0 0001 0009 N 12 3.0 12 SAS Initiator 1 0001 0009 N 12 3.0 12 SAS Initiator 2 0001 0009 N 12 3.0 12 SAS Initiator 3 0001 0009 N 12 3.0 12 SAS Initiator 4 N 3.0 12 SAS Initiator 5 N 3.0 12 SAS Initiator 6 N 3.0 12 SAS Initiator 7 N 3.0 12 SAS Initiator Don't remember seeing anything erroneous in dmesg. Unfortunately, right now I cannot pepeat this sequence, because I've already dd'ed disks with zeroes and put them back into the production. Thanks. Eugene. From owner-freebsd-scsi@freebsd.org Mon Aug 7 09:16:06 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47AD7DC6298 for ; Mon, 7 Aug 2017 09:16:06 +0000 (UTC) (envelope-from janm@transactionware.com) Received: from mail3.transactionware.com (mail.transactionware.com [203.14.245.7]) by mx1.freebsd.org (Postfix) with SMTP id A33BE2ADC for ; Mon, 7 Aug 2017 09:16:03 +0000 (UTC) (envelope-from janm@transactionware.com) Received: (qmail 99899 invoked by uid 907); 7 Aug 2017 09:09:19 -0000 Received: from Unknown (HELO jmmacpro.tmst.com.au) (203.14.245.130) (smtp-auth username janm, mechanism plain) by mail3.transactionware.com (qpsmtpd/0.84) with (ECDHE-RSA-AES256-GCM-SHA384 encrypted) ESMTPSA; Mon, 07 Aug 2017 19:09:19 +1000 From: Jan Mikkelsen Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: BIO_FLUSH, drive write cache: Which drivers do the Right Thing? Message-Id: <6E659EED-746D-4936-9C53-702D72DA84B8@transactionware.com> Date: Mon, 7 Aug 2017 19:11:18 +1000 To: freebsd-scsi@freebsd.org X-Mailer: Apple Mail (2.3273) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Aug 2017 09:16:06 -0000 Hi, I'm looking at how ZFS interacts with drive-based write caches, so that = I can work out what is =E2=80=9Csafe=E2=80=9D. The premise is that drive write caching is safe because ZFS can flush = the drive write cache when necessary to ensure updates get to the = physical media in the correct order. For this to work, the actual = command has to get to the drive. This seems to work for ata(4), da(4) and amr(4). Other non-CAM RAID controllers, eg. mfi(4) don=E2=80=99t seem to support = this at all, including for JBOD drives like mfisyspd. For CAM-based controllers (eg. mpr(4) or mrsas(4)), it seems that = controllers like mpr(4) pass the SCSI command on as expected. RAID = controllers implementing CAM have to do the right thing to make it work; = mrsas(4) seems to find SYNCHRONIZE_CACHE and do something but I=E2=80=99m = not sure that it really sending a cache flush command to any underlying = drives. =46rom all of this, this is what I see: - ata(4) and amr(4) pass the flush command to the drive, allowing the = use of drive write cache. - da(4) on =E2=80=9Cplain SCSI=E2=80=9D controllers pass the command to = the drive, allowing the use of drive write cache. - da(4) for RAID controllers may or may not get the command to the = drives. - Other RAID controllers will not currently get the command to the = underlying drives, making drive write cache dangerous. Some questions: Does mrsas(4) really get a write cache flush to the drive? Does it vary = for the JBOD and RAID volume case? Is all this correct? What have I missed? Thanks, Jan Mikkesen From owner-freebsd-scsi@freebsd.org Mon Aug 7 15:27:56 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B65C6DD9705 for ; Mon, 7 Aug 2017 15:27:56 +0000 (UTC) (envelope-from stephen.mcconnell@broadcom.com) Received: from mail-io0-x22e.google.com (mail-io0-x22e.google.com [IPv6:2607:f8b0:4001:c06::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8005670834 for ; Mon, 7 Aug 2017 15:27:56 +0000 (UTC) (envelope-from stephen.mcconnell@broadcom.com) Received: by mail-io0-x22e.google.com with SMTP id j32so3866978iod.0 for ; Mon, 07 Aug 2017 08:27:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; h=from:references:in-reply-to:mime-version:thread-index:date :message-id:subject:to; bh=nud7rpMQAMlVJ2BG5GbZPXHhnCXDh9EQNlsoIWTkYpE=; b=UHJSKYPvB2Zhyue/aI6Fs9WN6sNnymU9u3USjDkrYVo+QPHfARB7meBMBB70Y8I0p0 dbMgoe9PcmWwpsinF1+YNLOr1RL2OS2nqGdDdi+0gDW5692pV+M/4vHFNcvkw/Hu5zuD 4PbVTzNNq3f2R7TyQICoCvjLzg36MlvL0QKis= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:references:in-reply-to:mime-version :thread-index:date:message-id:subject:to; bh=nud7rpMQAMlVJ2BG5GbZPXHhnCXDh9EQNlsoIWTkYpE=; b=UUpAPKzWYREH3kKYDBVsy2+GU53WwyYoZY5NVoUkMg4plFexttAXvKdh5Ciqil6luP tLmZNU9rOprFSzZUtbubJ1tKdcSzecCvPK7cPmAtu6Oy8FgdGmtvlt4OYUR3N/lYQBof oIu9JzwtFCr1VN3uKSnrSdRSGQAATiQyc2JRyo0LWJ8S/ad5zx2D7oLqfJsm3A9mrps6 1ojye28DxepzV5VGdhfhFgfcOMB0JXXNOyeweqYkRp8sqCakh8cYpilQufSEXbf2oaix 7z20dRyLa6zWbAx04NyyAUSg08WixoMr/hcy8LToqhIdD9LGZuIra7k27oBA56PdP9if Hv1A== X-Gm-Message-State: AIVw111rioW/fSyAcTDL0dLl6ysJSQa6Pt9v9xnWFGE8dFhY5r8/s87z BrLic0O1ZbScWvMbCmdYRVTsuyQ24ZDg X-Received: by 10.107.18.94 with SMTP id a91mr939309ioj.119.1502119675822; Mon, 07 Aug 2017 08:27:55 -0700 (PDT) From: Stephen Mcconnell References: <592FDE8C.1090609@omnilan.de> <59303484.1040609@omnilan.de> <59306503.4010007@omnilan.de> <59315A74.9050506@omnilan.de> <20170602153705.GA56018@mithlond.kdm.org> <593198C3.2080902@omnilan.de> <59319917.1050301@omnilan.de> <86a38661813a20d3b349920c2de8962e@mail.gmail.com> <4c93e12f2776d3c5d372b2652883d4af@mail.gmail.com> In-Reply-To: MIME-Version: 1.0 X-Mailer: Microsoft Outlook 14.0 Thread-Index: AQK5uw9AxlTbZs3SRUL7gsvMDeNX4QK127o/Aqh6HjgB6++7AwFsXC9MAdL6oGEBmtNDxAIdfU1yAZuZqewB6HGuVwJh2b5SAYdydE0CD9/RPp/tsuvg Date: Mon, 7 Aug 2017 09:27:54 -0600 Message-ID: Subject: RE: mps(4) blocks panic-reboot To: "Eugene M. Zheganin" , freebsd-scsi@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Aug 2017 15:27:56 -0000 Hi Eugene, Sorry, I just haven't had much time to finish this. I'll do what I can to get to it this week. Steve > -----Original Message----- > From: owner-freebsd-scsi@freebsd.org [mailto:owner-freebsd- > scsi@freebsd.org] On Behalf Of Eugene M. Zheganin > Sent: Saturday, August 05, 2017 11:12 AM > To: Stephen Mcconnell; freebsd-scsi@freebsd.org > Subject: Re: mps(4) blocks panic-reboot > > Hi, > > On 26.06.2017 20:35, Stephen Mcconnell wrote: > > The change I provided to Harry didn't work completely, and I haven't been > > able to find time to look into it further. Since it wasn't a complete fix > > I didn't commit it. I'm not sure when I'll get some time, but hopefully in > > the next week or two. > > > I really hate to bother anyone, but looks like this fix can really make > the difference - so - just in case - any news on this ? > > I'm terribly sorry, and thanks anyway. > Eugene. > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" From owner-freebsd-scsi@freebsd.org Wed Aug 9 14:27:40 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 787D7DCE654 for ; Wed, 9 Aug 2017 14:27:40 +0000 (UTC) (envelope-from mi+m@aldan.algebra.com) Received: from smtp.rcn.com (smtp.rcn.com [69.168.97.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 45B1474550 for ; Wed, 9 Aug 2017 14:27:40 +0000 (UTC) (envelope-from mi+m@aldan.algebra.com) X_CMAE_Category: , , X-CNFS-Analysis: v=2.2 cv=Aa3zJDfG c=1 sm=1 tr=0 a=2ZyxD0Un1vyHw3hS6RhAvw==:117 a=2ZyxD0Un1vyHw3hS6RhAvw==:17 a=kj9zAlcOel0A:10 a=4mwfk3PxCzGL85_U2lIA:9 a=CjuIK1q_8ugA:10 X-CM-Score: 0 X-Scanned-by: Cloudmark Authority Engine X-Authed-Username: YW5hdEByY24uY29t Authentication-Results: smtp02.rcn.cmh.synacor.com header.from=mi+m@aldan.algebra.com; sender-id=neutral Authentication-Results: smtp02.rcn.cmh.synacor.com smtp.mail=mi+m@aldan.algebra.com; spf=neutral; sender-id=neutral Authentication-Results: smtp02.rcn.cmh.synacor.com smtp.user=anat; auth=pass (PLAIN) Received-SPF: neutral (smtp02.rcn.cmh.synacor.com: 38.125.162.34 is neither permitted nor denied by domain of aldan.algebra.com) Received: from [38.125.162.34] ([38.125.162.34:37989] helo=[192.168.25.92]) by smtp.rcn.com (envelope-from ) (ecelerity 3.6.23.54417 r(Core:3.6.23.0)) with ESMTPSA (cipher=DHE-RSA-AES256-GCM-SHA384) id 03/59-53214-ADB1B895; Wed, 09 Aug 2017 10:27:38 -0400 From: "Mikhail T." Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (1.0) Date: Wed, 9 Aug 2017 10:27:37 -0400 Subject: Do I need SAS drives?.. Message-Id: <4DFBCE11-913A-4FC9-937D-463B4D49816C@aldan.algebra.com> To: freebsd-hardware@freebsd.org, freebsd-scsi@freebsd.org X-Mailer: iPhone Mail (14G60) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Aug 2017 14:27:40 -0000 My server has 8 "hot-plug" slots, that can accept both SATA and SAS drives. S= ATA ones tend to be cheaper for the same features (like cache-sizes), what a= m I getting for the extra money spent on SAS? Asking specifically about the protocol differences... It would seem, for exa= mple, SATA can not be as easily hot-plugged, but with camcontrol(8) that sho= uld not be a problem, right? What else? Thank you! --=20 Sent from mobile device, please, pardon shorthand. --=20 Sent from mobile device, please, pardon shorthand.= From owner-freebsd-scsi@freebsd.org Wed Aug 9 15:59:16 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 38B80DD017E; Wed, 9 Aug 2017 15:59:16 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-wr0-x22e.google.com (mail-wr0-x22e.google.com [IPv6:2a00:1450:400c:c0c::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C8A9476FBE; Wed, 9 Aug 2017 15:59:15 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-wr0-x22e.google.com with SMTP id y43so25542206wrd.3; Wed, 09 Aug 2017 08:59:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=dz8yUgnrMCg4tFPB9xlOlypi9GdLXMQSmmy65spPk5U=; b=PpDWgjR1C/YJsOX0HYB+/2kxBc6SwWrb5vODdZlr1BvRf0MfJZnKtPDEILe0gzRobd 1XQ1UAOKS00jST8RuIo5gfRYVD0hHuRcbrPrCqc3xRYQXQY1MY4eT37WmaeybjvYKhlJ A+uuaRtVFVlL5EhhDRPNG0XRYq499Uc7s7itVuMtKOVOvVm+wiPzipSzBvA7NfXgTUpZ QpLp51VVz5V9zCxbdmoNdUCuQ/4Yp5IN6iwmEL0Knw2huZ36xkCdGsLfb8Jsg8fnDE8N ku6sqe3E29tnoXT22EOkQ7y5V+3jgYtmu/7u52Hc5lfQ3MHM+0/TR7qgqokaCpMSQ8Ak uMJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=dz8yUgnrMCg4tFPB9xlOlypi9GdLXMQSmmy65spPk5U=; b=QcKy/mWo1dLMEaVTUKgFwKaaXfVQApu483BDV/CwUlUglVQTO0jYV4Ds3l2mTg7pkK L+BB3eiBsuC8k+vr+Y9uvDtkP6bYztRAybHvM5CeMrzGd1uPF3mkfNEipKobfskxvaSq vYD4xXbHJvYDfyOwRve+GmYgefowWV7zrI2Dwgu/TSj28PgSERH+SE/rbgJssnfdwTQl RZnpArHDD6pUEV8o8mv/zXUpgrMSfPLFML8qsxXxYQ2tBCvMBOFzsoVy+mQecFcI1tCW hpkFQr7o6tRRIifZ0HX7vx2nXM8GL47nrAhlag8jmein/smu3bQICNeKhdmw/6B/cuFK gAMQ== X-Gm-Message-State: AHYfb5joMo6aYf3exLx0RGu+7U71EJKzYnOGTFr2mGz+S/PRdf5HpT59 EnTdTFiPbdMub/WJLaZrmyzp/JvDlw== X-Received: by 10.223.174.209 with SMTP id y75mr5668371wrc.19.1502294354228; Wed, 09 Aug 2017 08:59:14 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.28.208.3 with HTTP; Wed, 9 Aug 2017 08:59:13 -0700 (PDT) In-Reply-To: <4DFBCE11-913A-4FC9-937D-463B4D49816C@aldan.algebra.com> References: <4DFBCE11-913A-4FC9-937D-463B4D49816C@aldan.algebra.com> From: Alan Somers Date: Wed, 9 Aug 2017 09:59:13 -0600 X-Google-Sender-Auth: NuITEyLAB4lYAZ4VdGC7OeHpXPM Message-ID: Subject: Re: Do I need SAS drives?.. To: "Mikhail T." Cc: freebsd-hardware@freebsd.org, FreeBSD-scsi Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Aug 2017 15:59:16 -0000 On Wed, Aug 9, 2017 at 8:27 AM, Mikhail T. wrote: > My server has 8 "hot-plug" slots, that can accept both SATA and SAS drives. SATA ones tend to be cheaper for the same features (like cache-sizes), what am I getting for the extra money spent on SAS? > > Asking specifically about the protocol differences... It would seem, for example, SATA can not be as easily hot-plugged, but with camcontrol(8) that should not be a problem, right? What else? Thank you! > -- > Sent from mobile device, please, pardon shorthand. Good question. First of all, hot-plugability has more to do with the controller than the protocol. Since you have a SAS controller, you should have no problem hot plugging SATA drives. But SAS drives still have a few advantages: 1) When a SATA drive goes into error recovery, it can lock up the bus indefinitely. This won't matter if your drives are directly connected to a SAS HBA. But if you have an expander with say, 4 SAS lanes going to the HBA, then a flaky SATA drive can reduce the bandwidth available to the good drives. 2) Even with NCQ, the SATA protocol is limited to queueing one or more write commands OR one or more read commands. You can't queue a mixture of reads and writes at the same time. SAS does not have that limitation. In this sense, SAS is theoretically more performant. However, I've never heard of anybody observing a performance problem that can be definitely blamed on this effect. 3) SAS drives have a lot of fancy features that you may not need or care about. For example, they often have features that are useful in multipath setups (dual ports, persistent reservations), their error reporting capabilities are more sophisticated than SMART, their self encrypting command set is more sophisticated, etc etc. 4) The SAS activity LED is the opposite of SATA's. With SATA, the LED is off for an idle drive or blinking for a busy drive. With SAS, it's on for an idle drive or blinking for a busy drive. This makes it easier to see at a glance how many SAS drives you have installed. I think some SATA drives have a way to change the LEDs behavior, though. 5) Desktop class SATA drives can spend an indefinite amount of time in error recovery mode. If your RAID stack doesn't timeout a command, that can cause your array to hang. But SAS drives and RAID class SATA drives will fail any command than spends too much time in error recovery mode. 6) But the most important difference isn't something you'll find on any datasheet or protocol manual. SAS drives are built to a higher standard of quality than SATA drives, and have accordingly lower failure rates. I'm guessing that you don't have an expander (since you only have 8 slots), so item 1 doesn't matter to you. I'll guess that item 3 doesn't matter either, or you wouldn't have asked this question. Item 5 can be dealt with simply by buying the higher end SATA drives. So item 6 is really the most important. If this system needs to have very high uptime and consistent bandwidth, or if it will be difficult to access for maintenance, then you probably want to use SAS drives. If not, then you can save some money by using SATA. Hope that helps. -Alan From owner-freebsd-scsi@freebsd.org Thu Aug 10 13:44:54 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 36C33DD33FE; Thu, 10 Aug 2017 13:44:54 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x231.google.com (mail-wm0-x231.google.com [IPv6:2a00:1450:400c:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BE26A1687; Thu, 10 Aug 2017 13:44:53 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x231.google.com with SMTP id f15so22964428wmg.1; Thu, 10 Aug 2017 06:44:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=D+xEiEBYveIASKYA1Wjt8vh9UegHK2Qh2mqs7H5oWyQ=; b=aH8vBoKkRzSWzQJtkmYOLVy2qdqdRiu22dYB4Nwxh2hy7ta0HGOfrIjIVGRJnZzN2B w0C/aL4O/WcvjVrpCiXuj/lG7So6CmIphzP7ZwiVyIOk/iscW5MN444pb9AVIo7hbSGK Rvw7kmJwX3oZLebjIwR+pAtKJnBJnITxhoOMc2UiZSNB2TkkfdkODgp9bbx/JEqkQqGJ Q/ALFxnbv5xgbIu6VuuBy8dTg3JB/6GqzaNuLgd9wFjCXDJ+IvyzyxkSP8CO5VVKVa0D 9nP0MIJTMQES++0J79GHbOFKsk2HGo83q0FBi8h1n0lOnRM7QICjnI7xiKNOvsiJPaK2 od7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=D+xEiEBYveIASKYA1Wjt8vh9UegHK2Qh2mqs7H5oWyQ=; b=lGizENCQBgU0LsS0tf9W4neoOTz5RneyZXARRqphtuiopHFbHdE5We8/MTlutDO45u A0TSnTV9wFNfiioE6ihX9mFz7J+0B54X8DZYMajrSKXt+4wXDutQ5jEj5nOFBzJSPUK/ f54lUA2U3gXl+/yfPvnS3OaHe4PfTk5H4Ny9kRlZo5b+gySSxT29Rvz12X+EX2ak3gy8 ki8IH9MWkbvsinW0YB2ejVT7Dbe2WlywD89DPZiaG0ZCOfzuQhgnU+bCHz4KyFRjJtuq PAkHHvLV1AqneVDlyJmIsLFprWDapNId5p7v+OjFAMhTs7lrM051G4udRZ1I1tzDH8Wx Mrgg== X-Gm-Message-State: AHYfb5jislAGrR76HNYaCLsGhFN3iiGnDHPt0yDwf8qKR6pELgJThAts krYw67haJyyVxjGmkfE= X-Received: by 10.28.9.9 with SMTP id 9mr7457520wmj.93.1502372692020; Thu, 10 Aug 2017 06:44:52 -0700 (PDT) Received: from ben.home (LFbn-1-6951-179.w90-116.abo.wanadoo.fr. [90.116.132.179]) by smtp.gmail.com with ESMTPSA id 60sm4388472wrd.20.2017.08.10.06.44.51 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 10 Aug 2017 06:44:51 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: Do I need SAS drives?.. From: Ben RUBSON In-Reply-To: Date: Thu, 10 Aug 2017 15:44:50 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4DFBCE11-913A-4FC9-937D-463B4D49816C@aldan.algebra.com> To: FreeBSD-scsi , freebsd-hardware@freebsd.org X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2017 13:44:54 -0000 > On 09 Aug 2017, at 17:59, Alan Somers wrote: >=20 > 3) SAS drives have a lot of fancy features that you may not need or > care about. For example, (...) their error > reporting capabilities are more sophisticated than SMART Really interesting answer Alan, thank you very much ! Slightly off-topic but I take this opportunity, how do you check SAS drives health ? I personally cron a background long test every 2 weeks (using = smartmontools). I did not experience SAS drive error yet, so not sure how this behaves. Does the drive reports to FreeBSD when its read or write error rate = cross a threshold (so that we can replace it before it fails) ? Or perhaps smartd will do ? As an example below a SAS error counter log returned by smartctl : Errors Corrected by Total Correction Gigabytes = Total ECC rereads/ errors algorithm processed = uncorrected fast | delayed rewrites corrected invocations [10^9 bytes] = errors read: 0 49 0 49 233662 73743.588 = 0 write: 0 3 0 3 83996 9118.895 = 0 verify: 0 0 0 0 28712 0.000 = 0 Thank you ! Ben From owner-freebsd-scsi@freebsd.org Thu Aug 10 14:01:49 2017 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 03535DD3AC9; Thu, 10 Aug 2017 14:01:49 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-wm0-x236.google.com (mail-wm0-x236.google.com [IPv6:2a00:1450:400c:c09::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8D4B6215A; Thu, 10 Aug 2017 14:01:48 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-wm0-x236.google.com with SMTP id f15so23400468wmg.1; Thu, 10 Aug 2017 07:01:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=Z7bKJx5yIn2Q8yVVDmcOhwU4mPEDxWqePFVlFI0CfL0=; b=fpv7yE6o23ctmSUYEsDpDcY3BcbpIQWgXQfyU20v4EuKRxgb9PbWkC7tKDVi/aghdj m3X8hUbIXqg5yOPZdaIL9S6ERpZRjc6iG8L9WLynZI9soqYrPnRubGjQg+IqbFWSb4jC RQJQtDe1/SL72IPErfeLcX7sUYFx2QeiZaUwtsIct1NNH+I0OhoOAR6vfiBjQsLwQn3Z Xa2+zDxY1DbkFMuLCYTyelm0VE1/atPJkM6AYwcP6Px8yyTvDFoQFOjtRKH9AoLMXp9o AYcSBI9uWS+sy371tZ7SKrfNSrXLWupbIvh10T9CGT7qocQ+GDlPUf9lWkPOGJcXGyHN 3HjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=Z7bKJx5yIn2Q8yVVDmcOhwU4mPEDxWqePFVlFI0CfL0=; b=gubTab91Sa7Pzv7gFWqeCouHvXc9A4HXUitTaf4tBfGehm3CC+TCndBB2P6LIoF3jZ UuaFVHA476Nv855kpuJjL7NdqGqnaMkuAqTKruAyXz+wYArtafF4vn50mEoOEs1o4Vr7 kc4MaM62MhjnEEbJN1IQFVEf3SlTHJTv7WW1UL2wEwhbyRKKx8ilftHJf6bgWhlxOdK6 5do8vfqYL6BvHWfAEOUNgAAfzvhIU7F9lDZ1vKMwhn87SZ8siZf0xub97bfsfaaYeL6t ZCaV0EE9qHMvOr3O243cDQzU4L5JNuzl38tExhQ6U43juM+pEwhSKOWkgqcASU3pfnHG 6n+A== X-Gm-Message-State: AHYfb5jDb5XFCT9C6tTQrDQKxbplH0vjxOGdn33sOwgzWlGyc9jlznHt TZXQKqF0rVwlwa+VS2iua1CtN33YxQ== X-Received: by 10.28.170.18 with SMTP id t18mr7603839wme.6.1502373706889; Thu, 10 Aug 2017 07:01:46 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.28.208.3 with HTTP; Thu, 10 Aug 2017 07:01:46 -0700 (PDT) In-Reply-To: References: <4DFBCE11-913A-4FC9-937D-463B4D49816C@aldan.algebra.com> From: Alan Somers Date: Thu, 10 Aug 2017 08:01:46 -0600 X-Google-Sender-Auth: V3Ta4p3yRSRdEMKZCbwrv6N80-I Message-ID: Subject: Re: Do I need SAS drives?.. To: Ben RUBSON Cc: FreeBSD-scsi , freebsd-hardware@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2017 14:01:49 -0000 On Thu, Aug 10, 2017 at 7:44 AM, Ben RUBSON wrote: >> On 09 Aug 2017, at 17:59, Alan Somers wrote: >> >> 3) SAS drives have a lot of fancy features that you may not need or >> care about. For example, (...) their error >> reporting capabilities are more sophisticated than SMART > > Really interesting answer Alan, thank you very much ! > Slightly off-topic but I take this opportunity, > how do you check SAS drives health ? > I personally cron a background long test every 2 weeks (using smartmontools). > I did not experience SAS drive error yet, so not sure how this behaves. > Does the drive reports to FreeBSD when its read or write error rate cross > a threshold (so that we can replace it before it fails) ? > Or perhaps smartd will do ? > > As an example below a SAS error counter log returned by smartctl : > Errors Corrected by Total Correction Gigabytes Total > ECC rereads/ errors algorithm processed uncorrected > fast | delayed rewrites corrected invocations [10^9 bytes] errors > read: 0 49 0 49 233662 73743.588 0 > write: 0 3 0 3 83996 9118.895 0 > verify: 0 0 0 0 28712 0.000 0 > > Thank you ! > > Ben smartmontools is probably the best way to read SAS error logs. Interpreting them can be hard, though. The Backblaze blog is probably the best place to get current advice. But the easiest thing to do is certainly to wait until something fails hard. With ZFS, you can have up to 3 drives' worth of redundancy, and hotspares too. -Alan