From owner-freebsd-fs@freebsd.org Fri Aug 31 01:35:29 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1EF97EFEA87 for ; Fri, 31 Aug 2018 01:35:29 +0000 (UTC) (envelope-from grant@grantgray.id.au) Received: from mail.agc.net.au (mail.evps.com.au [27.121.114.102]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 832917B921 for ; Fri, 31 Aug 2018 01:35:27 +0000 (UTC) (envelope-from grant@grantgray.id.au) Received: from localhost (localhost [127.0.0.1]) by mail.agc.net.au (Postfix) with ESMTP id C3A034719F72; Fri, 31 Aug 2018 11:35:15 +1000 (AEST) Received: from mail.agc.net.au ([127.0.0.1]) by localhost (mail.agc.net.au [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id vK0QaaZ7nOR1; Fri, 31 Aug 2018 11:35:13 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by mail.agc.net.au (Postfix) with ESMTP id 2508A4719F76; Fri, 31 Aug 2018 11:35:13 +1000 (AEST) X-Virus-Scanned: amavisd-new at mail.agc.net.au Received: from mail.agc.net.au ([127.0.0.1]) by localhost (mail.agc.net.au [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id Jz1k2IIdZmtw; Fri, 31 Aug 2018 11:35:13 +1000 (AEST) Received: from mail.agc.net.au (mail.agc.net.au [27.121.114.102]) by mail.agc.net.au (Postfix) with ESMTP id E0AE84719F72; Fri, 31 Aug 2018 11:35:12 +1000 (AEST) Date: Fri, 31 Aug 2018 11:35:12 +1000 (AEST) From: Grant Gray To: "M. Casper Lewis" Cc: freebsd-fs Message-ID: <1121631966.257797.1535679312447.JavaMail.zimbra@grantgray.id.au> In-Reply-To: <20180831011607.GX1473@genomecenter.ucdavis.edu> References: <20180831003436.GW1473@genomecenter.ucdavis.edu> <707525919.257415.1535675999891.JavaMail.zimbra@grantgray.id.au> <20180831011607.GX1473@genomecenter.ucdavis.edu> Subject: Re: Failing ZFS log devices/panic MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [27.121.114.102] X-Mailer: Zimbra 8.8.8_GA_1728 (ZimbraWebClient - FF62 (Linux)/8.8.8_GA_1703) Thread-Topic: Failing ZFS log devices/panic Thread-Index: Y9NH0oaWIVPIVIcEYmh0PDvqJFoPqg== X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 31 Aug 2018 01:35:29 -0000 I'm going to defer to people with more experience with this hardware combination than myself, but I will say I've had compatibility issues with SATA SSD's on LSI SAS controllers in the past. In my case this manifested as the SSD's disappearing off the bus and not returning. I've currently got an issue between some HGST SATA disks and a SAS3008 HBA where mixing SAS and SATA disks on the same port results in intermittent I/O errors on the HGST SATA disks, but not other SAS devices. If feasible, it may be useful to move the SSD's onto a plain SATA controller as a diagnostic step. ----- On 31 Aug, 2018, at 11:16 AM, M. Casper Lewis mclewis@genomecenter.ucdavis.edu wrote: > On Fri, Aug 31, 2018 at 10:39:59AM +1000, Grant Gray wrote: >> SAS? SATA? Make/model? HBA? > > SATA. Currently Intel SSDSC2BB24 but like I said, we've tried several. > > HBA is a Supermicro AOC-S3008L-L8I (reports itself as a LSI3008-IT). > > Firmware has been updated to latest on all controllers. > > We've purchased an LSI 9300-8i and scheduled downtime to install it, to > see if we can rule out a bad controller. > >> Are you mixing SAS and SATA on the same bus? > > No, the pool is on separate HBAs. Only the cache and log devices are > attached to this HBA. > >> Pool configuration? > > 8 11-disk raidz3s. We've tried various numbers and configurations of > log devices, anywhere from 4 to 8 devices, both mirrored and unmirrored. > > -- > M. Casper Lewis | mclewis@ucdavis.edu > Systems Administrator | Voice: (530) 754-7978 > Genome Center | > University of California, Davis |