From owner-freebsd-fs@freebsd.org Sat Aug 5 18:20:37 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5D9E3DC2686 for ; Sat, 5 Aug 2017 18:20:37 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x236.google.com (mail-wm0-x236.google.com [IPv6:2a00:1450:400c:c09::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E215B82A02 for ; Sat, 5 Aug 2017 18:20:36 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x236.google.com with SMTP id t201so41079422wmt.1 for ; Sat, 05 Aug 2017 11:20:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=TL19KXzkSIWLXTJNP07eHGl8EWGYo7oU0xKTKaAjlRM=; b=lnC6g8qwyNiD0l+jm48QCTdG7Bi925E9TVoplwN6aesnycGgpSEDwkdKtSZlhJJR6l Cqig2KjR81jvjSt0iMbVxwhf0so5O1H/dBYjYmGhvCeH7mhLsoT1OZv4QHImAg6KrzRb CmtcTDxQP2AIlS9Z45ve3wxar6WWHwpK8X9aMYvNFIyiEEUijrSzOKt0Tbc+vuobnEEb kYXsZFNUNGFn29ueC+A52Rr0WThYubU66WQXx4PmYws6JsLiXP4LU0hbuaNUUUtuaz30 HIUKa/CRFAv2XBGCoPjzUPSvaxGFcW/mXVhMHKTO5uzk7UR6UIrimKTmvtb0S/6aFA+Y owDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=TL19KXzkSIWLXTJNP07eHGl8EWGYo7oU0xKTKaAjlRM=; b=PfJ3NWG2PuekZ2cou/BBMVd/8hcWfQpxhaI2KaA+l7b9muVBwFvffpk0xNLK0qEnJE JU9aES78l5tC52pgPEob3Ox7M9G4zFiyUzhvS4KDmatCL4MacgP2bNZFzpLyEoDb8z0F q1BeZ2YhslFk/UXttWWeKjEVASlz0/95Uh7s0QruImRRBmyALKMrOP4sMN59Ct0DXwE6 rQ1gu3b+VHyt1EbWgsj47aqN925roVNv6vCzRDUHBe2I1eoRCmaD4UTZKl37KKMMNCsB lfJH5MJ30R1iJwO9gOXbvtQH250EQK/8hupeyheLhWjNFkK+sCZOeHFd6oimGpek7UMg 3sag== X-Gm-Message-State: AHYfb5hF3m4pE0tV+ONP3MpXavp0g7S51G4SRiEVBidrhFVQ2d4abH3C lNMRVap5z6Bqe2/Le90= X-Received: by 10.28.187.86 with SMTP id l83mr3274206wmf.165.1501957234448; Sat, 05 Aug 2017 11:20:34 -0700 (PDT) Received: from ben.home (LFbn-1-6951-179.w90-116.abo.wanadoo.fr. [90.116.132.179]) by smtp.gmail.com with ESMTPSA id g17sm6214534wrc.28.2017.08.05.11.20.33 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 05 Aug 2017 11:20:33 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: a strange and terrible saga of the cursed iSCSI ZFS SAN From: Ben RUBSON In-Reply-To: <8ff8d88c-ec9a-5586-b131-c9bc1fe17d95@norma.perm.ru> Date: Sat, 5 Aug 2017 20:20:32 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <7F068B61-6167-4C58-A925-7956B1D023D6@gmail.com> References: <1bd10b1e-0583-6f44-297e-3147f6daddc5@norma.perm.ru> <8D11D326-6611-4798-AE3E-9411200A9556@gmail.com> <25a4d73c-698e-38bd-7957-f2895a01e351@norma.perm.ru> <8ff8d88c-ec9a-5586-b131-c9bc1fe17d95@norma.perm.ru> To: Freebsd fs X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Aug 2017 18:20:37 -0000 > On 05 Aug 2017, at 19:59, Eugene M. Zheganin = wrote: >=20 > Hi, >=20 > On 05.08.2017 22:50, Eugene M. Zheganin wrote: >> Hi, >>=20 >> On 05.08.2017 22:22, Ben RUBSON wrote: >>>> On 05 Aug 2017, at 19:08, Eugene M. Zheganin = wrote: >>>>=20 >>>> (...) >>>>=20 >>>> 1) autumn 2016, SAN is set up, supermicro server, external JBOD, = sandisk ssds, several redundant pools, FreeBSD 11.x (probably release, = don't really remember - see below). >>> Hi Eugene, >>>=20 >>> Which SAS adapter ? >>> SAS 2008 ? >> Exactly. What's wrong with it ? I'm running latest (or at least not = that old) firmware on em. >>=20 > No, wait, my bad, it's LSISAS3008. mpr(4) one. Nothing wrong but things such as verifying the firmware version. Certainly you use the same (IT ?) version on all your 6 HBA. Not sure about the 3008, but we had a discussion on this list some = months ago about 2008 firmwares, some were said to be bad. You could also enable some HBA debugging, we never know : sysctl dev.mpr.0.debug_level=3D0x1B Ben