From owner-freebsd-scsi@FreeBSD.ORG Mon Dec 8 20:44:21 2014 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4C0ACEC1; Mon, 8 Dec 2014 20:44:21 +0000 (UTC) Received: from mail-wg0-x229.google.com (mail-wg0-x229.google.com [IPv6:2a00:1450:400c:c00::229]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CE94ADA2; Mon, 8 Dec 2014 20:44:20 +0000 (UTC) Received: by mail-wg0-f41.google.com with SMTP id y19so7165695wgg.28 for ; Mon, 08 Dec 2014 12:44:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=TimU8P/2Nx68oJeOPhPdBTH5rIRopXZuqUcz6X7qGOg=; b=k9CTa15f0my4Ny6SDjTyLh9Uo6J4in3/HVEwIdpzncCmHh+NTCoxsCWLcCYQKLOFeB nsgwLUquIOPNV3xL97ZghsvBxeMd6KExkGIPx5WhlXi8IHcLzh/aRZut37uLEKFIcvdd sXsEBKx7zXLl/mun5WGrpjKfagNsq46ZBPQGIfmqPYaBvlUmnZi5omkw1IrNkWsQntM0 lO0JzRXEnQbgWeMmubEI4ykmbseXd0e7ozMoaMezzFzZJqpzf+ghiWKaQR9DqeATnznR bMyv0frxA7WyNaUSYJ71F+1RquHdRlyPx9UXGL6xTlVG9TUZMe7BAxpHWrvexqasU0SO cyfw== MIME-Version: 1.0 X-Received: by 10.194.23.202 with SMTP id o10mr48606844wjf.73.1418071459202; Mon, 08 Dec 2014 12:44:19 -0800 (PST) Received: by 10.217.152.195 with HTTP; Mon, 8 Dec 2014 12:44:19 -0800 (PST) In-Reply-To: References: <54822835.3080800@crystal.harvard.edu> Date: Mon, 8 Dec 2014 13:44:19 -0700 Message-ID: Subject: Re: LSI SAS 3008 card - 35 out of 36 disks detected From: Jason Wolfe To: Alan Somers Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD-scsi X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Dec 2014 20:44:21 -0000 On Mon, Dec 8, 2014 at 11:20 AM, Alan Somers wrote: > On Mon, Dec 8, 2014 at 8:58 AM, Justin O'Conor > wrote: >> Hi All, >> Thanks, this is encouraging. smp_discover ses0|1 see 36 sata disks. This is from the 10.1 install. >> > > There are certainly some inconsistencies in the smp_discover > responses. For example, the SEP on ses0 (phy identifier: 36) has > "connector type: SAS virtual connector" and "connector element index: > 24" But the SEP on ses1 (phy identifier: 28) has "connector type: No > information" and "connector element index: 0". Also note that "phy > identifier: 12" on ses1 has "connector element index: 0". That would > be the first slot on the rear expander, if the slots and phys are > numbered the same way. My best guess is that phy 12 and phy 28 mapped > to the same map_idx in mpr_mapping.c:1168. So the information for the > SEP overwrote the information for the first disk slot. If my guess is > true, then recabling your chassis as you suggested wouldn't help. > However, you might try the attached but untested patch. It will > prevent the SEP from being added to the mapping table while printing a > useful error message. If I'm correct, then the patch will let you use > all 36 disk slots, but you won't have ses1 anymore. > > In the meantime, I'll try to reproduce your problem. I have all the > required equipment in my lab. > > -Alan > I believe this is the same issue we ran into a few years ago on the LSI2008, where the ses0 device would map over the boot disk. It's long and spans over multiple months, so just the relevant bits: Initial report: https://lists.freebsd.org/pipermail/freebsd-scsi/2012-February/005243.html The issue seems to be a shortcoming in the detection method where it has no problem assigning the ses over an already mapped disk, LSI's initial response was to use the LSI config utility and map the drives manually: https://lists.freebsd.org/pipermail/freebsd-scsi/2012-February/005243.html https://lists.freebsd.org/pipermail/freebsd-scsi/2012-February/005267.html https://lists.freebsd.org/pipermail/freebsd-scsi/2012-March/005343.html This was not an option for us as we had 2000 of these devices in the field, and entering the LSI BIOS would be a large undertaking. In the end after an internal dialogue with LSI guys, Kashyap was kind enough to write a one off for us, that never made it upstream. It simply assigns the ses device to max target + 1 when a conflict is found. The core issue seems to be with the way LSI detects and assigns devices on FreeBSD, so this is by no means 'proper', but it's sound enough so resolve the issue for us on the LSI2008. In case it's interesting to anyone: http://nitrology.com/mps_fix-fbsd10stable.diff Jason