From owner-freebsd-scsi@freebsd.org Fri Jul 29 13:41:03 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 16805BA6395 for ; Fri, 29 Jul 2016 13:41:03 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 017551E69 for ; Fri, 29 Jul 2016 13:41:03 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: by mailman.ysv.freebsd.org (Postfix) id F1916BA6390; Fri, 29 Jul 2016 13:41:02 +0000 (UTC) Delivered-To: scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F1128BA638F; Fri, 29 Jul 2016 13:41:02 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 6F7091E65; Fri, 29 Jul 2016 13:41:01 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from [IPv6:2a02:2698:22:ed44:708f:11f3:83d6:57da] (dynamic-2a02-2698-22-0-0.perm.ertelecom.ru [IPv6:2a02:2698:22:ed44:708f:11f3:83d6:57da] (may be forged)) (authenticated bits=0) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPSA id u6TDet8s040803 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Fri, 29 Jul 2016 18:40:57 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1469799657; bh=+sgo2uRLnwb7/dkRFqVx9WUeuG10af0r7Y0L9D6VYZ0=; h=To:Cc:From:Subject:Date; b=v0+kA4/hvzG+Bmp3IdEjmahhrZ2ECBvHfb6JlWmW37gNcVq96d6PR/tI6PpBlebs4 H1aDFndA7Z9YvB6iu1uw+R7S1VXE9ksgXl4UuZAQfrc1uJ0L4FP8AZlfvcoMk6eNzq T4g14TDkpUmbMKZJHBLZgPoxq31iGSIjPPMnBlLw= To: scsi@freebsd.org Cc: stable@freebsd.org From: "Eugene M. Zheganin" Subject: mrsas(4) Message-ID: <52b7278e-2a5e-5ce7-5937-c512ef3982bc@norma.perm.ru> Date: Fri, 29 Jul 2016 18:41:01 +0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jul 2016 13:41:03 -0000 Hi. I'm experiencing some weird troubles with an LSI MegaRAID SAS9341-4i controller: sometimes, all of a sudden, it reports that disk was reattached. I'm using zfs, and a redundant pool, so, besides the fact that it's bad by itself, everything should continue to work, but instead my FreeBSD starts to livelock, and all disk/zfs commands stop working - they just hang (however, I'm able to log in, launch gstat and other utilities, and most of the services seem to be unaffected, however, iSCSI target starts to behave weirdly too). I wrote a letter to the LSI support, but I guess they will first ask me to try the latest driver, which for some reason isn't commited in the tree yet. So, two questions: - why the latest driver (6.710.11.00) isn't in the main tree, it's from december 2015, so at least it seems like it should have been made it to the 11.x. In 11.x I still see the 06.709.07.00-fbsd ? may be I don't unerstand something. - has anyone seen anything similar on mrsas ? I'm kinda open to the ideas. To be honest, I suspect the controller and the firmware, because with factory-default firmware version I had seen all sorts of madness, including disk detaching after requesting SMART data with smartmontools. This was reproducible and solved with the firmware update, but considering the amount of fixes that each new version does contain, I suppose there's many of the bugs that left unfixed. Thanks. Eugene. From owner-freebsd-scsi@freebsd.org Fri Jul 29 13:56:49 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 08288BA68E8 for ; Fri, 29 Jul 2016 13:56:49 +0000 (UTC) (envelope-from ultima1252@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id DC9FC1962 for ; Fri, 29 Jul 2016 13:56:48 +0000 (UTC) (envelope-from ultima1252@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id D89BDBA68E4; Fri, 29 Jul 2016 13:56:48 +0000 (UTC) Delivered-To: scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D8305BA68E3; Fri, 29 Jul 2016 13:56:48 +0000 (UTC) (envelope-from ultima1252@gmail.com) Received: from mail-yw0-x232.google.com (mail-yw0-x232.google.com [IPv6:2607:f8b0:4002:c05::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 97FC91961; Fri, 29 Jul 2016 13:56:48 +0000 (UTC) (envelope-from ultima1252@gmail.com) Received: by mail-yw0-x232.google.com with SMTP id r9so115733008ywg.0; Fri, 29 Jul 2016 06:56:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=CepXkmaBrQa7Z5UMXBac8n4sh5jfo8oZeuz7ZSBJ7/8=; b=Q+SuEdv3BCjLZ9b7nroSv90h75b7WFSA1NR8qPXiNPkgIA1ROX4GCDgRajiVAwNdKo Lk0c+cLG9s5O3JAbWjaeY+38g2vtemKLHWlw1ZYI3a7fkqMo7txrFTvu6J/B0tE3cVs4 GUMvkIYjNc7aTn1VxvIA+j112QABiovwia64d231bMNlrthlCunFl2znpKT5CjyAu3NU iTx+EPOYNEYeD/FymIq1XhDgRRVvMBEm6Ro8hLX2Ha4vArJx4WctIwrPtxVlXw4JqY1y DaldR3adB2PBcKKN3UFg46DcYRgAVmbVvXGoKNpquDVHz+UaeiHS/yUACbXaIdsH2iKL +e9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=CepXkmaBrQa7Z5UMXBac8n4sh5jfo8oZeuz7ZSBJ7/8=; b=f5M9IW49D7m+vSmGVUQJ7f7iRTzpqg/LiaPYm2hkuW0PDExivlDneERWfJaQM5OX7w 5xYPQ4n2Dtrj9LTMEqv/CNVA1zSvkuiq4WgZI2GXewGYZgdF/gn+onmF1RY5Co/rfqXl 2RS5Y9MaNy38TmErGkVm7HzCBCKTmI/rs6Uw0Xf9tNCHIoA1I6TQhI3afccglaiIIBUI NirjaF2tz1PIeeDXswQQ4r8V1DragkaXDJVo5Kx/wBwYbsqRAXpaDpwO/mUz4tq+Q+nx bZEopPiRttaYNCLGq+iZvPX5V0RrX2GB9iUfWRElH8jEPtbl4IPq0hRjQmnSZa6flfH5 5zGA== X-Gm-Message-State: AEkoouskHl9Ya8xszYN2auEgPy3VAwMKPQfqvuCYp1OitbMPJJtyuk8D1aGUqsolicFewlW66kMXNY4cIzVmXg== X-Received: by 10.129.164.9 with SMTP id b9mr34916780ywh.145.1469800607801; Fri, 29 Jul 2016 06:56:47 -0700 (PDT) MIME-Version: 1.0 Received: by 10.129.51.150 with HTTP; Fri, 29 Jul 2016 06:56:47 -0700 (PDT) In-Reply-To: <52b7278e-2a5e-5ce7-5937-c512ef3982bc@norma.perm.ru> References: <52b7278e-2a5e-5ce7-5937-c512ef3982bc@norma.perm.ru> From: Ultima Date: Fri, 29 Jul 2016 09:56:47 -0400 Message-ID: Subject: Re: mrsas(4) To: "Eugene M. Zheganin" Cc: scsi@freebsd.org, stable@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.22 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jul 2016 13:56:49 -0000 Hello, I have experienced this twice within the last month. It was not with this same controller, however it was not the controllers fault. Random disks will drop and reattach. For me, this was power supply problem and both times replacement fixed the issue. If you haven't checked into the power supply. I would suggest testing it. Especially If it was previously working fine. On Fri, Jul 29, 2016 at 9:41 AM, Eugene M. Zheganin wrote: > Hi. > > I'm experiencing some weird troubles with an LSI MegaRAID SAS9341-4i > controller: sometimes, all of a sudden, it reports that disk was > reattached. I'm using zfs, and a redundant pool, so, besides the fact that > it's bad by itself, everything should continue to work, but instead my > FreeBSD starts to livelock, and all disk/zfs commands stop working - they > just hang (however, I'm able to log in, launch gstat and other utilities, > and most of the services seem to be unaffected, however, iSCSI target > starts to behave weirdly too). I wrote a letter to the LSI support, but I > guess they will first ask me to try the latest driver, which for some > reason isn't commited in the tree yet. So, two questions: > > - why the latest driver (6.710.11.00) isn't in the main tree, it's from > december 2015, so at least it seems like it should have been made it to the > 11.x. In 11.x I still see the 06.709.07.00-fbsd ? may be I don't unerstand > something. > > - has anyone seen anything similar on mrsas ? I'm kinda open to the ideas. > To be honest, I suspect the controller and the firmware, because with > factory-default firmware version I had seen all sorts of madness, including > disk detaching after requesting SMART data with smartmontools. This was > reproducible and solved with the firmware update, but considering the > amount of fixes that each new version does contain, I suppose there's many > of the bugs that left unfixed. > > > Thanks. > > Eugene. > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-scsi@freebsd.org Fri Jul 29 14:25:19 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4B256BA7028 for ; Fri, 29 Jul 2016 14:25:19 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 3797D1AF7 for ; Fri, 29 Jul 2016 14:25:19 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: by mailman.ysv.freebsd.org (Postfix) id 33291BA7025; Fri, 29 Jul 2016 14:25:19 +0000 (UTC) Delivered-To: scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 32B1ABA7023; Fri, 29 Jul 2016 14:25:19 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EB0211AF4; Fri, 29 Jul 2016 14:25:18 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id A634628431; Fri, 29 Jul 2016 16:25:08 +0200 (CEST) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id E0E4628412; Fri, 29 Jul 2016 16:25:06 +0200 (CEST) Message-ID: <579B6742.9050200@quip.cz> Date: Fri, 29 Jul 2016 16:25:06 +0200 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:35.0) Gecko/20100101 Firefox/35.0 SeaMonkey/2.32 MIME-Version: 1.0 To: "Eugene M. Zheganin" , scsi@freebsd.org CC: stable@freebsd.org Subject: Re: mrsas(4) References: <52b7278e-2a5e-5ce7-5937-c512ef3982bc@norma.perm.ru> In-Reply-To: <52b7278e-2a5e-5ce7-5937-c512ef3982bc@norma.perm.ru> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jul 2016 14:25:19 -0000 Eugene M. Zheganin wrote on 07/29/2016 15:41: > Hi. > > I'm experiencing some weird troubles with an LSI MegaRAID SAS9341-4i > controller: sometimes, all of a sudden, it reports that disk was > reattached. I'm using zfs, and a redundant pool, so, besides the fact > that it's bad by itself, everything should continue to work, but instead > my FreeBSD starts to livelock, and all disk/zfs commands stop working - > they just hang (however, I'm able to log in, launch gstat and other > utilities, and most of the services seem to be unaffected, however, > iSCSI target starts to behave weirdly too). I wrote a letter to the LSI > support, but I guess they will first ask me to try the latest driver, > which for some reason isn't commited in the tree yet. So, two questions: > > - why the latest driver (6.710.11.00) isn't in the main tree, it's from > december 2015, so at least it seems like it should have been made it to > the 11.x. In 11.x I still see the 06.709.07.00-fbsd ? may be I don't > unerstand something. > > - has anyone seen anything similar on mrsas ? I'm kinda open to the > ideas. To be honest, I suspect the controller and the firmware, because > with factory-default firmware version I had seen all sorts of madness, > including disk detaching after requesting SMART data with smartmontools. > This was reproducible and solved with the firmware update, but > considering the amount of fixes that each new version does contain, I > suppose there's many of the bugs that left unfixed. mrsas is very strange so I moved to mfi driver for Dell PowerEdge T130 with PERC H330. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=200993 Miroslav Lachman From owner-freebsd-scsi@freebsd.org Fri Jul 29 15:14:47 2016 Return-Path: Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8B748BA7DB8 for ; Fri, 29 Jul 2016 15:14:47 +0000 (UTC) (envelope-from mike@sentex.net) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 760591840 for ; Fri, 29 Jul 2016 15:14:47 +0000 (UTC) (envelope-from mike@sentex.net) Received: by mailman.ysv.freebsd.org (Postfix) id 71E5FBA7DB6; Fri, 29 Jul 2016 15:14:47 +0000 (UTC) Delivered-To: scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 716ABBA7DB5; Fri, 29 Jul 2016 15:14:47 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [IPv6:2607:f3e0:80:80::2]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "smarthost.sentex.ca", Issuer "smarthost.sentex.ca" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id CC956183F; Fri, 29 Jul 2016 15:14:46 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (lava.sentex.ca [IPv6:2607:f3e0:0:5::11]) by smarthost2.sentex.ca (8.15.2/8.15.2) with ESMTPS id u6TFEjks008005 (version=TLSv1 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 29 Jul 2016 11:14:45 -0400 (EDT) (envelope-from mike@sentex.net) Received: from [IPv6:2607:f3e0:0:4:5c30:ed1b:e203:c55c] ([IPv6:2607:f3e0:0:4:5c30:ed1b:e203:c55c]) by lava.sentex.ca (8.14.9/8.14.9) with ESMTP id u6TFEhPJ063598; Fri, 29 Jul 2016 11:14:44 -0400 (EDT) (envelope-from mike@sentex.net) Subject: Re: mrsas(4) To: "Eugene M. Zheganin" , scsi@freebsd.org References: <52b7278e-2a5e-5ce7-5937-c512ef3982bc@norma.perm.ru> Cc: stable@freebsd.org From: Mike Tancsa Organization: Sentex Communications Message-ID: <08745358-b7d6-ea3f-e97b-58ef3c04ef41@sentex.net> Date: Fri, 29 Jul 2016 11:14:37 -0400 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: <52b7278e-2a5e-5ce7-5937-c512ef3982bc@norma.perm.ru> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.78 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Jul 2016 15:14:47 -0000 On 7/29/2016 9:41 AM, Eugene M. Zheganin wrote: > Hi. > > I'm experiencing some weird troubles with an LSI MegaRAID SAS9341-4i > - has anyone seen anything similar on mrsas ? I'm kinda open to the > ideas. To be honest, I suspect the controller and the firmware, because A somewhat related datapoint as I dont have that exact controller, but I am using the driver with a similar class card with very stable results on RELENG_10. However, smartctl is broken when trying to read from them. mrsas0@pci0:1:0:0: class=0x010400 card=0x080915d9 chip=0x005d1000 rev=0x02 hdr=0x00 vendor = 'LSI Logic / Symbios Logic' device = 'MegaRAID SAS-3 3108 [Invader]' It has a pair of SSD drives that are used in a zfs array for logging and caching. 0-zoo-# camcontrol devlist at scbus0 target 0 lun 0 (pass0,da0) at scbus0 target 1 lun 0 (pass1,da1) at scbus2 target 0 lun 0 (ada0,pass2) at scbus3 target 0 lun 0 (ada1,pass3) at scbus4 target 0 lun 0 (ada2,pass4) at scbus6 target 0 lun 0 (pass5,ses0) at scbus7 target 0 lun 0 (ada3,pass6) at scbus8 target 0 lun 0 (ada4,pass7) at scbus9 target 0 lun 0 (ada5,pass8) at scbus10 target 0 lun 0 (ada6,pass9) at scbus13 target 0 lun 0 (pass10,ses1) 0-zoo-# I am not able to get smartctl to work with them unfortunately. MegaCli -AdpAllInfo -aAll Adapter #0 ============================================================================== Versions ================ Product Name : LSI 3108 MegaRAID ROMB Serial No : FW Package Build: 24.1.1-0001 Mfg. Data ================ Mfg. Date : 00/00/00 Rework Date : 00/00/00 Revision No : Battery FRU : N/A Image Versions in Flash: ================ BIOS Version : 6.13.00_4.14.05.00_0x06010600 Ctrl-R Version : 5.01-0004 FW Version : 4.210.10-2910 NVDATA Version : 3.1310.00-0054 Boot Block Version : 3.00.00.00-0009 -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/