Date: Wed, 13 Feb 2019 20:02:15 +0200 (EET) From: Krasimir Velkov <moerke@abv.bg> To: freebsd-scsi@freebsd.org Subject: SATA affiliation missing from SATA phys Message-ID: <503097732.142480.1550080935357@nm42.abv.bg>
next in thread | raw e-mail | index | archive | help
Hey guys, I have a couple of freebsd 12.0 controllers (each has an LSI 9300-8e HBA) and one JBOD box with a single LSI 3x40 SAS EXPANDER. The JBOD box is connected to both controllers and according to all information I found on the Internet, SATA disks CANNOT be accessed by two SAS Initiators at the same time and have SATA affiliations. The problem is that the aforementioned affiliations are not being honored and both initiators end up accessing the SATA disks which leads to an affiliation conflict and one of the initiators freezing. Here is an example for a SATA disk: smp_rep_phy_sata --phy=0 /dev/ses4 Report phy SATA response: expander change count: 83 phy identifier: 0 STP I_T nexus loss occurred: 0 affiliations supported: 0 affiliation valid: 0 STP SAS address: 0x500304801eeba8c0 register device to host FIS: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 affiliated STP initiator SAS address: 0x0 STP I_T nexus loss SAS address: 0x0 affiliation context: 0 current affiliation contexts: 0 maximum affiliation contexts: 0 there is no affiliated STP initiator SAS address and sending operations to the phy works for both initiator hosts. I found absolutely no information on how to enable/disable SATA affiliation for the SAS EXPANDER. Tried issuing commands with both camcontrol and smp_phy_control without any success: camcontrol smppc /dev/ses4 -vvv -p 0 -o sataportsel camcontrol: error sending command (pass6:mpr5:0:200:0): PHY CONTROL. 40 91 00 00 00 00 00 00 00 00 07 00 00 ... (pass6:mpr5:0:200:0): CAM status: SMP Status Error (pass6:mpr5:0:200:0): SMP status: SMP Function Failed (0x2) I have absolutely no idea how to proceed. Has anybody bumped into something similar? Thanks, Krasimir Velkov From owner-freebsd-scsi@freebsd.org Fri Feb 15 17:24:24 2019 Return-Path: <owner-freebsd-scsi@freebsd.org> Delivered-To: freebsd-scsi@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 87B1314E1B78 for <freebsd-scsi@mailman.ysv.freebsd.org>; Fri, 15 Feb 2019 17:24:24 +0000 (UTC) (envelope-from peter.blok@bsd4all.org) Received: from smtpq2.mnd.mail.iss.as9143.net (smtpq2.mnd.mail.iss.as9143.net [212.54.34.165]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0B7B77300E; Fri, 15 Feb 2019 17:24:22 +0000 (UTC) (envelope-from peter.blok@bsd4all.org) Received: from [212.54.34.120] (helo=smtp12.mnd.mail.iss.as9143.net) by smtpq2.mnd.mail.iss.as9143.net with esmtp (Exim 4.86_2) (envelope-from <peter.blok@bsd4all.org>) id 1guhDh-0006IK-Cu; Fri, 15 Feb 2019 18:24:13 +0100 Received: from 5ed231fb.cm-7-3a.dynamic.ziggo.nl ([94.210.49.251] helo=wan0.bsd4all.org) by smtp12.mnd.mail.iss.as9143.net with esmtp (Exim 4.86_2) (envelope-from <peter.blok@bsd4all.org>) id 1guhDh-0007Cg-AG; Fri, 15 Feb 2019 18:24:13 +0100 Received: from newnas (localhost [127.0.0.1]) by wan0.bsd4all.org (Postfix) with ESMTP id 0A771F7; Fri, 15 Feb 2019 18:24:13 +0100 (CET) X-Virus-Scanned: amavisd-new at bsd4all.org Received: from wan0.bsd4all.org ([127.0.0.1]) by newnas (newnas.bsd4all.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id isY8P2QlQoxm; Fri, 15 Feb 2019 18:24:12 +0100 (CET) Received: from [192.168.1.65] (unknown [192.168.1.65]) by wan0.bsd4all.org (Postfix) with ESMTPSA id 4B86EED; Fri, 15 Feb 2019 18:24:12 +0100 (CET) From: peter.blok@bsd4all.org Message-Id: <426C1AF9-7E30-4174-88C0-1FBFE6341A1D@bsd4all.org> Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Re: iSCSI stall Date: Fri, 15 Feb 2019 18:24:11 +0100 In-Reply-To: <7142e6ae-638d-a667-87d2-f14c881236a9@FreeBSD.org> Cc: freebsd-scsi@freebsd.org To: Alexander Motin <mav@freebsd.org> References: <3579EC93-825D-4908-B177-673363111BEF@bsd4all.org> <6329694E-4BF7-4467-A3A7-2556AA0C3FD6@bsd4all.org> <7142e6ae-638d-a667-87d2-f14c881236a9@FreeBSD.org> X-Mailer: Apple Mail (2.3445.102.3) X-SourceIP: 94.210.49.251 X-Ziggo-spambar: / X-Ziggo-spamscore: 0.0 X-Ziggo-spamreport: CMAE Analysis: v=2.3 cv=SYKJicZu c=1 sm=1 tr=0 a=7fK1ynn72W3Z/oi6DA4Tww==:17 a=CFTnQlWoA9kA:10 a=6I5d2MoRAAAA:8 a=6Q3WNqvRAAAA:8 a=2GToYnZcBhCU6EHnupkA:9 a=QEXdDO2ut3YA:10 a=ioCW4vYGWj-Fin33:21 a=_W_S_7VecoQA:10 a=IjZwj45LgO3ly-622nXo:22 a=I8PBwKCn76L9oNdl0isp:22 X-Ziggo-Spam-Status: No X-Spam-Status: No X-Spam-Flag: No X-Rspamd-Queue-Id: 0B7B77300E X-Spamd-Bar: +++ Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of peter.blok@bsd4all.org designates 212.54.34.165 as permitted sender) smtp.mailfrom=peter.blok@bsd4all.org X-Spamd-Result: default: False [3.43 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(0.00)[+a:smtp.ziggo.nl/16]; MV_CASE(0.50)[]; MX_GOOD(-0.01)[smtp.bsd4all.org]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.39)[-0.395,0]; RCVD_IN_DNSWL_LOW(-0.10)[165.34.54.212.list.dnswl.org : 127.0.5.1]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:33915, ipnet:212.54.32.0/20, country:NL]; MID_RHS_MATCH_FROM(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[251.49.210.94.zen.spamhaus.org : 127.0.0.11]; ARC_NA(0.00)[]; RCVD_COUNT_FIVE(0.00)[6]; R_DKIM_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; DMARC_NA(0.00)[bsd4all.org]; NEURAL_SPAM_MEDIUM(0.84)[0.835,0]; BAD_REP_POLICIES(0.10)[]; IP_SCORE(0.00)[country: NL(0.02)]; NEURAL_SPAM_LONG(0.60)[0.598,0]; FROM_NO_DN(0.00)[]; RBL_SENDERSCORE(2.00)[165.34.54.212.bl.score.senderscore.com] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SCSI subsystem <freebsd-scsi.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-scsi>, <mailto:freebsd-scsi-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-scsi/>; List-Post: <mailto:freebsd-scsi@freebsd.org> List-Help: <mailto:freebsd-scsi-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-scsi>, <mailto:freebsd-scsi-request@freebsd.org?subject=subscribe> X-List-Received-Date: Fri, 15 Feb 2019 17:24:24 -0000 Hi Alexander, I suspected re-ordering too and that=E2=80=99s why I changed max_tags to = 1, but that didn=E2=80=99t work the way I expected. In the wireshark = trace I still saw multiple CDB READ arriving,without a data transfer. If = one of the CDBs was a SYNC CACHE everything stalled without any data = transfer. When I did send a TUR, the data transfer happened and = everything continued. The CDB read had to be orgination from some ZFS (meta)operation, because = bulk of the data was arroving thru zfs recv and most of the CDB were = writes. It did not stall all the time in similar sequences. I did not have the exact target source code, nor build capability and = time to debug, so I=E2=80=99m very glad those commits fixed it. Peter > On 13 Feb 2019, at 16:34, Alexander Motin <mav@freebsd.org> wrote: >=20 > Hi Peter, >=20 > If those commits helped (I did hope they improve performance in some > configurations like this), then I guess this NAS has some problems = with > ordered tag handling in its iSCSI target, and this change workarounded > it. Our iSCSI initiator does not specially care about ordered = requests, > just forwards the flag straight to the target, so it should not change > anything on our side. >=20 > On 13.02.2019 04:07, peter.blok@bsd4all.org = <mailto:peter.blok@bsd4all.org> wrote: >> Hi, >>=20 >> Did not get any response on this, but it seems the commits 344072, = 344075 and 344076 have fixed the issue. My assumption is that the target = doesn=E2=80=99t handle re-ordering with non-offset CDBs very well. >>=20 >> Normally every time when the backup (zfs recv on target) ran, it hung = a couple of times and now it went thru without a glitch and without any = quirks. >>=20 >> Peter >>=20 >>=20 >>> On 16 Jan 2019, at 10:05, peter.blok@bsd4all.org wrote: >>>=20 >>> Hi, >>>=20 >>> I have a ReadyNAS RND4000 as an iSCSI target. The target is the only = device in a ZFS pool and is receiving backups thru zfs receive. The = initiator is FreeBSD 11.x (recent stable at this moment), but this = probably happens with 12.0 as well. >>>=20 >>> During a zfs receive it stalls. Nothing happens anymore. When I send = a TUR via camcontrol everything continues. Any other target on any other = platform works fine. >>>=20 >>> When I take a network trace it looks as if it is around SYNCHRONIZE = CACHE. If this takes too long in the target and before the reply is back = some new READ CDBs are send, it seems to hang only showing Nop-In and = Nop-out. The moment I send a TUR it continues replying with the data of = the READs that were owed. >>>=20 >>> It is not a big deal, but it irritates me and I would like to create = a work-around for this, assuming it is caused by the target and there is = probably no way to fix it there. One test was to create a quirk to = reduce the tags to 1, but that didn=E2=80=99t help. Although it is 1 it = still seems to send multiple CDB concurrently. >>>=20 >>> In the past I have attempted to port FreeBSD to the ReadyNAS, but = got stuck on fan control. I wasn=E2=80=99t able to control the fan and = it made too much noise. I was able to get the network up and running. >>>=20 >>> Any tips? >>>=20 >>>=20 >>>=20 >>>=20 >>> _______________________________________________ >>> freebsd-scsi@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-scsi >>> To unsubscribe, send any mail to = "freebsd-scsi-unsubscribe@freebsd.org" >>=20 >=20 > --=20 > Alexander Motin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?503097732.142480.1550080935357>