From owner-freebsd-stable@freebsd.org Thu Jul 5 16:17:21 2018 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3ED7010398D3 for ; Thu, 5 Jul 2018 16:17:21 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C730C85938 for ; Thu, 5 Jul 2018 16:17:20 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (ezra.dcm1.omnilan.net [78.138.80.135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id w65GHIZB019562 for ; Thu, 5 Jul 2018 18:17:18 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 973ABD6F for ; Thu, 5 Jul 2018 18:17:18 +0200 (CEST) Subject: ctld(8) 11.2-release lockup with w2k16 [Was: Re: ctld(8), multiple 'portal-group' on same socket (individual 'discovery-auth-group' restrictions)] To: FreeBSD Stable References: <5444C94C.4050705@omnilan.de> <20141021104308.GA5990@brick.home> From: Harry Schmalzbauer Organization: OmniLAN Message-ID: <3613504a-b017-bc3a-cd62-54d8bb051ea1@omnilan.de> Date: Thu, 5 Jul 2018 18:17:18 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <20141021104308.GA5990@brick.home> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Greylist: ACL 130 matched, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [78.138.80.130]); Thu, 05 Jul 2018 18:17:18 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: 78.138.80.135; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.27 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Jul 2018 16:17:21 -0000 Am 21.10.2014 um 12:43 schrieb Edward Tomasz Napierała: > On 1020T1035, Harald Schmalzbauer wrote: >> Hello, >> >> I'm trying to move from istgt(1) to ctld(8), but it seems my setup isn't >> possible with ctld. >> Besides missing support for virtual-DVDs ('UnitType DVD' in istgt) and >> real ODD-devices ('UnitType pass' in istgt), > Yup, we don't implement virtual DVDs and passthrough. Especially the > latter would be a nice feature to have. Hello Edward, my current problem is unrelated. But this old mail illustrates the timeframe I've been happily using ctld(8) without problems :-) Thanks! Recently, I discovered that WindowsServerBackup fails with Win2k16 (never used 2k12). Old initiators running 2008R2 (or ESXi 5.5) are still able to use ctld(8) ZVOL targets for WindowsServerBackup on 11.2-release without problems. I haven't had time to do much analysis and I'm lacking skills/equipment to do them down at debugger level, but I wanted to ask if you're aware about problems with Windows Server 2016 as ctld(8) initiator. The Symptoms: The system locks up for about 30-60 seconds with iSCSI load from w2k16. When the lockup happens, systat(1) shows 25% intr usage (which is one core) and not even the login session is responsive anymore. Neither updating userland-output nor reacting to input. But, the input is queued and gets processed after the lockup releases. The lockup vanishes as soon as iSCSI session was reset: Jun 28 06:14:09 bansta kernel: WARNING: 172.24.32.172 (iqn.1991-05.com.microsoft:dafus.mgn.mo1.psw-online.de): no ping reply (NOP-Out) after 5 seconds; dropping connection Jun 28 06:14:09 bansta kernel: WARNING: 172.24.32.172 (iqn.1991-05.com.microsoft:dafus.mgn.mo1.psw-online.de): waiting for CTL to terminate 94 tasks Jun 28 06:14:09 bansta kernel: WARNING: 172.24.32.172 (iqn.1991-05.com.microsoft:dafus.mgn.mo1.psw-online.de): tasks terminated Sometimes it's possible to transfer 30GB before the lockup happens, sometimes even a NTFS-quick-format leads to the lockup. Yesterday I used istgt(1) instead of ctld(8) to export the exactly same ZVOL using the exactly same network backend, with exactly the same initiator. The lockup hasn't occured anymore, the complete WindowsServerBackup taks finishes successfully on the Windows Server 2016 initiator.  So I strongly suspect a ctld(8) locking problem. Like mentioned, target backed is a ZFS volume.  I already used a HDD as target backed (and observed a much better performance, which drops even if I use a UFS vnode backend on the same HDD), but I'm not sure anymore whether the lockup also occured... For now I can't tell anything helpfuly, just describe the symptoms and ask if you have any hints for me what to try next to narrow down the problem, or if this is a already known problem. Thanks, -harry