From owner-freebsd-fs@FreeBSD.ORG Wed Mar 10 17:46:19 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C62F910656A7 for ; Wed, 10 Mar 2010 17:46:19 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 2F42F8FC21 for ; Wed, 10 Mar 2010 17:46:19 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 6B7D445EB2; Wed, 10 Mar 2010 18:46:16 +0100 (CET) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id CC59745E8E; Wed, 10 Mar 2010 18:46:11 +0100 (CET) Date: Wed, 10 Mar 2010 18:46:12 +0100 From: Pawel Jakub Dawidek To: Kevin Day Message-ID: <20100310174612.GF1715@garage.freebsd.pl> References: <7418ECC2-55C1-4A28-82EA-0972AFE745EF@dragondata.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="0rSojgWGcpz+ezC3" Content-Disposition: inline In-Reply-To: <7418ECC2-55C1-4A28-82EA-0972AFE745EF@dragondata.com> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org Subject: Re: iscsi over HAST backed storage partial success X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 17:46:20 -0000 --0rSojgWGcpz+ezC3 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Mar 09, 2010 at 05:03:41PM -0600, Kevin Day wrote: >=20 > I'm running istgt (iscsi target) using HAST backed storage. For the most = part, it seems to work really well. I have ucarp running to change the IP t= hat istgt is bound to, and modified the ucarp scripts to start/stop istgt d= epending on which side is the master. If I shut down the primary, the secon= dary takes over and all seems well. >=20 > However, if I reboot the secondary, the primary starts freezing up for lo= ng periods: >=20 > Mar 9 22:46:27 cs04 hastd: [iscsi1] (primary) Unable to r: Socket is not= connected. > Mar 9 22:46:27 cs04 hastd: [iscsi1] (primary) Unable to co: Connection r= efused. > Mar 9 22:46:42 cs04 last message repeated 3 times > Mar 9 22:46:53 cs04 istgt[14298]: ABORT_TASK > Mar 9 22:47:35 cs04 last message repeated 3 times > Mar 9 22:48:02 cs04 hastd: [iscsi1] (primary) Unable to co: Operation ti= med out. > Mar 9 22:48:02 cs04 istgt[14298]: CmdSN(45748), OP=3D0x2a, ElapsedTime= =3D74 cleared=20 > Mar 9 22:48:02 cs04 istgt[14298]: istgt_iscsi.c: 640:istgt_iscsi_write_p= du: ***ERROR*** iscsi_write() failed (errno=3D32) > Mar 9 22:48:02 cs04 istgt[14298]: istgt_iscsi.c:3327:istgt_iscsi_op_task= : ***ERROR*** iscsi_write_pdu() failed > Mar 9 22:48:02 cs04 istgt[14298]: istgt_iscsi.c:3867:istgt_iscsi_execute= : ***ERROR*** iscsi_op_task() failed =20 > Mar 9 22:48:02 cs04 istgt[14298]: istgt_iscsi.c:4337:worker: ***ERROR***= iscsi_execute() failed > Mar 9 22:48:02 cs04 istgt[14298]: CmdSN(490802), OP=3D0x2a, ElapsedTime= =3D73 cleared > Mar 9 22:48:02 cs04 istgt[14298]: CmdSN(28387), OP=3D0x2a, ElapsedTime= =3D73 cleared=20 > Mar 9 22:48:14 cs04 istgt[14298]: ABORT_TASK > Mar 9 22:48:52 cs04 last message repeated 2 times > Mar 9 22:49:22 cs04 hastd: [iscsi1] (primary) Unable to co: Operation ti= med out. >=20 > As soon as the secondary comes back online, everything starts behaving ag= ain and all is well. >=20 > Is this expected behavior at this point, or should hastd not block like t= his? It shouldn't of course block like this. There is a separate thread responsible for reconnecting which shouldn't interact with I/O threads. I'll try to reproduce and will let you know. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --0rSojgWGcpz+ezC3 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkuX2uMACgkQForvXbEpPzT7ewCgu70tq0+kx0OyRe4PQAaR1HV3 KAsAn2mW+sMKaLvKpvfJWILmNUiQfYX6 =ONha -----END PGP SIGNATURE----- --0rSojgWGcpz+ezC3--