From owner-svn-src-head@FreeBSD.ORG Wed Jun 18 17:13:19 2014 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2F5D6808; Wed, 18 Jun 2014 17:13:19 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1013C2B03; Wed, 18 Jun 2014 17:13:19 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s5IHDIUO060299; Wed, 18 Jun 2014 17:13:18 GMT (envelope-from trasz@svn.freebsd.org) Received: (from trasz@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s5IHDIYw060298; Wed, 18 Jun 2014 17:13:18 GMT (envelope-from trasz@svn.freebsd.org) Message-Id: <201406181713.s5IHDIYw060298@svn.freebsd.org> From: Edward Tomasz Napierala Date: Wed, 18 Jun 2014 17:13:18 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r267610 - head/sys/cam/ctl X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jun 2014 17:13:19 -0000 Author: trasz Date: Wed Jun 18 17:13:18 2014 New Revision: 267610 URL: http://svnweb.freebsd.org/changeset/base/267610 Log: Rework session termination in iSCSI target to actually wait for any outstanding commands to be properly aborted by CTL. Without it, in some cases (such as files backing the LUNs stored on failing disk drives), terminating a busy session would result in panic. Reviewed by: mav@ (earlier version) Sponsored by: The FreeBSD Foundation Modified: head/sys/cam/ctl/ctl_frontend_iscsi.c Modified: head/sys/cam/ctl/ctl_frontend_iscsi.c ============================================================================== --- head/sys/cam/ctl/ctl_frontend_iscsi.c Wed Jun 18 17:04:25 2014 (r267609) +++ head/sys/cam/ctl/ctl_frontend_iscsi.c Wed Jun 18 17:13:18 2014 (r267610) @@ -1045,7 +1045,7 @@ cfiscsi_session_terminate_tasks(struct c { struct cfiscsi_data_wait *cdw, *tmpcdw; union ctl_io *io; - int error; + int error, last; #ifdef notyet io = ctl_alloc_io(cs->cs_target->ct_softc->fe.ctl_pool_ref); @@ -1102,12 +1102,31 @@ cfiscsi_session_terminate_tasks(struct c CFISCSI_SESSION_DEBUG(cs, "removing csw for initiator task tag " "0x%x", cdw->cdw_initiator_task_tag); #endif + /* + * Set nonzero port status; this prevents backends from + * assuming that the data transfer actually succeeded + * and writing uninitialized data to disk. + */ + cdw->cdw_ctl_io->scsiio.io_hdr.port_status = 42; cdw->cdw_ctl_io->scsiio.be_move_done(cdw->cdw_ctl_io); TAILQ_REMOVE(&cs->cs_waiting_for_data_out, cdw, cdw_next); uma_zfree(cfiscsi_data_wait_zone, cdw); } CFISCSI_SESSION_UNLOCK(cs); #endif + + /* + * Wait for CTL to terminate all the tasks. + */ + for (;;) { + refcount_acquire(&cs->cs_outstanding_ctl_pdus); + last = refcount_release(&cs->cs_outstanding_ctl_pdus); + if (last != 0) + break; + CFISCSI_SESSION_WARN(cs, "waiting for CTL to terminate tasks, " + "%d remaining", cs->cs_outstanding_ctl_pdus); + pause("cfiscsi_terminate", 1); + } } static void @@ -1124,19 +1143,22 @@ cfiscsi_maintenance_thread(void *arg) CFISCSI_SESSION_UNLOCK(cs); if (cs->cs_terminating) { - cfiscsi_session_terminate_tasks(cs); - callout_drain(&cs->cs_callout); + /* + * We used to wait up to 30 seconds to deliver queued + * PDUs to the initiator. We also tried hard to deliver + * SCSI Responses for the aborted PDUs. We don't do + * that anymore. We might need to revisit that. + */ + callout_drain(&cs->cs_callout); icl_conn_shutdown(cs->cs_conn); icl_conn_close(cs->cs_conn); /* - * XXX: We used to wait up to 30 seconds to deliver queued PDUs - * to the initiator. We also tried hard to deliver SCSI Responses - * for the aborted PDUs. We don't do that anymore. We might need - * to revisit that. + * At this point ICL receive thread is no longer + * running; no new tasks can be queued. */ - + cfiscsi_session_terminate_tasks(cs); cfiscsi_session_delete(cs); kthread_exit(); return;