From owner-freebsd-scsi@FreeBSD.ORG Mon Mar 12 00:21:23 2012 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 45C2A106566C for ; Mon, 12 Mar 2012 00:21:23 +0000 (UTC) (envelope-from jwd@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 1741C8FC19; Mon, 12 Mar 2012 00:21:23 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.5/8.14.5) with ESMTP id q2C0LMwj044227; Mon, 12 Mar 2012 00:21:22 GMT (envelope-from jwd@freefall.freebsd.org) Received: (from jwd@localhost) by freefall.freebsd.org (8.14.5/8.14.5/Submit) id q2C0LMOE044226; Mon, 12 Mar 2012 00:21:22 GMT (envelope-from jwd) Date: Mon, 12 Mar 2012 00:21:22 +0000 From: John To: "freebsd-scsi @ freebsd. org" Message-ID: <20120312002122.GA95330@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.3i Cc: aoyama@peach.ne.jp Subject: istgt does not appear to shut down cleanly X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Mar 2012 00:21:23 -0000 Hi Folks, I've been putting together a zfs server which exports zvols via istgt: #istgt -V istgt version 0.4 istgt extra version 20111008 and about a 1 month old 9-stable: 9.0-STABLE FreeBSD 9.0-STABLE #0 r231864M: Fri Feb 17 23:21:34 UTC 2012 When adding a new LUC/LUN combination, I want to terminate and then restart the daemon. However, istgt does not appear to completely exit. + /bin/ps -Ho pid,lwp,inblk,oublk,%cpu,%mem,stat,time,mwchan,xstat,command -p 18261 PID LWP INBLK OUBLK %CPU %MEM STAT TIME MWCHAN XSTAT COMMAND 18261 102021 68 5 0.0 0.0 S 0:00.01 kqread 0 istgt -c /config/pool0/istgt/test1.istgt.conf -p /config/pool0/istgt/test1.pid -D -t all 18261 102065 0 0 0.0 0.0 I 0:00.00 sigwai 0 istgt -c /config/pool0/istgt/test1.istgt.conf -p /config/pool0/istgt/test1.pid -D -t all 18261 102066 0 0 0.0 0.0 I 0:00.00 uwait 0 istgt -c /config/pool0/istgt/test1.istgt.conf -p /config/pool0/istgt/test1.pid -D -t all 18261 102067 0 0 0.0 0.0 I 0:00.00 uwait 0 istgt -c /config/pool0/istgt/test1.istgt.conf -p /config/pool0/istgt/test1.pid -D -t all 18261 102068 0 0 0.0 0.0 I 0:00.00 uwait 0 istgt -c /config/pool0/istgt/test1.istgt.conf -p /config/pool0/istgt/test1.pid -D -t all 18261 102069 0 0 0.0 0.0 I 0:00.00 uwait 0 istgt -c /config/pool0/istgt/test1.istgt.conf -p /config/pool0/istgt/test1.pid -D -t all + kill -INT 18261 I then loop waiting for the pidfile to be removed: + [ -s /config/pool0/istgt/test1.pid ] + sleep 0.1 + [ -s /config/pool0/istgt/test1.pid ] + sleep 0.1 + [ -s /config/pool0/istgt/test1.pid ] + sleep 0.1 + [ -s /config/pool0/istgt/test1.pid ] + sleep 0.1 + [ -s /config/pool0/istgt/test1.pid ] and then finally for the process to exit (it doesn't matter how long I wait), but it never does: + jot 10 + sleep 0.1 + kill -0 18261 ... + kill -0 18261 + sleep 0.1 + kill -0 18261 + /bin/ps -Ho pid,lwp,inblk,oublk,%cpu,%mem,stat,time,mwchan,xstat,command -p 18261 PID LWP INBLK OUBLK %CPU %MEM STAT TIME MWCHAN XSTAT COMMAND 18261 102021 68 5 0.0 0.0 S 0:00.01 uwait 0 istgt -c /config/pool0/istgt/test1.istgt.conf -p /config/pool0/istgt/test1.pid -D -t all 18261 102065 0 0 0.0 0.0 I 0:00.00 sigwai 0 istgt -c /config/pool0/istgt/test1.istgt.conf -p /config/pool0/istgt/test1.pid -D -t all Finally I just forcefully kill it.. + kill -9 18261 Above, the LUC threads appear to exiting, but that's all. From looking at the source: istgt_remove_pidfile(istgt); istgt_close_log(); istgt->config = NULL; istgt_free_config(config); istgt_set_state(istgt, ISTGT_STATE_SHUTDOWN); /* stop signal thread */ rc = pthread_join(sigthread, NULL); I think it's hanging on the pthread_join(). Does anyone else have any experience with this issue? Or a better way to do this? Many Thanks!! John