From owner-freebsd-current@FreeBSD.ORG Tue Oct 20 08:36:23 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CF9C01065672; Tue, 20 Oct 2009 08:36:23 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-fx0-f210.google.com (mail-fx0-f210.google.com [209.85.220.210]) by mx1.freebsd.org (Postfix) with ESMTP id 363F98FC0C; Tue, 20 Oct 2009 08:36:22 +0000 (UTC) Received: by fxm6 with SMTP id 6so5815546fxm.43 for ; Tue, 20 Oct 2009 01:36:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :content-type:content-transfer-encoding; bh=yvqpJCUaHWuiZ3vhKpjLYzy4UxVPAYKbIotRAkhB9XM=; b=lMOAn9Fgoac+l9AT2GbArpfjpRwN1X/aYe/XUZpNhGQ0twefRAFS3Yz+ACT7mCJsCU AgWPDrWynex8kB1IBacn2dnmUr2iR7qRttqO6WEtbLmJ/xr0l9gn+w1pkj/Kaqe8rpxy m8KX7NG/sxjw7XZ/s1ZK+Jsh7cuK7Ngrq4ox8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; b=mLRC8TOpiBYKOOakss+IZcpMNFmFuRJE4UwOOK3w7IEV0fktbED9J4mBHu9JB2s/Rl KkfFlslhFbkp+k2kd9YlUjvpeRzB4MYE43LHFkKFEhbKvsgjDdknEIXuUCf3XEbMxN3K W0ElTSbSiVyjecyczLStoLalY11giAU7eKp90= Received: by 10.103.64.16 with SMTP id r16mr2600303muk.90.1256027781877; Tue, 20 Oct 2009 01:36:21 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (pc.mavhome.dp.ua [212.86.226.226]) by mx.google.com with ESMTPS id s10sm1338393mue.22.2009.10.20.01.36.20 (version=SSLv3 cipher=RC4-MD5); Tue, 20 Oct 2009 01:36:21 -0700 (PDT) Sender: Alexander Motin Message-ID: <4ADD7683.7040907@FreeBSD.org> Date: Tue, 20 Oct 2009 11:36:19 +0300 From: Alexander Motin User-Agent: Thunderbird 2.0.0.23 (X11/20090901) MIME-Version: 1.0 To: Andrew Thompson References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: FreeBSD-Current , Scott Long Subject: Re: CAM problem X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2009 08:36:24 -0000 Andrew Thompson wrote: > I have a cam problem that is noticeable with usb devices. It relates to > the ordering of xpt_release_device() and the CAM_DEV_UNCONFIGURED flag > when yanking a device that has stalled. This then causes a problem with > the usb explore thread which will end up waiting on simfree forever, > blocking any further usb attach/detach on the controller. > > Hopefully my printfs can show the problem. I have replaced the pointers > returned from xpt_alloc_device() with pretty names, is the one in > question. > > <...unplug...> > > ugen1.3: at usbus1 (disconnected) > umass0: at uhub2, port 1, addr 3 (disconnected) > umass_detach: > usb_cam_action, device GONE > usb_cam_action, device GONE > usb_cam_action, device GONE > xpt_find_bus: ref=6 -> 7 > usb_cam_action, device GONE > usb_cam_action, device GONE As I can see, you are returning CAM_TID_INVALID error here. There is no special error handling for this error, comparing to CAM_SEL_TIMEOUT. If you return CAM_SEL_TIMEOUT there, device will be killed immediately and probably workaround this specific problem. > xpt_release_device dev3 failed, ref=3 unconf=0 > xpt_release_path: xpt_release_bus > xpt_release_bus: ref=7 -> 6 > (da0:umass-sim0:0:0:0): got CAM status 0x39 > (da0:umass-sim0:0:0:0): fatal error, failed to attach to device > (da0:umass-sim0:0:0:0): lost device > (da0:umass-sim0:0:0:0): removing device entry > > ^^^ USB disk had stalled on attach This thing drops reference as periph driver detached itself, but device is still treated as valid by XPT. > xpt_release_device dev3 failed, ref=1 unconf=0 > xpt_release_path: xpt_release_bus > xpt_release_bus: ref=6 -> 5 > xpt_release_device dev3 failed, ref=0 unconf=0 > > ^^^ last reference to dev3 dropped >From deallocation point of view, configured status handled the same as one more reference... > xpt_release_path: xpt_release_bus > xpt_release_bus: ref=5 -> 4 > xpt_release_device dev2 OK > xpt_release_target: xpt_release_bus > xpt_release_bus: ref=4 -> 3 > xpt_release_path: xpt_release_bus > xpt_release_bus: ref=3 -> 2 > umass_cam_detach_sim: calling xpt_bus_deregister > xpt_find_bus: ref=2 -> 3 > xpt_alloc_target: ref=3 -> 4 > xpt_alloc_device: device = dev4 > scsi_dev_async: set dev dev3 unconfigured > > ^^^ dev3 gets the CAM_DEV_UNCONFIGURED flag cleared here ... but removing configured status does not call deallocation, as unreferencing does. > xpt_bus_deregister: xpt_release_bus > xpt_release_bus: ref=4 -> 3 > xpt_release_device dev4 OK > xpt_release_target: xpt_release_bus > xpt_release_bus: ref=3 -> 2 > xpt_release_path: xpt_release_bus > xpt_release_bus: ref=2 -> 1 > umass_cam_detach_sim: > umass-sim0: waiting... ref = 1 > > ^^^ wait on "simfree" forever. I think correct solution will be to additionally increment reference counter before clearing CAM_DEV_UNCONFIGURED and decrement it back after setting CAM_DEV_UNCONFIGURED back. Check for CAM_DEV_UNCONFIGURED inside xpt_release_device() then could be removed or turned into assertion. -- Alexander Motin